Python version management with Pyenv

In the previous posts we have seen how to install different python versions in the system but in a cumbersome way, either using brew to install another version in your system (and modify manually the PATH variable to make them availiable) or download and compile specific versions or using conda for creating new environments using different python version. This is where pyenv comes very handy. Pyenv allows you to install different python versions in your system and select which one to use at every moment with a couple of easy commands. In this post we explore how to install and use pyenv, in my opinion the utlimate python version manager for any developer.

TLDR

Assuming you use zsh and on linux/macOS machine

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

SRC_FILE="${HOME}/.zshrc"
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ${SRC_FILE}
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ${SRC_FILE}
echo 'eval "$(pyenv init -)"' >> ${SRC_FILE}

then source ~/.zshrc and check if pyenv is installed

pyenv --version

Install a python verision and use it in your current shell

pyenv install 3.12.4
pyenv shell 3.12.4

Prerequisites to install pyenv

For linux

sudo apt-get update
sudo apt-get install -y git \
                        curl \
                        build-essential \
                        libssl-dev \
                        zlib1g-dev \
                        libbz2-dev \
                        libreadline-dev \
                        libsqlite3-dev \
                        libffi-dev \
                        libncurses5-dev \
                        libncursesw5-dev \
                        xz-utils \
                        tk-dev

For MacOs

brew update

brew install git \
             curl \
             openssl \
             readline \
             sqlite3 \
             xz \
             zlib

Install Pyenv

I prefer to install directly clonning the repository into the recommended installation path ~/.pyenv.

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

Now open ~/.zshrc (or ~/.bashrc) and append:

SRC_FILE="${HOME}/.zshrc"
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ${SRC_FILE}
echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ${SRC_FILE}
echo 'eval "$(pyenv init -)"' >> ${SRC_FILE}

finally just source ~/.zshrc (or source ~/.bashrc). Now try

whereis pyenv
pyenv --help

Uninstall Pyenv

Remove the directory

rm -fr ~/.pyenv

and the remove the lines you appended in ~/.zshrc or ~/.bashrc:

Installing a specific python version

Let’s say I want to install Python 3.11, I can take a look fo the available versions of 3.11 running

pyenv install --list | grep 3.11

then select 3.11.2

pyenv install 3.11.2

Let’s also install 3.12.4

pyenv install 3.12.4

Now check which versions you have installed running

pyenv versions

getting

* system (set by ~/.pyenv/version)
  3.11.2
  3.12.4

In asterisk the current active python version in the terminal, in this case system. To uninstall any version (e.g. 3.11.2) just do

pyenv uninstall 3.11.2

Setting python version

Pyenv allows you to define which installed python version to use at each time. There is the global, local and shell levels.

• global: sets the global python version in your system
• local : sets a python version for the current directory
• shell: sets python version for current shell.

Global python versions are set like 3.11.2 just type in your terminal

pyenv global 3.11.2

Now every time you run python in a new terminal, it is going to default to 3.11.2.

To set a local python version you type

pyenv local 3.11.2

in your desired directory. You will see it will create a new file called .pyenv-version containing 3.11.2. When you are in your terminal and have cded to that directory, pyenv will look for that file and point the python command to that specific version. This is very useful when developing on specific projects if you want to pin the python version to be used in that project. That being said, the developer obviously has to use pyenv to develop in that project.

Finally, to pin a python version for your current interactive shell you just run

pyenv shell 3.11.2

This will be effective as long as you work on that interactive shell, the moment you close it, it defaults to local or global python versions. I use the pyenv shell command for creating new virtual environments with specific python versions, but I will comment on that in a later post.

The hierarchy for the python version is the following, shell > local > global meaning that, if there is a shell python version, the terminal will take that, then if not it will default to the local then if that doesn’t exist it will default to the global version.

A bit of pyenv inner workings

I am not going to rewrite all the information in the pyenv README.md, check there for a fully detaile dexplanation on how pyenv works. I just want to highlight few things to get a birds eye view. Essentially what pyenv installation does is to prepend the path ~/.pyenv/shims, if you check closely what’s in there:

ls -lhat ~/.pyenv/shims

you will find a python and pip executables among others. Every time you run python your shell tries to find that executable in the directories in its PATH, starting from the first. By prepending ` ~/.pyenv/shims to the PATH, the python executable found will be ~/.pyenv/shims/python. Then, that python is actually an executable that redirects the call to pyenv which in turn decides wether to point to global, local or shell` python versions. This is very clean!

Different versions of python are installed under ~/.pyenv/versions. Without using shims you can call directly any python version you have installed on your bash/zsh shell, for instance if we want to call python 3.11.2:

~/.pyenv/versions/3.11.2/bin/python

but obviously is not good practice to install packages calling this python, instead you should create a new environment that copy your python version to your new environment in the current directory using venv binary from the desired python version (we’ll go on details on the next post):

Conda in pyenv

One of the main reasons why I wouldn’t install miniconda directly is because you can actually install it as a pyenv version although it is a bit tricky to set up if you still don’t want to mess your PATH variable. Check the miniconda versions

pyenv install --list | grep miniconda3

try to get the latest version: miniconda3-latest, let’s proceed to install it

pyenv install miniconda3-latest

Go ahead and make it available on shell, then check conda is on your command line

pyenv shell miniconda3-latest
conda --help
conda --version

If you ever need to update this version just run

conda update -n base -c defaults conda

Now you can create a new virtual environment called myenv2

conda create -n myenv2 python=3.12 -y

After this it gets trycky if you want to activate the environment. Normally you would run conda activate myenv2 but then conda complains that you need to run conda init zsh (in my cas since I use zsh) first, which writes some lines to ~/.zshrc that allow you to activate an environment by modifying the PATH. I pesonally don’t like letting conda control my environment PATH (that’s why I use pyenv after all), what one can do is to activate the base environment like

source ~/.pyenv/versions/miniconda3-latest/bin/activate

and then you can activate the environment as

conda activate myenv2

Many times I won’t use the activate functionality, I just call the python binary:

PYTHON_VERSION=miniconda3-latest
ENV=myenv2
${PYENV_ROOT}/versions/${PYTHON_VERSION}/envs/${ENV}/bin/python --version

this is pretty handy to use in bash scripts. See that every new environment we install using conda will be placed in the directory ~/.pyenv/versions/miniconda3-latest/envs/.

Conclusions

pyenv is a versatile python version manager. It simply works and allows you to use other managers like minicionda or mamba. It is my preferred way of managing python versions in a development environment by far.