5 min read Python

Introducing the Python Launcher for Unix

The problem

Let's say you have more than one version of Python installed on your machine. What version does python3 point to? If you said, "the newest version", you may actually be wrong. That's because python3 points at the last version of Python you installed, not necessarily the newest; python3 typically gets overwritten every time you install some version of Python.

This is a bit annoying when you simply want to work with the newest version of Python (e.g., creating a new virtual environment). It would be convenient if there was a command which just always used the newest version of Python that's installed ...

The solution

It turns out the Python installation of Windows came up with a solution to this annoyance over a decade ago: the Python Launcher for Windows. Through a command named py, you can get the newest version of Python and a convenient listing of all of the known installs of Python on your Windows machine (it's also the default way to launch Python from a python.org install on Windows when python3 is left off of PATH, although that's not the case for the latest PSF Windows Store installs). Being a Windows user at work, I came to find the command rather convenient to have and missed it anytime I did work under macOS or Linux. That felt weird and I decided to do something about that problem.

And so over 3 years ago I set out to re-implement the Python Launcher for Unix in Rust. On July 24, 2021, I launched 1.0.0 of the Python Launcher for Unix (thanks to Kushal Das and Dusty Phillips for double-checking my Rust code). This gives you a py command on Unix which will always use the newest version of Python. What this means is if you run:

py -m venv .venv

you will end up with a virtual environment using the newest version of Python that can be found on $PATH. For this one use alone the command is surprisingly useful. It's also great when you just want a REPL for the newest version of Python that you  have installed.

You can also set the $PY_PYTHON environment variable to specify the version of Python you want to use by default. For instance, let's say you installed CPython 3.10.0.rc1, but you want to use Python 3.9 for your day-to-day. You can set PY_PYTHON = 3.9 and that would cause py to use python3.9.

You can use py --list to see what Python versions you have installed where.

 3.9 │ /usr/local/bin/python3.9                
 3.8 │ /usr/local/opt/python@3.8/bin/python3.8 
 3.7 │ /usr/local/opt/python@3.7/bin/python3.7 
 2.7 │ /usr/bin/python2.7

All of this mirrors what the Python Launcher for Windows does (although the format of the list is different). There are some other things which are also implemented which are not quite as useful these days since Python 2 is behind us (i.e. you can specify restrictions like what major version of Python you want as well as the major.minor version), but if you are coming from Windows then you have  most of what you're used to from the py command (i.e. configuration files are not implemented and I didn't bother with -0 as an alias for --list).

Bonus features

For my workflow

Since I was writing this tool for convenience, I decided to push things out a bit and add some niceties to fit my personal workflow a bit better. 😁

For one thing, if you have a virtual environment in a .venv directory either in the current or any parent directory, it will be used instead of the newest version of Python that's installed. What this means is you don't need to activate your virtual environment anymore (although the Launcher will use any activated virtual environment as well)!

py -m venv .venv
py -c "import sys; print(sys.executable)"

I can then continue to use -m to use tools installed into the virtual environment, e.g. py -m pip without issue as well.

This has a surprising benefit for Starship users like myself. By setting the Python binary used to get the version of Python to py, Starship will tell me what version of Python will be used when I run the Python Launcher. Add in folder detection for .venv and now I have Starship tell me what version the virtual environment is for a directory. The FAQ entry on this topic covers how to update your starship.toml to make this all work.

I'm also a fish shell user, so I created tab completions for it. The neat thing about them it they are dynamic, so they will be accurate to your machine.

For others' workflows

I have also managed to do some things that help others out based on their workflow.

As my first serious CLI tool that I created, I decided to create a man page (from a Markdown file thanks to pandoc). I am also taking the changelog a bit more seriously than I usually do, so thanks to scriv and Ned Batchelder for making some changes to the tool to let me automate nearly my entire release process.

Thanks to Trey Hunner, Jeff Triplett, Ben Spaulding, Carol Willing, and Lorena Mesa being beta testers who happened to be pyenv users, as they contributed a FAQ entry on how to set it up such that the $PY_PYTHON environment variable gets set to what Python version pyenv is set to use.

I also happen to always be on the verge of giving Nushell a serious try, so I went ahead can came up with a one-liner which creates a table of your Python versions.

What the Launcher does ... in picture form

To help understand the overall flow that the Python Launcher for Unix takes in choosing which Python interpreter to use, I created a flowchart using Graphviz.

How to install

If you're interested in installing the Python Launcher for Unix, there are a bunch of options which are all covered in the README. I managed to get a formula into Homebrew/Linuxbrew. Bernát Gábor was nice enough to create an Arch AUR package. I have pre-built binaries for:

  1. RISC-V Linux
  2. AArch64 Linux
  3. x86-64 Linux
  4. Apple Silicon
  5. x86-64 macOS
  6. x86-64 NetBSD

That's basically every platform that I could easily get going under GitHub Actions or build directly on my macOS machine. And if you have cargo installed from Rust, cargo install python-launcher also works (although you do get the bonus features like the man page and fish tab completions).

What the Launcher doesn't do

One key thing to point out is the Python Launcher for Unix is not meant to be used in every situation where you may want to launch Python. For instance, if you know the path to the Python you want to use then just launch it directly. Same goes for if you have very specific requirements around what Python interpreter to use (e.g. Framework build on macOS or various bitness builds). The Launcher is purely a convenience and not meant to be The Launcher For All Things; this should never end up in a Docker container.

Otherwise I hope some of you find it useful!