3 min read WASI

State of WASI support for CPython: June 2023

I wanted to give an update on where WASI support for CPython stood today and what I see happening in the future.

💡
If you don't know what WASI is or how CPython fits into it, please read my other blog post on WebAssembly platforms.

Today

WASI is currently a tier 3 platform for CPython. I am publishing unofficial builds for Python 3.11 using WASI-SDK 16 and Python 3.12 using WASI-SDK 20 (the discrepancy in version support is due to some clock support changes that were fixed in Python 3.12). I have also cleaned up the CPython WASM notes on building CPython for WASI. And this WASI support is already seeing use, for instance by VS Code and its WebAssembly support (and thus how I get work time for all of this 😁).

I created a dev container for CPython. I call this out here because the dev container installs the requisite tooling to do a WASI build of CPython. This means people can more easily try building CPython for WASI using GitHub Codespaces.

There is now componentize-py to help you take Python code and create a WebAssembly component from it. I do know the authors are looking for feedback, so if you give a try then please do let them know how it went!

We hosted the inaugural WebAssembly summit at PyCon US! Probably the most visible thing, external to the discussions had in the room, is https://github.com/psf/webassembly as the place to track links out to important things such as where people are talking, documentation, etc. (think of it like "awesome-python-webassembly" 🕶️).

I have started participating in the Bytecode Alliance. There is now a dynamic languages SIG – which is being renamed to Guest Languages SIG at some point – that I am a sponsoring member of (although ironically I have never made a meeting as it's at 07:00 PT, but they are looking for a second meeting time that works better for those of us on Pacific time). Since that SIG is for basically all dynamic languages, there's a separate meeting just for Python as a guest runtime and bindings for it in the component model (which I do attend since it's at 12:00 PT every other week).

If I had to come up with a theme for the past year for me and WASI, it's been getting acquainted with what Christian had done for us and establishing connections and laying groundwork with everyone else doing WebAssembly stuff in Python (e.g., working with PyScript and Pyodide on general WebAssembly concerns).

The future

My main goal for the rest of 2023 is to get WASI support for CPython to tier 2. Part of that is setting up a new buildbot for CPython which I have direct access/control to as well as understand how it works. There is also various issues that need to be resolved before I'm ready to go to the Python steering council and ask for WASI support to be promoted to tier 2 (crucially, that's figuring out how to make it easy for other core developers to fix their own issues that come up thanks to WASI).

Once WASI support hits tier 2, I plan to work with the release managers to have official WASI builds of CPython. I plan to use my unofficial builds as a way to experiment with this. I expect it will be a "full" build with as much of the standard library built-in and not designed for e.g., IoT situations where you might optimize for file size. I expect a lot of folks will do their own build of CPython for WASI, but I would like to provide a build with as many features turned on as possible so people can get a feel for any limitations WASI may have for their code as well as a quick way to get up and going.

In terms of packaging, that's an active point of discussion in the Python guest runtime and bindings group in the Bytecode Alliance. Just the other day, some folks from that meeting got a hacked version of CPython and WASI Libc together that contains an experimental dlopen and dlsym such that they could import a WASI-compiled version of ujson and NumPy! It's just a hack, but it's an exciting one as once there's an answer on how to link .so files into a WASI build of CPython, then we can start talking about what needs to happen for WASI wheels to be uploaded to PyPI!

I'm also interested in trying to use Wizer to pre-initialize the interpreter to see if start-up time can be faster on WASI builds than CPython when it's natively compiled. I'm not sure how much we can actually do ahead of time prior to command-line flags influencing things, but it would be interesting to see how much can be done upfront (maybe even preemptively importing a bunch of modules?).

There's also things we are talking about doing with the WASI build of CPython in VS Code, but those discussions are a bit too early to go into detail about. But we are bandying about ideas around shipping Python as an extension itself so there's always a way to get an interpreter, making it so you can run tool extensions like our Black extension in the browser (which would be great for those times when you forgot to run Black before creating your PR and now you're failing CI 😅), and maybe even writing more of the Python extension in Python itself.