Python 3.13: Free Threading and a JIT Compiler

Free Threading Makes the GIL Optional

Free threading is an attempt to remove the Global Interpreter Lock (GIL) from CPython, which has traditionally been the biggest obstacle to achieving thread-based parallelism when performing CPU-bound tasks. In short, the GIL allows only one thread of execution to run at any given time, regardless of how many cores your CPU is equipped with. This prevents Python from leveraging the available computing power effectively.

There have been many attempts in the past to bypass the GIL in Python, each with varying levels of success. You can read about these attempts in the tutorial on bypassing the GIL. While previous attempts were made by third parties, this is the first time that the core Python development team has taken similar steps with the permission of the steering council, even if some reservations remain.

The removal of the GIL would have significant implications for the Python interpreter itself and especially for the large body of third-party code that relies on it. Because free threading essentially breaks backward compatibility, the long-term plan for its implementation is as follows:

1. Experimental: Free threading is introduced as an experimental feature and isn’t a part of the official Python distribution. You must make a custom Python build to disable the GIL.
2. Enabled: The GIL becomes optional in the official Python distribution but remains enabled by default to allow for a transition period.
3. Disabled: The GIL is disabled by default, but you can still enable it if needed for compatibility reasons.

There are no plans to completely remove the GIL from the official Python distribution at the moment, as that would cause significant disruption to legacy codebases and libraries. Note that the steps outlined above are just a proposal subject to change. Also, free threading may not pan out at all if it makes single-threaded Python run slower than without it.

Until the GIL becomes optional in the official Python distribution, which may take a few more years, the Python development team will maintain two incompatible interpreter versions. The vanilla Python build won’t support free threading, while the special free-threaded flavor will have a slightly different Application Binary Interface (ABI) tagged with the letter “t” for threading.

This means that C extension modules built for stock Python won’t be compatible with the free-threaded version and the other way around. Maintainers of those external modules will be expected to distribute two packages with each release. If you’re one of them, and you use the Python/C API, then you’ll learn how to target CPython’s new ABI in the final section of this tutorial.

JIT Compiles Python to Machine Code

As an interpreted language, Python takes your high-level code and executes it on the fly without the need for prior compilation. This has both pros and cons. Some of the biggest advantages of interpreted languages include better portability across different hardware architectures and a quick development time due to the lack of a compilation step. At the same time, interpretation is much slower than directly executing code native to your machine.

Languages like C and C++ leverage Ahead-of-Time (AOT) compilation to translate your high-level code into machine code before you ship your software. The benefit of this is faster execution since the code is already in the computer’s mother tongue. While you no longer need a separate program to interpret the code, you must compile it separately for all target platforms that you want supported. You should also handle platform-specific differences yourself.

There’s a middle ground between code compilation and interpretation. For example, Java takes the best of both worlds by compiling its code into portable bytecode, which is well-suited for both efficient and cross-platform execution. Additionally, Java uses the Just-In-Time (JIT) compilation approach, which basically means converting high-level instructions into native code right before it runs on the target machine.

This approach has some drawbacks, including a delayed startup time and non-deterministic performance, making it unfit for real-time computing, such as financial trading systems. On the other hand, JIT has advantages over the classic AOT compilation. Because a JIT compiler can collect invaluable information at runtime that isn’t available statically, it can further optimize the resulting machine code, tailoring it to the specific data patterns.

Up to now, you could take advantage of various JIT compilers for Python through external tools and libraries only. Some of them, like PyPy and Pyjion, offered more or less general-purpose JIT compilers, while others, such as Numba, focused on specific use cases like numerical computation.

The new experimental JIT compiler in Python 3.13 uses a fairly recent algorithm named copy-and-patch, which you can learn about in the official paper published in the proceedings of the ACM on Programming Languages in 2021. The basic idea behind this compilation technique boils down to finding a suitable template with pre-compiled machine code for the target CPU and filling it with the missing information, such as memory addresses of variables.

While copy-and-patch follows a relatively crude approach, ensuring fast just-in-time compilation, it yields surprisingly good results. Note that there’s currently no expectation for Python’s JIT to provide any meaningful performance improvements. At best, it should be on par with plain Python without the JIT, which is an achievement in itself considering the extra steps involved.

The long-term plan is to enhance Python’s JIT to the point where it actually makes a noticeable difference in code execution performance without taking much additional memory.

In the early stages of development of Python 3.13, it wasn’t possible to build the interpreter with both free threading and the JIT compiler enabled. You had to choose one or the other. Now, with the first release candidate available for download, both features have been fully integrated and can work together.

Next, you’ll learn where to get Python 3.13 with these experimental features supported.

The Official Distribution Channels

There are many ways to install Python on your computer. For example, you can run Python in Docker using one of the official images, which conveniently include pre-release versions. As mentioned earlier, none of the official distribution channels ship with free threading and JIT by default.

Having said that, if you’re on macOS or Windows, then you can use the official installers to customize Python’s installation process:

When you run the installer, choose Customize installation and click Next to skip the optional features. On the Advanced Options page, select the Download free-threaded binaries (experimental) checkbox at the bottom.

This will install separate Python binaries for the stock and free-threaded versions. On Windows, when you list the available interpreters using the Python launcher, you’ll see two variants of Python 3.13:

PS> py --list
 -V:3.13t *       Python 3.13 (64-bit, freethreaded)
 -V:3.13          Python 3.13 (64-bit)

One is the standard build of Python 3.13, while the other, denoted with the letter “t,” is the free-threaded version. The asterisk (*) indicates the default binary to launch when you request Python 3.13 with the py -3.13 command.

Notice that while you can enable free threading using the official installer, there’s no analogous option for the experimental JIT compiler. That’s where alternative Python installation methods come into play, and you’ll take a closer look at them next.

Python Version Manager: pyenv

A popular tool for managing multiple Python versions on your computer is pyenv. It allows you to install many interpreters and switch between them quickly. It even comes with alternative Python interpreters like MicroPython and full-fledged distributions like Anaconda. Although pyenv works predominantly on Unix-like operating systems, if you’re on Windows, you can try the pyenv-win fork or use WSL.

Once you’ve successfully installed pyenv, you can list the Python interpreters available for download and optionally narrow down the results by using the Unix grep command or findstr on Windows:

• Windows
• Linux + macOS

PS> pyenv install --list | findstr 3.13
3.13.0a1-arm
3.13.0a1-win32
3.13.0a1
(...)
3.13.0rc1-arm
3.13.0rc1-win32
3.13.0rc1

$ pyenv install --list | grep 3.13
  3.13.0rc1
  3.13.0rc1t
  3.13-dev
  3.13t-dev
  pypy2.7-7.3.13-src
  pypy2.7-7.3.13
  pypy3.9-7.3.13-src
  pypy3.9-7.3.13
  pypy3.10-7.3.13-src
  pypy3.10-7.3.13

This will filter the list, showing only names that include the string 3.13. When using pyenv-win on Windows, you’ll see just CPython versions. In contrast, on Linux and macOS, you might encounter other Python implementations as well. For instance, the highlighted lines correspond to CPython, while the remaining ones represent PyPy versions that match your search criteria.

As you can see from the Linux and macOS output above, there were four CPython versions and a few PyPy interpreters whose names contained the string 3.13 at the time of writing. You can disregard the latter because PyPy follows a slightly different versioning scheme, which involves the underlying Python version followed by a specific PyPy release number.

But why are there four CPython 3.13 versions? First of all, the ones ending with a -dev suffix represent the development version of Python, which contains unstable code that’s under active development. Generally speaking, you shouldn’t use these versions unless you’re testing or experimenting with the bleeding edge.

The rc1 suffix denotes the first release candidate of Python 3.13, while the trailing letter “t” indicates its free-threaded variant as opposed to the default version. Therefore, you can use pyenv to install a stock version of Python 3.13, as well as one with free-threading support. Type the following command to install both in one go:

$ pyenv install 3.13.0rc1 3.13.0rc1t

Just like the official installers, pyenv doesn’t bundle Python 3.13 with the experimental JIT support. After all, this little command-line tool merely reflects the available Git tags in the python/cpython repository on GitHub. However, unlike the official Python installers, which download pre-compiled binaries for your operating system, pyenv automates the compilation of the CPython source code, letting you customize the process.

To install Python 3.13 with the experimental JIT enabled using pyenv, you can specify custom build flags by setting the PYTHON_CONFIGURE_OPTS environment variable accordingly:

$ PYTHON_CONFIGURE_OPTS='--enable-experimental-jit' \
  pyenv install 3.13.0rc1

In this case, you enable the JIT on top of the stock Python, 3.13.0rc1, which doesn’t come with free threading. If you’d like to have both features enabled simultaneously, then you can either pick 3.13.0rc1t as the basis or append the --disable-gil configuration option to the environment variable.

Sometimes, it’s useful to specify custom names for the same version of Python built with different options. To do so, you can use the pyenv-suffix plugin and the associated environment variable:

$ PYENV_VERSION_SUFFIX="-stock" \
  pyenv install 3.13.0rc1

$ PYENV_VERSION_SUFFIX="-nogil" \
  PYTHON_CONFIGURE_OPTS='--disable-gil' \
  pyenv install 3.13.0rc1

$ PYENV_VERSION_SUFFIX="-jit" \
  PYTHON_CONFIGURE_OPTS='--enable-experimental-jit' \
  pyenv install 3.13.0rc1

$ PYENV_VERSION_SUFFIX="-nogil-jit" \
  PYTHON_CONFIGURE_OPTS='--disable-gil --enable-experimental-jit' \
  pyenv install 3.13.0rc1

$ pyenv versions
* system (set by PYENV_VERSION environment variable)
  3.13.0rc1-jit
  3.13.0rc1-nogil
  3.13.0rc1-nogil-jit
  3.13.0rc1-stock

This way, you’ll have multiple copies of the same baseline Python built with different feature sets. Note that you’ll need to install the pyenv-suffix plugin before proceeding with these commands. Otherwise, pyenv will ignore PYENV_VERSION_SUFFIX entirely, and you’ll end up overwriting existing installations under the same name!

Alternatively, you can install the python-build plugin, which comes with pyenv, as a standalone tool to gain more control. By doing this, you’ll be able to give the installed Python interpreters arbitrary names instead of just adding suffixes to their predefined versions:

$ PYTHON_CONFIGURE_OPTS='--disable-gil --enable-experimental-jit' \
  python-build -v 3.13.0rc1 $(pyenv root)/versions/py3.13

This command will compile Python 3.13 with both free threading and the experimental JIT, placing it under the py3.13 alias in pyenv.

While pyenv hides many tedious details about making custom Python builds, there may be times when you’ll want the ultimate control. In the next section, you’ll get an idea of what it takes to compile Python 3.13 with the experimental features from the source code by hand.

Custom Build From Source Code

You can get a copy of the Python 3.13 source code from GitHub by cloning the associated tag. For example, you can use this command:

$ git clone --branch v3.13.0rc1 --depth=1 https://github.com/python/cpython.git

Specifying the --branch parameter allows you to clone only the specific line of development rather than the complete repository with its entire history, which saves quite a bit of time. The name v3.13.0rc1 points to the final commit in the release candidate. The other parameter, --depth=1, limits the number of commits to fetch to the most recent one.

Alternatively, if you don’t have a Git client installed, then you can download a zipped archive with similar content using the GitHub web interface:

Locate the green button labeled Code, click on it, and choose Download ZIP from the modal window that pops up.

You can also download a pre-release version of Python from the official python.org website. The gzip tarball with the source code is comparable in size to the ZIP archive provided by GitHub.

Assuming you have all the necessary build tools and libraries installed, which you’ll learn about in the next section, you can change your current working directory into the cloned or downloaded CPython folder and go from there. For platform-specific instructions, refer to the Python Developer’s Guide.

The first step is to configure the build by running the configure script:

$ cd cpython/
$ ./configure --disable-gil --enable-experimental-jit --enable-optimizations

This script collects various information about your operating system, library versions, build options, preferred installation location, and so on. It then uses that information to adapt Python to your specific environment. Optionally, you can use it to perform cross-compilation, targeting another system or platform.

Here’s a quick recap of how to enable Python 3.13’s experimental features:

• Free Threading: To compile Python with free-threading support, you need to configure the build with the --disable-gil option.
• JIT Compiler: To compile Python with the experimental JIT compiler, you need to configure the build using the --enable-experimental-jit option.

While --disable-gil is a Boolean flag that can be either turned on or off, the --enable-experimental-jit switch takes optional values:

Only the yes and yes-off values enable support for the JIT compiler. When you don’t specify any value for this option, then it has the same effect as typing yes, which is the default value.

Unless you’re in the middle of implementing a new feature in CPython and need to recompile its source code often, it’s usually a good idea to --enable-optimizations when building a production-grade Python interpreter. This flag will improve the performance and memory management of the resulting code, which would otherwise be less than optimal. On the other hand, it’ll make the build take longer.

Because building Python from scratch requires many dependencies and can look different on various operating systems, there’s no one-size-fits-all solution. To streamline the process, you’ll find detailed step-by-step instructions based on Docker in the following section.

Install the Necessary Build Tools and Libraries

First, run a fresh Docker container using an official Ubuntu image maintained by Canonical:

$ docker run --name ubuntu --hostname ubuntu -it ubuntu

This will pull the latest ubuntu image from Docker Hub and run a new container based on it. If you’re reading this in the future, then you may want to explicitly request a specific snapshot of that image to ensure reproducibility:

$ docker pull ubuntu@sha256:8a37d68f4f73ebf3d4efafbcf66379bf3728902a8038616808f04e34a9ab63ee

The SHA-256 digest above represents the exact version of the image used at the time of making this tutorial. Future versions of the image may come with slightly different system packages, potentially causing problems.

The -it option in the docker run command instructs Docker to run your new container in an interactive mode. This lets you type commands as if you logged in to a remote server. The --name and --hostname parameters are optional but will allow you to find your container more easily using Docker’s command-line interface.

$ docker start ubuntu
$ docker attach ubuntu

The very first thing you should typically do after starting a new Docker container is upgrade your system packages. Here’s how you can do this in the Ubuntu container:

root@ubuntu:/# apt update
root@ubuntu:/# apt upgrade -y

This ensures that your container has the latest security patches and software updates. Additionally, when you’re starting from scratch, the apt update command retrieves a list of packages available for your Debian-based distribution. Depending on your base Docker image, that list might be initially empty, preventing you from installing any new packages.

Next, you can install the required dependencies to compile Python 3.13 from source code:

root@ubuntu:/# DEBIAN_FRONTEND=noninteractive apt install -y \
               wget unzip build-essential pkg-config zlib1g-dev \
               python3 clang

Note that you instruct the apt package manager to use a non-interactive mode by setting the DEBIAN_FRONTEND environment variable accordingly. This will skip the interactive prompt that the tzdata package would display to ask about your preferred time zone during installation. When set to noninteractive, this variable makes the tool use Etc/UTC as the default timezone without bothering you.

You may be wondering how the python3 package found its way into the list of dependencies needed to compile Python. Oddly enough, it may sound like you must already have Python to compile Python itself. Indeed, you need an earlier version of the Python interpreter, but only if you want to build Python 3.13 with the JIT support. That’s because the JIT build tool, which generates binary stencils for the copy-and-patch compiler, is a pure-Python script.

Notice the you also listed the clang package, which is another requirement for the JIT compiler in Python 3.13. Apart from the build-essential package, which brings the venerable GNU C compiler, you must also include the competing Clang compiler from the LLVM toolchain. Here’s why:

Clang is specifically needed because it’s the only C compiler with support for guaranteed tail calls (musttail), which are required by CPython’s continuation-passing-style approach to JIT compilation. (Source)

Fortunately, it’s a build-time-only dependency for the JIT to work. So, you won’t need LLVM to run the compiled Python interpreter afterward.

root@ubuntu:/# apt install -y gdb lcov libbz2-dev libffi-dev \
libgdbm-compat-dev libgdbm-dev liblzma-dev libncurses-dev \
libreadline-dev libsqlite3-dev libssl-dev lzma lzma-dev tk-dev uuid-dev

Now that your Docker container has all the essential build tools and libraries, you’re ready to proceed to downloading the Python source code.

Download Python Source Code Into the Container

While still in your Docker container, download an archive of CPython’s source code tagged as 3.13, either from GitHub or python.org, and extract it into a temporary location. You may replace a release candidate in the ZIP_FILE variable below with the final release if it’s available when you read this tutorial:

root@ubuntu:/# BASE_URL=https://github.com/python/cpython/archive/refs/tags
root@ubuntu:/# ZIP_FILE=v3.13.0rc1.zip
root@ubuntu:/# wget -P /tmp $BASE_URL/$ZIP_FILE
root@ubuntu:/# unzip -d /tmp /tmp/$ZIP_FILE

Here, you use Wget to download the ZIP file from the specified URL into the /tmp folder. In the next step, you’ll configure the build by enabling the experimental features introduced in Python 3.13.

Build Python With Free Threading and JIT Support

Navigate into the parent folder containing the Python source code, which you extracted from the downloaded archive in the previous step:

root@ubuntu:/# cd /tmp/cpython-3.13.0rc1/

Make sure to adjust the path as necessary if you opted for a more recent Python 3.13 release.

Now, you can run the configure script with custom build flags to enable free threading and the JIT compiler:

root@ubuntu:/tmp/cpython-3.13.0rc1# ./configure --disable-gil \
                                                --enable-experimental-jit \
                                                --enable-optimizations

This will configure your build environment and generate a new Makefile in the current working directory, which you can invoke with the make command:

root@ubuntu:/tmp/cpython-3.13.0rc1# make -j $(nproc)

Calling make without providing a specific target will trigger the default one, which is conventionally named all.

By using the -j option, you specify the number of jobs to run simultaneously. The nproc command returns the number of processing units on your computer for parallel execution. Despite running this within a Docker container, you still have access to all the CPU cores available on the host system. That’s the default behavior unless you specify otherwise when creating your Docker container.

Running multiple jobs in parallel will significantly speed up the compilation process, but you may need to wait a few minutes anyway. The good news is that you won’t be compiling Python often.

To verify if the compilation succeeded, try executing the resulting python binary file from the local folder:

root@ubuntu:/tmp/cpython-3.13.0rc1# ./python
Python 3.13.0rc1 experimental free-threading build
⮑ (main, Aug 26 2024, 15:10:57) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Great! The version information in the header confirms that you’ve made a custom build of Python 3.13. Additionally, your build includes free threading, as indicated by the printed message.

To check whether your build also comes with the experimental JIT compiler, you can define the following function in your shell, which takes the path to a python executable:

root@ubuntu:/tmp/cpython-3.13.0rc1# \
> function ha