Async programming in Python: a survival guide

If you’re a regular Python developer, you probably heard of one of the new buzzy features recently added to the language: async programming (the other one being type annotations). These new features are becoming more and more a part of modern python idioms, and sooner or later you’re going to face a codebase using them. New frameworks, such as FastAPI, leverage these features to create fast and concise programs.

In this guide, we’ll go over some of the fundamentals and applications of this (not so) new programming technique.

Concurrency and paralellism

Before proceeding to the language-specific examples, it’s important to cover up some fundamentals. The first concept we have to know is the difference between concurrency and parallelism. Parallel processes occur at the same time, for example when you have a multi-core machine. It’s a strategy suitable for CPU-bound processes, like mathematical simulations and machine learning, where most time is spent on heavy calculations. On the other hand, concurrent processes are defined as tasks that finish in overlapping time. It doesn’t mean they are running in the same instant though. One example would be multitasking in a single-core machine. The concurrent model usually works well when we have IO-bound processes, that is, processes that spend a lot of time idle, waiting for some data to come in or to be sent out.

There are a few ways this concurrency can be achieved. One could be using threads and letting the OS do the scheduling automatically through time slicing. Another way would be to use a technique called cooperative multitasking. In cooperative multitasking, your functions should be instrumented by adding a few placeholding keywords, async and await, for instance, to mark the locations in which your program is expected to be waiting for a long time, and therefore can have its execution paused. When the function reaches that point, it gives control back to the scheduler and so allows it to leverage this CPU spare time to run other functions. Methods and functions that behave this way are called coroutines.

The entity responsible for running the coroutines is called the Event Loop. It acts as a scheduler, determining which coroutines should be resumed next. The event loop continuously monitors the underlying OS for events that might happen, such as IO or scheduled time events, generated by a sleep call. This is implemented by having a loop constantly making a system call, like select, to monitor the file descriptions used by the coroutines. The select call informs the loop about which descriptors are ready to be written to/read from, and by association, which coroutines are ready to be resumed.

Enough theoretical stuff, let’s get into Python.

The asyncio module

At the time of writing, Python has a few implementations of Event Loops, but let’s stick with the built-in library called asyncio for the sake of the examples. It provides a set of tools to run and manage coroutines.

Consider the snippet below:

import asyncio

async def sleep(name):
    print(f"  {name}: Starting sleep")
    await asyncio.sleep(1)
    print(f"  {name}: Ending sleep")

async def main():
    print("Runnning coroutines sequentially:")
    await sleep("A")
    await sleep("B")

    print("Running coroutines concurrently:")
    await asyncio.gather(
        sleep("C"),
        sleep("D"),
    )

asyncio.run(main())

The code is pretty self-explanatory, but the two interesting things here are the use of the async keyword to define the coroutines, and the usage of the await keyword, which defines points in the code where the function can be paused/resumed. At the end of the script, we called asyncio.run to start the loop, handing over the main coroutine. All coroutines should be either awaited or passed to the event loop.

Pro tip

Rule number one of async programming is: never block the event loop! To do so, it’s essential that within the coroutines, from the calling function down to the operating system, there shouldn’t be any blocking calls, such as sleep, synchronously read/write from files/sending network requests, etc. This means that even though you’re not obliged to convert all your codebase to use async at once, the parts that you do should only be using libraries prepared to be run from event loops.

The good news is that most packages implementing async provide a compatible interface with their synchronous counterparts:

• requests: httpx, aiohttp
• Flask: FastAPI, Quart
• psycopg2: asyncpg

ASGI: a new standard supporting WebSockets

One of the great applications of async in web development is implementing WebSocket servers. As opposed to regular HTTP request/response cycles, which are synchronous and stateless, when using WebSockets, the server and the client must keep a stateful connection active that can last for hours, exchanging messages once in a while, so the server needs to be able to keep lots of processes running on his end.

Having a server running hundreds or thousands of threads and processes doesn’t scale well, but, with cooperative multitasking, we have a feasible alternative. Assuming that most of the time, the tasks are going to be idle, waiting for some data or user interaction, event loops seemed a really good fit to keep the multitasking overhead at bay.

The problem is: the whole application stack must be rewritten to achieve this. Web frameworks like Django were not prepared to be used this way. And even the WSGI protocol, used to exchange data between the server and the web apps, had to the rethought.

That’s when the ASGI standard came about, a new protocol for integration of frameworks and the webservers supporting async. With the advent of this new interface, many new tools were created or improved to support the emerging standard, including frameworks like Django (using Channels), as well as webservers like Daphne and Uvicorn. The ASGI interface also added the capability for the servers to implement Server Pushes over both WebSockets and HTTP/2, enabling the backend to send data to the browser without being requested. This enables the creating of new software architectures that were almost impossible before.

Spawning tasks

Until now, we have covered the following ways of running the coroutines:

• Passing it to the event loop using asyncio.run
• Awaiting on them
• Running multiple coroutines concurrently by using asyncio.gather

But, what if we wanted to spawn a task dynamically ? For instance, suppose we have a main loop handling events coming from the standard input, and in certain cases, we should spawn a long-running process in parallel to avoid blocking the loop.

The following example depicts this scenario. A third-party library called aiofiles is used, because the native read system call would block the loop.

import asyncio
import aiofiles

async def delayed_echo(text):
    await asyncio.sleep(1)
    print(f"  Your text back, after 1 sec: {text}")

async def main():
    async with aiofiles.open("/dev/stdin", "r") as f:
        while True:
            text = await f.readline()
            task = asyncio.create_task(delayed_echo(text))

asyncio.run(main())

As you can see, the asyncio.create_task function is provided to allow running on-demand tasks concurrently. It’s the async equivalent of creating threads. The function returns a task handle as well, although it’s being ignored in our case. The handle allows the application to cancel or to join the task waiting for its conclusion.

Conclusion

This article gives a glimpse of the fundamentals and will help you start to get acquainted with async programming. This is a really strong buzzword right now, not only in the python world but in web software development in general. There’s a whole ecosystem of apps and tools being developed to leverage this new approach. I think it’s really something worth going deeper if you want to juice up your skills as a developer capable of writing modern Python code.

The emerging of WebSocket-enabled servers also impacts front-end development. Maybe it could mean a shift in what it’s known as best practice for web software design nowadays. This is noticeable by the bubbling up of new tools as Hotwire, Livewire, and Liveview.

My suggestion is to go beyond this post and read other authors as well, there’s a lot of material out there. When you feel comfortable, try to build something meaningful as you go over the asyncio documentation exploring more code examples.