tryexceptpass: Threaded Asynchronous Magic and How to Wield It

Photo Credit: Daniel Schwen via Wikipedia Threaded Asynchronous Magic and How to WieldIt A dive into python’s asyncio tasks and eventloops

Ok let’s face it. Clock speeds no longer govern the pace at which computer processors improve. Instead we see increased transistor density and higher core counts. Translating to software terms, this means that code won’t run faster, but more of it can run in parallel.

Although making good use of our new-found silicon real estate requires improvements in software, a lot of programming languages have already started down this path by adding features that help with parallel execution. In fact, they’ve been there for years waiting for us to take advantage.

So why don’t we? A good engineer always has an ear to the ground, listening for the latest trends in his industry, so let’s take a look at what Python is building for us.

What do we have sofar?

Python enables parallelism through both the threading and the multiprocessing libraries. Yet it wasn’t until the 3.4 branch that it gave us the asyncio library to help with single-threaded concurrency. This addition was key in providing a more convincing final push to start swapping over from version 2.

The asyncio package allows us to define coroutines. These are code blocks that have the ability of yielding execution to other blocks. They run inside an event loop which iterates through the scheduled tasks and executes them one by one. A task switch occurs when it reaches an await statement or when the current task completes.

Task execution itself happens the same as in a single-threaded system. Meaning, this is not an implementation of parallelism, it’s actually closer to multithreading. We can perceive the concurrency in situations where a block of code depends on external actions.

This illusion is possible because the block can yield execution while it waits, making anything that depends on external IO, like network or disk storage, a great candidate. When the IO completes, the coroutine receives an interrupt and can proceed with execution. In the meantime, other tasks execute.

The asyncio event loop can also serve as a task scheduler. Both asynchronous and blocking functions can queue up their execution as needed.

Tasks

A Task represents callable blocks of code designed for asynchronous execution within event loops. They execute single-threaded, but can run in parallel through loops on different threads.

Prefixing a function definition with the async keyword turns it into an asynchronous coroutine. Though the task itself will not exist until it’s added to a loop. This is usually implicit when calling most loop methods, but asyncio.ensure_future(your_coroutine) is the more direct mechanism.

To denote an operation or instruction that can yield execution, we use the await keyword. Although it’s only available within a coroutine block and causes a syntax error if used anywhere else.

Please note that the async keyword was not implemented until Python version 3.5. So when working with older versions, use the @asyncio.coroutine decorator and yield from keywords instead.

Scheduling

In order to execute a task, we need a reference to the event loop in which to run it. Using loop = asyncio.get_event_loop() gives us the current loop in our execution thread. Now it’s a matter of calling loop.run_until_complete(your_coroutine) or loop.run_forever() to have it do some work.

Let’s look at a short example to illustrate a few points. I strongly encourage you to open an interpreter and follow along:

import time import asyncio async def do_some_work(x): print("Waiting " + str(x)) await asyncio.sleep(x) loop = asyncio.get_event_loop() loop.run_until_complete(do_some_work(5))

Here we defined do_some_work() as a coroutine that waits on the results of external workload. The workload is simulated through asyncio.sleep .

Running the code may be surprising. Did you expect run_until_complete to be a blocking call? Remember that we’re using the event loop from the current thread to execute the task. We’ll discuss alternatives in more detail later. So for now, the important part is to understand that while execution blocks, the await keyword still enables concurrency.

For a better picture, let’s change our test code a bit and look at executing tasks in batches:

tasks = [asyncio.ensure_future(do_some_work(2)), asyncio.ensure_future(do_some_work(5)] loop.run_until_complete(asyncio.gather(*tasks))

Introducing the asyncio.gather() function enables results aggregation. It waits for several tasks in the same thread to complete and puts the results in a list.

The main observation here is that both function calls did not execute in sequence. It did not wait 2 seconds, then 5, for a total of 7 seconds. Instead it started to wait 2s, then moved on to the next item which started to wait 5s, returning when the longer task completed, for a total of 5s. Feel free to add more print statements to the base function if it helps visualize.

This means that we can put long running tasks with awaitable code in an execution batch, then ask Python to run them in parallel and wait until they all complete. If you plan it right, this will be faster than running in sequence.

Think of it as an alternative to the threading package where after spinning up a number of Threads , we wait for them to complete with .join() . The major difference is that there’s less overhead incurred than creating a new thread for each function.

Of course, it’s always good to point out that your millage may vary based on the task at hand. If you’re doing compute-heavy work, with little or no time waiting, then the only benefit you get is the grouping of code into logical batches.

Running a loop in a different thread

What if instead of doing everything in the current thread, we spawn a separate Thread to do the work for us.

from threading import Thread import asyncio def start_loop(loop): asyncio.set_event_loop(loop) l.run_forever() new_loop = asyncio.new_event_loop() t = Thread(target=start_loop, args=(new_loop,)) t.start()

Notice that this time we created a new event loop through asyncio.new_event_loop() . The idea is to spawn a new thread, pass it that new loop and then call thread-safe functions (discussed later) to schedule work.

The advantage of this method is that work executed by the other event loop will not block execution in the current thread. Thereby allowing the main thread to manage the work, and enabling a new category of execution mechanisms.

Queuing work in a different thread

Using the thread and event loop from the previous code block, we can easily get work done with the call_soon() , call_later() or call_at() methods. They are able to run regular function code blocks (those not defined as coroutines) in an event loop.