Blocking tasks in Tornado

Every now and then a new discussion is raised on Tornado’s mailling list about what is the best way to execute blocking tasks. It turns out that there are 3 feasible options, in order of increasing complexity:

  • Optimize blocking calls. Often, a slow DB query, or an overly complicate template are the blocking bottleneck. Rather than complicating the webserver, the first thing to try is to speed them up. This is sufficient 99% of the time.
  • Execute the slow task in a separate thread or process. This means off-loading the task to a different thread (or process) to the one running the IOLoop, which is then free to accept other requests.
  • Use an asynchronous driver/library to run the task. For example, something like gevent, motor and the like.

This blog post is about the second option, in particular using Python’s concurrent.futures package.

For example, consider this simple web server, with a blocking “SleepHandler” handler:

import time

import tornado.ioloop
import tornado.web

class MainHandler(tornado.web.RequestHandler):

    def get(self):
        self.write("Hello, world %s" % time.time())

class SleepHandler(tornado.web.RequestHandler):

    def get(self, n):
        self.write("Awake! %s" % time.time())

application = tornado.web.Application([
    (r"/", MainHandler),
    (r"/sleep/(\d+)", SleepHandler),

if __name__ == "__main__":

Try to visit http://localhost:8888/sleep/10 in one tab and http://localhost:8888/ in another: you’ll see that “Hello, world” is not printed in the second tab until the first one has finished, after 10 seconds. Effectively, the first call is blocking the IOLoop, who cannot serve the second tab.

You can make the “SleepHandler” Tornado-friendly by executing it in another thread. Below is a decorator that can be used to “unblock” it:

from concurrent.futures import ThreadPoolExecutor
from functools import partial, wraps

import tornado.ioloop
import tornado.web

EXECUTOR = ThreadPoolExecutor(max_workers=4)

def unblock(f):

    def wrapper(*args, **kwargs):
        self = args[0]

        def callback(future):

            partial(f, *args, **kwargs)
            lambda future: tornado.ioloop.IOLoop.instance().add_callback(
                partial(callback, future)))

    return wrapper

class SleepHandler(tornado.web.RequestHandler):

    def get(self, n):
        return "Awake! %s" % time.time()

Very simply, the unblock decorator submits the decorated function to the thread pool, which returns a future; a callback is added to this future to return control to the IOLoop, by calling add_callback, which eventually will call self.finish and conclude the request.

Note that the decorated function must be itself be decorated with tornado.web.asynchronous, in order to not call self.finish too soon! Moreover, self.write is not thread-safe (thanks mrjoes!) therefore it must be called in the main thread with the future’s result as parameter.

Full code is below, available on gist.

January 22, 2013
407 words

tornado futures python

Contact me

Hi! I'm Lorenzo Bolla, Senior Software Architect in London, UK.

When I'm not programming in Python, Rust, Go or Haskell, I am playing either basketball, piano or chess.