Coroutines, event loops, and the history of Python generators

Audience level:
March 10th 1:20 p.m. – 2:15 p.m.


This talk traces lightweight concurrency from Python 2.2's generators, which enabled semi-coroutines as a mechanism for scheduling "weightless" threads; to PEP 342, which created true coroutines, and hence made event-driven programming easier; to 3rd party libraries built around coroutines, from older GTasklet and to the current Greenlet/gevent and Twisted Reactor.


This talk aims to provide both a practical guide and theoretical underpinnings to the use of generator-based lightweight concurrency in Python.

  1. Lightning tour of generator constructs. Why generator-based scheduling is particularly useful for event-based programming.
  2. Simple example of a "trampoline" or scheduler.
  3. Slightly fleshed out example of scheduler with discussion of data-passing issues.
  4. Examples using GTasklet to make coroutine code look more like familiar sequential code (the framework is based on greenlets rather than generators, but accomplishes similar purpose).
  5. Brief examples of Twisted Reactors and Deferreds.
  6. Limits of generator-based concurrency (i.e. doesn't help with multiple cores and multiple servers). "Throw at the wall" list of ways to generalize to larger scales than single cores.