Coroutines, event loops, and the history of Python generators
- Type:
- Talk
- Audience level:
- Experienced
- Category:
- Concurrency
March 10th 1:20 p.m. – 2:15 p.m.
Description
This talk traces lightweight concurrency from Python 2.2's generators, which enabled semi-coroutines as a mechanism for scheduling "weightless" threads; to PEP 342, which created true coroutines, and hence made event-driven programming easier; to 3rd party libraries built around coroutines, from older GTasklet and peak.events to the current Greenlet/gevent and Twisted Reactor.
Abstract
This talk aims to provide both a practical guide and theoretical underpinnings to the use of generator-based lightweight concurrency in Python.
- Lightning tour of generator constructs. Why generator-based scheduling is particularly useful for event-based programming.
- Simple example of a "trampoline" or scheduler.
- Slightly fleshed out example of scheduler with discussion of data-passing issues.
- Examples using GTasklet to make coroutine code look more like familiar sequential code (the framework is based on greenlets rather than generators, but accomplishes similar purpose).
- Brief examples of Twisted Reactors and Deferreds.
- Limits of generator-based concurrency (i.e. doesn't help with multiple cores and multiple servers). "Throw at the wall" list of ways to generalize to larger scales than single cores.