Change the future

Thursday 1:20 p.m.–4:40 p.m.

Python 3 Metaprogramming

David Beazley

Audience level:
Experienced
Category:
Core Python (Language, Stdlib)

Description

Some of the most significant changes in Python 3 are related to metaprogramming. In this tutorial, I'll cover decorators, class decorators, descriptors, and metaclasses. However, the focus will be on idioms and examples that are only made possible using features that are unique to Python 3. For instance, making free use of function annotations, signatures, new metaclass features and more.

Abstract

(ed. This is a very rough draft. Will revise and flesh out more if accepted).

  1. Introduction to Concepts. A high-level overview of what metaprogramming is about and why you would care about it. In short, it's about avoiding code repetition (Don't Repeat Yourself), but also essential for understanding the advanced magic found in frameworks.

  2. Function objects. A look at function objects and what's inside. Includes function annotations, attributes, underlying code objects, signatures, and so forth. Needed to set up later sections that focus on manipulating function objects.

  3. Closures. Using closures to make code. I'll show how you can often use closures to write things that look like simple macros.

  4. Decorators. Overview of writing function decorators. Examples will freely make use of Python-3 only features such as signatures and annotations. Will probably do some advanced examples doing things such as enforcing contracts or multiple dispatch.

  5. The Object System. A brief overview of the Python object system and how it's put together. Will discuss the implementation of objects, inheritance, the MRO (C3 algorithm), super() and related topics. Will also discuss the concept of mixin classes as they pertain to later topics.

  6. The descriptor protocol. Overview of writing descriptor objects and customizing attribute access. This section is needed as background for later class-based examples involving decorators and metaclasses.

  7. Class decorators. Examples that show how class objects can be modified through the use of decorators. This will include instantiating descriptors and using class decorators as an alternate to mixins.

  8. Metaclasses. A detailed introduction and overview to what a metaclass actually is. Will present numerous examples of metaclasses in action with some contrast to class decorators. Special attention will be given to the prepare() method of metaclasses that enable entirely new kinds of metaprogramming techniques in Python 3. Will also include some material on abstract base classes.

  9. (Maybe). Import hooks. I might talk about some things you can do with the import hook machinery. Not sure about time though. Could be dropped.

By the end of the tutorial, I hope to have built up a fairly advanced framework related to data modeling/type systems. It will use all of the features together to do some pretty amazing things.