The Python programming language has a "batteries included" philosophy where a wide range of tools and libraries are available out of the box as part of the standard library. In addition to the packages available in the standard library, thousands of third-party programs and packages are available on the Python Package Index (PyPI). How do you distribute your own software, and how do you discover and install software from other developers? This presentation covers the current methods and the upcoming improvements implemented by the new distutils2/packaging library.

There are a number of Python libraries to help package your software for distribution, but they all leave a lot to be desired. distutils has been available in the standard library for years but leaves out many important features of modern packaging managers. The third-party setuptools package aims to fill some of the gaps and is currently used by many Python developers. distribute is a fork of setuptools that is more actively maintained and supports Python 3. The various options are confusing and lack standardization.

The absence of good packaging standards has complicated the development of Python package managers. Installers have trouble finding the most recent version of packages due to the lack of a complete standard for package version numbers. Pip must create a nonstandard record of files installed for each package to support uninstallation. Setuptools had to introduce its own method of defining dependencies to work with the way packages are distributed. These nonstandard additions have helped address real problems, but have contributed to the fragmentation of the Python packaging ecosystem.

The state of Python packaging is a mess, but the future is bright. New standards have been accepted to address these issues, and a new packaging library named packaging will be in the Python 3.3 standard library scheduled for release this summer. A backport of this library named distutils2 is already available for Python 2.4 to 3.2. How can you support these new standards in your Python packages?

The following topics will be covered:

• Overview of the current packaging libraries/tools and their limitations:
  • distutils, setuptools, and distribute
  • easy_install, pip, and buildout
• Improvements introduced by the new packaging standards:
  • Improved database of installed packages to support uninstallation (PEP 376)
  • Standardized method for defining dependencies (PEP 345)
  • Standardized package version numbers (PEP 386)
  • Static package metadata via setup.cfg (obsoletes setup.py)
  • More granular classification of data files to support OS packagers
• Implementation of the new standards: distutils2/packaging:
  • distutils2 for Python 2.4 to 3.2, packaging in the 3.3+ standard library
  • Modular design to support third-party tools
  • Includes pysetup, a barebones Python package manager
• Supporting distutils2/packaging in your Python packages:
  • Overview of setup.cfg
  • Using hooks to extend distutils2/packaging commands to support complex installations
  • Supporting older standards and distutils2/packaging simultaneously
• What happens to tools like pip? They will be updated to use packaging/distutils2 instead of setuptools/distribute.