Julien Danjou: Packaging Python software with pbr

Packaging python has been a painful experience for long. The history of the various distribution that Python offered along the years is really bumpy, and both the user and developer experience has been pretty bad.

Fortunately, things improved a lot in the recent years, with the reconciliation of setuptools and distribute .

Though in the context of the OpenStack project, a solution on top of setuptools has been already started a while back. Its usage is now spread across a whole range of software and libraries.

This project is called pbr , for Python Build Reasonableness . Don't be afraid by the OpenStack colored themed of the documentation it is a bad habit of OpenStack folks to not advertise their tooling in an agnostic fashion. The tool has no dependency with the cloud platform, and can be used painlessly with any package.

How it works

pbr takes inspiration from distutils2 (a now abandoned project) and uses a setup.cfg file to describe the packager's intents. This is how a setup.py using pbr looks like:

import setuptools setuptools.setup(setup_requires=['pbr'], pbr=True)

Two lines of code it's that simple. The actual metadata that the setup requires is stored in setup.cfg :

[metadata] name = foobar author = Dave Null author-email = foobar@example.org summary = Package doing nifty stuff license = MIT description-file = README.rst home-page = http://pypi.python.org/pypi/foobar requires-python = >=2.6 classifier = Development Status :: 4 - Beta Environment :: Console Intended Audience :: Developers Intended Audience :: Information Technology License :: OSI Approved :: Apache Software License Operating System :: OS Independent Programming Language :: Python [files] packages = foobar

This syntax is way easier to write and read than the standard setup.py .

pbr also offers other features such as:

automatic dependency installation based on requirements.txt automatic documentation building and generation using Sphinx automatic generation of AUTHORS and ChangeLog files based on git history automatic creation of the list of files to include using git version management based on git tags

All of this comes with little to no effort on your part.

Using flavors

One of the feature that I use a lot, is the definition of flavors. It's not tied particularly to pbr it's actually provided by setuptools and pip themselves but pbr setup.cfg file makes it easy to use.

When distributing a software, it's common to have different drivers for it. For example, your project could support both PostgreSQL or mysql but nobody is going to use both at the same time. The usual trick to make it work is to add the needed library to the requirements list (e.g. requirements.txt ). The upside is that the software will work directly with either RDBMS, but the downside is that this will install both libraries, whereas only one is needed. Using flavors, you can specify different scenarios:

[extras] postgresql = psycopg2 mysql = pymysql

When installing your package, the user can then just pick the right flavor by using pip to install the package:

$ pip install foobar[postgresql] This will install foobar , all its dependencies listed in requirements.txt , plus whatever dependencies are listed in the [extras] section of setup.cfg matching the flavor. You can also combine several flavors, e.g.: $ pip install foobar[postgresql,mysql]

would install both flavors.

pbr is well-maintained and in very active development, so if you have any plans to distribute your software, you should seriously consider including pbr in those plans.