Building an open source library is not unlike building a democracy. The citizens of the programming community can freely choose whether or not to use the library, as well as how and whether to contribute back to the project. Meanwhile, the code of conduct lays out guidance that protects the rights of those citizens. As for governance, open source libraries are increasingly introducing mechanisms to enable different contributors and maintainers to cycle in and out of leadership roles. Finally, the API serves as a kind of rule-of-law, specifying how the library is to be used and providing direction as to how and where new components of the library can be integrated.
The majority of resources on API design frame the problem in terms of object-oriented programming concepts; which behaviors to encapsulate, how to organize the hierarchy to enable shared behavior across classes, how to handle multiple inheritance, etc. However, it’s often equally important to design around the citizenry of the project.
A key challenge of building an open source project is developing an API that serves a range of use cases, encourages a diversity of contributions, and can withstand the tests of time. When we set out to create the open source machine learning diagnostics library Yellowbrick, we were faced with tough decisions: will our users mostly be experienced ML practitioners or beginners? Should we prioritize ease of use or ease of contribution? How best to coordinate between the existing APIs of our two main dependencies, scikit-learn and Matplotlib?
Ultimately, these decisions come down to the same challenge faced by any democracy — namely, that everyone wants power! In open source API design, the central tradeoff is of complexity, and complexity tends to be inversely proportional to control. Making an API easy to use requires developers to abstract away a tremendous amount of complexity — complexity which then makes it increasingly more difficult for users to contribute.
In this talk, we give a guided tour through the various checks and balances involved in the implementation of the Yellowbrick API. We walk through our decisions, including some of our biggest challenges, successes, and lessons learned along the way, in the hopes that our case study can help inform the creators of new libraries in the Python ecosystem!