Opening Government With Python
log in to bookmark this presentaton
Novice / Poster
How can Python make your government more accountable? Take a look at how the Open State Project has begun the process of opening up state legislatures via web scraping and the creation of developer-friendly APIs. In this session the lead developers on the Open State Project will discuss best practices and lessons learned in web scraping, python data validation, and pitfalls of political data.
Abstract
The [Open State Project http://openstates.sunlightlabs.com] is an open source effort to enable citizen engagement by freeing data from all state legislatures and making it available in a common format via a RESTful API and regular data dumps.
The project goal is rather ambitious and as such has encountered a number of interesting problems that we feel would be of interest to the wider Python community. We also feel that by making people aware of what we are doing we can encourage other developers to attempt to affect similar change.
Among the problems that we will aim to cover in the poster session:
- web scraping tools and tips
- our solution to data validation of potentially untrustworthy scraped data
- useful Python utilities for hard problems (fuzzy string matching, name standardization, and more)
- political and legal considerations for such a project
- suggestions for other civic-minded Python projects