PyCon 2019 in Cleveland, Ohio

Sunday 10 a.m.–1 p.m. in Expo Hall

Thinking Inside the Box: How Python Helped Us Adapt to An Existing Data Ingestion Pipeline

Eddie Schuman


We will cover how we used Python to adapt to a large institutional processing setup. We used Python to create the definitions, configuration files, and supplementary metadata for each of the weather radars we worked with. We used a variety of custom tools to interface with existing systems and processes that would have been infeasible to work with otherwise. We took advantage of one of Python’s greatest strengths: its flexibility. We used it to perform the bulk of our data processing with NumPy, created custom utility functions to encourage code reuse, and created custom scripts for interfacing with the institutional data processing framework we worked within.