A Python framework developed for 'roundtrip' processing of an Atmospheric Lagrangian Particle Dispersion Model (LPDM) is presented. Contents of the framework include a custom package for the generations of input and configuration files, f2py created modules for controlling program flow and reading the large volumes of output data, and processing and plotting routines for displaying the output to the web using Plone.
The talk will have three components: Development of the main processing package, development of the web-based tools for output, and an overview of the (ongoing) refactoring process.
The primary codebase is a Python package that was written specifically for the processing of LPDM model output. LPDMs are used extensively in the scientific community including researchers at NASA, NOAA, and throughout Europe for forecasting and analysis of the atmospheric transport of gasses and dust. The presented package initially served to plot output data, but has grown recently in complexity and function. It now provides tools for generating input data, manipulating the model itself, and distributing jobs for parallel processing using the multiprocessing package.
The model data is without value if it is not available for use. Through the use of Python, we have transitioned from the generation of limited gif output files, to creating a 'Through The Web' browsing experience which allows for custom plotting of the data, cycling through model forecast products, and managing the code base.
The development of the package was largely a learning experience. From the time the package was originally created the developer has learned a great deal about the meaning of 'pythonic'. A large effort has been undertaken recently to refactor the code to improve it, and more importantly make it more 'team friendly' as the development efforts have grown. The package is planned to have a v1.0 release date coincident with PyCON. [Stay Tuned... ]
This talk will present the technical details of the Python package that underlies most the functionality of the code base. Further, we will present 'Lessons Learned' from the perspective of a non-expert programmer/developer and show how Python was used to replace several proprietary products in our workflow.