Running ultra large telescopes in Python
log in to bookmark this presentaton
Running large scale projects is always a challenge. At SKA South Africa, it was decided some time ago that the expressiveness of Python outweighs its potential drawbacks. In this talk I'll present the problem domain of running a large telescope, with a special focus of how to handle it in Python.
Almost everything in this project is written in Python. The production servers run Python for monitoring, control and data processing. Our full testing infrastructure (e.g. simulators) are written in Python. Python technologies in use include NumPy, SciPy and h5py.
On of the central points of our implementation is the KATCP network protocol, which is built in a way that it's possible to introspect devices around you. It has implementations using a normal select loop, twisted, Ruby and C. I'll describe how this protocol works, why it's better than others available and how we manage a large set of devices almost purely within Python.
Lessons learned while implementing software for SKA could be used for any other large-scale high-complexity python scientific projects.
The SKA will be a two thousand antenna radio telescope with an effective collecting area of one square kilometer . Two countries are bidding to host this astronomical instrument -- South Africa and Australia. Construction of the SKA is expected to start in 2016 and be completed around 2024.
South Africa's bid includes the construction of a sixty-four antenna array named MeerKAT which will itself be a leading radio astronomy instrument until the SKA is completed . KAT7, a seven antenna prototype of MeerKAT is already in place on the MeerKAT site.
Many important parts of the KAT7 system use Python, including::
- monitoring of the entire system
- simulators for the entire system
- all low-level devices except the correlator and antenna motor controller
- control of antenna positions
- control of all ancillary devices
- generation and presentation of data for user displays
- post-processing of data
- analysis of data (via NumPy, SciPy and CASA)
A number of Python libraries have been released by the project under open source licenses::
- PySPEAD (the correlator data transfer protocol) 
- KATCP (the monitoring and control protocol) 
- an HTML5 backend for matplotlib 
- CASPER correlator interface utilities 
 http://en.wikipedia.org/wiki/Square_Kilometre_Array  http://en.wikipedia.org/wiki/MeerKAT  http://github.com/sratcliffe/PySPEAD  http://pypi.python.org/pypi/katcp/0.3.0  http://code.google.com/p/mplh5canvas/  http://pypi.python.org/pypi/corr/0.5.1