Transient Key Project● Static sources are boring● Transient – something that changes● Detect sources● Store extracted information● Associate with existing observations● Did something change?● This is where the Python fun starts!
Why Python● A lot of existing code● Good (scientific) libraries● Good astronomical libraries● Easy to use● Get it working first, then fast● NumPy is quite fast when used correctly
What do we use?● Python● NumPy / SciPy● Django● MonetDB● Pyrap (casacore wrapper) – Does all astro shizzle you can imagine
Distributed Computation● Home made (ASTRON)● SSH based● Calling remote scripts● Shared code base (NFS)● Protocol for passing arguments and results● Uses logging module a lot● Difficult to debug● Difficult to profile● Hadoop probably better
The Lofar Cluster● 100 nodes● 24 cores each● 20 TB raid5 storage● 40 Gbps InfiniBand● But for AARTFAAC we build our own● Eventually
Main loop● Extract source properties● Store properties● Do association● Detect transients
The data● 1 datacube per second● 10 frequency bands● In the future 10 images per second● In the future 4 different polarization● Non stop