The GAdget DataFrame Library for analyzing GADGET simulation data.

1. Extending Pandas:
Jacob Hummel, PhD
@astrohummel
Hire me!
GADFLY
The GAdget DataFrame LibrarY

6. GADGET: GAlaxies with Dark matter and Gas intEracT

7. Side note: Astronomers love horribly
tortured acronyms.
• SAURON, GADZOOKS!, THINGS, SPUD, SINFONI
• ACBAR: Arcminute Cosmology Bolometer Array Receiver
(It’s a photon trap!)

8. I want to analyze my data using pandas,
but loading GADGET data can be a pain.
Over the course of a PhD it eventually pays off – easier access à more science!

9. Major pain points:
1. Stored in custom-formatted hdf5 files.
2. Data is way too big to fit in memory.
Need some way to get data into pandas while keeping
everything properly indexed and allowing for effective
memory management.

10. My solution was to subclass
pandas.DataFrame:
Then, implement methods to load the data (h5py) and enable dynamic
memory management.

11. Gadfly has several other features, but I’ll
just highlight one:
• The ability to visualize your data is crucial.
• Unfortunately, GADGET data really needs to
be volume rendered to easily interpret.
• Requires a triply-nested for loop--> SLOW!
• Solution: use Numba.

13. Gadfly is available on Github:
A complete description of the framework is
available in this paper:
Seeking data science opportunites in Salt Lake City, Utah.
@astrohummel Web: