• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Python and MongoDB in Astronomy
 

Python and MongoDB in Astronomy

on

  • 863 views

An intro to galactic Astronomy for programmers and how Python and MongoDB fit into our development cycle.

An intro to galactic Astronomy for programmers and how Python and MongoDB fit into our development cycle.

Statistics

Views

Total Views
863
Views on SlideShare
853
Embed Views
10

Actions

Likes
1
Downloads
0
Comments
0

2 Embeds 10

http://www.slashdocs.com 6
http://paper.li 4

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Python and MongoDB in Astronomy Python and MongoDB in Astronomy Presentation Transcript

    • Python and MongoDB in AstronomyDan Foreman-MackeyCenter for Cosmology and Particle PhysicsDepartment of Physics @ NYU In collaboration with: David W. Hogg (NYU), Larry Widrow (Queen’s), Dustin Lang (Princeton), Jonathan Sick (Queen’s), Micha Gorelick (NYU) and many others...
    • Astronomy 101 How to Study the Cosmos Python, MongoDB, etc. Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Astronomy 101 How to Study the Cosmos Python, MongoDB, etc. Andromeda Case Studies The Internet The Milky WayDan Foreman-Mackey CCPP@NYU dfm.github.com
    • The Universe Galaxies Stars PlanetsDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the The Universe UniverseMade of? Galaxies Stars Planets Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the The Universe UniverseMade of? Galaxies Stars Are there other Earth- Planets like planets? Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the The Universe UniverseMade of? Galaxies Stars Are there other Earth- Planets like planets? Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Source: Wikipedia (Adam Evans)Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Rotational Speed RadiusDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Observed Rotational Speed RadiusDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Observed Rotational Speed WTF? RadiusDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Rotational Speed ? Observed WTF? RadiusDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Rotational Speed ? Observed WTF? Radius ?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Observed Rotational Speed WTF? Radius ? ?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Dark MatterDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Size of the Universe PyGotham b le rvaobse Time Source: NASA / WMAP Science Team Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Dark Energy 73% Heavy Elements 0.03% Dark Matter 23% Atoms 4% Source: NASA / WMAP Science Team WMAP Year 7 (Larson et al. 2011)Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Dark Energy 73%Source: DFM & Widrow (in prep) Heavy Elements 0.03% Dark Matter 23% Atoms 4% Source: NASA / WMAP Science Team WMAP Year 7 (Larson et al. 2011) Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Source: http://apod.nasa.gov Dark Energy 73%Source: DFM & Widrow (in prep) Heavy Elements 0.03% Dark Matter 23% Atoms 4% Source: NASA / WMAP Science Team WMAP Year 7 (Larson et al. 2011) Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • What is the Universe Made of? Source: http://apod.nasa.gov Dark Energy 73%Source: DFM & Widrow (in prep) Heavy Elements 0.03% Dark Matter 23% Atoms 4% Source: NASA / WMAP Science Team WMAP Year 7 (Larson et al. 2011) Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU Credit: The Millennium Simulation Project dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU Credit: The Millennium Simulation Project dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU Credit: The Millennium Simulation Project dfm.github.com
    • Data in AstronomyDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Credit: Jonathan Sick jonathansick.caDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Credit: Jonathan Sick jonathansick.caDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy MegaCam: 340 MegaPixels Credit: Jonathan Sick jonathansick.caDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy MegaCam: 340 MegaPixels Credit: NASADan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Imaging Source: NASA / ESADan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Imaging Spectroscopy Source: NASA / ESADan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Source: Riaud & Schneider (2007) Imaging Spectroscopy Spectroscopy Source: NASA / ESADan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy Source: Riaud & Schneider (2007) Imaging Spectroscopy Spectroscopy Source: NASA / ESADan Foreman-Mackey CCPP@NYU dfm.github.com
    • a Data in Astronomy lot is Open ofDan Foreman-Mackey CCPP@NYU dfm.github.com
    • a Data in Astronomy lot is Open of ! and there’s a lot of itDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy is Open 1990– 2000– 1997–2001 SDSSHubble archive.stsci.edu/hst sdss.org 2MASS www.ipac.caltech.edu/2mass GAIAPan-STARRS LSST PlannedDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy is Open 1990– 2000– 1997–2001 SDSSHubble archive.stsci.edu/hst sdss.org 2MASS www.ipac.caltech.edu/2mass GAIAPan-STARRS LSST PlannedDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Data in Astronomy is Open 1990– 2000– 1997–2001 SDSSHubble archive.stsci.edu/hst sdss.org 2MASS www.ipac.caltech.edu/2mass GAIAPan-STARRS LSST PlannedDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in?Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in? + Scientific Python StackDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in? + Scientific Python StackDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Where does Python fit in? + Scientific Python StackDan Foreman-Mackey CCPP@NYU dfm.github.com
    • What about ? Easy Flexible Pythonic ScalableDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSSDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS 728 SESAR ET AL. Ses S07 Labela Ntot A 84 B 144 C 54 D 8 E 11 F 11 Source: Sesar et al. (2010) G 10 H 7 I 4 J 26 K 8Dan Foreman-Mackey CCPP@NYU L 3 dfm.github.com M 5
    • Case Studies Variable Stars in Stripe 82 SDSSDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSSDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS 800k “Fields” ~ 12TB Imaging data > 1M “Target Stars”Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS Photons/Brightness TimeDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS Y Np(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS Y Np(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Stars Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS YN p(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Stars M Y M Ypconst ⌘ [(1 Pbad )pgood + Pbad pbad ] pvar ⌘ [(1 Pbad )pvar,good + Pbad pbad ] i=1 i=1 Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS YN p(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Stars M Y M Ypconst ⌘ [(1 Pbad )pgood + Pbad pbad ] pvar ⌘ [(1 Pbad )pvar,good + Pbad pbad ] i=1 i=1 Runs Runs Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS YN p(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Stars M Y M Ypconst ⌘ [(1 Pbad )pgood + Pbad pbad ] pvar ⌘ [(1 Pbad )pvar,good + Pbad pbad ] i=1 i=1 Runs Runs ⇤ ⇤ pgood ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ ) pvar,good ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ + ⌃2 ) var “Constant & Good” “Variable & Good” ⇤ pbad ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ + ⌃2 ) bad “Bad” Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS YN p(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1 Stars M Y M Ypconst ⌘ [(1 Pbad )pgood + Pbad pbad ] pvar ⌘ [(1 Pbad )pvar,good + Pbad pbad ] i=1 Npars = Nstars + Nruns + 6 i=1 Runs ~ 0 , ~ ⇤ , , ⌘, ⌃2 , PRuns , Pbad } 2 ⇥ = {f f var var , ⌃bad ⇤ ⇤ pgood ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ ) pvar,good ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ + ⌃2 ) var “Constant & Good” “Variable & Good” ⇤ pbad ⌘ N (Ci↵ |fi0 f↵ , 2 i↵ + 2 i↵ + ⌃2 ) bad “Bad” Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSSDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSSDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS N Y p(X|⇥) = [(1 Pvar )pconst (X ↵ |⇥) + Pvar pvar (X ↵ |⇥)] ↵=1Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies Variable Stars in Stripe 82 SDSS N X f ⇤ (t) = A0 + [An sin(!t) + Bn cos(!t)] n=1Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Case Studies CFHT4.7 Gigapixel mosaic of M31 Source: Jonathan Sick (Queen’s University)Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com Jonathan Sick Source: (Queen’s University)
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com Jonathan Sick Source: (Queen’s University)
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com Jonathan Sick Source: (Queen’s University)
    • Case Studies CFHTCosmic ray removal Flat-fielding Sky subtraction ... Mosaic making MongoDB Persistent Metadata + GeoSpatial Indexing img1.fits img2.fits ... img4000.fits Source: Jonathan Sick (Queen’s University) Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Astrometry.net Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Astrometry.net Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case Studies Lang & Hogg (2011) ~2500 JPGsDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case Studies Source: Lang & Hogg (2011)Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case Studies Source: Lang & Hogg (2011)Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case Studies Source: Lang & Hogg (2011)github.com/dfm/MarkovPyDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Crowdsourcing Comet Holmes Case StudiesDan Foreman-Mackey CCPP@NYU dfm.github.com
    • Growing Datasets UserInteraction Number Data And MuchCrunching Management More... ( )Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Growing Datasets UserInteraction Easy! Number Data And MuchCrunching Management More... ( )Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Growing Big Questions Datasets UserInteraction Easy! Number Data And MuchCrunching Management More... ( )Dan Foreman-Mackey CCPP@NYU dfm.github.com
    • Dan Foreman-Mackey Center for Cosmology & Particle Physics (NYU) dfm.github.com @__dfm__Dan Foreman-Mackey CCPP@NYU dfm.github.com