Download as PDF, PPTX
Uploaded byEUDAT
SKA Regional Sciences Centres - A Platform for Global Astronomy
The document discusses the role of regional science centres (SRCs) in supporting data-intensive radio astronomy with the Square Kilometre Array (SKA). SRCs will host SKA science archives, provide access to data and computing resources, and support for data analysis. They will form a global collaborative network to maximize access to and reuse of SKA data on an international scale. The SRC model could be extended to other large scientific projects that produce massive datasets and rely on distributed global communities for research.
More Related Content
Similar to SKA Regional Sciences Centres - A Platform for Global Astronomy
SKA Regional Sciences Centres - A Platform for Global Astronomy
- 1. ASTRON is partof the Netherlands Organisation for Scientific Research (NWO) Netherlands Institute for Radio Astronomy SKA Regional Science Centres A Platform for Global Astronomy EUDAT Conference, “Putting the EOSC vision into practice” Porto, Portugal, January 23, 2018 Michael Wise Head of Astronomy, ASTRON
- 2. ASTRON is partof the Netherlands Organisation for Scientific Research (NWO) Netherlands Institute for Radio Astronomy SKA and Data Intensive Astronomy Global Analysis Infrastructure for SKA Science Challenges and Boundary Conditions Current Initiatives and Relevance for the EOSC Talk Outline SKA Regional Science Centres A Platform for Global Astronomy
- 3. Today’s Astronomy isThe History of the Universe Cosmic Dawn (First Stars & Galaxies) Exploration of the Unknown Galaxy Evolution (Normal Galaxies at z~2-3) Cosmology (Dark Matter, Large-scale Structure) Testing General Relativity (Strong Gravity, Gravitational Waves) Cradle of Life (Planets, Molecules, SETI)
- 4. Michael Wise /EUDAT Conference / January 23, 2018 Title 4 ALMA JVLA E-ELT SKA Euclid CTAJWSTLSST FAST ALMA LOFAR Athena
- 5. Exponential Growth of Data Volumes DataIntensive Astronomy • From data poverty to data glut • From data sets to data streams • From static to dynamic, evolving data • From offline to real-time analysis • From centralized to distributed resources …and Complexity User interaction with the data has become the bottleneck in research!
- 6. Michael Wise /EUDAT Conference / January 23, 2018 Data Intensive Astronomy 6 § Science is increasingly driven by large data sets § Massive data collections and large scientific collaborations § Most science extraction is based on the archived data § Current instruments already producing petascale datasets New science infrastructures will produce exascale data!
- 7. Modern sky surveysobtain ~ 1012 –1018 bytes of images Catalogs ~ 108 –109 objects (stars, galaxies, etc.) and measure ~ 102 –104 numbers per object The Era of Big Surveys
- 8. Modern sky surveysobtain ~ 1012 –1018 bytes of images Catalogs ~ 108 –109 objects (stars, galaxies, etc.) and measure ~ 102 –104 numbers per object The Era of Big Surveys SDSS: 4.3 MB/sec, ~40 TB total LSST: 160 MB/sec, ~13 TB/night, 30 PB over 5 years LOFAR: ~100 TB/night, 6-10 PB/yr archived data MWA: 16 GB/sec, 6 PB/yr archived data ASKAP: spectral line data 80 PB/yr, Cont./HI sky 1-4 PB/yr CTA: 3-10 PB/yr archived data, raw data 10x higher SKA: 0.1-3 EB/yr archived science-ready data
- 9. Bolshoi Simulation (Klypinet al. 2011) Numerical Simulations Complex theoretical models are expressed as numerical simulations …and then must be matched against large collections of observational data ⇒ Can easily generate TBs per hour
- 10. Formation and Evolutionof Galaxies • The Dawn of Galaxies: Searching for the Epoch of First Light • 21-cm Emission and Absorption Mechanisms • Preheating the IGM • SKA Imaging of Cosmological HI • Large Scale Structure and Galaxy Evolution • A Deep SKA HI Pencil Beam Survey • Large scale structure studies from a shallow, wide area survey • The Ly- α forest seen in the 21-cm HI line • High Redshift CO • Deep Continuum Fields • Extragalactic Radio Sources • The SubmicroJansky Sky • Probing Dark Matter with Gravitational Lensing • Activity in Galactic Nuclei • The SKA and Active Galactic Nuclei • Sensitivity of the SKA in VLBI Arrays • Circum-nuclear MegaMasers • H2O megamasers • OH Megamasers • Formaldehyde Megamasers • The Starburst Phenomenon • Interstellar Processes • HII Regions: High Resolution Imaging of Thermal Emission • Centimetre Wavelength Molecular Probes of the ISM • Supernova Remnants • The Origin of Cosmic Rays • Interstellar Plasma Turbulence • Recombination Lines • Magnetic Fields • Rotation Measure Synthesis • Polarization Studies of the Interstellar Medium in the Galaxy and in Nearby External Galaxies • Formation and Evolution of Stars • Continuum Radio Emission from Stars • Imaging the Surfaces of Stars • Red Giants and Supergiant Stars • Star Formation • Protostellar Cores • Protostellar Jets • Uncovering the Evolutionary Sequence • Magnetic Fields in Protostellar Objects • Cool Star Astronomy • The Radio Sun • Observing Solar Analogs at Radio Wavelengths • Where are the many other Radio Suns? • Flares and Microflares • X-ray Binaries • Relativistic Electrons from X-ray Transients • The Faint Persistent Population • Imaging of Circumstellar Phenomena • Stellar Astrometry • Supernovae • Radio Supernovae • The Radio After-Glows of Gamma-ray Bursts • Pulsars • Pulsar Searches • Pulsar Timing• Radio Pulsar Timing and General Relativity • Solar System Science • Thermal Emission from Small Solar System Bodies • Asteroids • Planetary Satellites • Kuiper Belt Objects • Radar Imaging of Near Earth Asteroids • The Atmosphere and Magnetosphere of Jupiter • Comet Studies • Solar Radar • Coronal Scattering • Formation and Evolution of Life • Detection of Extrasolar Planets • Pre-Biotic Interstellar Chemistry • The Search for Extraterrestrial Intelligence SKA Science
- 11. Michael Wise /EUDAT Conference / January 23, 2018 §Australia §Canada §China §India §Italy §Netherlands §New Zealand §South Africa §Sweden §UK Potential new members: Spain, Portugal, Germany, France, others… 11 SKA Headquarters at Jodrell Bank, UK The Square Kilometre Array Host country for SKA-Mid Host country for SKA-Low
- 12. Michael Wise /EUDAT Conference / January 23, 2018 12 SKA1 MID in South Africa
- 13. Michael Wise /EUDAT Conference / January 23, 2018 13 SKA1 LOW in Australia Size of the Netherlands
- 14. Michael Wise /EUDAT Conference / January 23, 2018 SKA System Data Flow 14 300 Pbytes / year of fully processed science data products
- 15. Michael Wise /EUDAT Conference / January 23, 2018 Title §Suggested Deep Field Initiative §Recommended by ILT board following Sept 16 meeting §Single group of KSP experts, focused on common analysis of a deep field §Intended to improve coordination of KSP commissioning activities §Catalyst to achieve best strategy to reach thermal-noise limited images §Shared early science results §Discussed with KSPs at PIs Meeting and in subsequent telecon §Consensus that single deep field and working group not optimal way forward §Does not address many outstanding high priority calibration issues 15 28PB LOFAR Long Term Archive Current Radio Astronomy Archives
- 16. Michael Wise /EUDAT Conference / January 23, 2018 Title §Suggested Deep Field Initiative §Recommended by ILT board following Sept 16 meeting §Single group of KSP experts, focused on common analysis of a deep field §Intended to improve coordination of KSP commissioning activities §Catalyst to achieve best strategy to reach thermal-noise limited images §Shared early science results §Discussed with KSPs at PIs Meeting and in subsequent telecon §Consensus that single deep field and working group not optimal way forward §Does not address many outstanding high priority calibration issues 16 7PB Long Term Archive LOFAR73PB 300PB SKA Phase1 Science Archive Future SKA Science Archive 2017 2024
- 17. Ian Bird (IanBird, CERN)
- 18. Michael Wise /EUDAT Conference / January 23, 2018 Impact of Science Archives 18 § Assumes the archives are persistent and maintained § Assumes archival data is open and accessible to users § Assumes data products stored are appropriate for general use § Assumes users retrieving data have resources to process to a science result Science archives are a multiplier for total science output Publications from Hubble Space Telescope
- 19. Michael Wise /EUDAT Conference / January 23, 2018 SKA Regional Centres 19 Simplified description but highlights important factors • a collaborative network Model for collaborative network of SRC SKA Observatory SRC 1 SRC 2 SRC n SRC 3 Users SRC Portal § SKA Regional Centres (SRCs) will host the SKA science archive § Provide access and distribute data products to users § Provide access to compute and storage resources § Provide analysis capabilities § Provide user support § Multiple regional SRCs, locally resourced and staffed Primary interface for SKA data analysis
- 20. Michael Wise /EUDAT Conference / January 23, 2018 20 Global Network of Centres
- 21. Science Data Centre € Science Data Centre Funding Agencies What isa Science Data Centre? Astronomers Place to find their data Place to analyze their data Place to find support Observatories Place to curate their data Place to develop new capabilities Place to support their users Industry Way to fund science Way to maximize investment Way to spur innovation Place to find new challenges Place to collaborate with academia Place to test new technologies
- 22.
- 23. Michael Wise /EUDAT Conference / January 23, 2018 Regional Centre Functionality 23 Data AnalysisData ProcessingData Discovery § Observation database § Associated metadata § Quick-look data products § Flexible catalog queries § Integration with VO tools § Publish data to VO § Reprocessing and calibration § High resolution imaging § Mosaicing § Source extraction § Catalog re-creation § DM searches § Multi-wavelength studies § Catalog cross-matching § Light-curve analysis § Transient classification § Feature detection § Visualization
- 24. Michael Wise /EUDAT Conference / January 23, 2018 Cross-Cutting Functionality 24 Data AnalysisData ProcessingData Discovery § Observation database § Associated metadata § Quick-look data products § Flexible catalog queries § Integration with VO tools § Publish data to VO § Reprocessing and calibration § High resolution imaging § Mosaicing § Source extraction § Catalog re-creation § DM searches § Multi-wavelength studies § Catalog cross-matching § Light-curve analysis § Transient classification § Feature detection § Visualization DADI WP OBELICS WP
- 25. Portable processing pipelines forradio astronomy data LOFAR-based EOSC pilot project
- 26. Detection, Classification, Inference ASKAPHI Cube (Jurek et al. 2010) Automated detection and calculation of galaxy rotation curves in HI surveys SKA cube ~ 0.9 PB
- 27. Michael Wise /EUDAT Conference / January 23, 2018 27 User Science Platform (image courtesy of LSST Development team) 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) LSST Data Products LSE-163 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) ontainer deployment, batch computing, storage, identity management, …) User Databases 5A, Champaign-Urbana, IL • July 25-27, 2017 derpinnings Middleware Notebookorkflows User Storage User Computing LSST Stack (or equivalent for other instances) ing, storage, identity management, …) LSST Data (e.g., images) 5L • July 25-27, 2017 Notebook User Computing LSST Stack other instances) ity management, …) LSST Databases 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) User Storage 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) LSST Data Products LSE-163 User Computing 5nagement Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 e Platform Underpinnings APIs and Middleware NotebookUser Workflows User Databases User Storage User Computing LSST Stack ter Infrastructure (or equivalent for other instances) ent, batch computing, storage, identity management, …) LSST Software Data Access Center Infrastructure (or equivalent for other research infrastructure) (container deployment, batch computing, storage, identity management, …)
- 28. Michael Wise /EUDAT Conference / January 23, 2018 28 User Science Platform (image courtesy of LSST Development team) 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) LSST Data Products LSE-163 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) ontainer deployment, batch computing, storage, identity management, …) User Databases 5A, Champaign-Urbana, IL • July 25-27, 2017 derpinnings Middleware Notebookorkflows User Storage User Computing LSST Stack (or equivalent for other instances) ing, storage, identity management, …) SKA Data (e.g., cubes) 5L • July 25-27, 2017 Notebook User Computing LSST Stack other instances) ity management, …) SKA Databases 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) User Storage 5NSF/DOE Data Management Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 Science Platform Underpinnings Portal APIs and Middleware NotebookUser Workflows LSST Files (e.g., Images) LSST Databases L1, L2, Cal, EFD User Databases User Storage User Computing LSST Stack Data Access Center Infrastructure (or equivalent for other instances) (container deployment, batch computing, storage, identity management, …) LSST Data Products LSE-163 User Computing 5nagement Review • NCSA, Champaign-Urbana, IL • July 25-27, 2017 e Platform Underpinnings APIs and Middleware NotebookUser Workflows User Databases User Storage User Computing LSST Stack ter Infrastructure (or equivalent for other instances) ent, batch computing, storage, identity management, …) SKA Software Data Access Center Infrastructure (or equivalent for other research infrastructure) (container deployment, batch computing, storage, identity management, …)
- 29. Advanced European Networkof E-infrastructures for Astronomy with the SKA EUDAT ConferenceJanuary 23, 2018 29 Open Questions Where will the SKA science archive data be hosted? How can we take optimal advantage of existing infrastructure? What are the processing requirements and technologies to consider? How will that data be transported from the sites to Europe? What interfaces, tools, and techniques will users need for analysis? How do we setup and operate an international network of SRCs?
- 30. EUDAT ConferenceJanuary 23,2018 30 § WP1: Project Management § WP2: Governance Structure and Business Models § WP3: Computing and Processing Requirements § WP4: Data Transport and Optimal European Storage Topologies § WP5: Data Access and Knowledge Creation § WP6: User Services Design and specification of a distributed, European SKA Regional Centre to support the astronomical community in achieving the scientific goals of the SKA EC Horizon 2020 (€3 million) 13 countries, 28 partners, SKAO, host countries, e-infrastructures (EGI, GÉANT, RDA), NREN’s Three year project (2017-2019)
- 31. Advanced European Networkof E-infrastructures for Astronomy with the SKA EUDAT ConferenceJanuary 23, 2018 31 European SKA Regional Centre European SKA Regional Centre ICT Technology Centre National Science & Data Centre 1 National Science & Data Centre 2 National Science & Data Centre 3 Cloud Services (Commercial) Cloud Services (Academic) HPC Services Software Services Service Provider § Create a European-scale, federated Regional Centre for the SKA § Provide resources for SKA science extraction to users § Coordination with ICT communities, industry, and service providers § Facilitate shared development, interoperability, and innovation § European counterpart for engagement with other SRCs internationally
- 32. Michael Wise /EUDAT Conference / January 23, 2018 32 Organizational Challenges § Large and distributed global user community § Exascale computation and data management § Integration of global research infrastructures § Long-term development and operations § Sustainable funding models for common infrastructure § Clash of cultural mindsets (structured HPC versus interactive analysis)
- 33. Michael Wise /EUDAT Conference / January 23, 2018 33 Shared vs. Dedicated From “VRE in the Data for Science Approach to Common Challenges”, Zhao, Martin, & Jeffery § Long-term curation § Committed resources § Analysis latency § HPC culture shock § Requires outside expertise § Mechanism to harness community investments (personal grants, etc.) § Preserve environment for discovery and innovation Shared infrastructure is a boundary condition, but there are issues:
- 34. The European OpenScience Cloud EOSC Data Stewardship and Interoperability Preserving and connecting our data Shared Infrastructure Storage and computing resources serving multiple research communities Science Platform Bringing researchers and their analysis to the data and infrastructure How do we put the EOSC vision into practice?
- 35. Michael Wise /EUDAT Conference / January 23, 2018 35 SKA Timeline 1995 2000 2007 2008 2012 Early R&D Development of Concepts; Technology Choices Preparatory Phase 2013% 2014% 2015% 2016% 2018% 2020% 2024% Preliminary%Design% Review/System% Requirements%Review% Cri?cal%Design% Review% Go/No%Go%Decision% for%Construc?on% of%SKA1% Construc?on% Proposal%brought% to%SKA%Board% Start%of%%SKA1% Construc?on% SKA1% completed% SKA1%early% science%% 2017%
- 36. GLEAM Survey (Hurley-Walkeret al. 2016) The SKA will produce an incredible amount of complex data Extracting SKA science will require a global research infrastructure Regional Data Centres will be the primary interface for researchers A European SKA Regional Centre will be a key part of that network The AENEAS project is a first step in establishing a European SRC The EOSC is a natural platform for supporting SKA science Summary