Download to read offline
Uploaded byIan Foster
ESnet6 and Smart Instruments
ESnet6 provides an ultra-fast and reliable network that enables new smart instruments for 21st century science. The network capacity has increased dramatically over time, with 2022 bandwidth being 500,000 times greater than 1993. This network allows rapid data transfer between facilities, such as replicating 7 petabytes of climate data between three labs. It also enables fast assembly and use of new instruments like high energy diffraction microscopy that can perform an analysis in 31 seconds. The integrated research infrastructure provided by Globus further supports use of remote resources and smart instruments that will drive scientific discovery.
![[.ppt]](https://cdn.slidesharecdn.com/ss_thumbnails/ppt1491-thumbnail.jpg?width=640&height=640&fit=bounds)
ESnet6 and Smart Instruments
- 1. How ESnet6 enablesnew smart instruments for 21st Century science Ian Foster Argonne National Laboratory The University of Chicago foster@anl.gov
- 2.
- 3.
- 4.
- 5. FitzRoy, The WeatherBook: A Manual of Practical Meteorology, 1863 … and uses data to discover extratropical frontal cyclones
- 6. Science cares aboutlatency, bandwidth, and reliability Pony express (1860) from St Joseph, Missouri, to Sacramento, California Latency: 10 days = 864000 seconds High Bandwidth: 50 letters x 1000 characters/letter = 50 KB / 864000 sec = 0.05 B/s Low Reliability: Accidents, floods, hostile locals, disease, etc. Poor
- 7. ESnet has workedsince 1986 to improve all three … ESnet6 network capacity, as of May 2022
- 8. … as measuredalso by bisection bandwidth 1993: 0.005 Gbps Bisection bandwidth in Gigabit/second 2022: 500,000 Gbps
- 9. https://dashboard.globus.org/esgf Shown: Statusas of March 10, 2022 4 to 6 GB/s 1.5 GB/s Replicating 7.4 petabytes of climate data from Livermore to Argonne and Oak Ridge National Labs 17,347,671 directories & 28,907,532 files From February 12 to May 4, 2022
- 10. This ultra-fast, stable,and reliable network enables the rapid assembly of new smart instruments
- 11. aining of DeepNeural Networks ources This ultra-fast, stable, and reliable network enables the rapid assembly of new smart instruments High energy diffraction microscopy: https://doi.org/10.48550/arXiv.2105.13967 7 seconds 7 + 19 + 5 = 31 s 5 seconds 19 seconds
- 12. High energy diffractionmicroscopy: https://doi.org/10.48550/arXiv.2105.13967 aining of Deep Neural Networks ources This ultra-fast, stable, and reliable network enables the rapid assembly of new smart instruments 7 seconds 7 + 19 + 5 = 31 s 5 seconds 19 seconds ModelTrain
- 13. This ultra-fast, stable,and reliable network enables the rapid assembly of new smart instruments
- 14. This ultra-fast, stable,and reliable network enables the rapid assembly of new smart instruments ModelTrain
- 15. This ultra-fast, stable,and reliable network enables the rapid assembly of new smart instruments Serial synchrotron crystallography: https://doi.org/10.1101/2022.04.06.487333
- 16. ModelTrain Mapping of smartinstruments to ESnet6-connected resources requires an integrated research infrastructure
- 17. For example, Globus,hybrid software-as-a-service for secure, reliable, performant data orchestration … https://globus.org
- 18. … and fortask orchestration
- 19. Polaris Bebop Cluster Argonne Leadership Computing Facility Laboratory Computing Research Center Eagle store APS Computing Orthros Cluster APSDM system Portal server Portal server Theta Advanced Photon Source Key: funcX agent Globus Connect agent Globus-accessible storage and computing (10,000s of systems) Endpoints are widely deployed at DOE facilities
- 20. Globus is anot-for-profit service operated for the research community by the University of Chicago, supported by ~200 subscribing institutions
- 21. ESNet6 ushers ina new era! An ultra-fast, predictable, and reliable network … … plus an integrated research infrastructure … enables new smart instruments that will drive discovery and innovation There’s still much to be done: • Build & apply new instruments • Reach the white space • Smart quantum instruments Thanks to ESnet, DOE ASCR, NSF, NIST, NIH, Argonne, Globus team, and many other partners