Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

GlobusWorld 2021: Arecibo Observatory Data Movement

The story of how Globus helped move petabytes of data from the Arecibo Observatory to TACC, and thereby save 50+ years of data for posterity and future research. Presented at the GlobusWorld 2021 conference by George Robb III.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

GlobusWorld 2021: Arecibo Observatory Data Movement

  1. 1. Arecibo Observatory Data Movement - so much more than data George B. Robb III, grobb3@es.net EPOC - Performance Chaser ESnet - Infrastructure Team National Science Foundation Award #1826994
  2. 2. Arecibo Observatory •Treasure of Astronomical Sciences since 1963 Inspiration for all, just a few highlights: • Active radio telescope • NEO, planetary, and atmospheric imaging • Full spectrum 1Hz - 10 GHz • Distance to Pleiades cluster • Metric tons of scientific discoveries and publications. https://www.naic.edu/ao/legacy-discoveries 5/12/2021 © 2021, Engagement and Performance Operations Center (EPOC) 2
  3. 3. Takes a hit, keeps on making science! • 2006 - NSF 15% Budget cut across astronomical sciences • 2007 - Arecibo budget slashed from$10.5 million to $8 million (NASA pitches in with ~$2.6 million to help operations budget). • 2011 - NSF implements Data Management Plan requirement. • 2015 - Facilities director Kerr quits due to funding clashes. • 2018 - University of Central Florida takes on stewardship. Operations working tirelessly making science to happen. 5/12/2021 3 © 2021, Engagement and Performance Operations Center (EPOC)
  4. 4. Takes another hit. • Hurricane Maria ( Sept 20, 2017 ) • Category 4 • Damage sustained • R&E ( I2) network connection lost and remained offline since 2012 to UPR 10GbE ! • Too expensive to repair due to budget cuts. • Operations resume when the power came back... And the science goes on! 5/12/2021 4 © 2021, Engagement and Performance Operations Center (EPOC)
  5. 5. Takes another hit 5/12/2021 5 © 2021, Engagement and Performance Operations Center (EPOC) •5.0 - 6.4 Earthquakes ( January 7-11, 2020 ) •Operations still going • shaken not stirred ( yes 007 reference)
  6. 6. Takes and yet another 5/12/2021 6 © 2021, Engagement and Performance Operations Center (EPOC) • Tropical Storm Isaias ( July 30, 2020 ) • And, SCIENCE vs operations • still going… • First [auxiliary] cable snap • (Aug 10, 2020) • Second cable [ primary ] snaps • (Nov 6, 2020)
  7. 7. Thermodynamics, Economics, and Gravity win. (Dec 1, 2020) 5/12/2021 7 © 2021, Engagement and Performance Operations Center (EPOC) D a t a C e n t e r B u i l d i n g 1 .
  8. 8. Thermodynamics, Economics, and Gravity win. (Dec 1, 2020) - In a SECOND all changed. 5/12/2021 8 © 2021, Engagement and Performance Operations Center (EPOC)
  9. 9. 5/12/2021 9 © 2021, Engagement and Performance Operations Center (EPOC) MISSION: Get the DATA to a safe stable state • 2 Petabytes of Golden Copy need to move. • Fiber is still cut! • Transfers in MB/s with PB to go. • Data movement at a scale of years • Network Attached Storage (NAS) “Appliances” being used with sneakernet • 100+ Terabytes at a time. ( Full capacity of NAS device ) • Onsite team hand carries this priceless data to the closest 10GbE links at University of Puerto Rico - Mayaguez (RUM) and Engine-4 Collaboration Space. Operations still going, harder than ever, hand carrying copies of priceless scientific data to safety.
  10. 10. 5/12/2021 10 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance: • Team forms to see what can be done to get the data moving to safe stable state ( we are in FULL DR ) . • EPOC - Architect solutions to implement • Globus - let's get the data on the wire • Anyone want to saturate a link ( Also, redline a NAS ) • UPR (RUM) and Engine-4 - Provide transport 10GbE link! • UCF - let’s facilitate the migrations • TACC - let’s catch and secure the data
  11. 11. 5/12/2021 11 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance - EPOC • Worked with UCF teams to understand the nature of the data. • Architect course of action: • People are the network (let’s help make this happen). • The the fabulous community really knocked this out of the park THANK YOU
  12. 12. 5/12/2021 12 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance - EPOC • Understand Problem set: • Site is not accepting new installations • Oh, Global PANDEMIC • We can’t just ship hardware. • Closest 10GbE link an hour drive away. • THANK YOU University of Puerto Rico (RUM) - UPR and Engine-4 • Network Attached Storage (movable storage appliance) devices small but, mighty solutions that can be tuned
  13. 13. 5/12/2021 13 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance - TACC •Offered to catch and distribute the data • Waved a magical storage wand to create a safe and secure 2 Petabyte landing zone. • 10GbE link analysis and monitoring • NetSage and dashboarding • Globus native landing zone ( woo hoo )!
  14. 14. 5/12/2021 14 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance - Globus •Always part of the team and joined in person with additional support - THANK YOU! • rsync vs. globus • ( spoiler: globus is an obvious winner ) • Timescale was years to move 2PB • Data Transfer Node (DTN) tuning only helps! >> Teams are pulling data from tape at this point!!!
  15. 15. 5/12/2021 15 © 2021, Engagement and Performance Operations Center (EPOC) Move that data! - Globus
  16. 16. 5/12/2021 16 © 2021, Engagement and Performance Operations Center (EPOC) Call for assistance - so much more than data. FULL Disaster Recovery - The instrument fell! • Community engaged immediately • Initial transfers 10s of Megabytes per second • Tuned Globus transfers 100s of Megabytes per second • Tools we have • Globus - share that data! • ESnet - https://es.net Fasterdata, DMZ, DTN, DME, Tuning, and much more. • EPOC - https://epoc.global • Supercharge your human network. • We use only tools available, no such thing as silver bullets • Hit the mission targets.
  17. 17. 5/12/2021 17 © 2021, Engagement and Performance Operations Center (EPOC) Words of caution ( or fear ) - FULL Disaster Recovery - THE INSTRUMENT FELL. ( think for a moment on this ) • The network is part of the instrument! • NSF’s 2011 Data Management Plan requirement • https://www.nsf.gov/bfa/dias/policy/dmp.jsp • Large facilities what is the current status of: • Data management plan • Replica site • Backups • Fiber paths
  18. 18. 5/12/2021 18 © 2021, Engagement and Performance Operations Center (EPOC) Words of caution ( or hope ) - FULL Disaster Recovery - THE INSTRUMENT FELL. • Call for assistance and engage your communities! • EPOC - https://epoc.global epoc@iu.edu • ESnet - https://es.net • DTN - https://fasterdata.es.net/science-dmz/DTN/ • DME - https://fasterdata.es.net/performance-testing/2019-2020-data-mobili ty-workshop-and-exhibition/2019-2020-data-mobility-exhibition/ • ISI - Center of Excellence pilot, for Data Management • https://www.nsf.gov/awardsearch/showAward?AWD_ID=1842042 National Science Foundation Award #1826994

×