The document discusses the challenges in data-intensive science and the potential of outsourcing mundane research tasks through Software-as-a-Service (SaaS) solutions. It highlights how automation can accelerate discovery by reducing the time researchers spend on administrative tasks and data handling. The Globus Online platform is presented as an example of leveraging cloud services to streamline data movement and enhance research efficiency.

1. Accelerating data-intensive scienceby outsourcing the mundaneIan Foster

2. Alfred North Whitehead (1911) Civilization advances by extending the number of important operations which we can perform without thinking about them

3. J.C.R. Licklider reflects on thinking (1960) About 85 per cent of my “thinking” time was spent getting into a position to think, to make a decision, to learn something I needed to know

4. For example … (Licklider again) At one point, it was necessary to compare six experimental determinations of a function relating speech-intelligibilityto speech-to-noise ratio. No two experimenters had used the same definition or measure of speech-to-noise ratio. Several hours of calculating were required to get the data into comparable form. When they were in comparable form, it took only a few seconds to determine what I needed to know.

5. Research hasn’t changed much in 300 yearsAnalyzedataCollectdataPublish resultsIdentify patternsDesign experimentPose questionTest hypothesesHypothesize explanation

6. Discovery 1960: Data collection dominates Janet Rowley: chromosome translocationsand cancer

7. 800,000,000,000 bases/day30,000,000,000,000 bases/year Discovery 2010: Data overflows

8. 42%!!Meanwhile, we drown in administriviaThe Federal Demonstration Partnership’s faculty burden survey

9. You can run a company from a coffee shop

10. Salesforce.com, Google,Animoto, …, …, caBIG,TeraGrid gatewaysSoftwarePlatformInfrastructureVarieties of “* as a Service” (*aaS)

11. Salesforce.com, Google,Animoto, …, …, caBIG,TeraGrid gatewaysSoftwarePlatformAmazon, GoGrid,Microsoft, Flexiscale, …InfrastructureVarieties of * as a service (*aaS)

12. Salesforce.com, Google,Animoto, …, …, caBIG,TeraGrid gatewaysSoftwareGoogle, Microsoft, Amazon, …PlatformAmazon, GoGrid,Microsoft, Flexiscale, …InfrastructureVarieties of * as a service (*aaS)

13. Perform important tasks without thinking Web presenceEmail (hosted Exchange)Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Data analytics Content distributionIaaS

14. Perform important tasks without thinkingWeb presenceEmail (hosted Exchange)Calendar Telephony (hosted VOIP) Human resources and payroll Accounting Customer relationship mgmt Data analytics Content distributionSaaSIaaS

15. What about small and medium labs?

16. Research IT is a growing burdenBig projects can build sophisticated solutions to IT problemsSmall labs and collaborations have problems with bothThey need solutions, not toolkits—ideally outsourced solutions

17. Medium science: Dark Energy SurveyBlanco 4m on Cerro TololoImage credit: Roger Smith/NOAO/AURA/NSFEvery night, they receive 100,000 files in IllinoisThey transmit these files to Texas for analysis (35 msec latency)Then move the results back to IllinoisThis whole process must run reliably & routinely

18. Open transfer sockets vs. time[Image: Don Petravick, NCSA]

19. A new approach to research ITGoal: Accelerate discovery and innovation worldwide by providing research IT as a serviceLeverage software-as-a-service (SaaS) toprovide millions of researchers with unprecedented access to powerful research tools, and enable a massive shortening of cycle times intime-consuming research processes

20. Time-consuming tasks in scienceRun experimentsCollect dataManage dataMove dataAcquire computersAnalyze dataRun simulationsCompare experiment with simulationSearch the literatureCommunicate with colleagues

21. Publish papers

22. Find, configure, install relevant software

23. Find, access, analyze relevant data

24. Order supplies

25. Write proposals

26. Write reports

27. …Time-consuming tasks in scienceRun experimentsCollect dataManage dataMove dataAcquire computersAnalyze dataRun simulationsCompare experiment with simulationSearch the literatureCommunicate with colleagues

28. Publish papers

29. Find, configure, install relevant software

30. Find, access, analyze relevant data

31. Order supplies

32. Write proposals

33. Write reports

34. …Data movement can be surprisingly difficult Discover endpoints, determine available protocols, negotiate firewalls, configure software, manage space, determine required credentials, configure protocols, detect and respond to failures, determine expected performance, determine actual performance, identify diagnose and correct network misconfigurations, integrate with file systems, …BA

35. Grid (aka federation) as a service Globus Toolkit Globus OnlineBuild the Grid Components for building custom grid solutionsglobustoolkit.orgUse the Grid Cloud-hostedfile transfer serviceglobusonline.org

36. Globus Online’s Web 2.0 architectureCommand line interfacelsalcf#dtn:/scpalcf#dtn:/myfile \nersc#dtn:/myfileHTTP REST interfacePOST https://transfer.api.globusonline.org/ v0.10/transfer <transfer-doc>Web interfaceFire-and-forget data movementMany files and lots of dataCredential managementPerformance optimizationExpert operations and monitoringGridFTP serversFTP serversHigh-performancedata transfer nodesGlobus Connecton local computers

37. Globus Connect to/from your laptop25

38. Almost always faster than other methods0.001 0.01 0.1 1 10 100 1000Megabyte/fileArgonne  NERSC

39. Monitoring provides deep visibility

41. Globus Online runs on the cloud

42. Data movers scale well on Amazon

43. 11 x 125 files200 MB each11 users12 sitesSaaS facilitates troubleshooting

44. Moving 586 Terabytes in two weeks

45. NSF XSEDE architecture incorporatesGlobus Toolkit and Globus Online XSEDE33

46. Next steps: Outsource additional activitiesAnalyzedataCollectdataPublish resultsIdentify patternsDesign experimentPose questionTest hypothesesHypothesize explanation

47. A use case for the next stepsMedical image data is acquired at multiple sitesUploaded to a commercial cloudQuality control algorithms appliedAnonymization procedures appliedMetadata extracted and storedAccess granted to clinical trial teamInteractive access and analysisMore metadata generated and storedAccess granted to subset of data for education

48. Required building blocksGroup management for data sharingScheduled September, 2011, for BIRN biomedicalMetadata managementCreate, update, query a hosted metadata catalogData publication workflowsData movement, naming, metadata operations, etc.Cloud storage accessAnd HTTP, WebDAV, SRM, iRODS, …Computation on shared dataE.g., via Galaxy workflow system

49. www.globusoline.org37

50. SummaryTo accelerate discovery, automate the mundaneData-intensive computing is particularly full of mundane tasksOutsourcing complexity to SaaS providers is a promising route to automationGlobus Online is an early experiment in SaaS for science

51. For more informationFoster, I. Globus Online: Accelerating and democratizing science through cloud-based services. IEEE Internet Computing(May/June):70-73, 2011.Allen, B., Bresnahan, J., Childers, L., Foster, I., Kandaswamy, G., Kettimuthu, R., Kordas, J., Link, M., Martin, S., Pickett, K. and Tuecke, S. Globus Online: Radical Simplification of Data Movement via SaaS. Preprint CI-PP-05-0611, Computation Institute, 2011.

52. Thank you!foster@anl.govfoster@uchicago.edu