DataFest 2019, Introduction to Science Gateways

1. Science Gateways
Raminder Singh
Team Lead, Data Science Research and Facilitation
FAS Research Computing
1

2. What are Science Gateways?
2
• Provide online resource for accessing data,
software, computing services, and equipment
specific to the needs of a science or engineering
discipline
2

3. • Science gateways have involvement since 2003
• TeraGrid science gateways program was designed to
address
– Increasing use of the web for science
– Provide accessibility to supercomputers through these
interfaces
3
History
TeraGrid was an eScience
infrastructure project to
combine resources at eleven
partner sites . The project
started in 2001 and operated
from 2004 through 2011 with
a $53 million in funding

4. 44

5. 5

6. 66
My Journey
• Started in 2009 at Indiana University Science Gateways
Group
• First project was to create a generic product using software
developed for LEAD Gateway.
– Make the code reusable for other gateway projects.
– Experiment with web 2.0 technologies.
• Understand Grid Infrastructure and Security
• Develop programmatic APIs to interact with different
schedulers
– PBS, SGE, LSF, UGE, SLURM
• Integrate with data transfer interfaces
– GridFTP (FTP + UDP) – Now Globus

7. 7

8. 8

9. 9

10. 1010
High Performance
Computing
● Bridges (PSC)
● Comet (SDSC)
● Stampede (TACC)
● SuperMIC (LSU)
● Wrangler (TACC)
Cloud Computing
● Jetstream
(IU/TACC)

11. 1111
XSEDE ECSS
• Extended Collaborative Support Services (ECSS)
– Performance analysis
– Optimization
– Efficient use of accelerators
– I/O optimization
– Data analytics
– Visualization
– Use of XSEDE by science gateways
– Workflow automation
• Campus Champions (CC)
– Work closely with research group to understand requirements
– Solve local challenges at campus
– Collaborate with ECSS and other XSEDE programes

12. https://www.xsede.org/ecosystem/science-gateways/gateways-listing
https://catalog.sciencegateways.org/ 12

13. 1313
Example Gateways

14. 14
Sample Gateways

15. 15

16. 1616

17. 17
•Web-based platform for computational biomedical research (analysis
and data integration)
•Developed at Penn State, Johns Hopkins, OHSU and Cleveland
Clinic with substantial outside contributions
•Open source under Academic Free License
•More than 6,500 citations
•More than 125 public Galaxy resources
•100+ public servers, many more non-public
•Both general-purpose and domain-specific

18. 18

19. • Software Frameworks
– Agave - https://agaveapi.co/
– Apache Airavata - https://airavata.apache.org/
– Apache Rave - https://rave.apache.org/
– CIPRES Workbench - http://www.phylo.org/
– Galaxy Framework - https://galaxyproject.github.io/
• Science Gateway Community Institute
https://sciencegateways.org/
19
What else come out?

20. sciencegateways.org
20

21. Learn More
• https://sciencegateways.org/resources
• Monthly webinars second Wednesday of each month
– https://sciencegateways.org/engage/webinars
• Gateways 2019
– https://sciencegateways.org/web/gateways2019
• Science Gateway bootcamp
– https://sciencegateways.org/engage/bootcamp
• PEARC Conference
– https://www.pearc19.pearc.org/
21

22. Gateway Tools
22

23. Apache Airavata
23

24. 24

25. 25

26. 26
Gateways at Harvard

27. 27
Galaxy at Harvard

28. 2828
CEPR Secure EnvironmentApplication
Local Data
Store
Screener Pre-K InterventionAdult Capacity Future New
Local Data
Store
Local Data
Store
Local Data
Store
Local Data
Store
Data Exchange
Research
Data
CEPW DW File Repository
Central Repository
Controlled flow

29. 2929
Open OnDemand: Access Cluster and interactive Apps

30. Our Focus
• Train new users
• Deploy applications useful for user groups
• Use containers like Singularity
– To quickly deploy application
– Promote reproducibility of science
• Create a community to develop applications
• Work with Harvard Dataverse for data
management/archival
• Automation of workflows
• Deployment of useful workflow tools
• Provide resource beyond Harvard
– Cloud
– XSEDE
30

31. Questions?
Thanks!! r_singh@g.harvard.edu
Twitter: @ramansingh1
31