Apache Airavata is a software platform that supports the creation and management of science gateways through a microservices architecture, promoting operational sustainability and collaboration in scientific research. The project, funded by NSF, emphasizes the integration of diverse technologies and tools to facilitate shared resources and services across various scientific domains. Key features include user identity management, scalable job and workflow management, and a focus on community-driven development and feedback to enhance functionality and user experience.

1. CI Software experiences with
Apache Airavata
Suresh Marru, Marlon Pierce
Sudhakar Pamidighantam, Chathuri Wimalasena
Research Technologies
Pervasive Technology Institute/UITS, Indiana University

2. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Outline
1. Motivation - SciGaP
• Operational Sustainability of Science Gateways
2. Reality vs Fantasy with emerging trends
• Micro-Services based architecture, DevOps,
Cloud Native, Big Data Tools…..

3. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Acknowledgement:
SciGaP project funded by NSF ACI Awards:
1339774, 1339856, and 1339649
PI: Marlon Pierce
Co-I: Suresh Marru
PI: Mark Miller
Co-I: Amit Majumdar
PI: Borries Demeler
GridChem Science
Gateway
CIPRES Science
Gateway
Ultrascan Science
Gateway
IU Cyberinfrastructure
Gateway
Neuroscience Gateway

4. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Knowledge and Expertise
Computational
Resources
Scientific
Instruments
Algorithms and
Models
Archived Data
and Metadata
Advanced Science Tools
Science Gateways:
Enabling & Democratizing Scientific Research

5. Scale number
of gateways
without having
to scale FTE’s
needed to
support them

6. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services

7. Science Gateway Platform as a Service (SciGaP)
User Identity
Management
Information, Monitoring &
Auditing
Application Programmer Interface
CIPRES
Science Gateways
Neuro
Science
Ultrascan BioVLABGAAMP
DES
SimWG
Param
Chem
Graphical Interfaces Admin Dashboards
XSEDE OSG
Future
Grid
Data
Nets
Campus
Clusters
Academic &
Commercial
Clouds
International
Grids
Data & Provenance
Management
Scalable SecureLoad Balanced Configurable Fault Tolerant Maintainable Performance
Job & Workflow
Management
Improve sustainability by converging on a single set of
hosted infrastructure services

8. Meritocratic open governance
Apache Way

9. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Airavata PMC
• Aleksander Slominski, IBM
• Amila Jayasekara, IU
• Ate Douma, One Hippo
• Chathura Herath, Knight Capital
• Chathuri Wimalasena, IU
• Chris A. Mattmann, NASA JPL
• Eran Chinthaka, Comprehend
• Heshan Suriyaarachchi,
• Lahiru Gunathilake, HighFive
• Marlon Pierce, IU
• Milinda Pathirage, IU
• Patanachai Tangchaisin, Ipsy
• Raminder Singh, IU
• Saminda Wijeratne, Georgia Tech
• Shahani Weerawarana, Auxenta
• Shameera Rathnayaka, IU
• Srinath Perera, WSO2
• Supun Nakandala, UoM
• Suresh Marru (Chair), IU
• Thilina Gunarathne, KPMG

10. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Apache Community:
Extend Airavata from your project or extend your
project from Airavata

11. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Plans for Continuous Integration & Deployment
• Use Ansible to setup a Apache Mesos Cluster
• To be deployed on Quarry Gateway Hosting at IU,
Amazon EC2, Rackspace and in future on JetStream
• Dockerize all Airavata and dependent services
• Use Marathon to deploy and maintain docker containers on
the mesos cluster.

12. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Airavata: A simple and open architecture to
facilitate external contributions

13. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
• Modern distributed
applications are rarely
composed of modules
written in a single language.
• Weaving together
innovations made in a range
of languages is a core
competency of successful
enterprises.
• Cross language
communications are a
necessity, not a luxury.
Gateways are Polyglot
* source: The Programmer’s Guide to Apache Thrift.

14. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Apache Airavata API’s built over Apache Thrift
Clean way to define IDLs with
richer data structures

15. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
API’s and CPI’s
• External clients interact with Airavata API
(based on Apache Thrift).
• Internally, components interact with each other
through Component Programming Interfaces (CPIs).

16. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Experiences
• Complex use cases evolve over time which challenges the need to
preserve a cleaner design.
• Support native clients in various languages.
• Thrift implementation of API/CPI services provide robust and scalable
servicers. Different implementations to switch based on QoS
(TSimpleServer, TThreadPoolServer TNonblockingServer,
THsHaServer).
• No need to marshall / unmarshall data – objects generated from
models are used internally as well.
• Auto-generation of Client-SDK’s makes it easy to do modifications to
the models.
• Convenient way to achieve backward compatibility.

17. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
What we learn from fellow Apache Projects:
• Working collaboratively.
• Constructive criticism is a blessing.
• Commit early and commit often.
• Test driven development
https://cwiki.apache.org/confluence/display/AIRA
VATA/Tests+in+Airavata
• Don’t be afraid to make a change,
after all version control systems (git,
svn) can be trusted.

18. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Community helps find New Stakeholders
• Current stakeholders have been slow to adopt
cloud computing.
• Scientific “big data” could change this.
• And we want Airavata to move beyond
academic computing.
• Where does Airavata fit in the Apache stack?

19. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Architecture Feedback:
architecture@airavata.apache.org
Not so fast, My friend

20. Apache Airavata Software ==> Science Gateways Platform as a Service (SciGaP) Hosted Services
Community Hangout
Mailing lists:
• architecture@airavata.apache.org
• dev@airavata.apache.org
• users@airavata.apache.org
Direct Contact:
Suresh Marru (smarru@iu.edu)
?? Questions ??