The Jetstream cloud is a collaboration between Cyverse partners TACC and University of Arizona, University of Chicago, Johns Hopkins University, and Indiana University to bring the flexibility and ease-of-use of CyVerse Atmosphere to the entire community of science, at a much larger scale. Jetstream is a cloud resource operated as part of XSEDE, and built from two independent OpenStack clusters, each capable of supporting thousands of virtual machines and data volumes. The clusters are integrated via the user-friendly "Atmosphere" interface developed by CyVerse, with authentication enabled by Globus, and, unlike the CyVerse cloud also offer full access to Openstack web service APIs. Jetstream features a diverse catalog virtual machine templates. One can launch a personal Galaxy server, do advanced biostatistics, use Matlab, or experiment with new technologies like Docker, all on Jetstream. This talk highlights the unique capabilities of Jetstream and provides information about how researchers from all over can access it.

1. funded by the National Science Foundation
Award #ACI-1445604
On-Demand Cloud Computing for Life
Sciences Research and Education
Matthew Vaughn(@mattdotvaughn)
ORCID 0000-0002-1384-4283
Director, Life Science Computing
Texas Advanced Computing Center
PI @ Jetstream | Cyverse | Araport | CODE@TACC

2. Overview
• What is Cloud Computing?
• Why do we need Cloud Computing?
• What is Jetstream?
• How is Jetstream different from CyVerse
Atmosphere?
• How can one get started using
Jetstream?

3. “…A model for enabling ubiquitous, convenient,
on-demand network access to a shared pool of
configurable computing resources (e.g.,
networks, servers, storage, applications and
services) that can be rapidly provisioned and
released with minimal management.” NIST
Definition of Cloud Computing
http://www.nist.gov/itl/csd/cloud-102511.cfm
What is Cloud Computing?

4. Why do we need Cloud Computing?

5. SCIENTIFIC COMPUTING THEN
• C/C++/FORTRAN/PERL/SHELL
• MPI
• LAPACK/BLAS/PETSC
• GRID ENGINE
• UNIX*
• X86/PPC/SPARC
• COPROCESSORS

6. SCIENTIFIC COMPUTING NOW
LANGUAGES
• Python 2 & 3
• R
• Julia
• Perl
• Matlab
• Java
• Scala, Clojure, etc
• .NET
• C/C++
• Swift
• Haskell
• Go
• Javascript
FRAMEWORKS
• MapReduce Hadoop, Storm,
Pachyderm, Cloudera
• Event & Streaming: Kinesis,
Azure Stream Analytics, Camel,
Streambase
• Deep/Machine Learning:
Watson, Azure BI, Tensorflow,
Caffe
• In-memory parsing: Kognito,
Apache Spark
• Containers: Docker, Rocket,
MESOS, Kubernetes
• Cloud: AWS, GCE, OpenStack,
vCloud, Azure
HARDWARE
•Many-core computing - 50-100
threads/node*
• Xeon / Xeon Phi
• GPU
• OpenPower
• ARM
• ShenWei
• Google TPU
• Multi-level memory architecture
• Hierarchical storage
• FPGAs
• Quantum-like systems

7. Why don’t we all use Cloud?
Cloud computing is complex to adopt
• It can require a high degree of technical
knowledge
• Users can incur unexpected costs
• Data management and movement can be
challenging
• There’s no systematic support for adopting
cloud technologies or services

8. Jetstream is a Managed Cloud tailored to science and
engineering

9. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Supported by NSF ACI, operated by your friends
Construction
Application / Community LeadsManagement & Operations
Vendors

10. It’s built to address multiple user perspectives
User Accomplishments Role
• Learned how to use the shell and how to work with Linux
• Mastered using R to develop plots for his manuscript
• Published VM to IU Scholarworks to allow reproducible analysis
Laboratory
scientist
• Launches an instance and has full sudo access to customize
• Developed a software with R and Python library dependencies
• She updates it regularly by creating VM image snapshots
Informatics
specialist
• Linked several Jetstream instances with Apache Hadoop
• Worked with XSEDE ECSS to import existing Amazon image
• Built a simple science gateway to allow others to use his tools
Core facility staff

11. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Leadership/national class machines
Institutional HPC, HTC systems
Jetstream serves the “Long Tail”

12. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Jetstream serves the “Long Tail”
Researchers, developers, and scientists who:
– Need between 1 and a few hundred cores
• RIGHT NOW
• For the foreseeable future, but not forever
– Want to fully customize the OS and configuration for their
research computing environment
– Are working with cloud-native applications & workflows
– Use interactive mode for their computing & analytics

13. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
• Jetstream is also helpful for:
– Science gateway operators
• Run web applications, databases, and services (front-end)
• Use it for on-demand provisioned compute capacity
– STEM educators teaching a variety of subjects
• Create a reference VM appliance
• Provision an entire classroom
• Minimize need for local IT support
• It democratizes access to cloud-native technologies and approaches
– no credit card or PO needed
Jetstream serves the “Long Tail”

14. Jetstream is an innovative combination of powerful
hardware and sophisticated software

15. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Hardware Systems Overview

16. Platform Overview
Atmosphere APIGlobus Auth
Atmo Services XSEDE Services
OpenStack CEPH
Indiana University
OpenStack CEPH
TACC
OpenStack CEPH
Potentially, Others
Web App

17. Jetstream is easy-to-use Cyberinfrastructure

22. GateOne Web Shell
• Zero install SSH client
• Allows tablets or
Windows to easily
use the system
• Supports screen-
casting and terminal
sharing
Can also use native SSH
client with SSH keys

23. Jetstream is programmable Cyberinfrastructure

24. Jetstream enables access modes uncommon in
academic research computing
Web Services
• OpenStack API
• Compatible with all clients + libraries
• Python, Java, Go + CLI
• Atmosphere API – Pre-release
• Preview docs @
http://docs.atmospherev2.apiary.io/
• CLI later in 2017
Finicky
Automation, Orchestration, and Workflow
• Marathon/MESOS
• Docker Compose, Machine, Swarm
• Kubernetes
• CloudMan & Elasticluster
• OpenStack Magnum
Configuration Management Tools
• Vagrant / Terraform
• Chef
• Ansible
• Puppet
In Dev

25. Annotate a Genome Using WQ-Maker
Data analysis in Rstudio
Run your own JupyterHub
Publish a VM with DOI
Operate a Galaxy server
Teach a Bioinformatics class
Experiment with Deep Learning
Master Docker or K8S
Mine data using ArcGIS
Develop web services

26. Jetstream is a little different from CyVerse Atmosphere
(but not too much)

27. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Compared to CyVerse Atmosphere, Jetstream…
• Is allocated via XSEDE competitive process
• Gets its support from XSEDE-sponsored staff
• Uses XSEDE credentials and SSH keys for login
• Supports more APIs and automation
• Has more, bigger virtual machine hosts & better
networking
• Is not integrated with CyVerse Data Store

28. How can one get started using Jetstream?

29. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
It’s pretty easy:
• Sign up for a free XSEDE account (portal.xsede.org)
• Get an allocation (startups take 1 business day to approve)
• Start using it use.jetstream-cloud.org
• Read more and get help at www.jetstream-cloud.org
• Cite Jetstream in your papers (instructions on web)
• Follow the project on Twitter @jetstream_cloud

30. Questions and Discussion
Slides at slideshare.net/mattdotvaughn

31. Spin Up Your Own Cluster
elasticluster start slurm-js-iu
“Launch a 3-node SLURM
cluster with Gluster storage”
Cluster in a box
• Slurm, SGE, Torque
• Centos, Ubuntu
• Hadoop
• Gluster, Ceph, NFS
• Ansible
-> XSEDE XNIT
• compatible clusters

32. Use Marathon, MESOS, & Docker

33. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
What comes next?
• Passed acceptance (handily) in May
• OpenStack clouds, CEPH storage, and software components functional
– Substantial software integration and development required post-
acceptance. Terra incognita!
• Running in “Early operations mode”
– Extra maintenance days
– A few rough edges (especially for power users)
• Full production - Sep 1, 2016
• More features and capabilities will come
• Public roadmap soon

34. Free tier makes it really easy to get
started on public cloud$
Our idea: Let any user with active
XSEDE User Portal account use a
small (but functional) slice of
Jetstream
• Get an XSEDE account
• Sign in to XSEDE User Portal
• Click “Trial Jetstream Access”
button
• Get access to Jetstream in about
30 minutes

35. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Jetstream in context
• HTC system
• GPGPUs
• Virtual Clusters
• Gateways
• Data Intensive
• Portal-based
configuration
• Services
• Data sets
• Large Memory
• Data Intensive
Computing
• Managed VM
• Self-service VM
• Gateways
• Minimal disk
• Extensible
COMET WRANGLER BRIDGES JETSTREAM

36. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Jetstream in context (2)
Blue Waters & Stampede 1/2
Comet, Bridges, Regional HPC, Public Cloud$

37. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Summary
Jetstream is a public, production cloud platform
• Offers on-demand interactive shell + VNC
• Supports configurable software environments
• Enables configurable computing resources
• Encourages work with cloud-native software
• Empowers novice, intermediate, & expert users

38. QUESTIONS?
vaughn@tacc.utexas.edu
@mattdotvaughn
www.slideshare.net/mattdotvaughn
@jetstream_cloud
http://use.jetstream-cloud.org/
http://portal.xsede.org
https://github.com/jetstream-cloud

39. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
How can I use Jetstream?
• An XSEDE User Portal (XUP) account is required. They are free!
Get one at https://portal.xsede.org
• Read the Allocations Overview -
https://portal.xsede.org/allocations-overview
• Write a successful allocation request – start with a Startup or
Education request - https://portal.xsede.org/successful-requests

40. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
Where can I get help or learn more?
• Production:
– Web app: http://use-jetstream-cloud.org/
– User guides: https://portal.xsede.org/jetstream
– XSEDE KB: https://portal.xsede.org/knowledge-base
– Email: help@jetstream-cloud.org
– Campus Champions: https://www.xsede.org/campus-
champions
– Training Videos / Virtual Workshops (TBD)

41. Expanding NSF XD’s reach and impact
Around 299,000 researchers, educators, & learners received NSF support in
2012-2013
– Only 1.5% completed a computation, data analysis, or visualization
task on XSDEDE program resources
– Less than 3% had an XSEDE Portal account
– 70% of researchers surveyed* claimed to be resource constrained
Why don’t they use XSEDE systems?
– Activation energy is pretty high
– HPC resources are scarce and not well-matched to their needs
– They just don’t need that much capability
* https://www.xsede.org/xsede-nsf-release-cloud-survey-report

42. Funded by the National Science Foundation
Award #ACI-1445604
http://jetstream-cloud.org/
What is Jetstream?
A production cloud platform for NSF-sponsored researchers
• Provides on-demand interactive computing and analysis
• Enables configurable environments and architectures
• Supports computational reproducibility and sharing
• Democratizes access to cloud-native software
• Focused on ease of use for all adopters
Expands the community of users who benefit from NSF investment in
shared cyberinfrastructure

43. Flavor vCPUs RAM Storage Per Node
m.tiny 1 2 20 46
m.small 2 4 40 23
m.medium 6 16 130 7
m.large 10 30 230 4
m.xlarge 22 60 460 2
m.xxlarge 44 120 920 1
VM Host Configuration
• Dual Intel E-2680v3 “Haswell”
• 24 physical cores/node @ 2.5
GHz (Hyperthreading on)
• 128 GB RAM
• Dual 1 TB local disks
• 10GB dual uplink NIC
• Running Centos7+KVM
Hypervisor
Hardware Specifics
CEPH Storage
• 20x Dell 730xd per cloud
• 2x10Gbs bonded NIC per 730xd
• Running CEPH 0.94.5 Hammer
• Configured as OpenStack Storage
• Storage is XSEDE-allocated
• Implemented on backend as OpenStack Volumes
• Each user gets 10 volumes up to 500GB total storage
• Exploring object storage as well but that’s in the future

44. Key components: What is OpenStack?
• Free and open source software for creating
private and public clouds
• Started in 2010 by Rackspace and NASA,
now managed by OpenStack Foundation
• Widespread adoption across industry and
public sector
• Modular architecture with a 6-month update
cycle
https://www.openstack.org/

45. Key components: Everything else
Component Function
Atmosphere Web application + middleware for providing user-friendly
provider-agnostic IaaS
Globus Auth Provides powerful identity and access management functions
that are easily integrated into web and mobile applications
XSEDE Services Centralized account management, allocation, and reporting
CEPH Distributed object store and file system designed to provide
excellent performance, reliability and scalability