“Pacific Research Platform Science Drivers”
Opening Remarks
PRP Science Driver PI Workshop
UC Davis
March 23, 2016
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
http://lsmarr.calit2.net
1
Vision: Creating a West Coast “Big Data Freeway”
Connected by CENIC/Pacific Wave
Use Lightpaths to Connect
All Data Generators and Consumers,
Creating a “Big Data” Freeway
Integrated With High Performance Global Networks
“The Bisection Bandwidth of a Cluster Interconnect,
but Deployed on a 20-Campus Scale.”
This Vision Has Been Building for 25 Years
DOE ESnet’s Science DMZ: A Scalable Network
Design Model for Optimizing Science Data Transfers
• A Science DMZ integrates 4 key concepts into a unified whole:
– A network architecture designed for high-performance applications,
with the science network distinct from the general-purpose network
– The use of dedicated systems for data transfer
– Performance measurement and network testing systems that are
regularly used to characterize and troubleshoot the network
– Security policies and enforcement mechanisms that are tailored for
high performance science environments
http://fasterdata.es.net/science-dmz/
Science DMZ
Coined 2010
The DOE ESnet Science DMZ and the NSF “Campus Bridging” Taskforce Report Formed the Basis
for the NSF Campus Cyberinfrastructure Network Infrastructure and Engineering (CC-NIE) Program
Based on Community Input and on ESnet’s Science DMZ Concept,
NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways
Red 2012 CC-NIE Awardees
Yellow 2013 CC-NIE Awardees
Green 2014 CC*IIE Awardees
Blue 2015 CC*DNI Awardees
Purple Multiple Time Awardees
Source: NSF
How Prism@UCSD Transforms Big Data Microbiome Science:
Preparing for Knight/Smarr 1 Million Core-Hour Analysis
12 Cores/GPU
128 GB RAM
3.5 TB SSD
48TB Disk
10Gbps NIC
Knight Lab
10Gbps
Gordon
Prism@UCSD
Data Oasis
7.5PB,
200GB/s
Knight 1024 Cluster
In SDSC Co-Lo
CHERuB
100Gbps
Emperor & Other Vis Tools
64Mpixel Data Analysis Wall
120Gbps
40Gbps
1.3Tbps
Next Step: The Pacific Research Platform Creates
a Regional End-to-End Science-Driven “Big Data Freeway System”
NSF CC*DNI Grant
$5M 10/2015-10/2020
PI: Larry Smarr, UC San Diego Calit2
Co-Pis:
• Camille Crittenden, UC Berkeley CITRIS,
• Tom DeFanti, UC San Diego Calit2,
• Philip Papadopoulos, UC San Diego SDSC,
• Frank Wuerthwein, UC San Diego Physics
and SDSC
Pacific Research Platform Regional Collaboration:
Multi-Campus Science Driver Teams
• Jupyter Hub
• Biomedical
– Cancer Genomics Hub/Browser
– Microbiome and Integrative ‘Omics
– Integrative Structural Biology
• Earth Sciences
– Data Analysis and Simulation for Earthquakes and Natural Disasters
– Climate Modeling: NCAR/UCAR
– California/Nevada Regional Climate Data Analysis
– CO2 Subsurface Modeling
• Particle Physics
• Astronomy and Astrophysics
– Telescope Surveys
– Galaxy Evolution
– Gravitational Wave Astronomy
• Scalable Visualization, Virtual Reality, and Ultra-Resolution Video 7
GPU JupyterHub:
2 x 14-core CPUs
256GB RAM
1.2TB FLASH
3.8TB SSD
Nvidia K80 GPU
Dual 40GbE NICs
And a Trusted Platform
Module
GPU JupyterHub:
1 x 18-core CPUs
128GB RAM
3.8TB SSD
Nvidia K80 GPU
Dual 40GbE NICs
And a Trusted Platform
Module
PRP UC-JupyterHub Backbone
UCB Next Step: Deploy Across PRP UCSD
Source: John Graham, Calit2
Next Step: Global Research Platform
Building on CENIC/Pacific Wave and GLIF
Current
International
GRP Partners
UC Davis PRP Science Drivers
• Particle Physics Data Analysis
– UCSD: A.Yagil, F. Wuerthwein (team leader; UC Davis: M. Chertok, J. Conway, R. Erbacher, M. Mulhearn, M.
Tripathi); UCI: A. Lankford, A. Taffard, D. Whiteson; UCSC: A. Seiden, J. Nielsen, B. Schumm; Caltech: H.
Newman; UCSB: C. Campagnari; UCR: R. Clare, O. Long, S. Wimpenny
• Telescope Surveys Team:
– LBNL: Peter Nugent; UCD: Tony Tyson; Caltech/IPAC/JPL, UCB, Stanford/SLAC, UCI, UCSC, UW
• Microbiome and Integrative ‘Omics Team:
– UCSD: Rob Knight; UCD: David Mills, Carlito Labrilla; Caltech: Sarkis Mazmanian; UCSF: Sergio Baranzini
• Data Analysis and Simulation for Earthquakes and Natural Disasters Team:
– UCB: Steve Mahin, with UCSD, UCD, UCLA, UCI, USC, Stanford, OSU, and UW
• Scalable Visualization and Virtual Reality Team:
– UCSD: Falko Kuester, Tom Levy, Jurgen Schulze; UIC: Maxine Brown; UHM, Jason Leigh; UCD: Louise Kellogg;
UCI: Magda El Zarki, Walt Scacchi; UCM, Marcelo Kallmann, Nicola Lecari; UvA: Cees de Laat.

Pacific Research Platform Science Drivers

  • 1.
    “Pacific Research PlatformScience Drivers” Opening Remarks PRP Science Driver PI Workshop UC Davis March 23, 2016 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net 1
  • 2.
    Vision: Creating aWest Coast “Big Data Freeway” Connected by CENIC/Pacific Wave Use Lightpaths to Connect All Data Generators and Consumers, Creating a “Big Data” Freeway Integrated With High Performance Global Networks “The Bisection Bandwidth of a Cluster Interconnect, but Deployed on a 20-Campus Scale.” This Vision Has Been Building for 25 Years
  • 3.
    DOE ESnet’s ScienceDMZ: A Scalable Network Design Model for Optimizing Science Data Transfers • A Science DMZ integrates 4 key concepts into a unified whole: – A network architecture designed for high-performance applications, with the science network distinct from the general-purpose network – The use of dedicated systems for data transfer – Performance measurement and network testing systems that are regularly used to characterize and troubleshoot the network – Security policies and enforcement mechanisms that are tailored for high performance science environments http://fasterdata.es.net/science-dmz/ Science DMZ Coined 2010 The DOE ESnet Science DMZ and the NSF “Campus Bridging” Taskforce Report Formed the Basis for the NSF Campus Cyberinfrastructure Network Infrastructure and Engineering (CC-NIE) Program
  • 4.
    Based on CommunityInput and on ESnet’s Science DMZ Concept, NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways Red 2012 CC-NIE Awardees Yellow 2013 CC-NIE Awardees Green 2014 CC*IIE Awardees Blue 2015 CC*DNI Awardees Purple Multiple Time Awardees Source: NSF
  • 5.
    How Prism@UCSD TransformsBig Data Microbiome Science: Preparing for Knight/Smarr 1 Million Core-Hour Analysis 12 Cores/GPU 128 GB RAM 3.5 TB SSD 48TB Disk 10Gbps NIC Knight Lab 10Gbps Gordon Prism@UCSD Data Oasis 7.5PB, 200GB/s Knight 1024 Cluster In SDSC Co-Lo CHERuB 100Gbps Emperor & Other Vis Tools 64Mpixel Data Analysis Wall 120Gbps 40Gbps 1.3Tbps
  • 6.
    Next Step: ThePacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Freeway System” NSF CC*DNI Grant $5M 10/2015-10/2020 PI: Larry Smarr, UC San Diego Calit2 Co-Pis: • Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UC San Diego SDSC, • Frank Wuerthwein, UC San Diego Physics and SDSC
  • 7.
    Pacific Research PlatformRegional Collaboration: Multi-Campus Science Driver Teams • Jupyter Hub • Biomedical – Cancer Genomics Hub/Browser – Microbiome and Integrative ‘Omics – Integrative Structural Biology • Earth Sciences – Data Analysis and Simulation for Earthquakes and Natural Disasters – Climate Modeling: NCAR/UCAR – California/Nevada Regional Climate Data Analysis – CO2 Subsurface Modeling • Particle Physics • Astronomy and Astrophysics – Telescope Surveys – Galaxy Evolution – Gravitational Wave Astronomy • Scalable Visualization, Virtual Reality, and Ultra-Resolution Video 7
  • 8.
    GPU JupyterHub: 2 x14-core CPUs 256GB RAM 1.2TB FLASH 3.8TB SSD Nvidia K80 GPU Dual 40GbE NICs And a Trusted Platform Module GPU JupyterHub: 1 x 18-core CPUs 128GB RAM 3.8TB SSD Nvidia K80 GPU Dual 40GbE NICs And a Trusted Platform Module PRP UC-JupyterHub Backbone UCB Next Step: Deploy Across PRP UCSD Source: John Graham, Calit2
  • 9.
    Next Step: GlobalResearch Platform Building on CENIC/Pacific Wave and GLIF Current International GRP Partners
  • 10.
    UC Davis PRPScience Drivers • Particle Physics Data Analysis – UCSD: A.Yagil, F. Wuerthwein (team leader; UC Davis: M. Chertok, J. Conway, R. Erbacher, M. Mulhearn, M. Tripathi); UCI: A. Lankford, A. Taffard, D. Whiteson; UCSC: A. Seiden, J. Nielsen, B. Schumm; Caltech: H. Newman; UCSB: C. Campagnari; UCR: R. Clare, O. Long, S. Wimpenny • Telescope Surveys Team: – LBNL: Peter Nugent; UCD: Tony Tyson; Caltech/IPAC/JPL, UCB, Stanford/SLAC, UCI, UCSC, UW • Microbiome and Integrative ‘Omics Team: – UCSD: Rob Knight; UCD: David Mills, Carlito Labrilla; Caltech: Sarkis Mazmanian; UCSF: Sergio Baranzini • Data Analysis and Simulation for Earthquakes and Natural Disasters Team: – UCB: Steve Mahin, with UCSD, UCD, UCLA, UCI, USC, Stanford, OSU, and UW • Scalable Visualization and Virtual Reality Team: – UCSD: Falko Kuester, Tom Levy, Jurgen Schulze; UIC: Maxine Brown; UHM, Jason Leigh; UCD: Louise Kellogg; UCI: Magda El Zarki, Walt Scacchi; UCM, Marcelo Kallmann, Nicola Lecari; UvA: Cees de Laat.

Editor's Notes

  • #8 Campus Cyberinfrastructure – Network Infrastructure and Engineering (CC-NIE) Campus Cyberinfrastructure – Infrastructure, Innovation, and Engineering (CC-IIE) Campus Cyberinfrastructure – Data, Networking, and Innovation (CC-DNI) NSF 15-534 incorporates Data Infrastructure Building Blocks (CC-DNI-DIBBs) – Multi-Campus / Multi-Institution Model Implementation from Program Solicitation NSF 14-530