Download to read offline
![• FIONAs PCs [a.k.a ESnet DTNs]:
– ~$8,000 Big Data PC with:
– 1 CPU
– 10/40 Gbps Network Interface Cards
– 3 TB SSDs or 100+ TB Disk Drive
– Extensible for Higher Performance to:
– +Up to 38 Intel CPUs
– +Up to 8 GPUs [4M GPU Core Hours/Week]
– +NVMe SSDs for 100Gbps Disk-to-Disk
– +Up to 160 TB Disks for Data Posting
– $700 10Gpbs FIONAs Being Tested
• FIONettes are $250 FIONAs
– 1Gbps NIC With USB-3 for Flash Storage or SSD
Big Data Science Data Transfer Nodes (DTNs)-
Flash I/O Network Appliances (FIONAs)
Phil Papadopoulos, SDSC &
Tom DeFanti, Joe Keefe & John Graham, Calit2
Key Innovation: UCSD Designed Flash I/O Network Appliances (FIONAs)
To Provide Disk-to-Disk Data Transfer at Full Speed on 10/40/100G Networks
FIONAS—10/40G, $8,000
FIONette—1G, $250](https://image.slidesharecdn.com/ucimleventmay2018-180803175027/85/Distributed-Cyberinfrastructure-to-Support-Big-Data-Machine-Learning-4-320.jpg)
Uploaded byLarry Smarr
Distributed Cyberinfrastructure to Support Big Data Machine Learning
The document discusses advancements in distributed cyberinfrastructure for big data machine learning, detailing the development and functionality of flash I/O network appliances (Fionas) at UC San Diego for efficient data transfer and analysis. It highlights NSF grants supporting multi-campus collaboration and the integration of machine learning capabilities into research infrastructure, including the use of GPUs for AI training on massive datasets. The document emphasizes ongoing innovation in machine learning applications and infrastructure within the Pacific Research Platform and collaborating institutions.
More Related Content
Similar to Distributed Cyberinfrastructure to Support Big Data Machine Learning
Distributed Cyberinfrastructure to Support Big Data Machine Learning
- 1. “Distributed Cyberinfrastructure to SupportBig Data Machine Learning” Panel on the Future of Machine Learning California Institute for Telecommunications and Information Technology University of California, Irvine May 24, 2018 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net 1
- 2. Based on CommunityInput and on ESnet’s Science DMZ Concept, NSF Has Made Over 200 Campus-Level Awards in 44 States Source: Kevin Thompson, NSF
- 3. How UCSD DMZNetwork Transforms Big Data Microbiome Science: Preparing for Knight/Smarr 1 Million Core-Hour Analysis Knight Lab FIONA 10Gbps Gordon Prism@UCSD Data Oasis 7.5PB, 200GB/s Knight 1024 Cluster In SDSC Co-Lo CHERuB 100Gbps Emperor & Other Vis Tools 64Mpixel Data Analysis Wall 120Gbps 40Gbps 1.3Tbps
- 4. • FIONAs PCs[a.k.a ESnet DTNs]: – ~$8,000 Big Data PC with: – 1 CPU – 10/40 Gbps Network Interface Cards – 3 TB SSDs or 100+ TB Disk Drive – Extensible for Higher Performance to: – +Up to 38 Intel CPUs – +Up to 8 GPUs [4M GPU Core Hours/Week] – +NVMe SSDs for 100Gbps Disk-to-Disk – +Up to 160 TB Disks for Data Posting – $700 10Gpbs FIONAs Being Tested • FIONettes are $250 FIONAs – 1Gbps NIC With USB-3 for Flash Storage or SSD Big Data Science Data Transfer Nodes (DTNs)- Flash I/O Network Appliances (FIONAs) Phil Papadopoulos, SDSC & Tom DeFanti, Joe Keefe & John Graham, Calit2 Key Innovation: UCSD Designed Flash I/O Network Appliances (FIONAs) To Provide Disk-to-Disk Data Transfer at Full Speed on 10/40/100G Networks FIONAS—10/40G, $8,000 FIONette—1G, $250
- 5. Logical Next Step:The Pacific Research Platform Networks Campus DMZs to Create a Regional End-to-End Science-Driven “Big Data Superhighway” System (GDC) NSF CC*DNI Grant $5M 10/2015-10/2020 PI: Larry Smarr, UC San Diego Calit2 Co-PIs: • Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2/QI, • Philip Papadopoulos, UCSD SDSC, • Frank Wuerthwein, UCSD Physics and SDSC Letters of Commitment from: • 50 Researchers from 15 Campuses • 32 IT/Network Organization Leaders NSF Program Officer: Amy Walton Source: John Hess, CENIC
- 6. PRP National-Scale ExperimentalDistributed Testbed: Using Internet2 to Connect Early-Adopter Quilt Regional R&E Networks Original PRP Extended PRP Testbed Announced at Internet2 Global Summit May 8, 2018
- 7. PRP’s First 2.5Years: Connecting Multi-Campus Application Teams and Devices Earth Sciences
- 8. Data Transfer RatesFrom 40 Gbps DTN in UCSD Physics Building, Across Campus on PRISM DMZ, Then to Chicago’s Fermilab Over CENIC/ESnet Based on This Success, Würthwein Will Upgrade 40G DTN to 100G For Bandwidth Tests & Kubernetes Integration With OSG, Caltech, and UCSC Source: Frank Würthwein, OSG, UCSD/SDSC, PRP
- 9. FIONA8 FIONA8 100G Epyc NVMe 40G160TB 100G NVMe 6.4T SDSU 100G Gold NVMe March 2018 John Graham, UCSD 100G NVMe 6.4T Caltech 40G 160TB UCAR FIONA8 UCI FIONA8 FIONA8 FIONA8 FIONA8 FIONA8 FIONA8 FIONA8 FIONA8 sdx-controller controller-0 Calit2 100G Gold FIONA8 SDSC 40G 160TB UCR 40G 160TB USC 40G 160TB UCLA 40G 160TB Stanford 40G 160TB UCSB 100G NVMe 6.4T 40G 160TB UCSC 40G 160TB Hawaii Running Kubernetes/Rook/Ceph On PRP Allows Us to Deploy a Distributed PB+ of Storage for Posting Science Data Rook/Ceph - Block/Object/FS Swift API compatible with SDSC, AWS, and Rackspace Kubernetes Centos7
- 10. UC San DiegoJaffe Lab (SIO) Scripps Plankton Camera Off the SIO Pier with Fiber Optic Network
- 11. Over 1 BillionImages So Far! Requires Machine Learning for Automated Image Analysis and Classification Phytoplankton: Diatoms Zooplankton: Copepods Zooplankton: Larvaceans Source: Jules Jaffe, SIO ”We are using the FIONAs for image processing... this includes doing Particle Tracking Velocimetry that is very computationally intense.”-Jules Jaffe
- 12. New NSF CHASE-CIGrant Creates a Community Cyberinfrastructure: Adding a Machine Learning Layer Built on Top of the Pacific Research Platform Caltech UCB UCI UCR UCSD UCSC Stanford MSU UCM SDSU NSF Grant for High Speed “Cloud” of 256 GPUs For 30 ML Faculty & Their Students at 10 Campuses for Training AI Algorithms on Big Data NSF Program Officer: Mimi McClure
- 13. FIONA8: Adding GPUsto FIONAs Supports Data Science Machine Learning Multi-Tenant Containerized GPU JupyterHub Running Kubernetes / CoreOS Eight Nvidia GTX-1080 Ti GPUs 32GB RAM, 3TB SSD, 40G & Dual 10G ports Source: John Graham, Calit2
- 14. 48 GPUs for OSGApplications UCSD Adding >350 Game GPUs to Data Sciences Cyberinfrastructure - Devoted to Data Analytics and Machine Learning SunCAVE 70 GPUs WAVE + Vroom 48 GPUs FIONA with 8-Game GPUs 95 GPUs for Students CHASE-CI Grant Provides 96 GPUs at UCSD for Training AI Algorithms on Big Data Plus 288 64-bit GPUs On SDSC’s Comet
- 15. Next Step: Surroundingthe PRP Machine Learning Platform With Clouds of GPUs and Non-Von Neumann Processors Microsoft Installs Altera FPGAs into Bing Servers & 384 into TACC for Academic Access CHASE-CI 64-TrueNorth Cluster 64-bit GPUs 4352x NVIDIA Tesla V100 GPUs
- 16. Pattern Computer WasJust Announced - We Will Provide Access Through CHASE-CI HE UC CCD ICD May 23, 2018 Mark Anderson, CEO Announcing Pattern Computer Reduction of 10,000 Variables to 39 For Microbiome Protein Families Smarr, et. al (2018) www.patterncomputer.com/img/pdf/KEGGs_5.22_final.pdf
- 17. Calit2 Has EstablishedLabs On Both UC San Diego and UC Irvine Campuses For Machine Learning on von Neumann and NvN Processors Charless Fowlkes, Director Ken Kreutz Delgado, Director
- 18. CHASE-CI’s ML ResearchersAre Exploring Mapping Machine Learning Algorithm Families Onto Novel Architectures Qualcomm Institute 1. Deep & Recurrent Neural Networks (DNN, RNN) 2. Reinforcement Learning (RL) 3. Variational Autoencoder (VAE) and Markov Chain Monte Carlo (MCMC) 4. Support Vector Machine (SVM) 5. Sparse Signal Processing (SSP) and Sparse Baysian Learning (SBL) 6. Latent Variable Analysis (PCA, ICA)