Driving Applications on the UCSD Big Data Freeway System
 

Like this? Share it with your network

Share

Driving Applications on the UCSD Big Data Freeway System

on

  • 345 views

Keynote lecture by Calit2 Director Larry Smarr to the Cubic and UC San Diego Innovation Workshop on February 26, 2014 explores driving applications on the UCSD Big Data freeway system.

Keynote lecture by Calit2 Director Larry Smarr to the Cubic and UC San Diego Innovation Workshop on February 26, 2014 explores driving applications on the UCSD Big Data freeway system.

Statistics

Views

Total Views
345
Views on SlideShare
333
Embed Views
12

Actions

Likes
1
Downloads
1
Comments
0

2 Embeds 12

http://lsmarr.calit2.net 6
http://www.slideee.com 6

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Driving Applications on the UCSD Big Data Freeway System Presentation Transcript

  • 1. “Driving Applications on the UCSD Big Data Freeway System” Keynote Lecture Cubic and UC San Diego Innovation Workshop UC San Diego February 26, 2014 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net 1
  • 2. The Data-Intensive Discovery Era Requires High Performance Cyberinfrastructure • Growth of Digital Data is Exponential – “Data Tsunami” • Driven by Advances in Digital Detectors, Computing, Networking, & Storage Technologies • Shared Internet Optimized for Megabyte-Size Objects • Need Dedicated Photonic Cyberinfrastructure for Gigabyte/Terabyte Data Objects • Finding Patterns in the Data is the New Imperative – Data-Driven Applications – Data Mining – Visual Analytics – Data Analysis Workflows Source: SDSC
  • 3. The White House Announcement Has Galvanized U.S. Campus CI Innovations
  • 4. CERN’s CMS Experiment Generates Massive Amounts of Data
  • 5. UCSD is a Tier-2 LHC Data Center: CMS Flow into UCSD Physics Dept. Peaks at 2.4 Gbps Source: Frank Wuerthwein, Physics UCSD
  • 6. Dan Cayan USGS Water Resources Discipline Scripps Institution of Oceanography, UC San Diego much support from Mary Tyree, Mike Dettinger, Guido Franco and other colleagues Sponsors: California Energy Commission NOAA RISA program California DWR, DOE, NSF Planning for climate change in California substantial shifts on top of already high climate variability UCSD Campus Climate Researchers Need to Download Results from Remote Supercomputer Simulations to Make Regional Climate Change Forecasts
  • 7. average summer afternoon temperature average summer afternoon temperature 7GFDL A2 1km downscaled to 1km Hugo Hidalgo Tapash Das Mike Dettinger
  • 8. Protein Data Bank (PDB) Needs Bandwidth to Connect Resources and Users • Archive of experimentally determined 3D structures of proteins, nucleic acids, complex assemblies • One of the largest scientific resources in life sciences Source: Phil Bourne and Andreas Prlić, PDBHemoglobin Virus
  • 9. Protein Data Bank Usage Is Growing Over Time • More than 300,000 Unique Global Visitors per Month • Up to 300 Concurrent Users • ~10 Structures are Downloaded per Second 7/24/365 • Increasingly Popular Web Services Traffic Source: Phil Bourne and Andreas Prlić, PDB
  • 10. Collaboration Between EVL’s CAVE2 and Calit2’s VROOM Over 10Gb Wavelength EVL Calit2 Source: NTT Sponsored ON*VECTOR Workshop at Calit2 March 6, 2013
  • 11. Global Innovation Centers are Being Connected with 10,000 Megabits/sec Clear Channel Lightpaths Source: Maxine Brown, UIC and Robert Patterson, NCSA 100 Gbps Commercially Available; Research on 1 Tbps
  • 12. Creating a Big Data Freeway System: Use Optical Fiber with 1000x Shared Internet Speeds NSF CC-NIE Has Awarded Prism@UCSD Optical Switch Phil Papadopoulos, SDSC, Calit2, PI
  • 13. Arista Enables SDSC’s Massively Parallel 10G Switched Data Analysis Resource 12
  • 14. High Performance Wireless Research and Education Network http://hpwren.ucsd.edu/ National Science Foundation awards 0087344, 0426879 and 0944131
  • 15. approximately 50 miles: Note: locations are approximate MVFDMTGY MPO SMER CNM UCSD to CI and PEMEX 70+ miles to SCI PL MLO MONP CWC P480 USGC SO LVA2 BVDA RMNA Santa Rosa GVDA KNW WMC RDM CRY SND BZN AZRY FRD WIDC KYVW PFO BDC KSW DHL SLMS SCS CRRS GLRS DSME WLA P506 P510 P499 GMPK IID2 P509 P500 P494 P497 155Mbps FDX 6 GHz FCC licensed 155Mbps FDX 11 GHz FCC licensed 45Mbps FDX 6 GHz FCC licensed 45Mbps FDX 11 GHz FCC licensed 45Mbps FDX 5.8 GHz unlicensed 45Mbps-class HDX 4.9GHz 45Mbps-class HDX 5.8GHz unlicensed ~8Mbps HDX 2.4/5.8 GHz unlicensed ~3Mbps HDX 2.4 GHz unlicensed 115kbps HDX 900 MHz unlicensed 56kbps via RCS network via Tribal Digital Village Network dashed = planned B08 1 P486 Backbone/relay node Astronomy science site Biology science site Earth science site University site Researcher location Native American site First Responder site NSS S SDSU P474 P478 DESC P473 POTR P066 P483 CE Red circles: HPWREN supplied cameras Yellow circles: SD County supplied cameras HPWREN Topology, 360 Degree Cameras Source: Hans Werner Braun, HPWREN PI
  • 16. Various Real-Time Network Cameras for Environmental Observations Source: Hans Werner Braun, HPWREN PI
  • 17. San Diego County Digital Weather Stations: High Spatial Density Reads Out Time-Changing Atmosphere Source: Jessica Block, Calit2
  • 18. Trigger real-time computer-generated alerts, if: condition “A” AND condition “B” AND condition “C” OR condition “D” exists, in which case several San Diego emergency officers are being paged or emailed during such alert conditions, based on HPWREN data parameterization by a CDF Division Chief.This system has been in operation since 2004. Date: Wed, 4 Aug 2010 09:31:05 -0700 Subject: URGENT weather sensor alert LP: RH=26.1 WD=135.2 WS=1.9 FM=6.8 AT=80.7 at 20100804.093100 More details at http://hpwren.ucsd.edu/Sensors/ Relative Humidity Wind speed Wind direction Fuel moisture Source: Hans Werner Braun, HPWREN PI
  • 19. By Measuring the State of My Body and “Tuning” It Using Nutrition and Exercise, I Became Healthier 2000 Age 41 2010 Age 61 1999 1989 Age 51 1999 I Arrived in La Jolla in 2000 After 20 Years in the Midwest and Decided to Move Against the Obesity Trend I Reversed My Body’s Decline By Quantifying and Altering Nutrition and Exercise http://lsmarr.calit2.net/repository/LS_reading_recommendations_FiRe_2011.pdf
  • 20. I Used a Variety of Emerging Personal Sensors To Quantify My Body & Drive Behavioral Change Withings/iPhone- Blood Pressure Zeo-Sleep Azumio-Heart Rate MyFitnessPal- Calories Ingested FitBit - Daily Steps & Calories Burned Withings WiFi Scale - Daily Weight
  • 21. From One to a Billion Data Points Defining Me: Big Data Coming to the Electronic Medical Record (EMR) Billion: My Full DNA, MRI/CT Images Million: My DNA SNPs, Zeo, FitBit Hundred: My Blood VariablesOne: My WeightWeight Blood Variables SNPs Microbial Genome Today’s EMR Tomorrow’s EMR
  • 22. Visualizing Time Series of 150 LS Blood and Stool Variables, Each Over 5-10 Years Calit2 64 megapixel VROOM
  • 23. Only One of My Blood Measurements Was Far Out of Range--Indicating Chronic Inflammation Normal Range <1 mg/L Normal 27x Upper Limit Episodic Peaks in Inflammation Followed by Spontaneous Drops Complex Reactive Protein (CRP) is a Blood Biomarker for Detecting Presence of Inflammation
  • 24. Consumer Self Measurement is Exploding Totally Outside of the Medical Complex From the First San Francisco QS Meetup in 2008 To 116 Cities in 37 Countries in Four Years
  • 25. The Self-Monitoring Business Has Reached Market Takeoff • MyFitnessPal – 40 Million Users – Aug 2013 Raised $18M Series A, Led by Kleiner Perkins • Fitbit – Has Raised ~$70M • BodyMedia Was Bought by Jawbone – For ~$100M • Zeo Sleep Monitor – Closed Down in 2013 More Mergers Likely as the Shakeout Continues
  • 26. Mobile Health Market Projected to be $30B-$60B by 2015 Source: Rick Valencia, Qualcomm Life mHealth Technology Progression
  • 27. Platforms Enable Expanding Ecosystems Empowering Many to Serve Diverse Customer Sets Source: Kristian Rauhala, PEAR Sports LLC