• Share
  • Email
  • Embed
  • Like
  • Private Content
Supercomputer End Users: the OptIPuter Killer Application
 

Supercomputer End Users: the OptIPuter Killer Application

on

  • 1,123 views

08.08.13

08.08.13
Keynote
DREN Networking and Security Conference
Title: Supercomputer End Users: the OptIPuter Killer Application
San Diego, CA

Statistics

Views

Total Views
1,123
Views on SlideShare
1,105
Embed Views
18

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 18

http://lsmarr.calit2.net 18

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Supercomputer End Users: the OptIPuter Killer Application Supercomputer End Users: the OptIPuter Killer Application Presentation Transcript

    • Supercomputer End Users: the OptIPuter Killer Application Keynote DREN Networking and Security Conference San Diego, CA August 13, 2008 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
    • Abstract During the last few years, a radical restructuring of optical networks supporting e-Science projects has occurred around the world. U.S. universities are beginning to acquire access to high bandwidth lightwaves (termed "lambdas") on fiber optics through the National LambdaRail, Internet2's Circuit Services, and the Global Lambda Integrated Facility. The NSF-funded OptIPuter project explores how user controlled 1- or 10- Gbps lambdas can provide direct access to global data repositories, scientific instruments, and computational resources from the researcher's Linux clusters in their campus laboratories. These end user clusters are reconfigured as "OptIPortals," providing the end user with local scalable visualization, computing, and storage. Integration of high definition video with OptIPortals creates a high performance collaboration workspace of global reach. An emerging major new user community are end users of NSF’s TeraGrid and DODs HPCMP, allowing them to directly optically connect to the remote Tera or Peta-scale resources from their local laboratories and to bring disciplinary experts from multiple sites into the local data and visualization analysis process.
    • Interactive Supercomputing Collaboratory Prototype: Using Analog Communications to Prototype the Fiber Optic Future “What we really have to do is eliminate distance between individuals who want to interact with other people and SIGGRAPH 1989 with other computers.” ― Larry Smarr, Director, NCSA Illinois Boston “We’re using satellite technology… to demo what It might be like to have high-speed fiber-optic links between advanced computers in two different geographic locations.” ― Al Gore, Senator Chair, US Senate Subcommittee on Science, Technology and Space
    • Chesapeake Bay Simulation Collaboratory : vBNS Linked CAVE, ImmersaDesk, Power Wall, and Workstation Alliance Project: Collaborative Video Production via Tele-Immersion and Virtual Director Alliance Application Technologies Environmental Hydrology Team Alliance 1997 4 MPixel PowerWall UIC Donna Cox, Robert Patterson, Stuart Levy, NCSA Virtual Director Team Glenn Wheless, Old Dominion Univ.
    • ASCI Brought Scalable Tiled Walls to Support Visual Analysis of Supercomputing Complexity 1999 LLNL Wall--20 MPixels (3x5 Projectors) An Early sPPM Simulation Run Source: LLNL
    • 60 Million Pixels Projected Wall Driven By Commodity PC Cluster At 15 Frames/s, The System Can Display 2.7 GB/Sec 2002 Source: Philip D. Heermann, DOE ASCI Program
    • Challenge—How to Bring This Visualization Capability to the Supercomputer End User? 2004 35Mpixel EVEREST Display ORNL
    • The OptIPuter Project: Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data Scalable Adaptive Graphics Environment (SAGE) Now in Sixth and Final Year Picture Source: Mark Ellisman, David Lee, Jason Leigh Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
    • Challenge: Average Throughput of NASA Data Products to End User is ~ 50 Mbps Tested May 2008 Internet2 Backbone is 10,000 Mbps! Throughput is < 0.5% to End User http://ensight.eos.nasa.gov/Missions/aqua/index.shtml
    • Dedicated 10Gbps Lambdas Provide Cyberinfrastructure Backbone for U.S. Researchers 10 Gbps per User ~ 200x Shared Internet Throughput Interconnects Two Dozen State and Regional Internet2 Dynamic Optical Networks Circuit Network Under Development NLR 40 x 10Gb Wavelengths Expanding with Darkstrand to 80
    • 9Gbps Out of 10 Gbps Disk-to-Disk Performance Using LambdaStream between EVL and Calit2 9.3 Throughput in Gbps 9.35 9.3 9.25 9.22 9.2 9.15 CaveWave 9.1 9.01 9.02 9.05 TeraWave 9 8.95 8.9 8.85 San Diego to Chicago Chicago to San Diego CAVEWave: TeraGrid: 20 senders to 20 receivers (point to point ) 20 senders to 20 receivers (point to point ) Effective Throughput = 9.01 Gbps Effective Throughput = 9.02 Gbps (San Diego to Chicago) (San Diego to Chicago) 450.5 Mbps disk to disk transfer per stream 451 Mbps disk to disk transfer per stream Effective Throughput = 9.30 Gbps Effective Throughput = 9.22 Gbps (Chicago to San Diego) (Chicago to San Diego) 465 Mbps disk to disk transfer per stream 461 Mbps disk to disk transfer per stream Dataset: 220GB Satellite Imagery of Chicago courtesy USGS. Each file is 5000 x 5000 RGB image with a size of 75MB i.e ~ 3000 files Source: Venkatram Vishwanath, UIC EVL
    • NLR/I2 is Connected Internationally via Global Lambda Integrated Facility Source: Maxine Brown, UIC and Robert Patterson, NCSA
    • OptIPuter / OptIPortal Scalable Adaptive Graphics Environment (SAGE) Applications MagicCarpet Bitplayer Streaming Blue Marble Streaming animation dataset from San Diego of tornado simulation to EVL using UDP. using UDP. 6.7Gbps 516 Mbps ~ 9 Gbps in Total. SAGE Can Simultaneously Support These Applications Without Decreasing Their Performance SVC JuxtaView Locally streaming Locally streaming the aerial HD camera live photography of downtown video using UDP. Chicago using TCP. 538Mbps 850 Mbps Source: Xi Wang, UIC/EVL
    • OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid Distributed Applications/ Web Services Visualization Telescience SAGE JuxtaView Data Services LambdaRAM Vol-a-Tile Distributed Virtual Computer (DVC) API DVC Configuration DVC Runtime Library DVC Services DVC Job DVC Scheduling Communication DVC Core Services Resource Namespace Security High Speed Storage Identify/Acquire Management Management Communication Services Globus PIN/PDC GRAM GSI XIO RobuStore Discovery and Control GTP XCP UDT IP Lambdas CEP LambdaStream RBUDP
    • Two New Calit2 Buildings Provide New Laboratories for “Living in the Future” • “Convergence” Laboratory Facilities – Nanotech, BioMEMS, Chips, Radio, Photonics – Virtual Reality, Digital Cinema, HDTV, Gaming • Over 1000 Researchers in Two Buildings – Linked via Dedicated Optical Networks UC Irvine www.calit2.net Preparing for a World in Which Distance is Eliminated…
    • The Calit2 1/4 Gigapixel OptIPortals at UCSD and UCI Are Joined to Form a Gbit/s HD Collaboratory UCSD Wall to Campus Switch at 10 Gbps Calit2@ UCI wall Calit2@ UCSD wall NASA Ames Visit Feb. 29, 2008 UCSD cluster: 15 x Quad core Dell XPS with Dual nVIDIA 5600s UCI cluster: 25 x Dual Core Apple G5
    • Cisco Telepresence Provides Leading Edge Commercial Video Teleconferencing • 191 Cisco TelePresence  85,854 TelePresence  13,450 Meetings Avoided in Major Cities Globally Meetings Scheduled to Date Travel – US/Canada: 83 CTS Average to Date 3000, 46 CTS 1000  Weekly Average is 2,263 (Based on 8 Participants) – APAC: 17 CTS 3000, Meetings 4 CTS 1000 ~$107.60 M To Date  108,736 Hours – Japan: 4 CTS 3000, 2  Cubic Meters of Emissions CTS 1000  Average is 1.25 Hours Saved 16,039,052 (6,775 – Europe: 22 CTS Cars off the Road) 3000, 10 CTS 1000 – Emerging: 3 CTS 3000 Uses QoS Over Shared Internet ~ 15 mbps • Overall Average Utilization is 45% Cisco Bought WebEx Source: Cisco 3/22/08
    • e-Science Collaboratory Without Walls Enabled by Uncompressed HD Telepresence Over 10Gbps iHDTV: 1500 Mbits/sec Calit2 to UW Research Channel Over NLR May 23, 2007 John Delaney, PI LOOKING, Neptune Photo: Harry Ammons, SDSC
    • OptIPlanet Collaboratory Persistent Infrastructure Supporting Microbial Research Photo Credit: Alan Decker Feb. 29, 2008 Ginger Armbrust’s Diatoms: Micrographs, Chromosomes, Genetic Assembly iHDTV: 1500 Mbits/sec Calit2 to UW Research Channel Over NLR UW’s Research Channel Michael Wellings
    • OptIPortals Are Being Adopted Globally AIST-Japan Osaka U-Japan KISTI-Korea CNIC-China UZurich NCHC-Taiwan SARA- Netherlands Brno-Czech Republic U. Melbourne, EVL@UIC Calit2@UCSD Calit2@UCI Australia
    • Green Initiative: Can Optical Fiber Replace Airline Travel for Continuing Collaborations ? Source: Maxine Brown, OptIPuter Project Manager
    • AARNet International Network
    • Launch of the 100 Megapixel OzIPortal Over Qvidium Compressed HD on 1 Gbps CENIC/PW/AARNet Fiber No Calit2 Person Physically Flew to Australia to Bring This Up! January 15, 2008 Covise, Phil Weber, Jurgen Schulze, Calit2 CGLX, Kai-Uwe Doerr , Calit2 www.calit2.net/newsroom/release.php?id=1219
    • Victoria Premier and Australian Deputy Prime Minister Asking Questions www.calit2.net/newsroom/release.php?id=1219
    • University of Melbourne Vice Chancellor Glyn Davis in Calit2 Replies to Question from Australia
    • OptIPuterizing Australian Universities in 2008: CENIC Coupling to AARNet UMelbourne/Calit2 Telepresence Session May 21, 2008 Two Week Lecture Tour of Australian Research Universities by Larry Smarr October 2008 Phil Scanlan Founder- Australian American Leadership Dialogue www.aald.org AARNet's roadmap: by 2011 up to 80 x 40 Gbit channels
    • First Trans-Pacific Super High Definition Telepresence Meeting Using Digital Cinema 4k Streams 4k = 4000x2000 Pixels = 4xHD Streaming 4k 100 Times with JPEG 2000 the Resolution Compression ½ gigabit/sec of YouTube! Lays Technical Basis for Global Keio University Digital President Anzai Cinema Sony UCSD NTT Chancellor Fox SGI Calit2@UCSD Auditorium
    • From Digital Cinema to Scientific Visualization: JPL Supercomputer Simulation of Monterey Bay 4k Resolution = 4 x High Definition Source: Donna Cox, Robert Patterson, NCSA Funded by NSF LOOKING Grant
    • Rendering Supercomputer Data at Digital Cinema Resolution Source: Donna Cox, Robert Patterson, Bob Wilhelmson, NCSA
    • EVL’s SAGE Global Visualcasting to Europe September 2007 Gigabit Streams Image Viewing Image Viewing Image Image Image Image Source Replication Viewing Viewing OptIPortals at OptIPortal at EVL Russian OptIPuter OptIPuter OptIPortal OptIPortal at Chicago Academy of servers at SAGE- at SARA Masaryk Sciences CALIT2 Bridge at Amsterdam University Moscow San Diego StarLight Brno Oct 1 Chicago Source: Luc Renambot, EVL
    • Creating a California Cyberinfrastructure of OptIPuter “On-Ramps” to NLR & TeraGrid Resources UC Davis UC Berkeley UC San Francisco UC Merced UC Santa Cruz Creating a Critical Mass of UC Los Angeles OptIPuter End Users on UC Santa Barbara UC Riverside a Secure LambdaGrid UC Irvine UC San Diego CENIC Workshop at Calit2 Sept 15-16, 2008 Source: Fran Berman, SDSC , Larry Smarr, Calit2
    • CENIC’s New “Hybrid Network” - Traditional Routed IP and the New Switched Ethernet and Optical Services ~ $14M Invested in Upgrade Now Campuses Need to Upgrade Source: Jim Dolgonas, CENIC
    • The “Golden Spike” UCSD Experimental Optical Core: Ready to Couple Users to CENIC L1, L2, L3 Services Quartzite Communications To 10GigE cluster node interfaces Goals by Core Year 3 2008: CENIC L1, L2 >= 60 endpoints at 10 GigE Quartzite Wavelength Services Selective >= 30 Packet switched Core ..... Switch Lucent >= 30 Switched wavelengths To 10GigE cluster node interfaces and other switches >= 400 Connected endpoints To cluster nodes ..... Glimmerglass Approximately 0.5 Tbps To cluster nodes ..... GigE Switch with Arrive at the “Optical” Center Production OOO Dual 10GigE Upliks of Hybrid Campus Switch 32 10GigE Switch To cluster nodes ..... GigE Switch with Dual 10GigE Upliks Force10 ... To Packet Switch CalREN-HPR GigE Switch with Dual 10GigE Upliks other Research nodes Cloud GigE Funded by 10GigE NSF MRI Campus Research 4 GigE 4 pair fiber Grant Cloud Cisco 6509 Juniper T320 OptIPuter Border Router Source: Phil Papadopoulos, SDSC/Calit2 (Quartzite PI, OptIPuter co-PI)
    • Calit2 Sunlight Optical Exchange Contains Quartzite 10:45 am Feb. 21, 2008
    • Block Layout of UCSD Quartzite/OptIPuter Network Glimmerglass OOO Switch ~50 10 Gbps Lightpaths 10 More to Come Quartzite Application Specific Embedded Switches
    • Calit2 Microbial Metagenomics Cluster- Next Generation Optically Linked Science Data Server Source: Phil Papadopoulos, SDSC, Calit2 512 Processors ~200TB ~5 Teraflops Sun 1GbE X4500 ~ 200 Terabytes Storage and Storage 10GbE Switched 10GbE / Routed Core
    • Calit2 3D Immersive StarCAVE OptIPortal: Enables Exploration of High Resolution Simulations Connected at 50 Gb/s to Quartzite 15 Meyer Sound Speakers + Subwoofer 30 HD Projectors! Passive Polarization-- Optimized the Polarization Separation and Minimized Attenuation Source: Tom DeFanti, Greg Dawe, Calit2 Cluster with 30 Nvidia 5600 cards-60 GB Texture Memory
    • Next Step: Experiment on OptIPuter/OptIPortal with Remote Supercomputer Power User 1 Billion Light-year Pencil From a 20483 Hydro/N-Body Simulation M. Norman, R. Harkness, P. Paschos Working on Putting in Calit2 StarCAVE Structure of the Intergalactic Medium 1.3 M SUs, NERSC Seaborg 170 TB output Source: Michael Norman, SDSC, UCSD
    • The Livermore Lightcone: 8 Large AMR Simulations Covering 10 Billion Years “Look Back Time” • 1.5 M SU on LLNL Thunder • Generated 200 TB Data • 0.4 M SU Allocated on SDSC DataStar for Data Analysis Alone 5123 Base Grid, 7 Levels of Adaptive Refinement65,000 Spatial Dynamic Range Livermore Lightcone Tile 8 Source: Michael Norman, SDSC, UCSD
    • An 8192 x 8192 Image Extracted from Tile 8: How to Display/Explore? Digital Cinema Working on Image Putting it on Calit2 HIPerWall OptIPortal
    • 2x
    • 4x
    • 8x
    • 16x
    • 300 Million Pixels of Viewing Real Estate For Visually Analyzing Supercomputer Datasets HDTV Digital Cameras Digital Cinema Goal: Link Norman’s Lab OptIPortal Over Quartzite, CENIC, NLR/TeraGrid to Petascale Track 2 at Ranger@TACC and Kraken@NICS by October 2008