This document discusses how high-performance cyberinfrastructure and dedicated 10Gbps networks enable new levels of discovery for data-intensive research. It provides examples of how universities are using these resources for projects in fields like cosmology, ocean observing, and microbial analysis. Specifically, it describes how networks like National LambdaRail provide connectivity between campuses and commercial clouds, and how tools like OptIPortals allow researchers to remotely analyze large datasets in real-time using visualization techniques.
Making Sense of Information Through Planetary Scale ComputingLarry Smarr
09.03.01
Invited Presentation to the
Diamond Exchange—Brave New World
Title: Making Sense of Information Through Planetary Scale Computing
Monterey, CA
How to Terminate the GLIF by Building a Campus Big Data Freeway SystemLarry Smarr
12.10.11
Keynote Lecture
12th Annual Global LambdaGrid Workshop
Title: How to Terminate the GLIF by Building a Campus Big Data Freeway System
Chicago, IL
New Applications of SuperNetworks and the Implications for Campus NetworksLarry Smarr
07.10.09
Speaker
Fall 2007 Internet2 Member Meeting
Town and Country Resort and Convention Center
Title: New Applications of SuperNetworks and the Implications for Campus Networks
San Diego, CA
Calit2: Experiments in Living in the Virtual/Physical WorldLarry Smarr
10.12.15
Invited Talk
"Cultivating Networked Centers of Excellence" CineGrid International Workshop 2010
Title: Calit2: Experiments in Living in the Virtual/Physical World
Making Sense of Information Through Planetary Scale ComputingLarry Smarr
09.03.01
Invited Presentation to the
Diamond Exchange—Brave New World
Title: Making Sense of Information Through Planetary Scale Computing
Monterey, CA
How to Terminate the GLIF by Building a Campus Big Data Freeway SystemLarry Smarr
12.10.11
Keynote Lecture
12th Annual Global LambdaGrid Workshop
Title: How to Terminate the GLIF by Building a Campus Big Data Freeway System
Chicago, IL
New Applications of SuperNetworks and the Implications for Campus NetworksLarry Smarr
07.10.09
Speaker
Fall 2007 Internet2 Member Meeting
Town and Country Resort and Convention Center
Title: New Applications of SuperNetworks and the Implications for Campus Networks
San Diego, CA
Calit2: Experiments in Living in the Virtual/Physical WorldLarry Smarr
10.12.15
Invited Talk
"Cultivating Networked Centers of Excellence" CineGrid International Workshop 2010
Title: Calit2: Experiments in Living in the Virtual/Physical World
Remote Telepresence for Exploring Virtual WorldsLarry Smarr
08.01.26
Foundational Talk
Virtual World and Immersive Environments
NASA Ames
Title: Remote Telepresence for Exploring Virtual Worlds
Mountain View, CA
The independent study on how Cloud Computing can be used to introduce a new Next Generation Sequencing method in terms of better understanding of the limitations of existing Next Generation Sequencing Methods.
Dr. Patrick Bradshaw presents an overview of his program, Sensory Information Systems, at the AFOSR 2013 Spring Review. At this review, Program Officers from AFOSR Technical Divisions will present briefings that highlight basic research programs beneficial to the Air Force.
High Performance Cyberinfrastructure Enables Data-Driven Science in the Globa...Larry Smarr
10.10.28
Invited Speaker
Grand Challenges in Data-Intensive Discovery Conference
San Diego Supercomputer Center, UC San Diego
Title: High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World
La Jolla, CA
Practical steps for non-machine learners on how to prepare your medical image dataset for deep learning modelling.
Here we use a fundus image dataset as an example that might have controls (healthy eyes) and glaucomatous fundus images with three different severities. In glaucoma, the optic disc is of a special interest so we want to annotate that from the images using a bounding box to help the deep learning training.
Image restoration techniques covered such as denoising, deblurring and super-resolution for 3D images and models.
From classical computer vision techniques to contemporary deep learning based processing for both ordered and unordered point clouds, depth maps and meshes.
For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/sept-2017-alliance-vitf-courtney
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Dr. Patrick Courtney, MBA, of tec-connection and the Standards in Laboratory Automation (SiLA) Consortium delivers the presentation "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications" at the Embedded Vision Alliance's September 2017 Vision Industry and Technology Forum. In his presentation, Courtney covers the following topics:
▪ Motivation: the need and the market
▪ Big applications today: NGS case study
▪ Improvement curve: Carlson’s curve and what this means
▪ The next applications for imaging
A Novel Three-Dimensional Adaptive Localization (T-Dial) Algorithm for Wirele...iosrjce
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
Remote Telepresence for Exploring Virtual WorldsLarry Smarr
08.01.26
Foundational Talk
Virtual World and Immersive Environments
NASA Ames
Title: Remote Telepresence for Exploring Virtual Worlds
Mountain View, CA
The independent study on how Cloud Computing can be used to introduce a new Next Generation Sequencing method in terms of better understanding of the limitations of existing Next Generation Sequencing Methods.
Dr. Patrick Bradshaw presents an overview of his program, Sensory Information Systems, at the AFOSR 2013 Spring Review. At this review, Program Officers from AFOSR Technical Divisions will present briefings that highlight basic research programs beneficial to the Air Force.
High Performance Cyberinfrastructure Enables Data-Driven Science in the Globa...Larry Smarr
10.10.28
Invited Speaker
Grand Challenges in Data-Intensive Discovery Conference
San Diego Supercomputer Center, UC San Diego
Title: High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World
La Jolla, CA
Practical steps for non-machine learners on how to prepare your medical image dataset for deep learning modelling.
Here we use a fundus image dataset as an example that might have controls (healthy eyes) and glaucomatous fundus images with three different severities. In glaucoma, the optic disc is of a special interest so we want to annotate that from the images using a bounding box to help the deep learning training.
Image restoration techniques covered such as denoising, deblurring and super-resolution for 3D images and models.
From classical computer vision techniques to contemporary deep learning based processing for both ordered and unordered point clouds, depth maps and meshes.
For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/sept-2017-alliance-vitf-courtney
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Dr. Patrick Courtney, MBA, of tec-connection and the Standards in Laboratory Automation (SiLA) Consortium delivers the presentation "The Reverse Factory: Embedded Vision in High-Volume Laboratory Applications" at the Embedded Vision Alliance's September 2017 Vision Industry and Technology Forum. In his presentation, Courtney covers the following topics:
▪ Motivation: the need and the market
▪ Big applications today: NGS case study
▪ Improvement curve: Carlson’s curve and what this means
▪ The next applications for imaging
A Novel Three-Dimensional Adaptive Localization (T-Dial) Algorithm for Wirele...iosrjce
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
An End-to-End Campus-Scale High Performance Cyberinfrastructure for Data-Inte...Larry Smarr
12.04.19
The Annual Robert Stewart Distinguished Lecture
Iowa State University
Title: An End-to-End Campus-Scale High Performance Cyberinfrastructure for Data-Intensive Research
Ames, IA
Why Researchers are Using Advanced NetworksLarry Smarr
07.07.03
Remote Talk from Calit2 to:
Building KAREN Communities for Collaboration Forum
KIWI Advanced Research and Education Network
University of Auckland, Auckland City, New Zealand
Title: Why Researchers are Using Advanced Networks
La Jolla, CA
High Performance Cyberinfrastructure Required for Data Intensive Scientific R...Larry Smarr
11.06.08
Invited Presentation
National Science Foundation Advisory Committee on Cyberinfrastructure
Title: High Performance Cyberinfrastructure Required for Data Intensive Scientific Research
Arlington, VA
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.08
Fourth Lecture in the
Australian American Leadership Dialogue Scholar Tour
Swinburne University
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Hawthorn, Australia
End-to-end Optical Fiber Cyberinfrastructure for Data-Intensive Research: Imp...Larry Smarr
10.10.13
Featured Speaker EDUCAUSE 2010
Anaheim Convention Center
Title: End-to-end Optical Fiber Cyberinfrastructure for Data-Intensive Research: Implications for Your Campus
Anaheim, CA
How Global-Scale Personal Lightwaves are Transforming Scientific ResearchLarry Smarr
07.03.22
Distinguished Lecturer
Technology for a Changing World Series
Baskin School of Engineering, UCSC
Title: How Global-Scale Personal Lighwaves are Transforming Scientific Research
Santa Cruz, CA
How Global-Scale Personal Lighwaves are Transforming Scientific ResearchLarry Smarr
07.03.08
Speaker
Distinguished Lecturer Series
Department of Computer Science
Title: How Global-Scale Personal Lighwaves are Transforming Scientific Research
UC Davis
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...Larry Smarr
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
Riding the Light: How Dedicated Optical Circuits are Enabling New ScienceLarry Smarr
06.08.15
Invited Talk
Future of Imaging Plenary Session
SPIE Optics and Photonics Convention
Title: Riding the Light: How Dedicated Optical Circuits are Enabling New Science
San Diego, CA
Positioning University of California Information Technology for the Future: S...Larry Smarr
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
Shrinking the Planet—How Dedicated Optical Networks are Transforming Computat...Larry Smarr
08.12.16
Invited Presentation to the
Advisory Committee on Cyberinfrastructure
National Science Foundation
Title: Shrinking the Planet—How Dedicated Optical Networks are Transforming Computational Science and Collaboration
Arlington, VA
High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biom...Larry Smarr
11.04.06
Joint Presentation
UCSD School of Medicine Research Council
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biomedical Sciences
06.07.26
Invited Talk
Cyberinfrastructure for Humanities, Arts, and Social Sciences, A Summer Institute, SDSC
Title: The OptIPuter and Its Applications
La Jolla, CA
Similar to High Performance Cyberinfrastructure Discovery Tools for Data Intensive Research (20)
2. Abstract
High performance cyberinfrastructure (10Gbps dedicated
optical channels end-to-end) enable new levels of discovery
for data-intensive research projects—such as cosmological
simulations, ocean observing, and microbial metagenomics.
In addition to the national optical fiber infrastructure provided
by National LamdbaRail, we need local campus high
performance research cyberinfrastructure to provide ―on-
ramps,‖ as well as compute and storage clouds, to augment
the emerging remote commercial clouds.
3. Dedicated 10,000Mbps (10Gbps) Supernetworks
Enable Remote Visual Analysis of Big Data
National
LambdaRail
Interconnects
Two Dozen
State and Regional
Optical Networks
Also Dynamic
Circuit Network
NLR 80 x 10Gb Wavelengths Is Now Available
4. NSF’s OptIPuter Project: Using Supernetworks
to Meet the Needs of Data-Intensive Researchers
OptIPortal–
Termination
Device for the
OptIPuter
10Gbps
Backplane
5. Intergalactic Medium on 2 Billion Light Year Scale
Exploring Cosmology With Supercomputers,
Supernetworks, and Supervisualization
• Supercomputer Output Science:
Norman,
– 148 TB Movie Output Harkness,
(0.25 TB/file) Paschos,
– 80 TB Diagnostic Dumps SDSC
(8 TB/file) Visualization:
• Connected at 10Gbps Insley, ANL;
Wagner SDSC
– Oak Ridge to ANL to SDSC
ANL * Calit2 * LBNL * NICS * ORNL * SDSC
6. End-to-End 10Gbps Cyberinfrastructure for
Petascale High Performance Computing End Users
log of gas temperature log of gas density
Mike Norman, SDSC
Using OptIPortal
to Analyze
Petascale
Cosmological
Big Data
7. Calit2 Microbial Metagenomics Cluster-Users Can
Connect by Shared Internet or 10Gbps Optical Paths
Source: Phil
Papadopoulos,
512 Processors SDSC, Calit2
~5 Teraflops
~ 200 Terabytes Storage
Nearly 4000 Users
Over 75 Countries
8. Using 10 Gbps Big Data Access and Analysis-
Collaboration Between Calit2 and U Washington
Photo Credit: Alan Decker
Feb. 29, 2008
Ginger Armbrust’s
Diatom Chromosomes
iHDTV: 1500 Mbits/sec
Calit2 to UW Over NLR
UW Research Channel
9. MIT Using OptIPortal to Analyze 10km
Coupled Ocean Microbial Simulation
MIT’s Ed DeLong &
Darwin Project Team
10. The NSF-Funded Ocean Observatory Initiative– a
Complex System of Systems Cyberinfrastructure
Source: Matthew
Arrott, Calit2
Program Manager
for OOI CI
11. Taking Sensornets to the Ocean Floor:
Remote Interactive HD Imaging of Deep Sea Vent
1 cm. Source:
John Delaney
and
Research
Channel,
University of
Washington
12. NSF OOI is a $400M Program
-OOI CI is $34M Part of OOI
Science Program
Source:
25 to 30 Years
Matthew Arrott,
Calit2 Program
Construction Manager for OOI
Cyberinfrastructure
Program 5 Years
30 Software Engineers
Housed at Calit2@UCSD
13. OOI CI is Built on National LambdaRail’s
and Internet2’s DCN Optical Infrastructure
Source: John
Orcutt, Matthew
Arrott, SIO/Calit2
14. High Definition Video Connected OptIPortals:
Virtual Working Spaces for Data Intensive Research
Source: Falko
Kuester, Kai Doerr
Calit2; Michael
Sims, NASA
15. Analyzing Big Data in 3D Stereo:
The NexCAVE OptIPortal
Source: Tom
DeFanti,
Calit2@UCSD
Array of JVC
HDTV 3D LCD
Screens
Calit2’s KAUST
NexCAVE =
22.5MPixels
16. CENIC,
“Blueprint 10Gbe
N x for the Digital I2DCN
NLR, University”--Report
of the
UCSD Research Cyberinfrastructure Design Team
research.ucsd.edu/documents/rcidt/RCIDTReportFinal2009.pdf
Gordon –
HPC
Cluster System April 24, 2009
Condo
DataOasis
Triton – (Central) Storage
Petadata
Analysis
Scientific
Instruments
Digital Data Campus Lab
Collections Cluster OptIPortal
Source: Philip Papadopoulos, SDSC, Calit2 UCSD
17. California and Washington Universities Are Testing
a 10Gbps Connected Commercial Data Cloud
• Amazon Experiment for Big Data
– Only Available Through CENIC and
Pacific NW GigaPOP
• Private 10Gbps Peering Paths
– Includes Amazon EC2 Computing and
S3 Storage Services
This is a production cluster with it’s own Force10 e1200 switch. It is connected to quartzite and is labeled as the “CAMERA Force10 E1200”. We built CAMERA this way because of technology deployed successfully in Quartzite