Using Supercomputers and Supernetworks to Explore the Ocean of Life Moore Foundation PI Meeting [email_address] July 17, 2...
Abstract Calit2, in partnership with J. Craig Venter Institute in Rockville, MD, and UCSD's SDSC and Scripps Institution o...
Challenge: Average Throughput of NASA Data Products  to End User is 10-100 Mbps  Tested July 2007 http://ensight.eos.nasa....
Calit2’s Direct Access Core Architecture  Creates a SuperNetwork Metagenomics Server Traditional User Response Request Sou...
The OptIPuter Project: Creating High Resolution Portals  Over Dedicated Optical Channels to Global Science Data Picture So...
CAMERA Builds on Cyberinfrastructure Grid, Workflow, and Portal Projects in a Service Oriented Architecture Cyberinfrastru...
e-Science Collaboratory Without Walls  Enabled by Uncompressed HD Telepresence Photo: Harry Ammons, SDSC John Delaney, PI ...
EVL’s Scalable Adaptive Graphics Environment Creates a High Performance Windowed OptIPortal MagicCarpet Streaming Blue Mar...
OptIPortal–  Termination Device for the OptIPuter Global Backplane Source: Falko Kuester, Calit2@UCI NSF Infrastructure Gr...
An Emerging High Performance Collaboratory for Microbial Metagenomics  NW! CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI Op...
Interactive Exploration of Marine Genomes  Using 100 Million Pixels Ginger Armburst (UW), Terry Gaasterland (UCSD SIO)
Use of Tiled Display Wall OptIPortal  to Interactively View Microbial Genome Acidobacteria bacterium Ellin345 Soil Bacteri...
Use of Tiled Display Wall OptIPortal  to Interactively View Microbial Genome Source:  Raj Singh, UCSD
Use of Tiled Display Wall OptIPortal  to Interactively View Microbial Genome Source:  Raj Singh, UCSD
CAMERA is Partnering to Port  Metagenomic Community Software to the OptIPortal Collaboration Between Microbial Genomics Gr...
3D OptIPortal Calit2 StarCAVE  Telepresence “Holodeck” 60 GB Texture Memory,  Renders Images 3,200 Times the Speed of  Sin...
Metagenomic Challenge--Enormous Biodiversity: Very Little of GOS Metagenomic Data Assembles Well <ul><li>Use Reference Gen...
Use of Self Organizing Maps to Identify Species Massive Computation on the Japanese Earth Simulator Human Fugu Arabidopsis...
Using SOM, Sargasso Sea  Metagenomic Data Yields 92 Microbial Genera ! Eukaryotes Prokaryotes Viruses Mitochondria Chlorop...
Moore Foundation Funded the Venter Institute to Provide the Full Genome Sequence of 155+ Marine Microbes Phylogenetic Tree...
DOE Genomic Encyclopedia of Bacteria and Archaea  (GEBA) / Bergey Solution: Deep Sampling Across Phyla Source: Eddie Rubin...
Calit2, SDSC, EVL, and SIO are Creating  Environmental Observatory Control Rooms
Towards a Total Knowledge Integration System  for the Coastal Zone—SensorNets Linked to OptIPuter <ul><li>Moorings </li></...
Ocean Observatory Initiative -- Initial Stages <ul><li>OOI Implementing Organizations </li></ul><ul><ul><li>Regional Scale...
Upcoming SlideShare
Loading in …5
×

Using Supercomputers and Supernetworks to Explore the Ocean of Life

866 views

Published on

07.07.17
Moore Foundation PI Meeting
Calit2@UCSD
Title: Using Supercomputers and Supernetworks to Explore the Ocean of Life
La Jolla, CA

Published in: Economy & Finance, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
866
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Using Supercomputers and Supernetworks to Explore the Ocean of Life

    1. 1. Using Supercomputers and Supernetworks to Explore the Ocean of Life Moore Foundation PI Meeting [email_address] July 17, 2007 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
    2. 2. Abstract Calit2, in partnership with J. Craig Venter Institute in Rockville, MD, and UCSD's SDSC and Scripps Institution of Oceanography, is creating a Community Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA), funded by the Gordon and Betty Moore Foundation. CAMERA collaborates closely with DoE's Joint Genome Institute. The CAMERA computational and storage cluster containing the metagenomic data can be accessed via the web over novel dedicated 10 Gb/s light pipes (termed &quot;lambdas&quot;) through the National LambdaRail, providing direct connection to the scalable Linux clusters in individual user laboratories. These clusters are reconfigured as &quot;OptIPortals,&quot; providing the end user with local scalable visualization, computing, and storage. Currently over 1000 web users are registered from over 40 countries and a dozen OptIPortal sites are under construction.
    3. 3. Challenge: Average Throughput of NASA Data Products to End User is 10-100 Mbps Tested July 2007 http://ensight.eos.nasa.gov/Missions/icesat/index.shtml
    4. 4. Calit2’s Direct Access Core Architecture Creates a SuperNetwork Metagenomics Server Traditional User Response Request Source: Phil Papadopoulos, SDSC, Calit2 + Web Services <ul><ul><li>Sargasso Sea Data </li></ul></ul><ul><ul><li>Sorcerer II Expedition (GOS) </li></ul></ul><ul><ul><li>JGI Community Sequencing Project </li></ul></ul><ul><ul><li>Moore Marine Microbial Project </li></ul></ul><ul><ul><li>NASA and NOAA Satellite Data </li></ul></ul><ul><ul><li>Community Microbial Metagenomics Data </li></ul></ul>Flat File Server Farm W E B PORTAL Dedicated Compute Farm (1000s of CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10,000s of CPUs) Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns Data- Base Farm 10 GigE Fabric
    5. 5. The OptIPuter Project: Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data Picture Source: Mark Ellisman, David Lee, Jason Leigh Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI Univ. Partners: SDSC, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5M Over Five Years Now In the Fifth Year
    6. 6. CAMERA Builds on Cyberinfrastructure Grid, Workflow, and Portal Projects in a Service Oriented Architecture Cyberinfrastructure: Raw Resources, Middleware & Execution Environment NBCR Rocks Clusters Virtual Organizations Web Services KEPLER Workflow Management Vision Telescience Portal Located in Calit2@UCSD Building National Biomedical Computation Resource an NIH supported resource center
    7. 7. e-Science Collaboratory Without Walls Enabled by Uncompressed HD Telepresence Photo: Harry Ammons, SDSC John Delaney, PI LOOKING, Neptune May 23, 2007 1500 Mbits/sec Calit2 to UW Research Channel Over NLR
    8. 8. EVL’s Scalable Adaptive Graphics Environment Creates a High Performance Windowed OptIPortal MagicCarpet Streaming Blue Marble dataset from San Diego to EVL using UDP. 6.7Gbps JuxtaView Locally streaming the aerial photography of downtown Chicago using TCP. 850 Mbps Bitplayer Streaming animation of tornado simulation using UDP. 516 Mbps SVC Locally streaming HD camera live video using UDP. 538Mbps ~ 9 Gbps in Total. SAGE Can Simultaneously Support These Applications Without Decreasing Their Performance Source: Xi Wang, UIC/EVL
    9. 9. OptIPortal– Termination Device for the OptIPuter Global Backplane Source: Falko Kuester, Calit2@UCI NSF Infrastructure Grant Data from the Transdisciplinary Imaging Genetics Center 50 Apple 30” Cinema Displays Driven by 25 Dual-Processor G5s 265 MPixel Wall Under Construction [email_address] Source: Falko Kuester, UCSD/Calit2
    10. 10. An Emerging High Performance Collaboratory for Microbial Metagenomics NW! CICESE UW JCVI MIT SIO UCSD SDSU UIC EVL UCI OptIPortals OptIPortal UC Davis UMich LANL DOE JGI
    11. 11. Interactive Exploration of Marine Genomes Using 100 Million Pixels Ginger Armburst (UW), Terry Gaasterland (UCSD SIO)
    12. 12. Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome Acidobacteria bacterium Ellin345 Soil Bacterium 5.6 Mb Source: Raj Singh, UCSD
    13. 13. Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome Source: Raj Singh, UCSD
    14. 14. Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome Source: Raj Singh, UCSD
    15. 15. CAMERA is Partnering to Port Metagenomic Community Software to the OptIPortal Collaboration Between Microbial Genomics Group, Max Planck Institute for Marine Microbiology, and CAMERA / Rocks Group
    16. 16. 3D OptIPortal Calit2 StarCAVE Telepresence “Holodeck” 60 GB Texture Memory, Renders Images 3,200 Times the Speed of Single PC Source: Tom DeFanti, Greg Dawe, Calit2 Connected at 200 Gb/s 30 HD Projectors!
    17. 17. Metagenomic Challenge--Enormous Biodiversity: Very Little of GOS Metagenomic Data Assembles Well <ul><li>Use Reference Genomes to Recruit Fragments </li></ul><ul><ul><li>Compared 334 Finished and 250 Draft Microbial Genomes </li></ul></ul><ul><li>Only 5 Microbial Genera Yielded Substantial and Uniform Recruitment </li></ul><ul><ul><li>Prochlorococcus, Synechococcus, Pelagibacter, Shewanella, and Burkholderia </li></ul></ul>Source: Douglas Rusch, et al. (PLOS Biology March 2007)
    18. 18. Use of Self Organizing Maps to Identify Species Massive Computation on the Japanese Earth Simulator Human Fugu Arabidopsis Rice C. Elegans Drosophilia www.es.jamstec.go.jp/publication/journal/jes_vol.6/pdf/JES6_22-Abe.pdf T. Abe, H. Sugawara, S. Kanaya, T. Ikemura Journal of the Earth Simulator, Volume 6, October 2006, 17–23 SOM Created from an Unsupervised Neural Network Algorithm to Analyze Tetranucleotide Frequencies in a Wide Range of Genomes 10kb Moving Window
    19. 19. Using SOM, Sargasso Sea Metagenomic Data Yields 92 Microbial Genera ! Eukaryotes Prokaryotes Viruses Mitochondria Chloroplasts Input Genomes: 1500 Microbes 40 Eukaryotes 1065 Viruses 642 Mitochondria 42 Chloroplasts 5kb Window T. Abe, H. Sugawara, S. Kanaya, T. Ikemura Journal of the Earth Simulator, Volume 6, October 2006, 17–23
    20. 20. Moore Foundation Funded the Venter Institute to Provide the Full Genome Sequence of 155+ Marine Microbes Phylogenetic Trees Created by Uli Stingl, Oregon State Blue Means Contains One of the Moore 155 Genomes www.moore.org/microgenome/trees.aspx
    21. 21. DOE Genomic Encyclopedia of Bacteria and Archaea (GEBA) / Bergey Solution: Deep Sampling Across Phyla Source: Eddie Rubin, DOE JGI 2007 Goal: Finish ~100 Bacterial and Archaeal Genomes from Culture Collections Project Lead -- Jonathan Eisen (JGI/UC Davis) Acidobacteria Bacteroides Fibrobacteres Gemmimonas Verrucomicrobia Planctomycetes Chloroflexi Proteobacteria Chlorobi Firmicutes Fusobacteria Actinobacteria Cyanobacteria Chlamydia Spriochaetes Deinococcus-Thermus Aquificae Thermotogae TM6 OS-K Termite Group OP8 Marine GroupA WS3 OP9 NKB19 OP3 OP10 TM7 OP1 OP11 Nitrospira Synergistes Deferribacteres Thermudesulfobacteria Chrysiogenetes Thermomicrobia Dictyoglomus Coprothmermobacter Well sampled phyla No cultured taxa
    22. 22. Calit2, SDSC, EVL, and SIO are Creating Environmental Observatory Control Rooms
    23. 23. Towards a Total Knowledge Integration System for the Coastal Zone—SensorNets Linked to OptIPuter <ul><li>Moorings </li></ul><ul><li>Ships </li></ul><ul><li>Autonomous Vehicles </li></ul><ul><li>Satellite Remote Sensing </li></ul><ul><li>Drifters </li></ul><ul><li>Long Range HF Radar </li></ul><ul><li>Near-Shore Waves/Currents </li></ul><ul><li>COAMPS Wind Model </li></ul><ul><li>Nested ROMS Models </li></ul><ul><li>Data Assimilation and Modeling </li></ul><ul><li>Data Systems </li></ul>Pilot Project Components www.sccoos.org/ Yellow—Proposed Initial OptIPuter Backbone
    24. 24. Ocean Observatory Initiative -- Initial Stages <ul><li>OOI Implementing Organizations </li></ul><ul><ul><li>Regional Scale Node </li></ul></ul><ul><ul><ul><li>$150m, UW </li></ul></ul></ul><ul><ul><li>Global/Coastal Scale Nodes </li></ul></ul><ul><ul><ul><li>$120m, to be Awarded </li></ul></ul></ul><ul><ul><li>Cyberinfrastructure </li></ul></ul><ul><ul><ul><li>$30m, SIO/Calit2 UCSD </li></ul></ul></ul><ul><li>6 Year Development Effort </li></ul>Source: John Orcutt, Matthew Arrott, SIO/Calit2

    ×