Distributed Interactive Computing Environment (DICE)

716 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
716
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Distributed Interactive Computing Environment (DICE)

  1. 1. Distributed Interactive Computing Environment http://www.arl.hpc.mil/SciVis/dice
  2. 2. Challenge : Stovepiped Applications Platforms HPC Codes CFD Code 1 CSM Code 1 CCM Code 2 CCM Code 1 Preprocessing Runtime Analysis Tools Postprocessing
  3. 3. Additional Challenges • Code Setup can be Difficult • Enormous Amounts of Data • Heterogeneous Environment • Security
  4. 4. Distributed Interactive Computing Environment Graphical User Interface and Runtime Visualization Common Distributed Data Model and Format High Performance Computing Resource User’s Workstation
  5. 5. Fully Integrated FMD GUI
  6. 6. Fully Integrated ZnsFlow GUI
  7. 7. Fully Integrated CTH GUI
  8. 8. What is DICE ? A Distributed Computing Environment Composed of Configurable Mega-Components • Heterogeneous DSM * • Data Organization • Graphical User Interface • Visualization User-Ready Computing Environments Customized for Specific Technology Areas
  9. 9. Network Distributed Global Memory Clients VectorPut VectorGet Put Get 0 Barrier Semaphore 10M 30M 50M Servers Process Local Address Space System Shared Memory Disk File Physical Data Storage
  10. 10. Light Data Heavy Data • 3D Structured Mesh • X, Y, Z Values • 300 x 200 x 300 • Material Volume Fractions • Number of Materials • Cell Temperatures • Material Names • Total Residual Mass
  11. 11. User Application Scripting and Graphical User Interface Tools Visual Generators Plots Planes Surfaces High Performance Computing Code DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... Network Distributed Global Memory ( NDGM ) with GSSAPI Shared Memory Facility TCP/IP Sockets Operating System Distributed Resources Visualization Toolkit (vtk) OpenGL Mesa File I/O Process Control
  12. 12. DICE Data Model and Format Convenience Layer Structured Data Unstructured Data Groups DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... FileSystem Serial and Parallel Global Access to Secondary Storage ( Globus - GASS ) Distributed Buffer of Bytes Network Distributed Global Memory with GSSAPI Other ...
  13. 13. Hierarchical Data Format Version 5 Group Attribute Group Group Attribute DataSet • Host Independent Data Format • Access via API only • Physical Storage thru Drivers • Includes Data Compression • Includes SubSelections Not Allowed Group DataSet Group DataSet Attribute Attribute Attribute DataSet
  14. 14. eXtensible Markup Language • Similar to HTML Format • Human Readable • Pervasive on the WEB • Strict Hierarchy with No Inheritance < Parent Name=“Top”> • Many Parsing Tools Already Exist < Child Name=“Left”> Character Data for Left < /Child > Parent <Child Name=“Right” > Character Data for Right < /Child > Child Child Character Data for Left Character Data for Right < /Parent > Parse XML Document Object Model In-Memory Data Structures Serialize
  15. 15. N 3 5 3 1 4 1 2 N 3 Polyline 4 4 5 3 4 N 2 Polyvertex 4 1 2 2 1 Triangle Quadrilateral 3 1 2 Polygon 6 5 5 3 4 1 3 1 Tetrahedron 2 Pyramid 8 2 3 4 2 Wedge 1 7 5 6 3 4 1 Hexahedron 2 j i j k 2DSMesh DICE Object Directory Base Topologies i 3DSMesh
  16. 16. <Topology Type=“Tetrahedra” NumElements=“3”> Light Data </Topology> <Geometry> <Topology Type=“3DSmesh” <Vertices> Dimensions=“10 20 30”> <Index Location=“XML” Type=“List”> </Topology> 1000 1001 21001 21000 …… <Geometry> </Index> <Points Type=“XYZ” Location=“HDF” > <Points Type=“XYZ” Location=“HDF” > NDGM:File1.h5:/Grid1/Geometry/MyPoints NDGM:File1.h5:/Grid1/Geometry/MyPoints <Region Type=“Slab” Location=“XML” > 1 2 10 2 3 20 3 4 30 </Region> </Points> </Vertices> </Geometry> </Points> </Geometry> 100 x 200 x 300 HPC Code HDF5 NDGM GASS MPI-I/O ....
  17. 17. Complexity of Model Defined in Light Data Domain Domain Grid XML Grid Attribute Topology 100 Hexahedra HPC Code Geometry XYZPoints.h5 HDF5 File Attribute NDGM:Pressure.h5 NDGM:Temperature.h5 HDF5 NDGM HPC Code I/O is Natural and Efficient GASS
  18. 18. XML Contains Parameters for Variable Substitution Domain Domain Grid XML Grid Parameter TimeStep = 0 to 99 Topology 100 Hexahedra HPC Code Geometry XYZPoints.h5 HDF5 Attribute NDGM:&TimeStep;/Pressure.h5 HDF5 for TimeStep 0 HDF5 for TimeStep 1 HDF5 for TimeStep 2
  19. 19. Parallel / Distributed Entities Post-Processing Tools Pre-Processing Tools HPC Codes Runtime Tools Convenience Layer eXtensible Markup Language ( XML ) Hierarchical Data Format Verision 5 ( HDF5 ) Distributed Data Hub
  20. 20. DICE Data Reader for EnSight Data Reader uses DICE Object Directory API EnSight DICE Object Directory
  21. 21. DICE Visualization using vtk HDF5 File HDF5 NDGM Visualization “Subnet” DICE Object Directory GASS Source Filter Filter Filter Polygons
  22. 22. Adding Performance Friendly Functionality System Programming Language C C++ Fortran77 Fortran90 etc. Compiled Object SWIG Tcl Wrapper Perl Wrapper Python Wrapper Java Wrapper } New Scripting Command via Dynamically Loadable Library
  23. 23. C++ DiceFloat64Array MyData, NewData; DiceInt64Array MyIndex; MyData.SetNumberOfElements( 10000 ); Allocate 10,000 Doubles MyData.Generate(0.0, .9999 ); Set Individual Values MyIndex.SetNumberOfElements( 100 ); MyIndex.Generate( 0, 99 ); MyIndex *= 2; Array Operation +, -, *, / NewData = MyData[ MyIndex ]; Array Indexing Scripting ( Tcl, Perl, etc. ) DiceHDF H5Object Object Oriented Scripting Interface H5Object Open NDGM:FileName.h5:/DataSetName Open HDF5 File, Query Number Type and Shape DiceMemory MemoryObject MemoryObject SetArray [ DiceFloat32Array ] Request Automatic Conversion to Float32 H5Object Read [ MemoryObject cget -this ] MemoryObject has Number Type and Shape set NewData [ MemoryObject GetArray ] DiceExpr $NewData[ where( $NewData > .5 ) ] = .5 Clip Values to .5
  24. 24. DICE Web Interface Under Construction DICE on HPC Resource using Off Screen Rendering SSL Kerberos kinit • Every URL Request is Intercepted to Verify Session Id • DICE Runs as a User Process • HTML FORMS result in Tcl Procedure Invocation • Graphics Updated via Server-Push ( Netscape only ) or Client-Pull • Intended for Limited Interfaces from Remote ( Slow Connection ) or Graphically Challenged Client Machines.
  25. 25. MPI Finite Element Visualization Data Mining Steering Coupling HDF5 NDGM MPI Finite Volume
  26. 26. “Gently Coupling” Codes from Separate Disciplines Distributed Data Hub Finite Volume Structural Mechanics Global Monitor Small Scale Analysis Injection Flow Analysis Runtime Visualization
  27. 27. COTS, GOTS and “Semi-Automatic” Visualization DICE and the HPC Environment X / OpenGL DICE User Interface XML and Http ( SSL + Kerberos ) DICE Server ( User Process ) DICE Object Directory XML, HDF5 I/O One or More Cooperating HPC Codes

×