0
Distributed
Interactive
Computing
Environment

http://www.arl.hpc.mil/SciVis/dice
Challenge : Stovepiped Applications

Platforms

HPC Codes
CFD Code 1

CSM Code 1
CCM Code 2
CCM Code 1

Preprocessing
Runt...
Additional Challenges

• Code Setup can be Difficult
• Enormous Amounts of Data
• Heterogeneous Environment
• Security
Distributed Interactive Computing Environment
Graphical User
Interface and Runtime
Visualization

Common Distributed
Data ...
Fully Integrated FMD GUI
Fully Integrated ZnsFlow GUI
Fully Integrated CTH GUI
What is DICE ?
A Distributed Computing Environment Composed of Configurable
Mega-Components

• Heterogeneous DSM *
• Data ...
Network Distributed Global Memory
Clients

VectorPut
VectorGet

Put
Get

0

Barrier
Semaphore

10M

30M

50M

Servers
Proc...
Light Data

Heavy Data

• 3D Structured Mesh

• X, Y, Z Values

• 300 x 200 x 300

• Material Volume Fractions

• Number o...
User Application
Scripting and Graphical User Interface Tools

Visual Generators
Plots
Planes
Surfaces

High Performance
C...
DICE Data Model and Format
Convenience Layer

Structured Data
Unstructured Data
Groups

DICE Object Directory
Object Orien...
Hierarchical Data Format
Version 5
Group
Attribute

Group

Group

Attribute DataSet

• Host Independent Data
Format
• Acce...
eXtensible Markup Language
• Similar to HTML Format
• Human Readable
• Pervasive on the WEB
• Strict Hierarchy with No Inh...
N

3

5

3
1

4

1
2

N

3

Polyline

4

4

5

3

4

N

2

Polyvertex

4

1

2

2

1

Triangle

Quadrilateral

3
1

2

Pol...
<Topology Type=“Tetrahedra” NumElements=“3”>

Light Data

</Topology>
<Geometry>

<Topology Type=“3DSmesh”

<Vertices>

Di...
Complexity of Model Defined in Light Data

Domain
Domain

Grid
XML
Grid
Attribute

Topology
100 Hexahedra

HPC
Code

Geome...
XML Contains Parameters for Variable Substitution

Domain
Domain

Grid
XML
Grid
Parameter TimeStep = 0 to 99

Topology
100...
Parallel / Distributed Entities
Post-Processing Tools

Pre-Processing Tools
HPC Codes

Runtime Tools

Convenience Layer
eX...
DICE Data Reader for EnSight

Data Reader uses
DICE Object
Directory API
EnSight
DICE Object Directory
DICE Visualization using vtk
HDF5
File

HDF5
NDGM

Visualization
“Subnet”

DICE Object
Directory

GASS

Source

Filter

Fi...
Adding Performance Friendly Functionality

System Programming Language
C C++ Fortran77 Fortran90 etc.

Compiled Object

SW...
C++
DiceFloat64Array

MyData, NewData;

DiceInt64Array

MyIndex;

MyData.SetNumberOfElements( 10000 );

Allocate 10,000 Do...
DICE Web Interface
Under Construction

DICE on HPC Resource
using Off Screen Rendering
SSL
Kerberos kinit

• Every URL Req...
MPI Finite Element

Visualization
Data Mining
Steering
Coupling

HDF5
NDGM

MPI Finite Volume
“Gently Coupling” Codes from Separate Disciplines

Distributed Data Hub

Finite Volume
Structural Mechanics

Global Monito...
COTS, GOTS and
“Semi-Automatic”
Visualization

DICE and the HPC Environment
X / OpenGL

DICE User
Interface

XML and Http
...
Upcoming SlideShare
Loading in...5
×

Distributed Interactive Computing Environment (DICE)

231

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
231
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Distributed Interactive Computing Environment (DICE)"

  1. 1. Distributed Interactive Computing Environment http://www.arl.hpc.mil/SciVis/dice
  2. 2. Challenge : Stovepiped Applications Platforms HPC Codes CFD Code 1 CSM Code 1 CCM Code 2 CCM Code 1 Preprocessing Runtime Analysis Tools Postprocessing
  3. 3. Additional Challenges • Code Setup can be Difficult • Enormous Amounts of Data • Heterogeneous Environment • Security
  4. 4. Distributed Interactive Computing Environment Graphical User Interface and Runtime Visualization Common Distributed Data Model and Format High Performance Computing Resource User’s Workstation
  5. 5. Fully Integrated FMD GUI
  6. 6. Fully Integrated ZnsFlow GUI
  7. 7. Fully Integrated CTH GUI
  8. 8. What is DICE ? A Distributed Computing Environment Composed of Configurable Mega-Components • Heterogeneous DSM * • Data Organization • Graphical User Interface • Visualization User-Ready Computing Environments Customized for Specific Technology Areas
  9. 9. Network Distributed Global Memory Clients VectorPut VectorGet Put Get 0 Barrier Semaphore 10M 30M 50M Servers Process Local Address Space System Shared Memory Disk File Physical Data Storage
  10. 10. Light Data Heavy Data • 3D Structured Mesh • X, Y, Z Values • 300 x 200 x 300 • Material Volume Fractions • Number of Materials • Cell Temperatures • Material Names • Total Residual Mass
  11. 11. User Application Scripting and Graphical User Interface Tools Visual Generators Plots Planes Surfaces High Performance Computing Code DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... Network Distributed Global Memory ( NDGM ) with GSSAPI Shared Memory Facility TCP/IP Sockets Operating System Distributed Resources Visualization Toolkit (vtk) OpenGL Mesa File I/O Process Control
  12. 12. DICE Data Model and Format Convenience Layer Structured Data Unstructured Data Groups DICE Object Directory Object Oriented Support for Data Model, XML, ... Hierarchical Data Format ( HDF5 ) Data Format : Number Type, Data Space ... FileSystem Serial and Parallel Global Access to Secondary Storage ( Globus - GASS ) Distributed Buffer of Bytes Network Distributed Global Memory with GSSAPI Other ...
  13. 13. Hierarchical Data Format Version 5 Group Attribute Group Group Attribute DataSet • Host Independent Data Format • Access via API only • Physical Storage thru Drivers • Includes Data Compression • Includes SubSelections Not Allowed Group DataSet Group DataSet Attribute Attribute Attribute DataSet
  14. 14. eXtensible Markup Language • Similar to HTML Format • Human Readable • Pervasive on the WEB • Strict Hierarchy with No Inheritance < Parent Name=“Top”> • Many Parsing Tools Already Exist < Child Name=“Left”> Character Data for Left < /Child > Parent <Child Name=“Right” > Character Data for Right < /Child > Child Child Character Data for Left Character Data for Right < /Parent > Parse XML Document Object Model In-Memory Data Structures Serialize
  15. 15. N 3 5 3 1 4 1 2 N 3 Polyline 4 4 5 3 4 N 2 Polyvertex 4 1 2 2 1 Triangle Quadrilateral 3 1 2 Polygon 6 5 5 3 4 1 3 1 Tetrahedron 2 Pyramid 8 2 3 4 2 Wedge 1 7 5 6 3 4 1 Hexahedron 2 j i j k 2DSMesh DICE Object Directory Base Topologies i 3DSMesh
  16. 16. <Topology Type=“Tetrahedra” NumElements=“3”> Light Data </Topology> <Geometry> <Topology Type=“3DSmesh” <Vertices> Dimensions=“10 20 30”> <Index Location=“XML” Type=“List”> </Topology> 1000 1001 21001 21000 …… <Geometry> </Index> <Points Type=“XYZ” Location=“HDF” > <Points Type=“XYZ” Location=“HDF” > NDGM:File1.h5:/Grid1/Geometry/MyPoints NDGM:File1.h5:/Grid1/Geometry/MyPoints <Region Type=“Slab” Location=“XML” > 1 2 10 2 3 20 3 4 30 </Region> </Points> </Vertices> </Geometry> </Points> </Geometry> 100 x 200 x 300 HPC Code HDF5 NDGM GASS MPI-I/O ....
  17. 17. Complexity of Model Defined in Light Data Domain Domain Grid XML Grid Attribute Topology 100 Hexahedra HPC Code Geometry XYZPoints.h5 HDF5 File Attribute NDGM:Pressure.h5 NDGM:Temperature.h5 HDF5 NDGM HPC Code I/O is Natural and Efficient GASS
  18. 18. XML Contains Parameters for Variable Substitution Domain Domain Grid XML Grid Parameter TimeStep = 0 to 99 Topology 100 Hexahedra HPC Code Geometry XYZPoints.h5 HDF5 Attribute NDGM:&TimeStep;/Pressure.h5 HDF5 for TimeStep 0 HDF5 for TimeStep 1 HDF5 for TimeStep 2
  19. 19. Parallel / Distributed Entities Post-Processing Tools Pre-Processing Tools HPC Codes Runtime Tools Convenience Layer eXtensible Markup Language ( XML ) Hierarchical Data Format Verision 5 ( HDF5 ) Distributed Data Hub
  20. 20. DICE Data Reader for EnSight Data Reader uses DICE Object Directory API EnSight DICE Object Directory
  21. 21. DICE Visualization using vtk HDF5 File HDF5 NDGM Visualization “Subnet” DICE Object Directory GASS Source Filter Filter Filter Polygons
  22. 22. Adding Performance Friendly Functionality System Programming Language C C++ Fortran77 Fortran90 etc. Compiled Object SWIG Tcl Wrapper Perl Wrapper Python Wrapper Java Wrapper } New Scripting Command via Dynamically Loadable Library
  23. 23. C++ DiceFloat64Array MyData, NewData; DiceInt64Array MyIndex; MyData.SetNumberOfElements( 10000 ); Allocate 10,000 Doubles MyData.Generate(0.0, .9999 ); Set Individual Values MyIndex.SetNumberOfElements( 100 ); MyIndex.Generate( 0, 99 ); MyIndex *= 2; Array Operation +, -, *, / NewData = MyData[ MyIndex ]; Array Indexing Scripting ( Tcl, Perl, etc. ) DiceHDF H5Object Object Oriented Scripting Interface H5Object Open NDGM:FileName.h5:/DataSetName Open HDF5 File, Query Number Type and Shape DiceMemory MemoryObject MemoryObject SetArray [ DiceFloat32Array ] Request Automatic Conversion to Float32 H5Object Read [ MemoryObject cget -this ] MemoryObject has Number Type and Shape set NewData [ MemoryObject GetArray ] DiceExpr $NewData[ where( $NewData > .5 ) ] = .5 Clip Values to .5
  24. 24. DICE Web Interface Under Construction DICE on HPC Resource using Off Screen Rendering SSL Kerberos kinit • Every URL Request is Intercepted to Verify Session Id • DICE Runs as a User Process • HTML FORMS result in Tcl Procedure Invocation • Graphics Updated via Server-Push ( Netscape only ) or Client-Pull • Intended for Limited Interfaces from Remote ( Slow Connection ) or Graphically Challenged Client Machines.
  25. 25. MPI Finite Element Visualization Data Mining Steering Coupling HDF5 NDGM MPI Finite Volume
  26. 26. “Gently Coupling” Codes from Separate Disciplines Distributed Data Hub Finite Volume Structural Mechanics Global Monitor Small Scale Analysis Injection Flow Analysis Runtime Visualization
  27. 27. COTS, GOTS and “Semi-Automatic” Visualization DICE and the HPC Environment X / OpenGL DICE User Interface XML and Http ( SSL + Kerberos ) DICE Server ( User Process ) DICE Object Directory XML, HDF5 I/O One or More Cooperating HPC Codes
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×