Grid Technologies in Disaster Management Presentation Transcript
Grid Technologies in Disaster Management & Infrastructure Protection Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47404 [email_address] http://grids.ucs.indiana.edu/ptliupages/presentations/CIGridGDINmar28-04.ppt http:// www.infomall.org http://www.grid2002.org
Grid is NOT a mesh in this talk!!!!
The Grid represents “Internet Scale Distributed Computing” or the “Managed Internet”
Major commercial and research Initiative – IBM HP Oracle Sun SGI Platform
We describe how one can use familiar system of systems (Grid of Grids) language for critical infrastructure and emergency response (command and control) Grids
We give infrastructure examples from flood and earthquake cases with HPC simulations linked to other Grid resources
Discuss GIS and Collaboration Grids
e-Business e-Science and the Grid
e-Business captures an emerging view of corporations as dynamic virtual organizations linking employees, customers and stakeholders across the world.
e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses.
The Grid or CyberInfrastructure integrates the best of the Web, Agents, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything .
A deluge of data of unprecedented and inevitable size must be managed and understood.
People , computers , data and instruments must be linked.
On demand assignment of HPC resources, experts, computers, networks and storage resources
e-Defense and e-Crisis
Grids support Command and Control and provide Global Situational Awareness
Link commanders and frontline troops to themselves and to archival and real-time data; link to what-if simulations
Dynamic heterogeneous wired and wireless networks
Security and fault tolerance essential
System of Systems; Grid of Grids
The command and information infrastructure of each ship is a Grid; each fleet is linked together by a Grid; the President is informed by and informs the national defense Grid
Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support
Grid supports On-demand HPC Simulations
Define and Build DoD relevant Services – Collaboration, Sensors, GIS, Database etc.
Raw (HPC) Resources Middleware Portal Services System Services System Services System Services Application Service Libraries User Services “ Core” Grid Typical Grid Architecture Application Service Application Service Re-use Re-use Application Customization Each service should be able to run independently on separate machines Service Oriented Architectures scale better than traditional Distributed objects Database
A typical Web or Grid Service
In principle, services can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining)
The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python
Payment Credit Card Warehouse Shipping control WSDL interfaces WSDL interfaces Web Services Web Services Security Catalog Portal Service
We are familiar with a hierarchy Lines of Code Methods Objects Programs Packages
Grids extend the software hierarchy to larger distributed systems and link it to a corresponding resource hierarchy
Overlay and Compose Grids of Grids Methods Services Functional Grids CPUs Clusters Compute Resource Grids MPPs Databases Federated Databases Sensor Sensor Nets Data Resource Grids
Information Grid Enterprise Grid Compute Grid Campus Grid Teacher Students Dynamic light-weight Peer-to-peer Collaborative Training Grid Composing an Overlay Grid R2 R1
Education Grid Inservice Teachers Preservice Teachers School of Education Teacher Educator Grids Informal Education (Museum) Grid Student/Parent … Community Grid Science Grids CI Grids Typical Science Grid Service such as Research Database or simulation Transformed by Grid Filter to form suitable for education Learning Management or LMS Grid Publisher Grid Campus or Enterprise Administrative Grid Education as a Grid of Grids Digital Library Grid
Solid Earth Research Virtual Observatory (SERVO)
NASA funded: JPL, UC Davis, UC Irvine, USC, Brown, Indiana
Web/Grid-services and portlet based Problem Solving Environment (PSE)
Couples data with simulation, pattern recognition software, and visualization
Enable investigators to seamlessly merge multiple data sets and models, and create new queries.
Spaced-based observational data
Ground-based sensor data (GPS, seismicity)
Published/historical fault measurements
Pattern recognition software
International Version iSERVO
Australia China and Japan as an APEC (Asia-Pacific Economic Cooperation) Initiative – next meeting Beijing July 2004
Database Database Research Simulations Analysis and Visualization Portal Repositories Federated Databases Data Filter Services Crisis Data Streaming Data Sensors SERVOGrid Research Crisis Customization Services From Research to CI/Crisis Cell Phone Mobile Network Geoscience Research and Crisis Grids Other Interdependent CI and Crisis Grids ? Discovery Services GIS
Database Database Analysis and Visualization Portal Repositories Federated Databases Data Filter Services Streaming Data Sensors SERVOGrid Research Education Customization Services From Research to Education Education Grid Computer Farm Geoscience Research and Education Grids Field Trip Data ? Discovery Services Research Simulations GIS
HPC Simulation Data Filter Distributed Filters massage data For simulation Other Grid and Web Services Analysis Control Visualize SERVOGrid (Complexity) Computing Model Grid OGSA-DAI Grid Services This Type of Grid integrates with Parallel computing Multiple HPC facilities but only use one at a time Many simultaneous data sources and sinks Grid Data Assimilation Data Filter Data Filter Data Filter Data Filter
Crisis and CI Grids
We first form Critical Infrastructure CI Grids from the “atomic” functional and resource Grids and Services
This gives water (flood), gas, postal, agricultural etc. Grids
Then we customize, compose and overlay CI with other Grids (such as weather, census data) for
Emergency Response (Command and Control) or Crisis Grids
Critical Infrastructure CIGrids Flood CIGrid Gas CIGrid … … Gas Services and Filters Security Workflow Messaging Notification Metadata Data Access/Storage Registry Physical Network Compute Sensors Visualization Collaboration openGIS Services Flood Services and Filters Electricity CIGrid
openGIS Grid Semantics
Note GIS (Geographical Information System) Grid at heart of all these Grids
Geography Markup Language (GML) is an XML encoding for the specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model.
We are building CI specific ontologies in terms of GML to define faults, satellites etc.
Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements.
This becomes the specification of portals to different composite Grids
Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors.
Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.
GIS Grid Services I
Web Feature Service (WFS) supports the query and discovery of geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature.
Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena.
Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS).
Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.
GIS Grid Services II
Web Terrain Service (WTS) augments WMS with advanced visualization including 3D terrains.
Catalog Service - Web Profile (CS-W) is a catalog service that will be built on a general Grid metadata service
Sensor Collection Service (SCS) fetches observations from a sensor or group of sensors and will be integrated with research on Grid sensor services
Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors.
Web Notification Service (WNS) will be replaced by standard Grid notification service
Portals need to provide services for security, customization, layout, rendering
Jetspeed and GridSphere are two well known portals
Commercial systems like IBM WebSphere similar
These Portals are very important as they encourage “ component ” model for user interfaces and so this fits service model so every service can be packaged with its (document fragment) user interface
P ortlets are good as they support they mixing and matching of services for a particular composed/overlay Grid to allow this to be easily reflected in user interface
New JSR168 portlet Java Standard
NSF Middleware Initiative NMI funding collection of open source Grid portlets
The OGCE Computing Grid Portal
Provides Portlets for
Management of user proxy certificates
Remote file Management via Grid FTP
Grid Event/Logging service
Access to OGSA services
Access to directory services
Specialized Application Factory access
Access to Metadata Index tools
User searchable index
Real Time Collaboration
Download from open source http://www.ogce.org
Example Capability: File Management
Grid FTP portlet– Allow User to manage remote file spaces
Uses stored proxy for authentication
Upload and download files
Third party file transfer
Request that GridFTP server A send a file to GridFTP server B
Does not involve traffic through portal server
Portal Server User GridFTP Server A GridFTP Server B GridFTP Service 1 of many Portlets Jetspeed
Watershed Entry Page: Select a County FloodGrid Portal
Watershed Boundaries in Black Open source GIS MapServer presented as a simple portlet
Data Download Portal This is where you can download ArcView compatible data files for your watershed. Some of this gets hooked to the runoff model in our connected crisis grid model.
Flood Grid Parallel Simulations Sensors GIS Data Archives provide terrain, elevation, and land use data Grid Workflow Links Data, Sensors, Model
SERVOGrid HPC Applications
Codes range from simple “rough estimate” codes to parallel, high performance applications.
Disloc : handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space.
Simplex : inverts surface geodetic displacements for fault parameters using simulated annealing downhill residual minimization.
GeoFEST : Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces.
Virtual California : Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space
RDAHMM : Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another.
PARK : Boundary element program to calculate fault slip velocity history based on fault frictional properties; a model for unstable slip on a single earthquake fault.
PDPC : Phase Dynamics Probability Change
Preprocessors, mesh generators
Visualization tools: RIVA , GMT
QuakeSim Portal Shots SERVOGrid linking HPC, Data and Visualization SAR Data
Run Finley – Simulation from Australia
Global-MMCS 2.0 XGSP MCU
We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services.
We will deploy it globally with first release end of May 2004.
The function of A/V media server will be distributed using NaradaBrokering architecture.
Media Servers mix and convert A/V streams
Open XGSP MCU based on the following open source projects
openh323 is basis of H323 Gateway
NIST SIP stack is basis of SIP Gateway
NaradaBrokering is open source messaging from Indiana
Java Media Framework basis of Media Servers
XGSP Web Service MCU Architecture Gateways convert to uniform XGSP Messaging High Performance (RTP) and XML/SOAP and .. Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing NB Scales as distributed Web Services NaradaBrokering SIP H323 Access Grid Native XGSP Admire Media Servers Filters Session Server XGSP-based Control NaradaBrokering All Messaging
A/V Collaboration Systems
GlobalMMCS Grid federates existing A/V Protocols
H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment.
The Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences
enhanced Mbone A/V tools ( VIC, RAT )
Internet 2 network ( Multicast support )
Grid Services in GlobalMMCS I
The audio mixer creates a mixed audio stream from all the audio streams in the session
Video mixing makes the unicast users watch the pictures of multiple participants in a meeting through one video stream
visualize the VS set in the session, embedded into the control panel of each endpoint, which
Image grabbers capture video streams and save them as static JPEG files.
All the media processing components can be distributed among the pool of the media servers connected to NaradaBrokering infrastructures.
This generalizes to a HPC farm of “stream servers” doing image processing etc.
Perhaps need 100 node cluster to support 10,000 simultaneous A/V streams
H.323, SIP Gateway Servers, A/V Session Server
H.323 and SIP gateway transform their protocol specific messages into XGSP signaling messages so that H.323 and SIP A/V endpoints could communicate with the XGSP A/V session server
The session server implements session management logics
creating/destroying A/V sessions
allowing endpoints to join/leave session
Allowing users to make audio/video selection, managing A/V application components
0 10 20 30 40 50 60 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Delay (Milliseconds) Packet Number Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms NaradaBrokering-RTP JMF-RTP
Polycom, Access Grid and RealVideo views of multiple streams using GlobalMMCS A/V Web Service
Unicast AG Portlet
WS Display WS Viewer Master WS Display WS Viewer NaradaBrokering Message Bus Collaboration as a WS Set up Session with GlobalMMCS defining participants Shared Output Port Collaboration Other Participants Text Chat Whiteboard GIS Services etc. are AUTOMATICALLY Collaborative if Built as a Grid Services WS Display WS Viewer Application or Content source WSDL Web Service F I U O F I R O
Integration of PDA, Cell phone and Desktop Grid Access