Establishing a User-Driven, World-Class Oceanographic Data ...Presentation Transcript
Establishing a User-Driven, World-Class Oceanographic Data Center by the Right People, in the Right Place , and at the Right Time L. Charles Sun National Center for Ocean Research 20-24 June, 2005, Taipei, Taiwan
Time, Place, and People
Steps in Establishing an NODC
Mission and Role of an NODC
QC and QA
Products and Services
Organizational Considerations and Chart
IDARS, Argo & GTSPP: Three examples of “Collaboratories”
Data Portal: “Gateway” to Ocean Data
Climate Data Portal: The Proven Prototype
Other Technologies for the Collaboratory
Time, Place, and People
Time: Since 1975 ~
Place: The Center of the world
People: We are the right people
Steps in Establishing an NODC - I
Recruit a team of interested parties to propose a mission and organizational model for the center.
Construct a draft mission.
Conduct negotiations with the potential partners.
Steps in Establishing an NODC - II
Prepare a draft administrative organization.
Prepare a final version of the mission and information on partnerships for final approval.
Organization Chart Office of the Director Director Deputy Director Associate Director Ocean Dynamics Chief Data Base Management Chief Information Technology Chief Staff Data Processing Research Data and Product Development Data Archival Database Development and Maintenance Networking Operating System Maintenance Hardware/software purchase and Maintenance Library Chief Service
Mission of an NODC
To safeguard versions of oceanographic data and information.
To provide high quality data to a wide variety of users in a timely and useful manner.
Roles of an NODC
Conventional role – as a minimum
Contemporary role – in response to advances in data collection and information technology
Conventional Role - I
Receive data, perform quality control, archive and disseminate it on request.
Keep copies of all or part of its data holdings in the format in which the data were received.
Developing and protecting national archives of oceanographic data
Conventional Role - II
Produce and provide inventories of its holdings on request.
Referral of the users to sources of additional data and information not stored in the NODC.
Participate in international oceanographic data and information exchange.
Contemporary Role - I
Receive data via electronic networks on a daily basis, process the data immediately, and provide outputs to the user or to the data collectors for data in question.
Report the results of quality control directly to data collectors as part of the quality assurance module for the system.
Contemporary Role - II
Process and publish data on the Internet and on CD/DVD-ROMs.
Publish statistical studies and atlases of oceanographic variables.
Performing a level of quality control on its data holdings
Quality Control and Assurance
Data can be detected easily by a data center
Obvious errors such as an impossible date and time and location
Data cannot usually be detected by a data center
Subtle errors such as an instrument may be off calibration
Information Technologies - I
Local Area Networking
Wide Area Networking – the Internet (and the GTS)
Information Technologies - II
Graphics Capability (Graphical Information System)
Software Development & Implementation
Hardware procurement & Maintenance
Products Development - I
Work with the client to determine what the real need. Examples of data products include atlases, datasets of ocean observations filtered by area, time and variables observed
Products Development - II
Review the world wide web sites of existing NODCs for ideas and examples of data and Information products.
Providing directory and inventory information
Acting as a referral center
Receiving data for specific processing followed by delivery of the processed data
A centralized data center
A distributed data center
Centers of Data : “Data Portals” or “Virtual Collaboratories”
Data Center Center of Data A Center of Data B Center of Data C
What is a Collaboratory? The fusion of computers and electronic communications has the potential to dramatically enhance the output and productivity of researchers. A major step toward realizing that potential can come from combining the interests of the scientific community at large with those of the computer science and engineering community to create integrated, tool-oriented computing and communication systems to support scientific collaboration. Such systems can be called " collaboratories ." From "National Collaboratories - Applying Information Technology for Scientific Research," Committee on a National Collaboratory, National Research Council. National Academy Press, Washington, D. C., 1993.
Acknowledgement Soreide, N. N. and L. C. Sun, 1999: Virtual Collaboratory: How Climate Research can be done Collaboratively using the Internet. U.S. – China Symposium and Workshop on Climate variability, September 21-24, 1999, Beijing, China Presented by Len Pietrafesa, North Carolina State University.
International Steering Committee Collaboratory Partner Collaboratory Partner Collaboratory Partner Collaboratory Partners & Customers Providers of Data & Information Users of Data & Information Observations & Satellite Groups Modeling & Forecasting Groups Research Groups New Users Educational Administrators General Public Structure of the Collaboratory for Ocean Research
IDARS * as an example...
Real-Time Coastal Water Temperature Data
Real-Time Argo Profile Data
Real-Time Global Temperature and Salinity Profile Data
Time Series Data
NOAA CoastWatch AVHRR SST Images
http://www.nodc.noaa.gov/idars/ * Interactive Data Access and Retrieval System
Argo as an example...
GTSPP * as an example... * G lobal T emperature- S alinity P rofile P rogram
Argo and GTSPP
Argo and GTSPP set a standard in the international ocean data management community
Data dissemination in near-real time
Researcher involvement has assured data quality
Benefits of data dissemination
Wide use of Argo and GTSPP data
Traditional research, modeling, forecasting groups
Related disciplines, educational, administrative, public
With recent advances in technology, we can do much more...
Distributed Object Technology
Data servers and datasets are objects – software packages of procedures and data that contain their own context
Solid, commercial underpinning for distributed object technology in the ocean sciences
Adaptability and Scalability of distributed object systems
Distributed object systems in the commercial arena
Are robust, reducing system maintenance and upkeep costs
Supported by Object Management Group (OMB)
standards body for Internet Inter-ORB Protocol (IIOP) distributed object protocols
CORBA/IIOP and Java RMI/IIOP
consortium of large (Fortune 500) companies
Cross platform independence, compliance with standards
The Data Portal: a “gateway” to ocean data
Why do we need a Data Portal?
Each center of data provides a highly customized Web sites for their data
but different datasets have different navigation and interface characteristics
so the user faces a bewildering spectrum of data access interfaces and locations
Data Portal is single, uniform, consistent “gateway” to ocean data in a common format
User goes to a single location and sees a consistent interface
Complements the customized data access
Data & Information Users Distributed data Observed data Satellite data Data and information products Model outputs Visualization Uniform network access
Web Browser Java Application User Network CORBA* Client Support Java Servlet Graphics One or more Web Servers TAO data support CORBA* Data Observing System Server Data Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Server Data Portal
Web Browser Java Application User Network CORBA* Client Support Java Servlet Graphics One or more Web Servers Drifter Data support CORBA* Data TAO data support CORBA* Data Observing System Servers Data Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Servers Data Portal
Web Browser Java Application User Network CORBA* Client Support Java Servlet Graphics One or more Web Servers Drifter Data support CORBA* Data TAO data support CORBA* Data Observing System Servers In-Situ/Satellite data support CORBA* Data In-Situ/Satellite Data Servers Model data support CORBA* Data Model Output Servers Data Gridded data support CORBA* Data Gridded Data Servers Common Object Request Broker Architecture (CORBA) is an industry standard Middleware. CORBA is used in the NOAAServer software from which this effort will leverage. Based on performance indicators, Java Remote Method Invocation (RMI), an alternative middleware, could easily be substituted for CORBA. CORBA* Network Data Servers Data Portal
How do we build a Data Portal?
Build on a proven prototype
connects 5 geographically distributed data servers in Silver Spring, Boulder, Seattle
CORBA for network connections
unified interactive Java graphics
data from distributed servers are co-plotted together on the same axis on the users desktop
Prototype Data Portal: CDP * Silver Spring MD * Climate Data Portal Boulder CO Seattle WA Honolulu HI
Climate Data Portal Sample Plots
Data Selection : Web Interface
Utilizes CORBA for network connections.
Utilizes EPIC Web Technology:
Searches data by keywords, location and time ranges.
Web Interface s creen Shots
Other Technologies for the Collaboratory:
Networks (100 Megabits/sec today, 10 Gigabits/sec in future)
Next Generation Internet (NGI) and Internet 2
Interactive Java graphics
3D, Virtual reality
high-speed telecommunications systems for advanced collaboration applications
tele-immersion systems allow individuals at different locations to share a single virtual environment
Use networks not airplanes for collaboration
Virtual Reality lets the scientist touch the data, move into it, and see it from different viewpoints
The realism of virtual reality enables the scientist and the lay person to understand complex ideas more easily
Scientists using virtual reality affirm this new technology discloses features of their data and model outputs which were undiscovered with standard visualization techniques
Virtual reality can be approachable and affordable
Widens audience for scientific data and information
Government administrators and decision makers
Educators and students
Some examples follow…
Courtesy of Nancy N. Soreides, PMEL
Why use Virtual Reality? Virtual reality modeling language (VRML) rendering of temperatures and sea surface topography along the equator in the tropical Pacific, viewed from South America, showing the dynamics of El Nino and La Nina. Using an inexpensive PC and a web browser with a free plug-in, the images can be rotated, animated, and zoomed. Changes in the equatorial Pacific during El Nino and La Nina are clearly understood by scientist and layman. http://www.pmel.noaa.gov/toga-tao/vis/vrml/ or http://www.pmel.noaa.gov/vrml El Nino La Nina Courtesy of Nancy N. Soreides, PMEL
Stereographic Virtual Reality 3D, interactive virtual reality visualizations are not difficult for a scientist to create or to view, from the web or from the desktop, and the effect can be enhanced dramatically by including the capability of stereographic viewing. With a PC and a 99-cent pair of red/green sci-fi glasses, the spheres and vectors will pop out of the page in stereo, revealing the true 3D location of the fish, the steep slopes of the bathymetry, and the vertical motions near the submarine canyon. The images can be rotated, animated and zoomed. http://www.pmel.noaa.gov/~hermann/vrml/stereo.html Fish larvae and velocity vectors in a submarine canyon, from a circulation model of Pribolof Canyon in the Bering Sea. Use red/green glasses to see images on the right in stereo. Stereo Stereo Courtesy of Nancy N. Soreides, PMEL
Immersive devices provide the graphical illusion of being in a three-dimensional space by displaying visual output in 3D and stereo, and by allowing navigation through the space.
Navigating through our virtual environments and viewing the data from different vantage points greatly increases our ability to perform analysis of scientific data.
The impact of such visualizations in person is stunning, and must be experienced by the scientist to be fully comprehended .
Users of these advanced immersion technologies affirm that no other techniques provide a similar sense of presence and insight into their datasets .
Immersive Virtual Reality Courtesy of Nancy N. Soreides, PMEL
The CAVE The CAVE is a multi-person, high resolution, 3D graphics video and audio virtual environment. The size of a small room (10x10x10 foot), it consists of rear-projected screen walls and a front-projected floor. Using special "stereoscopic" glasses inside a CAVE, scientists are fully immersed in their data. Images appear to float in space, with the user free to "walk" around them, yet maintain a proper perspective. The CAVE was the first virtual reality technology to allow multiple users to immerse themselves fully in the same virtual environment at the same time. View of the CAVE Scientist inside the CAVE CAVES have been deployed in academia, government, and industry, including NASA, NCAR, NCSA, Argon National Laboratory, Caterpillar Corp., General Motors, among others. http://www.pyramidsystems.com/CAVE.html Courtesy of Nancy N. Soreides, PMEL
The ImmersaDesk Courtesy of Nancy N. Soreides, PMEL
“ The development of scientific data manipulation and visualization capabilities requires an integrated systems approach … [including] the end-to-end flow of data from generation to storage to interactive visualization, and must support data retrieval, data mining, and sophisticated interactive presentation and navigation capabilities.”
“ Data Exploration of petabyte databases will required both technology development and altered work patterns for research scientists and engineers.”*
* Data and Visualization Corridors, Report on the 1998 DVC Workshop
Series, Edited by Paul H. Smith and John van Rosendale, Sponsored by the
Department of Energy and the National Science Foundation, 1998.