Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWIPS II briefing


Published on

  • Be the first to comment

AWIPS II briefing

  1. 1. National Weather Service The Evolution of AWIPS NURO 5 September 2007 Jason Tuell Office of Science and Technology 1
  2. 2. Overview • The Why and What of AWIPS Evolution • Objectives • Re-architecture Approach • Roadmap • What does AWIPS II mean to ensembles? • Summary 2
  3. 3. Why Change? • Case for change briefed to NWS Corporate Board – Nov 2004 • AWIPS Present State Summary • Hardware – AWIPS hardware was in good shape • Communications Infrastructure – AWIPS communications infrastructure was in OK shape • Data – AWIPS Data was in need of improvements • Software – AWIPS software was in critical need of improvements – Costly software development, maintenance and inability to meet NWS and customer needs • Corporate board direction to focus on addressing software shortcomings • Plan and requirements developed • Shaped portions of the AWIPS O&M re-compete activity 3
  4. 4. What is AWIPS Evolution? • AWIPS Evolution • A long-term project which delivers a modern, robust software infrastructure that provides the foundation for future system level enhancements • AWIPS II • Implements a modern Services Oriented Architecture (SOA) infrastructure • First output of AWIPS Evolution and provides the foundation for all subsequent improvements • AWIPS II Extended - Creation of a seamless weather enterprise spanning NWS operations • Migration of NAWIPS into the AWIPS II SOA • Delivery of thin client – Support to the Weather Service Offices and Center Weather Support Units • Integration of “orphan” systems (e.g., Weather Even Simulator) • AWIPS II Enterprise Enhancements • Data Delivery Enhancements – “Smart push-smart pull” data access – Katrina satellite WAN back up • Integrated visual collaboration • Visualization Enhancements • Information Generation Enhancements 4
  5. 5. AWIPS Evolution Objectives • Establish Service Oriented Architecture for AWIPS and NAWIPS • Create a seamless weather enterprise that supports all levels of NWS operations from National Centers to WSOs • Build a common development environment that will be used by all developers • Establish infrastructure for GIS integration • Enable access to data independent of its location, i.e., provide access to data not resident locally at the WFO or RFC. • Provide infrastructure for real time graphical collaboration between • WFOs, RFCs and centers for enhanced internal collaboration • Other NOAA entities and • Trusted partners, e.g., Emergency Managers • Implement a Common AWIPS visualization environment (CAVE) used by all applications • Standardize generation of NWS products and services 5
  6. 6. AWIPS II Re-Architecture Approach • Perform “black-box” conversion • Preserve existing functionality, look and feel on top of new infrastructure • Thorough field validation and acceptance before deployment • No loss of functionality • Deployed system current with deployed AWIPS capability (i.e., OB9) • Use open source projects - No proprietary code • JAVA and open source projects enable AWIPS II to be platform and OS independent – No plans to move from Linux • ADE enables collaborative development • OS, Platform independence allows non-Linux based research to be easily integrated into AWIPS II 6
  7. 7. AWIPS II Features • AWIPS Development Environment (ADE) • Used by all AWIPS developers (National, Regional, & Local) • Developers concentrate on new capabilities, not re-implementing existing ones (i.e. screen I/O, communications protocols, data access routines, logging routines, or other previously developed capabilities) • Software can be developed on a variety of platforms • Robust infrastructure for improved software O&M • Use of plug-ins: visualization extensions; new data types and transforms • System level, remediation, core services reduce system complexity • Improved support for local requirements (e.g., local apps, scripts, plug-ins) • Common AWIPS Visualization Environment (CAVE) • Provides a common development and execution environment for AWIPS GUIs (e.g. D2D, NMAP, GFE, etc.) • Ability to pan/zoom large data sets (Raster & Vector) with flexibility over data rendering • GIS tools • Thin Client (Web Browser) enabled • Dynamic Load balancing • Processing dynamically allocated among available CPUs 7
  8. 8. AWIPS II Migration Roadmap Strategy 2006 2007 2008 2009 2010 AE OSIP Gates 2 3 4a 4b MPLS Meshed Topology OBx 7 8 8.3 9 10 OB 9 Dev & Test Deployment New Release ADE Training Development Paradigm SW CTR PIP ADE Training NWS New Capability Development in ADE (AWIPS II) Analysis RTS IRAD ADE Development O & M Transition O & M Transition Prep & Coordination Migration Planning Baseline Application Migration AWIPS II 1.0 Note: Task bar colors are ADE Local App Training OTE / Deployment Support “User” Functional Tests For speaker reference only Local App Migration C&A OTE = Calendar Year Deployment Deployment Planning = Fiscal Year Field Ops Training 9/05/07 -- ITO, ESA 8
  9. 9. AWIPS Evolution Roadmap 2007 2008 2009 2010 2011 2012 2013 2014 Baseline Application Migration AWIPS II AWIPS II OTE / Deployment Governance Model NAWIPS Migration AWIPS II Thin Client Extended WES Integration CHPS AWIPS II Data Delivery IOC FOC Enhancements Phase 1 Collaboration Phase 2 Phase 3 = Calendar Year = Fiscal Year Information Generation IOC FOC IOC Visualization 9
  10. 10. AWIPS Evolution Governance Model • What is it? • Governance model controls the development, test, integration, configuration management, deployment and support of the new system -- both hardware and software • Why? • AWIPS II offers new levels of flexibility and extensibility • New rules needed to take advantage of system capabilities and also define limits • Tension between unlimited modifications and ability to support the system • Sample issues for consideration • Monolithic configurations no longer required -- how do we manage site specific configurations? • Plug ins can be down loaded and installed on demand • Scripting that modifies AWIPS menus, functions • Flexible Governance Model needed for limited open source implementation 10
  11. 11. AWIPS II What does it mean to ensembles? • NAWIPS migration • Same infrastructure supporting both NAWIPS and AWIPS applications • Algorithms • Common library for meteorological and hydrological calculations and tools • Data delivery • Smart push-smart pull may provide more flexibility for access to large data sets • Visualization • Supports new tools and techniques for visualizing ensemble data • Information Generation • New architecture for product generation and delivery will allow tailored products capturing uncertainty 11
  12. 12. AWIPS II Training • Strategic Training Plan developed • Training targeted for the following groups: • ESAs • ITOs • AWIPS and application focal points • Developers (both baseline and local) • NCF • SST • Training Organizations involved in planning, developing and implementing courses 12
  13. 13. AWIPS II Testing • Layered testing strategy • Different testing phases overlap – functionality tested multiple times in different settings • ADE usage • Not formal testing, but expect to get feedback as developers use the ADE • Algorithm V&V – • Assist algorithm choice • Verify port of algorithms • V&V • Task order validation • Executed by dev orgs in controlled setting • Performance testing – Evaluate system against known and developing benchmarks • Acceptance testing – part of the task order acceptance process • Risk reduction • Side by side testing of new system by field personnel in a “lab” environment • Testing at limited number sites in manner that doesn’t impact operations • Operational Test & Evaluation (OT&E) • Formal testing of the entire system • Tests system interfaces, operations, support, training, etc. 13
  14. 14. AWIPS II Risks and Challenges • Organizational challenges • Significant off contract workload on Government • Matrixed structure increases project complexity • Performance • Supporting the short fuse warning mission • Handling large global data sets • Schedule • Completing the migration and testing • Migration of local applications • Local applications outside the baseline • Not a Raytheon responsibility 14
  15. 15. Summary • New architecture defined and infrastructure delivered • ADE 1.0 delivered June 14, 2007 • Application migration underway • Migration Plan delivered June 2007 • AWIPS baseline migration to be completed FY09 • Local applications migration planning in progress • Incremental approach will extend AWIPS II architecture throughout NWS • AWIPS II and subsequent enhancements will provide more flexible foundation for delivering products that capture uncertainty 15
  16. 16. National Weather Service Back Up 16
  17. 17. AWIPS II Local Applications • Survey – completed August 31, 2007 • 4191 local applications identified • 37% critical, 44 % important, 19% less important • Migration plan to address approach based on survey results • Training requirements and approach to be refined based on survey • Raytheon to provide sample migration and code samples for approach • “Integrate with” vice “interface to” • Level of effort required uncertain • Raytheon estimate that 80% of local apps will be able to be rewritten in Javascript, without extensive programming in the ADE 17
  18. 18. AWIPS II What gets us excited so far… • Dynamic load balancing • Failover handled automatically • Enables consideration of tailored hardware configurations • Mathematically intensive calculations handed off to the graphics card • Significant performance improvements • Progressive disclosure of all data • Imagery via quad tree tiling, grids and observations • Integrated thin client • Allows baseline solution to be extended to CWSUs, WSOs, and IMETs • Integrated drawing and graphical collaboration • Tools built into the infrastructure, implemented in 2011 • Built in GIS via geotools library • Scripting level access to practically all system level services and functions • LESS CODE • Potential order of magnitude reduction in amount of software with increase in functionality 18
  19. 19. AWIPS II GIS Capabilities • GIS functionality integrated into AWIPS II architecture via GeoTools • GeoTools • Open Source Java code library • Provides OGC compliant methods for manipulation of geospatial data • Equivalent to ESRI’s arc engine • AWIPS II will support JavaScript access to GeoTools library • ADE 1.0 supports limited access – Read, write shape files – Create GeoTiffs – Spatial queries of ingested data • Task Order 10 will deliver full JavaScript access • GeoTools does not make AWIPS a stand alone GIS system • Reference: • GeoTools home: • GeoTools & Eclipse: 19
  20. 20. AWIPS Evolution Outcomes • Short-term (1-3 years) • Shorten transition of research to operations • Improve software O&M and technology refresh – Fewer DRs and TTs – Focus on hardening and productionizing for life cycle support • Minimize adverse impacts on operations from software and hardware upgrades • Long-term (3-10 years) • Increase integration of AWIPS and National Center AWIPS • Improve performance and functionality of AWIPS • Improve collaboration at all levels of NWS operations • Increase access to all environmental data for decision making 20
  21. 21. AWIPS Evolution Data Delivery • OSIP Project 05-040 • Enables “smart push - smart pull” data delivery • Implements a discovery service within the SOA • Access to data not available locally • Freedom from the tyranny of the SBN • Enables consideration of new data delivery architecture – What data to you broadcast over SBN? – What data do you make available on servers? • Schedule • IWT starting Q4 2007 to define concept of operations and operational requirements • IOC - 2011 - software implementation for remote data access • FOC 2012 - enterprise configuration (servers, comms, etc.) that enables remote data access 21
  22. 22. AWIPS Evolution Collaboration • OSIP Project 05-041 • Objective • Integrated graphical collaboration throughout the NWS Weather Enterprise and beyond • Phase 1 - Integrated collaboration between all levels of NWS operations • Phase 2 - Collaboration between NWS offices and other NOAA entities • Phase 3 -Collaboration between NWS offices and trusted external partners, e.g., Emergency Managers • Schedule • Phase 1 IOC - 2011 • Phase 2 IOC - 2012 • Phase 2 IOC - 2013 22
  23. 23. AWIPS Evolution Information Generation & Visualization • OSIP Projects 05-042 (IG) and 05-021 (Vis) • Information Generation objective • Re-architect generation of all NWS products and services • Separation of content generation from formatting and dissemination • Enable faster response to emerging customer demands • Visualization objective • Common user interface - standardize User Interfaces across applications • 3-D visualization • Improve user interfaces based on latest principles and research 23
  24. 24. What is AWIPS? NEXRAD GOES/POES/NPOESS 169 separate AWIPS systems at 137 geographical locations Warnings NCEP Models Watches AWIPS Communications Advisories Forecasts ASOS AWIPS Workstations and Service provided to Servers 3066 US Counties Buoys, 24 hrs/day, River Gauges ~900 Workstations (total) 365 days/yr. ~1200 Servers (total) 24
  25. 25. Enterprise Architecture “AWIPS is the Integrating USERS Element of the nt En LOCAL EXTERNAL Lightning vir Modernized Weather me Cooperative USERS; Detection FAA Data on on COMPUTER me Service” vi r Observers SYSTEMS; n En OBS PLATFORMS AND NETWORKS USERS MDCRS t Lightning Commercial Aircraft PROFILER Data Data Atmospheric Profiles Official User Products, Observations, & (May Interface Locally Graphics NWSTG Upper Air NCEP GTS Observing at Site or Centrally VIA NWSTG Guidance, Gateway Observations, for International & Non- & Offical User World-Wide Data MICROART Upper Air NCF NOAA Domestic products Products Collection & Ocean Distribution Observations Obs LDAD NCEP Retrospective - Calibration Data NCDC Users Data AWIPS - Numerical Predictions - Large-scale POES Data Modeling - Interactive Data Class 4 GOES I-M, Polar Analysis NIDS -Archiving (NCF) Orbiter, and Forecaster Guidance Level 2 & POES Other Satellite Data Data Level 3 Data Surface Weather Radar Observations NESDIS POES Products Official User Spacecraft NEXRAD Official User - Satellite Product Gen Products - Satellite Operation Products Forecaster - RADAR Reflectivity Measurement ASOS - Product Formulation WALLOPS - RADAR Product Gen NWWS - Data Integration & Analysis CDA Satellite -Surface NWR - Forecasting Distribution & Observing - Services Dissemination DCP Processing (Services Data GOES Dissemination) Data USERS USERS GOES USERS I-M Systems Legend Spacecraft - Radiance Measurements Data Collection - DCP Data Collection & Figure 6-8 of Dissemination Platforms (DCP) Remote Surface Distribution Observing NOAA EA v 1.6 Modeling & Product Generation Environment Observing * Note some systems perform multiple functions, e.g. LDAD disseminates information and receives observations 25
  26. 26. Architectural Compliance • Collaboration on Achieving Enterprise Solutions • AWIPS is the Integrating Element of the Modernized Weather Service • AWIPS Uses Enterprise Red Hat Linux support contract • AWIPS Contract supports Climate, Weather and Water, Transportation, and extensible to Coastal and Ocean Resources • Reuse of existing assets • HazCollect uses AWIPS IT infrastructure for message handling/creation and routing to dissemination systems • Shared use of NOAA-Net for Wide Area Network in progress • AWIPS provides NEXRAD visualization and data collection functions • AWIPS software used by other Government projects - RSA II, WES, FX-Net • Usage of Standards • Complies with statutory requirements – Information Quality (Section 515) & Accessibility (Section 508) – Government Paperwork Elimination Act (GPEA) – FISMA/NIST Standards and Guidance 26
  27. 27. AWIPS Evolution An EA Perspective • EA perspective focuses on hydrological and meteorological mission operations • Administrative functions not part of AWIPS • Supercomputing functions outside of AWIPS • AWIPS II compliant with current DMIT data standards • Some services, e.g., discovery service, not implemented as part of AWIPS II • Will be implemented in the Data Delivery system enhancement • Exploring partnerships with other agencies within and outside of NOAA • NESDIS and NPOES NDE • NASA SPORT and NASA Goddard 27
  28. 28. Open Source Projects Used in ADE OS Project Functionality ADE Role ANT Build scripting Build system Mule + Spring Enterprise Service Bus + Container Runtime Environment ActiveMQ Java Messaging System Broker JMS Broker Jibx XML to Object Serialization Canonical XML Message GeoTools + JTS GIS capabilites GIS primitives Tomcat Web Server Test Client Server Baltik Scalable Vector Graphics Tools SVG tools Ehcache Event Driven Clusterable Cache Cache Framework Log4j Java Logging API Log manager Jogl Java API to OpenGL CAVE rendering interface Eclipse RCP GUI plug-in based framework CAVE framework Eclipse IDE Java Integrated Development Development environment MC4J Console JMX Management Console Remote management console 28
  29. 29. SOA Architecture – Logical Layered View Layers Separated By Simple APIs Client/Presentation Services JMX CAVE Security Services /Demilitarized Zone (DMZ) External Programs Enterprise Service Bus - Communication Mission Services Layer Mbean IngestSrv NotifySrv ProductSrv Hydro Models PersistSrv AutoBldSrv AdapterSrv IndexSrv SubscribeSrv UtilitySrv LAPS StagingSrv PurgeSrv FORTRAN/C/C++ Command Line Programs Data Access Layer <<Java>> <<Java>> <<abstract>> HDF5DataStore DataLayer BaseDao HDF5 API Hibernate Platform Layer Localization HDF5 PostgreSQL Store Data Persistence Metadata Spatial Store Index Index 29
  30. 30. AWIPS-II ADE High Level System Services SOA Services Running in an ESB Container CORE SOA Services JMX Remote WAN Visible Service Management ADE 1.0 Updated T06 Client Mbean Mbean Mbean Mbean Mbean CAVE IngestSrv PersistSrv IndexSrv ProductSrv AdapterSrv Visualization StagingSrv PurgeSrv Client FileSystem Meta Data Index uEngine Exec Adapter Data Rendering METAR Plug In RDBMS via JDBC Spatial Data Base Manage Subscription JNI Adapter Radar Plug In CAVE Bundles HDF5 persistence GFE IFPServer Data Interrogation Satellite Plug In GRIB Plug In CAVE Procedures Wx Drawing Wx Warning Radar All Tilts Hydro Visualization Mbean Mbean Mbean NotifySrv AutoBldSrv UtilitySrv Collaboration Subscription Notify uEngine Localization Data Color Tables XMPP Maps and Topo Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints Services Independent of End Points 30