Integrating GIS utility data in the UK


Published on

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This shows where assets are but in fact this should be where pipes might be.
  • DTI has wide range of responsibilities: company law, trade, business growhth, innovation, employment law, energy, science, consumer law… in 2005 it was called the depth of productivity, energy and industry… TMA and level of accuracy
  • Integrating GIS utility data in the UK

    1. 1. The VISTA project: Integrating UK utility data Beck, Boukhelifa, Cohn, Fu & Parker
    2. 2. Overview
    3. 3. The Utility Underworld <ul><ul><ul><li>Massive network of buried services: gas, water, electricity, telephone, cable, sewage, drains … </li></ul></ul></ul><ul><ul><ul><li>Need to know asset location for planning and maintenance </li></ul></ul></ul><ul><ul><ul><li>Many databases, varying accuracy and provenance </li></ul></ul></ul><ul><ul><ul><li>Context </li></ul></ul></ul><ul><ul><ul><ul><li>~4M street openings p.a. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Direct costs of £1B p.a. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Indirect costs of £3B-£5B p.a. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Safety! </li></ul></ul></ul></ul>
    4. 4. A Congested View
    5. 5. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    6. 6. VISTA Consortium Visualising Integrated information on buried assets to reduce streetworks VISTA
    7. 7. VISTA Project <ul><li>4 year government funded project </li></ul><ul><li>23+ Utility partners, other universities (Nottingham and Leeds) </li></ul><ul><li>Aims to reduce cost of street works in the UK </li></ul><ul><ul><ul><li>Motivation: Traffic Management Act </li></ul></ul></ul><ul><li>Facilitate sharing and exchange of knowledge about buried assets </li></ul><ul><ul><ul><li>Digitise existing paper maps </li></ul></ul></ul><ul><ul><ul><li>Integrate utility data </li></ul></ul></ul><ul><ul><ul><li>Visualize the integrated data set </li></ul></ul></ul>
    8. 8. The Vision (details) <ul><li>Tagline: Swift, safe, cost-effective streetworks </li></ul><ul><li>Objectives for Leeds University </li></ul><ul><ul><li>Agree a core set of attributes </li></ul></ul><ul><ul><li>Provide a framework for integrating and accessing this data </li></ul></ul><ul><ul><li>Investigate presentation needs for different classes of user </li></ul></ul><ul><ul><li>Design appropriate presentation techniques </li></ul></ul>
    9. 9. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    10. 10. Utility Data <ul><li>Heterogeneous in nature </li></ul><ul><ul><li>Modern data is predominantly, but not exclusively, digital (GIS: vector) </li></ul></ul><ul><ul><ul><li>Available in paper / raster / vector </li></ul></ul></ul><ul><ul><li>No common format or standard for data </li></ul></ul><ul><ul><li>Captured over the past 200 years </li></ul></ul><ul><ul><li>Variation in data quality </li></ul></ul><ul><ul><ul><li>Only 50% of buried infrastructure location known accurately (Marvin and Slater (1997)). </li></ul></ul></ul><ul><ul><li>Relative vs. Absolute positioning </li></ul></ul>
    11. 11. The current picture
    12. 14. Current Practice <ul><li>Utility packs </li></ul><ul><ul><ul><li>Combined service drawings are rare </li></ul></ul></ul><ul><ul><ul><li>Reliance on Ordnance Survey backdrop for visual integration of asset information </li></ul></ul></ul><ul><ul><ul><li>User preference for n+1 maps </li></ul></ul></ul><ul><ul><ul><ul><li>Users would like combined service drawings </li></ul></ul></ul></ul><ul><li>Preparation </li></ul><ul><ul><ul><li>Manual process </li></ul></ul></ul><ul><ul><ul><li>Informal design </li></ul></ul></ul><ul><ul><ul><li>No standard format or symbology </li></ul></ul></ul><ul><li>Vista aims to automate map production </li></ul>
    13. 15. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    14. 16. Utility Data: Problem Domain <ul><li>Heterogeneous in Practice </li></ul><ul><li>Different ways of storing asset data </li></ul><ul><ul><li>Paper – CAD – GIS </li></ul></ul><ul><ul><li>Raster to Vector conversion </li></ul></ul><ul><ul><ul><li>Employed spatial grammar techniques through genetic algorithms </li></ul></ul></ul><ul><li>Different ways of storing digital asset data </li></ul><ul><ul><li>Different syntactic models </li></ul></ul><ul><ul><li>Global Schema based integration </li></ul></ul><ul><ul><ul><li>During prototype phase use ETL software (FME from Safe) </li></ul></ul></ul><ul><ul><ul><li>Will recommend that OGC interoperable sources are implemented </li></ul></ul></ul><ul><ul><ul><ul><li>A mixed model is inevitable in the short term </li></ul></ul></ul></ul><ul><li>Different ways of structuring digital asset data </li></ul><ul><ul><li>Different syntactic and schematic models </li></ul></ul><ul><ul><li>Integration based on a common utility data model (global schema)‏ </li></ul></ul><ul><ul><ul><li>Resolving schematic heterogeneity </li></ul></ul></ul>
    15. 17. Utility Data: Problem Domain <ul><li>Different ways of describing asset data </li></ul><ul><ul><li>Semantic inconsistency </li></ul></ul><ul><ul><li>Ontology/Global Thesauri employed at the data level </li></ul></ul><ul><ul><ul><li>Resolving semantic heterogeneity </li></ul></ul></ul><ul><ul><ul><ul><li>When the same asset type is given different names by different companies </li></ul></ul></ul></ul><ul><ul><ul><ul><li>When different asset types are given the same name by different companies </li></ul></ul></ul></ul><ul><li>Different ways of sharing and representing asset data </li></ul><ul><ul><li>Paper – CAD – GIS </li></ul></ul><ul><ul><ul><li>Different symbols and conventions </li></ul></ul></ul><ul><ul><ul><li>Uncertainty </li></ul></ul></ul><ul><ul><li>User/domain tailored visualisations </li></ul></ul>
    16. 18. Integration constraints <ul><ul><li>Require low operational impact </li></ul></ul><ul><ul><li>Organisations are unlikely to change their internal data model </li></ul></ul><ul><ul><li>Organisations must retain full data autonomy </li></ul></ul>
    17. 19. The Vision (details) <ul><li>Aim: Swift, safe, cost-effective streetworks </li></ul><ul><li>Objectives </li></ul><ul><ul><li>Agree a core set of attributes </li></ul></ul><ul><ul><li>Provide a framework for integrating and accessing this data </li></ul></ul><ul><ul><li>Investigate presentation needs for different classes of user </li></ul></ul><ul><ul><li>Design appropriate presentation techniques </li></ul></ul>
    18. 20. Agree a core set of attributes <ul><li>Currently 29 Global Schema fields grouped into 11 types </li></ul><ul><li>Keep It Simple – flat file approach </li></ul><ul><li>Semantically transparent field-names </li></ul><ul><li>Distinguished between core and non-core data </li></ul><ul><ul><li>Asset x 10 fields (5 core) </li></ul></ul><ul><ul><li>Condition x 1 field (0 core) </li></ul></ul><ul><ul><li>Confidence x 1 field (0 core) </li></ul></ul><ul><ul><li>Date x 1 field (0 core) </li></ul></ul><ul><ul><li>Detection System x 1 field (0 core) </li></ul></ul><ul><ul><li>Dimension x 5 fields (5 core) </li></ul></ul><ul><ul><li>Domain x 4 fields (2 core) </li></ul></ul><ul><ul><li>GIS x 2 fields (1 core) </li></ul></ul><ul><ul><li>Location x 3 fields (1 core) </li></ul></ul><ul><ul><li>Rehabilitation work x 1 field (0 core) </li></ul></ul><ul><ul><li>Risk x 1 field (0 core) </li></ul></ul>
    19. 21. A framework for integration Overview
    20. 22. A framework for integration Syntactic (format) integration <ul><li>Essentially resolving the differences in GIS format between each utility dataset. Ideally will be done using a syntactically interoperable approach (OGC WFS). </li></ul><ul><li>Currently use FME middleware to convert the source data into the target format (ORACLE). This could be scripted to deal with data refreshing from different locations. </li></ul><ul><li>At implementation, there is likely to be a hybrid approach (middleware/WFS) in the short/medium term. </li></ul>
    21. 23. A framework for integration Schematic (structure) integration <ul><li>To generate the metadata that describes the relationship between the source data and the global schema . </li></ul><ul><li>These may be direct 1-1 mappings or they could represent transformations: </li></ul><ul><ul><li>Scaling numerical data </li></ul></ul><ul><ul><li>Calculated values (conversion of ‘depth of invert’ to ‘depth of cover’) </li></ul></ul><ul><ul><li>Compound data needs splitting </li></ul></ul><ul><ul><li>Atomised data needs compounding </li></ul></ul><ul><li>Furthermore, as utility data can be sparsely populated some error checking may be required. </li></ul><ul><li>Rule generation is a complex, iterative, process. </li></ul>
    22. 24. A framework for integration Schematic (structure) integration <ul><li>Metadata rules generated in RadiusStudio from 1Spatial </li></ul><ul><ul><li>Simple user interface. </li></ul></ul><ul><ul><li>Can do complex mapping and error checking. </li></ul></ul><ul><ul><li>RadiusStudio is a web service. </li></ul></ul><ul><li>We will be researching into techniques to deploy this metadata in XSLT. </li></ul><ul><ul><li>XSLT could be used to store all metadata natively. </li></ul></ul><ul><ul><li>However, is difficult to edit, design and share. </li></ul></ul><ul><ul><li>Domain knowledge is essential so the rules must be understood by the end-users. </li></ul></ul>
    23. 26. A framework for integration Semantic (term) integration <ul><li>Generation of a cross domain global thesaurus </li></ul><ul><li>Thesaurus: a tree of terms linked together by hierarchical, and associative or equivalence relationships. </li></ul><ul><ul><ul><li>Can be converted into an OWL ontology </li></ul></ul></ul><ul><ul><ul><li>Developed within MultiTes Pro </li></ul></ul></ul><ul><li>Articulated in Oracle </li></ul><ul><li>Reconciling semantic heterogeneities </li></ul>
    24. 27. A framework for integration Overview
    25. 28. Integrated Data <ul><li>The approach allows data from different utility domains to be successfully integrated </li></ul><ul><ul><li>Network data </li></ul></ul><ul><ul><li>Furniture data </li></ul></ul><ul><li>This provides greater flexibility for data presentation </li></ul><ul><ul><li>Other attributes can be used during data visualisation </li></ul></ul><ul><li>Global schema continually under development </li></ul><ul><ul><li>Requirements for telecoms </li></ul></ul><ul><ul><li>3d features </li></ul></ul><ul><ul><li>Changes are easy to implement </li></ul></ul><ul><ul><ul><li>because of the way the metadata is collected, stored and shared </li></ul></ul></ul>
    26. 30. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    27. 31. Developing a utility web service
    28. 32. Pilot Projects <ul><li>Infrastructure: Traditional Web GIS </li></ul><ul><ul><li>VISTA: ORACLE (datastore) </li></ul></ul><ul><ul><ul><li>Source utility data – materialized </li></ul></ul></ul><ul><ul><ul><li>Schematically and semantically integrated </li></ul></ul></ul><ul><ul><li>VISTA: GeoServer (WFS delivery) </li></ul></ul><ul><ul><ul><li>Consumes data from Oracle </li></ul></ul></ul><ul><ul><ul><li>Delivers OGC compliant WFS </li></ul></ul></ul><ul><ul><ul><ul><li>Via a secure link between Leeds and Developer servers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Integrated data </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Materialized views of integrated data (faster delivery) </li></ul></ul></ul></ul><ul><ul><li>Developer Service </li></ul></ul><ul><ul><ul><li>Consumes VISTA WFS </li></ul></ul></ul><ul><ul><ul><li>Developer controls how the data is rendered </li></ul></ul></ul><ul><ul><ul><ul><li>Can be made fit for multiple different end-users (Planning, on site, etc.) </li></ul></ul></ul></ul><ul><ul><ul><li>Renders integrated data to ‘accredited’ users </li></ul></ul></ul>
    29. 33. Pilot Projects <ul><li>East Midlands x 2 </li></ul><ul><ul><li>Utilities: </li></ul></ul><ul><ul><ul><li>Anglian Water </li></ul></ul></ul><ul><ul><ul><li>Central Networks </li></ul></ul></ul><ul><ul><ul><li>National Grid </li></ul></ul></ul><ul><ul><ul><li>Severn Trent </li></ul></ul></ul><ul><ul><ul><li>United Utilities </li></ul></ul></ul><ul><ul><ul><li>Yorkshire Water </li></ul></ul></ul><ul><ul><li>Portal Developer </li></ul></ul><ul><ul><ul><li>Jacobs </li></ul></ul></ul><ul><ul><ul><li>Anglian Water </li></ul></ul></ul><ul><li>Scotland </li></ul><ul><ul><li>Data partners </li></ul></ul><ul><ul><ul><li>Perth and Kinross Council </li></ul></ul></ul><ul><ul><ul><li>Scottish Water </li></ul></ul></ul><ul><ul><ul><li>Transport Scotland </li></ul></ul></ul><ul><ul><li>Portal Developer </li></ul></ul><ul><ul><ul><li>Symology </li></ul></ul></ul>
    30. 34. Bespoke Visualization
    31. 35. Bespoke utility visualization <ul><li>Uncertainty Visualization </li></ul><ul><ul><ul><li>Include aspects of uncertainty onto the display </li></ul></ul></ul><ul><li>Incorporating Aesthetics </li></ul><ul><ul><ul><li>Reduce clutter and visual complexity </li></ul></ul></ul><ul><li>Ontology-based Visualization </li></ul><ul><ul><ul><li>Use utility ontology to drive the visualization </li></ul></ul></ul><ul><li>End user requirements are crucial </li></ul>
    32. 36. Uncertainty visualization EXAMPLES A. future environmental setting -> colour D. longitude / magnitude -> glyphs B. contours -> line style H. local uncertainty -> volume C. regional uncertainty -> quadtree E. noise -> overlaid grid G. kind of object -> blur F. shape -> texture
    33. 37. Visualization Prototype USE OF BLUR 2D SVG map of utility Data Gas Sewer Water OS
    34. 38. Visualization Prototype USE OF BACKGROUND COLOUR <ul><li>Unified 3-colour scheme </li></ul><ul><ul><ul><li>green : certain </li></ul></ul></ul><ul><ul><ul><li>yellow : probable </li></ul></ul></ul><ul><ul><ul><li>red : uncertain </li></ul></ul></ul>
    35. 39. Aesthetics and Clutter <ul><li>Close proximity between assets </li></ul><ul><li>Line crossings and busy junctions </li></ul><ul><li>Missing 3D information </li></ul><ul><li>Label overlap </li></ul><ul><li>Backdrop information </li></ul>
    36. 40. Aesthetics and Complexity <ul><li>Bends (b) </li></ul><ul><li>Crosses (c) </li></ul><ul><li>Angles (m) </li></ul><ul><li>Orthogonality (o) </li></ul><ul><li>Symmetry (s) </li></ul>[Purchase, 98]
    37. 41. Aesthetics – A graph example
    38. 42. Detect Detect cluttered areas
    39. 43. Then simplify
    40. 44. Aesthetics – Is it possible to improve this! <ul><li>Promote area of interest </li></ul><ul><ul><li>Declutter </li></ul></ul><ul><ul><li>Graph the non-essential areas </li></ul></ul><ul><ul><ul><li>Retain context </li></ul></ul></ul>
    41. 45. Ontology Driven Visualization To be developed
    42. 46. Ontology-based visualization <CONCEPT> <DESCRIPTOR>Hydrant</DESCRIPTOR> <NT level=&quot;1&quot;>Combined Hydrant </NT> <NT level=&quot;1&quot;>Fire Hydrant </NT> <NT level=&quot;1&quot;>Washout Hydrant</NT> <BT level=&quot;1&quot;>Water Furnishing and Fixture <BT level=&quot;2&quot;>Water Asset</BT> </BT> <SN>Device for extracting large volumes of water from the network. Used to flush out the network, provide water for fire fighting or provide a temporary connection.</SN> <SC>Water</SC> </CONCEPT> <CONCEPT> <NON-DESCRIPTOR>Impounding Reservoir</NON-DESCRIPTOR> <USE>Raw Water Storage</USE> <SN>….</SN> <SC>Water</SC> </CONCEPT> Ontology
    43. 47. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    44. 48. Implementation issues Operational Impact <ul><ul><li>Require low operational impact </li></ul></ul><ul><ul><li>Organisations are unlikely to change their internal data model </li></ul></ul><ul><ul><li>Organisations must retain full data autonomy </li></ul></ul><ul><ul><li>Changes, such as WFS, should have additional business benefits </li></ul></ul>
    45. 49. Implementation issues Data Currency <ul><ul><li>Data Currency </li></ul></ul><ul><ul><li>Data currency should fit the purpose of the end use </li></ul></ul><ul><ul><ul><li>Back office planning </li></ul></ul></ul><ul><ul><ul><li>Field requirements </li></ul></ul></ul><ul><ul><li>Recognise that there is always lag in the system </li></ul></ul>
    46. 50. Implementation issues Other issues <ul><ul><li>Other issues </li></ul></ul><ul><ul><li>Response time </li></ul></ul><ul><ul><ul><li>Virtual vs. materialised </li></ul></ul></ul><ul><ul><li>Scale of integration </li></ul></ul><ul><ul><ul><li>Localised? </li></ul></ul></ul><ul><ul><ul><li>National? </li></ul></ul></ul><ul><ul><li>Data Security </li></ul></ul><ul><ul><ul><li>Requires unpicking </li></ul></ul></ul><ul><ul><ul><li>Always better than sharing data on CD! </li></ul></ul></ul>
    47. 51. Implementation issues Impact on architecture <ul><ul><li>Virtual or Materialised: </li></ul></ul><ul><ul><li>The utility industry needs to decide </li></ul></ul><ul><ul><li>What are the applications? </li></ul></ul><ul><ul><li>What are there requirements? </li></ul></ul><ul><ul><ul><li>Utility requirements are significantly different to disaster/emergency response </li></ul></ul></ul>
    48. 52. Overview <ul><ul><li>Introduction - VISTA </li></ul></ul><ul><ul><li>Utility Data </li></ul></ul><ul><ul><li>Data Integration </li></ul></ul><ul><ul><li>Data Delivery and Visualization </li></ul></ul><ul><ul><li>Implementation Issues </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
    49. 53. Conclusions <ul><li>VISTA has ‘proved the concept’ for dynamic integration of heterogeneous utility data in the UK </li></ul><ul><li>Developed a cross domain utility thesaurus </li></ul><ul><li>Developed a number of visualization mechanisms </li></ul><ul><li>Recognised a number of implementation issues </li></ul><ul><li>Further Work </li></ul><ul><li>Develop a rich cross domain ontology </li></ul><ul><ul><ul><li>Links to FGDC, the Common Information Model (IEC 61970-301 etc) </li></ul></ul></ul><ul><li>Ontology driven integration process </li></ul><ul><li>Ontology driven visualization process </li></ul><ul><li>Once stabilised the whole system should be modelled in UML </li></ul>
    50. 54. Any Questions? <ul><li> </li></ul><ul><li> </li></ul> e-mail: