• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
The FME Oven: Never Too Many Ingredients
 

The FME Oven: Never Too Many Ingredients

on

  • 780 views

 

Statistics

Views

Total Views
780
Views on SlideShare
780
Embed Views
0

Actions

Likes
0
Downloads
13
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Linear referenced values are added in a previous step

The FME Oven: Never Too Many Ingredients The FME Oven: Never Too Many Ingredients Presentation Transcript

  • The FME Oven – Never Too Many Ingredients
    Kurt Hartman
    Director of Technology, Accurate Assessment Group Ltd.
  • Agenda
    Introduction to Accurate Assessment Group
    Case Study – Video Logging
    Case Study – ERCB Data
    Questions
  • Introduction – Client Map
  • Introduction – Webmap Clients
    Urban Clients
    City of Wetaskiwin
    Town of Barrhead
    *Town of Beaverlodge
    *Town of Edson
    *Town of Fox Creek
    Town of High Level
    Town of Peace River
    Town of Redwater
    *Town of Sexsmith
    Town of Stettler
    *Town of Two Hills
    *Town of Wembley
    *Town of Valleyview
    *Village of Derwent
    *Village of Myrnam
    *Village of Willingdon
    * Regional Sites
    Rural Clients
    Brazeau County
    County of Athabasca
    Camrose County
    *County of Grande Prairie No. 1
    County of Minburn
    County of St. Paul
    County of Stettler
    *County of Two Hills No. 21
    County of Wetaskiwin
    Kneehill County
    Lamont County
    *Municipal District of Greenview No. 16
    Municipal District of Opportunity No. 17
    Rural Municipality of Wood Buffalo
    Smoky Lake County
    Westlock County
    Wheatland County
    Woodlands County
    *Yellowhead County
  • Introduction – Municipal Information Integration
  • Video Logging
    Video logging: a method of displaying video data within a GIS
    Video is captured using a vehicle equipped with digital video cameras, precision GPS and on-board computers
  • Video Logging
    Precise digital images are captured at regular intervals from GPS-equipped vehicles traveling at regular road speeds up to 100 km/ hr.
    • Video is post-processed using asset extraction/ identification software.
    • Assets can be located 80-100 meters from the vehicle.
  • Video Logging – Ingredients...
    5,100,000 images
    1,700,000 points
    X,Y,Z
    GPS heading
    Image name
    Clip name
    Date/Time
    Road network
  • Video Logging – Out of the oven...
    4,800,000 images
    1,600,000 linear referenced events
    Linear reference values/keys
    Direction of travel
    Image name
    Image path
    Batch files to create necessary directories
    Batch files to rename and move image files
  • Video Logging – Challenge 1
    Attaching points to the correct road
  • Video Logging – Solution 1
    Use Labeller to determine the orientation of the road in the vicinity of the point
    • Compare road orientation to GPS heading using Expression Evaluator
  • Video Logging – Solution 1 (Cont’d)
    Using a Tester, determine if the difference between the road and heading azimuths are acceptable
    • Depending on the results from the Tester, assign the direction of travel for the point
  • Video Logging – Solution 1 (Overview)
  • Video Logging – Challenge 2
    More than one pass on the same road
    Blue and black points are going the same direction
  • Video Logging – Solution 2
    Use StatisticsCalculator to determine for each video clip/road combination:
    Smallest linear reference value
    Largest linear reference value
    Total number of points
  • Video Logging – Solution 2 (Cont’d)
    Use ExpressionEvaluator to determine the coverage that each clip has per road
  • Video Logging – Solution 2 (Cont’d)
    Use a series of 3 Testers to validate which records should be included in the final dataset:
    Test 1:
    If the point is part of the only video clip on that road and it covers more than 10% of the road
    If it passes, include it
    If it fails, forward it on to Test 2
  • Video Logging – Solution 2 (Cont’d)
    Test 2
    If the point is part of a clip that covers more than 25% of the road and the total coverage on the road is less than 110%
    This would handle scenarios where more than one clip is needed to cover a road
    If it passes, include it
    If it fails, forward it on to Test 3
  • Video Logging – Solution 2 (Cont’d)
    Test 3
    If the point is part of a clip that covers more than 75% of the road
    If it reaches this test, then it is likely a road that has more than one pass
    To determine which of the passes gets included we include additional variables
    Largest amount of coverage
    Most images
    Most recent date
    If it fails, forward it on to the Unused feature
  • Video Logging – Solution 2 (Overview)
  • Video Logging – Challenge 3
    Around 5,000,000 images (about 1.4 TB) requires intelligent file management
    Developed a file structure that takes into account:
    Year of image
    Road name
    Alberta Township Survey township identifier
    Which camera (front, side, rear)
  • Video Logging – Solution 3
    Points that are to be included in the final dataset are also forwarded to the Create Batch File process
    Using a series of Testers, Concatenators and StringReplacers the Create Batch File:
    Create batch files that make the necessary directory structure
    Creates batch files that move and rename the image files
    Pushes the new image name and path back into the final dataset
  • Video Logging – Solution 3 (Overview)
  • Video Logging – Final Translation
  • Video Logging – Finished Product
  • Video Logging – Benefits
    Predictable result
    Reproducible result
    36 person hours to create translation
    1 person hour to run and validate
    170 steps completed with 1 mouse click
  • ERCB Data
    ERCB – Energy Resources Conservation Board
    Maintains Oil & Gas data for Alberta
    Wells
    Pipelines
    Facilities
  • ERCB Data – Ingredients…
    2 shape files
    9 text files
    Cryptic field names
    Uses a lot of codes and abbreviations
    Eg: Pipeline material type = “G”
  • ERCB Data – Out of the Oven…
    4 feature classes
    Meaningful field names
    User-friendly data structure
    Replace codes and abbreviations with “English” descriptions
    Eg: Pipeline material type = “Composite”
  • ERCB Data
    Transformers used (118 in total):
    Joiner
    StringConcatenator
    SubstringExtractor
    FeatureMerger
    AttributeValueMapper
    PointConnector
    AttributeRenamer
    Tester
  • ERCB Data - Overview
  • ERCB Data - Benefits
    Create user-friendly dataset
    Predictable result
    Reproducible result
    118 steps completed with 1 mouse click
  • ERCB Data – Finished Product
  • Thank You!
    Questions?
    For more information:
    Kurt Hartman
    Kurt@aag-gis.com
    Accurate Assessment Group Ltd.
    www.aag-gis.com