Successfully reported this slideshow.
Your SlideShare is downloading. ×

Processing Rasters from Satellites, Drones, & More

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 25 Ad
Advertisement

More Related Content

Slideshows for you (20)

Similar to Processing Rasters from Satellites, Drones, & More (20)

Advertisement

More from Safe Software (20)

Advertisement

Processing Rasters from Satellites, Drones, & More

  1. 1. PROCESSING RASTERS FROM SATELLITES, DRONES, & MORE
  2. 2. Why do we care about rasters? Rasters are essential for background maps, spatial analytics, and visualizing data over time.
  3. 3. BRINGING RASTERS INTO YOUR WORKFLOWS ● Landsat 8 ● Sentinel 2 ● Planet ● Earth on AWS ● 54 new GDAL-based formats ● GDAL Generic Raster Reader ● GDAL VRT Raster Reader ● PDF Reader (also 2D & text data) Remote Sensing Sources New in FME 2018 ● Drones/UAVs ● Sensors Other Sources
  4. 4. Automation is key.
  5. 5. Top Raster Transformers Anything is possible for rasters in FME! safe.com/transformers
  6. 6. first Get data from Planet and other sources Read imagery and vector layers. Process Mosaic rasters, style vector data, combine, tile. Output as .png Store raster tiles in the cloud. Make available everywhere to anyone. third Automate Do this whenever new images become available. last DEMO: UP-TO-DATE HYBRID BASEMAP second
  7. 7. Check it live
  8. 8. HOW ABOUT MILLIONS OF TILES? WAIT A MINUTE...
  9. 9. first Get images Read Planet imagery of California wildfires. Process Store Save to local or cloud storage. third Analyze Make tools for visual and analytical change comparison. last DEMO: MONITOR AND ANALYZE CHANGE DAILY Derive new products like NIR, NDVI, NDWI. second
  10. 10. Check it live
  11. 11. first Get images from Planet Collect images for a selected timeframe. Process Clip to extents of desired area. Add video to map Prepare HTML output with LeafletJS. MAKE A GEOREFERENCED SATELLITE VIDEO third lastsecond Output as .mp4 Integrate with FFmpeg and output a video.
  12. 12. Scenario: NATURAL HAZARD WORKFLOWS Photo by C. Hitchcock
  13. 13. NATURAL HAZARDS: CHALLENGES ● Diverse data types needed. ○ Vector and raster ○ Open and proprietary ○ Spatial and non-spatial ● Limited IT infrastructure. Need to make data accessible. ● Urgency! Disasters mean short timelines. ● Need to automate processing and distribution.
  14. 14. KEY WORKFLOWS ● Impact assessment. ○ population and transportation ○ identify critical infrastructure ○ hazard sources and extents ● Data integration. ● Risk, probability, scenarios, time. ● UAV surveys, automation.
  15. 15. DEMO Vancouver Flood Hazard Risk Assessment
  16. 16. Bonus: Flood Notifications FME Knowledge Center Tutorial: http://fme.ly/flood
  17. 17. EXAMPLE: SEARCH AND RESCUE UAVs 1. Build route for UAV. 2. UAV flies autonomously and collects data. 3. Post-process – geolocate, assemble results into photomosaic.
  18. 18. Automated route generation SEARCH GRID GENERATION Automated image geolocation
  19. 19. UAVs FOR DAMAGE ASSESSMENT ● UAVs are easy and safe to deploy. ● Rapidly assess impact to people and infrastructure. ● Data guides the level of response. Story: Renato Salvaleon develops UAV systems at Southern Co, supported by FME Server automation. safe.com/uav UAV storm damage assessment by Southern Co.
  20. 20. MEETING NATURAL HAZARD CHALLENGES ● Diverse data types needed. ○ Vector and raster ○ Open and proprietary ○ Spatial and non-spatial ● Limited IT infrastructure. Need to make data accessible. ● Urgency! Disasters mean short timelines. ● Need to automate data processing and distribution. Rapid prototyping
  21. 21. Automation is key.
  22. 22. RESOURCES ● San Francisco Transit demo: http://fme.ly/sftransit ● Blog - earthquake notifications: http://fme.ly/earthquake ● Christchurch earthquake story: http://fme.ly/christchurch ● Flood notification tutorial: http://fme.ly/flood ● Risk analyzer by con terra: georiskanalyzer.com ● Blog - get started with drones: http://fme.ly/drones ● FME Server for Fort McMurray wildfire: http://fme.ly/atco ● Search “remote sensing” or “UAV” on safe.com
  23. 23. Q&A

Editor's Notes

  • We don’t care about rasters; we care about rasters mean - is it a picture, elevation model, etc. A raster is just a means to achieving the solution we want - we care about the environment or the city or whatever else it represents.
  • Rasters are coming to us in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows.

    Planet is a huge archive - billions of images - humans can’t manage all of this manually. We need automation.
  • Some useful ones worth mentioning.
    FeatureReader is not a raster transformer, but we want to mention it here because it plays an important role when reading satellite data. Instead of a traditional approach - read data, then figure out what to keep for further processing, we first figure out what we need (coverage, weather, quality, date range), and then read it using all those pre-processed values as parameters. MapnikRasterizer - for making a really nice output - maps or anything (chocolate packages)


    FeatureReader - let features drive what data you get from Planet. e.g. GIS boundaries
    RasterMosaicker - combine all the images we get
    MapnikRasterizer - we have lots of mapnik resources & webinars
    NDVI - the key is to say it’s a custom transformer - can add functionality that’s missing
  • http://fme.ly/sftransit or http://demos.fmeserver.com/planet-satellite-map/
  • server checks regularly for new images that meet criteria
  • https://blog.safe.com/2018/01/elastic-processing-cloud-docker-fme-server/

    Generating millions of tiles normally takes a full day… we were able to do it in 25 mins.
  • http://demos.fmeserver.com/farmland-monitoring-planet/
  • https://s3-us-west-2.amazonaws.com/safe-scenarios/Planet/video/20170907234358091020257.htm
  • We’re talking about assessing risks and damage from floods, earthquakes, forest fires, climate change, hurricanes, and more.
    Image 1: Alpine Shire Council wanted to improve bushfire risk assessments. With FME, they designed an app that gives real-time analysis of risk. (If you search “alpine shire council” on safe.com you can find this story.)
    Image 2: I85 Atlanta Bridge Collapse: a guy lit a grocery cart on fire and the fire grew and ultimately collapsed the bridge. This was a Southern Co project to assess the damage to infrastructure.
    Image 3: was an FME UC presentation - “Helping rebuild a city with FME” - after the Christchurch earthquake

    __

    Other examples:
    Critical infrastructure protection: FortisBC
    Humanitarian and disaster response: Hurricane, Syria, Myanmar (OGC Testbed13)
    UAVs allow for rapid assessment and response: UAV damage assessment and cell network support in Puerto Rico
  • Image of SFO flooding scenario from OGC TestBed 11. FME was used to read NetCDF data from a high resolution flood forecast model and convert this into a KML time series of raster images for display in Google Earth.

    Several key points from this slide are based on findings from the following study:
    https://unstats.un.org/unsd/geoinfo/RCC/docs/rccap20/16_Robert%20Deakin_New%20Zealand%20-%20Canterbury%20SDI%20Paper.pdf
    Twentieth United Nations Regional Cartographic Conference for Asia and the Pacific Jeju, 6 - 9 October 2015 Item 7(b) of the provisional agenda Invited Papers Canterbury SDI: lessons learned from post-earthquake recovery

    The root causes underlying these issues were identified as:
     The lack of data sharing agreements in place;
     No data sharing channels (such as web services), or standardisation of formats and data models
     No catalogue or registry of available data sources
     A lack of training or practice on how to pull together data sources
    Problems manifested as:
    • Inability to find key people who could supply data;
    • Inability to access the data;
    • Reluctance to supply data because of perceived poor quality;
    • Reluctance to supply data because the requestor was perceived as not needing it;
    • Reluctance to supply data because it was not known how the requestor might use it;
    • Privacy issues;
    • Supplying spatial data in unhelpful formats.
    Problems related to integrating data from multiple agencies included:
    • No plans on how to integrate data representing aspects such as people, places, things, events and concepts, from multiple agencies;
    • No plans on how to integrate data from a single agency into a recipient’s business system;
    • Data-models were different and the attributes contained different data. Schemas were not available;
    • Abstractions were different (e.g. mobile toilet delivery being recorded as a number against a street name by some people, and against an address by others);
    Geometry data types were different (whether the feature was stored as a point, a line, or a polygon); • Datasets were at different levels of completion; • Data was in different formats, e.g. WFS, ArcGIS, MS Access, Excel, CSV files, PDFs, paper forms and maps. Getting expertise was also a problem: • People were thrust into positions they were not equipped to deal with; • Information was captured by people without experience of spatial data; • Data was managed by people without experience of spatial data; • No access to technical expertise; • Out-of-town resources were unfamiliar with the city (e.g. common road names and place names); • Out-of-town resources were unfamiliar with the systems they were working in.

    __

    Opportunities with FME
    Common data models, spatial reference
    Open standards
    Cloud based, decentralized
    Mobile devices
    Crowd sourcing
    APIs, loosely coupled
    Web sources – leave in place
    Real time, non-spatial
    Model based, rapid prototyping
    Easy updates

    FME Capabilities
    Formats R&W
    Web Access, Publish
    Processing Geometry: Vector, raster, point clouds
    Processing records: ETL, schema mapping

    https://www.safe.com/solutions/for-industries/emergency-services-and-policing/

    Increase Disaster Response Efficiency
    Build Situational AwarenessSwiftly gather, combine, and analyse tabular and location-aware information from 350+ sources, providing facts for critical decision making in real-time.Leverage Citizen Engagement Harvest intelligence from citizens via social media, including geotagged photos, and return life-saving information back to the public.Hands-Free Data Flow Reduce manual efforts during emergencies by creating automated workflows that leverage complex event processing (CEP) technology to immediately deliver critical intelligence using real-time notifications.
  • Integrating various sources is key.
    Impact assessment requires 4D basemap (time) to fully evaluate impacts.
    Hazard impact extents
    Real time updates from the field, often non-spatial (messages, spreadsheets)
    Intersect impact zones with infrastructure to determine severity
  • Our example demo scenario - flood risk assessment. Open in Workbench.

    Vancouver Flood Hazard Risk Assessment:
    Read infrastructure layers from geodatabase and assign criticality
    Load elevation model for Vancouver region
    Read live water level feed from nearest NOAA monitoring station
    Generate flood severity polygons with FloodAreaExtractor
    Assess flood impact on infrastructure
    Calculate HazardRisk = severity * criticality, and use to style output

    FloodRiskAreas:
    Subtract flood level from each cell
    Classify cells by flood severity
    Convert severity levels to areas


    Hazard impact extents
    Real time updates from the field, often non-spatial (messages, spreadsheets)
    Map extents in 4D (time)
    Intersect impact zones with infrastructure to determine severity
  • https://knowledge.safe.com/articles/44550/flood-notification-scenario.html

    https://environment.data.gov.uk/flood-monitoring/id/floods?min-severity=4
  • UAVs are a help for marine search and rescue because you can search large areas quickly and safely. Consider near-shore areas that are hard to access by land or sea – UAVs can fill this gap.
    RCMSAR used FME to build the route, i.e. generate search grid waypoints. Then UAV flies the grid (autonomously, to reduce human error and maximize coverage given limited flight time). Then they used FME to assemble the results into a photomosaic.
    Automation is key for flight planning and post-processing. Need to optimize the route, make sure you comply with regulations, process the data that’s gathered, build a catalogue or dashboard, distribute the data, generate flight statistics.
  • Telemetry based georeferencing Export to KML for Google Earth review
  • See Presentation by Renato, Steve, Dean at FME 2017 UC:
    https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/

    UAS at Southern Co:
    https://www.southerncompany.com/innovation/unmanned-aircraft-systems.html

    http://proceedings.esri.com/library/userconf/geoconx16/papers/geoconx_51.pdf

  • Another Key Automation Workflow: Resilience Scenario Analysis
    Modelling / simulation (FME supports NetCDF 4, multi-D raster)
    Scenarios stress / test response systems
    Requires automation to iterate through impact analysis
    Big users are analyzers of climate change and extreme weather impacts
  • Again: rasters are coming to us from satellites & UAVs in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows.
  • Image: California earthquake story
    Powered by FME Notification Services
    trigger workflows to run when new data becomes available
    serve up custom reports automatically by email
    FME cloud based
    Stephanie Halpin

    See FME UC 2017:
    UAV presentation - Southern Co. (Renato, Steve & Dean)
    https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/
    FME and Disaster Response
    https://www.safe.com/presentation/fme-for-disaster-response/

×