Processing Rasters from Satellites, Drones, & More
•Download as PPTX, PDF•
1 like•4,588 views
This document discusses how rasters from satellites, drones, and other sources can be processed and analyzed using FME. It provides examples of bringing raster data into workflows from various sources like Landsat 8, Sentinel 2, and Planet. It also discusses how FME can be used to automate raster processing, mosaicking, styling, combining, and outputting tiles. Examples are given for creating up-to-date hybrid basemaps and monitoring and analyzing wildfire changes daily. Additional examples discuss building georeferenced satellite videos and using FME for natural hazard workflows like flood risk assessment and search and rescue with UAVs. Automation and accessing data are emphasized as important for meeting challenges of natural disaster response.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Processing Rasters from Satellites, Drones, & More
Similar to Processing Rasters from Satellites, Drones, & More (20)
More from Safe Software
More from Safe Software (20)
Recently uploaded
Recently uploaded (20)
Processing Rasters from Satellites, Drones, & More
- 1. PROCESSING RASTERS FROM SATELLITES, DRONES, & MORE
- 2. Why do we care about rasters? Rasters are essential for background maps, spatial analytics, and visualizing data over time.
- 3. BRINGING RASTERS INTO YOUR WORKFLOWS ● Landsat 8 ● Sentinel 2 ● Planet ● Earth on AWS ● 54 new GDAL-based formats ● GDAL Generic Raster Reader ● GDAL VRT Raster Reader ● PDF Reader (also 2D & text data) Remote Sensing Sources New in FME 2018 ● Drones/UAVs ● Sensors Other Sources
- 5. Top Raster Transformers Anything is possible for rasters in FME! safe.com/transformers
- 7. first Get data from Planet and other sources Read imagery and vector layers. Process Mosaic rasters, style vector data, combine, tile. Output as .png Store raster tiles in the cloud. Make available everywhere to anyone. third Automate Do this whenever new images become available. last DEMO: UP-TO-DATE HYBRID BASEMAP second
- 9. HOW ABOUT MILLIONS OF TILES? WAIT A MINUTE...
- 10. first Get images Read Planet imagery of California wildfires. Process Store Save to local or cloud storage. third Analyze Make tools for visual and analytical change comparison. last DEMO: MONITOR AND ANALYZE CHANGE DAILY Derive new products like NIR, NDVI, NDWI. second
- 11. Check it live
- 12. first Get images from Planet Collect images for a selected timeframe. Process Clip to extents of desired area. Add video to map Prepare HTML output with LeafletJS. MAKE A GEOREFERENCED SATELLITE VIDEO third lastsecond Output as .mp4 Integrate with FFmpeg and output a video.
- 14. Scenario: NATURAL HAZARD WORKFLOWS Photo by C. Hitchcock
- 15. NATURAL HAZARDS: CHALLENGES ● Diverse data types needed. ○ Vector and raster ○ Open and proprietary ○ Spatial and non-spatial ● Limited IT infrastructure. Need to make data accessible. ● Urgency! Disasters mean short timelines. ● Need to automate processing and distribution.
- 16. KEY WORKFLOWS ● Impact assessment. ○ population and transportation ○ identify critical infrastructure ○ hazard sources and extents ● Data integration. ● Risk, probability, scenarios, time. ● UAV surveys, automation.
- 17. DEMO Vancouver Flood Hazard Risk Assessment
- 18. Bonus: Flood Notifications FME Knowledge Center Tutorial: http://fme.ly/flood
- 19. EXAMPLE: SEARCH AND RESCUE UAVs 1. Build route for UAV. 2. UAV flies autonomously and collects data. 3. Post-process – geolocate, assemble results into photomosaic.
- 20. Automated route generation SEARCH GRID GENERATION Automated image geolocation
- 21. UAVs FOR DAMAGE ASSESSMENT ● UAVs are easy and safe to deploy. ● Rapidly assess impact to people and infrastructure. ● Data guides the level of response. Story: Renato Salvaleon develops UAV systems at Southern Co, supported by FME Server automation. safe.com/uav UAV storm damage assessment by Southern Co.
- 22. MEETING NATURAL HAZARD CHALLENGES ● Diverse data types needed. ○ Vector and raster ○ Open and proprietary ○ Spatial and non-spatial ● Limited IT infrastructure. Need to make data accessible. ● Urgency! Disasters mean short timelines. ● Need to automate data processing and distribution. Rapid prototyping
- 24. RESOURCES ● San Francisco Transit demo: http://fme.ly/sftransit ● Blog - earthquake notifications: http://fme.ly/earthquake ● Christchurch earthquake story: http://fme.ly/christchurch ● Flood notification tutorial: http://fme.ly/flood ● Risk analyzer by con terra: georiskanalyzer.com ● Blog - get started with drones: http://fme.ly/drones ● FME Server for Fort McMurray wildfire: http://fme.ly/atco ● Search “remote sensing” or “UAV” on safe.com
- 25. Q&A
Editor's Notes
- We don’t care about rasters; we care about rasters mean - is it a picture, elevation model, etc. A raster is just a means to achieving the solution we want - we care about the environment or the city or whatever else it represents.
- Rasters are coming to us in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows. Planet is a huge archive - billions of images - humans can’t manage all of this manually. We need automation.
- Some useful ones worth mentioning. FeatureReader is not a raster transformer, but we want to mention it here because it plays an important role when reading satellite data. Instead of a traditional approach - read data, then figure out what to keep for further processing, we first figure out what we need (coverage, weather, quality, date range), and then read it using all those pre-processed values as parameters. MapnikRasterizer - for making a really nice output - maps or anything (chocolate packages) FeatureReader - let features drive what data you get from Planet. e.g. GIS boundaries RasterMosaicker - combine all the images we get MapnikRasterizer - we have lots of mapnik resources & webinars NDVI - the key is to say it’s a custom transformer - can add functionality that’s missing
- http://fme.ly/sftransit or http://demos.fmeserver.com/planet-satellite-map/
- server checks regularly for new images that meet criteria
- https://blog.safe.com/2018/01/elastic-processing-cloud-docker-fme-server/ Generating millions of tiles normally takes a full day… we were able to do it in 25 mins.
- http://demos.fmeserver.com/farmland-monitoring-planet/
- https://s3-us-west-2.amazonaws.com/safe-scenarios/Planet/video/20170907234358091020257.htm
- We’re talking about assessing risks and damage from floods, earthquakes, forest fires, climate change, hurricanes, and more. Image 1: Alpine Shire Council wanted to improve bushfire risk assessments. With FME, they designed an app that gives real-time analysis of risk. (If you search “alpine shire council” on safe.com you can find this story.) Image 2: I85 Atlanta Bridge Collapse: a guy lit a grocery cart on fire and the fire grew and ultimately collapsed the bridge. This was a Southern Co project to assess the damage to infrastructure. Image 3: was an FME UC presentation - “Helping rebuild a city with FME” - after the Christchurch earthquake __ Other examples: Critical infrastructure protection: FortisBC Humanitarian and disaster response: Hurricane, Syria, Myanmar (OGC Testbed13) UAVs allow for rapid assessment and response: UAV damage assessment and cell network support in Puerto Rico
- Image of SFO flooding scenario from OGC TestBed 11. FME was used to read NetCDF data from a high resolution flood forecast model and convert this into a KML time series of raster images for display in Google Earth. Several key points from this slide are based on findings from the following study: https://unstats.un.org/unsd/geoinfo/RCC/docs/rccap20/16_Robert%20Deakin_New%20Zealand%20-%20Canterbury%20SDI%20Paper.pdf Twentieth United Nations Regional Cartographic Conference for Asia and the Pacific Jeju, 6 - 9 October 2015 Item 7(b) of the provisional agenda Invited Papers Canterbury SDI: lessons learned from post-earthquake recovery The root causes underlying these issues were identified as: The lack of data sharing agreements in place; No data sharing channels (such as web services), or standardisation of formats and data models No catalogue or registry of available data sources A lack of training or practice on how to pull together data sources Problems manifested as: • Inability to find key people who could supply data; • Inability to access the data; • Reluctance to supply data because of perceived poor quality; • Reluctance to supply data because the requestor was perceived as not needing it; • Reluctance to supply data because it was not known how the requestor might use it; • Privacy issues; • Supplying spatial data in unhelpful formats. Problems related to integrating data from multiple agencies included: • No plans on how to integrate data representing aspects such as people, places, things, events and concepts, from multiple agencies; • No plans on how to integrate data from a single agency into a recipient’s business system; • Data-models were different and the attributes contained different data. Schemas were not available; • Abstractions were different (e.g. mobile toilet delivery being recorded as a number against a street name by some people, and against an address by others); Geometry data types were different (whether the feature was stored as a point, a line, or a polygon); • Datasets were at different levels of completion; • Data was in different formats, e.g. WFS, ArcGIS, MS Access, Excel, CSV files, PDFs, paper forms and maps. Getting expertise was also a problem: • People were thrust into positions they were not equipped to deal with; • Information was captured by people without experience of spatial data; • Data was managed by people without experience of spatial data; • No access to technical expertise; • Out-of-town resources were unfamiliar with the city (e.g. common road names and place names); • Out-of-town resources were unfamiliar with the systems they were working in. __ Opportunities with FME Common data models, spatial reference Open standards Cloud based, decentralized Mobile devices Crowd sourcing APIs, loosely coupled Web sources – leave in place Real time, non-spatial Model based, rapid prototyping Easy updates FME Capabilities Formats R&W Web Access, Publish Processing Geometry: Vector, raster, point clouds Processing records: ETL, schema mapping https://www.safe.com/solutions/for-industries/emergency-services-and-policing/ Increase Disaster Response Efficiency Build Situational AwarenessSwiftly gather, combine, and analyse tabular and location-aware information from 350+ sources, providing facts for critical decision making in real-time.Leverage Citizen Engagement Harvest intelligence from citizens via social media, including geotagged photos, and return life-saving information back to the public.Hands-Free Data Flow Reduce manual efforts during emergencies by creating automated workflows that leverage complex event processing (CEP) technology to immediately deliver critical intelligence using real-time notifications.
- Integrating various sources is key. Impact assessment requires 4D basemap (time) to fully evaluate impacts. Hazard impact extents Real time updates from the field, often non-spatial (messages, spreadsheets) Intersect impact zones with infrastructure to determine severity
- Our example demo scenario - flood risk assessment. Open in Workbench. Vancouver Flood Hazard Risk Assessment: Read infrastructure layers from geodatabase and assign criticality Load elevation model for Vancouver region Read live water level feed from nearest NOAA monitoring station Generate flood severity polygons with FloodAreaExtractor Assess flood impact on infrastructure Calculate HazardRisk = severity * criticality, and use to style output FloodRiskAreas: Subtract flood level from each cell Classify cells by flood severity Convert severity levels to areas Hazard impact extents Real time updates from the field, often non-spatial (messages, spreadsheets) Map extents in 4D (time) Intersect impact zones with infrastructure to determine severity
- https://knowledge.safe.com/articles/44550/flood-notification-scenario.html https://environment.data.gov.uk/flood-monitoring/id/floods?min-severity=4
- UAVs are a help for marine search and rescue because you can search large areas quickly and safely. Consider near-shore areas that are hard to access by land or sea – UAVs can fill this gap. RCMSAR used FME to build the route, i.e. generate search grid waypoints. Then UAV flies the grid (autonomously, to reduce human error and maximize coverage given limited flight time). Then they used FME to assemble the results into a photomosaic. Automation is key for flight planning and post-processing. Need to optimize the route, make sure you comply with regulations, process the data that’s gathered, build a catalogue or dashboard, distribute the data, generate flight statistics.
- Telemetry based georeferencing Export to KML for Google Earth review
- See Presentation by Renato, Steve, Dean at FME 2017 UC: https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/ UAS at Southern Co: https://www.southerncompany.com/innovation/unmanned-aircraft-systems.html http://proceedings.esri.com/library/userconf/geoconx16/papers/geoconx_51.pdf
- Another Key Automation Workflow: Resilience Scenario Analysis Modelling / simulation (FME supports NetCDF 4, multi-D raster) Scenarios stress / test response systems Requires automation to iterate through impact analysis Big users are analyzers of climate change and extreme weather impacts
- Again: rasters are coming to us from satellites & UAVs in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows.
- Image: California earthquake story Powered by FME Notification Services trigger workflows to run when new data becomes available serve up custom reports automatically by email FME cloud based Stephanie Halpin See FME UC 2017: UAV presentation - Southern Co. (Renato, Steve & Dean) https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/ FME and Disaster Response https://www.safe.com/presentation/fme-for-disaster-response/