Introducting RasterFrames
•Download as PPTX, PDF•
1 like•205 views
The underlying purpose of RasterFrames™ is to allow data scientists and software developers to process and analyze geospatial-temporal raster data with the same flexibility and ease as any other Spark Catalyst data type. At its core is a user-defined type (UDT) called TileUDT, which encodes a GeoTrellis Tile in a form the Spark Catalyst engine can process. Furthermore, we extend the definition of a DataFrame to encompass some additional invariants, allowing for geospatial operations within and between RasterFrames to occur, while still maintaining necessary geo-referencing constructs.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Introducting RasterFrames
Similar to Introducting RasterFrames (20)
Recently uploaded
Recently uploaded (20)
Introducting RasterFrames
- 1. Introducing GIS Raster Support for Spark DataFrames Simeon H.K. Fitch Co-Founder & VP of R&D, Astraea, Inc. ™ ™
- 2. Astræa • With exploding population growth and finite resources, we need to have tools to better plan for sustainable growth • We empower individuals to ask complex questions of the world 2 See the earth. As it was, as it is, as it could be.℠
- 3. Astræa Earth Engine Analytics-Ready Earth-Observing Imagery Open-Source Foundation Scalable Core Built on RasterFrames Business-Centric Model Deployment
- 4. • Provides ability to work with global-scale remote sensing imagery in a convenient and familiar format • Simplifies processing and analyzing of terabyte raster datasets • Delivers HPC, advanced ML/AI, and remote sensing data to users • Open Source; aspiring LocationTech project
- 5. RasterFrames 5 GeoTrellis Layers Map Algebra Layer Operations Statistical Analysis Ti l eLayer RDD Machine Learning Visualization Your Application GeoTIFF RasterFrame Spark Dat aSour ce Spark Dat aFr ame join
- 7. Get Involved! • Check out the documentation – Quick start – More examples • Join the community and help out • Join conversation on Gitter • Provide feedback http://rasterframes.io/
- 8. Coming Soon… • Python support • Spatial joins between arbitrary RasterFrames • Vector-based filtering (GeoMesa?) • Additional DataSources (PostGIS rasters?) • Performance optimizations • More documentation and examples
- 9. Thank You! http://rasterframes.io/ github: https://github.com/s22s/raster-frames gitter: https://gitter.im/s22s/raster-frames Simeon H.K. Fitch email: fitch@astraea.earth github: metasim twitter: @metasim
- 10. APPENDIX
- 11. Quick Start 1. Install sbt: http://www.scala-sbt.org/ 2. > sbt new s22s/raster-frames.g8 3. (Accept defaults) 4. > cd my-rasterframes-project 5. > sbt run
- 12. Why This is Hard: Dimensionality 12 Spatial (500m → 5m → 30cm) Temporal (Refresh rates: Weeks → Daily → Hourly) Spectral (4 bands → 200 bands) Planet DigiGlobe Landsat8 Planetary Resources Metadata • Coordinate Reference System • Temporal/Spatial Extent • QA Flags • Calibration parameters +
- 13. Why This is Hard: Data Footprint 13 As resolution scales, image size explodes Data footprint for one football field size multiband raster (single point in time!) • 30 meters • 8 band • 0.5 GB/image Landsat8 (NASA) • 3 meters • 4 band • 16 GB/image Planet PlanetScope Ortho • 30 centimeters • 4 band • 1.0 TB/image DigiGlobe • 10 m Resolution • 200 band (hyper-spectral) • 50 TB/ image? Planetary Resources
Editor's Notes
- A little about who we are and what we’re up to My role at Astræa is to apply the art and discipline of software engineering to make data scientists efficient and effective in solving these problems
- Moving beyond maps
- To effectively and efficiently deliver the power of high-performance computing, advanced machine learning, and remote sensing to our users RasterFrames provides the ability to work with global EO data in a data frame format, familiar to most data scientists
- Users, users, users Let us help you get started
- \ ]”\] [p90-=1`˘¿l;?