A talk by Rob Emanuele given at FedGeoDay 2016 about using GeoMesa, GeoWave, and GeoTrellis to work with geospatial data on Apache Spark and Accumulo.

1. APPLICATIONS WITH
GEOSPATIALLY ENABLE YOUR
Rob Emanuele @lossyrob @

2. APPLICATIONS WITH
GEOSPATIALLY ENABLE YOUR
Rob Emanuele @lossyrob @

3. What we’ll be covering…
What does “processing geospatial data at scale”
mean?
What are Accumulo and Spark?
What is LocationTech?
How GeoMesa, GeoWave, and GeoTrellis can
GeoSpatially enable your projects.

4. PROCESSING GEOSPATIAL DATA
@ SCALE

5. PROCESSING GEOSPATIAL DATA
@ SCALE

6. • 170 X 180 km
• 2gb each.
• 11 bands
• 700 scenes per day
• 1.4 TB / day
• 255,500 scenes / year
• 0.25 PB / year
Landsat 8

7. Landsat 8 on
• All Landsat 8 scenes from 2015 and beyond.
• Selection of cloud-free scenes from 2013 and 2014.

8. Landsat 8 on
645,763 scenes

9. Landsat 8 on
≈1 Petabyte

11. 64 GB

12. 32 Landsat 8 Scenes

13. This many people’s phones could hold all the Landsat 8 AWS is
holding.

14. PROCESSING GEOSPATIAL DATA
@ SCALE

15. PROCESSING GEOSPATIAL DATA
@ SCALE

16. PROCESSING GEOSPATIAL DATA
@ SCALE

20. Apache Spark
a distributed computation engine.
An API that lets you work with distributed data
as a collection.
Written in Scala, with language bindings for use
with Java, Python, and R.

21. Hey Flyers Fans, what is the total count of
Landsat 8 Scenes on your phones?

23. Data
Node
Data
Node
Data
Node
Name
Node
Master
Tablet
Server
Tablet
Server
Tablet
Server
Accumulo
BigTable clone (columnar database)
Records stored on HDFS
Lexicographically sorted table index

25. Hey Flyers Fans, what is the total count of
Landsat 8 Scenes on your phones per month?

27. PROCESSING GEOSPATIAL DATA
@ SCALE

28. PROCESSING GEOSPATIAL DATA
@ SCALE

32. WHAT IS ?

36. SPACE FILLING CURVES

37. Hey Flyers Fans, what is the total count of
Landsat 8 Scenes on your phones per month, per country?

38. Hey Flyers Fans, what is the total count of
Landsat 8 Scenes on your phones per country?

39. SPACE FILLING CURVES

40. Z curve

41. Hilbert Curve

43. Hey Flyers Fans, what is the total count of
Landsat 8 Scenes on your phones A) per month, B) per country,
C) per both?

45. Geo +
accessed through

48. SELECT
tweet.text,
user.name
FROM
tweet, user
WHERE
bbox(tweet.location, -115, 45, -110, 50) AND
tweet.user_id = user.user_id
+

51. Geo +
accessed through
GEOWAVE

55. Geo +
Geo (mostly rasters) +

56. 100 spot instance m3.xlarge workers @ $0.04 /
hr = $4.00 / hr
400 CPUs / ≈1.5 TB memory
1 master m3.xlarge on-demand instance @
$0.26 / hr
EMR cluster charge, $0.07 / hr
$4.37 / hr
Rendering elevation
with hillshade + NLCD on AWS EMR

59. NED 1/3 arc second + NLCD

60. NED 1/3 arc second + NLCD

61. NED 1/3 arc second + NLCD

62. Hey Flyers Fans, can you take the average pixel value of each scene’s
band and derive a EPSG:3857 tile set of PNGs to be served on web
maps?

64. Collaboration

65. Current Collaboration

66. Current Collaboration

67. Future Collaboration

68. Future Collaboration

69. • Come see us at our booth
• Join the locationtech-iwg mailing list
• Share you big geospatial data challenges
Get involved!

70. THANKYOU
@lossyrob
gitter.im/geotrellis/geotrellis
github.com/geotrellis/geotrellis
remanuele@azavea.com