The document discusses using AWS for geospatial workloads and big data. It provides examples of customers like DELWP and Transport for NSW using AWS to analyze satellite imagery, store petabytes of data, and build analytics tools to gain insights. Architectures shown involve using Amazon S3 for storage, Amazon Athena for querying, Amazon SageMaker for machine learning, and serverless technologies. Real-time analytics of geospatial data from sensors is discussed along with delay pattern recognition and continually retraining models. Dashboards are also highlighted for providing operational insights.

1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Running Geospatial workloads
on AWS
Aileen Gemma Smith
CEO
Vizalytics
Herman Coomans
Senior Manager, Solutions Architecture
Amazon Web Services

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Finding Value in Data is a Journey
Business monitoring
Business insights
New opportunity
Business optimisation
Business transformation
Evolving Tools and Methods
AI/MLSQL Query

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Ibis Rookery, Victoria
Image source: Wikimedia Commons

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Customer Story: DELWP
Required analysis of water bodies in Victoria
Steps to complete:
• Gather satellite data (time required: 25 years)
• Store data up to Petabytes
• Build compute cluster
• Perform analytics

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Elevation
models
Aerial
imagery
Climate
models
Satellite
imagery
High-resolution
radar
aws.amazon.com/earth

7. Image:
Geosciences Australia

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Why Use AWS for Big Data?
Agility Scalability
Get to insights faster
Broadest and deepest
capabilities
Low cost
Data migrations made easy

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Back in 2015…
People expect access to EO data and
products
New EO satellite missions
Will require fundamental changes to
how EO data are analysed
Image:
Geosciences Australia

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The CSIRO/Geosciences Australia Solution
Amazon Simple
Storage Service (S3)
Image:
CSIRO

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Customer Story: DELWP
Required analysis of water bodies in Victoria over 25 years
Steps to complete:
• Gather satellite data (time required: 25 years)
• Store data up to Petabytes
• Build compute cluster
• Perform analytics

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Architecture
Geospatial Data Lake
Region
Amazon Simple Storage
Service (S3)
Consumer
Amazon AthenaAmazon Elasticsearch
Service
Amazon SageMaker
Notebook
Model
Amazon RDS
Postgres
Polygons

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Customer Story – Transport for NSW: Vizalytics
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.
Sydney's Central Station indicator board, 1906 2018, Museum of Applied Arts & Sciences, accessed 4 April 2019,
<https://ma.as/212227>

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
High level view of the customer problem
• Disparate data
• Too much data
• Inconsistent metadata
• Customer teams don’t have necessary skill sets
• Desire for rapid prototyping
• Want to move from being reactive to being proactive
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Keep creating more
and more data
Get insights about
your dataAND
Weather
Transit
Live Geo
Spatial
Internal Data
Open Data
Permits
Events
Semi Static
GIS Data
Contextual Graph Analytics
Smart Dashboards
Visualisations
Actionable Insight
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Geospatial Context – Over 6000km of Rail
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Geospatial Context – 100s of Stations
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Geospatial Context – 1000s of Fixed Track Sensors
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Geospatial Context – 100s of Moving Sensors (Trains)
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Building a Geospatial Data Lake
Points
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.
Context
Polygons
Lines

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Real Time Position Data
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Using the Data - Delay Pattern Recognition
What kind of delays are we classifying?
Why is it important to the user?
• Is it track-segment related?
• Is it equipment related?
• Is it likely to impact future trips on these tracks?
• Will there be knock on delays to other trips (on other tracks)?
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Serverless Processes – Training/Retraining
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Model Training
• Rolling window of scheduled vs. actual performance (e.g., last 12
months)
• Why continually retrain?
• Equipment and infrastructure changes and evolves
• Ridership is not constant
• Data sets evolve
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dashboard – Operationally relevant insight in action
Vizalytics Content © 2019, Vizalytics Technology, Inc. All rights reserved.
NSW TrainLink Performance Insight Dashboard – Data for illustrative
purposes only

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How to
copy your petabytes of geospatial
data down from the Cloud

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Share, don’t copy
Geospatial Data Lake
Region
Amazon
WorkSpaces
Amazon Simple Storage
Service (S3)
Customer 1 Customer 2 Customer 3 Customer 4 Customer 5
Amazon AthenaAmazon Elasticsearch
Service
Amazon EC2 AWS Lambda Amazon SageMaker
Users
Amazon EMR

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Characteristics of modern applications
Internet-scale and transactional
Users: 1M+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Mobile, IoT, devices
Scale: Up-out-in
Economics: Pay-as-you-go
Developer access: Instant API accessSocial mediaRide hailing Media streaming Dating

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Further Reading
Tap your badge after the presentation to have these links emailed to you
https://aws.amazon.com/big-data/
https://aws.amazon.com/earth/
https://aws.amazon.com/rds/
https://aws.amazon.com/S3/

36. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aileen Gemma Smith Herman Coomans