In the last 10 years, Department of Environment, Land, Water and Planning (DELWP) has procured half a petabyte of aerial photography, satellite imagery, and point cloud data worth more than $45million. In addition, a range of faster products critical to the delivery of services are produced continuously. DELWP’s faster data needs is expected to grow overtime and an enterprise system to manage, publish and enable discovery and analysis of this growing catalogue is essential. Find out how DELWP is implementing pioneering data science strategies to successfully integrate disparate systems and data silos, reduce risks and increase compliance, and support a rapid pace of innovation to improve DELWP’s services to the Victorian public. Project Manager with DELWP’s Digital First program, Alena Moison will share the department’s journey through digital transformation, key successes and learning points, as well as how they worked with Bulletproof to implement solutions on AWS. Speakers: Giles Bill, Bulletproof Networks Sam Mason, Bulletproof Networks Alena Moison, Department of Environment, Land, Water and Planning

2. Department of Environment, Land, Water,
and Planning’s (DELWP) Data Lake:
Investing in Asset Wealth for Public/Community Benefit
Alena Moison, DELWP
Sam Mason, Bulletproof
Giles Bill, Bulletproof

3. We are the leading
cloud
services
provider in A/NZ

4. 17+
years (2000)
ASX
listed (2014)
120+
employees
$47.2m
FY16 revenue
700+
customers
5
Locations in
A/NZ & US

5. Cloud Empowered Digital Transformation
Bulletproof

7. What can you expect today?
An Overview of :
• DELWP’s challenges and risks
• What is a Data Lake?
• Bulletproof’s Data Lake solution
• Solution architecture overview
• Next steps

8. Digital First Initiative /
DELWP’s Digital First Vision
To become a high
performing organisation
known for service
excellence, innovation and
putting the customer at the
centre by leveraging digital.
Alena Moison
Spatial Data Operations

9. Digital First Initiative /
What we do
DELWP’s Data Lake 10
We are focused on creating a liveable, inclusive and sustainable Victoria
with thriving natural environments –
where the community is at the centre of everything we do.
Climate Change
Forest Fire
Management
Victoria
Planning
Wildlife
Parks and forest
Coasts and marine
Property and land
titles
Water and
catchments
Energy
Local Government
Environment
Heritage

10. Digital First Initiative /
DELWP’s Digital Reality
DELWP’s Data Lake 11
0.5 petabytes
Aerial & satellite imagery
& point cloud data
>$45m worth of
Geospatial data assets
10’s of thousands
of raster models
Data Growth of +3000%
over the last 10 years

11. Digital First Initiative /Digital First Initiative /
DELWP’s Challenges
DELWP’s Data Lake 12
DELWP’s current raster data services are:
• Inefficient
• Inadequate
• Disjointed

12. Digital First Initiative /Digital First Initiative /
Risks for DELWP
DELWP’s Data Lake 13
• High and increasing costs
• Reliance on key personnel
• Data security & DR
• Inability to scale
• Internal silos

13. What is a Data Lake?

14. Data is a problem…

15. Data is a problem…
90%
Less than 2 years old

16. Data is a problem…
90%
Less than 2 years old
90%
Unstructured/Semi-structured

17. Data is a problem…
90%
Less than 2 years old
90%
Unstructured/Semi-structured
33% of business leaders don’t trust it anyway!

18. The Data Lake
An Enterprise Data Repository
• Store ALL data and store it raw
• Low-cost, scalable, secure & durable medium
• Storage & Compute independent
• Centralised security

19. Leverage your existing investment in data
• Catalogue (Search & Discovery, Publish & Share)
• Semantics delayed until use
• Query datasets in-place (Athena/Presto etc.)
• Integrate new data sources easily

20. • Invest in enterprise data wealth
• Dissolve corporate data silos
• Lower TCO across enterprise
Data Lake benefits

21. Data Lake benefits
• Encourages experimentation
• Reduces risk
• Reduced Time-to-Insight

22. What Bulletproof Implemented for DELWP
AWS Data Lake/Science Project details

23. Service Architecture

24. Component Architecture
Example Bundle Manifest (truncated)
{
"publicationStatus":"draft",
"sizeInBytes":2427834,
"assetEnvelopeUpperCornerLong":"142.06779353677902",
"assetEnvelopeLowerCornerLat":"-38.019847309483744",
"assetCoordinateSystem":"EPSG:WGS 84",
"platform":"air",
"publiclyAvailable":false,
"storageClass":"standard",
"resolutionUnit":"cm",
"fingerprint":"108916f8d5a4f4f6fec58ce1a53d4ce2",
"coordinateSystem":"mga54",
"contentType":null,
"resolutionMagnitude":20,
"timestamp":"2015-05-01T00:00:00.000+10:00",
"assetEnvelopeLowerCornerLong":"142.06574467631495",
"format":"tif",
"version":null,
"searchable":true,
"sensorBandNumberOrScale":null,
"entries":[
"s3://bp-delwp-qa-dev/landing/cobboboonee-wrights-
swamp_2015may01_air_cir_20cm_mga54.tif"
],
"deleted":false,
"name":"cobboboonee-wrights-swamp_2015may01_air_cir_20cm_mga54",
"location":"cobboboonee-wrights-swamp",
"sensor":"cir",
………………
}

25. Ingestion flow

26. Consumption flow

27. Digital First Initiative /
Key Learnings
DELWP’s Data Lake 28
• Legacy environment & legacy thinking: bring people
along on the journey
• Take a proof of concept/pilot approach
• Value of bringing in the right expertise to cut through
the noise

28. Digital First Initiative /
Benefits
Infinite, secure
storage
Uplift digital
capability
Discover
data easily
Increase
collaboration
Improved staff
satisfaction
Access data
faster
Support
communities
Scale & flex
DELWP’s Data Lake

29. Digital First Initiative /Digital First Initiative /
Next Steps for DELWP
DELWP’s Data Lake 30
• Begin to measure and provide insights into the value of the data
• Seek minimal, incremental changes to user experience
• Engage more participants in the Platform
• Take an Agile & DevOps approach
• Product Backlog determines next development and release
iteration.

30. DELWP’s Data Lake
What’s next?

31. Data Sources:
• Ingestion and storage
across disparate data
sources
Processing:
• Leverage on demand HPC clusters for
processing jobs
Rules:
• Additional ingestion sources
• Configurable rules based processing
• Incorporate external systems into workflow
processes
• Intelligent bundle storage management to
optmise costs
External Parties:
• Potential Monetisation of data through API
• Discoverable and versioned APIs
• Portal integration
External Parties:
• Potential Monetisation of data through API
• Discoverable and versioned APIs
• Portal integration
What’s next?

32. How Bulletproof can help
Our offerings

33. Cloud-Empowered Digital Transformation

34. If you want to find out more
• Visit us at stand: G13
• Contact us to discuss your requirements
contactus@bulletproof.net | 1300 663 903
• Enter our draw to win a Parrot Drone 2.0
• AWS will do a talk about Data Lake at 2.45pm straight after this
session

35. Thank you!