SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
Many disciplines are on the wrong side of speed - there is a tradeoff with development speed and security, data science, compliance, etc. Let us look at disciplines that have succeeded in shifting left by integrating development, and learn successful patterns: testing, DevOps, agile, DataOps.
Many disciplines are on the wrong side of speed - there is a tradeoff with development speed and security, data science, compliance, etc. Let us look at disciplines that have succeeded in shifting left by integrating development, and learn successful patterns: testing, DevOps, agile, DataOps.
17.
www.scling.com
Big data workflows
immutable datasets
functional transform processing
data pipelines
homogeneous, coordinated workflows
17
18.
www.scling.com
immutable datasets
functional transform processing
data pipelines
homogeneous, coordinated workflows
democratised data
minimal operations
minimal risk deployments
reproducibility
Big data workflows
18
19.
www.scling.com
Big data workflows
immutable datasets
functional transform processing
data pipelines
homogeneous, coordinated workflows
democratised data
minimal operations
minimal risk deployments
reproducibility
19
Real value of big data.
Not achieved by late adopters.
20.
www.scling.com
Ghost of shifts present
*Ops all the things
20
22.
www.scling.com
Enterprise culture big data
Hadoop / Spark / Flink
+
traditional workflows
mutable data
microservices
functional teams
heterogeneous data platform
=
worst of two worlds
22
23.
www.scling.com
Enterprise culture DevOps
Kubernetes / cloud
+
test in staging
handover deployment
no prod access
(Dev)Ops team
...
23
24.
www.scling.com
How to shift left
common goals
same definition of success
incentives, not gateways
common tools & environments
common processes
common teams
24
26.
www.scling.com
Factory - improving processes
26
Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496
AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
27.
www.scling.com
1994: OS/2 Warp CID installation
27
Grmbl, who
reinstalled my
machine?
28.
www.scling.com
IT craft to factory
28
Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496
AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
Security Waterfall
Application
delivery
Traditional
operations
Traditional
QA
Infrastructure
DevSecOps Agile
Containers
DevOps CI/CD
Infrastructure
as code
29.
www.scling.com
Security Waterfall
IT craft to factory
29
Application
delivery
Traditional
operations
Traditional
QA
Infrastructure
DB-oriented
architecture
DevSecOps Agile
Containers
DevOps CI/CD
Infrastructure
as code
Data factories,
data pipelines,
DataOps
Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496
AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
31.
www.scling.com
DevSecOps by induction
validate security of system
make secure easy
validate security of changes
ignore perfection - focus on delta
31
32.
www.scling.com
Ghost of shifts yet to come
machine learning
security
compliance
are still at ends with speed
32
35.
www.scling.com
Towards sustainable production ML
35
Multiple models,
parameters, features
Assess ingress data quality
Repair broken data from
complementary source
Choose model and parameters based
on performance and input data
Benchmark models
Try multiple models,
measure, A/B test
37.
www.scling.com
ComplianceOps by induction
validate compliance of system
make compliant easy
validate compliance of change
ignore perfection - focus on delta
37
38.
www.scling.com
long game
very long
ladders
recipes
people & teams
craft to process
focus on delta
Doing the shift
38
39.
www.scling.com
Doing the shift
long game
very long
ladders
recipes
people & teams
craft to process
focus on delta
learn faster by collaboration
39
40.
www.scling.com
Data value = data + domain expertise + data practices
40
Disrupt?
https://xkcd.com/1831/
Adapt?
+ 1000s of failures...
41.
www.scling.com
Data value = data + domain expertise + data practices
41
Disrupt?
https://xkcd.com/1831/
Adapt?
+ 1000s of failures...
42.
www.scling.com
Data value = data + domain expertise + data practices
42
Data lake
Stream storage
Client data +
domain expertise
Practices from
data leaders
Disrupt?
https://xkcd.com/1831/
Collaborate?
Data-value-as-a-service
Adapt?
+ 1000s of failures...
43.
www.scling.com
The ghost of Winston W. Royce (1929-1995)
43
44.
www.scling.com
Tech has massive impact on society
44
Product?
Supplier?
Employer?
Make an active
choice whether to
have an impact!
Cloud?
0 likes
Be the first to like this
Views
Total views
150
On SlideShare
0
From Embeds
0
Number of Embeds
0
You have now unlocked unlimited access to 20M+ documents!
Unlimited Reading
Learn faster and smarter from top experts
Unlimited Downloading
Download to take your learnings offline and on the go
You also get free access to Scribd!
Instant access to millions of ebooks, audiobooks, magazines, podcasts and more.
Read and listen offline with any device.
Free access to premium services like Tuneln, Mubi and more.