Advertisement
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left
The right side of speed - learning to shift left

Check these out next

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
smallerror
Testing the Data Warehouse—Big Data, Big Problems
Testing the Data Warehouse—Big Data, Big Problems
TechWell
How to design and implement a data ops architecture with sdc and gcp
How to design and implement a data ops architecture with sdc and gcp
Joseph Arriola
Stored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s Guide
VoltDB
Open Data Science Conference Big Data Infrastructure – Introduction to Hadoop...
Open Data Science Conference Big Data Infrastructure – Introduction to Hadoop...
DataKitchen
Future of Data Engineering
Future of Data Engineering
C4Media
Machine learning in production
Machine learning in production
Turi, Inc.
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Big Data Spain
1 of 44

The right side of speed - learning to shift left

Jan. 5, 2021••
Download to read offline
Software

Many disciplines are on the wrong side of speed - there is a tradeoff with development speed and security, data science, compliance, etc. Let us look at disciplines that have succeeded in shifting left by integrating development, and learn successful patterns: testing, DevOps, agile, DataOps.

Lars Albertsson
Founder & Data Engineer
Advertisement
Advertisement
Advertisement

Recommended

DataOps - Lean principles and lean practicesDataOps - Lean principles and lean practices
DataOps - Lean principles and lean practicesLars Albertsson772 views•29 slides
Data ops in practiceData ops in practice
Data ops in practiceLars Albertsson2.9K views•26 slides
Data ops in practice - Swedish styleData ops in practice - Swedish style
Data ops in practice - Swedish styleLars Albertsson398 views•59 slides
Data democratisedData democratised
Data democratisedLars Albertsson306 views•32 slides
Understanding DataOps and Its Impact on Application QualityUnderstanding DataOps and Its Impact on Application Quality
Understanding DataOps and Its Impact on Application QualityDevOps.com486 views•22 slides
Measuring Data Quality with DataOpsMeasuring Data Quality with DataOps
Measuring Data Quality with DataOpsSteven Ensslen490 views•30 slides

More Related Content

Slideshows for you(20)

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
smallerror•1.4K views
Testing the Data Warehouse—Big Data, Big ProblemsTesting the Data Warehouse—Big Data, Big Problems
Testing the Data Warehouse—Big Data, Big Problems
TechWell•1.8K views
How to design and implement a data ops architecture with sdc and gcpHow to design and implement a data ops architecture with sdc and gcp
How to design and implement a data ops architecture with sdc and gcp
Joseph Arriola•362 views
Stored Procedure Superpowers: A Developer’s GuideStored Procedure Superpowers: A Developer’s Guide
Stored Procedure Superpowers: A Developer’s Guide
VoltDB•453 views
Open Data Science Conference Big Data Infrastructure – Introduction to Hadoop...Open Data Science Conference Big Data Infrastructure – Introduction to Hadoop...
Open Data Science Conference Big Data Infrastructure – Introduction to Hadoop...
DataKitchen•2.1K views
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
C4Media•1.7K views
Machine learning in productionMachine learning in production
Machine learning in production
Turi, Inc.•7.8K views
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Stream Processing as Game Changer for Big Data and Internet of Things by Kai ...
Big Data Spain•1.1K views
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
The Enterprise and Connected Data, Trends in the Apache Hadoop Ecosystem by A...
Big Data Spain•495 views
Stream Scaling in PravegaStream Scaling in Pravega
Stream Scaling in Pravega
DataWorks Summit•638 views
What’s New with Databricks Machine LearningWhat’s New with Databricks Machine Learning
What’s New with Databricks Machine Learning
Databricks•370 views
Netflix Data Engineering @ Uber Engineering MeetupNetflix Data Engineering @ Uber Engineering Meetup
Netflix Data Engineering @ Uber Engineering Meetup
Blake Irvine•4.7K views
Gluent Extending Enterprise Applications with HadoopGluent Extending Enterprise Applications with Hadoop
Gluent Extending Enterprise Applications with Hadoop
gluent.•859 views
End to End Supply Chain Control TowerEnd to End Supply Chain Control Tower
End to End Supply Chain Control Tower
Databricks•610 views
Natalie Godec - AirFlow and GCP: tomorrow's health service data platformNatalie Godec - AirFlow and GCP: tomorrow's health service data platform
Natalie Godec - AirFlow and GCP: tomorrow's health service data platform
matteo mazzeri•285 views
Developing and Implementing a QA Plan During Your Legacy Data to S1000DDeveloping and Implementing a QA Plan During Your Legacy Data to S1000D
Developing and Implementing a QA Plan During Your Legacy Data to S1000D
dclsocialmedia•807 views
Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to...Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to...
Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to...
Big Data Spain•3.7K views
Scaling Databricks to Run Data and ML Workloads on Millions of VMsScaling Databricks to Run Data and ML Workloads on Millions of VMs
Scaling Databricks to Run Data and ML Workloads on Millions of VMs
Matei Zaharia•1.5K views
Challenges of Operationalising Data Science in ProductionChallenges of Operationalising Data Science in Production
Challenges of Operationalising Data Science in Production
iguazio•323 views
Data Science and Enterprise Engineering with Michael Finger and Chris RobisonData Science and Enterprise Engineering with Michael Finger and Chris Robison
Data Science and Enterprise Engineering with Michael Finger and Chris Robison
Databricks•479 views

Similar to The right side of speed - learning to shift left(20)

From the Drawing Board to the Trenches: Building a Production-ready ApplicationFrom the Drawing Board to the Trenches: Building a Production-ready Application
From the Drawing Board to the Trenches: Building a Production-ready Application
Hristo Iliev•107 views
CloudCamp Chicago - Big Data & Cloud May 2015 - All SlidesCloudCamp Chicago - Big Data & Cloud May 2015 - All Slides
CloudCamp Chicago - Big Data & Cloud May 2015 - All Slides
CloudCamp Chicago•1K views
Monitoring ICS CommunicationsMonitoring ICS Communications
Monitoring ICS Communications
Digital Bond•3.3K views
Optimizing Spark Deployments for Containers: Isolation, Safety, and Performan...Optimizing Spark Deployments for Containers: Isolation, Safety, and Performan...
Optimizing Spark Deployments for Containers: Isolation, Safety, and Performan...
Spark Summit•2K views
Serverless 101 in MontrealServerless 101 in Montreal
Serverless 101 in Montreal
Aaron Williams•82 views
Dylan Beattie "Architecture: The Stuff That's Hard to Change"Dylan Beattie "Architecture: The Stuff That's Hard to Change"
Dylan Beattie "Architecture: The Stuff That's Hard to Change"
Fwdays•235 views
Continuous (Non)-Functional Testing of Microservices on k8s Continuous (Non)-Functional Testing of Microservices on k8s
Continuous (Non)-Functional Testing of Microservices on k8s
QAware GmbH•463 views
2011-08-10 In-Q-Tel Technology Focus Day, Trends & Observations in Open Sourc...2011-08-10 In-Q-Tel Technology Focus Day, Trends & Observations in Open Sourc...
2011-08-10 In-Q-Tel Technology Focus Day, Trends & Observations in Open Sourc...
Shawn Wells•224 views
Glassbeam: Ad-hoc Analytics on Internet of Complex Things with Apache Cassand...Glassbeam: Ad-hoc Analytics on Internet of Complex Things with Apache Cassand...
Glassbeam: Ad-hoc Analytics on Internet of Complex Things with Apache Cassand...
DataStax Academy•1K views
2019 Performance Monitoring and Management Trends and Insights2019 Performance Monitoring and Management Trends and Insights
2019 Performance Monitoring and Management Trends and Insights
OpsRamp•1.6K views
Continuous delivery for databasesContinuous delivery for databases
Continuous delivery for databases
DevOpsGroup•5.3K views
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and LogstashKeeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Keeping Up with the ELK Stack: Elasticsearch, Kibana, Beats, and Logstash
Amazon Web Services•2.3K views
Cloud Computing: Powering the Future of Development and TestingCloud Computing: Powering the Future of Development and Testing
Cloud Computing: Powering the Future of Development and Testing
TechWell•588 views
Flight East 2018 Presentation–A DevOps State of Mind: Continuous Security wit...Flight East 2018 Presentation–A DevOps State of Mind: Continuous Security wit...
Flight East 2018 Presentation–A DevOps State of Mind: Continuous Security wit...
Synopsys Software Integrity Group•898 views
Building Microservices in the cloud at AutoScout24Building Microservices in the cloud at AutoScout24
Building Microservices in the cloud at AutoScout24
Christian Deger•1.2K views
Streaming Cyber Security into Graph: Accelerating Data into DataStax Graph an...Streaming Cyber Security into Graph: Accelerating Data into DataStax Graph an...
Streaming Cyber Security into Graph: Accelerating Data into DataStax Graph an...
Keith Kraus•1.1K views
Achieving Secure DevOps: Overcoming the Risks of Modern Service DeliveryAchieving Secure DevOps: Overcoming the Risks of Modern Service Delivery
Achieving Secure DevOps: Overcoming the Risks of Modern Service Delivery
Perforce•934 views
Cloud Security & Compliance - JPMC OverviewCloud Security & Compliance - JPMC Overview
Cloud Security & Compliance - JPMC Overview
▫️Canturk▫️ ▪️Isci▪️•24 views
Cloud Security & Compliance - JPMC OverviewCloud Security & Compliance - JPMC Overview
Cloud Security & Compliance - JPMC Overview
Canturk Isci•104 views
BGOUG "Agile Data: revolutionizing database cloning'BGOUG "Agile Data: revolutionizing database cloning'
BGOUG "Agile Data: revolutionizing database cloning'
Kyle Hailey•1.9K views
Advertisement

More from Lars Albertsson(20)

Data engineering in 10 years.pdfData engineering in 10 years.pdf
Data engineering in 10 years.pdf
Lars Albertsson•783 views
The 7 habits of data effective companies.pdfThe 7 habits of data effective companies.pdf
The 7 habits of data effective companies.pdf
Lars Albertsson•240 views
Holistic data application qualityHolistic data application quality
Holistic data application quality
Lars Albertsson•389 views
Secure software supply chain on a shoestring budgetSecure software supply chain on a shoestring budget
Secure software supply chain on a shoestring budget
Lars Albertsson•260 views
Ai legal and ethicsAi legal and ethics
Ai legal and ethics
Lars Albertsson•195 views
Mortal analytics - Covid-19 and the problem of data qualityMortal analytics - Covid-19 and the problem of data quality
Mortal analytics - Covid-19 and the problem of data quality
Lars Albertsson•415 views
The lean principles of data opsThe lean principles of data ops
The lean principles of data ops
Lars Albertsson•403 views
Engineering data qualityEngineering data quality
Engineering data quality
Lars Albertsson•1.2K views
Eventually, time will kill your data processingEventually, time will kill your data processing
Eventually, time will kill your data processing
Lars Albertsson•410 views
Taming the reproducibility crisisTaming the reproducibility crisis
Taming the reproducibility crisis
Lars Albertsson•520 views
Eventually, time will kill your data pipelineEventually, time will kill your data pipeline
Eventually, time will kill your data pipeline
Lars Albertsson•932 views
Kubernetes as data platformKubernetes as data platform
Kubernetes as data platform
Lars Albertsson•873 views
Don't build a data science teamDon't build a data science team
Don't build a data science team
Lars Albertsson•881 views
Big data == lean dataBig data == lean data
Big data == lean data
Lars Albertsson•223 views
Privacy by designPrivacy by design
Privacy by design
Lars Albertsson•1.9K views
Test strategies for data processing pipelines, v2.0Test strategies for data processing pipelines, v2.0
Test strategies for data processing pipelines, v2.0
Lars Albertsson•2.7K views
10 ways to stumble with big data10 ways to stumble with big data
10 ways to stumble with big data
Lars Albertsson•1.4K views
Protecting privacy in practiceProtecting privacy in practice
Protecting privacy in practice
Lars Albertsson•9.7K views
Testing data streaming applicationsTesting data streaming applications
Testing data streaming applications
Lars Albertsson•3.9K views
A primer on building real time data-driven productsA primer on building real time data-driven products
A primer on building real time data-driven products
Lars Albertsson•949 views

Recently uploaded(20)

Asdialer | How to Choose the Right Auto Dialer for Your NeedsAsdialer | How to Choose the Right Auto Dialer for Your Needs
Asdialer | How to Choose the Right Auto Dialer for Your Needs
Aresync•0 views
L2-3.FA19.pptL2-3.FA19.ppt
L2-3.FA19.ppt
UbaidURRahman78•0 views
pmk.docxpmk.docx
pmk.docx
fairy70•2 views
Trends & Innovation in Cyber and DigitaltechTrends & Innovation in Cyber and Digitaltech
Trends & Innovation in Cyber and Digitaltech
Liming Zhu•0 views
Micro Services.pdfMicro Services.pdf
Micro Services.pdf
ELHEITMohamedWassim•0 views
Software Architecture.pptSoftware Architecture.ppt
Software Architecture.ppt
MuhammadTalha416221•0 views
SE_RE-II-CH5 (3).pdfSE_RE-II-CH5 (3).pdf
SE_RE-II-CH5 (3).pdf
AZKANAAZ1•0 views
The AnomalyThe Anomaly
The Anomaly
megedwards5•2 views
Delphi Software Development | ChetuDelphi Software Development | Chetu
Delphi Software Development | Chetu
Chetu•6 views
CLOUD_COMPUTING_AWS_TRAINING.pptxCLOUD_COMPUTING_AWS_TRAINING.pptx
CLOUD_COMPUTING_AWS_TRAINING.pptx
MohammadSamiuddin10•0 views
Heartless Sky (Zodiac Academy, #7)Heartless Sky (Zodiac Academy, #7)
Heartless Sky (Zodiac Academy, #7)
megedwards5•3 views
It Consulting & Services - Black Basil TechnologiesIt Consulting & Services - Black Basil Technologies
It Consulting & Services - Black Basil Technologies
Black Basil Technologies•9 views
The Eye of the World (The Wheel of Time, #1)The Eye of the World (The Wheel of Time, #1)
The Eye of the World (The Wheel of Time, #1)
megedwards5•2 views
Automatic System for Detection and Classification of Brain TumorsAutomatic System for Detection and Classification of Brain Tumors
Automatic System for Detection and Classification of Brain Tumors
Fatma Sayed•2 views
Customer Engagement ProgramCustomer Engagement Program
Customer Engagement Program
Prasanna Hegde•0 views
Sakha - Corporate ProfileSakha - Corporate Profile
Sakha - Corporate Profile
Sakha Global•19 views
Firewalls (1).pptFirewalls (1).ppt
Firewalls (1).ppt
adnanetnzr•0 views
Some benefits of hiring Offshore Development Team.pdfSome benefits of hiring Offshore Development Team.pdf
Some benefits of hiring Offshore Development Team.pdf
Go-Tech Solution•0 views
Introduction to DatabasesIntroduction to Databases
Introduction to Databases
Muhammad Jahanzaib•2 views
Letters From Father Christmas, Centenary EditionLetters From Father Christmas, Centenary Edition
Letters From Father Christmas, Centenary Edition
megedwards5•2 views
Advertisement

The right side of speed - learning to shift left

  1. www.scling.com The right side of speed - learning to shift left Lars Albertsson, Scling 1
  2. www.scling.com Left is up 2 Winston W Royce: "Managing the development of large software systems"
  3. www.scling.com Speed vs X tradeoff? security compliance Speed or quality operations tech debt 3
  4. www.scling.com Enterprise culture - friction as a service analyst analyst architect architect developer developer risk manager vs risk manager security officer security officer tester tester ops engineer ops engineer 4
  5. www.scling.com Is there a tradeoff? Separation pitches teams and interests against each other 5
  6. www.scling.com Ghost of shifts past 70s - 90s: reign of waterfall 90s: flood of IT failures dot com crash 6
  7. www.scling.com Extreme programming 7
  8. www.scling.com A ladder to climb 8 Test certified program Google ~2006
  9. www.scling.com Agile 9
  10. www.scling.com Revenue models or role models 10Credits: @johncutlefish, @henrikkniberg
  11. www.scling.com DevOps 11
  12. www.scling.com No speed tradeoff! 12
  13. www.scling.com Collaboration is 13 people workflows coupling / decoupling not tools
  14. www.scling.com The most disruptive tech changes collaboration RDBMS version control GNU + Linux git + Github containers cloud Hadoop 14
  15. www.scling.com Big data 15
  16. www.scling.com Hadoop file storage batch processing single dimension database 16
  17. www.scling.com Big data workflows immutable datasets functional transform processing data pipelines homogeneous, coordinated workflows 17
  18. www.scling.com immutable datasets functional transform processing data pipelines homogeneous, coordinated workflows democratised data minimal operations minimal risk deployments reproducibility Big data workflows 18
  19. www.scling.com Big data workflows immutable datasets functional transform processing data pipelines homogeneous, coordinated workflows democratised data minimal operations minimal risk deployments reproducibility 19 Real value of big data. Not achieved by late adopters.
  20. www.scling.com Ghost of shifts present *Ops all the things 20
  21. www.scling.com DataOps 21
  22. www.scling.com Enterprise culture big data Hadoop / Spark / Flink + traditional workflows mutable data microservices functional teams heterogeneous data platform = worst of two worlds 22
  23. www.scling.com Enterprise culture DevOps Kubernetes / cloud + test in staging handover deployment no prod access (Dev)Ops team ... 23
  24. www.scling.com How to shift left common goals same definition of success incentives, not gateways common tools & environments common processes common teams 24
  25. www.scling.com Craft - improving artifacts 25AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
  26. www.scling.com Factory - improving processes 26 Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496 AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
  27. www.scling.com 1994: OS/2 Warp CID installation 27 Grmbl, who reinstalled my machine?
  28. www.scling.com IT craft to factory 28 Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496 AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG Security Waterfall Application delivery Traditional operations Traditional QA Infrastructure DevSecOps Agile Containers DevOps CI/CD Infrastructure as code
  29. www.scling.com Security Waterfall IT craft to factory 29 Application delivery Traditional operations Traditional QA Infrastructure DB-oriented architecture DevSecOps Agile Containers DevOps CI/CD Infrastructure as code Data factories, data pipelines, DataOps Lotus final assembly, Brian Snelson, https://www.flickr.com/photos/32659528@N00/2868525496 AMC Javelin, CZmarlin, https://commons.wikimedia.org/wiki/File:AMC_Javelin_1971-74_purple_blown_custom.JPG
  30. www.scling.com DevSecOps 30 SECURITY
  31. www.scling.com DevSecOps by induction validate security of system make secure easy validate security of changes ignore perfection - focus on delta 31
  32. www.scling.com Ghost of shifts yet to come machine learning security compliance are still at ends with speed 32
  33. www.scling.com MLOps 33 DATA SCIENCE
  34. www.scling.com Naive ML 34
  35. www.scling.com Towards sustainable production ML 35 Multiple models, parameters, features Assess ingress data quality Repair broken data from complementary source Choose model and parameters based on performance and input data Benchmark models Try multiple models, measure, A/B test
  36. www.scling.com ComplianceOps 36 COMPLIANCE
  37. www.scling.com ComplianceOps by induction validate compliance of system make compliant easy validate compliance of change ignore perfection - focus on delta 37
  38. www.scling.com long game very long ladders recipes people & teams craft to process focus on delta Doing the shift 38
  39. www.scling.com Doing the shift long game very long ladders recipes people & teams craft to process focus on delta learn faster by collaboration 39
  40. www.scling.com Data value = data + domain expertise + data practices 40 Disrupt? https://xkcd.com/1831/ Adapt? + 1000s of failures...
  41. www.scling.com Data value = data + domain expertise + data practices 41 Disrupt? https://xkcd.com/1831/ Adapt? + 1000s of failures...
  42. www.scling.com Data value = data + domain expertise + data practices 42 Data lake Stream storage Client data + domain expertise Practices from data leaders Disrupt? https://xkcd.com/1831/ Collaborate? Data-value-as-a-service Adapt? + 1000s of failures...
  43. www.scling.com The ghost of Winston W. Royce (1929-1995) 43
  44. www.scling.com Tech has massive impact on society 44 Product? Supplier? Employer? Make an active choice whether to have an impact! Cloud?
Advertisement