Chris Reddington
Cloud Solution Architect at Microsoft UK
Blogger, Podcaster & Vlogger @ www.cloudwithchris.com
Consider a follow or subscribe!
Twitter: https://twitter.com/reddobowen
YouTube: https://youtube.com/c/CloudWithChris
GitHub: https://github.com/chrisreddington
Steph Locke
Cloud Solution Architect Manager at Microsoft UK
Ex-Data&AI MVP
Blog: https://stephlocke.info
Twitter: https://twitter.com/theStephLocke
LinkedIn: https://linkedin.com/in/StephanieLocke
GitHub: https://github.com/stephlocke
©Microsoft Corporation
Azure
azure.com/developervelocity
©Microsoft Corporation
Azure
Key findings
Top drivers for business performance: developer tools, product
management capabilities, culture, and talent management
Open source is the #1 differentiator for top performers
Accelerating impact: AI/ML, Live site automation, product
telemetry
Public cloud adoption is a key differentiatordriving business
performance for non-digital companies
Top performers: empowering citizen developers score 33% higher
on innovation
</>
©Microsoft Corporation
Azure
©Microsoft Corporation
Azure
©Microsoft Corporation
Azure
©Microsoft Corporation
Azure
©Microsoft Corporation
Azure
An evolving journey of Developer Velocity
Buildingthe
Foundation
Transforming
the Future
Accelerating the
Impact
Talent management
Product management
capabilities
Automated live site
management
Public cloud adoption
Effective developer toolchain
Open source adoption and
capabilities
AI / ML
Culture of psychological safety
and ‘fail fast & learn’
Empowering citizen developers
Usage of granular product
telemetry
</>
AI
65%
47%
#1 differentiator
30%
20%
4x higher 33%
</>
AI
What is DevOps?
Development + operations
DevOps is the union of people, processes,
and technologies to deliver continuous
value to users.
Here’s how Microsoft builds
apps of the future
Adopting a
growth mindset
Creating a culture of
customer obsession
Aligning to shared
objectives
DevOps across Microsoft
http://aka.ms/DevOps-Stories
105K
Engineers use the
DevOps platform
2M
Git commits
per month
4.4M
Builds per month
500M
Test executions
per day
5M
Work items
viewed per day
500K
Work items
updated per day
85,000
Deployments per day
Internal Microsoft Engineering System Activity, July 2019
Create clarity
Measure outcomes
not outputs
Objectives and key results (OKRs)
Example objective: Grow a strong and
happy customer base
1. Increase external NPS from 21 to 35
2. Increase docs SAT from 55 to 65
3. New pipeline flow has an Apdex score of 0.9
4. Queue time for jobs is 5 seconds or less
KRs are measures for the quarter.
Best KRs are leading indicators.
Encourage ambitious KRs: 70% of the
improvement target scores green.
Create clarity
Change what you track
Focus on measuring only the most critical and impactful KPIs:
Customer usage
How much value are
users getting?
Acquisition
Retention
Engagement
Satisfaction (NPS)
Feature usage
Pipeline
throughput
How efficient is the
DevOps process?
Time to build
Time to test
Time to deploy
Time to improve
Failed and flaky automation
Live-site health
How quickly can you
detect and fix issues?
Time to detect, time to
communicate, time to mitigate
Customer impact, customer
support metrics
Incident prevention items
Aging live-site problems
SLA per customer
Be customer obsessed
Ensure products are live
in production and
collecting feedback
Collect telemetry data that examines the
hypothesis that motivated the deployment.
Gather information and making incremental changes is
key to improvement.
Treating the goal as a question rather than a statement of
fact motivates developers to continue testing their ideas.
Evolve to full stack teams
Evolve the organization (1 of 3)
Program
management
Development Testing
Evolve to full stack teams
Evolve the organization (2 of 3)
Program
management
Engineering Ops/SRE
Evolve to full stack teams
Evolve the organization (3 of 3)
Feature team
Our journey continues…
©Microsoft Corporation
Azure
Change how you collaborate, develop, and deliver
Collaborate on code
Leverage innersource practices to build better
products and create a culture of openness by:
Building upon open-source projects that use global contributors
Customizing, adapting, and reusing existing internal code
Collaborating with developers across your organization
Appointing a smaller team to make final decisions
Leveraging open-
source software and
innersource practices
enables your team to
innovate and deliver
value faster.
Change how you collaborate, develop, and deliver
Drive toward automation
Long running
functional tests
Shifted to unit tests from
automated functional tests
Count
of
tests
M62 M63 M64 M65 M66 M67 M68 M69 M70 M71 M72 M73 M74 M75 M76 M77 M78 M79 M80 M81 M82 M83 M84 M85 M86 M87 M88 M89 M90 M91 M92 M93 M94 M95 M96 M97 M98 M99
Change how you collaborate, develop, and deliver
Utilize pull requests
Control changes
Pull requests are the first gate to
control how changes get merged to
the master.
Limit human error and ensure only
high-quality code gets merged with
detailed permissions, policies, and
status checks.
Enhance security from dev to production
Secure the software supply chain
Apply policies
and controls
proactively
Rely on security
automation
Build security checks
into your CI/CD
pipelines
Strengthen your
security posture
Enhance security from dev to production
Build security practices into your delivery
Key security practices:
Pre-commit
• Threatmodeling
• IDE securityplug-in
• Pre-commithooks
• Secure coding
standards
• Peer review
Commit (CI)
• Static code analysis
• Securityunit tests
• Dependency
management
Deploy (CD)
• Infra as code (IaC)
• Securityscanning
• Cloud configuration
• Securityacceptance
tests
Analyze
• Security
Smoke tests
• Configuration
checks
• Penetration tests
Operate
• Continuous
monitoring
• Threatintelligence
• Penetration testing
• Blameless
postmortems
An evolving journey of Developer Velocity
Buildingthe
Foundation
Transforming
the Future
Accelerating the
Impact
Talent management
Product management
capabilities
Automated live site
management
Public cloud adoption
Effective developer toolchain
Open source adoption and
capabilities
AI / ML
Culture of psychological safety
and ‘fail fast & learn’
Empowering citizen developers
Usage of granular product
telemetry
</>
AI
Thank You!
Questions?
https://aka.ms/developervelocity-report
https://azure.com/developervelocity
http://aka.ms/developervelocity-pbi
https://aka.ms/developervelocity-tdm-webinars
https://aka.ms/developervelocity-bdm-webinars

Developer Velocity

  • 2.
    Chris Reddington Cloud SolutionArchitect at Microsoft UK Blogger, Podcaster & Vlogger @ www.cloudwithchris.com Consider a follow or subscribe! Twitter: https://twitter.com/reddobowen YouTube: https://youtube.com/c/CloudWithChris GitHub: https://github.com/chrisreddington Steph Locke Cloud Solution Architect Manager at Microsoft UK Ex-Data&AI MVP Blog: https://stephlocke.info Twitter: https://twitter.com/theStephLocke LinkedIn: https://linkedin.com/in/StephanieLocke GitHub: https://github.com/stephlocke
  • 5.
  • 6.
  • 7.
    Key findings Top driversfor business performance: developer tools, product management capabilities, culture, and talent management Open source is the #1 differentiator for top performers Accelerating impact: AI/ML, Live site automation, product telemetry Public cloud adoption is a key differentiatordriving business performance for non-digital companies Top performers: empowering citizen developers score 33% higher on innovation </>
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
    An evolving journeyof Developer Velocity Buildingthe Foundation Transforming the Future Accelerating the Impact Talent management Product management capabilities Automated live site management Public cloud adoption Effective developer toolchain Open source adoption and capabilities AI / ML Culture of psychological safety and ‘fail fast & learn’ Empowering citizen developers Usage of granular product telemetry </> AI
  • 14.
  • 15.
  • 16.
  • 17.
    What is DevOps? Development+ operations DevOps is the union of people, processes, and technologies to deliver continuous value to users.
  • 18.
    Here’s how Microsoftbuilds apps of the future Adopting a growth mindset Creating a culture of customer obsession Aligning to shared objectives
  • 19.
    DevOps across Microsoft http://aka.ms/DevOps-Stories 105K Engineersuse the DevOps platform 2M Git commits per month 4.4M Builds per month 500M Test executions per day 5M Work items viewed per day 500K Work items updated per day 85,000 Deployments per day Internal Microsoft Engineering System Activity, July 2019
  • 20.
    Create clarity Measure outcomes notoutputs Objectives and key results (OKRs) Example objective: Grow a strong and happy customer base 1. Increase external NPS from 21 to 35 2. Increase docs SAT from 55 to 65 3. New pipeline flow has an Apdex score of 0.9 4. Queue time for jobs is 5 seconds or less KRs are measures for the quarter. Best KRs are leading indicators. Encourage ambitious KRs: 70% of the improvement target scores green.
  • 21.
    Create clarity Change whatyou track Focus on measuring only the most critical and impactful KPIs: Customer usage How much value are users getting? Acquisition Retention Engagement Satisfaction (NPS) Feature usage Pipeline throughput How efficient is the DevOps process? Time to build Time to test Time to deploy Time to improve Failed and flaky automation Live-site health How quickly can you detect and fix issues? Time to detect, time to communicate, time to mitigate Customer impact, customer support metrics Incident prevention items Aging live-site problems SLA per customer
  • 22.
    Be customer obsessed Ensureproducts are live in production and collecting feedback Collect telemetry data that examines the hypothesis that motivated the deployment. Gather information and making incremental changes is key to improvement. Treating the goal as a question rather than a statement of fact motivates developers to continue testing their ideas.
  • 23.
    Evolve to fullstack teams Evolve the organization (1 of 3) Program management Development Testing
  • 24.
    Evolve to fullstack teams Evolve the organization (2 of 3) Program management Engineering Ops/SRE
  • 25.
    Evolve to fullstack teams Evolve the organization (3 of 3) Feature team
  • 26.
  • 27.
  • 28.
    Change how youcollaborate, develop, and deliver Collaborate on code Leverage innersource practices to build better products and create a culture of openness by: Building upon open-source projects that use global contributors Customizing, adapting, and reusing existing internal code Collaborating with developers across your organization Appointing a smaller team to make final decisions Leveraging open- source software and innersource practices enables your team to innovate and deliver value faster.
  • 29.
    Change how youcollaborate, develop, and deliver Drive toward automation Long running functional tests Shifted to unit tests from automated functional tests Count of tests M62 M63 M64 M65 M66 M67 M68 M69 M70 M71 M72 M73 M74 M75 M76 M77 M78 M79 M80 M81 M82 M83 M84 M85 M86 M87 M88 M89 M90 M91 M92 M93 M94 M95 M96 M97 M98 M99
  • 30.
    Change how youcollaborate, develop, and deliver Utilize pull requests Control changes Pull requests are the first gate to control how changes get merged to the master. Limit human error and ensure only high-quality code gets merged with detailed permissions, policies, and status checks.
  • 31.
    Enhance security fromdev to production Secure the software supply chain Apply policies and controls proactively Rely on security automation Build security checks into your CI/CD pipelines Strengthen your security posture
  • 32.
    Enhance security fromdev to production Build security practices into your delivery Key security practices: Pre-commit • Threatmodeling • IDE securityplug-in • Pre-commithooks • Secure coding standards • Peer review Commit (CI) • Static code analysis • Securityunit tests • Dependency management Deploy (CD) • Infra as code (IaC) • Securityscanning • Cloud configuration • Securityacceptance tests Analyze • Security Smoke tests • Configuration checks • Penetration tests Operate • Continuous monitoring • Threatintelligence • Penetration testing • Blameless postmortems
  • 33.
    An evolving journeyof Developer Velocity Buildingthe Foundation Transforming the Future Accelerating the Impact Talent management Product management capabilities Automated live site management Public cloud adoption Effective developer toolchain Open source adoption and capabilities AI / ML Culture of psychological safety and ‘fail fast & learn’ Empowering citizen developers Usage of granular product telemetry </> AI
  • 34.
  • 35.