How do you migrate over 65,000 of the most demanding software engineers from infrastructure built up over decades of high-intensity work into a common engineering system based on modern software development technologies and best practices?

2. martinwo@microsoft.com
@martinwoodward
http://visualstudio.com/devops

3. I work here

6. I work here

7. Visual Studio Team Services
Continuous
Delivery for
Every Team,
Every App,
Every Platform Continuous
Delivery
Agile Planning
Dashboards
Kanban Boards
Taskboards
Build and Test
Git Source Control
Modern Code Workflow
Continuous Integration
Continuous Testing
Package Management
Open Source Compliance
Plan +
Track
Monitor +
Learn
Release
Develop +
Test
Delivery
Deployment of app and
infrastructure
PaaS, IaaS and
Containers
Monitoring
Telemetry
Diagnostics
Analysis

10. Sprint 1
August 2010
Sprint 29
June 2012
VSTS Preview
Sprint 64
April 2014
VSTS GA
Sprint 67
June 2014
1ES
Sprint 92
Nov 2015
Extensions
Sprint 102
June 2016
GVFS
Sprint 124
Sep 2016

14. Increase flow of value Shorten cycle times Continuously Improve

17. • Original estimate
• Completed hours
• Lines of Code
• Team capacity
• Team burndown
• Team velocity
• # of bugs found
Things we don’t watch• Acquisition
• Engagement
• Satisfaction
• Churn
• Feature Usage
Usage
• Time to Detect
• Time to Communicate
• Time to Mitigate
• Customer Impact
• Incident Prevention Items
• Aging Live Site Problems
• SLA per Customer
• Customer Support Metrics
Live Site Health
• Time to Build
• Time to Self Test
• Time to Deploy
• Time to Learn
Velocity

18. Found one of the top customers with low availability.
Proactively reached out and resolved their issue.

23. •
•
•
•
•
•
•

25. Before
• Redundant alerts for same the issue
• Needed to set right thresholds and
tune often
• Stateless alerts contributed to
further noise
After
• Every alert must be actionable and
represent a real issue with the
system.
• Alerts should create a sense of
urgency – false alerts dilutes that

26. Health model in action
• 3 errors for memory and
performance
• All 3 related to same
code defect
• APM component mapped to feature team
• Auto-dialer engaged Global DRI
Eliminated alert noise
~928 alerts per week to
~22 and reduced DRI
escalations by ~56%

27. •
•

28. “One time” deployment commands in OneNote, email
Set-Options “-p 0”
Imagine a dozen more steps like that…
And then…someone misses a step half way through
We once broke pre-production for a day

29. 




30. 1 2 3 4 5

31. https://github.com/Netflix/Hystrix/wiki

32. •
•
•
•
•
•
•
•
•

33.  Double blind test
 Full disclosure at or near end
vs.
 Share tactics & lessons learned
 Continued evolution
Assume Breach - Use War Games to the learn attacks and practice response

35. Alignment
Autonomy
“Let’s try to give our teams three things….
Autonomy, Mastery, Purpose”

36. Phone VS/PS
Phone VS/PS
Phone VS/PS
Phone VS/PS
Phone VS/PSUnique PS accounts and
workflows
1 year migration plan
WDG Product Studio Account
WDG VSTS Account

37. Can you add all of my
custom workflow fields
to the bug template
that everyone uses?
Can you add a new
work item type that is
only used by my feature
team?
Is it okay if I
track my team’s
hardware
purchases in
VSTS?
Can we store every
Watson report as a bug
even if we’re not sure it
needs to be fixed?
Is it okay if I create new
rules that everyone has
to follow for how to
track dependencies?
Is it cool if I add a
few hundred new
default values?
I prefer
Tags, let’s
use those!
I prefer
Keywords,
let’s use
those!
Yes!

40. Work Management
Information Tools
Coding Tools
Testing Tools
Build Tools
Debugging Tools
Developer
Time
Source: Internal Microsoft Telemetry analysis of 4700 engineers in WDG

44. Teams
 Physical team rooms
 Cross discipline
 10-12 people
 Self managing
 Clear charter and goals
 Intact for 12-18 months
 Own features in production
 Own deployment of features

45. UI
API
Data

46. UI
API
Data

47. Employee choice, not
manager driven
Typically <20%
change, but 100% get
to make a choice
Cross-pollinate talent
and micro-culture
Sticky Note Exercise - Self Forming Teams

48. Sprint
3 weeks
1
Plan
3 sprints
3
Season
6 months
6
Scenario
18 months
Teams are responsible for the detail
Leadership is responsible
for driving the big picture

49. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
18 month scenario
6 month plan
CY17 H1

50. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
18 month scenario
6 month plan

51. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
18 month scenario
6 month plan

52. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
18 month scenario
6 month plan

53. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
18 month scenario
6 month plan

54. TIME
VALUE

55. TIME
VALUE

57. 3 weeks
SpringFallSpring Fall

58. What we accomplished
Progressive
Deployment
Week 1 Week 2 Week 3
Week 1 Week 2 Week 3Week 2 Week 3
Sprint 98Sprint 97 Sprint 99
The sprint plan

61. SpringFallSpring Fall
3 sprints

64. SpringFallSpring Fall

68. engineers on
your team# 5 ?x =

69. Rule: If your bug count exceeds your bug cap… stop working
on new features until you’re back under the cap.
5 50x =10

71. Code Test & Stabilize Code Test & Stabilize
Code
Complete
Planning

73. • Tests that anyone can run anywhere (inc production)
• Shifted to unit tests from automated functional tests
• Core tests run before pull request
• Fast and 100% reliable build and test is critical
• Rolling tests run after commit

75. PR Validations
(Impacted Test runs
and tests)
PR Validations
Build and static tools
PR Creation
Create PR
VSO.PR
Build+L0
VSO.L1
OnPrem.SelfTest
RM.SelfTest
Manual
Tfs.SelfHost
VSO.CredScan
Cred Scanner
Policheck
CI – Required
Runs
CI - Test
(Release Defs)
CI - Build
(Build Defs)
Branch
Update
Push
VSO.CI
SPS.SelfTest
Tfs.SelfTest
Onprem.SelfTest Report
RM.SelfTest Report
Spsext.SelfTest
Tfs.SelfHost Report
VSO.Packaging.CI
VSO.SDKSample.
CI

76. Customer Focussed
Production First Mindset
Team Autonomy + Enterprise Alignment
Shift Left Quality
Infrastructure as Flexible Resource

78. Shared abstraction layer
Single master branch, multiple release branches
Update 1
Update 2
Update N
Visual Studio Team Services
Team Foundation Server
One code base with multiple delivery streams

79. Shared Platform Services (SPS)
North Central
TFS SU1
North Central
AT
AT
AT
JA
JA
JA
Blob
TFS SU7
Australia
TFS SU0
West Central
Containerized Services

80. •
•
•
•
•
• If you are starting out today and cloud native, consider
PaaS, Service Fabric + Azure Functions
• We need to ship same code to on-prem & cloud

82. After
• 3-week sprints
• Vertical teams
• Team rooms
• Continual Planning & Learning
• PM & Engineering
• Continual customer engagement
• Everyone in master
• 8-12 person teams
• Publicly shared roadmap
• Zero debt
• Mockups in PPT
• Inner source
• Flattened organization hierarchy
• User satisfaction determines success
• Features shipped every sprint
Before
• 4-6 month milestones
• Horizontal teams
• Personal offices
• Long planning cycles
• PM, Dev, Test
• Yearly customer engagement
• Feature branches
• 20+ person teams
• Secret roadmap
• Bug debt
• 100 page spec documents
• Private repositories
• Deep organizational hierarchy
• Success is a measure of install numbers
• Features shipped once a year

83. https://aka.ms/ignite.mobileapp
Please
evaluate
this session https://myignite.microsoft.com/evaluations
Your input is
important!