DevOps, pilier de la transformation stratégique de Microsoft. Sam Guckenheimer, Corporate Product Strategist, Visual Studio and Cloud Services Munil Shah, Partner Director Software Engineering, Visual Studio Team Services

1. http://visualstudio.com/devops

2. http://aka.ms/devops

3. The Journey
Sprint 1
August 2010
Sprint 113
January 2017
Sprint 29
June 2012
Visual Studio Team Services
“Public Preview”

4. And We Support One Engineering System
If you want to go fast, go alone. If you want to go far, go together.

5. Our learnings as
a SaaS provider

6. Our learnings as
a SaaS provider

7. Team rooms

9. What we accomplished
Progressive
Deployment
Week 1 Week 2 Week 3
Week 1 Week 2 Week 3Week 2 Week 3
Sprint 98Sprint 97 Sprint 99
The sprint plan

12. Our learnings as
a SaaS provider

13. One code base with multiple delivery streams
Single master branch, multiple release branches
Shared abstraction layer
Update 1
Update 2
Update N
Visual Studio Team Services
Team Foundation Server

15. Tests should be written at the lowest level possible
Write once, run anywhere - includes production system
Product is designed for testability
Test code is product code, only reliable tests survive
Testing infrastructure is a shared service
L0: Run against raw drop. Only access binaries. Run in the build. Must be fast and reliable.
L1: Run against raw drop, and can hit resources on machine (e.g. SQL test)
L2: Run against “special” deployed product. We analyzed sources of timing issues and eliminated them.
L3: Run against production

17. Our learnings as
a SaaS provider

18. … and proactively reach out to customers with low availability
Found one of the top customers with low
availability. Proactively reached out and resolved
their issue.

19. UserVoice Top Customers

20. Our learnings as
a SaaS provider

22. Existing experience Baseline:
36% conversion to project
Hypothesis: Can simplify path to “,magic moment”
After 3 experiments, with different treatments:
50% to 100% customers
conversion to project (+18%)

23. All code is deployed, but feature flags control exposure

24. All code is deployed, but feature flags control exposure

25. ON
OFF
Feature
Flag

26. ON
OFF
Feature
Flag

27. ON
OFF
Feature
Flag

28. ON
OFF
Feature
Flag

29. ON
OFF
Feature
Flag

30. ON
OFF
Feature
Flag

32. Our learnings as
a SaaS provider

33. Application
Insights Analytics
(Project Kusto) for
• text search and
queries over
structured and semi-
structured data
• high volume ingestion
• fast queries over very
large data sets

34. Our learnings as
a SaaS provider

35. 1 2 3 4 5

38. Redundant alerts for same the issue
Needed to set right thresholds and tune often
Stateless alerts contributed to further noise
Every alert must be actionable and represent a real issue with the system
Alerts should create a sense of urgency – false alerts dilute that
Consolidate alerts so that only actionable alerts are sent to team
Autoroute according a health model based on suspect route cause

39. Found 3 errors for memory
and performance
All 3 errors are related to
the same code defect
Eliminated alert noise:
~928 alerts per week to ~22
Reduced DRI escalations by ~56%
APM component mapped to feature team
Auto-dialer engaged Global DRI

40. 40
 Double blind test
 Full disclosure at or near end
vs.
 Share tactics & lessons learned
 Continued evolution
https://www.youtube.com/watch?v=i9qf3VdfcjE

41. Our learnings as
a SaaS provider

42. Demo:
How we use Visual Studio Team Services
for Microsoft’s One Engineering System

44. Our learnings as
a SaaS provider

46. Ronny Kohavi
Distinguished Engineer,
Analysis and Experimentation

49. Live Site Health
Time to Detect
Time to Communicate
Time To Mitigate
Customer Impact
Incident prevention items
Aging live site problems
Customer support metrics
SLA per customer account
(SLA, MPI, top drivers)
Engineering
Bug cap per engineer
Aging bugs in important
categories
Pass rate & coverage by
test level
Velocity
Time to build
Time to self test
Time to deploy
Time to learn
(Telemetry pipe)
Usage
Acquisition
Engagement
Dedication
Churn
Feature usage

51. Appendix:
Our Dashboards, Backlog, Pipeline

58. Appendix: Our Telemetry Examples