Devops Metrics

  • 13,042 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • The post is very informative. It is a pleasure reading it. I have also bookmarked you for checking out new posts.


    http://www.jaibalajiproperties.com/indirapuram-flats.html
    Are you sure you want to
    Your message goes here
  • Mind blowingly broad in its scope, brain crushingly hard to follow 100% without the talk track, lots to think about on the flight home...
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
13,042
On Slideshare
0
From Embeds
0
Number of Embeds
8

Actions

Shares
Downloads
242
Comments
2
Likes
14

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. DEVOPS METRICS ANDREW SHAFER - PATRICK DEBOIS - ISRAEL GATThursday 16 June 2011
  • 2. Patrick Debois Jedi @patrickdebois http://jedi.be/blog http://www.vectronicsappleworld.com/collection/articlepics/joysticks/snap23.jpgThursday 16 June 2011
  • 3. Israel Gat Cutter Consortium @agileexec http://theagileexecutive.comThursday 16 June 2011
  • 4. Andrew Shafer - Rabble Rouser - Cloudscaling @littleidea - http://stochasticresonance.wordpress.comThursday 16 June 2011
  • 5. http://www.flickr.com/photos/quinnanya/3588989219Thursday 16 June 2011
  • 6. http://www.flickr.com/photos/zen/3233023066/Thursday 16 June 2011
  • 7. To Compare http://www.flickr.com/photos/thebusybrain/2492945625/Thursday 16 June 2011
  • 8. http://www.flickr.com/photos/topshampatti/219953012/Thursday 16 June 2011
  • 9. To UnderstandThursday 16 June 2011
  • 10. To ProtectThursday 16 June 2011
  • 11. To Improve http://www.flickr.com/photos/vizzzual-dot-com/2655969483/Thursday 16 June 2011
  • 12. Describing a Metric http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 13. Describing a Metric •basis of measurement http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 14. Describing a Metric •basis of measurement •assumptions http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 15. Describing a Metric •basis of measurement •assumptions •level and usage http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 16. Describing a Metric •basis of measurement •assumptions •level and usage •desired trend http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 17. Describing a Metric •basis of measurement •assumptions •level and usage •desired trend •when to use http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 18. Describing a Metric •basis of measurement •assumptions •level and usage •desired trend •when to use •when to stop http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 19. Describing a Metric •basis of measurement •assumptions •level and usage •desired trend •when to use •when to stop •how to game http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 20. Describing a Metric •basis of measurement •assumptions •level and usage •desired trend •when to use •when to stop •how to game •warnings http://www.flickr.com/photos/chandramarsono/4324373384/Thursday 16 June 2011
  • 21. Beware of Vanity Metrics UNACTIONABLE INACCESSIBLE UNAUDITABLE http://www.flickr.com/photos/centralasian/5829549813/Thursday 16 June 2011
  • 22. Red is good , Green is worthless • http://www.reliableplant.com/Read/21305/lean-metrics-red-is-good,-green-is-worthlessThursday 16 June 2011
  • 23. Thursday 16 June 2011
  • 24. “In truth, a good case could be made that if your knowledge is meagre and unsatisfactory, the last thing in the world you should do is make measurements; the chance is negligible that you will measure the right things accidentally.” George Miller http://stochasticresonance.wordpress.com/2008/08/03/miles-per-gallon/Thursday 16 June 2011
  • 25. What we can learn from monitoring http://www.experimentalistsanonymous.com/stuff/paralleluniverse/images/2edmoneyshot.jpgThursday 16 June 2011
  • 26. Level 0 - Individual ResourceThursday 16 June 2011
  • 27. Availability Processes, Port,SSH Login Network, Memory, CPUThursday 16 June 2011
  • 28. Availability Processes, Port,SSH Login Network, Memory, CPUThursday 16 June 2011
  • 29. Capacity Processes, Port,SSH Login Network, Memory, CPUThursday 16 June 2011
  • 30. Capacity Processes, Port,SSH Login Network, Memory, CPUThursday 16 June 2011
  • 31. Health Thresholds http://www.smiley-faces.org/wallpaper/smiley-face-wallpaper-001.jpg http://www.geocities.jp/nikonikocalendar/index_en.htmlThursday 16 June 2011
  • 32. Health Thresholds http://www.smiley-faces.org/wallpaper/smiley-face-wallpaper-001.jpg http://www.geocities.jp/nikonikocalendar/index_en.htmlThursday 16 June 2011
  • 33. Metric versus Diagnostic Does it contribute to Yes Measurement the Metric bottom line? No Diagnostic Pete Behrens,Trail Ridge Consulting http://www.slideshare.net/krivitsky/agile-metrics-presentation-65425Thursday 16 June 2011
  • 34. Level 1 - Services LevelThursday 16 June 2011
  • 35. Performance SSH, HTTP, NTPThursday 16 June 2011
  • 36. Performance SSH, HTTP, NTP # Servers/AdminThursday 16 June 2011
  • 37. Performance SSH, HTTP, NTP # Servers/Admin # Line of Code/DeveloperThursday 16 June 2011
  • 38. “Gameable” http://kithicor.org/wp/wp-content/uploads/2010/01/gamerEvolution.jpgThursday 16 June 2011
  • 39. Level 2 - Team LevelThursday 16 June 2011
  • 40. Aggregation LB-WEB WEB WEB WEB 1 2 3Thursday 16 June 2011
  • 41. Aggregation DEV Project Team DEV DEV DEVThursday 16 June 2011
  • 42. Burn up chartThursday 16 June 2011
  • 43. Team UtilizationThursday 16 June 2011
  • 44. “Velocity also has a direction component” http://www.randomstew.com/wp-content/uploads/mooneye.jpgThursday 16 June 2011
  • 45. What about efficiency? http://www.flickr.com/photos/jdiggans/92981578/Thursday 16 June 2011
  • 46. http://sievertschreiber.files.wordpress.com/2009/12/good-code-is-measured-in-wtf-per-minute.jpgThursday 16 June 2011
  • 47. Incentives & Behaviors http://www.flickr.com/photos/turtlemadness/2439050717/Thursday 16 June 2011
  • 48. Development Operations Level 3 - Integration Levelhttp://www.xgn.nl/images/upload/20081020225036.jpg http://www.kawaiikakkoiisugoi.com/wp-content/uploads/2011/03/pacman-screen.jpgThursday 16 June 2011
  • 49. Integration WEB APP DBThursday 16 June 2011
  • 50. Collaboration DEV QA OPS # of deploys / dayhttp://www.sage.org/lists/sage-members-archive/2001/msg01208.htmlhttp://www.datacenterknowledge.com/archives/2009/12/30/how-many-servers-can-one-admin-manage/http://fc04.deviantart.net/fs70/f/2010/108/2/3/Pac_Man_Widescreen_by_molotov_arts.jpgThursday 16 June 2011
  • 51. Counting Collaboration More interaction doesn’t mean a better party • http://www.nature.com/news/2010/101013/full/news.2010.538.htmlThursday 16 June 2011
  • 52. Measuring Collaboration? “Play Nice Study” http:///www.healthylarimer.org/documents/PlayNice.pptThursday 16 June 2011
  • 53. Levels Collaboration Collaboration Coalition Coordination CooperationThursday 16 June 2011
  • 54. Cooperation provide information somewhat defined roles formal communication decisions made independentlyThursday 16 June 2011
  • 55. •share information & resources •defined roles •frequent communication Coordination •some shared decision makingThursday 16 June 2011
  • 56. •share ideas •share resources Coalition •frequent & prioritized communication •all members have a vote on decisionsThursday 16 June 2011
  • 57. Collaboration •members belong to one system •frequent communication characterized by mutual trust •consensus on all decisionsThursday 16 June 2011
  • 58. Knowledge sharing http://www.au.af.mil/au/awc/awcgate/navy/onr_metrics_on_collab.pdfThursday 16 June 2011
  • 59. Team Resilience 10th Floor Test Truck Factor to take a random machine in your infrastructure ,drop it from the 10th floor, and be able to restore your infrastructure to working within 5-10 minute The number of people on your team who have to be hit with a truck before the project is in serious trouble Resilience should be a 4th management objective, alongside Better/Faster/Cheaper John Allspaw http://www.kitchensoap.com/ Thursday 16 June 2011
  • 60. Black Hole Theory Absorbs light, to see the effects you need to look at nearby objects http://www.collaborate.com/blog/2010/01/28/musing-metrics-collaborationThursday 16 June 2011
  • 61. Quest for proof • Innovating and/or providing decision support • Saving time or Money (tangible but less than (tangible) quality) • Increase Quality • Easing access to and (tangible... but less so) interactions with subject- matter experts (intangible) http://communityandbusiness.groupsite.com/group/Collaboration20/post/measuring-the-value-of-collaboationThursday 16 June 2011
  • 62. More vs Better Widgets Add more servers Tune existing servers Process Improvement Quality Improvement Easier More Difficult!! http://socialconsulting.wordpress.com/2009/02/24/measuring-collaboration/Thursday 16 June 2011
  • 63. Collaboration Challenge “Maximize Team Utility”Thursday 16 June 2011
  • 64. Level 4 - Business LevelThursday 16 June 2011
  • 65. AHA KA-CHING!Thursday 16 June 2011
  • 66. Return on Investment http://www.flickr.com/photos/hanekomu/4481674104/Thursday 16 June 2011
  • 67. Inventory (WIP)Thursday 16 June 2011
  • 68. Time to Market http://stefanroock.wordpress.com/2010/03/02/kanban-definition-of-lead-time-and-cycle-time/Thursday 16 June 2011
  • 69. Mean Time Between Failure http://museumdosgames.blogspot.com/2010/04/game-over.htmlThursday 16 June 2011
  • 70. Mean Time Between ResolutionThursday 16 June 2011
  • 71. Technical DebtThursday 16 June 2011
  • 72. LeaksThursday 16 June 2011
  • 73. Debt Metaphor not all debt is created equal http://www.flickr.com/photos/shoot-art/4536509717/Thursday 16 June 2011
  • 74. Customer Responsiveness Jim HighsmithThursday 16 June 2011
  • 75. Types of Tech debt “oops” “yeah I know” unintentional intentional “we’ll fix it the next minor “we’ll fix it the next major revision” revision” short term (tactical) long term (strategical) http://forums.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspxThursday 16 June 2011
  • 76. Gartner • http://www.castsoftware.com/resources/materials/analyst/Gartner-How_to_Monetize_Application_Technical_Debt.pdfThursday 16 June 2011
  • 77. Vicious Cycle of The Vicious Cycle of TechnicalDebt Technical Debt (More) Relentless Pressure Take Diminished Technical Dev Velocity Debt Technical Fail to Pay Debt Debt Back Accrues Neglect Maintenance http://theagileexecutive.com/2010/09/20/how-to-break-the-vicious-cycle-of-technical-debt/ Israel Gat 1Thursday 16 June 2011
  • 78. Symptoms - Measurable • Poor Customer Responsiveness • Long Delivery Times • Late deliveries • Lots of defects (Toxic Code) • Rising Development costs • Frustrated & Poor performing Teams http://www.theagileengineer.com/public/Home/Home_files/TechnicalDebt_published.pdfThursday 16 June 2011
  • 79. Secret Sauce http://radar.oreilly.com/archives/2007/10/Ops-ROI-jesserobbins.htmlThursday 16 June 2011
  • 80. The Team Knows http://kingsinsight.com/2010/07/31/ask-the-team-about-technical-debt-the-creeping-doom-graph/Thursday 16 June 2011
  • 81. Failure Value Demand % Demand As a Team http://farm4.static.flickr.com/3167/2769773425_16c235dc28.jpgThursday 16 June 2011
  • 82. Level 5 - End User LevelThursday 16 June 2011
  • 83. End User Perspective • END to END monitoring • User Experience (browser testing) • External parties (Email, ISP, Plugins)Thursday 16 June 2011
  • 84. Participatory Design http://www.informaworld.com/smpp/content~db=all~content=a790786652Thursday 16 June 2011
  • 85. “Crowd” - Monitoring Sudden Change in User Activity Pattern Detection of problem? A/B TestingThursday 16 June 2011
  • 86. Spend more time Automation => creatively , early in the process Traditional Engineering http://www.informaworld.com/smpp/content~db=all~content=a790786652Thursday 16 June 2011
  • 87. Design Strategies To Design is Human To Design does not require super powers Design used to be a collective or shared ability http://www.amazon.com/Research-Design-Thinking-Nigel-Cross/dp/9062757960 p 4 http://www.psych.lancs.ac.uk/people/lindenball.htmlThursday 16 June 2011
  • 88. Open Innovation Wikipedia Open Source SharedConfig Management Shared Infrastructures http://en.wikipedia.org/wiki/Open_innovation http://www.clickadvisor.com/downloads/Bughin_Next_Step_in_Open_Innovation.pdfThursday 16 June 2011
  • 89. Novice vs Expert • depth first • breath first , depth dives • vertical oriented • horizontal oriented • can not deal with • uncertainty = options uncertainty Linden Ballhttp://www.amazon.com/Research-Design-Thinking-Nigel-Cross/dp/9062757960 p 147 http://www.psych.lancs.ac.uk/people/lindenball.htmlThursday 16 June 2011
  • 90. Continuous Evolution “Problem space evolves with the solution” http://www.amazon.com/Research-Design-Thinking-Nigel-Cross/dp/9062757960 http://www.psych.lancs.ac.uk/people/lindenball.htmlThursday 16 June 2011
  • 91. Cultural TaxonomyThursday 16 June 2011
  • 92. Cultural TaxonomyThursday 16 June 2011
  • 93. tl;dr • avoid vanity metrics • insight > data • don`t hate the player • hate the game • business value good • beware of technical debtThursday 16 June 2011
  • 94. Questions?Thursday 16 June 2011
  • 95. Thursday 16 June 2011
  • 96. Thank you!Thursday 16 June 2011