Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JavaOne - Performance Focused DevOps to Improve Cont Delivery

2,268 views

Published on

These are the slides of my JavaOne presentation. The abstract goes like this:
How do companies developing business-critical Java enterprise Web applications increase releases from 40 to 300 per year and still remain confident about a spike of 1,800 percent in traffic during key events such as Super Bowl Sunday or Cyber Monday? It takes a fundamental change in culture. Although DevOps is often seen as a mechanism for taming the chaos, adopting an agile methodology across all teams is only the first step. This session explores best practices for continuous delivery with higher quality for improving collaboration between teams by consolidating tools and for reducing overhead to fix issues. It shows how to build a performance-focused culture with tools such as Hudson, Jenkins, Chef, Puppet, Selenium, and Compuware APM/dynaTrace

Published in: Technology
  • Be the first to comment

JavaOne - Performance Focused DevOps to Improve Cont Delivery

  1. 1. 11 Andreas Grabner http://apmblog.compuware.com @grabnerandi
  2. 2. 2
  3. 3. 3
  4. 4. 4
  5. 5. 5 Testing is Important – and gives Confidence
  6. 6. 6 But are we ready for “The Real” world?
  7. 7. 7 Measure Performance during the game Ball Possession: 40 : 60 Fouls: 0 : 0 Score: 0 : 0 Minute 1 - 5
  8. 8. 8 Measure Performance during the game Minute 6 - 35 Ball Possession: 80 : 20 Fouls: 2 : 12 Score: 0 : 0
  9. 9. 9 Deep Dive Analysis
  10. 10. 10 Options “To Fix” the situation
  11. 11. 11 Not always a happy ending  Minute 90 Ball Possession: 80 : 20 Fouls: 4 : 25 Score: 3 : 0
  12. 12. 12 FRUSTRATED FANS!! 12
  13. 13. 13 How does that relate to Software?
  14. 14. 1414 From Deploy to … Deploy Promotion/Event Problems Ops Playbook War Room Timeline
  15. 15. 1515 The “War Room” – back then 'Houston, we have a problem‘ NASA Mission Control Center, Apollo 13, 1970
  16. 16. 1616 The “War Room” – NOW Facebook – December 2012
  17. 17. 1717 Problem: Unclear End User Problem Descriptions
  18. 18. 1818 Statuc Quo: Ops Runbook – System Unresponsive
  19. 19. 1919 Problem: Unclear Ops Problem Descriptions
  20. 20. 2020 Status Quo: Ops Runbook – High Resource Usage
  21. 21. 2121 Lack of data?
  22. 22. 2222
  23. 23. 23 Answers to the right questions
  24. 24. 2424 What are the real questions? Individual Users? ALL users? Is it the APP? Or Delivery Chain? Code problem? Infrastructure? One transaction? ALL transactions? In AppServer? In Virtual Machine?
  25. 25. 2525 Problem: What Devs would like to have
  26. 26. 2626 Problem: What Devs would like to have Top Contributor is related to String handling 99% of that time comes from RegEx Pattern Matching Page Rendering is the main component
  27. 27. 2727 Its getting this …Its like getting this …
  28. 28. 28 … when you need to see this!
  29. 29. 2929 RECAP Status Quo: We don’t like “War Rooms”
  30. 30. 3030 Problem: Attitudes like this don’t help either Image taken from https://www.scriptrock.com/blog/devops-whats-hype-about/ Shopzilla CIO (in 2010): “… when they get in the war room - the developers and ops teams describe the problem as the enemy, not each other”
  31. 31. 3131 Problem: Very “expensive” to work on these issues ~80% of problems caused by ~20% patterns YES we know this 80%Dev Time in Bug Fixing $60BDefect Costs BUT
  32. 32. 3232 TOP PROBLEM PATTERNS • Focus on Web and Java
  33. 33. 3333 Top Problem Patterns: Resource Pools
  34. 34. 3434 Top Problem Patterns: Resource Pools
  35. 35. 3535 Deployment Mistakes lead to internal Exceptions
  36. 36. 3636 Deployment Mistakes lead to high logging overhead
  37. 37. 3737 Production Deployment leads to Log SYNC Issues
  38. 38. 3838 Long running SQL with Production Data
  39. 39. 3939 N+1 Query Problem
  40. 40. 4040 Reading and processing too much data in App
  41. 41. 4141 Memory Leaks in Cache Layer with Production Data Still crashes Problem fixed!Fixed Version Deployed
  42. 42. 4242 Synchronization Issues under real load
  43. 43. 4343 BLOATED Web Sites 17! JS Files – 1.7MB in Size Useless Information! Even might be a security risk!
  44. 44. 4444 Missing or incorrect configured browser caches 62! Resources not cached 49! Resources with short expiration
  45. 45. 4545 SLOW or Failing 3rd Party Content
  46. 46. 4646 Want MORE of these and more details? http://apmblog.compuware.com
  47. 47. 4747 Lots of Problems that could have been avoided • BUT WHY are they still making it to Production?
  48. 48. 4848 Missing Focus on Performance
  49. 49. 4949 Different Goals for Dev and Ops
  50. 50. 5050 Disconnected Teams despite “Shared Responsibility”
  51. 51. 5151
  52. 52. 5252 How to make the Enterprise Crew happy?
  53. 53. 5353
  54. 54. 5454 Solution: DevOps + Performance Focus Culture “Shared Responsibility” Agile Process for ALL Teams Performance as Key Requirement X-Team Collaboration and Education Automation Measurement, Collaboration and Deployment Automate Performance and Architectural Problem Detection Measurement “Visible” KPIs for each Team Focus on Performance, Architectural and Deployment Measures Sharing Expertise, Tool and Data Sharing “Easy” sharing of Performance, Deployment and Production Data http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/
  55. 55. 5555 Culture: EXTEND Requirements with … Performance Scalability Testability Deployability Deployability
  56. 56. 5656 Sharing: DON’T EXCLUDE anyone from Agile Process Stand-Ups Sharing Tools Feedback
  57. 57. 5757 Measurement: Define KPIs accepted by all teams # of SQL Executions # of Log Lines MBs / Uses Time for Deployment Time for Rollback Response TimesPerf Test Code Coverage
  58. 58. 5858 AUTOMATION, AUTOMATION, AUTOMATION Performance Scalability Shared Tools Automatic Feedback
  59. 59. 5959 DevOps Collaboration – TODO LIST FOR YOU!! Access to Production Data Shared Reporting and Task Management Diagnostic Tools Shared Performance KPIs and Tooling Known How Exchange
  60. 60. 6060 Recap – Problem – Root Cause – Solution - Result DevOps + Performance Culture Automation Measurement Collaboration
  61. 61. 6161 TIPS FOR DEVS
  62. 62. 6262 Performance Focus in Test Automation 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test Framework Results Architectural Data We identified a regresesion Problem solved Lets look behind the scenes Exceptions probably reason for failed tests Problem fixed but now we have an architectural regression Problem fixed but now we have an architectural regression Now we have the functional and architectural confidence
  63. 63. 6363 Performance Focus in Test Automation Analyzing All Unit / Performance Tests Analyzing Metrics such as DB Exec Count Jump in DB Calls from one Build to the next
  64. 64. 6464 Performance Focus in Test Automation Cross Impact of KPIs
  65. 65. 6565 Performance Focus in Test Automation Embed your Architectural Results in Jenkins
  66. 66. 6666 Performance Focus in Test Automation Here is the difference! Compare Build that shows BAD Behavior! With Build that shows GOOD Behavior!
  67. 67. 6767 Performance Focus in Test Automation CalculateUserStats is the new Plugin that causes problems
  68. 68. 6868 Remember – DevOps requires Cultural Change Share Integrate Collaborate Performance
  69. 69. 6969 Elevate our DevOps Investment - REDUCE 80% Dev Time for Bug Fixing $60B Costs by Defects
  70. 70. 70 © 2011 Compuware Corporation — All Rights Reserved© 2011 Compuware Corporation — All Rights Reserved 70 Participate in Compuware APM Discussion Forums apmcommunity.compuware.com Like us on Facebook facebook.com/CompuwareAPM Join our LinkedIn group Compuware APM User Group Follow us on Twitter twitter.com/CompuwareAPM Read our Blog About:Performance Watch our Videos & product Demos youtube.com/Compuware www.compuware.com/APM Thank You
  71. 71. 71 © 2011 Compuware Corporation — All Rights Reserved Simply Smarter

×