Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
11
Make it part of your Agile Delivery
2
3
4
5
Testing is Important – and gives Confidence
6
But are we ready for “The Real” world?
7
Measure Performance during the game
Ball Possession: 40 : 60
Fouls: 0 : 0
Score: 0 : 0
Minute 1 - 5
8
Measure Performance during the game
Minute 6 - 35
Ball Possession: 80 : 20
Fouls: 2 : 12
Score: 0 : 0
9
Deep Dive Analysis
10
Options “To Fix” the situation
11
Not always a happy ending 
Minute 90
Ball Possession: 80 : 20
Fouls: 4 : 25
Score: 3 : 0
12
FRUSTRATED FANS!!
12
13
How does that
relate to
Software?
1414
From Deploy to …
Deploy Promotion/Event Problems Ops Playbook War Room
Timeline
1515
The “War Room” – back then
'Houston, we have a problem‘
NASA Mission Control Center, Apollo 13, 1970
1616
The “War Room” – NOW
Facebook – December 2012
1717
Problem: Unclear End User Problem Descriptions
1818
Statuc Quo: Ops Runbook – System Unresponsive
1919
Problem: Unclear Ops Problem Descriptions
2020
Status Quo: Ops Runbook – High Resource Usage
2121
Lack of data?
2222
23
Answers to the right questions
2424
What are the real questions?
Individual Users? ALL users?
Is it the APP? Or Delivery Chain?
Code problem? Infrastruct...
2525
Problem: What Devs would like to have
2626
Problem: What Devs would like to have
Top Contributor is related to
String handling
99% of that time comes from
RegEx...
2727
Its getting this …Its like getting this …
28
… when you need to see this!
2929
Problem: Attitudes like this don’t help either
Image taken from https://www.scriptrock.com/blog/devops-whats-hype-abo...
3030
Problem: Very “expensive” to work on these issues
~80% of problems
caused by ~20% patterns
YES we know this
80%Dev Ti...
3131
TOP PROBLEM
PATTERNS
• Taken From Production
Environments
3232
Top Problem Patterns: Resource Pools
3333
Top Problem Patterns: Resource Pools
3434
Deployment Mistakes lead to internal Exceptions
3535
Deployment Mistakes lead to high logging overhead
3636
Production Deployment leads to Log SYNC Issues
3737
Long running SQL with Production Data
3838
N+1 Query Problem
4040
Memory Leaks in Cache Layer with Production Data
Still crashes
Problem fixed!Fixed Version Deployed
4242
BLOATED Web Sites
17! JS Files – 1.7MB in Size
Useless Information!
Even might be a security risk!
4343
Missing or incorrect configured browser caches
62! Resources not cached
49! Resources with short expiration
4444
SLOW or Failing 3rd Party Content
4545
Want MORE of these and more details?
http://apmblog.compuware.com
4646
Lots of Problems that could have been avoided
• BUT WHY are they still making it to Production?
4747
Missing Focus on Performance
4848
Different Goals for Dev and Ops
4949
Disconnected Teams despite “Shared Responsibility”
5050
Solution: DevOps + Performance Focus
5151
BEST PRACTICES
5252
Culture Become ONE Team
5353
Culture Testability
5454
Automate & Measure …Performance
5555
Automate & Measure …Scalability
5656
Automate Deployment
5757
How? Performance Focus in Test Automation
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPur...
5858
How? Performance Focus in Test Automation
Analyzing All Unit /
Performance Tests
Analyze Perf
Metrics
Identify
Regres...
5959
How? Performance Focus in Test Automation
Cross Impact of KPIs
6060
How? Performance Focus in Test Automation
Embed your Architectural
Results in Jenkins
6161
Share Tools
6262
Share Results
6363
Getting control over your weekend again …
Enjoy a beer with friends?
Instead of pizza and soda
with your colleagues?
64
6565
YOU HAVE TIME FOR THE REAL …
6666
DevOps Automation In-Action
• Automate Load Test Analysis and Regression Detection
6767
DevOps Automation In-Action
• Automate Load Test Analysis and Regression Detection
6868
DevOps: Actionable Data to Ops
• Input for Capacity and Deployment Planning
Number of Requests on The
App Server we w...
6969
IF WE DO ALL THAT
80% Dev Time for Bug Fixing
$60B Costs by Defects
7070
Want MORE of these and more details?
http://apmblog.compuware.com
71
© 2011 Compuware Corporation — All Rights Reserved
Simply Smarter
Upcoming SlideShare
Loading in …5
×

StarWest 2013 Performance is not an afterthought – make it a part of your Agile Delivery

2,059 views

Published on

This presentation was given at StarWest 2013 in Anaheim, CA and also broadcasted through the Virtual Conference.
It shows how important it is to focus on performance throughout continuous delivery in order to avoid the most common performance problem patterns that still cause applications to crash and engineers spending their weekends and nights in a firefighting/war room situation

Published in: Technology
  • Be the first to comment

StarWest 2013 Performance is not an afterthought – make it a part of your Agile Delivery

  1. 1. 11 Make it part of your Agile Delivery
  2. 2. 2
  3. 3. 3
  4. 4. 4
  5. 5. 5 Testing is Important – and gives Confidence
  6. 6. 6 But are we ready for “The Real” world?
  7. 7. 7 Measure Performance during the game Ball Possession: 40 : 60 Fouls: 0 : 0 Score: 0 : 0 Minute 1 - 5
  8. 8. 8 Measure Performance during the game Minute 6 - 35 Ball Possession: 80 : 20 Fouls: 2 : 12 Score: 0 : 0
  9. 9. 9 Deep Dive Analysis
  10. 10. 10 Options “To Fix” the situation
  11. 11. 11 Not always a happy ending  Minute 90 Ball Possession: 80 : 20 Fouls: 4 : 25 Score: 3 : 0
  12. 12. 12 FRUSTRATED FANS!! 12
  13. 13. 13 How does that relate to Software?
  14. 14. 1414 From Deploy to … Deploy Promotion/Event Problems Ops Playbook War Room Timeline
  15. 15. 1515 The “War Room” – back then 'Houston, we have a problem‘ NASA Mission Control Center, Apollo 13, 1970
  16. 16. 1616 The “War Room” – NOW Facebook – December 2012
  17. 17. 1717 Problem: Unclear End User Problem Descriptions
  18. 18. 1818 Statuc Quo: Ops Runbook – System Unresponsive
  19. 19. 1919 Problem: Unclear Ops Problem Descriptions
  20. 20. 2020 Status Quo: Ops Runbook – High Resource Usage
  21. 21. 2121 Lack of data?
  22. 22. 2222
  23. 23. 23 Answers to the right questions
  24. 24. 2424 What are the real questions? Individual Users? ALL users? Is it the APP? Or Delivery Chain? Code problem? Infrastructure? One transaction? ALL transactions? In AppServer? In Virtual Machine?
  25. 25. 2525 Problem: What Devs would like to have
  26. 26. 2626 Problem: What Devs would like to have Top Contributor is related to String handling 99% of that time comes from RegEx Pattern Matching Page Rendering is the main component
  27. 27. 2727 Its getting this …Its like getting this …
  28. 28. 28 … when you need to see this!
  29. 29. 2929 Problem: Attitudes like this don’t help either Image taken from https://www.scriptrock.com/blog/devops-whats-hype-about/ Shopzilla CIO (in 2010): “… when they get in the war room - the developers and ops teams describe the problem as the enemy, not each other”
  30. 30. 3030 Problem: Very “expensive” to work on these issues ~80% of problems caused by ~20% patterns YES we know this 80%Dev Time in Bug Fixing $60BDefect Costs BUT
  31. 31. 3131 TOP PROBLEM PATTERNS • Taken From Production Environments
  32. 32. 3232 Top Problem Patterns: Resource Pools
  33. 33. 3333 Top Problem Patterns: Resource Pools
  34. 34. 3434 Deployment Mistakes lead to internal Exceptions
  35. 35. 3535 Deployment Mistakes lead to high logging overhead
  36. 36. 3636 Production Deployment leads to Log SYNC Issues
  37. 37. 3737 Long running SQL with Production Data
  38. 38. 3838 N+1 Query Problem
  39. 39. 4040 Memory Leaks in Cache Layer with Production Data Still crashes Problem fixed!Fixed Version Deployed
  40. 40. 4242 BLOATED Web Sites 17! JS Files – 1.7MB in Size Useless Information! Even might be a security risk!
  41. 41. 4343 Missing or incorrect configured browser caches 62! Resources not cached 49! Resources with short expiration
  42. 42. 4444 SLOW or Failing 3rd Party Content
  43. 43. 4545 Want MORE of these and more details? http://apmblog.compuware.com
  44. 44. 4646 Lots of Problems that could have been avoided • BUT WHY are they still making it to Production?
  45. 45. 4747 Missing Focus on Performance
  46. 46. 4848 Different Goals for Dev and Ops
  47. 47. 4949 Disconnected Teams despite “Shared Responsibility”
  48. 48. 5050 Solution: DevOps + Performance Focus
  49. 49. 5151 BEST PRACTICES
  50. 50. 5252 Culture Become ONE Team
  51. 51. 5353 Culture Testability
  52. 52. 5454 Automate & Measure …Performance
  53. 53. 5555 Automate & Measure …Scalability
  54. 54. 5656 Automate Deployment
  55. 55. 5757 How? Performance Focus in Test Automation 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test Framework Results Architectural Data We identified a regresesion Problem solved Lets look behind the scenes Exceptions probably reason for failed tests Problem fixed but now we have an architectural regression Problem fixed but now we have an architectural regression Now we have the functional and architectural confidence
  56. 56. 5858 How? Performance Focus in Test Automation Analyzing All Unit / Performance Tests Analyze Perf Metrics Identify Regressions
  57. 57. 5959 How? Performance Focus in Test Automation Cross Impact of KPIs
  58. 58. 6060 How? Performance Focus in Test Automation Embed your Architectural Results in Jenkins
  59. 59. 6161 Share Tools
  60. 60. 6262 Share Results
  61. 61. 6363 Getting control over your weekend again … Enjoy a beer with friends? Instead of pizza and soda with your colleagues?
  62. 62. 64
  63. 63. 6565 YOU HAVE TIME FOR THE REAL …
  64. 64. 6666 DevOps Automation In-Action • Automate Load Test Analysis and Regression Detection
  65. 65. 6767 DevOps Automation In-Action • Automate Load Test Analysis and Regression Detection
  66. 66. 6868 DevOps: Actionable Data to Ops • Input for Capacity and Deployment Planning Number of Requests on The App Server we will need to handle Might need to tune GC Settings to reduce GC Overhead CPU is going to be tight with these machines – also impacted by GC Activity Input on Thread Pool Configuration Memory Usage for expected load still provides enough “headroom”
  67. 67. 6969 IF WE DO ALL THAT 80% Dev Time for Bug Fixing $60B Costs by Defects
  68. 68. 7070 Want MORE of these and more details? http://apmblog.compuware.com
  69. 69. 71 © 2011 Compuware Corporation — All Rights Reserved Simply Smarter

×