0
Wolfgang Gottesheim

1
2
3
4
Testing is Important – and gives Confidence

5
But are we ready for “The Real” world?

6
Measure Performance during the game

Minute 1 - 5

Ball Possession: 40 : 60
Fouls:
0:0
Score:
0:0
7
Measure Performance during the game

Minute 6 - 35

Ball Possession: 80 : 20
Fouls:
2 : 12
Score:
0:0
8
Deep Dive Analysis

9
Options “To Fix” the situation

10
Not always a happy ending 

Minute 90

Ball Possession: 80 : 20
Fouls:
4 : 25
Score:
3:0
11
Frustration

12
How does that
relate to
Software?
13
From Deployment to…
Deploy

14

Promotion/Event

Problems

Ops Playbook

War Room
The “War Room” – back then
'Houston, we have a problem‘
NASA Mission Control Center, Apollo 13, 1970

15
The “War Room” – NOW

Facebook – December 2012
16
Problem: Unclear End User Problem Descriptions

17
Problem: Unclear Ops Problem Descriptions

18
Status Quo: Problem – “War Rooms” create Tension

19
THE APP LIFECYCLE

20
Status Quo
Business
Requirements

Friction
Change
Unit
Testing

Development
21

Integration
Testing

Stability

Load/Perfo...
Operating in Silos: Different tools, different goals
Business

$$$

Requirements

Unit
Testing

Integration
Testing

Load/...
Feedback cycles – slow and tedious?
Business

$$$

Requirements
PDF Reports

Unit
Testing

Integration
Testing

Load/Perfo...
Status Quo
Business
Requirements

Confidence
Friction
Change
Unit
Testing

Development
27

Integration
Testing

Stability
...
Wouldn‘t we love one single solution?

Business
Production

Development
28

QA
What Devs would like to have

29
What Devs would like to have
Top contributor is save of
Mage_Core_Model_Abstract

70% of that time comes from
Sales_Model_...
Problem: Perception of each other …
What Ops thinks about Dev & Test …

…and how Ops looks for Dev and Test
31
Problem: “Walls” results in more Tension and Ignorance

32
Problem: Very “expensive” to work on these issues

YES we know this

BUT

~

80%
caused by

33

of problems

~20%

pattern...
TOP PROBLEM
PATTERNS
• Focus on Web and PHP

34
Top Problem Patterns: Does this sound familiar?
• Resource Pool Exhaustion
– Misconfiguration or failed deployment, e.g: d...
Top Problem Patterns: Does this sound familiar?
• Resource Pool Exhaustion (same as before – just different Pool)
– Using ...
Top Problem Patterns: Does this sound familiar?
• Deployment Issues leading to heavy logging resulting in high I/O and CPU...
Top Problem Patterns: Does this sound familiar?
• Too many and too slow Database Queries
– Dev and Test need to have “prod...
Top Problem Patterns: Does this sound familiar?
• Too much data requested from Database
– Dev and Test need to have “produ...
Top Problem Pattern: Oversized Pages
• Not following WPO (Web Performance Optimization Rules)
– Non optimized content, e.g...
Top Problem Pattern: Caches
• Not leveraging Browser-side Caching
– Misconfigured CDNs or missing cache settings -> automa...
Top Problem Pattern: 3rd Party Content
• Slow or failing 3rd party content
– Impacts page load time; Ops is required to mo...
Lots of Problems that could have been avoided
• BUT WHY are they still making it to Production?

43
Missing Focus on Performance

44
Different Goals for Dev and Ops

45
Disconnected Teams despite “Shared Responsibility”

46
50
How to make the Enterprise Crew happy?

51
52
Solution: DevOps + Performance Focus
Automation

Measurement

Measurement, Collaboration and Deployment

“Visible” KPIs fo...
Culture: EXTEND Requirements with …

Performance

Scalability

Deployability
54

Testability

Deployability
Sharing: DON’T EXCLUDE anyone from Agile Process

Stand-Ups

55

Sharing Tools

Feedback
Measurement: Define KPIs accepted by all teams

Perf Test Code Coverage

# of SQL Executions

Time for Rollback

Response ...
AUTOMATION, AUTOMATION, AUTOMATION

Performance

57

Scalability

Shared Tools

Automatic Feedback
DevOps Collaboration – TODO LIST FOR YOU!!
Shared Reporting and Task Management

Known How Exchange

Access to Production ...
Feedback cycles – fast and precise!
Business

Development
59

QA

Production
Feedback cycles – fast and precise!
Business

Development
60

QA

Production
Recap – Problem – Root Cause – Solution - Result
DevOps +
Performance Culture

Automation

Measurement

Collaboration

61
TIPS FOR DEVS

62
DevOps Automation in Action
Lets look behind the
scenes

• Most problems can be identified in CI

Architectural Data

Test...
Performance Focus in Test Automation

Analyzing All Unit / Performance Tests

Jump in DB Calls
from one Build to the
next
...
Performance Focus in Test Automation

Cross Impact of KPIs
69
Performance Focus in Test Automation

Embed your Architectural Results
in Jenkins

70
Performance Focus in Test Automation
Compare Build that shows BAD Behavior!

With Build that shows GOOD Behavior!

Here is...
Performance Focus in Test Automation

CalculateUserStats is the
new Plugin that causes
problems

72
Remember – DevOps requires Cultural Change

Share

73

Integrate

Collaborate

Performance
Thank You
Participate in Compuware
APM Discussion Forums
apmcommunity.compuware.com

Follow us on Twitter
twitter.com/Comp...
Simply Smarter

http://tinyurl.com/wpc13eval
75

© 2011 Compuware Corporation — All Rights Reserved
Upcoming SlideShare
Loading in...5
×

Lassen Sie sich nicht von schlechter Performance Ihr Wochenende verderben!

209

Published on

Wie oft sind Sie schon am Wochenende im Büro gesessen, um kurzfristig aufgetretene Probleme zu lösen? 80 Prozent der Probleme werden immer und immer wieder durch dieselben Ursachen ausgelöst (wie z. B. die fehlerhafte Verwendung/Konfiguration von Resource Pools, beim Deployment "vergessene" Änderungen oder Dateien, Memory Leaks, datenabhängige Performanceprobleme). In dieser Session zeigen wir, wie Sie diese „Stolpersteine“ in Ihrer Produktionsumgebung identifizieren können, welche Informationen Ihre Entwickler benötigen, um sie effizient beheben zu können, und wie Ihre Tester die Behebung automatisiert verifizieren können.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
209
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Lassen Sie sich nicht von schlechter Performance Ihr Wochenende verderben!"

  1. 1. Wolfgang Gottesheim 1
  2. 2. 2
  3. 3. 3
  4. 4. 4
  5. 5. Testing is Important – and gives Confidence 5
  6. 6. But are we ready for “The Real” world? 6
  7. 7. Measure Performance during the game Minute 1 - 5 Ball Possession: 40 : 60 Fouls: 0:0 Score: 0:0 7
  8. 8. Measure Performance during the game Minute 6 - 35 Ball Possession: 80 : 20 Fouls: 2 : 12 Score: 0:0 8
  9. 9. Deep Dive Analysis 9
  10. 10. Options “To Fix” the situation 10
  11. 11. Not always a happy ending  Minute 90 Ball Possession: 80 : 20 Fouls: 4 : 25 Score: 3:0 11
  12. 12. Frustration 12
  13. 13. How does that relate to Software? 13
  14. 14. From Deployment to… Deploy 14 Promotion/Event Problems Ops Playbook War Room
  15. 15. The “War Room” – back then 'Houston, we have a problem‘ NASA Mission Control Center, Apollo 13, 1970 15
  16. 16. The “War Room” – NOW Facebook – December 2012 16
  17. 17. Problem: Unclear End User Problem Descriptions 17
  18. 18. Problem: Unclear Ops Problem Descriptions 18
  19. 19. Status Quo: Problem – “War Rooms” create Tension 19
  20. 20. THE APP LIFECYCLE 20
  21. 21. Status Quo Business Requirements Friction Change Unit Testing Development 21 Integration Testing Stability Load/Performance Testing QA Production
  22. 22. Operating in Silos: Different tools, different goals Business $$$ Requirements Unit Testing Integration Testing Load/Performance Testing Logs, Exceptions, Stack Traces Throughput, Response Time CPU, Memory, Disk, Network Development QA Production 22
  23. 23. Feedback cycles – slow and tedious? Business $$$ Requirements PDF Reports Unit Testing Integration Testing Load/Performance Testing XML Report Reports, Log Files Development 26 QA Log Files Production
  24. 24. Status Quo Business Requirements Confidence Friction Change Unit Testing Development 27 Integration Testing Stability Load/Performance Testing Feedback QA Production
  25. 25. Wouldn‘t we love one single solution? Business Production Development 28 QA
  26. 26. What Devs would like to have 29
  27. 27. What Devs would like to have Top contributor is save of Mage_Core_Model_Abstract 70% of that time comes from Sales_Model_Quote Reverse call stack allows me to identify the entry point, e.g. index.php 30
  28. 28. Problem: Perception of each other … What Ops thinks about Dev & Test … …and how Ops looks for Dev and Test 31
  29. 29. Problem: “Walls” results in more Tension and Ignorance 32
  30. 30. Problem: Very “expensive” to work on these issues YES we know this BUT ~ 80% caused by 33 of problems ~20% patterns
  31. 31. TOP PROBLEM PATTERNS • Focus on Web and PHP 34
  32. 32. Top Problem Patterns: Does this sound familiar? • Resource Pool Exhaustion – Misconfiguration or failed deployment, e.g: default config from dev – Actual resource leak -> can be identified with Unit/Integration Tests 35
  33. 33. Top Problem Patterns: Does this sound familiar? • Resource Pool Exhaustion (same as before – just different Pool) – Using the same deployment tools in Test and Ops can prevent this – Testing with real load can detect that 36
  34. 34. Top Problem Patterns: Does this sound familiar? • Deployment Issues leading to heavy logging resulting in high I/O and CPU – Using the same deployment tools in Test and Ops can prevent this – Analyzing Log Output per Component in Dev prevents this problem 37
  35. 35. Top Problem Patterns: Does this sound familiar? • Too many and too slow Database Queries – Dev and Test need to have “production-like” database – Updates on a “Sample Databases” won’t show slow updates – Access patterns can be identified with Unit Tests 38
  36. 36. Top Problem Patterns: Does this sound familiar? • Too much data requested from Database – Dev and Test need to have “production-like” database – Otherwise these problem patterns can only be found in prod – Educate Developers on “the power of SQL” – instead of loading everything in memory and performing filters/aggregations/… in the App 39
  37. 37. Top Problem Pattern: Oversized Pages • Not following WPO (Web Performance Optimization Rules) – Non optimized content, e.g: compression, merging, … – Educate developers and automate WPO checks 17! JS Files – 1.7MB in Size Useless Information! Even might be a security risk! 40
  38. 38. Top Problem Pattern: Caches • Not leveraging Browser-side Caching – Misconfigured CDNs or missing cache settings -> automate cache configuration deployment – Educate developers; Educate testers to do “real life” testing (CDN, …) 62! Resources not cached 49! Resources with short expiration 41
  39. 39. Top Problem Pattern: 3rd Party Content • Slow or failing 3rd party content – Impacts page load time; Ops is required to monitor 3rd party services – Educate devs to optimize loading; Educate test to include 3rd party testing 42
  40. 40. Lots of Problems that could have been avoided • BUT WHY are they still making it to Production? 43
  41. 41. Missing Focus on Performance 44
  42. 42. Different Goals for Dev and Ops 45
  43. 43. Disconnected Teams despite “Shared Responsibility” 46
  44. 44. 50
  45. 45. How to make the Enterprise Crew happy? 51
  46. 46. 52
  47. 47. Solution: DevOps + Performance Focus Automation Measurement Measurement, Collaboration and Deployment “Visible” KPIs for each Team Automate Performance and Architectural Problem Detection Focus on Performance, Architectural and Deployment Measures Culture “Shared Responsibility” Agile Process for ALL Teams Performance as Key Requirement X-Team Collaboration and Education Sharing Expertise, Tool and Data Sharing “Easy” sharing of Performance, Deployment and Production Data http://www.opscode.com/blog/2010/07/16/what-devops-means-to-me/ 53
  48. 48. Culture: EXTEND Requirements with … Performance Scalability Deployability 54 Testability Deployability
  49. 49. Sharing: DON’T EXCLUDE anyone from Agile Process Stand-Ups 55 Sharing Tools Feedback
  50. 50. Measurement: Define KPIs accepted by all teams Perf Test Code Coverage # of SQL Executions Time for Rollback Response Times MBs / Uses # of Log Lines Time for Deployment 56
  51. 51. AUTOMATION, AUTOMATION, AUTOMATION Performance 57 Scalability Shared Tools Automatic Feedback
  52. 52. DevOps Collaboration – TODO LIST FOR YOU!! Shared Reporting and Task Management Known How Exchange Access to Production Data Diagnostic Tools Shared Performance KPIs and Tooling 58
  53. 53. Feedback cycles – fast and precise! Business Development 59 QA Production
  54. 54. Feedback cycles – fast and precise! Business Development 60 QA Production
  55. 55. Recap – Problem – Root Cause – Solution - Result DevOps + Performance Culture Automation Measurement Collaboration 61
  56. 56. TIPS FOR DEVS 62
  57. 57. DevOps Automation in Action Lets look behind the scenes • Most problems can be identified in CI Architectural Data Test Framework Results Build # Test Case Status Build 17 testPurchase OK 12 0 120ms testSearch OK 3 1 68ms testPurchase FAILED 12 5 60ms testSearch OK 3 1 68ms testPurchase OK 75 0 230ms testSearch OK 3 1 68ms Build 18 Build 19 Build 20 67 # SQL # Excep CPU 12 0 120ms Exceptions probably reason 3 1 68ms testSearch identified a regresesion OK We for failed tests Problem fixed but now we have an Problem solved architectural regression Now we have the functional and architectural confidence testPurchase OK
  58. 58. Performance Focus in Test Automation Analyzing All Unit / Performance Tests Jump in DB Calls from one Build to the next Analyzing Metrics such as DB Exec Count 68
  59. 59. Performance Focus in Test Automation Cross Impact of KPIs 69
  60. 60. Performance Focus in Test Automation Embed your Architectural Results in Jenkins 70
  61. 61. Performance Focus in Test Automation Compare Build that shows BAD Behavior! With Build that shows GOOD Behavior! Here is the difference! 71
  62. 62. Performance Focus in Test Automation CalculateUserStats is the new Plugin that causes problems 72
  63. 63. Remember – DevOps requires Cultural Change Share 73 Integrate Collaborate Performance
  64. 64. Thank You Participate in Compuware APM Discussion Forums apmcommunity.compuware.com Follow us on Twitter twitter.com/CompuwareAPM Like us on Facebook Read our Blog facebook.com/CompuwareAPM About:Performance Join our LinkedIn group Watch our Videos & product Demos Compuware APM User Group youtube.com/Compuware www.compuware.com/APM 74 © 2011 Compuware Corporation — All Rights Reserved
  65. 65. Simply Smarter http://tinyurl.com/wpc13eval 75 © 2011 Compuware Corporation — All Rights Reserved
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×