Performance Metrics for your Build Pipeline - presented at Vienna WebPerf October Meetup
•Download as PPTX, PDF•
2 likes•1,400 views
Software Performance Metrics that you should look at throughout your Build Pipeline and not just when your app crashes in productiong. Find performance and scalability problems as soon as executing your first Unit Test. Simply focus on metrics such as #SQLs, #LogMessages, #Objects on Heap, ...
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Performance Metrics for your Build Pipeline - presented at Vienna WebPerf October Meetup
Similar to Performance Metrics for your Build Pipeline - presented at Vienna WebPerf October Meetup (20)
More from Andreas Grabner
More from Andreas Grabner (20)
Recently uploaded
Recently uploaded (20)
Performance Metrics for your Build Pipeline - presented at Vienna WebPerf October Meetup
- 1. 1 Checking Performance Along your Build Pipeline @grabnerandi http://apmblog.compuware.com http://bit.ly/dttrial
- 2. 2 Commit Stage • Compile • Execute Unit Test • Code Analysis • Build installers Automated Acceptance Testing Automated Capacity Testing Manual testing • Key showcases • Exploratory testing Release
- 3. 3 Who is doing it? How many successful deployments can they do? 300 Deployments / Year 10+ Deployments / Day 50-60 Deployments / Day Every 11.6 seconds
- 4. 4 More on Amazons Story Deploying every 11.6s 75% fewer outages since 2006 90% fewer outage minutes ~0.001% of deployments cause a problem Instantaneous automatic rollback
- 5. Recommended Book https://itrevolution.wufoo.com/forms/phoenix-project-ebook-offer/ 5 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
- 6. Status Quo: Unreliable Software Impacts Business 6 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
- 7. The “War Room” – back then 'Houston, we have a problem‘ NASA Mission Control Center, Apollo 13, 1970 7
- 8. The “War Room” – NOW Facebook – December 2012 8
- 9. Status Quo: Bugfixing in Production happens YES we know this 80%Dev Time in Bug Fixing $60B Defect Costs BUT ~80% of problems caused by ~20% patterns 9 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
- 10. 10 4 Situations on WHY this happens, HOW to avoid it
- 11. 11
- 12. “Blindly” (Re)use 12 Existing Components
- 13. 13 Requirement: We need a report
- 14. 14 Using Hibernate results in 4k+ SQL Statements to display 3 items! Hibernate Executes 4k+ Statements Individual Execution VERY FAST But Total SUM takes 6s
- 15. 15 Requirement: We need a fancy UI
- 16. 16 Using Telerik Controls Results in 9s for Data-Binding of UI Controls #1: Slow Stored Procedure Depending on Request execution time of this SP varies between 1 and 7.5s #2: 240! Similar SQL Statements Most of these 240! Statements are not prepared and just differ in things like Column Names
- 17. 17 Metrics: # Total SQLs # SQLs / Web Request # Same SQLs / Request Transferred Rows Dev: “Learn” Frameworks Test: With realistic Data
- 18. 18
- 20. 20 Business Impact requires Action!
- 21. 21 Solution: Cache to the RESCUE!!
- 22. 22 Implementation and Rollout Implemented InMemory Cache Worked well in Load Testing
- 23. 23 Result: Out of Memory Crashes!! Still crashes Fixed Version Deployed Problem fixed!
- 24. 24 Metrics: Heap Size, # Objects Allocated, # Objects in Cache Cache Hit Ratio Test: With realistic Data
- 25. 25
- 26. 26 #No “Agile” Deployment
- 27. 27 Load Spike resulted in Unavailability Ad on air
- 28. 28 Alternative: “GoDaddy goes DevOps” 1h before SuperBowl KickOff 1h after Game ended
- 29. 29 Behind the Scenes
- 30. 30 Metrics: Availability Page Size, # Objects # Hosts, # Connections DevOps: “Feature” Switches
- 31. 31
- 32. 32 #Push without a Plan
- 33. 33 Mobile Landing Page of Super Bowl Ad 434 Resources in total on that page: 230 JPEGs, 75 PNGs, 50 GIFs, … Total size of ~ 20MB
- 34. 34 m.store.com redirects to www.store.com ALL CSS and JS files are redirected to the www domain This is a lot of time “wasted” especially on high latency mobile connections
- 35. 35 Metrics: Load Time, # Resources (Images, …), # HTTP 3xx, 4xx, 5xx Dev: Build for Mobile Test: Test on Mobile Ops: Monitor Mobile
- 36. 36
- 37. # of Requests / User # of Log Messages # of Exceptions 37 Page Size # Objects In Cache # Objects Allocated Cache Hit Ratio # of Images # of SQLs Availability # SQLs per Request # HTTP 3xx, 4xx
- 38. 38 Commit Stage • Compile • Execute Unit Test • Code Analysis • Build installers Automated Acceptance Testing Performance Tests Automated Capacity Testing Manual testing • Key showcases • Exploratory testing Release Unit & Integration Tests Functional Tests Production Monitoring
- 39. How? Performance Focus in Test Automation Analyzing All Unit / Performance Tests Analyze Perf Metrics Identify Regressions
- 40. How? Performance Focus in Test Automation
- 41. How? Performance Focus in Test Automation Identify Regressions Cross Impact of KPIs
- 42. 4242
- 43. 43
- 44. 44 If we do all that
- 45. Want MORE of these and more details? http://apmblog.compuware.com 45
- 46. 46 FREE Products & More Info •dynaTrace Free Trial – 30 Days Free Trial on http://bit.ly/dttrial – After that totally free for local usage •Our Blog: http://apmblog.compuware.com •Follow Me: @grabnerandi
Editor's Notes
- Web Architectural Metrics # of JS Files, # of CSS, # of redirects Size of Images