Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automated Performance Testing for Desktop Applications by Ciprian Balea

PowerPoint presentation by 3Pillar's Ciprian Balea, QA Lead, which was delivered at the Romanian Testing Conference (RTC) 2014 in Cluj-Napoca, Romania on May 15, 2014.

  • Login to see the comments

  • Be the first to like this

Automated Performance Testing for Desktop Applications by Ciprian Balea

  1. 1. Automated Performance Testing for Desktop Applications by Ciprian Balea
  2. 2. 2 THE SCARY QUESTION OF APPLICATION DEVELOPMENT © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential How does it perform in real conditions? Never as first expected!
  3. 3. 3 BENCHMARKING • What impact does the app have on the system running it? (is the system still reliable) • How does the app perform when running on the host system? (does it keep all functionality) © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Ideally, • consuming as little resources as possible without loosing functionality • the app should not break /slow the system down
  4. 4. 4 HOW TO PREDICT THE FUTURE? © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Run it multiple times and average out. A single run doesn’t accurately predict real life usage.
  5. 5. 5 HOW TO GET A CREDIBLE RESULT? © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Avoid inconsistencies induced by the outside world => no Internet connection, own webserver & assets
  6. 6. 6 WHAT’S OUR APPLICATION ABOUT? © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential A Custom Web Browser
  7. 7. 7 HARDWARE SETUP © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  8. 8. 8 BIG PROBLEM #1: HOW TO KNOW HOW MUCH TIME THE PAGE TOOK TO LOAD © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Based on the HTML content, loading time cannot be computed (element loading order is not consistent across multiple runs). Small solution:
  9. 9. 9 BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Small solution (a):
  10. 10. 10 BIG PROBLEM #2: HOW TO READ SYSTEM RESOURCE USAGE WITHOUT 3RD PARTY TOOLS © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Small solution (b):
  11. 11. 11 BIG PROBLEM #3: HOW TO MANIPULATE THE APPLICATION’S GUI © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Small solution:
  12. 12. 12 WHAT DATA WE COLLECT? • Test 1 • Webpage load time • CPU usage • Memory usage • Disk I/O • Test 2 • Timing a “complex” operation © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  13. 13. 13 TEST 1: READING SYSTEM STATISTICS © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  14. 14. 14 HOW IS THE DATA COLLECTED? © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Not at the start of the game Not at the end of the game Throughout the entire game
  15. 15. 15 HOW IS THE DATA COLLECTED? © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  16. 16. 16 RUNNING THE TEST • One test sequence: • clear cache • open the browser • start reading system data • load webpage • close the browser • save result in an Excel spreadsheet • Full test: • Restore the “clean” snapshot of the test environment • Automatically runs the test sequence for 1000 times on the native browser • Automatically runs the test sequence for 1000 times on the custom browser © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  17. 17. 17 RUNNING THE TEST © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  18. 18. 18 TEST 2: TIMING A “COMPLEX” OPERATION © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  19. 19. 19 “COMPLEX” OPERATION © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential Open the browser Load a webpage Bookmark page Go to homepage Open bookmark Wait for page to load
  20. 20. 20 RESULTS © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  21. 21. 21 RESULTS © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  22. 22. 22 CONCLUSIONS Performance testing is very important as it can reveal performance flaws. It is very time and effort consuming to do it manually. It can be automated with some initial effort, but the returned value is huge. Offers very conclusive results on what needs to be improved in the application to boost performance. © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  23. 23. 23 Q&A © Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
  24. 24. THANK S!Now let’s get to work! 24© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

×