Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Reliably Measure and Optimize Graphics Performance of Your Android Games

2,052 views

Published on

Watch a live presentation at http://offer.bitbar.com/measure-and-optimize-graphics-performance-of-android-games

Good graphics performance is closely related to fantastic user experience. That’s why we’ve integrated the unique GameBench profiling tool into Testdroid Cloud. This is a powerful resource for any developer who wants to spot performance bottlenecks and optimize their games across an array of Android devices with different silicon, screen sizes, memory, and many other variances in software and hardware.

Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/

Published in: Technology
  • Be the first to comment

How to Reliably Measure and Optimize Graphics Performance of Your Android Games

  1. 1. 11 August 2015 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com How to Reliably Measure and Optimize Graphics Performance of Your Android Games W E B I N A R Sharif Sakr Director of Business Dev. sharif@gamebench.net
  2. 2. Agenda • Testdroid Statistics from Q2'2015 • Testing Mobile Games – Why and How? • Optimize Graphics Performance with Help of Gamebench - Introduction • Demonstration • Q&A
  3. 3. TESTDROID STATISTICS
  4. 4. Q2'2015 – Test Run Stats
  5. 5. Why Mobile Games Fail?
  6. 6. iOS 8.0 iOS 8.0.2 iOS 8.1 iOS 8.1.1 iOS 8.2 iOS 8 ALL KitKat 4.4 KitKat 4.4.2 KitKat 4.4.3 KitKat 4.4.4 KitKat ALL Lollipop 5.0 Lollipop 5.0.1 Lollipop 5.0.2 Lollipop ALL Android 'M' 0 5 10 15 20 25 30 35 Failed test runs. Percentage (%).
  7. 7. Some Observations 1 2
  8. 8. Some Observations 3 4
  9. 9. Some Observations 5 6
  10. 10. Android Fragmentation 2015 by Open Signal report August 2015 Testdroid Cloud >95%
  11. 11. Android Fragmentation 2015 by Open Signal report August 2015 Testdroid Cloud + Gamebench >85%
  12. 12. How Many Devices is Enough? ~90% market coverage can be achieved with 128 devices ~20% market coverage can be achieved with 14 devices US Market 30 Android devices = ~2/3 market Global Market 80 Android devices = ~1/2 market
  13. 13. TESTING MOBILE GAMES
  14. 14. Correct behaviour across platforms and browsers Integration with web back- ends Typically need to fully utilize HW (CPU+GPU) Resource (e.g. battery) consumption OpenGL ES 2/3 Performance! Functionality and usability Screen orientations, connectivity, user profiles Robustness Robustness and security! Brand Compliances, verification with back-ends and data Different 'Verticals'
  15. 15. Manual Testing is not Agile
  16. 16. Test Automation is Agile
  17. 17. OS versions Chipsets CPU + GPU Tens of OEMs Memory Displays (resolutions, physical hw) OEM mods Other hardware (connectivity calibration) Relation to other software Where Test Automation Can Help
  18. 18. Why Real Devices Are Must-to-Have • Emulators cannot help you to test... • User Experience • Usability • Hardware • Software • Infrastructure 0 % = the percentage of your app users that use emulator to run your app!
  19. 19. Why Testing Games Matter? • Graphics performance capability is the most important thing for mobile games when forming UX • Too many games fail to work properly with low-end/mid- range devices due hardware performance • The real understanding of game's performance capability can be only figured out by testing on real devices Understanding How quickly gamer assimilates what the game is about, how to fully play with it, what are the functionalities. Usability The main User Interface, menus, design and how usable those are Playing Experience Intuition to follow the game and go forward with its instructions and game-play Monetization Are gamers willing to buy – invest – in game
  20. 20. Types of Game Testing • Functionality of the game-play, featuresFunctional Testing • Done always when new features /regressions are includedRegression Testing • How game runs on different configurationsCompatibility Testing • Different languages, geo-focused materialsLocalization Testing • Endurance test to determine if system can handle the loadSoak Testing • Measures the capacity of the systemStress Testing • Simplest form of performance testing, measures how system handles certain loadsLoad Testing • Isolation of the environment (e.g. from network) to see how game worksHermetic Testing Feature -based testing Performance testing End-user testing
  21. 21. Gameflow - Performance • Compared to regular app testing, mobile game testing may seem to be more challenging • e.g. OpenGL ES content not action-identifiable • Graphics and UI – Identical look&feel across all devices and variants • Usability controls in the context of game • Navigation/game-play, Progress, Delays between "game-stages" • Animations, Fonts, Other graphics assets • Screen Resolution and Orientation
  22. 22. • Studies: Majority of users leave game due sluggish, general slowness or other bad performance behavior • It's very important to understand how game behaves on different devices, under different circumstances Performance Makes +/- UX
  23. 23. Performance Testing • Performance, Load, Stress, Spike, Soak • Measure the real user experience • Execute tests with all possible devices (SW&HW) • Discover the impact of server loads, app performance (CPU/GPU utilization)
  24. 24. Components in Game Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database
  25. 25. Starts at Design Level App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Game Design, GFX Assets, Files, Optimization
  26. 26. Performance Testing w/HW App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Real Devices and Platform
  27. 27. Compatibility Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Variations
  28. 28. Reference Testing App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database
  29. 29. Back-End Performance App Middleware Platform Connectivity/Network Back-end GAME Game Engine Android iOS ... API & services Server Database Game with Back-End Integration / Google Play Services
  30. 30. Game Testing Process
  31. 31. Trends in Mobile Game Testing • Image recognition 15 seconds
  32. 32. GAMEBENCH
  33. 33. A traffic light system for mobile apps and devices
  34. 34. 50%
  35. 35. People notice performance
  36. 36. Performance = promotion
  37. 37. OEM and Chip makers Developers – Games, Apps, VR Journalists Testing / QA 40k+ downloads in less than 1 yr since launch, >250 users any day Market traction
  38. 38. App and Game Developers Chip makers OEM Consumers Journalists and Enthusiasts Used to certify performance before release Used by retailers to match consumers’ usage profiles to mobile devices Helps with device and app performance reviews
  39. 39. Copyright GameBench Limited Q&A
  40. 40. Who are the people behind GameBench •Sri Kannan Iyer: CEO & Founder (London) •Over 15 years experience in the mobile •ARM, MediaTek, Analog Devices and Intel •MSEE (UT Austin), MBA (Oxford) •4 patents, 1 book chapter, speaker at several conferences •Karthik Hariharakrishnan: CTO & Founder (Bristol) •Over 10 years in mobile engineering •ARM, Motorola, National Instruments •MSEE (University of Illinois), 3 patents Six employees based in Bristol & London Get in touch!
  41. 41. sharif@gamebench.net
  42. 42. Demonstration Gamebench with Real Devices from Testdroid Cloud
  43. 43. THANK YOU! More information about frameworks, devices and testing at www.testdroid.com

×