Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

SauceCon 2017: Mobile Test Pyramid Slide 1 SauceCon 2017: Mobile Test Pyramid Slide 2 SauceCon 2017: Mobile Test Pyramid Slide 3 SauceCon 2017: Mobile Test Pyramid Slide 4 SauceCon 2017: Mobile Test Pyramid Slide 5 SauceCon 2017: Mobile Test Pyramid Slide 6 SauceCon 2017: Mobile Test Pyramid Slide 7 SauceCon 2017: Mobile Test Pyramid Slide 8 SauceCon 2017: Mobile Test Pyramid Slide 9 SauceCon 2017: Mobile Test Pyramid Slide 10 SauceCon 2017: Mobile Test Pyramid Slide 11 SauceCon 2017: Mobile Test Pyramid Slide 12 SauceCon 2017: Mobile Test Pyramid Slide 13 SauceCon 2017: Mobile Test Pyramid Slide 14 SauceCon 2017: Mobile Test Pyramid Slide 15 SauceCon 2017: Mobile Test Pyramid Slide 16 SauceCon 2017: Mobile Test Pyramid Slide 17 SauceCon 2017: Mobile Test Pyramid Slide 18 SauceCon 2017: Mobile Test Pyramid Slide 19 SauceCon 2017: Mobile Test Pyramid Slide 20 SauceCon 2017: Mobile Test Pyramid Slide 21 SauceCon 2017: Mobile Test Pyramid Slide 22 SauceCon 2017: Mobile Test Pyramid Slide 23 SauceCon 2017: Mobile Test Pyramid Slide 24 SauceCon 2017: Mobile Test Pyramid Slide 25 SauceCon 2017: Mobile Test Pyramid Slide 26 SauceCon 2017: Mobile Test Pyramid Slide 27 SauceCon 2017: Mobile Test Pyramid Slide 28 SauceCon 2017: Mobile Test Pyramid Slide 29 SauceCon 2017: Mobile Test Pyramid Slide 30 SauceCon 2017: Mobile Test Pyramid Slide 31 SauceCon 2017: Mobile Test Pyramid Slide 32 SauceCon 2017: Mobile Test Pyramid Slide 33

YouTube videos are no longer supported on SlideShare

View original on YouTube

3

Share

Download to read offline

SauceCon 2017: Mobile Test Pyramid

Download to read offline

How does an organization effectively decide when to use browser emulation (in case of hybrid apps), mobile simulators/emulators and real devices? In this talk, Kwo Ding will review his mobile test pyramid approach and discuss best practices about when to use what. He will also talk about how to best structure and execute these tests.

Related Books

Free with a 30 day trial from Scribd

See all

SauceCon 2017: Mobile Test Pyramid

  1. 1. Kwo Ding Test Automation Consultant SauceCon 7 June 2017 Mobile Test PyramidUI integration testing for mobile
  2. 2. Kwo Ding Freelance Test Automation Consultant About Kwo Over 10 years experience in software testing Currently active in the financial industry Specialized in implementation of test (automation) strategies Experienced and certified Scrum Master, Product Owner and Lean Green Belt Specialized in automation of web, mobile and API testing Based in the Netherlands Who Am I? kwo@dingit.net https://www.linkedin.com/in/kwoding https://www.facebook.com/kwoding @dingkwo
  3. 3. Mobile Testing Challenges
  4. 4. Browsers Simulators/Emulators Real devices What are their strengths and how to apply?
  5. 5. Test Pyramid Unit Service UI Mike Cohn, Succeeding with Agile (2009)
  6. 6. Test Guidelines Test as isolated as possible Automate as much as possible Zero tolerance on flakiness Fast feedback (test as early as possible) Regression = broken build No risk = no test NFRs are just as important as funcAonals
  7. 7. Mobile Test PyramidTest Guidelines Test as isolated as possible Automate as much as possible Zero tolerance on flakiness Fast feedback (test as early as possible) Regression = broken build No risk = no test NFRs are just as important as funcAonals Real devices Simulators/Emulators Browsers
  8. 8. Browsers Mobile testing on desktop browsers
  9. 9. Pros and ConsUsage of desktop browser testing for mobile Matter of milliseconds to launch a browser, also headless possible Fast execution Easily set up 10+ browser instances per machine Scalable Ability to use browsers on different operating systems Cross platform Mobile simulation in desktop browsers is still using the desktop browser Mobile simulation uses desktop browser version No native keyboards, incoming calls etc. No native integration Incredibly fast, but it’s just not a device Just not a device…
  10. 10. Focus AreasDesktop Browser Testing Isolated browser tests performing full functional validations Resizing browsers and toggling user agents Functional system testing Responsive design Use equivalent desktop browsers No extensive visual checks because the rendering is different than actual devices Cross-browser Overall visual layout
  11. 11. Demo Browser Testing
  12. 12. So mobile testing on desktop browsers is great, right?
  13. 13. Mobile Simulation in Browser vs Device
  14. 14. Mobile Simulation in Browser vs Device
  15. 15. Mobile Simulation in Browser vs Device
  16. 16. Mobile Simulation in Browser vs Device
  17. 17. Simulators/Emulators Closer to the actual mobile experience
  18. 18. Pros and ConsUsage of mobile simulators/emulators Simulators/emulators are easy to set up, just download, install and run Easy to set up and free Virtualization means scalable and also running in parallel on one machine Scalable Ability to test native APIs such as incoming calls and GPS injection Native API integration Simulators are fast, because they only have to simulate the software part. Emulators based on the Intel architecture are fast. Simulators or Intel-based emulators are fast Manufacturer’s skins are available, but the device behavior is still based on stock Vanilla versions only CPU/memory usage of machine in case of simulators. Emulators try to simulate the hardware No real resource usage Connectivity with NFC, Bluetooth, network connections No real interoperability Emulators based on the ARM architecture are slow, which is actually the main architecture for Android devices Slow ARM-based emulators Contrast/brightness inaccurate in light/dark environment Inaccurate color display in light/dark Easy to debug simulator/emulators, already hooked up to the machine to access logs Debugging possibilities
  19. 19. Focus AreasSimulator/Emulator Testing Click paths throughout the application GPS injection, file attachments, incoming calls etc. Functional end-user flows Native API integration Emulators are limited to vanilla versions Visuals (vanilla only) Touch interactions such as swipe and tap comes closer to the user experience of a device than browser emulation Touch interactions
  20. 20. Demo Simulator/Emulator Testing
  21. 21. Real Devices The real thing…
  22. 22. Pros and ConsUsage of real devices Ability to test native APIs not only with injections for automation, but also actual NFC touch for example Native APIs in real conditions Some real devices are just faster than emulators due to the simulation of hardware, especially compared to the ARM-based emulators Can be faster than emulators Real devices come with a price, usually you pay per device/cradle Costs A new device is usually not available on-the-fly, even with cloud solutions New device means procurement Actual network conditions, battery/CPU/memory usage, actual manufacturer’s sauce on top of the OS Just the real thing… iOS apps need to be signed with Development Distribution Certificate and Provisioning Profile for automation Development iOS build required for automation
  23. 23. Focus AreasReal Device Testing Validating usability such as actual click areas, touch actions and voice over CPU/memory usage, battery, network strengths Interruption (incoming calls, push notifications), resource fighting (camera, GPS), NFC, Bluetooth Usability Performance Native API integration Focus on devices which are not available as simulators/emulators Real OS from manufacturers With specific manufacturer’s OS it comes with built-in browsers which are different than the standard and well-known browsers such as Chrome Visuals Manufacturer’s sauce Specific built-in browsers
  24. 24. Demo Real Device Testing
  25. 25. Best practicesMobile Testing Use native gestures for usability testing such as touch and tap Scroll elements into view before clicking Use native keyboards instead of injecting text or “sendKeys” Webview ≠ mobile browser rendering Take technical click areas into account (“fat finger click”) Launch a “clean” device before test Generic Automation
  26. 26. Real devices: real-life conditions Performance testing, native API integration such as incoming calls, GPS, background apps, NFC etc. Real devices: visual and usability testing Visual validations and usability flows Simulators/Emulators: functional end-user flows and visuals Fast feedback on end-user flows with touch interactions and visuals Browsers: system testing and responsive design In case of hybrid and web apps: Responsive design and system testing Mobile Test Pyramid Real devices Simulators/Emulators Browsers
  27. 27. Contact Kwo Ding Freelance Test Automation Consultant kwo@dingit.net https://www.linkedin.com/in/kwoding https://www.facebook.com/kwoding @dingkwo
  • mayukhg

    Oct. 16, 2019
  • OlenaKosiakevych

    Oct. 11, 2017
  • ssuser4f9d1b

    Jun. 23, 2017

How does an organization effectively decide when to use browser emulation (in case of hybrid apps), mobile simulators/emulators and real devices? In this talk, Kwo Ding will review his mobile test pyramid approach and discuss best practices about when to use what. He will also talk about how to best structure and execute these tests.

Views

Total views

1,337

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

30

Shares

0

Comments

0

Likes

3

×