Download to read offline
![Why bugs occur?
Android Apps
Application Framework
Inter-Process Communication Proxies
Android System Services and Mangers
Hardware Abstraction Layer
Linux Kennel (With Hardware Drivers)
]
]
Higher Level
System
Lower Level
System](https://image.slidesharecdn.com/optimizingdeviceandtestcoveragewebinar-190306050031/85/Optimizing-device-and-test-coverage-webinar-18-320.jpg)
Uploaded bypCloudy
Optimizing device and test coverage webinar
The document discusses strategies for optimizing device and test coverage in the context of increasing device fragmentation, particularly within the Android ecosystem. It outlines methods for selecting devices based on market penetration and diversity, while also highlighting the potential limitations of focusing solely on popular devices. Additionally, it provides insights into common issues that arise from device-specific factors and emphasizes the need for data-driven testing approaches.
Optimizing device and test coverage webinar
- 1. How to optimizeDevice and Test Coverage?
- 2. Agenda The ever growingdevice fragmentation and its effect Traditional methods to devise device & test coverage What’s the surprise @ pCloudy ?
- 3. 24,093 distinct Androiddevices
- 4. The ever-growing versionsof Android Oreo moves up to 21.5 percent from 19.2 percent. This along with a small drop in Marshmallow’s numbers brings last year’s version of Android to just 6.7 percent below Nougat.
- 5. Let’s do somemath! • 16 device display categories • 20 different common resolutions • 15 Android versions • 400 Android device Manufacturers • 4 Major cellular networks • 16 x 20 x 15 x 400 x 4 = 768000 permutations Is it feasible?
- 6. Devising the right device matrix Basedon Market Penetration By using Purchased data By using Analytics Based on Device Diversity How do you find your right device coverage?
- 7. Selecting Devices basedon Market Penetration Determine the set of devices with your target operating system that will have the highest occurrence of accessing your application. For Example- > What will you prioritize if you’ve more number of Samsung & Xiomi users than iPhone users?
- 8. Optimizing coverage throughDevice Planner
- 9. Market Share Approachfor Consumer Apps Pros You can typically gather approximate market share data and utilize it to prioritize your test device list You choose devices with the highest market share and test on as many as you have time and resources for. Cons You could be missing a whole class of devices that are less popular but might hold a significant portion of the market. You could be wasting resources testing on two devices that are very similar Purchased Data Analytics Source:
- 10. Selecting Devices basedon Device Diversity Quad Core or above CPU RAM >=3 GB Display >= 5“ Retina, Full-HD display Latest OS Version Dual Core CPU RAM <=1 GB Screen size <= 5“ No Retina or Full HD Display OS less than 1 year Single Core CPU RAM up to 1GB Display size <4” Low screen resolution Older OS older browser High End Devices Medium Range Devices Slow Devices A B C
- 11. Group Google Pixel 3OnePlus 6 Samsung S9+Vivo 9 Oppo F7 RAM CPU Display OS 4GB 5.7" 5.5" 8GB 5.8" 4GB 6.3" 4GB 6.3" Android 8 4GB Octa core Octa core Quad core Octa core Quad core Android 9 Android 8 Android 8 Android 9 A
- 12. Group B Galaxy CoreI8262Sony Xperia SL(LT26II) Micromax Canvas Superfone Asus Zenfone4 Oppo Joy RAM CPU Display OS 1GB 4.3" 4.3" 1GB 5.0" 1GB 4.0" 1GB 4.0" Android 4.4 1GB Dual core Dual core Dual core Dual core Dual Core Android 4.1.2 Android 4.0.4 Android 4.0.4 Android 4.4
- 13. Group C Sony XperiaTipo HTC A310E Explorer LG Optimus Net Samsung Galaxy Chat RAM Processor Display OS 512 MB 850 MHz 3.2" Android 4.0 Android 2.3 Android 4.0 3.0" 512 MB 3.2" 850 MHz 650 MHz 512 MB512 MB 3.2" 800 MHz Android 2.3
- 14. Optimizing your test& device coverage the pCloudy way!
- 15. Terminologies Manifest File: Everyproject in Android includes a manifest file stored in the root directory of its project hierarchy. What information do you get here? •Key information about the app without having to read other files in the package or run the application. •Defines which components of the application will be accessible externally to other apps, and how those components interact with other apps. •Declares which components and resources (icons, text strings, etc.) to use in which cases, in particular with relation to other apps.
- 16. Terminologies Target API Level:The Target Framework (also known as compileSdkVersion ) is the specific Android framework version (API level) that your app is compiled for at build time. • This setting specifies what APIs your app expects to use when it runs. • It has no effect on which APIs are actually available to your app when it is installed Android versions - Each release of Android goes by multiple names: • The Android version, such as Android 9.0 • A code (or dessert) name, such as Pie • A corresponding API level, such as API level 28
- 17.
- 18. Why bugs occur? AndroidApps Application Framework Inter-Process Communication Proxies Android System Services and Mangers Hardware Abstraction Layer Linux Kennel (With Hardware Drivers) ] ] Higher Level System Lower Level System
- 19. What does ouranalysis say? Type Root Cause Percentage of Issues Issue Examples Device - Specific Problematic Driver Implementation 56% OS Customization 36% Peculiar hardware composition 20% Non-device Specific Android Platform API evolution 67% Original Android System bugs 12%
- 20. What does ouranalysis say?
- 21. • Let’s nottest…let’s predict…
- 22. The path ahead! •The digital age force us to test smarter! Optimizing test & device coverage is the first step. • Effective use of Data Analysis to take testing to the next level • pCloudy is at the forefront of innovation to keep you ahead
- 23.
Editor's Notes
- #2 Thanks for introduction. Good morning or good afternoon depending on which part of world you are in. It’s a real pleasure to be part of group of experts and a global audience. Thanks a lot for Joining in. Let me begin with a brief intro about myself. My name is Avinash. I am a co-founder of pCloudy.com, one the fastest growing Mobile App Testing platforms. So I hope you are settled and we are ready to roll.
- #3 I like solving problems of Testing teams and we created pcloudy.com with a goal to simplify lives or let me be precise “Testing lives of” of App developers and Testers. And we have made some exciting progress. So, today I am going to focus on three key areas Digital Transformation and world of Apps Quality @ speed and what can do it beyond what you would have already seen Lastly, how pcloudy helps you take it beyond.
- #5 Here is a checklist most organizations use to consider themselves agile. If they have moved some dev project from waterfall to agile or have daily standup meetings, or have got a scrum master or think about CD all the time they think they are agile. But the real question is Are they really agile? Let’s see further.
- #8 For Example- if you support both Android and iOS, and your application will be used across millions of Google Nexus and Moto G devices but only thousands of iPhones, you prioritize testing on the Google Nexus and Moto G above the iPhone device. So This test plan will consist of testing on devices which are prioritized by your market analysis.
- #9 For Example- if you support both Android and iOS, and your application will be used across millions of Google Nexus and Moto G devices but only thousands of iPhones, you prioritize testing on the Google Nexus and Moto G above the iPhone device. So This test plan will consist of testing on devices which are prioritized by your market analysis.
- #10 With the help of Market share approach you can typically gather approximate market share data and utilize it to prioritize your test device list. You choose devices with the highest market share and test on as many as you have time and resources for. So if your QA cycle allows for testing across 10 devices, you test on the 10 devices with the highest market share. Disadvantage The Market Share Approach ensures that you test on the most popular devices. However, by focusing only on market share, you could be missing a whole class of devices that are less popular, but when combined, still comprise a significant portion of the market. And you could be wasting resources testing on two devices that are very similar—your application is extremely likely to work on both of these devices if it works on either one. The next section describes an alternative approach that takes into account both market share and device characteristics to optimize your test strategy.
- #13 With the help of Market share approach you can typically gather approximate market share data and utilize it to prioritize your test device list. You choose devices with the highest market share and test on as many as you have time and resources for. So if your QA cycle allows for testing across 10 devices, you test on the 10 devices with the highest market share. Disadvantage The Market Share Approach ensures that you test on the most popular devices. However, by focusing only on market share, you could be missing a whole class of devices that are less popular, but when combined, still comprise a significant portion of the market. And you could be wasting resources testing on two devices that are very similar—your application is extremely likely to work on both of these devices if it works on either one. The next section describes an alternative approach that takes into account both market share and device characteristics to optimize your test strategy.