Achieving 100% mobile test coverage perfecto mobile


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Achieving 100% mobile test coverage perfecto mobile

  1. 1. Achieving 100% Test Coverage for Your Mobile Footprint – Issues, Options and Strategies An InfoStretch and Perfecto Mobile Webinar November 12th, 2013
  2. 2. Agenda Panelists Introduction Mobile Testing Challenges How Can You Optimize Quality for the Mobile App Lifecycle Planning Definition Execution Automation How Does Perfecto’s Mobile Cloud Solves these challenges Q&A
  3. 3. Introduction to Panelists Manish Mathuria Eran Kinsbruner Founder and CTO at InfoStretch Corporation Director of Product Marketing at Perfecto Mobile
  4. 4. InfoStretch - The Mobile Application Lifecycle Enablers Matured Technology to Support Lifecycle Needs Mobile Applications Requirements Sustained User Creative Design Experience and Development Performance and Scalability MobiXpert Optimize Quality Attesto Elemento Qmetry Synchro
  5. 5. Mobile Services – Custom Services for Custom Needs at Your Fingertips Mobile Strategy Alignment of Mobility Goals with Business Strategy Experience with a Vast Range of Operating Systems and Form Factors SLA Driven Delivery with 30% Reduction in Time to Market Powered by: Mobile Development Specialized mobile solutions in retail, travel, digital entertainment and hospitality Reusable tools, frameworks and technology platform for easier maintenance Developed and deployed over 300 apps in the app stores worldwide Mobile QA Mobile Sustenance Private Cloud with 3000+ Devices 30 Day SLA for New Device and OS Upgrades 200,000+ Apps Tested with1500+ Device Test Cases Real Time App Store Review Tracking, Remediation and Verification Readymade Test Packs in Retail, Banking, Travel, Hospitality and Entertainment Proactive Identification of User Experience Related Issues with Intelligent Heuristics
  6. 6. About Perfecto Mobile 6
  7. 7. Mobile Testing is NOT Easy More Apps Shorter release Iterations New devices Growing Team
  8. 8. Android Fragmentation and Portability 11,868 distinct Android devices, up from 3,997 Android devices last year
  9. 9. Delivering Quality in a Complex World is Hard OS Style Screen Version Device 9
  10. 10. Mobile Quality IS a critical Challenge… Mobile is a Market Platform
  11. 11. Plans to Scale Success! That won’t happen again … How do we manage this? V 1.0 Let’s get it out the door … KISS Disappointment! Read Only Complex Txns Quality Forget KISS let’s Scale Time V 2.0
  12. 12. Phases of Mobile Quality Device Selection - Which OS/Device Combination is Ideal for My App? Planning Definition Execution Types of Tests - Which Ones Provide me High Coverage? Processes - How Do I Manage All the Testing Processes? How Do I Handle Automation? Devices - How Do we Acquire Them? And Data Plans? Remote Access - How Do I Do This Securely and Exclusively Conditioning - How Do I Expose the Device to Potential Defects? Automation – Is there ROI? Automation – Which Tools Automation Automation – Which Test Cases? Release & Sustenance Post Release Support - How do I Manage the New OS’s and Devices Identifying Defects - How do I Respond and Identify Defects to Ensure High Customer Satisfaction? End User Feedback and Usage – How do I Monitor this from App Stores
  13. 13. Planning – Device Selection Criteria Objective: Maximum Coverage at Minimum Cost (&time) Type of the App User Personas Geography • Games • Social • Business • Business • Teen • Social • Asia Pac • Americas etc. App Functions • Streaming • Screen Res • GPS etc. Device Popularity Form Factor OS Output: Device/OS/Test Matrix
  14. 14. Definition – With Test Management Tools In-Depth Test Management Solution CONTROL HUB FOR QA Integrates with complementary ALM tools, Simple and Easy to use
  15. 15. Execution - Mobile Labs Geographies Languages 3000 + Devices • Americas • Europe • Asia Pac • EFIGS, Arabic, Japanese, Chinese, Indian languages • Data Plans • Global Networks
  16. 16. Mobile Private Cloud Test Lab Secured, Dedicated, Automated Private Cloud Client A Private Cloud Client B Device Cradle      Device Docking Module Remote access for verification Certification Guidelines Audit trail for certification Automated Tests/Tasks Seamless provision of new devices  Readily available and deployable lab ( Cap-Ex Savings)  Capability to test for global coverage (Reduce Time to market)  Access to and testing on pre-market devices (Enhances competitiveness)
  17. 17. Mobile Testing and Automation Mobile HTML5 Based Automation Native Platform Automation Technologies Platform Independent Mobile Automation
  18. 18. Typical Challenges 1 How to Select Test Cases and Devices for Mobile Automation? 2 Scripting Challenges and Dealing with Fragmentation 3 Test Execution Challenges
  19. 19. Test Case Selection Tool and Resource Capability • System/Peripheral Interaction • Interaction between multiple apps/ OS-App /Multi Domain • Location aware, peripheral, battery life, system alerts etc. Stability/Change • Best Practices for other GUI Automation apply • UI components that will change less. • Clear understood business processes Capability/ Complexity / Priority • Less graphically intensive , more data and business scenarios • Medium complexity test cases - > High - > Low • Smoke - > High - > Medium - > Low
  20. 20. Scripting Challenges and Fragmentation
  21. 21. Widgets Render Different – Same OS
  22. 22. Inherent Device/Platform Differences
  23. 23. Form Factor Challenges Droid X2 Droid Charge iPhone 4
  24. 24. Dealing with Fragmentation Browser Open Browser Go To Sign On QTP/Selenium Make Payment Text Verify Confirm Payment Sign Off Make Payment Test case Address ATM Search Browser Open Select Acct View Schedule Payment Sync Text Quick Links Nick Name Browser Go To Sign On Sign Off ZIP ATM Search Make Payment Sync Image Confirm Payment Component Library Framework iOS Devices Android HTC Aria Android HTC Droid BB Torch BB 8520 Device Implementations for Sign On (10+) LG CU620 LG VX9000
  25. 25. Develop a Strong Test Recovery Automation Tests will break due to • • • • • Device characteristics Test script errors Automation tool instability Device instability Data instability Invest in Robust Test Recovery System • Develop agents that can restart test components on devices e.g. iwebdriver • Enhance test case recovery, capability to restart test case. Invest in Logging from the test cases • Details trace of the test • Immediately tells you the failures
  26. 26. Automation Tool Selection Criteria Type of Application Sharing/collaboration based decision Multi - Native, Hybrid, Web based Cloud (Public/Private), HW Dependency (Local) Are we using the right Automation tool? Technologhy based decision Integration to QC, Script support, Emulator support, DOM/OCR Object Recognition (Hybrid) Automation is a key for both Functional and Non Functional Testing HW based decision Cross Platform support, Device variations
  27. 27. Performance & End-User Experience – Key for Success Per Device version Pre-release testing (iOS7 Beta) Apple Bug Reporter Source: LINK Android defect database
  28. 28. Invest in Continuous Integration What is CI? • Automate execution of test cases • Integrated automated reporting, and notifications • Prominent tools – Jenkins, hudson Why CI? • There is no Automation until its Automated  • Well justified effort, easy ROI, pays off in days • Automated trending and device comparisons. Invest in a Test Automation lab of devices • Dedicated automation devices
  29. 29. Perfecto Mobile Continuous Integration Workflow Submit  Build  Test …. In Minutes vs. Hours
  30. 30. Demo
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.