T10
Mobile Testing
5/2/2013 11:15:00 AM

Mobile Testing Methodologies: Trends,
Successes, and Pitfalls
Presented by:
Eran ...
Eran Kinsbruner
Eran Kinsbruner is director of product marketing at PerfectoMobile, one of the leading mobile cloud and
au...
4/16/2013

Mobile Testing Methodologies Trends,
Successes, and Pitfalls
StarEast - 2013
Enabling Enterprise mobility
Exten...
4/16/2013

∙
∙
∙
∙
∙

Founded 2006
80+ employees
Office in Boston, London, Israel
Over 600 customers
Fortune 500 enterpris...
4/16/2013

Global Mobile Market Stats

Source: nielsen, 2012

Mobile OS market expected
to grow (even more)
FireFox OS

PE...
4/16/2013

Cost Of Mobile Quality Rises

Mobile accounts for 11% of U.S e-Commerce

Existing Mobile OS’s – End of 2012

So...
4/16/2013

Worldwide smartphone vendors Market share

Source: IDC Worldwide Mobile Phone Tracker, January 24, 2013

Market...
4/16/2013

Market trends - 2013
Mobile OS battle continues
•New open source competitors trying to influence the eco system...
4/16/2013

World Quality report 2012-2013
2012Mobile Testing is Behind the Curve

Source: CapGemini

Mobile Environment
Fr...
4/16/2013

Countless screen resolutions

Source: opensignalmaps, 2012 (Android only)

Consumer Reaction – Less Tolerant th...
4/16/2013

HTML5 Trend Involves Risks
To help protect y our priv acy , PowerPoint prev ented this external picture from be...
4/16/2013

Mobile Introduces various challenges
Performance
Automation

AGILE ALM

Functional

Native

Hybrid

Web

Applic...
4/16/2013

Mobile OS Regression bugs – What does it have to do
with my Mobile Testing?

Per Device version
Pre-release tes...
4/16/2013

Apple iOS – UI Guidelines

Source:
Source:
http://developer.apple.com/library/ios/#
http://developer.apple.com/...
4/16/2013

Risks Management

Mobile Quality Risk Analysis

Cost of quality in mobile
grows and impacted by
• Availability
...
4/16/2013

Mobile end-user Less Tolerant
end-

Listen to the
market
Obviously someone forgot
to do their regression
testin...
4/16/2013

Device governance
A clear process as part of the “mobile project” lifecycle
Define platforms and categories
Def...
4/16/2013

Enterprises OWNS the risk for Mobile quality
Mobile Malware - Trojans, viruses and rootkits migrating from trad...
4/16/2013

Mobile Is Mandated To Strict Regulations

Source: LINK

Source: Link

Mobile Testing Best Practices

Perfecto M...
4/16/2013

This is not a mobile test lab!
10 Devices
3 Extra Devices

Many Cables

3 Workstations

One Chair

Building Blo...
4/16/2013

World Quality report 2012-2013
2012-

Source: CapGemini

Building Blocks Of Mobile Testing Strategy
Consider ou...
4/16/2013

(True) Mobile
Cloud
Avoid device
management

Real Devices
+ Emulators
Real
smartphones

Manual +
Automation

AL...
4/16/2013

END

Perfecto Mobile

21
Upcoming SlideShare
Loading in …5
×

Mobile Testing Methodologies: Trends, Successes, and Pitfalls

828 views
629 views

Published on

In today's dynamic mobile marketplace—where new handsets and mobile operating systems are released every day—your ability to deal with these changes which impact your mobile product is vital. The mobile application lifecycle today must be short; must be of great quality; cover a myriad of handsets with different sizes, layouts, and enhanced capabilities; and, of course, cover as many operating systems as possible. This lifecycle requires a new methodology and approach. Eran Kinsbruner describes the mobile project challenges and provides real life examples of ways to overcome them. Take back the main mobile market trends and forecasts together with the key automation tools available for your use today. Learn the differences between the various mobile cloud and automation tools to help you select the right tool for your project. See how you can ramp up a successful mobile project, avoid the common pitfalls, and shorten the time to market—all while delivering a top-notch quality product.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
828
On SlideShare
0
From Embeds
0
Number of Embeds
58
Actions
Shares
0
Downloads
54
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Mobile Testing Methodologies: Trends, Successes, and Pitfalls

  1. 1. T10 Mobile Testing 5/2/2013 11:15:00 AM Mobile Testing Methodologies: Trends, Successes, and Pitfalls Presented by: Eran Kinsbruner Perfecto Mobile Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Eran Kinsbruner Eran Kinsbruner is director of product marketing at PerfectoMobile, one of the leading mobile cloud and automation companies. Formerly CTO for mobile testing and Texas Instruments project manager at Matrix, Eran has been in testing since 1999 with experience that includes managing teams at Qulicke & Soffa, Sun Microsystems, General Electric, and NeuStar. The co-inventor of a test exclusion automated mechanism for mobile J2ME testing at Sun Microsystems, Eran has experience in the mobile testing world. You can find Eran on Facebook, Twitter @ek121268, LinkedIn, and his professional mobile testing blog at ek121268.wordpress.com.
  3. 3. 4/16/2013 Mobile Testing Methodologies Trends, Successes, and Pitfalls StarEast - 2013 Enabling Enterprise mobility Extending Software Quality to Mobile Perfectomobile Mobile Testing Center of Excellence Group Agenda About me and my company Mobile Market Trends Mobile Quality Trends Mobile Quality Challenges Mobile Risk Management Mobile Testing Best Practices Perfecto Mobile 1
  4. 4. 4/16/2013 ∙ ∙ ∙ ∙ ∙ Founded 2006 80+ employees Office in Boston, London, Israel Over 600 customers Fortune 500 enterprises, Telcos, SMBs ∙ SaaS Platform + Licenses ∙ Recurring revenue model Mobile Application Quality across the mobile application lifecycle Monitoring Testing Automation ALM Integration Offshoring Continuous Integration Public Cloud Private Cloud Mobile Market Trends Perfecto Mobile 2
  5. 5. 4/16/2013 Global Mobile Market Stats Source: nielsen, 2012 Mobile OS market expected to grow (even more) FireFox OS PEAK KEON Tizen Windows Phone Samsung/Intel New Mobile OS’s RIM’s BlackBerry OS 10 Ubuntu Z10 & Q10 SailFish By Jolla Perfecto Mobile 3
  6. 6. 4/16/2013 Cost Of Mobile Quality Rises Mobile accounts for 11% of U.S e-Commerce Existing Mobile OS’s – End of 2012 Source:IDC Samsung accounted for 63 Million out of the 159 Million in Q42012 Perfecto Mobile 4
  7. 7. 4/16/2013 Worldwide smartphone vendors Market share Source: IDC Worldwide Mobile Phone Tracker, January 24, 2013 Market insights – Forecasts and trends Mobile based shopping expected to grow by 24% in 2013 (U.S) Perfecto Mobile 5
  8. 8. 4/16/2013 Market trends - 2013 Mobile OS battle continues •New open source competitors trying to influence the eco system •Windows Phone and BlackBerry OS 10 – critical year for their positioning in the market Mobile apps become more complex – impact testing and automation HTML5 and Mobile web expected to grow significantly EMM establishment – Processes, Apps, Tools, Roles (CMOO) Mini-tablets and “Phablets” will successfully blur the lines between device types Mobile Quality Trends Perfecto Mobile 6
  9. 9. 4/16/2013 World Quality report 2012-2013 2012Mobile Testing is Behind the Curve Source: CapGemini Mobile Environment Fragmented Dynamic Complex Business critical Source: Google, 2012 Perfecto Mobile 7
  10. 10. 4/16/2013 Countless screen resolutions Source: opensignalmaps, 2012 (Android only) Consumer Reaction – Less Tolerant than In Desktop Perfecto Mobile 8
  11. 11. 4/16/2013 HTML5 Trend Involves Risks To help protect y our priv acy , PowerPoint prev ented this external picture from being automatically downloaded. To download and display this picture, click Options in the Message Bar, and then click Enable external content. Mobile Quality Challenges Perfecto Mobile 9
  12. 12. 4/16/2013 Mobile Introduces various challenges Performance Automation AGILE ALM Functional Native Hybrid Web Application Types On various browsers Sample Screen Sizes (Only Smartphone) QVGA: quarter VGA (240×320 pixels) HVGA: half VGA (320×480 pixels) WVGA: wide VGA (480×800 pixels) FWVGA: full wide VGA (480×854 pixels) nHD: one-ninth high definition (360×640 pixels) qHD: one-quarter high definition (540×960 pixels) Various Device/Tablets OEM’s Various OS Types and Versions 2.3.3, 4.0.3 4.3.3, 5.1 4.1.2, 4.2.2 6.1, 6.1.1 Mobile Security Threats Source: Techcrunch Perfecto Mobile 10
  13. 13. 4/16/2013 Mobile OS Regression bugs – What does it have to do with my Mobile Testing? Per Device version Pre-release testing (iOS6.1.1 Beta) Apple Bug Reporter Source: LINK Android defect database Mobile Web Objects – Automation Challenges Perfecto Mobile 11
  14. 14. 4/16/2013 Apple iOS – UI Guidelines Source: Source: http://developer.apple.com/library/ios/# http://developer.apple.com/library/ios/#documentation/UserExperience/Conceptual/Mobil eHIG/Introduction/Introduction.html Android OS – UI Guidelines Source: http://developer.android.com/design/index.html Perfecto Mobile 12
  15. 15. 4/16/2013 Risks Management Mobile Quality Risk Analysis Cost of quality in mobile grows and impacted by • Availability • TTM • Functionality and UX • New platform support (Mobile OS, Mobile devices) • Previous app version compatibility Perfecto Mobile 13
  16. 16. 4/16/2013 Mobile end-user Less Tolerant end- Listen to the market Obviously someone forgot to do their regression testing on a Galaxy S3 before releasing this version to market… Perfecto Mobile 14
  17. 17. 4/16/2013 Device governance A clear process as part of the “mobile project” lifecycle Define platforms and categories Define coverage Acceptable not to test - You can’t test everything Define change frequency Adapt to the market trends as they happen Define SLA for changes Define coverage as part of the ALM phase Coverage visibility reports to project managers and executives Device Governance is a risk management for mobile projects Device Governance – Define coverage You can’t cover everything But you should define a target, which is the acceptable risk iOS is not as simple as you think At least 3 HW platforms (3GS, 4, 4S, 5) (iPADs, iPODs) At least 3 SW platforms Perfecto Mobile prev major 15% prev minor 25% Other 5% Latest 55% 15
  18. 18. 4/16/2013 Enterprises OWNS the risk for Mobile quality Mobile Malware - Trojans, viruses and rootkits migrating from traditional online banking and designed specifically for the mobile marketplace. Researchers see an increase in mobile malware development - in pace with market growth. ThirdThird-Party Apps - Consumers love their smart phone and tablet applications, but often these apps come from third parties with questionable security practices. Or worse, the apps are created by fraudsters and loaded with malware. Unsecured Wi-Fi - The unsecured wireless network is a toll-free highway for fraudsters to gain Witollaccess to mobile devices, either to seize control of or gain access to account information. devices, information. User Behavior - Consumers are prone to download third-party apps, use unsecured wireless thirdapps, networks, open and click links in SMS text messages and e-mails, and lose their mobile devices. edevices. Mobile testing and Hardware (Payment card readers, healthcare units and more) Uncompromised necessity for testing mobile apps Should adhere to regulations (GLBA/FFIEC, RDC, FCRA, PCI DSS) around security, GLBA/FFIEC, RDC, FCRA, DSS) authentications, payment and more Usually support complex business work flows (deposit check via camera) Multi tier interactions (Databases, trading accounts, other services) High Availability and Monitoring Must address mobile end user actions Using Home and Back button during application run Network lose during transaction Roaming during user transaction Application behavior in various networks 3G, 4G, WiFi DSS Compliance test case example Perfecto Mobile 16
  19. 19. 4/16/2013 Mobile Is Mandated To Strict Regulations Source: LINK Source: Link Mobile Testing Best Practices Perfecto Mobile 17
  20. 20. 4/16/2013 This is not a mobile test lab! 10 Devices 3 Extra Devices Many Cables 3 Workstations One Chair Building Blocks Of Mobile Testing Strategy Choose the right tools and platform Understand your users device and platform choices Optimize device and platform testing Automate as much as possible using device agnostic scripting Use mobile cloud based solutions (Collaborative, Governed, Scalable, Cost Effective) Use CD/CI solutions as part of your mobile ALM Do not re-invent the wheel reExtend existing ALM tools to Mobile. Perfecto Mobile 18
  21. 21. 4/16/2013 World Quality report 2012-2013 2012- Source: CapGemini Building Blocks Of Mobile Testing Strategy Consider outsource to reduce mobile manual testing costs Test for performance before releasing to the market Make sure you do regression on older supported platforms Do not have your end users do your QA work Perform pro-active testing on new devices and mobile OS’s Monitor in production to assure end user experience and availability Listen to your end users and act upon their feedback (Markets have significant impact on your business and branding) Perfecto Mobile 19
  22. 22. 4/16/2013 (True) Mobile Cloud Avoid device management Real Devices + Emulators Real smartphones Manual + Automation ALM Integration Device agnostic Core level extension End-toEnd-to-End Mobile ALM Development Functional Real Tablets Performance Secure, Centralized Interactive Emulators Short learning curve Monitoring Erank@perfectomobile.com Perfecto Mobile 20
  23. 23. 4/16/2013 END Perfecto Mobile 21

×