Development and testing By: Stephen Frein

554 views
402 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
554
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Development and testing By: Stephen Frein

  1. 1. Mobile Device Testing Strategies Stephen Frein Comcast 5/8/2014
  2. 2. About Me • Director of QA for Comcast.com • Adjunct @ Drexel University • https://www.linkedin.com/in/stephenfrein • stephen.frein@gmail.com • www.frein.com
  3. 3. • Largest mass media and communications company in the world (by $) • Largest cable company / ISP in USA • Largest converged network in the world (data / voice / video) • Comcast.com in April of 2014: o 200 million + page views o 25+ million unique visitors
  4. 4. Stuff We'll Talk About • Importance of mobile • What should I test on? • Where do I keep devices?
  5. 5. Mobile Is BIG
  6. 6. Mobile Matters (More Than Ever) • US retail mobile ecommerce sales increased 68% from 2012 to 2013 (1) * • US mobile web usage rose by a similar amount (2) • More than half of Amazon holiday shoppers used a mobile device in 2013 (3) * Psst! The (1) is an endnote.
  7. 7. But Desktop Still Rules Web • Most mobile Internet usage (80+%) is driven by apps (4) • Mobile usage accounts for about 30% of web traffic (5)
  8. 8. Punchline Mobile is growing in importance, but your web testing efforts should probably emphasize desktop for the foreseeable future. (Apps are their own animal.) Desktop > Mobile
  9. 9. When Will That Change? Not in the next year . . .
  10. 10. Where Should I Focus? 7.1 7.0 6.1 Gingerbread KitKat Jellybean
  11. 11. What My Team Does • Establish coverage thresholds (browser / OS / device type must have x% traffic) • Tune test coverage based on usual factors: o Historical defect discovery rates o Nature of changes o Risk of leakage o Schedule pressure • Pray we got it right
  12. 12. Balancing Stats Your Stats (the focus) Industry Stats (a counterweight) Google Analytics, Adobe, Piwik, etc StatCounter, Chitika, Android, etc
  13. 13. Industry Stats Useful Links http://gs.statcounter.com/#mobile+tablet- browser-US-monthly-201304-201404 http://www.netmarketshare.com/ (some $) http://developer.android.com/about/dashb oards/index.html http://david-smith.org/iosversionstats/
  14. 14. General Observations • Today, iOS and Android are the real show • New iOS versions have rapid adoption • Android versions more fragmented • Android world is heavily phone-skewed • “Default” browsers the focus • Mobile Chrome creeping up in usage • Don’t ignore Windows entirely, but it is a lower priority • Other flavors may not be worth it
  15. 15. Where to Test? Physicals Cloud Emulators
  16. 16. Help Me Choose Approach Pros Cons Physicals • Realism • Audio / Video • Geo-lock • Management Cloud • Geo-flex • Management • Automation • Cost • Audio / Video • Speed Emulator • Cheap • Fast • Realism • Realism!!!
  17. 17. We Use Cloud (Mostly) • International team • “Hide and seek” got old • Audio / video not a priority for us • Good automation support • Private cloud helps o Automation o Test environment access “Who has the &%*@! iPad?”
  18. 18. Whose Cloud Offering? Start by looking at these two . . . . . . but there are many others.
  19. 19. Remember • Mobile matters, but desktop owns web • Stats key to your device selection – Android phones and iPhone / iPad are must-do – Set percentage standards, but then tune based on other factors • Don’t rely on emulators – consider cloud solutions for device hosting
  20. 20. Wrap-up Questions? Thanks!
  21. 21. Endnotes 1. http://www.forbes.com/sites/chuckjones/2013/10/02/ecommerce-is-growing-nicely-while-mcommerce-is-on-a-tear/ 2. http://bgr.com/2013/11/08/mobile-devices-internet-traffic-q3-2013/ 3. http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=1886961&highlight= 4. http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ 5. http://www.walkersandsdigital.com/Walker-Sands-Mobile-Traffic-Report-Q3-2013

×