Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mobile Testing, That's Just a Smaller Screen, Right? Stephen Janaway

1,329 views

Published on

My presentation on mobile testing from Mobile Apps Europe 2014.

Published in: Software
  • Test your mobile web and native mobile applications on multiple mobile devices that increasingly command adoption & usage for enterprise and everyday computing. https://goo.gl/caU3Xp
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Mobile Testing, That's Just a Smaller Screen, Right? Stephen Janaway

  1. 1. Mobile Testing, That’s Just a Smaller Screen, Right? Stephen Janaway © Stephen Janaway 2014
  2. 2. Who am I? Stephen Janaway • 14 years experience in mobile software testing. • Mobile devices. • Mobile networks. • Mobile applications. • Have worked for companies such as Ericsson, Motorola and Nokia. • Work for the NET-A-PORTER GROUP. • www.stephenjanaway.co.uk • @stephenjanaway © Stephen Janaway 2014
  3. 3. www.testinginthepub.com @testinginthepub Search for “Testing In The Pub’” on iTunes © Stephen Janaway 2014
  4. 4. Today • An overview of mobile. • Why is mobile different? • What makes mobile testing different? • The future of mobile testing. • Your mobile testing future. © Stephen Janaway 2014
  5. 5. Mobile devices are • Smart Phones • Feature Phones • Tablets • E-readers • Big screens, small screens, touch screens and keypads. © Stephen Janaway 2014
  6. 6. And now… • Fitness devices. • Smart watches. • Wearable devices. • Beacons. • Household gadgets. © Stephen Janaway 2014
  7. 7. Mobile devices are much more • Alarm Clocks • Maps • Link to loved ones • Hand held shops • Tickets • Music Players • Books • Games Players • Cameras • …… • These are key experiences. © Stephen Janaway 2014
  8. 8. My alarm didn’t go off…
  9. 9. I’m lost
  10. 10. WHY IS MOBILE DIFFERENT? © Stephen Janaway 2014
  11. 11. MOBILE USERS ARE MORE EMOTIONALLY ATTACHED
  12. 12. MOBILE USERS ARE NOT SECOND CLASS CITIZENS
  13. 13. © Stephen Janaway 2014 ?
  14. 14. BUT QUALITY IS CURRENTLY STILL NOT GREAT… AND USERS INCREASINGLY DON’T TOLERATE THIS
  15. 15. MOBILE USERS CAN LEAVE BAD FEEDBACK MUCH MORE QUICKLY
  16. 16. What Makes Mobile Testing Different? • Many platforms. • Many OS versions in the field. • Many different hardware configurations. • Devices with limited memory and processing capability. • Limited battery life. • Network connectivity. • Application stores. • Low maturity of automation solutions. © Stephen Janaway 2014
  17. 17. Usability Matters • Usability is more important in the mobile world: – Smaller screen. – More difficult to interact with and typically used one handed.
  18. 18. Brand Consistency Matters • Think about what a mobile user would want. • Something familiar. • Taking advantage of mobile functionality: – Location. – Mobile search. – M-commerce. • A design that can be viewed on a small screen and in bright light. © Stephen Janaway 2014
  19. 19. Mobile Introduces a Whole New Language © Stephen Janaway 2014
  20. 20. Mobile Introduces New Scenarios • Download and install. • Un-install. • Upgrade. • Rotation. • Interaction with Sensors. • Location. • External media. © Stephen Janaway 2014
  21. 21. Mobiles Are Complicated • While your application runs, the phone is: – Talking to the network – Working out where you are – Maybe receiving calls, texts, Facebook and Twitter updates. • What happens when you have your application running, and then get a call, and a text, whilst uploading video to YouTube? Test for it and find out.
  22. 22. Performance Matters • Limited processor speeds. • Limited on-board memory. • Typically not turned off very often. • Don’t forget - most apps are just clients to web services. Use your web performance testing skills. © Stephen Janaway 2014
  23. 23. Data Matters • Check for excessive data consumption. • Using on device data measuring tools. • Using a proxy. © Stephen Janaway 2014
  24. 24. The Mobile World © Stephen Janaway 2014
  25. 25. Subscriptions vs. People © Stephen Janaway 2014
  26. 26. Different OS’s © Stephen Janaway 2014
  27. 27. Different Mobile Browsers © Stephen Janaway 2014
  28. 28. The Future Of Mobile Testing © Stephen Janaway 2014
  29. 29. More Devices = More Testing (?) © Stephen Janaway 2014
  30. 30. OS’s © Stephen Janaway 2014 Source: gs.statcounter.com
  31. 31. Android versions Source: developer.android.com
  32. 32. Screen Sizes © Stephen Janaway 2014
  33. 33. Deciding What to Test • Know your predicated customers (not your current ones). • Find out what OS updates are coming up. • Get as much data on device usage that you can: – In app analytics. – Web searches. – Similar apps. © Stephen Janaway 2014
  34. 34. Options • Don’t use them at all – use simulators or user agents. • Buy all or some of them. • Borrow all or some of them. • Rent all or some of them. • Use a cloud based service (Perfecto Mobile, DeviceAnywhere). © Stephen Janaway 2014
  35. 35. My Recommendation • Buy 2 from the top, 2 from the middle and 2 from the bottom. • Understand what your customers use or will use. • Test first on simulators not only on simulators. • Use cloud services if needs be, to fill any gaps. • Never just use browser emulation for mobile websites. © Stephen Janaway 2014
  36. 36. Open Device Labs • Borrow devices, normally for free. • http://opendevicelab.com • Open Device Labs = shared community pools of internet connected devices for testing purposes. © Stephen Janaway 2014
  37. 37. Making More Use of Your Users © Stephen Janaway 2014
  38. 38. Dog fooding/ Alpha/ Beta • Get an internal user group going. • Distribute app via TestFlight, TestFairy or similar. • Alpha/ Beta test via app stores. © Stephen Janaway 2014
  39. 39. More Financial Transactions Will Happen On Mobile © Stephen Janaway 2014
  40. 40. Security • Big recent rise in the amount of malware targeted at mobile devices. © Stephen Janaway 2014
  41. 41. © Stephen Janaway 2014
  42. 42. Perception of Security © Stephen Janaway 2014
  43. 43. Smartphones Everywhere = Your App Everywhere © Stephen Janaway 2014
  44. 44. Accessibility • How readable is the app/ website with a large font? • Is there voice activation / control? • Are there text alternatives to images? • Tools: – TalkBack (Android) – Voice Over (iOS) © Stephen Janaway 2014
  45. 45. Voice Activation • Siri – iOS • Google Now – Android • Also on Windows Phone and Blackberry. • Is the app voice activated? • What happens when you activate voice with the app running? © Stephen Janaway 2014
  46. 46. Your Mobile Testing Future You Can Avoid The Common Mobile Testing Mistakes © Stephen Janaway 2014
  47. 47. Only Testing On Simulators © Stephen Janaway 2014
  48. 48. Thinking a Mobile Device Lab Is a One Time Investment © Stephen Janaway 2014
  49. 49. Staying In The Office © Stephen Janaway 2014
  50. 50. Don’t Just Test Here…
  51. 51. On Here, and At Here…
  52. 52. Out Here…
  53. 53. In Here…
  54. 54. Forgetting About Battery Life © Stephen Janaway 2014
  55. 55. Battery Life © Stephen Janaway 2014
  56. 56. Ignoring OS Updates © Stephen Janaway 2014
  57. 57. OS Updates • Always regression test your app or website before an OS update. • Get on the developer program for early OS versions. • Users update more quickly on iOS than Android. • Carriers often control update cycles. • Decide beforehand how many OS version you will/ can support. © Stephen Janaway 2014
  58. 58. Thinking Mobile Test Automation Is Like Web Test Automation © Stephen Janaway 2014
  59. 59. Automation Suggestions • Determine if you require cross platform support. • Avoid tools that use embedded servers or agents – they alter the production code. • Look for tools with active online communities. • Consider the future. © Stephen Janaway 2014
  60. 60. My Picks • Appium: – Cross platform. – Well supported user community. – About to be more closely integrated in Selenium 3. • iOS Driver: – Well supported user community. – About to be more closely integrated in Selenium 3. • Calabash: – Well supported user community. – Heavily used at the BBC. © Stephen Janaway 2014
  61. 61. So……
  62. 62. Mobile Test Strategy – Top 5 Tips 1. Understand the potential user: – Which platforms and devices will they use? – How will they use the app or website? 2. Understand what OS versions are being used and when new ones are coming. 3. Test using simulators, real devices and users. 4. Understand the cost of automation: – It is where desktop automation was 6 years ago. – Best combined with extensive unit and integration testing. 5. Use an exploratory approach to manual testing: – Mobile device projects are typically short. © Stephen Janaway 2014
  63. 63. “Be proud, your work will soon be in the hands of millions”
  64. 64. Stephen Janaway Questions? www.stephenjanaway.co.uk @stephenjanaway www.testinginthepub.com

×