SlideShare a Scribd company logo
Mobile Testing, That’s Just a 
Smaller Screen, Right? 
Stephen Janaway 
© Stephen Janaway 2014
Who am I? 
Stephen Janaway 
• 14 years experience in mobile software 
testing. 
• Mobile devices. 
• Mobile networks. 
• Mobile applications. 
• Have worked for companies such as 
Ericsson, Motorola and Nokia. 
• Work for the NET-A-PORTER GROUP. 
• www.stephenjanaway.co.uk 
• @stephenjanaway 
© Stephen Janaway 2014
www.testinginthepub.com 
@testinginthepub 
Search for “Testing In The Pub’” on iTunes 
© Stephen Janaway 2014
Today 
• An overview of mobile. 
• Why is mobile different? 
• What makes mobile testing different? 
• The future of mobile testing. 
• Your mobile testing future. 
© Stephen Janaway 2014
Mobile devices are 
• Smart Phones 
• Feature Phones 
• Tablets 
• E-readers 
• Big screens, small screens, touch screens and 
keypads. 
© Stephen Janaway 2014
And now… 
• Fitness devices. 
• Smart watches. 
• Wearable devices. 
• Beacons. 
• Household gadgets. 
© Stephen Janaway 2014
Mobile Testing, That's Just a Smaller Screen, Right?   Stephen Janaway
Mobile devices are much more 
• Alarm Clocks 
• Maps 
• Link to loved ones 
• Hand held shops 
• Tickets 
• Music Players 
• Books 
• Games Players 
• Cameras 
• …… 
• These are key experiences. 
© Stephen Janaway 2014
My alarm didn’t go off…
I’m lost
WHY IS MOBILE DIFFERENT? 
© Stephen Janaway 2014
MOBILE USERS ARE MORE 
EMOTIONALLY ATTACHED
MOBILE USERS ARE NOT 
SECOND CLASS CITIZENS
© Stephen Janaway 2014 
?
BUT QUALITY IS CURRENTLY STILL 
NOT GREAT… 
AND USERS INCREASINGLY 
DON’T TOLERATE THIS
MOBILE USERS CAN LEAVE BAD 
FEEDBACK MUCH MORE QUICKLY
What Makes Mobile Testing Different? 
• Many platforms. 
• Many OS versions in the field. 
• Many different hardware configurations. 
• Devices with limited memory and processing 
capability. 
• Limited battery life. 
• Network connectivity. 
• Application stores. 
• Low maturity of automation solutions. 
© Stephen Janaway 2014
Usability Matters 
• Usability is more important in the mobile world: 
– Smaller screen. 
– More difficult to interact with and typically used one 
handed.
Brand Consistency Matters 
• Think about what a mobile user would want. 
• Something familiar. 
• Taking advantage of mobile functionality: 
– Location. 
– Mobile search. 
– M-commerce. 
• A design that can be viewed on a small screen 
and in bright light. 
© Stephen Janaway 2014
Mobile Introduces 
a Whole New Language 
© Stephen Janaway 2014
Mobile Introduces New Scenarios 
• Download and install. 
• Un-install. 
• Upgrade. 
• Rotation. 
• Interaction with Sensors. 
• Location. 
• External media. 
© Stephen Janaway 2014
Mobiles Are Complicated 
• While your application runs, the phone is: 
– Talking to the network 
– Working out where you are 
– Maybe receiving calls, texts, Facebook and Twitter 
updates. 
• What happens when you have your application 
running, and then get a call, and a text, whilst 
uploading video to YouTube? Test for it and find 
out.
Performance Matters 
• Limited processor speeds. 
• Limited on-board memory. 
• Typically not turned off very often. 
• Don’t forget - most apps are just clients to 
web services. Use your web performance 
testing skills. 
© Stephen Janaway 2014
Data Matters 
• Check for excessive data consumption. 
• Using on device data measuring tools. 
• Using a proxy. 
© Stephen Janaway 2014
The Mobile World 
© Stephen Janaway 2014
Subscriptions vs. People 
© Stephen Janaway 2014
Different OS’s 
© Stephen Janaway 2014
Different Mobile Browsers 
© Stephen Janaway 2014
The Future 
Of Mobile Testing 
© Stephen Janaway 2014
More Devices = More Testing (?) 
© Stephen Janaway 2014
OS’s 
© Stephen Janaway 2014 
Source: gs.statcounter.com
Android versions 
Source: developer.android.com
Screen Sizes 
© Stephen Janaway 2014
Deciding What to Test 
• Know your predicated customers (not your 
current ones). 
• Find out what OS updates are coming up. 
• Get as much data on device usage that you 
can: 
– In app analytics. 
– Web searches. 
– Similar apps. 
© Stephen Janaway 2014
Options 
• Don’t use them at all – use simulators or user 
agents. 
• Buy all or some of them. 
• Borrow all or some of them. 
• Rent all or some of them. 
• Use a cloud based service (Perfecto Mobile, 
DeviceAnywhere). 
© Stephen Janaway 2014
My Recommendation 
• Buy 2 from the top, 2 from the middle and 2 from 
the bottom. 
• Understand what your customers use or will use. 
• Test first on simulators not only on simulators. 
• Use cloud services if needs be, to fill any gaps. 
• Never just use browser emulation for mobile 
websites. 
© Stephen Janaway 2014
Open Device Labs 
• Borrow devices, normally 
for free. 
• http://opendevicelab.com 
• Open Device 
Labs = shared community 
pools of internet 
connected devices for 
testing purposes. 
© Stephen Janaway 2014
Making More Use of Your Users 
© Stephen Janaway 2014
Dog fooding/ Alpha/ Beta 
• Get an internal user group going. 
• Distribute app via TestFlight, TestFairy or 
similar. 
• Alpha/ Beta test via app stores. 
© Stephen Janaway 2014
More Financial Transactions 
Will Happen On Mobile 
© Stephen Janaway 2014
Security 
• Big recent rise in the amount of malware targeted at 
mobile devices. 
© Stephen Janaway 2014
© Stephen Janaway 2014
Perception of Security 
© Stephen Janaway 2014
Smartphones Everywhere 
= Your App Everywhere 
© Stephen Janaway 2014
Accessibility 
• How readable is the app/ website with a large 
font? 
• Is there voice activation / control? 
• Are there text alternatives to images? 
• Tools: 
– TalkBack (Android) 
– Voice Over (iOS) 
© Stephen Janaway 2014
Voice Activation 
• Siri – iOS 
• Google Now – Android 
• Also on Windows Phone and Blackberry. 
• Is the app voice activated? 
• What happens when you activate voice with 
the app running? 
© Stephen Janaway 2014
Your Mobile Testing Future 
You Can Avoid The Common 
Mobile Testing Mistakes 
© Stephen Janaway 2014
Only Testing On Simulators 
© Stephen Janaway 2014
Thinking a Mobile Device Lab 
Is a One Time Investment 
© Stephen Janaway 2014
Staying In The Office 
© Stephen Janaway 2014
Don’t Just Test Here…
On Here, and At Here…
Out Here…
In Here…
Forgetting About Battery Life 
© Stephen Janaway 2014
Battery Life 
© Stephen Janaway 2014
Ignoring OS Updates 
© Stephen Janaway 2014
OS Updates 
• Always regression test your app or website 
before an OS update. 
• Get on the developer program for early OS 
versions. 
• Users update more quickly on iOS than 
Android. 
• Carriers often control update cycles. 
• Decide beforehand how many OS version you 
will/ can support. 
© Stephen Janaway 2014
Thinking Mobile Test Automation 
Is Like Web Test Automation 
© Stephen Janaway 2014
Automation Suggestions 
• Determine if you require cross platform 
support. 
• Avoid tools that use embedded servers or 
agents – they alter the production code. 
• Look for tools with active online communities. 
• Consider the future. 
© Stephen Janaway 2014
My Picks 
• Appium: 
– Cross platform. 
– Well supported user community. 
– About to be more closely integrated in Selenium 3. 
• iOS Driver: 
– Well supported user community. 
– About to be more closely integrated in Selenium 3. 
• Calabash: 
– Well supported user community. 
– Heavily used at the BBC. 
© Stephen Janaway 2014
So……
Mobile Test Strategy – Top 5 Tips 
1. Understand the potential user: 
– Which platforms and devices will they use? 
– How will they use the app or website? 
2. Understand what OS versions are being used and 
when new ones are coming. 
3. Test using simulators, real devices and users. 
4. Understand the cost of automation: 
– It is where desktop automation was 6 years ago. 
– Best combined with extensive unit and integration testing. 
5. Use an exploratory approach to manual testing: 
– Mobile device projects are typically short. 
© Stephen Janaway 2014
“Be proud, your work will soon 
be in the hands of millions”
Stephen Janaway 
Questions? 
www.stephenjanaway.co.uk 
@stephenjanaway 
www.testinginthepub.com

More Related Content

What's hot

David Raissipour - Mobile First is Always Right
David Raissipour - Mobile First is Always RightDavid Raissipour - Mobile First is Always Right
David Raissipour - Mobile First is Always Right
MassTLC
 
Mobile Usability
Mobile UsabilityMobile Usability
Mobile Usability
Jeff Wisniewski
 
Usability testing for mobile apps - Touch Tour Chennai
Usability testing for mobile apps - Touch Tour ChennaiUsability testing for mobile apps - Touch Tour Chennai
Usability testing for mobile apps - Touch Tour Chennai
RIA RUI Society
 
Webinar: Are you ready for your peak season?
Webinar: Are you ready for your peak season?Webinar: Are you ready for your peak season?
Webinar: Are you ready for your peak season?
Jennifer Finney
 
5 Keys to Your Best Automated Testing Strategy
5 Keys to Your Best Automated Testing Strategy5 Keys to Your Best Automated Testing Strategy
5 Keys to Your Best Automated Testing Strategy
SOASTA
 
Paid Apps Economy / Nir Pochter
Paid Apps Economy / Nir PochterPaid Apps Economy / Nir Pochter
Paid Apps Economy / Nir Pochter
geektimecoil
 
Synthetic and rum webinar
Synthetic and rum webinarSynthetic and rum webinar
Synthetic and rum webinar
SOASTA
 
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA Webinar: Process Compression For Mobile App Dev 120612SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA
 
Continuous delivery 101
Continuous delivery 101Continuous delivery 101
Continuous delivery 101
darkofabijan
 
7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing
SOASTA
 
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
Ismaël Héry
 
New Features in CloudTest & TouchTest
New Features in CloudTest & TouchTestNew Features in CloudTest & TouchTest
New Features in CloudTest & TouchTest
Jennifer Finney
 
MeasureWorks - Prepare your mobile User Experience for fast conversion
MeasureWorks - Prepare your mobile User Experience for fast conversionMeasureWorks - Prepare your mobile User Experience for fast conversion
MeasureWorks - Prepare your mobile User Experience for fast conversion
MeasureWorks
 
Secrets to Realistic Load Testing
Secrets to Realistic Load TestingSecrets to Realistic Load Testing
Secrets to Realistic Load Testing
SOASTA
 
Windows Phone Dev Day Bournemouth Presentation
Windows Phone Dev Day Bournemouth PresentationWindows Phone Dev Day Bournemouth Presentation
Windows Phone Dev Day Bournemouth Presentation
James Mundy
 
Widget iOS: how to use them well
Widget iOS: how to use them wellWidget iOS: how to use them well
Widget iOS: how to use them well
Alessio Roberto
 
Learn fast to build fast @ le Monde - Lean Kanban France 2014
Learn fast to build fast @ le Monde - Lean Kanban France 2014Learn fast to build fast @ le Monde - Lean Kanban France 2014
Learn fast to build fast @ le Monde - Lean Kanban France 2014
Ismaël Héry
 
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major EventsO'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
SOASTA
 
Soasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest WebinarSoasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest Webinar
Jennifer Finney
 
User Sentiment to Determine App Quality
User Sentiment to Determine App QualityUser Sentiment to Determine App Quality
User Sentiment to Determine App Quality
Apigee | Google Cloud
 

What's hot (20)

David Raissipour - Mobile First is Always Right
David Raissipour - Mobile First is Always RightDavid Raissipour - Mobile First is Always Right
David Raissipour - Mobile First is Always Right
 
Mobile Usability
Mobile UsabilityMobile Usability
Mobile Usability
 
Usability testing for mobile apps - Touch Tour Chennai
Usability testing for mobile apps - Touch Tour ChennaiUsability testing for mobile apps - Touch Tour Chennai
Usability testing for mobile apps - Touch Tour Chennai
 
Webinar: Are you ready for your peak season?
Webinar: Are you ready for your peak season?Webinar: Are you ready for your peak season?
Webinar: Are you ready for your peak season?
 
5 Keys to Your Best Automated Testing Strategy
5 Keys to Your Best Automated Testing Strategy5 Keys to Your Best Automated Testing Strategy
5 Keys to Your Best Automated Testing Strategy
 
Paid Apps Economy / Nir Pochter
Paid Apps Economy / Nir PochterPaid Apps Economy / Nir Pochter
Paid Apps Economy / Nir Pochter
 
Synthetic and rum webinar
Synthetic and rum webinarSynthetic and rum webinar
Synthetic and rum webinar
 
SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA Webinar: Process Compression For Mobile App Dev 120612SOASTA Webinar: Process Compression For Mobile App Dev 120612
SOASTA Webinar: Process Compression For Mobile App Dev 120612
 
Continuous delivery 101
Continuous delivery 101Continuous delivery 101
Continuous delivery 101
 
7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing7 steps to pragmatic mobile testing
7 steps to pragmatic mobile testing
 
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
Learn Fast to Build Fast @ le Monde - Lean Kanban France 2014
 
New Features in CloudTest & TouchTest
New Features in CloudTest & TouchTestNew Features in CloudTest & TouchTest
New Features in CloudTest & TouchTest
 
MeasureWorks - Prepare your mobile User Experience for fast conversion
MeasureWorks - Prepare your mobile User Experience for fast conversionMeasureWorks - Prepare your mobile User Experience for fast conversion
MeasureWorks - Prepare your mobile User Experience for fast conversion
 
Secrets to Realistic Load Testing
Secrets to Realistic Load TestingSecrets to Realistic Load Testing
Secrets to Realistic Load Testing
 
Windows Phone Dev Day Bournemouth Presentation
Windows Phone Dev Day Bournemouth PresentationWindows Phone Dev Day Bournemouth Presentation
Windows Phone Dev Day Bournemouth Presentation
 
Widget iOS: how to use them well
Widget iOS: how to use them wellWidget iOS: how to use them well
Widget iOS: how to use them well
 
Learn fast to build fast @ le Monde - Lean Kanban France 2014
Learn fast to build fast @ le Monde - Lean Kanban France 2014Learn fast to build fast @ le Monde - Lean Kanban France 2014
Learn fast to build fast @ le Monde - Lean Kanban France 2014
 
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major EventsO'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
O'Reilly Webcast: How Nordstrom Prepares Its Site for Holidays and Major Events
 
Soasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest WebinarSoasta New Features in CloudTest & TouchTest Webinar
Soasta New Features in CloudTest & TouchTest Webinar
 
User Sentiment to Determine App Quality
User Sentiment to Determine App QualityUser Sentiment to Determine App Quality
User Sentiment to Determine App Quality
 

Viewers also liked

Retrospective Meeting Knowledge Sharing
Retrospective Meeting Knowledge SharingRetrospective Meeting Knowledge Sharing
Retrospective Meeting Knowledge Sharing
Philip Wang
 
Ale2012 retrospective
Ale2012 retrospectiveAle2012 retrospective
Ale2012 retrospective
Marc Löffler
 
What makes a failed Retro - Muon Nguyen
What makes a failed Retro - Muon NguyenWhat makes a failed Retro - Muon Nguyen
What makes a failed Retro - Muon Nguyen
Scrum Breakfast Vietnam
 
Mobile Users Are Different
Mobile Users Are DifferentMobile Users Are Different
Mobile Users Are Different
Stephen Janaway
 
Retrospective - the most ignored event in Scrum
Retrospective - the most ignored event in ScrumRetrospective - the most ignored event in Scrum
Retrospective - the most ignored event in Scrum
Sanjay Saini
 
Lightening Talk: lama sutra of retrospective
Lightening Talk: lama sutra of retrospectiveLightening Talk: lama sutra of retrospective
Lightening Talk: lama sutra of retrospective
Agileee
 
Why I Lost My Job As a Test Manager and What I Learnt As a Result
Why I Lost My Job As a Test Manager and What I Learnt As a ResultWhy I Lost My Job As a Test Manager and What I Learnt As a Result
Why I Lost My Job As a Test Manager and What I Learnt As a Result
Stephen Janaway
 
Testing Your Emotions
Testing Your EmotionsTesting Your Emotions
Testing Your Emotions
Stephen Janaway
 
The Art of the Retrospective: How to run an awesome retrospective meeting
The Art of the Retrospective: How to run an awesome retrospective meetingThe Art of the Retrospective: How to run an awesome retrospective meeting
The Art of the Retrospective: How to run an awesome retrospective meeting
Chris Smith
 
Build the future of mobile apps with facebook mobile app europe berlin sept 14
Build the future of mobile apps with facebook   mobile app europe berlin sept 14Build the future of mobile apps with facebook   mobile app europe berlin sept 14
Build the future of mobile apps with facebook mobile app europe berlin sept 14
Julien Lesaicherre
 
Apportioning Monoliths
Apportioning MonolithsApportioning Monoliths
Apportioning Monoliths
Kevin Goldsmith
 
The Spotify Tribe
The Spotify TribeThe Spotify Tribe
The Spotify Tribe
Kevin Goldsmith
 

Viewers also liked (12)

Retrospective Meeting Knowledge Sharing
Retrospective Meeting Knowledge SharingRetrospective Meeting Knowledge Sharing
Retrospective Meeting Knowledge Sharing
 
Ale2012 retrospective
Ale2012 retrospectiveAle2012 retrospective
Ale2012 retrospective
 
What makes a failed Retro - Muon Nguyen
What makes a failed Retro - Muon NguyenWhat makes a failed Retro - Muon Nguyen
What makes a failed Retro - Muon Nguyen
 
Mobile Users Are Different
Mobile Users Are DifferentMobile Users Are Different
Mobile Users Are Different
 
Retrospective - the most ignored event in Scrum
Retrospective - the most ignored event in ScrumRetrospective - the most ignored event in Scrum
Retrospective - the most ignored event in Scrum
 
Lightening Talk: lama sutra of retrospective
Lightening Talk: lama sutra of retrospectiveLightening Talk: lama sutra of retrospective
Lightening Talk: lama sutra of retrospective
 
Why I Lost My Job As a Test Manager and What I Learnt As a Result
Why I Lost My Job As a Test Manager and What I Learnt As a ResultWhy I Lost My Job As a Test Manager and What I Learnt As a Result
Why I Lost My Job As a Test Manager and What I Learnt As a Result
 
Testing Your Emotions
Testing Your EmotionsTesting Your Emotions
Testing Your Emotions
 
The Art of the Retrospective: How to run an awesome retrospective meeting
The Art of the Retrospective: How to run an awesome retrospective meetingThe Art of the Retrospective: How to run an awesome retrospective meeting
The Art of the Retrospective: How to run an awesome retrospective meeting
 
Build the future of mobile apps with facebook mobile app europe berlin sept 14
Build the future of mobile apps with facebook   mobile app europe berlin sept 14Build the future of mobile apps with facebook   mobile app europe berlin sept 14
Build the future of mobile apps with facebook mobile app europe berlin sept 14
 
Apportioning Monoliths
Apportioning MonolithsApportioning Monoliths
Apportioning Monoliths
 
The Spotify Tribe
The Spotify TribeThe Spotify Tribe
The Spotify Tribe
 

Similar to Mobile Testing, That's Just a Smaller Screen, Right? Stephen Janaway

Stephen janaway mobile testing - that's just a smaller screen, right
Stephen janaway  mobile testing - that's just a smaller screen, rightStephen janaway  mobile testing - that's just a smaller screen, right
Stephen janaway mobile testing - that's just a smaller screen, right
Romania Testing
 
An Introduction To Mobile Software Testing
An Introduction To Mobile Software TestingAn Introduction To Mobile Software Testing
An Introduction To Mobile Software Testing
Stephen Janaway
 
Mobile Testing, That's Just A Smaller Screen, Right?
Mobile Testing, That's Just A Smaller Screen, Right?Mobile Testing, That's Just A Smaller Screen, Right?
Mobile Testing, That's Just A Smaller Screen, Right?
Stephen Janaway
 
Mobile Testing Trends and Innovations
Mobile Testing Trends and InnovationsMobile Testing Trends and Innovations
Mobile Testing Trends and Innovations
TechWell
 
Four Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingFour Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance Testing
SOASTA
 
Harness Mobile Usability Testing to Improve Your Mobile Experience
Harness Mobile Usability Testing to Improve Your Mobile Experience Harness Mobile Usability Testing to Improve Your Mobile Experience
Harness Mobile Usability Testing to Improve Your Mobile Experience
UserZoom
 
Mobile App Testing Strategy
Mobile App Testing StrategyMobile App Testing Strategy
Mobile App Testing Strategy
Software Assurance LLC
 
Real Devices or Emulators: Wen to use What for Automated Testing
Real Devices or Emulators: Wen to use What for Automated TestingReal Devices or Emulators: Wen to use What for Automated Testing
Real Devices or Emulators: Wen to use What for Automated Testing
Sauce Labs
 
Mobile Testing Trends
Mobile Testing TrendsMobile Testing Trends
Mobile Testing Trends
Software Guru
 
The Mobile Testing Checklist
The Mobile Testing ChecklistThe Mobile Testing Checklist
The Mobile Testing Checklist
SOASTA
 
Usability Testing Goes Mobile
Usability Testing Goes MobileUsability Testing Goes Mobile
Usability Testing Goes Mobile
TechWell
 
Tap into mobile app testing@TDT Iasi Sept2013
Tap into mobile app testing@TDT Iasi Sept2013Tap into mobile app testing@TDT Iasi Sept2013
Tap into mobile app testing@TDT Iasi Sept2013
Tabăra de Testare
 
Agile Mobile Strategies on a Shoestring Budget
Agile Mobile Strategies on a Shoestring BudgetAgile Mobile Strategies on a Shoestring Budget
Agile Mobile Strategies on a Shoestring Budget
Aaron Grant
 
Rauch delighting mobile customers with content for apps, videos, and a social...
Rauch delighting mobile customers with content for apps, videos, and a social...Rauch delighting mobile customers with content for apps, videos, and a social...
Rauch delighting mobile customers with content for apps, videos, and a social...
Marta Rauch
 
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
Eneko Jon Bilbao
 
Testing at the Speed of Mobile: Adopting Continuous Integration with Agile
Testing at the Speed of Mobile: Adopting Continuous Integration with AgileTesting at the Speed of Mobile: Adopting Continuous Integration with Agile
Testing at the Speed of Mobile: Adopting Continuous Integration with Agile
Keynote Mobile Testing
 
App testing and publishing
App testing and publishingApp testing and publishing
App testing and publishing
Niall Roche
 
TestElf Informational Presentation
TestElf Informational PresentationTestElf Informational Presentation
TestElf Informational Presentation
TestElf
 
Best Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App TestingBest Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App Testing
Bitbar
 
7 Steps to Pragmatic Mobile Testing
7 Steps to Pragmatic Mobile Testing7 Steps to Pragmatic Mobile Testing
7 Steps to Pragmatic Mobile Testing
SOASTA
 

Similar to Mobile Testing, That's Just a Smaller Screen, Right? Stephen Janaway (20)

Stephen janaway mobile testing - that's just a smaller screen, right
Stephen janaway  mobile testing - that's just a smaller screen, rightStephen janaway  mobile testing - that's just a smaller screen, right
Stephen janaway mobile testing - that's just a smaller screen, right
 
An Introduction To Mobile Software Testing
An Introduction To Mobile Software TestingAn Introduction To Mobile Software Testing
An Introduction To Mobile Software Testing
 
Mobile Testing, That's Just A Smaller Screen, Right?
Mobile Testing, That's Just A Smaller Screen, Right?Mobile Testing, That's Just A Smaller Screen, Right?
Mobile Testing, That's Just A Smaller Screen, Right?
 
Mobile Testing Trends and Innovations
Mobile Testing Trends and InnovationsMobile Testing Trends and Innovations
Mobile Testing Trends and Innovations
 
Four Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingFour Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance Testing
 
Harness Mobile Usability Testing to Improve Your Mobile Experience
Harness Mobile Usability Testing to Improve Your Mobile Experience Harness Mobile Usability Testing to Improve Your Mobile Experience
Harness Mobile Usability Testing to Improve Your Mobile Experience
 
Mobile App Testing Strategy
Mobile App Testing StrategyMobile App Testing Strategy
Mobile App Testing Strategy
 
Real Devices or Emulators: Wen to use What for Automated Testing
Real Devices or Emulators: Wen to use What for Automated TestingReal Devices or Emulators: Wen to use What for Automated Testing
Real Devices or Emulators: Wen to use What for Automated Testing
 
Mobile Testing Trends
Mobile Testing TrendsMobile Testing Trends
Mobile Testing Trends
 
The Mobile Testing Checklist
The Mobile Testing ChecklistThe Mobile Testing Checklist
The Mobile Testing Checklist
 
Usability Testing Goes Mobile
Usability Testing Goes MobileUsability Testing Goes Mobile
Usability Testing Goes Mobile
 
Tap into mobile app testing@TDT Iasi Sept2013
Tap into mobile app testing@TDT Iasi Sept2013Tap into mobile app testing@TDT Iasi Sept2013
Tap into mobile app testing@TDT Iasi Sept2013
 
Agile Mobile Strategies on a Shoestring Budget
Agile Mobile Strategies on a Shoestring BudgetAgile Mobile Strategies on a Shoestring Budget
Agile Mobile Strategies on a Shoestring Budget
 
Rauch delighting mobile customers with content for apps, videos, and a social...
Rauch delighting mobile customers with content for apps, videos, and a social...Rauch delighting mobile customers with content for apps, videos, and a social...
Rauch delighting mobile customers with content for apps, videos, and a social...
 
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
 
Testing at the Speed of Mobile: Adopting Continuous Integration with Agile
Testing at the Speed of Mobile: Adopting Continuous Integration with AgileTesting at the Speed of Mobile: Adopting Continuous Integration with Agile
Testing at the Speed of Mobile: Adopting Continuous Integration with Agile
 
App testing and publishing
App testing and publishingApp testing and publishing
App testing and publishing
 
TestElf Informational Presentation
TestElf Informational PresentationTestElf Informational Presentation
TestElf Informational Presentation
 
Best Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App TestingBest Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App Testing
 
7 Steps to Pragmatic Mobile Testing
7 Steps to Pragmatic Mobile Testing7 Steps to Pragmatic Mobile Testing
7 Steps to Pragmatic Mobile Testing
 

Recently uploaded

Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
3610stuck
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
revolutionary575
 
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
andrehoraa
 
UMiami degree offer diploma Transcript
UMiami degree offer diploma TranscriptUMiami degree offer diploma Transcript
UMiami degree offer diploma Transcript
attueb
 
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
simran hot girls
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
Nextskill Technologies
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
Zyple Software
 
How to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at ScaleHow to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at Scale
Anchore
 
09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching
quanhoangd129
 
03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching
quanhoangd129
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
aadhiyaeliza
 
06. Ruby Array & Hash - Ruby Core Teaching
06. Ruby Array & Hash - Ruby Core Teaching06. Ruby Array & Hash - Ruby Core Teaching
06. Ruby Array & Hash - Ruby Core Teaching
quanhoangd129
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
neshakor5152
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
bahubalikumar09988
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
rachitkumar09887
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
quanhoangd129
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
XongoLab Technologies LLP
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
87tomato
 
TEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with YouTEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with You
marcofolio
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
dream girl
 

Recently uploaded (20)

Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
 
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
Test Polarity: Detecting Positive and Negative Tests (FSE 2024)
 
UMiami degree offer diploma Transcript
UMiami degree offer diploma TranscriptUMiami degree offer diploma Transcript
UMiami degree offer diploma Transcript
 
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
 
ERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in CoimbatoreERP Software Solutions Provider in Coimbatore
ERP Software Solutions Provider in Coimbatore
 
SAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple SoftwareSAP implementation steps PDF - Zyple Software
SAP implementation steps PDF - Zyple Software
 
How to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at ScaleHow to Secure Your Kubernetes Software Supply Chain at Scale
How to Secure Your Kubernetes Software Supply Chain at Scale
 
09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching09. Ruby Object Oriented Programming - Ruby Core Teaching
09. Ruby Object Oriented Programming - Ruby Core Teaching
 
03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching03. Ruby Variables & Regex - Ruby Core Teaching
03. Ruby Variables & Regex - Ruby Core Teaching
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
 
06. Ruby Array & Hash - Ruby Core Teaching
06. Ruby Array & Hash - Ruby Core Teaching06. Ruby Array & Hash - Ruby Core Teaching
06. Ruby Array & Hash - Ruby Core Teaching
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
 
08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching08. Ruby Enumerable - Ruby Core Teaching
08. Ruby Enumerable - Ruby Core Teaching
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
 
TEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with YouTEQnation 2024: Sustainable Software: May the Green Code Be with You
TEQnation 2024: Sustainable Software: May the Green Code Be with You
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
 

Mobile Testing, That's Just a Smaller Screen, Right? Stephen Janaway

  • 1. Mobile Testing, That’s Just a Smaller Screen, Right? Stephen Janaway © Stephen Janaway 2014
  • 2. Who am I? Stephen Janaway • 14 years experience in mobile software testing. • Mobile devices. • Mobile networks. • Mobile applications. • Have worked for companies such as Ericsson, Motorola and Nokia. • Work for the NET-A-PORTER GROUP. • www.stephenjanaway.co.uk • @stephenjanaway © Stephen Janaway 2014
  • 3. www.testinginthepub.com @testinginthepub Search for “Testing In The Pub’” on iTunes © Stephen Janaway 2014
  • 4. Today • An overview of mobile. • Why is mobile different? • What makes mobile testing different? • The future of mobile testing. • Your mobile testing future. © Stephen Janaway 2014
  • 5. Mobile devices are • Smart Phones • Feature Phones • Tablets • E-readers • Big screens, small screens, touch screens and keypads. © Stephen Janaway 2014
  • 6. And now… • Fitness devices. • Smart watches. • Wearable devices. • Beacons. • Household gadgets. © Stephen Janaway 2014
  • 8. Mobile devices are much more • Alarm Clocks • Maps • Link to loved ones • Hand held shops • Tickets • Music Players • Books • Games Players • Cameras • …… • These are key experiences. © Stephen Janaway 2014
  • 9. My alarm didn’t go off…
  • 11. WHY IS MOBILE DIFFERENT? © Stephen Janaway 2014
  • 12. MOBILE USERS ARE MORE EMOTIONALLY ATTACHED
  • 13. MOBILE USERS ARE NOT SECOND CLASS CITIZENS
  • 15. BUT QUALITY IS CURRENTLY STILL NOT GREAT… AND USERS INCREASINGLY DON’T TOLERATE THIS
  • 16. MOBILE USERS CAN LEAVE BAD FEEDBACK MUCH MORE QUICKLY
  • 17. What Makes Mobile Testing Different? • Many platforms. • Many OS versions in the field. • Many different hardware configurations. • Devices with limited memory and processing capability. • Limited battery life. • Network connectivity. • Application stores. • Low maturity of automation solutions. © Stephen Janaway 2014
  • 18. Usability Matters • Usability is more important in the mobile world: – Smaller screen. – More difficult to interact with and typically used one handed.
  • 19. Brand Consistency Matters • Think about what a mobile user would want. • Something familiar. • Taking advantage of mobile functionality: – Location. – Mobile search. – M-commerce. • A design that can be viewed on a small screen and in bright light. © Stephen Janaway 2014
  • 20. Mobile Introduces a Whole New Language © Stephen Janaway 2014
  • 21. Mobile Introduces New Scenarios • Download and install. • Un-install. • Upgrade. • Rotation. • Interaction with Sensors. • Location. • External media. © Stephen Janaway 2014
  • 22. Mobiles Are Complicated • While your application runs, the phone is: – Talking to the network – Working out where you are – Maybe receiving calls, texts, Facebook and Twitter updates. • What happens when you have your application running, and then get a call, and a text, whilst uploading video to YouTube? Test for it and find out.
  • 23. Performance Matters • Limited processor speeds. • Limited on-board memory. • Typically not turned off very often. • Don’t forget - most apps are just clients to web services. Use your web performance testing skills. © Stephen Janaway 2014
  • 24. Data Matters • Check for excessive data consumption. • Using on device data measuring tools. • Using a proxy. © Stephen Janaway 2014
  • 25. The Mobile World © Stephen Janaway 2014
  • 26. Subscriptions vs. People © Stephen Janaway 2014
  • 27. Different OS’s © Stephen Janaway 2014
  • 28. Different Mobile Browsers © Stephen Janaway 2014
  • 29. The Future Of Mobile Testing © Stephen Janaway 2014
  • 30. More Devices = More Testing (?) © Stephen Janaway 2014
  • 31. OS’s © Stephen Janaway 2014 Source: gs.statcounter.com
  • 32. Android versions Source: developer.android.com
  • 33. Screen Sizes © Stephen Janaway 2014
  • 34. Deciding What to Test • Know your predicated customers (not your current ones). • Find out what OS updates are coming up. • Get as much data on device usage that you can: – In app analytics. – Web searches. – Similar apps. © Stephen Janaway 2014
  • 35. Options • Don’t use them at all – use simulators or user agents. • Buy all or some of them. • Borrow all or some of them. • Rent all or some of them. • Use a cloud based service (Perfecto Mobile, DeviceAnywhere). © Stephen Janaway 2014
  • 36. My Recommendation • Buy 2 from the top, 2 from the middle and 2 from the bottom. • Understand what your customers use or will use. • Test first on simulators not only on simulators. • Use cloud services if needs be, to fill any gaps. • Never just use browser emulation for mobile websites. © Stephen Janaway 2014
  • 37. Open Device Labs • Borrow devices, normally for free. • http://opendevicelab.com • Open Device Labs = shared community pools of internet connected devices for testing purposes. © Stephen Janaway 2014
  • 38. Making More Use of Your Users © Stephen Janaway 2014
  • 39. Dog fooding/ Alpha/ Beta • Get an internal user group going. • Distribute app via TestFlight, TestFairy or similar. • Alpha/ Beta test via app stores. © Stephen Janaway 2014
  • 40. More Financial Transactions Will Happen On Mobile © Stephen Janaway 2014
  • 41. Security • Big recent rise in the amount of malware targeted at mobile devices. © Stephen Janaway 2014
  • 43. Perception of Security © Stephen Janaway 2014
  • 44. Smartphones Everywhere = Your App Everywhere © Stephen Janaway 2014
  • 45. Accessibility • How readable is the app/ website with a large font? • Is there voice activation / control? • Are there text alternatives to images? • Tools: – TalkBack (Android) – Voice Over (iOS) © Stephen Janaway 2014
  • 46. Voice Activation • Siri – iOS • Google Now – Android • Also on Windows Phone and Blackberry. • Is the app voice activated? • What happens when you activate voice with the app running? © Stephen Janaway 2014
  • 47. Your Mobile Testing Future You Can Avoid The Common Mobile Testing Mistakes © Stephen Janaway 2014
  • 48. Only Testing On Simulators © Stephen Janaway 2014
  • 49. Thinking a Mobile Device Lab Is a One Time Investment © Stephen Janaway 2014
  • 50. Staying In The Office © Stephen Janaway 2014
  • 51. Don’t Just Test Here…
  • 52. On Here, and At Here…
  • 55. Forgetting About Battery Life © Stephen Janaway 2014
  • 56. Battery Life © Stephen Janaway 2014
  • 57. Ignoring OS Updates © Stephen Janaway 2014
  • 58. OS Updates • Always regression test your app or website before an OS update. • Get on the developer program for early OS versions. • Users update more quickly on iOS than Android. • Carriers often control update cycles. • Decide beforehand how many OS version you will/ can support. © Stephen Janaway 2014
  • 59. Thinking Mobile Test Automation Is Like Web Test Automation © Stephen Janaway 2014
  • 60. Automation Suggestions • Determine if you require cross platform support. • Avoid tools that use embedded servers or agents – they alter the production code. • Look for tools with active online communities. • Consider the future. © Stephen Janaway 2014
  • 61. My Picks • Appium: – Cross platform. – Well supported user community. – About to be more closely integrated in Selenium 3. • iOS Driver: – Well supported user community. – About to be more closely integrated in Selenium 3. • Calabash: – Well supported user community. – Heavily used at the BBC. © Stephen Janaway 2014
  • 63. Mobile Test Strategy – Top 5 Tips 1. Understand the potential user: – Which platforms and devices will they use? – How will they use the app or website? 2. Understand what OS versions are being used and when new ones are coming. 3. Test using simulators, real devices and users. 4. Understand the cost of automation: – It is where desktop automation was 6 years ago. – Best combined with extensive unit and integration testing. 5. Use an exploratory approach to manual testing: – Mobile device projects are typically short. © Stephen Janaway 2014
  • 64. “Be proud, your work will soon be in the hands of millions”
  • 65. Stephen Janaway Questions? www.stephenjanaway.co.uk @stephenjanaway www.testinginthepub.com

Editor's Notes

  1. Hi everyone. I’m Stephen and I’m going to talk about Mobile Testing.
  2. I’ve been in the industry for 14 years. I have a website and blog Follow me on Twitter.
  3. I also curate a regular podcast all about software testing. You can get more details at www.testinginthepub.com.
  4. Here’s what I’m going to be talking about.
  5. They can also be medical devices, smart readers, hand held scanners, and much more. Basically anything with limited memory, portable, preferably with a data connection.
  6. Android Wear - http://googleblog.blogspot.co.uk/2014/03/sharing-whats-up-our-sleeve-android.html Android FIT APIs Wearable camera’s NEST thermostats
  7. Remember when we had to carry all this stuff about? Now it’s just in one device. It’s taken less than 30 years to get to this point. Most people are carrying around a small mobile computer in their pocket.
  8. It’s much more than just devices Imagine if something went wrong…
  9. You are late for that important meeting because your phone alarm clock does not work. IPHONE EXAMPLE
  10. You are lost due to a problem with your phone’s navigation software. IOS7 MAPS EXAMPLE. OK – so of course you could use a map but that won’t actually direct you. And maybe you don’t even have maps anymore.
  11. Actual Hardware – drop it, get it wet, stuff is removable Consumer electronics – fast paced, device jealousy.
  12. To people they are much more. Users are more emotionally attached to their mobiles than their PCs. PERCEPTION OF QUALITY Often a person’s phone is the first thing that they look at when they wake up. In fact it wakes them up!! HTC advert and personal experience. People classify themselves by what device they carry. It becomes part of them. The apps they use become status symbols. Think about users perceptions of quality on each device.
  13. MOBILE USERS WANT A MOBILE OPTIMISED EXPERIENCE BUT THEY DON’T WANT LESS….
  14. I JUST WANT TO POST A COMMENT. HEAR ME!! Mobile Web Mobile sales have already overtaken desktop sales, and mobile Internet usage is predicted to overtake desktop internet usage by 2014. It is only logical that mobile search will overtake desktop search at some point in the near future as well. Since 67 percent of users claim they are more likely to purchase from a mobile-friendly website
  15. Mobile users have far higher expectations of mobile apps when compared to other software like browser applications. The majority of reports and surveys state that nearly 80% of users delete an app after using it for the first time! The top four reasons for deletion are always bad design and usability, loading time and crashes immediately after installation. PERCEPTION OF REQUIRED QUALITY IS HIGHER Nearly 60% of users will delete an app that requires registration, while more than half of users expect an app to launch in under two seconds. If the app needs more time it gets deleted. Again more than half of users experience crashes the very first time they start an app.
  16. Apps stores enable this. Do you download apps with consistently bad review and 1 star? Often lots of other apps to choose from. YOUR BRAND IS YOUR BRAND, NO MATTER WHICH PLATFORM!!! Find out what your users want and understand more about your users via apps stores. So what’s important? Testing, at all levels of app design!! SO LET’S LOOK IN MORE DETAIL AT TESTING AN APPLICATION.
  17. What makes mobile testing different? 
We will touch on these much more in the rest of the presentation.
  18. So, as testers, what do we need to understand. Firstly...usability. Conduct real user tests. Observe users using the product. Make friends with your UX people. Use user profiles. Screen is smaller and used one handed. TEST LIKE THE USER DOES!!!! In app analytics. Website analytics. User persona’s. Similar applications.
  19. So what are you looking for when you test for mobile? Consistent brand Engagement - People typically have more than one device. They get distracted.
  20. You need to learn a whole new language when testing mobile.
  21. Think about things like screen rotation, as well.
  22. Mobile devices are low power, multi-tasking, lower performance and memory. Stress them. Test for the interactions.
  23. Q to the class – what should we think about? Typically two processors – main plus GPU. Max 3GB RAM. 16/32GB ROM. Memory leaks, etc.
  24. Users hate you wasting their data allowances. Test for it.
  25. Being able to test effectively for mobile means understanding the mobile world in detail.
  26. The number of mobile cellular subscriptions is about to equal the number of people. Last time I looked it was - 7,259,491
  27. Android is king. Note Series 40 which MS just killed.
  28. Interesting thing here is the leader is the Android browser worldwide NOT Chrome as shipped on most recent Android devices. AOSP… UC Browser is a 3rd party browser which uses compression technology.
  29. So here’s what I think we’ll be doing in the future of mobile testing.
  30. Here’s the challenge – how to ensure testing covers enough of them? How do you keep on top? More wearables means more devices to get. More complicated test labs. More interactions between devices. More difficult test scenarios. IT MEANS USERS ARE BECOMING MULTI DEVICE AND THEREFORE MORE DISTRACTED AND MORE TIME POOR. Apps need to really grab the attention.
  31. Tricky – lots of versions in live.
  32. And it’s also about screen sizes. And carrier software.
  33. Subscribe to the relevant updates/ seed notices from platform manufacturers Make friends with UX
  34. So what should you do? Not just simulators: They don’t fully emulate the device (some are better than others). They are not running on low performance hardware. Cannot use on real network connections. Security is normally permissive. Just be aware that cloud based services have some issues with latency, and try them out first to make sure you are happy using them.
  35. A great idea. Why not start one?
  36. As testers – start taking control of this.
  37. Users still don’t trust mobile as much as desktop for commerce. Browse on mobile, buy on desktop. But this is slowly changing with things like Paypal and google wallet. APPLE PAY. Barcode scanning, contactless payments, london tube, location based use cases. Security is becoming more and more important. What does this mean for testing?
  38. Test for security issues. Think about things like browser security, cookies, etc. Security settings. App permissions.
  39. It’s your companies brand, no matter which platform. As testers we need to ensure companies are taking security on mobile much more seriously. User and credit card data. Stored on device or server? Stored encrypted? Stored on SD or SIM card? Removed when the app is uninstalled? Security of channel. Wifi sniffing. Man-in-the-middle. User left logged in. Can the app be de-compiled? MobiSec and OWASP.
  40. And it’s not only real security issues. Perception of security. Application permissions. Watch out for these. Are they appropriate? This is a flashlight app (malicious example but sometimes it’s just due to lazy coding). TEST FOR THIS BEFORE LAUNCH. Users perception of security (affects mobile web too).
  41. What does this mean for testing? Localisation Different character sets Right left languages Is the app available in different languages? What happens if you change languages? What happens if you change keyboard language? Watch out for right-to-left languages – a frequent source of UI bugs.
  42. Wearables drive this.
  43. So – for the future why not make a commitment to avoid some of the common Mobile Testing Gotcha’s?
  44. Mistake number 1 As we mentioned before: They don’t fully emulate the device (some are better than others). They are not running on low performance hardware. Cannot use on real network connections. Security is normally permissive.
  45. Mistake number 2 A mobile device lab is not just for Christmas! This is consumer electronics. You need to keep it updated regularly. Think about open device labs as well.
  46. Mistake number 3 Users are not in your office!!!
  47. Don’t just test in perfect Wifi and perfect light. Test in low signal conditions. Opensignal.com is brilliant for this.
  48. Test on the train. Overloaded networks. – test in busy areas. Waterloo station is the rush hour for example  Test for swapping between 2G/ 3G/ Wifi. Speed.
  49. Deadspots Mention testers knowing where all the deadspots in Nokia offices are. Find a walking route with poor connections and record it using a fitness app.
  50. Mistake number 4!!
  51. Applications with poor battery life are hated by users. Moves being a good example. Some users just blame the device. Battery life – typically a problem with apps that use advertising incorrectly (Android especially). Very, very important. Typical scenarios: Normal Low battery Battery drain time Affect of poor/ no network connectivity. Does the app keep trying to connect?
  52. Mistake number 5 Ignoring OS updates or thinking that they don’t affect your app.
  53. iOS and Android (to Google Play devices), happen quickly. For the rest carriers control things and their acceptance testing adds significant delay. Always test before the update. Get on the developer programs and get early versions of the OS. But beware – they are not always the final functionality so be sure to also regression test in live after official update.
  54. Mistake number 6 Mobile test automation is like desktop test automation 6 years ago. You may want to consider pushing for more unit and integration tests before spending time on mobile UI automation.
  55. So – if I could sum up a mobile test strategy in 5 top tips, what would it be?
  56. And I’ll leave you with a quote from someone I used to work with in Nokia, which sums up why I think mobile and mobile testing is cool.
  57. As a project manager once said to me in Nokia: “Be proud, your work will soon be in the hands of millions”. No pressure on us testers, right? 
  58. Please ask any questions. REMEMBER TO REPEAT THE QUESTION BACK TO THE AUDIENCE BEFORE ANSWERING. I’m about for the rest of the conference so please come and have a chat. Or you can contact me through my website or via Twitter.