Your SlideShare is downloading. ×
Tap into mobile app testing@TDT Iasi Sept2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Tap into mobile app testing@TDT Iasi Sept2013

374
views

Published on

Tap into mobile app testing@TDT Iasi Sept2013

Tap into mobile app testing@TDT Iasi Sept2013

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
374
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
27
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Tap into Mobile App Testing Andreea Haras 11/09/2013
  • 2. Agenda • Introduction • Mobile Testing vs. Traditional Web-Based Testing • Challenges in Mobile Testing • Guidelines & Best Practices • Conclusion • Q&A
  • 3. About me • Andreea Haras – andreea.haras@gmail.com • 4+ years experience as software tester • Mobile testing enthusiastic • Interested in new technology
  • 4. Why mobile testing?
  • 5. Why mobile testing? • “The mobile market is no longer just communication. It’s internet, music, video, web, social networks, navigation and everything else... The mobile ecosystem now embraces anything a consumer wishes to do either on the move or on a handheld device at home” - Forrester
  • 6. Why mobile testing? • • “The mobile market is no longer just communication. It’s internet, music, video, web, social networks, navigation and everything else... The mobile ecosystem now embraces anything a consumer wishes to do either on the move or on a handheld device at home” - Forrester Applications running on mobile devices (e.g., mobile applications running on smart phones or new generation tablets) are becoming so popular that they are representing a revolution in the IT sector.
  • 7. Why mobile testing? • • • “The mobile market is no longer just communication. It’s internet, music, video, web, social networks, navigation and everything else... The mobile ecosystem now embraces anything a consumer wishes to do either on the move or on a handheld device at home” - Forrester Applications running on mobile devices (e.g., mobile applications running on smart phones or new generation tablets) are becoming so popular that they are representing a revolution in the IT sector. Over 1.6 million mobile applications have been developed, available in Apple App Store and Google Play (www.mobithinking.com)
  • 8. Why mobile testing? • • • • “The mobile market is no longer just communication. It’s internet, music, video, web, social networks, navigation and everything else... The mobile ecosystem now embraces anything a consumer wishes to do either on the move or on a handheld device at home” - Forrester Applications running on mobile devices (e.g., mobile applications running on smart phones or new generation tablets) are becoming so popular that they are representing a revolution in the IT sector. Over 1.6 million mobile applications have been developed, available in Apple App Store and Google Play (www.mobithinking.com) Estimates for downloads of apps in 2013 range from 56 to 82 billion (www.idc.com) and a prediction of 200 billion global downloads in 2017 that will be worth US$63.5 billion (www.portioresearch.com)
  • 9. Native, Traditional, Hybrid, Oh My! • Native Apps: coded with a specific programming language. They are fast, reliable, and powerful, but are tied to a mobile platform and downloaded from a central portal by OS (i.e. Apple App Store, Google Play). That means you must duplicate them using the appropriate programming language in order to target another mobile platform. Netflix, Pandora, Shazam.
  • 10. Native, Traditional, Hybrid, Oh My! • Native Apps: coded with a specific programming language. They are fast, reliable, and powerful, but are tied to a mobile platform and downloaded from a central portal by OS (i.e. Apple App Store, Google Play). That means you must duplicate them using the appropriate programming language in order to target another mobile platform. Netflix, Pandora, Shazam. • Traditional Mobile Web Apps: mobile web sites designed to match every webenabled phone. They’re accessed through the mobile device’s web browser and do not need to be downloaded/installed. They re-direct to the site’s M Site. Amazon, eBay, MSN are good examples.
  • 11. Native, Traditional, Hybrid, Oh My! • Native Apps: coded with a specific programming language. They are fast, reliable, and powerful, but are tied to a mobile platform and downloaded from a central portal by OS (i.e. Apple App Store, Google Play). That means you must duplicate them using the appropriate programming language in order to target another mobile platform. Netflix, Pandora, Shazam. • Traditional Mobile Web Apps: mobile web sites designed to match every web- • Hybrid Apps: rely on development frameworks and are written with the same enabled phone. They’re accessed through the mobile device’s web browser and do not need to be downloaded/installed. They re-direct to the site’s M Site. Amazon, eBay, MSN are good examples. technology used for websites and mobile web implementations. They are hosted or run inside a native container on a mobile device. They ensure cross-platform compatibility and can access the phone’s hardware (camera, GPS, user’s contacts, etc.). Foursquare, Twitter, Wikipedia and Facebook are good examples.
  • 12. Pop Quiz There are seven (yes, seven!) mobile operating systems that are recognized in the industry, other than the top 2 (iOS and Android), can you name one or more of the other five?
  • 13. Can you name the differences between Mobile Testing and Traditional Web-Based Testing?
  • 14. Mobile Testing vs. Traditional Web-Based Testing • Integration testing  Within areas of the Device Hardware  Battery, light, ringer, sleep, wake, usage, connectivity loss or change in speeds (“elevator test”), camera  Integration with other Apps on the device  Facebook, Twitter, etc.
  • 15. Mobile Testing vs. Traditional Web-Based Testing • Integration testing  Within areas of the Device Hardware  Battery, light, ringer, sleep, wake, usage, connectivity loss or change in speeds (“elevator test”), camera  Integration with other Apps on the device  Facebook, Twitter, etc. • Interruption Testing  How the App behaves with user-controlled and automatic interruptions
  • 16. Mobile Testing vs. Traditional Web-Based Testing • Integration testing  Within areas of the Device Hardware  Battery, light, ringer, sleep, wake, usage, connectivity loss or change in speeds (“elevator test”), camera  Integration with other Apps on the device  Facebook, Twitter, etc. • Interruption Testing  How the App behaves with user-controlled and automatic interruptions • Device-specific Gesture Testing  Pinching, Zooming, Shaking, Orientation, Eye Scrolling, Eye Pausing, Tilting, Swiping, Press and Hold, Tapping
  • 17. Pop quiz What mobile testing challenges do you think there are?
  • 18. Challenges in Mobile Testing • The time frames • Variety of Devices & OS Versions     Display density Different browsers Network connections Installation issues
  • 19. Challenges in Mobile Testing • The time frames • Variety of Devices & OS Versions     • Display density Different browsers Network connections Installation issues Simulating real life scenarios     Call interrupts Low battery Alarms Network issues
  • 20. More challenges in Mobile Testing • Usability  Accessibility  Intuitive  Mobile experience
  • 21. More challenges in Mobile Testing • Usability  Accessibility  Intuitive  Mobile experience • Automation  What to automate?  Tools  Effort vs Value
  • 22. More challenges in Mobile Testing • Usability  Accessibility  Intuitive  Mobile experience • Automation  What to automate?  Tools  Effort vs Value • Non functional testing  Performance  Security  Load testing
  • 23. Guidelines & Best Practices • Determining your Mobile Test Matrix • Setting up your Lab: Phones • Setting up your Lab: Tablets • Checklist • Key types of Mobile QA Testing: Manual or Automated Testing? • Things to remember • Test Automation Tool Selection – Few factors to consider
  • 24. Determining your Mobile Test Matrix The most difficult facet of the testing matrix is device fragmentation. There are nearly 4000 separate Android device models! How do you make sure that all users get a consistent experience? Device manufacturer market share (Q1 2013)
  • 25. Determining your Mobile Test Matrix Android device fragmentation Source: BGR
  • 26. Determining your Mobile Test Matrix Multiple approaches to build a “Mobile Test Matrix” • • • • Group devices based on the hardware and/or software specification and prioritize them User base – Build a test matrix based on the device and software version the user uses Use statistics from vendor market place (especially useful for native apps like games, utilities, etc.) Risk based and functionality based matrix
  • 27. Determining your Mobile Test Matrix Multiple approaches to build a “Mobile Test Matrix” • • • • Group devices based on the hardware and/or software specification and prioritize them User base – Build a test matrix based on the device and software version the user uses Use statistics from vendor market place (especially useful for native apps like games, utilities, etc.) Risk based and functionality based matrix Example: • • • Group 1, priority C: Small devices with a small CPU, RAM and low resolution. Older software versions and older browsers. Group 2, priority B: Mid-range devices with an avg. CPU, RAM (<512 MB), good screen size and resolution. The software is not the latest. Group 3, priority A: High-end devices with a dual/quad-core CPU, RAM (>512 MB) and a high screen resolution. Latest software versions.
  • 28. Setting up your Lab: Phones • • • • • Newest, highest resolution, high density, normal sized phone running latest version of OS. (Samsung Galaxy S4, iPhone 5S) Oldest, smallest, low density small sized phone running oldest supported version of OS. (iPhone 3, HTC Tattoo) Middle of the range phone matches highest usage, normal size, high density, normal resolution, highest usage OS (Samsung Galaxy S2) Phone with Highest Market Share (Samsung Galaxy S3, iPhone 4S) Budget smartphone - normal size, large screen, mdpi, old OS (Dell Venue)
  • 29. Setting up your Lab: Tablets • XLarge size high end tablet (10”), xhdpi, latest OS. (Nexus 10) • Large size Tablet (5-7”), med density, legacy OS (Kindle Fire) • Phablet (Phone/Tablet Hybrid) (5”) xhdpi, large size, latest OS, strange aspect ratio (LG Intuition, Samsung Note) • iOS running iOS 5 or lower (iPad 2) • iOS running iOS 6 (iPad mini, iPad 3)
  • 30. Checklist
  • 31. Key types of Mobile QA Testing: Manual or Automated Testing? Functional testing Performance testing Memory Leakage Testing Interrupt testing Usability testing Installation testing Upgrade testing
  • 32. Things to remember • • • • Start testing early. Use emulators / simulators Use the app on different carriers and network connections like 3G, 4G, WiFi, or LTE Combine testing tools and integrate them into a continuous integration system When possible, have a group of BETA testers to help with real-world testing • Consider the problem space: go mobile to test mobile • Don’t assume that because it works on one device, it will work on all
  • 33. Test Automation Tool Selection – Few factors to consider • • • • • • • • • • Supported devices Emulator support Availability of integrated simulators Types of connection supported Multi-device support (this is important to run same test on multiple devices) Compatibility with existing automation tools Security - VPN connections, data sharing, etc. Functionality supported - swipe, drag and drop, zoom, scrolling, pinching, rotating, etc Image / text recognition Debugging and test reporting
  • 34. Conclusions • Despite the challenges in mobile application testing, careful selection of target devices, using statistical techniques for test optimization and tools that maximize automation can ensure a cost effective mobile testing process.
  • 35. Conclusions • • Despite the challenges in mobile application testing, careful selection of target devices, using statistical techniques for test optimization and tools that maximize automation can ensure a cost effective mobile testing process. Given the stiff competition in the mobile apps market and variety of apps available for users to download, it is vital that you thoroughly test each and every aspect of the app before releasing it into the market, to ensure that the product meets the quality standard demanded by users.