Mobile Testing Tools 101

749 views

Published on

The burgeoning use of mobile devices has created enormous opportunities for organizations to leverage mobile to increase sales, advertise products, and collaborate with internal and external resources. However, with increasing usage, the need to perform testing on these devices is increasing significantly. This is not an easy task considering the number of devices, device operating systems, and operating system versions. To manage the number of variations, organizations rely on mobile testing tools to support their testing efforts. David Dang shares his experiences analyzing numerous mobile testing tool platforms for a prominent shopping network. Learn how identifying the "right" mobile testing tool depends on multiple factors such as supported devices, level of testing, resources, and required integration with other tools. Take back to share with your team a review of common tools on the market and the pros and cons of each.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
749
On SlideShare
0
From Embeds
0
Number of Embeds
30
Actions
Shares
0
Downloads
55
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Mobile Testing Tools 101

  1. 1. T22 Mobile Testing 5/2/2013 3:00:00 PM Mobile Testing Tools 101 Presented by: David Dang Zenergy Technologies, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. David Dang With more than fifteen years in IT, David Dang has focused on test automation, quality assurance, and test process improvement. David has extensive experience in designing, developing, and implementing advanced test automation frameworks that provide higher ROI by significantly reducing maintenance while increasing reuseability. David has worked with more than 100 companies to assist and drive their test automation and QA projects. He works with all the major license-based test automation tools and open source alternatives such as Selenium. David is a mobile automation expert; an HP Certified Instructor (CI) for QuickTest Professional, WinRunner, and QualityCenter; and frequent speaker at QA and testing conferences.
  3. 3. 4/16/2013 Mobile Testing 101 STAREAST 2013 Presented By: David Dang Slide 1 Software Peace of Mind in Action ??? Introduction Challenges of Mobile Testing Factors to consider when establishing a manual testing strategy to test mobile websites and mobile apps Conclusion Slide 2 Software Peace of Mind in Action 1
  4. 4. 4/16/2013 ??? Introduction Companies use mobile websites and applications to harness the power of mobile devices to increase sales, market products, create collaboration with internal resources, offer convenience to their customers, etc… With exploding usage, companies must perform more QA/testing on mobile websites and apps to minimize defect risks. Slide 3 Software Peace of Mind in Action ??? Discussion Why test web applications and apps on mobile devices? What are the challenges? How do you think mobile websites and apps affect the way we do testing? Slide 4 Software Peace of Mind in Action 2
  5. 5. 4/16/2013 ??? Challenges of Mobile Testing In addition to “traditional” testing challenges such as: Verify functionalities Verify business process flow Verify look and feel Verify data Verify user experience Mobile has other factors that impact testing. Slide 5 Software Peace of Mind in Action ??? Challenges of Mobile Testing Carriers – Verizon, AT&T, T-Mobile, Sprint, etc. The two common local network standards are CDMA and GSM. Each network has a different way of tunnelling the data into the mobile device. This tunnelling can have an impact on the speed and access to the web proxy of mobile application servers Devices – Smart Phones, Tablets, iTV New devices with more features and functionality are released often. Most of these devices must be tested to ensure the mobile application works correctly. The challenge is to keep up with the changes and ensure quality in the existing and new devices Slide 6 Software Peace of Mind in Action 3
  6. 6. 4/16/2013 ??? Challenges of Mobile Testing OS and OS version – iOS, Andriod, Blackberry, Windows With the mobile Operating System (OS) segment being so volatile , it is difficult to determine the trend within the marketplace. To add to the complexity, many devices run on different versions of the same OS. The challenge is to pick the “right” combination to test, seemingly an impossible task Location Based Service Location based service is a tracking service based on a mobile device’s current location. It can be used for target marketing of customers in a certain location. This can take the form of coupons, special offerings, or incentives. In addition, location based services can be used to analyze consumer participation based on a region of the country Slide 7 Software Peace of Mind in Action ??? Challenges of Mobile Testing Portability of test cases for mobile web applications and applications With the number of platforms, operating systems, and devices, it is critical that test cases can be ported from one device combination to the next. The challenge is to create a methodology that encourages reusability and portability. Network speed The network speed (Extended Coverage, 3G, and 4G) has an impact on the user experience. A slow responding mobile application will be interpreted by the user as poorly designed and low quality. The challenge is the ability to test across multiple network speeds to ensure the mobile application is functioning similarly across variations. Slide 8 Software Peace of Mind in Action 4
  7. 7. 4/16/2013 ??? Establish Manual Testing Strategies Agile vs. Waterfall development methodologies Agile – A software development methodology based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, crossfunctional teams Waterfall – A software development methodology based on a downward progression flow (like a waterfall) from Requirements Design Code Testing Production Slide 9 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Agile Impact Testing will occur earlier Timeline will be shortened Requires more “technical” testers Must determine how to do integration and regression testing Waterfall Impact Time to market; testing takes too long Too formal for small mobile websites and mobile apps Slide 10 Software Peace of Mind in Action 5
  8. 8. 4/16/2013 ??? Establish Manual Testing Strategies Mobile testing variations compared to “normal” application testing Mobile device functionality: portrait/landscape, swipe, pinch, external button on the device, etc. For mobile web application, mobile browser translation of html, dhtml, javascript, etc. For mobile application, different OS translations of mobile applications Mobile features: location based, network speed, on/off network, etc. Slide 11 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Mobile content vs. mobile website and app functionality Testing content look and feel • How do websites look on a Smart Phone, Tablet, and iTV based on screen compression and expansion? • Are there overflow of pages or objects? • How does the scrolling feature work on portrait and landscape? Test mobile website and app functionality • Reference data such as, all fifty states available on the drop down • Functionality such as login, search for item, shopping cart, etc. • Business process flow such as, create an order with an out of stock item Slide 12 Software Peace of Mind in Action 6
  9. 9. 4/16/2013 ??? Discussion Is it better to test on real devices or simulators? Slide 13 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Real Device Testing – The use of physical devices to test mobile websites and apps. The tester physically interacts with the device to determine the validity of the website or app. There are solutions on the market that offer real devices on the cloud; Device AnyWhere and Perfecto Mobile Mobile Simulator Testing – Mobile simulators/emulators are typically web-based or a computer application that mimics the behavior of a mobile device. It is used by QA to test how the website and/or application will appear and function in a mobile environment Slide 14 Software Peace of Mind in Action 7
  10. 10. 4/16/2013 ??? Establish Manual Testing Strategies Real Device Considerations Determine devices and OS combinations Use analytics to determine necessary coverage Develop a migration plan to move mobile apps onto the device Develop a device maintenance plan Consider real device alternatives; DeviceAnywhere or Perfecto Mobile Slide 15 Software Peace of Mind in Action ??? Establish Manual Testing Strategies – Analytics to Consider Slide 16 Software Peace of Mind in Action 8
  11. 11. 4/16/2013 ??? Establish Manual Testing Strategies Analytic to Consider Slide 17 Software Peace of Mind in Action ??? Establish Manual Testing Strategies DeviceAnywhere - DeviceAnywhere is a cloud-based Software-as-a-Service platform for testing and monitoring the usability, functionality, performance and availability of mobile apps and websites Slide 18 Software Peace of Mind in Action 9
  12. 12. 4/16/2013 ??? Establish Manual Testing Strategies Perfecto Mobile - The MobileCloud is a web-based Softwareas-a-Service platform that allows testers and developers to access a multitude of real mobile devices that are spread across different geo-locations Slide 19 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Mobile Simulator Considerations How closely the simulator mimics real device, e.g., OS, hardware, browser, etc. Purchase vs. open source simulator Will it work with both, mobile websites and mobile apps? Does it support multiple devices What functionality can be tested using the simulator; what are the gaps that must be covered by real devices Slide 20 Software Peace of Mind in Action 10
  13. 13. 4/16/2013 ??? Establish Manual Testing Strategies Some common simulation tools Opera Mini Simulator iOS Simulator Android Emulator MobiOne TestiPhone Slide 21 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Slide 22 Software Peace of Mind in Action 11
  14. 14. 4/16/2013 ??? Establish Manual Testing Strategies Real Device Pros Test what the end-user will experience Able to test mobile functionality, e.g., portrait/landscape, pinch, swipe Able to test mobile hardware, e.g., location based, network speed, etc. Able to test mobile websites and app’s look and feel of the device Experience the performance of the websites or apps Real Device Cons Expensive – device cost, phone plan, data plan, etc. Limited by number of same device and OS, e.g. two iphone 4s running iOS 5 Difficult to keep up-to-date on new devices Maintenance on all the devices Slide 23 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Simulator Pros Cheaper than real devices Easier to setup for testing (most of the simulators are webbased or standalone applications) Can test multiple instances (run simulators on multiple PCs) Simulator Cons Not the real device Hard to determine real user experience Cannot accurately test mobile functionality Slide 24 Software Peace of Mind in Action 12
  15. 15. 4/16/2013 ??? Establish Manual Testing Strategies Identify various levels of testing for mobile Unit testing • Test a unit of code • White box testing • Performed by development team Functional testing • Layout and content are correct • Mobile functionality such as pinching, swiping, and scrolling are working properly • Each function of the mobile website or mobile app is working properly • Perform both positive and negative tests Slide 25 Software Peace of Mind in Action ??? Establish Manual Testing Strategies Identify various levels of testing for mobile System testing • Business process driven testing • Test multiple functions at the same time • Ensure data is correct from one function to the next User acceptance testing • Verify the look and feel are consistent throughout the entire mobile website or app • Ensure all the functionalities are working properly • Ensure that the mobile functionality such as, pinching, swiping, and finger scrolling is working properly Slide 26 Software Peace of Mind in Action 13
  16. 16. 4/16/2013 ??? Establish Manual Testing Strategies Tips for manual testing strategies Understand the challenges of mobile testing Identify the device portfolio you must cover Understand the impact of the development methodology Determine if simulator, real device, or both, are needed Determine logical breakdown of activities including who does what, and at what time for QA/testing Slide 27 Software Peace of Mind in Action ??? Conclusion It is necessary and critical to test mobile websites and mobile apps to reduce the risk of defects and failures: Understand the challenges of testing on mobile devices Understand the factor that impacts manual testing strategy Slide 28 Software Peace of Mind in Action 14

×