It's a Phone First! How to Test Your Five-star Mobile Apps


Published on

Mobile application development shares many similarities-and some stark differences-with traditional web-based development. To build, test, and deploy five-star mobile applications, your organization needs-from inception-a focused test strategy to drive quality. Employing the wrong approaches and tools can leave your business sponsors and clients wondering what went wrong. Will Hurley outlines the current mobile landscape and explains what can and cannot be controlled in the mobile lifecycle. He explores the current landscape and limitations on tools for testing mobile apps, and offers guidance on what-and what not to-automate. With Will's guidance, you’ll learn how to establish a mobile lifecycle test strategy that is both leading edge and practical. From high-level feature classifications that are meaningful to your business to low-level test types and approaches that practitioners need to know, Will shares the knowledge you need to release a five-star app.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

It's a Phone First! How to Test Your Five-star Mobile Apps

  1. 1.           BW4 Concurrent Session  11/7/2012 10:15 AM                "It's a Phone First! How to Test Your Five-star Mobile Apps"       Presented by: Will Hurley Quality and Security Services                 Brought to you by:        340 Corporate Way, Suite 300, Orange Park, FL 32073  888‐268‐8770 ∙ 904‐278‐0524 ∙ ∙
  2. 2. Will Hurley Quality and Security Services A practice leader of quality management and security for Idea Integration Corporation, Will Hurley has been at the crest of numerous software development waves during his twenty-four year career. Will represented the US Air Force in the DoD’s first joint service system architecture, led CMM assessments, and supported the development of the CMMi. After leaving the Air Force, he consulted for industry-leading companies, helping them adopt and apply new development methods. In his current position, Will consults with clients in North America on achieving desired system and product lifecycle characteristics.
  3. 3. 28/09/2012 It’s a Phone First!  How to Test Your Five‐star Mobile Apps ©2012 Idea. All rights reserved 1 Today’s discussion Idea and I Acknowledgements Five Star Mobile Application improving mobile project outcomes and time to market. ©2012 Idea. All rights reserved 2 1
  4. 4. 28/09/2012 Since 1986, we’ve helped some of the world’s largest and most respected brands thrive in the digital space. space 3 ©2012 Idea. All rights reserved The Presenter Will Hurley Practice Executive Quality Mgmt. & Security Idea (781) 831-0389 ©2012 Idea. All rights reserved USAF Systems Engineer TBMCS Core Systems Lead Joint Service Architect GCCS USAF Data Administrator Consultant Small Business Owner Practice Leader 4 2
  5. 5. 28/09/2012 Acknowledgements This discussion captures mobile testing insights developed at Idea’s Global Solution Centers in Nashua, NH and Houston, TX. Idea’s insights are supported by more than 10 y years of mobile g pp y development and testing. Insights are not SDLC or tool specific. Data is updated on the quarter through a verity of sources. ComScore® Netbiscuits® Harris Interactive® OPA Crittercism 5 ©2012 Idea. All rights reserved 5 5 Star Mobile applications 6 ©2012 Idea. All rights reserved 6 3
  6. 6. 28/09/2012 5 Star Mobile radar chart ©2012 Idea. All rights reserved 7 Ecosystem insights Understanding the mobile ecosystem and trends increase test effectiveness and product quality characteristics. o Capture more than 80% of smartphone users with 3 network carriers. ih k i o Capture more than 85% of smart phone users with Android and iOS platforms. o Application use is growing faster than browser use. o Android users spend more time on carrier networks. o iOS users spend more time on wireless networks. ©2012 Idea. All rights reserved 8 4
  7. 7. 28/09/2012 Ecosystem carrier US Carrier Smart Phone Market Share As of Aug 2012, 114 million Americans own smart phones. p Verizon  31.60% 60 percent of all mobile devices acquired in the U.S. are smartphones. Verizon market share has grown fastest driven by iOS. Other 15% AT&T  26.30% Consider market share when signing contracts for cloud services. T‐Mobile  9.50% Sprint  9 50% Sprint 10.50% Tracfone  7.10% 9 ©2012 Idea. All rights reserved Ecosystem oem Samsung Galaxy S III leading 2012 sales may be passed by iPhone 5. US Smartphone OEM iPhone 4 took the top US device sales spot in 2011. Samsung,  25.6% Other,  15.8% iPhone 3GS took 2nd place. LG, 18.4% iPhone 4S took 3rd place. BlackBerry Curve 8530 4th place. RIM, 6.3% Apple,  16.3% 16 3% HTC Evo 4G 5th place. HTC, 6.4% Motorola,  11.2% HTC is the most active OEM for Android smartphones. 10 ©2012 Idea. All rights reserved 10 5
  8. 8. 28/09/2012 Ecosystem platform US OS Share Other 0.5% Android 52.2% Symbian 0.8% Microsoft 3.6% RIM 9.5% Apple 33.4% 66 percent of Americans ages 24-35 own a smartphone smartphone. Apple and Android market share are growing. Market share for all other platforms is shrinking. Android devices out number iPhones by 3-to-2 in the U.S. market. 11 ©2012 Idea. All rights reserved Platform fragmentation Fragmentation is a real but manageable issue for all platforms. A d id f i Android faces more i issues since it resides in an open ecosystem. Android OS adoption is slower than iOS. Trades reporting >½ of all iPhone users that can upgrade to iOS5 have. iOS6 expected to drive largest upg ade population date upgrade popu at o to date. Crash data are normalized as crashes as a percentage of each application launch. ©2012 Idea. All rights reserved 12 6
  9. 9. 28/09/2012 Ecosystem tablet Tablet Share (Platform) Other 12% iPads account for more than 70% of tablet network traffic. Apple 46% More than 80% of tablet traffic is wireless, not cellular. Tablet adoption is balanced between males and females, trending older with households earning $50K or more. Android 42% Kindle Fire 28% An estimated 74.1 million consumers are using tablets. iPad 2 31% ©2012 Idea. All rights reserved 13 Customer Experience insights Mobile users expect more. o Good design is not accidental. Get help if you need i d it. o o Good design is sticky. o Analytics portray the customer voice. Unstructured or unsolicited feedback has more action value than a survey. ©2012 Idea. All rights reserved 14 7
  10. 10. 28/09/2012 Experience components How easy is it for your customer to use your application. The application does what your customer wants it to do. How does the application make li ti k your customer feel. 15 ©2012 Idea. All rights reserved Customer experience Smartphone Usage Music,  24.50% Games,  31.80% Social net  or Blog,  35.70% ©2012 Idea. All rights reserved Text  Text message,  74.60% Used  browser,  48.60% Used App,  48.60% Four in ten mobile users are disappointed with applications from their favorite brands. Three-quarters of users surveyed believe a company's mobile app should be easier to use than its Web site. Users spend more time with mobile applications than their mobile browser. 82% of iOS and Android Apps crash once out of every 10 loads Less than 5% of mobile apps are still in use 3 months after downloading. 16 8
  11. 11. 28/09/2012 Experience sticky ©2012 Idea. All rights reserved 17 Experience design ©2012 Idea. All rights reserved 18 9
  12. 12. 28/09/2012 Experience themes Based on conversations, we found several common themes: • All purchasing because it’s required by their apartment complex complex. • All participants were referred to site via a brochure. • Participants pleased overall with the simplicity of the online process • Liked being able to select from presented options in order to simplify and speed up the process. • Appreciated the ability to get additional details where necessary. • A majority didn’t use these features. • All participants pay annually for their policy. ©2012 Idea. All rights reserved 19 Experience maturity o Functional requirements are only part of the customer o o o o experience. Key moments in customer interaction demand good design. If you don’t have experienced designers on your team bring them in. Understand accessible and emotional goals at every key moment. Don’t predict how people will respond to a new experience, experience test it it. o Allot the time to make incremental changes on all key moments. ©2012 Idea. All rights reserved 20 10
  13. 13. 28/09/2012 Functional insights It’s a phone first. o Test while preforming common smart phone tasks. k o It’s cheaper, faster and easier to test in emulated environments first. o Testing types are the same as Web based ntier applications o Manage functional and non-functional insights across p j g projects. o o Create a mobile test strategy document. Reduce tool and process resistance between groups. ©2012 Idea. All rights reserved 21 Manage scope Carriers Operating Systems OEM Devices Com & Sensors ©2012 Idea. All rights reserved 22 11
  14. 14. 28/09/2012 Manage knowledge ©2012 Idea. All rights reserved 23 Manage modularity • • • • • • • • • Display and Graphics Input Devices Sensors Permissions Intents Phone and messaging Deploy and Upgrade Analytics A l i Your application’s key moments ©2012 Idea. All rights reserved 24 12
  15. 15. 28/09/2012 Emulation(Simulation) testing Does not require the physical device. Can still use network services. Emulation requires experience with specific  platform development tools. Static security and performance testing can  be performed on source code. Emulation is best for: Investigating architectures and  application layers g g y Agile testing  of key features and  interactions  Evaluating code quality characteristics Research HTML5 characteristics. Planning for automation 25 ©2012 Idea. All rights reserved Device testing Does require a physical device. Testing on the device is best for: U E i User Experience – th l t 100 il the last 100 miles. Acceptance testing – working with  gestures, sensors and hardware. Soak Testing – how will the device  perform over time.  Provisioning – installation and  removal. Cloud providers and plugins for popular  automation tools support both functional  and performance testing. No need to run out and buy all platforms.  ©2012 Idea. All rights reserved 26 13
  16. 16. 28/09/2012 Automated testing Browser automation Tools can be used for ‘closest‐match’,  browser testing on or off the device. ‘Closest‐match’ browser automation  possible on some devices ‐ Safari 3  for the iPhone driven on a PC. User‐Agent header data to drive  compatibility testing logic. Work closely with development team to  manage and control Web 2.0 features. Third party libraries can speed  implementations. Physical devices and Cloud services can be  used to test physical platforms when  defects are detected. 27 ©2012 Idea. All rights reserved Automated testing Application automation Tools can be used in emulated  environments and on the device. Installation of some type of  automation controller is required on  the device. Controllers consume resources. Plug‐in and Cloud support is growing  and stable. Blu Ne eto ar oth Fie / lld Cloud services may not provide  object recognition. Additional fixtures may be required for  sensor (GPS, Accelerometer) and  hardware (camera, battery) automation. Applications may incorporate browser  engine. ©2012 Idea. All rights reserved 28 14
  17. 17. 28/09/2012 Functional details Create a mobile test strategy document. Tight control of manifest and OS support. Identify development and testing tool stack. Explain how you plan to manage fragmentation – OS, device, sensors, UI, marketplace. Test teams should be working in emulated, physical and cloud based environments. Test teams should manage insights across releases. HTML5 platform insights Carrier and Device insights OS / Platform insights Security insights Marketplace ©2012 Idea. All rights reserved 29 Security insights Protect your brand. o Address security issues programmatically and start now! o Static analysis is a highly effect approach to address coding issues. o Dynamic analysis, Security Penetration testing, is your last line of defense. o SD memory should be tested ©2012 Idea. All rights reserved 30 15
  18. 18. 28/09/2012 Security testing Static Application Security Testing (SAST) Dynamic Application Security Testing Static analysis tools can address unsafe Static analysis tools can address unsafe  practices earlier in the SDLC. Clang; AppValidator; AnalytiX SAST should be integrated with build and  reporting practices. Evaluate application permissions.  Mobile devices are susceptible to Mobile devices are susceptible to  common browser exploits and more.   Emulators or devices can be used to  ensure applications don’t bleed sensitive  data on to the device or to the network.  Keep up to date with known exploits and  add test cases to qualify the impact. Don’t build on or deliver on compromised  platforms. platforms ©2012 Idea. All rights reserved 31 Performance insights Protect your brand. o Static analysis is a effect approach to address performance issues. p o Dynamic analysis, Security Penetration testing is your last line of defense before delivery. o SD memory should be tested. ©2012 Idea. All rights reserved 32 16
  19. 19. 28/09/2012 Performance testing Static Application Performance Testing (SAPT) Performance Testing Static Analysis tools find issues earlier in Static Analysis tools find issues earlier in  the SDLC and improve coding practices. Load stress and longevity testing are Load, stress and longevity testing are  critical to the customer experience. Tools address code that slow application  performance. Load and stress testing are focused on  middle and backend service layers.  In a 53K line project we found 328  instances of code that would  contribute to performance issues. SAPT should be integrated with build and  reporting. Longevity testing should look at both  backend services and the mobile device. Performance tools should support  bandwidth throttling and common RESTful  protocols. ©2012 Idea. All rights reserved 33 Final thoughts o Cloud based testing services are valuable but… o Will not achieve expected ROI without proper investigation. investigation o o Lead to Scrumfall application lifecycles. Can slow delivery times. o Functional, performance, security and test automation should be considered for each mobile project. o Simplify the testing space through active p y g p g management. o o Diagram test environments. Support agile approaches for all team members. ©2012 Idea. All rights reserved 34 17
  20. 20. 28/09/2012 Final thoughts o Develop sticky applications with the 5 star insights. o Fill out our five start survey and see how your team stacks up! ©2012 Idea. All rights reserved 35 18