Downloaded 26 times
How to Prevent App Failures with Real User Conditions
- 1. Prevent App Failures TestReal User Conditions
- 2. 2 Sveta Kostinsky Dir. FieldEngineering
- 3. Meet your Users! 6/16/20163© 2015,Perfecto Mobile Ltd. All Rights Reserved. Georgia PeterSara SamRoss
- 4.
- 5. Mobile Calendar 2016 6/16/20165© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 6. Mobile application releasefrequency • Most successful apps release 1-4 updates a month • Release should balance between bug fixes and feature updates • Updates is a great marketing tool for developers to leverage • Developers can speak with their user base via release notes or comment in the reviews 6/16/2016 6© 2016, Perfecto Mobile Ltd. All Rights Reserved. 2017 36 per year App releases per year *The IDC predicts that by 2017, most app releases will occur 36 times per year
- 7. Problem 6/16/2016 7© 2015,PerfectoMobile Ltd. All Rights Reserved. The Digital Reality Users expect apps to work in the mobile moment (regardless of conditions) Failure to satisfy is costly
- 8. Problem 6/16/2016 8© 2015,PerfectoMobile Ltd. All Rights Reserved. The Mobile Experience Quality Challenge 1) Build common user environment condition profiles into test suites 2) Assemble multiple solutions to execute tests across profiles 3) Integrate user condition testing within Continuous integration pipelines 4) Analyze app performance blind spots
- 9. True user centrictesting Optimize your LAB environment, LAB configuration and the choice of user flows to reflect your users. 6/16/2016 9© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 10. 6/16/2016 10© 2015,PerfectoMobile Ltd. All Rights Reserved. Georgia PeterSara SamRoss
- 11. 6/16/2016 11© 2016,Perfecto Mobile Ltd. All Rights Reserved. End user conditions in your lab Name: Sara Location: London Device: iPhone 6, iOS 9.2 Carrier: BT Network condition: 3G Background apps: Instagram, Facebook, Uber Align coverage planning to business priorities Automate lab configuration to end user conditions Multiple devices Mobile | Tablet | Desktop End user conditions
- 12. Quality in thedigital era needs to align to the business 12 © 2015, Perfecto Mobile Ltd. All Rights Reserved. Marketing / Business Business-Engineering alignment Automated lab configuration Communicate Risk in business terms Name: Sara Location: London Age: 25 Device: New Sensitivity to UX: High Behavior: Fashion magazine designer Engineering Name: Sara Location: London Age: 25 Device: New Sensitivity to UX: High Behavior: Fashion magazine designer Name: Sara Location: London Device: iPhone6, iOS 9.2 Carrier: BT Network: 3G Background Apps: Instagram, Facebook
- 13. 6/16/2016 13© 2016,Perfecto Mobile Ltd. All Rights Reserved. Name: Sara Location: London Device: iPhone6, iOS 9.2 Carrier: BT Network: 3G Background Apps: Instagram Persona dimension in the coverage matrix
- 14. Real user condition- Network • vs. corporate firewall • LTE to WiFi seamless transition • WiFi assist on iOS9 • Offline mode testing 14
- 15. Real user condition- Network • Weak network signal – will I be charged twice?? • Changing network conditions flows – 3G/LTE • Have to test the end to end flow on IPV6 • Always helps to test on real carrier network – SIP • Apple Network link conditioner vs. external emulation TCP layer acknowledgements timing 15
- 16. Real user condition- Working with 3rd party applications • Sharing the device Camera application • Notifications, incoming calls, incoming text messages • Posting to third party application • Performing sign in via a 3rd party application • Context switching between the apps 16
- 17. Real user condition- Memory and CPU • iPhone photos application • Low memory conditions on the device • Hybrid apps and low memory • Thermal throttling of CPU – LTE connection 17
- 18. Real user condition- Rotation state and language • Testing different orientation • The need for visual analysis 6/16/2016 18© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 19. User Centric Quality- Lab with Real end user environment conditions Simple Functional Testing Is Not Enough
- 20. Perfecto Wind Tunnel- Lab Environment Easily bring your end user experience into the lab Offer a test environment that expands coverage to real world user conditions So that, 1. User experience blind spots are exposed 2. Mobile-user expectations are met 3. Business outcomes are assured 6/16/2016 21© 2015,Perfecto Mobile Ltd. All Rights Reserved.
- 21. Demo time 6/16/2016 22©2016, Perfecto Mobile Ltd. All Rights Reserved.
- 22. Lab Configuration Is yourLAB up-to-date? No, really up-to-date? Set up an agile lab, responsive to adoption trends. 6/16/2016 25© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 23. Optimize your lab 6/16/201626© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 24. User flow BI andflow analysis as part of Persona 27 Where should I spend most of my time?
- 25. 6/16/2016 28© 2016,Perfecto Mobile Ltd. All Rights Reserved.
- 26. Next Webinar June30: Why Apps Succeed • Panel Discussion of New Survey Results Register today: http://info.perfectomobile.com/Why-Apps-Succeed- Registration.html 6/16/2016 29© 2016, Perfecto Mobile Ltd. All Rights Reserved.
- 27. Please complete thesurvey! svetak@perfectomobile.com +1-339-927-4343 Sveta Kostinsky
Editor's Notes
- #5 What does the engineers reality looks like
- #6 7 new Apple devices 2 HTC devices 4 Galaxy devices Samsung rolling out support for Marshmallow in Q1 iOS 9 – 77%; iOS10 – 35% in October Explain that this is a planning calendar we have put for the developers to prepare for the upcoming releases
- #7 Since iOS7, Apps update automatically , so no need to worry about adoption App updates are one of the best marketing tools app developers have at their disposal Have your app updated on major OS launch day - you might be featured in app stores by Apple or Google Shorter feedback loop between developer and user TIP: Have your app updated on major OS launch day - you might be featured in app stores by Apple or Google
- #8 People will switch brands given bad experience. It’s easy to switch to a different application
- #10 You can not cover EVERYTHING and you can not afford yourself cover
- #15 Firewall blocking only part of your Whatsapp traffic – such as real time voice but not the control traffic - whatsapp can’t be used at work for example. The app says answering but the call is never complete. So need to turn off wifi, switch to 3G , then I forget to turn wifi back on it’s a bad user experience Some wifi’s don’t support IPV6 and will fall back to IPV4 so you need to see how this will work in production environment
- #16 Double charged by Citi Some WIFI’s netwroks don’t support IPV6 and will fall back to IPV4 so you need to see how this will work in production environment Indexing server “Elastic search” don’t support natively IPV6 as a field type - for example used for Data mining of logs of the app or “client ip address” field – is it represented in IPV4 or IPV6 SIP and carrier firewall as a sample of why you need to test with Carrier network as well as WIFI Is Apple link conditioner good enough to understand your app performance CORRECT TCP stack behavior adapting to poor network conditions and specifically an expected RFC 1122 implementation section known as TCP delayed acknowledgement When NLC on Apple devices is used, it does not respect those specifications and results in faulty representation of the network situation.
- #17 Instagram handoff
- #18 When the system dispatches a low-memory warning to your app, respond immediately. Low-memory warnings are your opportunity to remove references to objects that you do not need. Responding to these warnings is crucial because apps that fail to do so are more likely to be terminated.
- #20 Users are sensitive to battery consumption Doze is pretty cool: it activates when your phone is left idle for a period of time, while unplugged and with the screen off. It will stop some CPU and network activity operating in the background for some Android apps, thus saving battery. Doze mode can interrupt the background activity of your apps in order to make these battery savings. Ultra Power Saving Mode on Galaxy S6 takes it one step further; it makes everything grayscale and only lets you use features that are absolutely necessary, like the Phone and Messages apps. It also disables data, Wi-Fi, and Bluetooth when your Galaxy S6 is asleep. BAR adds a layer of intelligence and efficiency to the way your apps act when you are not using them. It allows apps to temporarily open in the background, receive an update, and then freeze again so it cannot use system resources or drain your battery. Power Saving Mode limits CPU performance, reduces screen brightness, and turns off things like vibration feedback to make the most of the battery life you have left. Low power mode – email fetch, back ground app refreshing and animations . Any time you place your iPhone face down on a surface, be it a desk or a bed, it's able to detect its orientation. As such, the screen will forgo lighting up when it receives a push notification. When your iOS device reaches 20 percent battery, you're automatically prompted to enable low-power mode. Users can easily see what is your app battery usage
- #23 How often do you update your lab devices? How precise is the lab configuration? Who owns this process? How often do you get the usage information and update your lab?
- #26 How often do you update your lab devices? How precise is the lab configuration? Who owns this process? How often do you get the usage information and update your lab?