Your SlideShare is downloading. ×
0
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Innovate 2013 session 1243 mobile testing.v3

483

Published on

Charts for joint presentation between IBM and AT&T on mobile app testing and performance analysis.

Charts for joint presentation between IBM and AT&T on mobile app testing and performance analysis.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
483
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Addressing the challenges of mobileapplication testingLeigh Williamson (@leighwillia), IBM Distinguished EngineerCharles Rankin (@mtgofiend), IBM Rational CTO TeamBalsu Thandu, Director AT&T Developer ProgramSession 1243© 2013 IBM Corporation
  • 2. 2Please note the followingIBM’s statements regarding its plans, directions, and intent are subject to change orwithdrawal without notice at IBM’s sole discretion.Information regarding potential future products is intended to outline our general productdirection and it should not be relied on in making a purchasing decision.The information mentioned regarding potential future products is not a commitment,promise, or legal obligation to deliver any material, code or functionality. Informationabout potential future products may not be incorporated into any contract. Thedevelopment, release, and timing of any future features or functionality described for ourproducts remains at our sole discretion.Performance is based on measurements and projections using standard IBMbenchmarks in a controlled environment. The actual throughput or performance that anyuser will experience will vary depending upon many factors, including considerationssuch as the amount of multiprogramming in the user’s job stream, the I/O configuration,the storage configuration, and the workload processed. Therefore, no assurance can begiven that an individual user will achieve results similar to those stated here.
  • 3. 3The BroadestPortfolio ofMobileSolutionsThe DeepestSet of ServicesExpertiseNew IndustryPartnershipsand Resourcesfor DevelopersThe Industry’s Most Comprehensive Mobile Portfolio
  • 4. 4The IBM MobileFirst offering portfolioAnalyticsSecurityManagementIBM and Partner ApplicationsBanking Insurance Transport Telecom GovernmentIndustry SolutionsHealthcareRetail AutomotiveApplication & Data PlatformStrategy&DesignServicesDevelopment&IntegrationServicesCloud & Managed ServicesDevices Network Servers
  • 5. 5Scan & CertifyInstrumentTestIntegrateObtain InsightManageDeployDesign & DevelopThe IBM mobile application development lifecycle
  • 6. 6Quality is critical for mobile applicationsCustomer facing (B2C) applications–‘Face of the Business’–Engage customer personally and driveloyalty–User experience is key to brand perceptionEmployee facing (B2E) applications–Increase worker productivity–Speed decision making and action–Poor user experience compromisesinvestment No more 1 star mobile apps !!
  • 7. 7How is mobile application testing different?Device platforms, fragmentation, and growth– iOS, Android, BlackBerry, Windows Phone, Symbian, etc.– Multiple form factors and screen resolutions– Device refresh every 24 monthsNew capabilities to test– Camera, GPS, direction, orientation, voice, etc.– Data and power consumptionMore network considerations– Multiple carriers, variable throughput and latency,disconnected use, network switching, etc.User experience is more critical
  • 8. 8Middle Tier ServerClient Tier Devices Back-end Data & ServicesTesting involves much more than just the mobile front-endTesting the end-to-end mobile solution
  • 9. 9What do mobile testers need?
  • 10. 10Test planning, reporting, and dashboardingTest case/suite/environment constructionIntegration with ALMIntegration with multiple execution enginesWhat do mobile testers need?
  • 11. 11Still most used mobile testing methodNecessary for some validationIntegration with test managementFrequently key in outsourcing andcrowd sourcing scenariosWhat do mobile testers need?
  • 12. 12Isolate front-end from back-endControl over data and interfaceDecrease expensive set-up costsWhat do mobile testers need?
  • 13. 13Network connectionsServer dataClient dataInjections and data flow analysisWhat do mobile testers need?
  • 14. 14Middle-tier and back-endMobile client codeMobile networkMobile client experienceWhat do mobile testers need?
  • 15. 15Device / platform agnosticApplication instrumented automationOptical / image recognition automationWhat do mobile testers need?
  • 16. 16Remotely accessibleShareable across teams / geographiesAccess to diverse selectionDevice clouds minimize cost of ownershipWhat do mobile testers need?
  • 17. 17IBM / Rational Testing SolutionMiddle Tier ServerClient Tier Devices Back-end Data & ServicesTest automation on deviceMobile device cloud accessRTW/RTVSSimulation of mid-tier or backend systemsTest automation of mid-tier or back-end systemsRQM Test PlanningConfiguration testingTest ConstructionTest ExecutionTeam CollaborationReporting & DashboardIBMAppScan
  • 18. 18Scan & CertifyInstrumentTestIntegrateObtain InsightManageDeployDesign & DevelopThe IBM mobile application development lifecycle
  • 19. AT&T Application Resource Optimizer(ARO)Balsu K. ThanduDirector, AT&T Developer Program
  • 20. How Fast is Your App?
  • 21. How Fast is Your App?
  • 22. How Fast is Your App?“Zombies appear tobe incapable ofrunning. The fastesthave been observedto move at a rate ofbarely one step per1.5 seconds.”– Max Brooks,Zombie Survival Guide
  • 23. How Important is Speed Anyway?Traditional Web (PC) Stats:•Yahoo! 400ms delay leads to 5-9% drop in traffic (1)•1 second of latency (2)– Customer satisfaction drops 16%– Conversion to sales drops 7%•10 Golden Principles of Successful Web Apps:– “Speed is always the most important feature” (3)“If speed is important on the web, it follows thatspeed is even more important for mobile”•71 % of users expect mobile sites to be as fast asdesktop (4)(1) slideshare.net/stoyan/dont-make-me-wait-or-building-highperformance-web-applications(2) http://econsultancy.com/us/blog/10195-the-need-for-online-speed-in-america(3) 10 Golden Principles Of Successful Web Apps http://www.businessinsider.com/10-golden-principles-successful-web-apps-2010-3(4) http://econsultancy.com/us/blog/9162-the-importance-of-speed-for-mobile-commerce
  • 24. What Do Mobile Users Expect?High Performance Apps•Speed– 64% of mobile users expect pages/apps to load in <4seconds (1)•Battery Life– Better Battery life (6.1 satisfied of 10) !!! (2)– 72% rate phone as very good or excellent• Only 21% rate battery life as very good or excellent (3)•Reasonable data usage– No one wants to hit their cap(1) http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf(2) http://www.businessnewsdaily.com/2200-smartphone-customer-satisfaction-battery-life.html(3) Personal communication from Swiftkey study
  • 25. What Do Mobile Users Get?Our Customers Expect More!!!!• Top Mobile frustrations “web page slow to load” (1)• Battery life top reported problem (2)(1) http://www.keynote.com/docs/reports/Keynote-2012-Mobile-User-Survey.pdf(2) http://www.businessnewsdaily.com/2200-smartphone-customer-satisfaction-battery-life.html
  • 26. As a Developer, What Can I Do?• Learn How Mobile is Different• Optimize:– Learn the best practices– Test your application– Fix issues before your customers discover them• Get ahead of the game – be ready for industry standardsand grading systemsIt can’t be MY developers, they’re the best• AT&T’s Application Resource Optimizer (ARO)– http://Developer.att.com/ARO– In 2012, <2% of tested apps required no optimization
  • 27. How is Mobile Different?InternetIP applicationsMobile applicationsWirelessStateMachineWirelessStateMachineMobile apps often do not consider the uniqueness of the wireless networkOptimized mobile applications improve user experience withimproved battery life and perceived faster speed
  • 28. It’s All ConnectedApplication traffic triggers RRC State transitions:•RRC State transitions control
  • 29. It’s All ConnectedApplication traffic triggers RRC State transitions:•RRC State transitions control– Speed/User Experience
  • 30. It’s All ConnectedApplication traffic triggers RRC State transitions:•RRC State transitions control– Speed/User Experience– Device battery life
  • 31. It’s All ConnectedApplication traffic triggers RRC State transitions:•RRC State transitions control– Speed/User Experience– Device battery life– Network efficiency
  • 32. AT&T Application Resource Optimizer:Develop Mobile Apps that run Faster, Smarter and Cleaner!AT&T Application Resource Optimizer (ARO)is a diagnostic tool created to help developersand testers understand how their app is behavingon any wireless networkEnabling a richer network-connected smartphoneexperience for your customers• Improved battery life• Faster response times• Efficient network handlingARO is Free, Cross Platform and Open Source-an essential tool for all mobile testers anddevelopersARO Analyzer
  • 33. How Does ARO Work?Transfer Trace Files Process TraceTest Your Application• PCAP only:• Network traffic collected• Native Collector adds:• Video• User Input• Peripheral Usage• CPU usage• Applies Radio and Power modelsto data• Graded against Best Practices• Learn how your applicationACTUALLY behaves!
  • 34. So, How Does ARO Make My AppFaster?ARO is the test kit to identify areas where performanceimprovements existSimple, common sense development best practices– Reducing connection times– Caching files– Eliminating ErrorsThe fixes identified by ARO will tune your applicationto higher performance and speed
  • 35. What Platforms Can I Test With?All of them!!!• All Platforms: Pcap/tcpdump network trace• Ex: Wi-Fi Hotspot (like Connectify) and Wireshark/NetMon to collect• iOS: Remote Virtual Interface uses Instruments to collect pcap over 3G/LTE• Android and Windows 8: Native Collector• Note: Android collector requires root
  • 36. Everyone Wins With ARO!“Using ARO we were able to cut thenetwork traffic of our iOSapplication down by ~30%!”
  • 37. Everyone Wins With ARO!“You will find that the new offeringsfrom Zynga will do much betterwith respect to battery and networkusage.”Words With Friends40% less energy200% less signalingFaster, more efficient user experience
  • 38. Everyone Wins With ARO!“Users concerned about dataconsumption and applicationperformance.”After only 1 week of development35% faster downloads70GB/month less data usage20% reduced battery usage
  • 39. Everyone Wins With ARO• Faster Response Times• Improved Battery Life• Efficient Data PlanUsage• Enriched UserExperienceEnd User• App-specific Analysis• Key Areas to Improve• Checks performance inways not usuallyavailable• Cross Platform andNetwork AgnosticDeveloper/Tester• Increased NetworkAvailability• Signaling LoadReduction• Efficient SpectrumUsage• Improved RadioAvailabilityWireless Carrier
  • 40. Rational and ARO a Winning Pairfor Mobile Application TestingWorking inside Rational, ARO can now be a part of yourday to day testing processes.
  • 41. © 2013 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.41AT&T Developer Programhttp://developer.att.comCommunicationsNewsletterEmailBlogSocial MediaContests & EventsDeveloper SummitMobile App HackathonsContestsFun EventsOnlineAPI PlatformAT&T Application ResourceOptimizer (ARO)Development ResourcesDevice SpecsSDKs & ToolsCommunity & SupportForumsWebcastsLive ChatTechnical Support
  • 42. thank you!More Information•http://developer.att.com/ARO•Balsu.Thandu@att.com
  • 43. 43
  • 44. 44Daily Apple TV giveawayComplete your session surveys online each day at a conference kiosk or onyour Innovate 2013 Portal!Each day that you complete all of that day’s session surveys, your name willbe entered to win the daily Apple TV!On Wednesday be sure to complete your full conference evaluation to receiveyour free conference t-shirt!
  • 45. 45© Copyright IBM Corporation 2013. All rights reserved. The informationcontained in these materials is provided for informational purposes only, and isprovided AS IS without warranty of any kind, express or implied. IBM shall not beresponsible for any damages arising out of the use of, or otherwise related to,these materials. Nothing contained in these materials is intended to, nor shallhave the effect of, creating any warranties or representations from IBM or itssuppliers or licensors, or altering the terms and conditions of the applicable licenseagreement governing the use of IBM software. References in these materials toIBM products, programs, or services do not imply that they will be available in allcountries in which IBM operates. Product release dates and/or capabilitiesreferenced in these materials may change at any time at IBM’s sole discretionbased on market opportunities or other factors, and are not intended to be acommitment to future product or feature availability in any way. IBM, the IBM logo,Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM productsand services are trademarks of the International Business Machines Corporation,in the United States, other countries or both. Other company, product, or servicenames may be trademarks or service marks of others.
  • 46. 46Acknowledgements and disclaimers© Copyright IBM Corporation 2013. All rights reserved.– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.IBM, the IBM logo, ibm.com, Rational, the Rational logo, Telelogic, the Telelogic logo, Green Hat, the Green Hat logo, and other IBM productsand services are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, orboth. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), thesesymbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks mayalso be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright andtrademark information” at www.ibm.com/legal/copytrade.shtmlIf you have mentioned trademarks that are not from IBM, please update and add the following lines:[Insert any special third-party trademark names/attributions here]Other company, product, or service names may be trademarks or service marks of others.Availability: References in this presentation to IBM products, programs, or services do not imply that they will be available in all countriesin which IBM operates.The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are providedfor informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to anyparticipant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is providedAS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwiserelated to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creatingany warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable licenseagreement governing the use of IBM software.All customer examples described are presented as illustrations of how those customers have used IBM products and the results they mayhave achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials isintended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenuegrowth or other results.

×