Downloaded 59 times
More Related Content
![[webinar] Best of Breed: Successful Test Automation Practices from Innovative...](https://cdn.slidesharecdn.com/ss_thumbnails/bestpracticesinqualitymanagementfrominnovativebrands-slides-200506184200-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to When & How to Successfully use Test Automation for Mobile Applications
When & How to Successfully use Test Automation for Mobile Applications
- 1. Testing for Android& iOS When, Where, and How to Successfully Use Test Automation Trent Peterson | @tdpeterson | trent@appthwack.com
- 2.
- 3. AGENDA What is testing? What’sthe problem? Why should I care? When: Choosing our battles Where: Best Practices How: Tools & Frameworks Q&A
- 4.
- 5. Types of tests • • • • • • • • Functional Non-functional Integration Unit Regression Smoke Usability Performance •Stress • Acceptance • Load • Sanity • End-to-End • System • Security And on and on…
- 6. Testing distilled forthis discussion Functional Testing Non-functional Testing
- 7.
- 8. Building things ishard.
- 9. WHY SHOULD ICARE?
- 10. So what’s thebig deal? 1,000,000+ apps in 1,000,000+ apps in 2 crashes and 84% will uninstall - Compuware as reported in TechCrunch
- 11. We’ve been herebefore.
- 12.
- 13. First, what’s ourgoal? Save time! BE REALISTIC Save money! • Increase regularity of testing Shrink QA! • Free up resources for other testing efforts • Enable previously impossible tests AUTOMATE EVERYTHING !
- 14. Deciding what toautomate How often will I test this? Is this something a human is good at? How much effort will automation require?
- 15. Some examples AUTOMATE MANUAL • Granularfunctionality (unit tests) • Repetitive tasks (update paths, navigation, etc.) • Performance • UX • Responsiveness and “feel” • New functionality
- 16. Our options (Machine) JVM Emulators RealDevices • FAST • Unit tests only* • Mocks • Cheap • Catch layout issues • Simulate calls/SMS • SLOW • Illusion of 1:1 with real devices • Exactly what end customers use • Performance data • Fast • Expensive: Purchase, maintai n, etc.
- 17. Our options (Human) QA Crowd EndUsers • Pros • Real feedback • Know the product • Expensive • Real feedback • Might know product • Expensive (usually) • Slow • Don’t do this.
- 18.
- 19. No barriers betweenDev & QA
- 20. Automation is asoftware project
- 21. Automation is atool, not a solution
- 22.
- 23. It’s all aboutdata
- 24.
- 25. Abstraction is important …butdon’t obsess
- 26.
- 27. Tool: Recorders GOOD BAD • Fast •Non-developers can create tests • Fragile • Difficult to maintain • Non-developers can create tests BEST PRACTICE • Treat as a fancy spy
- 28. Tool: UI ExerciserMonkey $ adb shell monkey –p my.sweet.app –v 1000
- 29. Framework: Android Testing SUMMARY •JUnit/Java • Instrumentation provides hooks into Android SDK to control Android-specific functionality • Very basic functionality supported • The foundation for most Android testing. Learn it, use it, love it.
- 30. Framework: Robolectric SUMMARY • JUnit •Runs in JVM • Mocks Android functionality with “shadow” objects • Super fast • Not everything is mocked (also, mocks) • Great for staying sane while developing
- 31. Framework: UI Automator SUMMARY •JUnit/Java • Extends testing framework with UI selectors and manipulators • Handle/control system dialogs, etc • 4.1+ only • Excellent choice if you have the luxury of supporting 4.1+.
- 32. Framework: Espresso SUMMARY • JUnit/Java •Extends testing framework with UI selectors and manipulators • Removes boilerplate code (sleeps, etc) • Very new and unstable • If you’re feeling adventurous give it a try
- 33. Framework: Robotium SUMMARY • JUnitextension • Runs on emulators and real devices • Oriented towards black-box testing and ensuring real-world outcomes • Limited to the app under test • Great choice for developers going beyond basic unit testing and testing UI.
- 34. Framework: Cucumber-based SUMMARY • Human-readabletests • Runs on emulators and real devices • Cross-platform • Complex scenarios require development • Behavior-driven design. Be careful! • Great choice for simple flows and teams with limited developer resources.
- 35. Framework: Appium SUMMARY • Supportsmany language adapters • Runs on emulators and real devices • Cross-platform • New, and not all WebDriver concepts map intuitively to native apps • Great choice for teams happy with Selenium and expanding to native.
- 36. Framework: UI Automation SUMMARY • • • • JavaScript Installswith Xcode Functions with other instruments Has a recorder (Careful) • Dev/maintenance of JS • Good choice to get basic tests up and running quickly.
- 37. Framework: XCTest SUMMARY • ObjectiveC • Installs with Xcode • Poor documentation • Obvious choice for unit testing.
- 38. Framework: KIF SUMMARY • ObjectiveC • Enables functional test automation capabilities by providing UI helpers • Relies on undocumented APIs • Great choice for devs automating functional tests.
- 39.
- 40. Tool: Bots Continuous Integration Executetests, etc.
- 41. Frameworks: Commercial GOOD BAD • Support •Added features and integrations • Longevity? • Proprietary • Often heavy and difficult to maintain • $$$ ADVICE • Evaluate very carefully and acknowledge lock-in.
- 42. When choosing, ask… Whatare we testing? What are our automation goals? Who will write the automated tests?
- 43.
- 44. AppThwack • • • • 100s of non-rooteddevices Results in minutes Built-in compatibility tests Supports all popular automation frameworks • Simple API and integrations • Focused, comprehensive reports
- 45. Q&A Trent Peterson |@tdpeterson | trent@appthwack.com
Editor's Notes
- #6 Picture courtesy of http://www.flickr.com/photos/schrierc/ (Creative Commons Attribution 2.0 Generic)
- #7 Picture courtesy of http://www.flickr.com/photos/conbon/ (Creative Commons Attribution 2.0 Generic)
- #9 Programming image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
- #11 TechCrunch article: http://techcrunch.com/2013/03/12/users-have-low-tolerance-for-buggy-apps-only-16-will-try-a-failing-app-more-than-twice/Trash can by Yellow Icon Design (http://www.yellowicon.com/) under GPU General Public License
- #12 Picture courtesy of http://www.flickr.com/photos/mulad/ (Creative Commons Attribution 2.0 Generic)
- #14 Suit image by http://www.flickr.com/photos/charlesfettinger/ (Creative Commons Attribution 2.0 Generic)
- #16 Picture courtesy of http://www.flickr.com/photos/wwarby/ (Creative Commons Attribution 2.0 Generic)
- #20 Wall image by http://www.flickr.com/photos/easylocum/ (Creative Commons Attribution 2.0 Generic)
- #21 Wall image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
- #22 Wall image by http://www.flickr.com/photos/zomgitsbrian/ (Creative Commons Attribution 2.0 Generic)
- #23 Wall image by http://www.flickr.com/photos/andreweason/ (Creative Commons Attribution 2.0 Generic)
- #24 Data image by http://www.flickr.com/photos/tensafefrogs/ (Creative Commons Attribution 2.0 Generic)
- #25 Wall image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
- #26 Wall image by http://www.flickr.com/photos/carbonnyc/ (Creative Commons Attribution 2.0 Generic)
- #28 Security image by http://www.flickr.com/photos/zigazou76/ (Creative Commons Attribution 2.0 Generic)
- #29 http://developer.android.com/tools/help/monkey.html
- #30 http://developer.android.com/tools/testing/testing_android.html
- #31 http://robolectric.org
- #32 http://developer.android.com/tools/testing/testing_ui.html
- #33 https://code.google.com/p/android-test-kit/
- #34 https://code.google.com/p/robotium/
- #36 http://appium.io/
- #37 https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
- #38 https://developer.apple.com/library/ios/documentation/ToolsLanguages/Conceptual/Xcode_Overview/UnitTestYourApp/UnitTestYourApp.html
- #39 https://github.com/kif-framework/KIF
- #40 http://square.github.io/spoon/
- #41 https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/000-About_Continuous_Integration/about_continuous_integration.html#//apple_ref/doc/uid/TP40013292
- #42 Chains image by http://www.flickr.com/photos/colinsd40/ (Creative Commons Attribution 2.0 Generic)
- #43 Surgical tools image by http://www.flickr.com/photos/wien/ (Creative Commons Attribution 2.0 Generic)
- #45 https://appthwack.com