• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
When & How to Successfully use Test Automation for Mobile Applications
 

When & How to Successfully use Test Automation for Mobile Applications

on

  • 525 views

A free webinar produced by the Technology Association of Oregon's Quality Assurance Forum Group. Presented by Trent Peterson, CEO and Co-Founder of Portland-based AppThwack, providing fast automated ...

A free webinar produced by the Technology Association of Oregon's Quality Assurance Forum Group. Presented by Trent Peterson, CEO and Co-Founder of Portland-based AppThwack, providing fast automated mobile device testing services for Android, iPhone, iPad, and mobile web apps. The recording to the webinar can be accessed here: http://youtu.be/r9Vm6ssSEU8

Statistics

Views

Total Views
525
Views on SlideShare
525
Embed Views
0

Actions

Likes
2
Downloads
16
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Picture courtesy of http://www.flickr.com/photos/schrierc/ (Creative Commons Attribution 2.0 Generic)
  • Picture courtesy of http://www.flickr.com/photos/conbon/ (Creative Commons Attribution 2.0 Generic)
  • Programming image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
  • TechCrunch article: http://techcrunch.com/2013/03/12/users-have-low-tolerance-for-buggy-apps-only-16-will-try-a-failing-app-more-than-twice/Trash can by Yellow Icon Design (http://www.yellowicon.com/) under GPU General Public License
  • Picture courtesy of http://www.flickr.com/photos/mulad/ (Creative Commons Attribution 2.0 Generic)
  • Suit image by http://www.flickr.com/photos/charlesfettinger/ (Creative Commons Attribution 2.0 Generic)
  • Picture courtesy of http://www.flickr.com/photos/wwarby/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/easylocum/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/zomgitsbrian/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/andreweason/ (Creative Commons Attribution 2.0 Generic)
  • Data image by http://www.flickr.com/photos/tensafefrogs/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/riebart/ (Creative Commons Attribution 2.0 Generic)
  • Wall image by http://www.flickr.com/photos/carbonnyc/ (Creative Commons Attribution 2.0 Generic)
  • Security image by http://www.flickr.com/photos/zigazou76/ (Creative Commons Attribution 2.0 Generic)
  • http://developer.android.com/tools/help/monkey.html
  • http://developer.android.com/tools/testing/testing_android.html
  • http://robolectric.org
  • http://developer.android.com/tools/testing/testing_ui.html
  • https://code.google.com/p/android-test-kit/
  • https://code.google.com/p/robotium/
  • http://appium.io/
  • https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
  • https://developer.apple.com/library/ios/documentation/ToolsLanguages/Conceptual/Xcode_Overview/UnitTestYourApp/UnitTestYourApp.html
  • https://github.com/kif-framework/KIF
  • http://square.github.io/spoon/
  • https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/000-About_Continuous_Integration/about_continuous_integration.html#//apple_ref/doc/uid/TP40013292
  • Chains image by http://www.flickr.com/photos/colinsd40/ (Creative Commons Attribution 2.0 Generic)
  • Surgical tools image by http://www.flickr.com/photos/wien/ (Creative Commons Attribution 2.0 Generic)
  • https://appthwack.com

When & How to Successfully use Test Automation for Mobile Applications When & How to Successfully use Test Automation for Mobile Applications Presentation Transcript

  • Testing for Android & iOS When, Where, and How to Successfully Use Test Automation Trent Peterson | @tdpeterson | trent@appthwack.com
  • Who?
  • AGENDA What is testing? What’s the problem? Why should I care? When: Choosing our battles Where: Best Practices How: Tools & Frameworks Q&A
  • INTRO TO TESTING
  • Types of tests • • • • • • • • Functional Non-functional Integration Unit Regression Smoke Usability Performance • Stress • Acceptance • Load • Sanity • End-to-End • System • Security And on and on…
  • Testing distilled for this discussion Functional Testing Non-functional Testing
  • THE PROBLEM
  • Building things is hard.
  • WHY SHOULD I CARE?
  • So what’s the big deal? 1,000,000+ apps in 1,000,000+ apps in 2 crashes and 84% will uninstall - Compuware as reported in TechCrunch
  • We’ve been here before.
  • CHOOSING OUR BATTLES
  • First, what’s our goal? Save time! BE REALISTIC Save money! • Increase regularity of testing Shrink QA! • Free up resources for other testing efforts • Enable previously impossible tests AUTOMATE EVERYTHING !
  • Deciding what to automate How often will I test this? Is this something a human is good at? How much effort will automation require?
  • Some examples AUTOMATE MANUAL • Granular functionality (unit tests) • Repetitive tasks (update paths, navigation, etc.) • Performance • UX • Responsiveness and “feel” • New functionality
  • Our options (Machine) JVM Emulators Real Devices • FAST • Unit tests only* • Mocks • Cheap • Catch layout issues • Simulate calls/SMS • SLOW • Illusion of 1:1 with real devices • Exactly what end customers use • Performance data • Fast • Expensive: Purchase, maintai n, etc.
  • Our options (Human) QA Crowd End Users • Pros • Real feedback • Know the product • Expensive • Real feedback • Might know product • Expensive (usually) • Slow • Don’t do this.
  • BEST PRACTICES
  • No barriers between Dev & QA
  • Automation is a software project
  • Automation is a tool, not a solution
  • Automate with precision
  • It’s all about data
  • Know your matrix
  • Abstraction is important …but don’t obsess
  • TOOLS and FRAMEWORKS
  • Tool: Recorders GOOD BAD • Fast • Non-developers can create tests • Fragile • Difficult to maintain • Non-developers can create tests BEST PRACTICE • Treat as a fancy spy
  • Tool: UI Exerciser Monkey $ adb shell monkey –p my.sweet.app –v 1000
  • Framework: Android Testing SUMMARY • JUnit/Java • Instrumentation provides hooks into Android SDK to control Android-specific functionality • Very basic functionality supported • The foundation for most Android testing. Learn it, use it, love it.
  • Framework: Robolectric SUMMARY • JUnit • Runs in JVM • Mocks Android functionality with “shadow” objects • Super fast • Not everything is mocked (also, mocks) • Great for staying sane while developing
  • Framework: UI Automator SUMMARY • JUnit/Java • Extends testing framework with UI selectors and manipulators • Handle/control system dialogs, etc • 4.1+ only • Excellent choice if you have the luxury of supporting 4.1+.
  • Framework: Espresso SUMMARY • JUnit/Java • Extends testing framework with UI selectors and manipulators • Removes boilerplate code (sleeps, etc) • Very new and unstable • If you’re feeling adventurous give it a try
  • Framework: Robotium SUMMARY • JUnit extension • Runs on emulators and real devices • Oriented towards black-box testing and ensuring real-world outcomes • Limited to the app under test • Great choice for developers going beyond basic unit testing and testing UI.
  • Framework: Cucumber-based SUMMARY • Human-readable tests • Runs on emulators and real devices • Cross-platform • Complex scenarios require development • Behavior-driven design. Be careful! • Great choice for simple flows and teams with limited developer resources.
  • Framework: Appium SUMMARY • Supports many language adapters • Runs on emulators and real devices • Cross-platform • New, and not all WebDriver concepts map intuitively to native apps • Great choice for teams happy with Selenium and expanding to native.
  • Framework: UI Automation SUMMARY • • • • JavaScript Installs with Xcode Functions with other instruments Has a recorder (Careful) • Dev/maintenance of JS • Good choice to get basic tests up and running quickly.
  • Framework: XCTest SUMMARY • Objective C • Installs with Xcode • Poor documentation • Obvious choice for unit testing.
  • Framework: KIF SUMMARY • Objective C • Enables functional test automation capabilities by providing UI helpers • Relies on undocumented APIs • Great choice for devs automating functional tests.
  • Tool: Spoon Spoon Instrumentation Tests
  • Tool: Bots Continuous Integration Execute tests, etc.
  • Frameworks: Commercial GOOD BAD • Support • Added features and integrations • Longevity? • Proprietary • Often heavy and difficult to maintain • $$$ ADVICE • Evaluate very carefully and acknowledge lock-in.
  • When choosing, ask… What are we testing? What are our automation goals? Who will write the automated tests?
  • SHAMELESS PLUG
  • AppThwack • • • • 100s of non-rooted devices Results in minutes Built-in compatibility tests Supports all popular automation frameworks • Simple API and integrations • Focused, comprehensive reports
  • Q&A Trent Peterson | @tdpeterson | trent@appthwack.com