Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From Manual to Automated Tests - STAC 2015

1,191 views

Published on

This is a talk I gave at Software Testing Atlanta Conference (STAC 2015).

ABSTRACT::
Learn about developing and maintaining automation tests for web and mobile applications. In this two part presentation, Shauvik will cover "Move Fast" - how to efficiently generate automation test scripts for manual test cases, and "Fix Tests" - strategies for maintaining tests between regressions or platforms.

Published in: Technology

From Manual to Automated Tests - STAC 2015

  1. 1. MOVE FAST AND FIX TESTS Shauvik Roy Choudhary @shauvik http://shauvik.com GOING FROM MANUAL TO AUTOMATED:
  2. 2. MOVE FAST AND FIX TESTS Shauvik Roy Choudhary @shauvik http://shauvik.com GOING FROM MANUAL TO AUTOMATED:
  3. 3. MOVE FAST AND FIX TESTS Shauvik Roy Choudhary @shauvik http://shauvik.com GOING FROM MANUAL TO AUTOMATED: Checks
  4. 4. ABOUT ME • PhD Thesis:
 Cross-platformTesting & Maintenance of Web & Mobile Apps • Industry:
 Google,Yahoo!, FullStory, Fujitsu Labs, IBM Research, HSBC, Goldman Sachs • Entrepreneurship:
 GeorgiaTechTI:GER program,VentureLab, ATDC,TechSquare Labs Shauvik Roy Choudhary PhD, GeorgiaTech Founder, CheckDroid
  5. 5. OUTLINE • MOTIVATION: Why Automate ? • MOVE FAST: How to Automate Quickly? • FIX TESTS: How to Resolve Automation Issues? • SCENARIO: Android UITesting • DISCUSSION
  6. 6. WHY AUTOMATETESTS? Software ManualTesting
  7. 7. WHY AUTOMATETESTS? Software ManualTesting
  8. 8. WHY AUTOMATETESTS? Software ManualTesting
  9. 9. WHY AUTOMATETESTS? Software ManualTesting
  10. 10. WHY AUTOMATETESTS? Software ManualTesting
  11. 11. PLATFORMS Web Tester
  12. 12. PLATFORMS Web Tester
  13. 13. PLATFORMS Web Tester
  14. 14. PLATFORMS Web Tester
  15. 15. PLATFORMS Web Tester
  16. 16. PLATFORMS Web Tester
  17. 17. PLATFORMS Web Tester
  18. 18. PLATFORMS Web Tester
  19. 19. Mobile Tester Lollipop Android 5.0 9
  20. 20. HOWTO AUTOMATE?
  21. 21. HOWTO AUTOMATE? WHAT Parts of your App
  22. 22. HOWTO AUTOMATE? HOW Infrastructure Tool Support WHAT Parts of your App
  23. 23. WHAT:TOTEST • Functional Tests
 Positive and negative scenarios • Non-Functional Tests
 Performance, Security, Compatibility… • Previous Issues (identified Manually)
 Never to occur in Regressions
  24. 24. HOW: INFRASTRUCTURE • Unit Tests
 xUnit for different programming languages • UI Tests
 Selenium/WebDriver for Web Apps
 Appium for Mobile (iOS,Android)
 Espresso for Android
  25. 25. IDEALTESTING PYRAMID by Mike Cohn
  26. 26. ICE-CREAM CONE
  27. 27. HOUR GLASS source: just-about.net
  28. 28. TESTING DIAMOND Source: toddlittleweb.com
  29. 29. CHRISTMASTREE source: just-about.net
  30. 30. UITESTS CLOSESTTO END-USERTESTING
  31. 31. MOVE FAST How to automate quickly?
  32. 32. RELIABLE INFRASTRUCTURE
  33. 33. VISUALTEST RECORDING
  34. 34. FIX TESTS How to fix broken tests?
  35. 35. UITESTING ISSUES
  36. 36. UITESTING ISSUES • FRAGILITY
  37. 37. UITESTING ISSUES • FRAGILITY • SPEED
  38. 38. UITESTING ISSUES • FRAGILITY • SPEED • FLAKINESS
  39. 39. 1. FRAGILETESTS
  40. 40. REDUCING FRAGILITY • Use durable selectors • IDs over XPath • Relative XPaths over Absolute XPaths • Other durable labels (class, content-desc..)
  41. 41. 2. SLOWTESTS
  42. 42. MAKINGTESTS FAST • Don’t use Sleeps • If absolutely necessary, sleep while polling • Replace slow resources with stubs
  43. 43. doSomething() Thread.sleep(1000) checkSomething()
  44. 44. doSomething() do{ Thread.sleep(1000) }while(!loaded) checkSomething()
  45. 45. FLAKINESS UIThread TestThread Work in Queue
  46. 46. FLAKINESS UIThread TestThread Work in Queue click
  47. 47. FLAKINESS UIThread TestThread Work in Queue click Motion down Motion up
  48. 48. FLAKINESS UIThread TestThread Work in Queue click Motion down Motion up assert
  49. 49. FLAKINESS UIThread TestThread Work in Queue click Motion down Motion up assert
  50. 50. FLAKINESS UIThread TestThread Work in Queue click Motion down Motion up assertsleep
  51. 51. FLAKINESS UIThread TestThread Work in Queue click Motion down Motion up assertsleep
  52. 52. TESTS
  53. 53. 3. FLAKYTESTS Image credits: Google, GTAC talk
  54. 54. REDUCING FLAKINESS • Have a hermetic test environment Source: googletesting.blogspot.com
  55. 55. REDUCING FLAKINESS • Have a hermetic test environment Source: googletesting.blogspot.com
  56. 56. FIX TESTS
  57. 57. 1. FRAGILETESTS FIX TESTS
  58. 58. 1. FRAGILETESTS 2. SLOWTESTS FIX TESTS
  59. 59. 1. FRAGILETESTS 2. SLOWTESTS 3. FLAKYTESTS Image credits: Google, GTAC talk FIX TESTS
  60. 60. ANDROID UI TESTING
  61. 61. MOBILETESTING PYRAMID Source: Daniel Knott,AdventuresInQA Author: Hand-On Mobile AppTesting
  62. 62. ANDROIDTESTINGTOOLS Image credits: BitBar
  63. 63. ESPRESSO: HOW IT WORKS UIThread TestThread Work in Queue Test Case Motion down Motion up Espresso action assert Blocked
  64. 64. ESPRESSO METHODS onView(Matcher) .perform(ViewAction) .check(ViewAssertion)
  65. 65. EXAMPLES onView(withId(R.id.greeting))
 .perform(click()); onView(withText("Hello Steve!"))
 .check(matches(isDisplayed()));
  66. 66. EXAMPLES onView(withId(R.id.greeting))
 .perform(click()); onView(withText("Hello Steve!"))
 .check(matches(isDisplayed())); Find
  67. 67. EXAMPLES onView(withId(R.id.greeting))
 .perform(click()); onView(withText("Hello Steve!"))
 .check(matches(isDisplayed())); Find Do Stuff
  68. 68. EXAMPLES onView(withId(R.id.greeting))
 .perform(click()); onView(withText("Hello Steve!"))
 .check(matches(isDisplayed())); Find Do Stuff Check
  69. 69. onView(Matcher<View>) ●  withId ●  withText ●  withContentDescription ●  isDisplayed ●  hasFocus ●  hasSibling ●  ... ●  mySpecialMatcher
  70. 70. perform(ViewAction) ●  click ●  longClick ●  doubleClick ●  typeText ●  scrollTo ●  ... ●  myCustomAction
  71. 71. check(ViewAssertion) ●  matches(...) ●  doesNotExist ●  myCustomAssertion
  72. 72. HIERARCHY withParent(Matcher) withChild(Matcher) hasDescendant(Matcher) isDescendantOfA(Matcher) hasSibling(Matcher) isRoot() UI PROPERTIES isDisplayed() isCompletelyDisplayed() isEnabled() hasFocus() isClickable() isChecked() isNotChecked() withEffectiveVisibility(…) isSelected() ROOT MATCHERS isFocusable() isTouchable() isDialog() withDecorView(…) isPlatformPopup() COMMON HAMCREST MATCHERS allOf(Matchers) anyOf(Matchers) is(...) not(...) endsWith(String) startsWith(String) SEE ALSO Preference matchers Cursor matchers USER PROPERTIES withId(…) withText(…) withTagKey(…) withTagValue(…) hasContentDescription(…) withContentDescription(…) withHint(…) withSpinnerText(…) hasLinks() hasEllipsizedText() hasMultilineText() INPUT supportsInputMethods(…) hasImeAction(…) CLASS isAssignableFrom(…) withClassName(…) Matchers CLICK/PRESS click() doubleClick() longClick() pressBack() pressImeActionButton() pressKey([int/EspressoKey]) pressMenuKey() closeSoftKeyboard() openLink(…) GESTURES scrollTo() swipeLeft() swipeRight() swipeUp() swipeDown() TEXT clearText() typeText(String) typeTextIntoFocusedView(String) replaceText(String) POSITION ASSERTIONS isLeftOf(Matcher) isRightOf(Matcher) isLeftAlignedWith(Matcher) isRightAlignedWith(Matcher) isAbove(Matcher) isBelow(Matcher) isBottomAlignedWith(Matcher) isTopAlignedWith(Matcher) LAYOUT ASSERTIONS noEllipsizedText(Matcher) noMultilineButtons() noOverlaps([Matcher]) View Actions matches(Matcher) doesNotExist() selectedDescendantsMatch(…) View Assertions onView(Matcher) .perform(ViewAction) .check(ViewAssertion) CHEAT SHEET 2.0
  73. 73. BARISTA EspressoTest Generation http://checkdroid.com/barista
  74. 74. DEMO
  75. 75. Tests
  76. 76. TEST CLOUD
  77. 77. WHY AUTOMATETESTS? Software ManualTesting
  78. 78. WHY AUTOMATETESTS? Software ManualTesting MOVE FAST How to automate quickly?
  79. 79. WHY AUTOMATETESTS? Software ManualTesting MOVE FAST How to automate quickly? 1. FRAGILETESTS 2. SLOWTESTS 3. FLAKYTESTS Image credits: Google, GTAC talk FIX TESTS
  80. 80. WHY AUTOMATETESTS? Software ManualTesting MOVE FAST How to automate quickly? 1. FRAGILETESTS 2. SLOWTESTS 3. FLAKYTESTS Image credits: Google, GTAC talk FIX TESTS ANDROID UI TESTING
  81. 81. DISCUSSION Are you currently automating tests? What are some challenges that you see?
  82. 82. THANKS! Shauvik Roy Choudhary @shauvik http://shauvik.com

×