Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (16)
Similar to I, For One, Welcome Our New Robot Overlords
Similar to I, For One, Welcome Our New Robot Overlords (20)
Recently uploaded
Recently uploaded (20)
I, For One, Welcome Our New Robot Overlords
- 1. I, For One, Welcome Our New Robot Overlords Accepting Mobile Automated Testing into your Heart
- 2. About Me: • Engineer at MEDL Mobile • Been doing Android & iOS development for 4 years
- 3. How many of you have a dedicated QA group?
- 4. How many of you are doing automated testing?
- 5. Old projects: small army of manual testers
- 6. New projects: only one or two testers
- 7. Manual Testing Problems Testers must be in the office Troubles with reproducing Flaky error cases Takes lots of time
- 8. I, for one, welcome our new robotic overlords
- 9. Pros of manual testing Testers invent new ways of going through the app If something is out of place, they can exploit it right away Can tell you how aggravating some of your flows are
- 10. “Don’t abandon manual testing completely” - Me (and your QA department)
- 13. Unit Tests Small, independent tests Tests only a small chunk of a class Fast to run Written mostly by developers
- 14. Functional Tests Test entire feature Cut across large sections of the app Can be written by developers, QA, or product owners
- 15. Needs from a testing solution Has to support iOS and Android One set of test cases shared between both apps
- 16. Needs from a testing solution Needs to be quick to pick up and set up
- 17. Needs from a testing solution Needs to be understandable/usable by non-technical people
- 18. UIAutomation/UIAutom ator iOS/Android specific frameworks One is Javascript, the other Java. Tests are not portable between platforms
- 19. Cucumber Tests are written in plain language “Given”, “When”, “Then” Desired by teams interested in Agile
- 20. Calabash Cucumber support, Ruby code Many built-in step definitions for mobile tests iOS and Android can use the same steps & step definitions FREE!
- 21. How to write Cucumber/Calabas h tests
- 22. Feature: Running a test As a Developer I want a sample feature So I can begin testing quickly
- 23. Feature: Running a test As a Developer I want a sample feature So I can begin testing quickly Scenario: Logging in Given I am about to login When I login as “User” Then I should see “My Profile
- 24. Feature: Running a test As a Developer I want a sample feature So I can begin testing quickly Scenario: Logging in Given I am about to login When I login as “User” Then I should see “My Profile Scenario: Invalid Login Given I am about to login When I login as “BadUser” Then I should see “Invalid Credentials”
- 25. Feature: Running a test As a Developer I want a sample feature So I can begin testing quickly @valid Scenario: Logging in Given I am about to login When I login as “User” Then I should see “My Profile @invalid Scenario: Invalid Login Given I am about to login When I login as “BadUser” Then I should see “Invalid Credentials”
- 26. Step Definitions Given(/^I am about to login$/) do … end
- 27. Step Definitions Given(/^I am about to login$/) do … end When(/^I login as “(.*?)”$/) do |user| … end
- 29. Step Definitions When(/^I login as “(.*?)”$/) do |user| … end
- 30. Step Definitions When(/^I login as “(.*?)”$/) do |user| … end USERS = { BadUser: { username: ‘baduser’, password: ‘BadPassword’ } }
- 31. Step Definitions When(/^I login as “(.*?)”$/) do |user| user = USERS[:user] touch(“Username”) keyboard_enter_text(user[:username]) touch(“Password”) keyboard_enter_text(user[:password]) touch(“Login”) end
- 33. Bad Tests are worse than no tests
- 34. Good tests should be slightly verbose
- 36. Good Tests should be Concise
- 37. Given I am on the Welcome Screen Then I press the Welcome button Then I tap the Username field And I type "username" Then I tap the Password field And I type "badPassword" And I tap the Login button Then I should see “Invalid Login"
- 38. Given I am on the Welcome Screen Then I press the Welcome button Then I tap the Username field And I type "username" Then I tap the Password field And I type "badPassword" And I tap the Login button Then I should see “Invalid Login" Given I am about to login When I log in as “BadUser” Then I should see “Invalid Login"
- 39. Good tests should be flexible
- 40. NOT
- 41. Good Tests must be independent
- 43. Good Tests should be run often
- 45. Good Tests should not know too much about what they’re testing
- 46. - (void)testSquare { assertEquals([sut squareOf:2], 2*2); }
- 47. - (void)testSquare { assertEquals([sut squareOf:2], 2*2); } - (void)testSquare { assertEquals([sut squareOf:2], 4); }
- 48. Manual testing is on a decline
- 49. Explored solutions for automated testing
- 50. Saw what goes into a Cucumber test
- 51. What makes a good test
Editor's Notes
- (Wait on this slide for them to read the comic)
- I’ve noticed a trend in the industry, especially in mobile We used to have QA departments that were quite large
- Now they seem to be getting smaller and smaller. But our apps are getting bigger and bigger. We are consistently being asked to do more with fewer people.
- Testers must be in the office - Can’t test while in meetings - LOTS OF MEETINGS - Can’t really test overnight Troubles with repro - Relies on how well everyone understands the app - Relies on how good a storyteller people are - Relies on common terminology Flaky error cases - Backend servers unreliable; testers pick the worst time to test - Heisenbugs Takes lots of time - Sometimes have to get deep in the app - Only go as fast as they can tap
- Testers invent new ways - They go from here, to over there, to here again Exploit right away Can tell how aggravating - They do it over and over and over again - Ask for shortcuts
- Generally written in the language that the app is written in Use platform facilities for those (XCTest and JUnit)
- We will be focusing on Functional Tests for this presentation
- Someone saw Cucumber code, and thought it was just something that the boss wrote down and told the engineers what to do.
- Calabash Cloud Built on top of the platform specific frameworks mentioned earlier
- Features are groups of tests Mostly there for commenting purposes
- Scenarios are the individual tests
- Tests can be assigned tags. Calabash can be told to run subsets of tests based on their tags Applying a tag to a feature applies that to all Scenarios under it.
- Step definitions are selected by using a regex
- Possible to add tokens into the regex, and use them as parameters to the test
- Since it is just Ruby code, we can include helper methods and data stores
- Bad tests give you the illusion of confidence Bad tests require the same amount of time and effort to maintain, but don’t give you any meaningful info
- Want to know exactly what failed
- Don’t lose the forest for the trees
- Take advantage of tools to encapsulate lower level steps
- Tests should not break simply because a label was changed Tests should not break simply because the server took a second longer to return
- Tests should not be used as setup for other tests One test failing should not cause others to fail
- Tests run often provide more feedback Tests not run are not providing any information
- Test should not break merely because the internal implementation changed Don’t test private state
- Do not rewrite the algorithm in your test
- Verbose yet concise, flexible, independent, run often, and should know little about what they are testing