The Chronicle of iPhone Automation -- From Frank to UISpec to UIAutomation to KIF


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Chronicle of iPhone Automation -- From Frank to UISpec to UIAutomation to KIF

  1. 1. The Chronicle of iPhone Automation<br />From Frank to UISpec to UIAutomation to KIF<br />HinlingYeung<br />QA Automation Engineer from Trulia, Inc<br />
  2. 2. Background<br />iPhone app<br />iPad app<br />Android app<br />Mobile site<br />more phone/device apps to come in a short future<br />Mobile traffic keeps increasing<br />
  3. 3. To help maintain our sanity…<br />Let’s automate some of our devices!<br />
  4. 4. Now let’s get back to the topic.<br />After watching Pete Hodgson’s presentation about Frank, I was very hopeful about iPhone automation…<br />
  5. 5. Then I follow the online documentations…<br />And successfully setup up Frank!<br />
  6. 6. However, when I actually start automating my first test case.<br />I hit my first wall:<br />
  7. 7. mbp-1350:cucumber hyeung$ cucumber features/login.feature<br />Feature: <br /> As an iOS developer<br /> I want to click the Log in button<br /> So I can see the warning message of missing email address and password<br /> Scenario: # features/login.feature:6<br />PING FAILED!!!!<br />FRANK! Given I launch the app # features/step_definitions/launch_steps.rb:1<br /> When I wait 30 seconds # features/step_definitions/frank_steps.rb:1<br /> When I touch the "No Thanks" Button # features/step_definitions/frank_steps.rb:29<br /> end of file reached (EOFError)<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:93:in `timeout'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2020:in `read_status_line'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:2009:in `read_new'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:1050:in `request'<br /> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:543:in `start'<br /> ./features/step_definitions/frank_steps.rb:30:in `/^I touch the "([^"]*)" Button$/'<br /> features/login.feature:9:in `When I touch the "No Thanks" Button'<br />Failing Scenarios:<br />cucumber features/login.feature:6 # Scenario: <br />1 scenario (1 failed)<br />3 steps (1 failed, 2 passed)<br />0m37.113s<br />
  8. 8. So…<br />Keep on debugging the ruby scripts, Frank, and possibly UISpec<br />OR<br />Let’s get to the bottom: UISpec<br />
  9. 9. Then I start setting up UISpec…<br />adding libraries…<br />Adding dependencies…<br />Adding header files…<br />Adding more header files…<br />Oops…linking error. Then adding .m files…<br />Add some test code:<br />-(void)itShouldShowErrorMessageAfterClickingLogInButton {<br /> [[app.find.labeltext:@"For Sale"] flash];<br /> [[app.find.labeltext:@"For Rent"] flash];<br /> [[app.find.labeltext:@"My Trulia"] flash];<br /> …<br />}<br />
  10. 10. Looks Really Promising…<br />Let’s try “touch” it instead of just “flash” it.<br />Then I hit another wall: <br />“touch” results in no reaction or EXC_BAD_ACESS error<br />
  11. 11. Sigh…<br />
  12. 12. I really need something that just work!<br />Then I talked to a friend of mine who works in the MobileMe automation team in Apple:<br />Me: What do you use to automate your iphone app?<br />Friend: Javascript<br />Me: Javascript? You mean Javascript JAVASCRIPT? The one we use on the browser?<br />Friend: yes. Check the apple dev center.<br />Results: UIAutomation from Apple<br />
  13. 13. UIAutomation with Instruments<br /><br />Add some accessibilityLabels<br />Setup the Instruments template<br />Write some Javascript<br />A little counter intuitive but click “record” and it starts running.<br />And it works!<br />
  14. 14.
  15. 15. Let’s integrate the tests with Jenkins<br />I hit another wall:<br />There is no good way to kick off the tests from command line and integrate my results to Jenkins!<br />
  16. 16. Then I found KIF…<br />KIF – Keep It Functional<br />By Square (Eric Firestone) --<br />Blog:<br />Couple linking errors later…<br />
  17. 17. YEAH!<br />It actually works from end to end with simple enough setup.<br />With instructions to start the test in command line<br />And quits after the test finish running.<br />And output results to the console.<br />Results are not in a CI-ready format but THAT’S OK IF THAT’S THE ONLY PROBLEM!<br />
  18. 18. Final Solution<br />KIF<br />Add accessibilityLabel to textField, Button, and View<br />isAccessibilityElement = YES<br />Symbiote from Frank to inspect the labels on app<br />WaxSim to launch app from command line<br />Modify the KIF log to output test results that can be turned into standard junit result format<br />