Automating UI testing <ul><li>Adam Siton </li></ul>
About me
Road map <ul><li>What are automated UI tests and why do we need them </li></ul><ul><li>Apple’s UIAutomation framework </li...
Why do we need automated tests <ul><li>Hopefully - this is a rhetorical question </li></ul>
We want a 5 star app <ul><li>A great design and marketing will get you a lot of users </li></ul><ul><li>But if your app is...
There are always more scenarios <ul><li>The number of test scenarios is always growing </li></ul><ul><ul><li>Each new feat...
Refactoring is alway dangerous <ul><li>Most of the times fixing bugs introduce new bugs </li></ul><ul><li>The sooner we fi...
Flexibility <ul><li>Automated tests can be run on multiple devices </li></ul><ul><li>And multiple SDKs </li></ul>
The solution - Automation <ul><li>We need a way to test all those scenarios automatically </li></ul><ul><li>Unit tests are...
UIAutomation
UIAutomation framework <ul><li>A JavaScript library introduced in iOS SDK 4.0 </li></ul><ul><li>Integrated in Instruments ...
UIAutomation elements <ul><li>Elements control is hierarchy based </li></ul><ul><li>UIAElement  - the base class </li></ul...
UIAutomation elements <ul><li>Target  - device operations (orientation, volume, shake, etc.) </li></ul>UIATarget.localTarg...
UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application  - Access and control of application-level UI eleme...
UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window  - contr...
UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></u...
UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></u...
UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></u...
Logging <ul><li>Since UIAutomation is based on view hierarchy, we need a simple way to examine the hierarchy </li></ul><ul...
Logging <ul><li>UIATarget.localTarget() </li></ul>
Logging <ul><li>UIATarget.localTarget(). frontMostApp() </li></ul>
Logging <ul><li>UIATarget.localTarget().frontMostApp(). mainWindow() </li></ul>
Logging <ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). tableViews()[0] </li></ul>
Logging <ul><li>UIATarget.localTarget().frontMostApp().mainWindow().tableViews()[0]. cells() </li></ul>
UIAutomation commands <ul><li>Tapping buttons </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().navigationBar()...
UIAutomation commands <ul><li>Scrolling </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). tableVi...
UIAutomation elements <ul><li>A complete list of elements and commands can be found at apple’s  UI Automation Reference Co...
Demo
UIAutomation - Summary <ul><li>“ Native” apple framework </li></ul><ul><li>Can be run both on device and simulator </li></...
Presenting - FoneMonkey
FoneMonkey <ul><li>Open source project release by  GorillaLogic </li></ul><ul><li>Records and plays back “higher-level” ev...
Adding FoneMonkey framework <ul><li>Duplicate your app build target to create a FoneMonkey testing target </li></ul>
Adding FoneMonkey framework <ul><li>Add FoneMonkey to your Project </li></ul>
Adding FoneMonkey framework <ul><li>Add FoneMonkey to your Testing Target </li></ul>
Adding FoneMonkey framework <ul><li>Add QuartzCore lib required by FoneMonkey to project </li></ul>
Adding FoneMonkey framework <ul><li>Add /Developer/Library/Frameworks/SenTestingKit </li></ul>
Using FoneMonkey <ul><li>When running the new target, the application is launched with the FoneMonkey console </li></ul><u...
Using FoneMonkey <ul><li>Start recording by tapping the Record button </li></ul><ul><li>Click the More button to view reco...
Using FoneMonkey <ul><li>Edit specific commands </li></ul><ul><li>Arguments values are per command </li></ul><ul><ul><li>T...
Using FoneMonkey <ul><li>Add verification steps </li></ul><ul><li>Support key-value verification of properties </li></ul>
Demo
Extending FoneMonkey <ul><li>There are many cases where we need to extend the basic functionality with custom gestures </l...
Adding touch events #import &quot;FoneMonkeyAPI.h&quot; #import &quot;MyView+FoneMonkey.h&quot; @implementation MyView (Fo...
Adding control events - (UIControlEvents)monkeyEventsToHandle { return UIControlEventEditingDidEnd; }  - (void) handleMonk...
Extending playback - (void) playbackMonkeyEvent:(FMCommandEvent*)event {    if (event.command isEqualToString:@&quot;Drag&...
What about gestures <ul><li>Unfortunately, FoneMonkey doesn’t have built in support for UIGestureRecognizers </li></ul><ul...
Extending gestures with Method swizzling <ul><li>Method swizzling  - Swapping method implementations in run time </li></ul...
Demo
FoneMonkey - Running tests
FoneMonkey script formats <ul><li>Every test recorded in FoneMonkey is saved in three formats: </li></ul><ul><ul><li>scrip...
Conclusion
FoneMonkey VS UIAutomation <ul><li>UIAutomation is a good framework with support for almost every UIKit component and gest...
Other alternatives <ul><li>FoneMonkey is my favorite UI testing framework </li></ul><ul><li>However, there are many other ...
Where to go from here <ul><li>Try it out! </li></ul><ul><ul><li>If you don’t need to extend functionality, integrating Fon...
<ul><li>Extending FoneMonkey does take some work </li></ul><ul><ul><li>This is actually how GorillaLogic probably makes mo...
Resources <ul><li>WWDC 2010 videos - Session 306 </li></ul><ul><li>David Reidy presentation </li></ul><ul><ul><li>http://w...
Thanks
Upcoming SlideShare
Loading in...5
×

Automating UI testing

11,308

Published on

The importance of automated unit tests and continuous integration is a well known fact, but in mobile applications unit tests just aren't enough. This presentation shows the importance of UI tests on iOS applications, and demonstrates how to build complex automated UI tests using the modern frameworks, focusing with Apple's UIAutomation framework and Gorilla Logic's FoneMonkey framework.

Published in: Technology, Education
1 Comment
14 Likes
Statistics
Notes
  • You can find the sample project I used in the presentation here - http://corecocoa.wordpress.com/2012/02/26/automating-ui-testing-on-the-ios/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
11,308
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
1
Likes
14
Embeds 0
No embeds

No notes for slide

Automating UI testing

  1. 1. Automating UI testing <ul><li>Adam Siton </li></ul>
  2. 2. About me
  3. 3. Road map <ul><li>What are automated UI tests and why do we need them </li></ul><ul><li>Apple’s UIAutomation framework </li></ul><ul><li>FoneMonkey </li></ul><ul><ul><li>This is where you should wake up </li></ul></ul>
  4. 4. Why do we need automated tests <ul><li>Hopefully - this is a rhetorical question </li></ul>
  5. 5. We want a 5 star app <ul><li>A great design and marketing will get you a lot of users </li></ul><ul><li>But if your app is buggy you will see a lot of those: </li></ul>
  6. 6. There are always more scenarios <ul><li>The number of test scenarios is always growing </li></ul><ul><ul><li>Each new feature = new scenarios </li></ul></ul><ul><ul><li>Each new view = new scenarios </li></ul></ul><ul><ul><li>Each bug discovered = new scenarios </li></ul></ul><ul><li>Impossible to test them all manually! </li></ul>
  7. 7. Refactoring is alway dangerous <ul><li>Most of the times fixing bugs introduce new bugs </li></ul><ul><li>The sooner we find a bug, the easier it is to fix it </li></ul>
  8. 8. Flexibility <ul><li>Automated tests can be run on multiple devices </li></ul><ul><li>And multiple SDKs </li></ul>
  9. 9. The solution - Automation <ul><li>We need a way to test all those scenarios automatically </li></ul><ul><li>Unit tests are great and important, but are focused on a specific feature - very hard to test how parts of the app work together </li></ul>
  10. 10. UIAutomation
  11. 11. UIAutomation framework <ul><li>A JavaScript library introduced in iOS SDK 4.0 </li></ul><ul><li>Integrated in Instruments </li></ul><ul><li>Can be run on the iPhone and Simulator </li></ul><ul><li>Touch based </li></ul>
  12. 12. UIAutomation elements <ul><li>Elements control is hierarchy based </li></ul><ul><li>UIAElement - the base class </li></ul><ul><ul><li>name </li></ul></ul><ul><ul><li>value </li></ul></ul><ul><ul><li>Elements </li></ul></ul><ul><ul><li>Parent </li></ul></ul>
  13. 13. UIAutomation elements <ul><li>Target - device operations (orientation, volume, shake, etc.) </li></ul>UIATarget.localTarget()
  14. 14. UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application - Access and control of application-level UI elements (windows, status bar, keyboard etc.) </li></ul></ul>UIATarget.localTarget() .frontMostApp()
  15. 15. UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window - control of the application’s window elements </li></ul></ul></ul>UIATarget.localTarget().frontMostApp() .mainWindow()
  16. 16. UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></ul></ul></ul><ul><ul><ul><ul><li>View </li></ul></ul></ul></ul>UIATarget.localTarget().frontMostApp().mainWindow() .tableViews()[0]
  17. 17. UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></ul></ul></ul><ul><ul><ul><ul><li>View </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>element </li></ul></ul></ul></ul></ul>UIATarget.localTarget().frontMostApp().mainWindow().tableViews()[0] .cells()[0]
  18. 18. UIAutomation elements <ul><li>Target </li></ul><ul><ul><li>Application </li></ul></ul><ul><ul><ul><li>Main window </li></ul></ul></ul><ul><ul><ul><ul><li>View </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>element </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Child element </li></ul></ul></ul></ul></ul>UIATarget.localTarget().frontMostApp().mainWindow().tableViews()[0].cells()[0] .elements()[“Chocolate Cake”]
  19. 19. Logging <ul><li>Since UIAutomation is based on view hierarchy, we need a simple way to examine the hierarchy </li></ul><ul><li>We do this by calling: </li></ul><ul><ul><li>UIATarget.localTarget(). logElementTree() </li></ul></ul>
  20. 20. Logging <ul><li>UIATarget.localTarget() </li></ul>
  21. 21. Logging <ul><li>UIATarget.localTarget(). frontMostApp() </li></ul>
  22. 22. Logging <ul><li>UIATarget.localTarget().frontMostApp(). mainWindow() </li></ul>
  23. 23. Logging <ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). tableViews()[0] </li></ul>
  24. 24. Logging <ul><li>UIATarget.localTarget().frontMostApp().mainWindow().tableViews()[0]. cells() </li></ul>
  25. 25. UIAutomation commands <ul><li>Tapping buttons </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().navigationBar(). buttons()[“Add”].tap(); </li></ul></ul><ul><li>Text input </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). textFields()[0].setValue(“new”) </li></ul></ul>
  26. 26. UIAutomation commands <ul><li>Scrolling </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). tableViews()[0].scrollToElementWithPredicate(“name beginsWith ‘Adam’); </li></ul></ul><ul><li>Sliding UIASlider </li></ul><ul><ul><li>UIATarget.localTarget().frontMostApp().mainWindow(). sliders()[0].dragToValue(0.5) </li></ul></ul>
  27. 27. UIAutomation elements <ul><li>A complete list of elements and commands can be found at apple’s UI Automation Reference Collection </li></ul><ul><li>More info at WWDC 2010 lecture - Automating User interface Testing with Instruments </li></ul>
  28. 28. Demo
  29. 29. UIAutomation - Summary <ul><li>“ Native” apple framework </li></ul><ul><li>Can be run both on device and simulator </li></ul><ul><li>Pretty good gesture support </li></ul><ul><li>Support for most UIKit components </li></ul><ul><li>Impossible to add functionality </li></ul><ul><li>Very fragile - small changes in UI cause massive changes in tests </li></ul><ul><li>No recording abilities </li></ul><ul><li>Must be a developer to add tests </li></ul><ul><li>Weak validation - only visibility and existence </li></ul><ul><li>Hard to integrate in CI </li></ul>
  30. 30. Presenting - FoneMonkey
  31. 31. FoneMonkey <ul><li>Open source project release by GorillaLogic </li></ul><ul><li>Records and plays back “higher-level” event </li></ul><ul><li>Flexible and extendable </li></ul>
  32. 32. Adding FoneMonkey framework <ul><li>Duplicate your app build target to create a FoneMonkey testing target </li></ul>
  33. 33. Adding FoneMonkey framework <ul><li>Add FoneMonkey to your Project </li></ul>
  34. 34. Adding FoneMonkey framework <ul><li>Add FoneMonkey to your Testing Target </li></ul>
  35. 35. Adding FoneMonkey framework <ul><li>Add QuartzCore lib required by FoneMonkey to project </li></ul>
  36. 36. Adding FoneMonkey framework <ul><li>Add /Developer/Library/Frameworks/SenTestingKit </li></ul>
  37. 37. Using FoneMonkey <ul><li>When running the new target, the application is launched with the FoneMonkey console </li></ul><ul><li>The console provides controls for recording, playback, validation and script management </li></ul>
  38. 38. Using FoneMonkey <ul><li>Start recording by tapping the Record button </li></ul><ul><li>Click the More button to view recorded commands </li></ul>
  39. 39. Using FoneMonkey <ul><li>Edit specific commands </li></ul><ul><li>Arguments values are per command </li></ul><ul><ul><li>Touch UIButton “myButton” </li></ul></ul><ul><ul><li>InputText UITextField “myTextField” new text </li></ul></ul>
  40. 40. Using FoneMonkey <ul><li>Add verification steps </li></ul><ul><li>Support key-value verification of properties </li></ul>
  41. 41. Demo
  42. 42. Extending FoneMonkey <ul><li>There are many cases where we need to extend the basic functionality with custom gestures </li></ul><ul><ul><li>double tap </li></ul></ul><ul><ul><li>Long press + drag gestures </li></ul></ul><ul><li>FoneMonkey can be extended for both recording and playback </li></ul>
  43. 43. Adding touch events #import &quot;FoneMonkeyAPI.h&quot; #import &quot;MyView+FoneMonkey.h&quot; @implementation MyView (FoneMonkey) - (NSString*) monkeyID {      return self.myViewId ? self.myViewId :         [super monkeyID]; } - (void) handleMonkeyTouchEvent:(NSSet*)touches withEvent:(UIEvent*)event {      UITouch* touch = [touches anyObject];      if (touch.tapCount == 2) {         [FoneMonkeyAPI record:self command:@&quot;DoubleTouch&quot; args:nil];      } else {         [super handleMonkeyTouchEvent:touches withEvent:event];      } }   @end
  44. 44. Adding control events - (UIControlEvents)monkeyEventsToHandle { return UIControlEventEditingDidEnd; } - (void) handleMonkeyEventFromSender:(id)sender forEvent:(UIEvent*)event { if (!self.editing && self.text != nil) { [FoneMonkeyAPI record:self command:FMCommandInputText args:[NSArray arrayWithObject:[self.text copy]]]; } else { [FoneMonkeyAPI continueRecording]; } }
  45. 45. Extending playback - (void) playbackMonkeyEvent:(FMCommandEvent*)event {    if (event.command isEqualToString:@&quot;Drag&quot;) {       [self performDrag];    } else { [super playBackMonkeyEvent:event]; } }
  46. 46. What about gestures <ul><li>Unfortunately, FoneMonkey doesn’t have built in support for UIGestureRecognizers </li></ul><ul><ul><li>Taps are OK, but every other gesture recognizer won’t be caught in handleMonkeyTouchEvent </li></ul></ul><ul><li>A very big problem since UIGestureRecognizers are widely used </li></ul><ul><li>Luckily - Can be solved easily with a little Method Swizzling </li></ul>
  47. 47. Extending gestures with Method swizzling <ul><li>Method swizzling - Swapping method implementations in run time </li></ul><ul><ul><li>The main difference from categories is that method swizzling let you use the original implementation </li></ul></ul>@implementation MyView (FoneMonkey) #import “ FoneMonkeyAPI.h” #import <objc/runtime.h> + (void) load      Method originalMethod = class_getInstanceMethod(self, @selector(handleSwipe:)); Method replacedMethod = class_getInstanceMethod(self, @selector(fm_handleSwipe:)); method_exchangeImplementations(originalMethod, replacedMethod); } @implementation MyView (FoneMonkey) - (void) fm_handleSwipe:(UIGestureRecognizer *)recognizer      [self fm_handleSwipe:recognizer]; [FoneMonkeyAPI record:self command:@&quot;Swipe&quot; args:nil]; }
  48. 48. Demo
  49. 49. FoneMonkey - Running tests
  50. 50. FoneMonkey script formats <ul><li>Every test recorded in FoneMonkey is saved in three formats: </li></ul><ul><ul><li>scriptName.fm - Native Mac Plist file used by the FoneMonkey console </li></ul></ul><ul><ul><li>scriptName.m - Drop-in Objective-C OCUnit test to be integrated with XCode </li></ul></ul><ul><ul><ul><li>Can be extended easily (loops etc.) </li></ul></ul></ul><ul><ul><ul><li>Can be run in Continuous Integration environment (behind the scope of this presentation) </li></ul></ul></ul><ul><ul><li>scriptName.js - UIAutomation Javascript for instruments tool </li></ul></ul>
  51. 51. Conclusion
  52. 52. FoneMonkey VS UIAutomation <ul><li>UIAutomation is a good framework with support for almost every UIKit component and gesture </li></ul><ul><li>It has 4 major down sides that make it very hard to use in the long term </li></ul><ul><ul><li>Impossible to extend </li></ul></ul><ul><ul><li>Very vulnerable to changes </li></ul></ul><ul><ul><li>Takes a long time to write tests </li></ul></ul><ul><ul><li>Can’t be integrated in a CI environment </li></ul></ul><ul><li>FoneMonkey has fewer support for components and gestures, but it answers the above bullets, so in the long run - it is the best solution in my opinion </li></ul>
  53. 53. Other alternatives <ul><li>FoneMonkey is my favorite UI testing framework </li></ul><ul><li>However, there are many other solutions. </li></ul><ul><li>The most famous are: </li></ul><ul><ul><ul><li>Selenium - http://seleniumhq.org </li></ul></ul></ul><ul><ul><ul><li>Frank - testingwithfrank.com </li></ul></ul></ul><ul><li>You can try out and see if they fit you better </li></ul><ul><ul><li>Everyone is entitled to his own wrong opinion </li></ul></ul>Frank
  54. 54. Where to go from here <ul><li>Try it out! </li></ul><ul><ul><li>If you don’t need to extend functionality, integrating FoneMonkey is less than a day. </li></ul></ul><ul><ul><li>Teach your QA team to record test scenarios </li></ul></ul><ul><ul><li>If you already have a CI environment - integrating FoneMonkey tests takes about 2-3 days </li></ul></ul>
  55. 55. <ul><li>Extending FoneMonkey does take some work </li></ul><ul><ul><li>This is actually how GorillaLogic probably makes monkey </li></ul></ul>But what if I do need extended functionality? <ul><li>Try to estimate the effort to integrate FoneMonkey vs the effort you spend on manual testing </li></ul><ul><li>If you need help or advice - my mailbox is always open: [email_address] </li></ul>
  56. 56. Resources <ul><li>WWDC 2010 videos - Session 306 </li></ul><ul><li>David Reidy presentation </li></ul><ul><ul><li>http://www.slideshare.net/DavidReidy/automated-ui-testing </li></ul></ul><ul><li>Alex Vollmer’s blog </li></ul><ul><ul><li>http://alexvollmer.com/posts/2010/07/03/working-with-uiautomation/ </li></ul></ul><ul><li>FoneMonkey docs - http://www.gorillalogic.com/books/fonemonkey-docs/home </li></ul><ul><li>FoneMonkey iOS Tutorial from IOS DevCon 2011 - http://www.gorillalogic.com/userfiles/fonemonkey/FoneMonkeyDevCon.pptx.pdf </li></ul><ul><li>Dr.Dobb’s blog </li></ul><ul><ul><li>http://drdobbs.com/open-source/231903414?pgno=1 </li></ul></ul>
  57. 57. Thanks

×