This document provides information about different types of automated tests for iOS applications, including: 1. Logic tests and application tests which check code functionality independently and in the context of an app. 2. Xcode test cases that can assert values, perform equality checks, and test for nil values. 3. Performance tests that measure execution time and compare performance of different code implementations. 4. UI automation tests that interact with user interfaces using JavaScript in Instruments to simulate user actions like taps and swipes. 5. Using Jenkins to manage test projects and write custom test scripts.

1. XCODE: AUTO TEST
Phineas

2. ABOUT AUTO TEST

3. ABOUT AUTO-TEST
Why?
If we're testing internals we should probably test
what user could see.
UI is the first thing users that see, so it is more
important than how it does whatever.

4. Hard to forget to test something.
Much expensive, time, labor, money, boring, etc.

5. ■ Logic tests. These tests check the correct functionality of a unit of
code by itself (not in an app). With logic tests you can put together
specific test cases to exercise a unit of code. You can also use logic
tests to perform stress-testing of your code to ensure that it behaves
correctly in extreme situations that are unlikely in a running app. These
tests help you produce robust code that works correctly when used in
ways that you did not anticipate.
■ Application tests. These tests check units of code in the context of
your app. You can use application tests to ensure that the connections
of your user-interface controls (outlets and actions) remain in place,
and that your controls and controller objects work correctly with your
object model as you work on your app. You can also use these tests to
perform hardware testing, such as getting the location of the device on
which your app is running.

6. UNIT TEST

7. TEST SCRIPT

8. INITIAL UNIT TEST
1. Static function will be executed
once
2. General function always run at
each test

9. XCTESTCASE
■ Fundamental Test
XCTAssert(expression, format...)
■ Boolean Tests
XCTAssertTrue(expression, format...)
XCTAssertFalse(expression, format...)

10. ■ Equality Tests
XCTAssertEqual(expression1, expression2, format...)
XCTAssertNotEqual(expression1, expression2, format...)
XCTAssertEqualWithAccuracy(expression1, expression2, accuracy, format...)
XCTAssertNotEqualWithAccuracy(expression1, expression2, accuracy,
format...)
■ Nil Tests
XCTAssertNil(expression, format...)
XCTAssertNotNil(expression, format...)
■ Unconditional Failure
XCTFail(format...)
XCTESTCASE

11. PERFORMATION TESTING
■ What are the absolute performance characteristics of
this code? Is the procedure bound by computation or
memory? What is the limiting behavior across different
sample sizes?
■ What are the relative performance characteristics of this
code, as compared to its alternatives? Which is faster,
methodA or methodB?

12. CFTimeInterval startTime = CACurrentMediaTime();
{
. . . . . . .
}
CFTimeInterval endTime = CACurrentMediaTime();
NSLog(@"Total Runtime: %g s", endTime - startTime);

13. uint64_t t = dispatch_benchmark(iterations, ^{
@autoreleasepool {
………
}
});
NSLog(@"[[NSMutableArray array] addObject:] Avg. Runtime: %llu ns", t);

14. measureBlock() {
………
}

15. ■ A performance test takes a block of code that you want to
evaluate and runs it ten times, collecting the average execution
time and the standard deviation for the runs. These statistics
combine to create a baseline for comparison, a means to
evaluate success or failure. To implement performance
measuring tests, you write methods using new API from
XCTest in Xcode 6 and later.

16. Test Suite 'Selected tests' started at 2014-12-11 01:21:27 +0000
Test Suite 'testerTests.xctest' started at 2014-12-11 01:21:27 +0000
Test Suite 'testerTests' started at 2014-12-11 01:21:27 +0000
Test Case '-[testerTests testPerformanceExample]' started.
2014-12-11 09:21:27.328 tester[724:13120] test start
/Users/sunxiaoshan/Desktop/Report/XCodeTest/tester/tester/testerTests/testerTests.m:164: Test Case '-[testerTests testPerformanceExample]' measured [Time, seconds]
average: 0.003, relative standard deviation: 3.742%, values: [0.002862, 0.002867, 0.002876, 0.003214, 0.003048, 0.002881, 0.002917, 0.002864, 0.002859, 0.002884],
performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "Local Baseline", baselineAverage: 1.810, maxPercentRegression: 10.000%,
maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100
2014-12-11 09:21:27.705 tester[724:13120] test end
Test Case '-[testerTests testPerformanceExample]' passed (0.378 seconds).
Test Suite 'testerTests' passed at 2014-12-11 01:21:27 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds
Test Suite 'testerTests.xctest' passed at 2014-12-11 01:21:27 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds
Test Suite 'Selected tests' passed at 2014-12-11 01:21:27 +0000.
Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.381) seconds

17. XCTESTEXPECTATION
XCTestExpectation *documentOpenExpectation = [self
expectationWithDescription:@"document open”];
[documentOpenExpectation fulfill];
[self waitForExpectationsWithTimeout:15
handler:^(NSError *error) {
if (error != nil) {
XCTFail(@"timeout error: %@", error);
}
}];

18. UI AUTOMATION TEST

19. How to do it
XCode tool: Automation
Script Language : JavaScript

20. How to get object
window.buttons()[0]
window.buttons()[“login”]

21. HOW TO START

22. HOW TO START (I)
Create a new trace
document in Instruments
using the Automation trace
template.

23. HOW TO START (II)
Click Add > Create.

24. HOW TO START (III)
In the Detail pane
Navigation bar, select
Script to enter the code for
your script.

25. VIEW STRUCTURE

26. ELEMENT HIERARCHY (I)
UIATarget.localTarget()
frontMostApp()
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

27. ELEMENT HIERARCHY
(II)
UIATarget.localTarget()
frontMostApp()
mainWindow()
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

28. ELEMENT HIERARCHY
(III)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

29. ELEMENT HIERARCHY
(IV)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
cells()[6]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

30. ELEMENT HIERARCHY (V)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
cells()[6]
elements()["Name"]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

31. ELEMENT HIERARCHY
(VI)
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html

32. INTERACTION

33. INTERACTION
element.tap()
touchAndHold()
doubleTap()
scrollToVisible()
captureScreenWithName()

34. JENKINS

35. It’s a tool that can manager projects easily and write
the custom script quickly by yourself.

36. JENKINS PLUGIN

37. NEW PROJECT

38. XCTOOL COMMAND LINE

39. REFERENCE

40. REFERENCE
https://developer.apple.com/library/mac/documentation/DeveloperTools/
Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/
UsingtheAutomationInstrument.html
https://developer.apple.com/library/ios/documentation/DeveloperTools/
Reference/UIAutomationRef/_index.html#//apple_ref/doc/uid/TP40009771
http://blog.manbolo.com/2012/04/08/ios-automated-tests-with-
uiautomation
http://justbm.blogspot.tw/2011/12/my-ios-ui-automation-testing.html
http://www.slideshare.net/DavidReidy/automated-ui-testing

41. http://nshipster.com/xctestcase/
https://developer.apple.com/Library/ios/
documentation/DeveloperTools/Conceptual/
testing_with_xcode/testing_3_writing_test_classes/
testing_3_writing_test_classes.html
https://developer.apple.com/legacy/library/
documentation/DeveloperTools/Conceptual/UnitTesting/
01-Unit-Test_Overview/overview.html
https://developer.apple.com/library/IOs/documentation/
DeveloperTools/Conceptual/testing_with_xcode/
testing_4_running_tests/testing_4_running_tests.html

42. http://jenkins-ci.org/