Testing sync engine

1. Testing Sync Engine

2. Testing (iOS) Sync Engine

3. Agenda • Overall tests stats • Our test strategies • MockTransportSession • Workflow • Problems • Future plans

4. Overall tests stats

5. • 3356 tests on OSX Overall tests stats

6. • 3356 tests on OSX • 3415 tests on iOS (+ apns tests) Overall tests stats

7. • 3356 tests on OSX • 3415 tests on iOS (+ apns tests) • ~ 7.5 minutes to build and run all the tests on one platform (build ~ 40s, tests ~ 7 min) Overall tests stats

8. • 3356 tests on OSX • 3415 tests on iOS (+ apns tests) • ~ 7.5 minutes to build and run all the tests on one platform (build ~ 40s, tests ~ 7 min) Overall tests stats

9. • Unit tests ~ 80% Overall tests stats

10. • Unit tests ~ 80% • Integration tests ~ 20% Overall tests stats

11. • Unit tests ~ 80% • Integration tests ~ 20% • Performance tests - 18 Overall tests stats

12. • Unit tests ~ 80% • Integration tests ~ 20% • Performance tests - 18 • Tests of tests Overall tests stats

13. • Unit tests ~ 80% • Integration tests ~ 20% • Performance tests - 18 • Tests of tests • Sorry, no UI tests Overall tests stats

14. Our test strategies

15. What we DON’T do: Our test strategies

16. What we DON’T do: • test against http:// (real backend) • test using http requests stubs (like OHHTTPStubs) Our test strategies

17. What we DON’T do: • test against http:// (real backend) • test using http requests stubs (like OHHTTPStubs) Why? • Stubs are not that flexible. Our test strategies

18. What we DON’T do: • test against http:// (real backend) • test using http requests stubs (like OHHTTPStubs) Why? • Stubs are not that flexible. • It’s still slow. Our test strategies

19. What we really do: • Test against fake backend Our test strategies

20. What we really do: • Test against fake backend Advantages • Fast • Flexible • Independent • Testable Our test strategies

21. What we really do: • Test against fake backend Advantages • Fast • Flexible • Independent • Testable Disadvantages • Need to write (and test!) code • Need to be in sync with real backend (can be improved) Our test strategies

22. MockTransportSession

23. WTF is THAT? MockTransportSession

24. WTF is THAT? ZMTransportSession ZMTransportSessi onErrorCode ZMBackgroundAc tivity ZMURLSessionCa ncelTimer ZMURLSessionSw itch ZMURLSession ZMTransportCodec ZMAccessToken ZMUserAgent ZMReachability ZMTransportData ZMTransportResp onse ZMTaskIdentiﬁerM ap ZMExponentialBa ckoff ZMAccessTokenH andler ZMTransportRequ estScheduler ZMNetworkSocket ZMDataBuffer ZMWebSocketFra me ZMWebSocketHan dshake ZMWebSocket ZMPushChannelC onnection ZMTransportPush Channel ZMStreamPairThre ad MockTransportSession

25. WTF is THAT? ZMTransportSession MockTransportSession

26. WTF is THAT? ZMTransportSession - is a delegate of a delegate of NSURLSession MockTransportSession

27. WTF is THAT? ZMTransportSession - is a delegate of a delegate of NSURLSession • Sends http requests to scheduler MockTransportSession

28. WTF is THAT? ZMTransportSession - is a delegate of a delegate of NSURLSession • Sends http requests to scheduler • Handles authentication MockTransportSession

29. WTF is THAT? ZMTransportSession - is a delegate of a delegate of NSURLSession • Sends http requests to scheduler • Handles authentication • Opens/closes web socket connection MockTransportSession

30. WTF is THAT? MockTransportSession - is a mock of ZMTransportSession MockTransportSession

31. WTF is THAT? MockTransportSession - is a mock of ZMTransportSession MockTransportSession - acts as a fake backend MockTransportSession

32. WTF is THAT? • Make fake requests MockTransportSession

33. WTF is THAT? • Make fake requests • Records requests and responds to them MockTransportSession

34. WTF is THAT? • Make fake requests • Records requests and responds to them • Manages internal database MockTransportSession

35. WTF is THAT? • Make fake requests • Records requests and responds to them • Manages internal database • Can simulate remote changes. MockTransportSession

36. WTF is THAT? • Make fake requests • Records requests and responds to them • Manages internal database • Can simulate remote changes. • Can send push channel (web socket) events MockTransportSession

37. • Before each test we insert objects in internal database (in memory) MockTransportSession

38. • Before each test we insert objects in internal database (in memory) • Test code make request using MockTransportSession MockTransportSession

39. • Before each test we insert objects in internal database (in memory) • Test code make request using MockTransportSession • It records and process incoming request (filtering by path and lots of ifs) MockTransportSession

40. • Before each test we insert objects in internal database (in memory) • Test code make request using MockTransportSession • It records and process incoming request (filtering by path and lots of ifs) • Performs some CRUD action on database depending on request (add message, create conversation, fetch conversation) MockTransportSession

41. • Before each test we insert objects in internal database (in memory) • Test code make request using MockTransportSession • It records and process incoming request (filtering by path and lots of ifs) • Performs some CRUD action on database depending on request (add message, create conversation, fetch conversation) • Creates and sends response. Optionally can delegate it to other object (i.e. test case). MockTransportSession

42. Workflow

43. • Finished feature - run all the tests Workflow

44. • Finished feature - run all the tests • Create pull request Workflow

45. • Finished feature - run all the tests • Create pull request • Review, merge Workflow

46. • Finished feature - run all the tests • Create pull request • Review, merge • Release script runs all tests locally Workflow

47. • Finished feature - run all the tests • Create pull request • Review, merge • Release script runs all tests locally • Pushes to remote branches (master/develop) Workflow

48. • Finished feature - run all the tests • Create pull request • Review, merge • Release script runs all tests locally • Pushes to remote branches (master/develop) • Xcode bots make integration on each push to master/ develop Workflow

49. • Finished feature - run all the tests • Create pull request • Review, merge • Release script runs all tests locally • Pushes to remote branches (master/develop) • Xcode bots make integration on each push to master/ develop • Each bot for each platform runs all the tests Workflow

50. • Finished feature - run all the tests • Create pull request • Review, merge • Release script runs all tests locally • Pushes to remote branches (master/develop) • Xcode bots make integration on each push to master/ develop • Each bot for each platform runs all the tests • Bots archive and upload binary to the cloud Workflow

51. Problems

52. Solved: • Tests isolation (mostly database state) Problems

53. Solved: • Tests isolation (mostly database state) • Handling asynchronous code Problems

54. Solved: • Tests isolation (mostly database state) • Handling asynchronous code Not yet solved: Problems

55. Solved: • Tests isolation (mostly database state) • Handling asynchronous code Not yet solved: • Reduce tests running time U don’t like me? =( Problems

56. Solved: • Tests isolation (mostly database state) • Handling asynchronous code Not yet solved: • Reduce tests running time • Sync with real backend Problems

57. Solved: • Tests isolation (mostly database state) • Handling asynchronous code Not yet solved: • Reduce tests running time • Sync with real backend • Flaky tests Problems

58. Future plans

59. • Split to smaller frameworks - to improve run time Future plans

60. • Split to smaller frameworks - to improve run time • Setup Buildasaur - to run tests on Github pull requests Future plans

61. • Split to smaller frameworks - to improve run time • Setup Buildasaur - to run tests on Github pull requests • Record/replay tests for MockTransportSession - to keep it in sync Future plans

62. That’s it!

Testing sync engine

Testing sync engine

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Testing sync engine

Similar to Testing sync engine(20)

Testing sync engine