This document summarizes key points from a GTAC 2015 conference in Boston on software testing. It discusses the speaker's background and details of the conference, including attendance numbers from various countries and the number of talks. The document then summarizes several talks on testing experience and tools. Common insights emphasized automation and speed. Testing strategies discussed included setting goals, transparency, and prioritizing automation. Specific tools mentioned include Octopus, Monkey, Diffy, Applitools Eyes, Flashback, and RpcReplay. The document concludes with additional testing thoughts and a question/answer section.

1. dinosu@kkbox.com
2015/12/4
GTAC 2015@Boston

2. ● Software Engineer in Test since 2004
● Biker
● Photographer
● Me on Internet
○ Twitter: @din0su
○ LinkedIn: dinosu
○ Github: dino-su
# whoami
Dino Su

4. 1562 applications from worldwide

5. ● 1362 attendance applications
● 200 speaker applications
● Iseral, Austrilla, South Africa, Finland, Taiwan

6. 25 talks in 14 hours

7. 32 speakers

10. next GTAC will be in 2017 Spring

11. Subject #1: Experience Sharing

12. Testing the tube
Keynote
Keynote

14. whole team testing

15. Step1: Team Commitment

16. ● Agree on metrics
● Progress transparency
● Allow team-based drill down
● Allow team-based metrics additions

19. ● product
● engineering
● operaions (Production & Release)
Dimensions of Quality

20. Step2: Automation

21. ● wherever possible
● big focus on mobile/device testing
● integration testing
● no manual regression
● push process
Push for automation

22. Step3: Set Goals

23. ● Start simple: cut by X in Y quarters
● Enforce via team commitments
● adjust as progress achieved
Goals

26. ● Testing the Tube
● Chromecast Testing
● Nest Automation Infrastructure
● Enabling Streaming Experiments at Netflix
● Effective Testing of a GPS Monitoring Station Receiver
● Chrome OS Test Automation Lab
More testing experience talks

27. Common Insights
Automation & Automation & Automation
Speed & Speed & Speed

28. Subject #2: Tools

29. Octopus from Uber

30. The problem

31. Solution

32. ● Test Host
● Test Target
● Channel
● Signal
○ read
○ write

34. ● Question: But why not mock instead of E2E ?
● Answer: It’s only used for core features to increase final
confidence for release.

35. code-less test automation

37. 1. ease of use
2. Android framework compatibility
3. code coverage
4. fault detection
Benchmark Criteria

41. Winner

42. Monkey test Tips
● How to setup monkey test
# setup
./gradlew installDebug installDebugAndroidTest
adb shell am instrument -w -e annotation com.kkbox.sqa.fuzz_test.test.Before
com.kkbox.sqa.fuzz_test.test/android.support.test.runner.AndroidJUnitRunner
# exercise
adb shell monkey -p com.skysoft.kkbox.android -v 30000
# teardown
adb shell am instrument -w -e annotation com.kkbox.sqa.fuzz_test.test.After
com.kkbox.sqa.fuzz_test.test/android.support.test.runner.AndroidJUnitRunner

43. ● 5 minutes as maximum (30000 events)

44. Diffy & Applitools

48. The tool saved Twitter engineers
from being layoff

49. ● Applitools Eyes
● The most powerful
image comparison
engine I've ever seen

55. unfortunately, it costs you $99 user
/ month

56. new network mocking tools
Flashback & RpcReplay

59. It makes integration testing become
more pleasure

60. Subject #3: Testing thoughts

61. component testing Soup Dumplings

70. 75% code coverage seems like a
right number

71. The Fun Part

72. 4am fire alarm

73. Google Cambridge

78. Q&A
dinosu@kkbox.com