Join Brian Jordan from Code.org as he takes you through the ins and outs of developing test automation for the world's premiere computer-science education platform. Code.org is a non-profit aimed at expanding access to computer science, by making computer science accessible to school-children everywhere, via its free online interactive curriculum and corresponding teacher-preparation programs. Currently used by 300,000 teachers and over 10 million students, Code.org is required to deliver a stable, robust product to all its stakeholders in the classroom — not an easy task, considering the product's unique game-like interactive features. Brian Jordan, software engineer at Code.org since 2014, introduces the complexities of Code.org's unique interactive product - a puzzle-progression-based computer-science curriculum - and describe how Code.org approaches Continuous Testing throughout the development cycle. He also presents Code.org automated testing suite, including: architecture, frameworks, tools, and best practices - and how those are designed to address complex QA issues, such as visual & functional testing, cross-browser & cross-device testing, and localization testing for their over 40 supported languages. Watch this webinar, and learn: * What Code.org open-source automated testing stack looks like * What visual testing with Applitools, cross-browser Selenium tests on Sauce Labs, and live-site monitoring with New Relic and Honeybadger look like in practice * About Code.org "Bug Collection": real live examples of bugs detected before they hit production * BONUS: Brian playbacks Code.org's full Selenium test-suite in high-speed. See Selenium drag-and-drop code its way through Code.org’s programming puzzles.

1. Continuous Testing
at Code.org
@bcjordan
Brian Jordan, software engineer at Code.org

2. Code.org

3. non-profit

4. expanding
participation in CS

5. Hour of Code

7. 230 million tried
Hour of Code

8. now what?

9. full curricula,
district partnerships,
professional
development,
policy change

10. K-12 CS Curriculum

11. 25k K-12
teachers trained

12. courses:
300k teachers
10mm students

15. how did Code.org
start automated testing?

16. development
in 2013-2014

17. why test?
what is Code.org's
testing context?

18. the context

19. the context

20. the context

21. the context

22. the context

23. so how do we
test all that?

24. how we started
Selenium testing

25. how our tests are
organized and run
selenium-webdriver
cucumber

28. takeaways

29. Build a library of
steps

30. Build a library of
test levels

31. Build a library of
annotations
@no_mobile
@no_ie9
@skip
@db_access@as_student
@eyes

32. who writes tests?

33. who/what
runs tests?

35. what challenges
did we end up facing?

36. challenge:
multiple browsers

37. solution:
selenium-webdriver
Sauce Labs

41. challenge:
testing new changes

42. solution:
chromedriver
sauce-connect

43. solution (soon):
pull request tests in CI

44. challenge:
interaction,
drag-and-drop

45. solution:
jQuery Simulate
$(selector).drag…

46. challenge:
visual, responsive
changes

47. hackathon solution?

48. not easy...

49. actual thing!

50. using eyes

51. converted feature

53. ignoring regions

59. what do tests look
like when running?

60. (not) wasting time

61. speeding up
test runs

62. 2014:
60-90 min of tests

63. parallelization
at runner level

64. 2015:
60-90 min of tests
to
20-30 min of tests

65. justifying
tests

66. the bug collection

67. missing button

68. duplicate button

69. Firefox getBBox()

70. JS % error: node build issue

71. Chrome 50 offsetWidth

72. my favorite
bug ever

77. who investigates
failures?
what about failures
pass tests?

78. dev of the day

80. what's next?

81. per-PR tests

82. managing more eyes tests

83. cross-browser responsive
eyes tests
github.com/tourdedave/responsive-web-testing

84. takeaways

85. start small

86. everybody
tests

87. periodically
invest in
speed-ups

88. code.org/help
github.com/code-dot-org
Thanks!
@bcjordan
Brian Jordan, software engineer at Code.org

89. code.org/help
github.com/code-dot-org
Thanks!
@bcjordan
Brian Jordan, software engineer at Code.org