SlideShare a Scribd company logo
Continuous Testing
Meets the Classroom
bcjordan.com
brian@code.org
@bcjordan
Brian Jordan, software engineer at Code.org
Code.org
non-profit
expanding
participation in CS
Hour of Code
200 million tried
Hour of Code
now what?
full curricula,
district partnerships,
professional
development,
policy change
K-12 CS Curriculum
20k K-12 teachers trained
intro courses:
260k teachers
8mil students
how do you even start
automated testing?
developing in
2013-2014
why test?
what is Code.org's
testing context?
the context
the context
the context
the context
the context
so how do we
test all that?
how we started
Selenium testing
how our tests are
organized and run
selenium-webdriver
cucumber
takeaways
Build a library of
steps
Build a library of
test levels
Build a library of
annotations
@no_mobile
@no_ie9
@skip
@db_access@as_student
who writes tests?
who/what
runs tests?
what challenges
did we end up facing?
challenge:
multiple browsers
solution:
selenium-webdriver
Sauce Labs
challenge:
testing local changes
solution:
chromedriver
sauce-connect
challenge:
interaction,
drag-and-drop
solution:
jQuery Simulate
$(selector).drag…
challenge:
handling
(intermittent)
failures
solution:
always fixing!
solution:
always fixing!
(try rerunning!)
what causes
intermittent failures?
Sauce Labs API
test_flakiness.rb
what do they look like
when running?
Speed Run!
(not) wasting time
speeding up
test runs
2014:
1-2h of tests
parallelization
at runner level
2016:
20-30min of tests
justifying
tests
the bug collection
Firefox getBBox()
my favorite
bug ever
who
investigates
failures?
dev of the day
what's next?
per-PR tests
takeaways
start small
everybody
tests
periodically
invest in
speed-ups
Q&A
bcjordan.com
brian@code.org
@bcjordan
Brian Jordan, software engineer at Code.org
code.org/help
github.com/code-dot-org

More Related Content

What's hot

What's hot (20)

Shift left-csun-sagar-barbhaya
Shift left-csun-sagar-barbhayaShift left-csun-sagar-barbhaya
Shift left-csun-sagar-barbhaya
 
Automated Testing With Jasmine, PhantomJS and Jenkins
Automated Testing With Jasmine, PhantomJS and JenkinsAutomated Testing With Jasmine, PhantomJS and Jenkins
Automated Testing With Jasmine, PhantomJS and Jenkins
 
Test Design + Environment Management: Scaling UI Automation for Agile
Test Design + Environment Management: Scaling UI Automation for AgileTest Design + Environment Management: Scaling UI Automation for Agile
Test Design + Environment Management: Scaling UI Automation for Agile
 
Getting Started with Selenium
Getting Started with SeleniumGetting Started with Selenium
Getting Started with Selenium
 
ASP.NET & Unit Testing
ASP.NET & Unit TestingASP.NET & Unit Testing
ASP.NET & Unit Testing
 
Manual, Visual, and Automated Testing For Web Apps
Manual, Visual, and Automated Testing For Web AppsManual, Visual, and Automated Testing For Web Apps
Manual, Visual, and Automated Testing For Web Apps
 
Getting Started With Selenium
Getting Started With SeleniumGetting Started With Selenium
Getting Started With Selenium
 
Cypress test techniques cucumber bdd framework,tdd,api tests course
Cypress test techniques cucumber bdd framework,tdd,api tests courseCypress test techniques cucumber bdd framework,tdd,api tests course
Cypress test techniques cucumber bdd framework,tdd,api tests course
 
Web Application Testing with Selenium
Web Application Testing with Selenium Web Application Testing with Selenium
Web Application Testing with Selenium
 
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)
DevQA: make your testers happier with Groovy, Spock and Geb (Greach 2014)
 
QA Automation testing online training
QA Automation testing online trainingQA Automation testing online training
QA Automation testing online training
 
Selenium and Appium Training from Sauce Labs
Selenium and Appium Training from Sauce LabsSelenium and Appium Training from Sauce Labs
Selenium and Appium Training from Sauce Labs
 
How To Use Selenium Successfully
How To Use Selenium SuccessfullyHow To Use Selenium Successfully
How To Use Selenium Successfully
 
How to Get Started with Cypress
How to Get Started with CypressHow to Get Started with Cypress
How to Get Started with Cypress
 
Selenium Testing on Chrome - Google DevFest Armenia 2015
Selenium Testing on Chrome - Google DevFest Armenia 2015Selenium Testing on Chrome - Google DevFest Armenia 2015
Selenium Testing on Chrome - Google DevFest Armenia 2015
 
Three Developer Behaviors to Eliminate 85 Percent of Accessibility Defects
Three Developer Behaviors to Eliminate 85 Percent of Accessibility DefectsThree Developer Behaviors to Eliminate 85 Percent of Accessibility Defects
Three Developer Behaviors to Eliminate 85 Percent of Accessibility Defects
 
Selenium Best Practices with Jason Huggins
Selenium Best Practices with Jason HugginsSelenium Best Practices with Jason Huggins
Selenium Best Practices with Jason Huggins
 
Testing Code.org's Interactive CS Curriculum
Testing Code.org's Interactive CS CurriculumTesting Code.org's Interactive CS Curriculum
Testing Code.org's Interactive CS Curriculum
 
Automated testing in javascript
Automated testing in javascriptAutomated testing in javascript
Automated testing in javascript
 
Visual Studio 2010 Testing for Developers
Visual Studio 2010 Testing for DevelopersVisual Studio 2010 Testing for Developers
Visual Studio 2010 Testing for Developers
 

Viewers also liked

Pivotal Failure - Lessons Learned from Lean Startup Machine DC
Pivotal Failure - Lessons Learned from Lean Startup Machine DCPivotal Failure - Lessons Learned from Lean Startup Machine DC
Pivotal Failure - Lessons Learned from Lean Startup Machine DC
Dave Haeffner
 
Agile testing for mere mortals
Agile testing for mere mortalsAgile testing for mere mortals
Agile testing for mere mortals
Dave Haeffner
 

Viewers also liked (20)

Selenium
SeleniumSelenium
Selenium
 
Web testing with Selenium
Web testing with SeleniumWeb testing with Selenium
Web testing with Selenium
 
Pivotal Failure - Lessons Learned from Lean Startup Machine DC
Pivotal Failure - Lessons Learned from Lean Startup Machine DCPivotal Failure - Lessons Learned from Lean Startup Machine DC
Pivotal Failure - Lessons Learned from Lean Startup Machine DC
 
Full Stack Testing Done Well
Full Stack Testing Done WellFull Stack Testing Done Well
Full Stack Testing Done Well
 
KISS Automation.py
KISS Automation.pyKISS Automation.py
KISS Automation.py
 
Agile testing for mere mortals
Agile testing for mere mortalsAgile testing for mere mortals
Agile testing for mere mortals
 
The Testable Web
The Testable WebThe Testable Web
The Testable Web
 
You do not need automation engineer - Sqa Days - 2015 - EN
You do not need automation engineer  - Sqa Days - 2015 - ENYou do not need automation engineer  - Sqa Days - 2015 - EN
You do not need automation engineer - Sqa Days - 2015 - EN
 
Selenium Basics
Selenium BasicsSelenium Basics
Selenium Basics
 
Web ui tests examples with selenide, nselene, selene & capybara
Web ui tests examples with  selenide, nselene, selene & capybaraWeb ui tests examples with  selenide, nselene, selene & capybara
Web ui tests examples with selenide, nselene, selene & capybara
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How ToCross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
 
Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...
Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...
Three Simple Chords of Alternative PageObjects and Hardcore of LoadableCompon...
 
Polyglot automation - QA Fest - 2015
Polyglot automation - QA Fest - 2015Polyglot automation - QA Fest - 2015
Polyglot automation - QA Fest - 2015
 
Bdd lessons-learned
Bdd lessons-learnedBdd lessons-learned
Bdd lessons-learned
 
Write Selenide in Python 15 min
Write Selenide in Python 15 minWrite Selenide in Python 15 min
Write Selenide in Python 15 min
 
Easy automation.py
Easy automation.pyEasy automation.py
Easy automation.py
 
Selenium Users Anonymous
Selenium Users AnonymousSelenium Users Anonymous
Selenium Users Anonymous
 
Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]
Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]
Selenide alternative in Python - Introducing Selene [SeleniumCamp 2016]
 
Cucumber Crash Course
Cucumber Crash CourseCucumber Crash Course
Cucumber Crash Course
 
Easy tests with Selenide and Easyb
Easy tests with Selenide and EasybEasy tests with Selenide and Easyb
Easy tests with Selenide and Easyb
 

Similar to Continuous Testing Meets the Classroom at Code.org

Best Practices for React Developer Test Technical Assessment for Hiring.pdf
Best Practices for React Developer Test Technical Assessment for Hiring.pdfBest Practices for React Developer Test Technical Assessment for Hiring.pdf
Best Practices for React Developer Test Technical Assessment for Hiring.pdf
DarshanaMallick
 

Similar to Continuous Testing Meets the Classroom at Code.org (20)

Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...
Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...
Tests for Every Branch Using CircleCI and Sauce Labs to Continuously Test CS ...
 
Fullstack JavaScript Developer - E-Degree
Fullstack JavaScript Developer - E-DegreeFullstack JavaScript Developer - E-Degree
Fullstack JavaScript Developer - E-Degree
 
Telerik Software Academy - Info Day - August 2012
Telerik Software Academy - Info Day - August 2012Telerik Software Academy - Info Day - August 2012
Telerik Software Academy - Info Day - August 2012
 
Софтуерна академия на Телерик - информационен ден (август 2012)
Софтуерна академия на Телерик - информационен ден (август 2012)Софтуерна академия на Телерик - информационен ден (август 2012)
Софтуерна академия на Телерик - информационен ден (август 2012)
 
Petri for kyiv.pptx
Petri for kyiv.pptxPetri for kyiv.pptx
Petri for kyiv.pptx
 
(M) brochure full stack development learning path
(M) brochure full stack development learning path(M) brochure full stack development learning path
(M) brochure full stack development learning path
 
Supporting Coding and Testing
Supporting Coding and TestingSupporting Coding and Testing
Supporting Coding and Testing
 
Resume- Zubair
Resume- ZubairResume- Zubair
Resume- Zubair
 
Yuvaraj-Resume
Yuvaraj-ResumeYuvaraj-Resume
Yuvaraj-Resume
 
Nagacv
NagacvNagacv
Nagacv
 
Kevin Nelms Baxter Resume
Kevin Nelms Baxter ResumeKevin Nelms Baxter Resume
Kevin Nelms Baxter Resume
 
shekhar_rawat_Java
shekhar_rawat_Javashekhar_rawat_Java
shekhar_rawat_Java
 
Der Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test MasterDer Test Manager ist tot - lang lebe der Test Master
Der Test Manager ist tot - lang lebe der Test Master
 
Digbijaya.biswal1234
Digbijaya.biswal1234Digbijaya.biswal1234
Digbijaya.biswal1234
 
Best Practices for React Developer Test Technical Assessment for Hiring.pdf
Best Practices for React Developer Test Technical Assessment for Hiring.pdfBest Practices for React Developer Test Technical Assessment for Hiring.pdf
Best Practices for React Developer Test Technical Assessment for Hiring.pdf
 
Lavina CV
Lavina CVLavina CV
Lavina CV
 
On The Road towards Continuous Delivery
On The Road towards Continuous DeliveryOn The Road towards Continuous Delivery
On The Road towards Continuous Delivery
 
Moodle quiz: towards post-paper e-assessment
Moodle quiz: towards post-paper e-assessmentMoodle quiz: towards post-paper e-assessment
Moodle quiz: towards post-paper e-assessment
 
Chennai Drupal Meet
Chennai Drupal MeetChennai Drupal Meet
Chennai Drupal Meet
 
From four to forty in four years - lessons from growing a team
From four to forty in four years - lessons from growing a teamFrom four to forty in four years - lessons from growing a team
From four to forty in four years - lessons from growing a team
 

More from Sauce Labs

Your Framework for Success: introduction to JavaScript Testing at Scale
Your Framework for Success: introduction to JavaScript Testing at ScaleYour Framework for Success: introduction to JavaScript Testing at Scale
Your Framework for Success: introduction to JavaScript Testing at Scale
Sauce Labs
 
Automating Hybrid Applications with Appium
Automating Hybrid Applications with AppiumAutomating Hybrid Applications with Appium
Automating Hybrid Applications with Appium
Sauce Labs
 
Quality at Speed: More API Testing, Less UI Testing
Quality at Speed: More API Testing, Less UI TestingQuality at Speed: More API Testing, Less UI Testing
Quality at Speed: More API Testing, Less UI Testing
Sauce Labs
 
Creating Digital Confidence with Test Automation
Creating Digital Confidence with Test AutomationCreating Digital Confidence with Test Automation
Creating Digital Confidence with Test Automation
Sauce Labs
 

More from Sauce Labs (20)

Simplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless ToolsSimplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless Tools
 
Testing on Mobile Devices with Location Services
Testing on Mobile Devices with Location ServicesTesting on Mobile Devices with Location Services
Testing on Mobile Devices with Location Services
 
Your Framework for Success: introduction to JavaScript Testing at Scale
Your Framework for Success: introduction to JavaScript Testing at ScaleYour Framework for Success: introduction to JavaScript Testing at Scale
Your Framework for Success: introduction to JavaScript Testing at Scale
 
Automating Hybrid Applications with Appium
Automating Hybrid Applications with AppiumAutomating Hybrid Applications with Appium
Automating Hybrid Applications with Appium
 
Quality at Speed: More API Testing, Less UI Testing
Quality at Speed: More API Testing, Less UI TestingQuality at Speed: More API Testing, Less UI Testing
Quality at Speed: More API Testing, Less UI Testing
 
Creating Digital Confidence with Test Automation
Creating Digital Confidence with Test AutomationCreating Digital Confidence with Test Automation
Creating Digital Confidence with Test Automation
 
Just Enough (Automated) Testing
Just Enough (Automated) TestingJust Enough (Automated) Testing
Just Enough (Automated) Testing
 
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
Using Axe to Add Accessibility Checks to Your Existing Selenium TestsUsing Axe to Add Accessibility Checks to Your Existing Selenium Tests
Using Axe to Add Accessibility Checks to Your Existing Selenium Tests
 
How Open Source Helps to Bring Back Product Obsession
How Open Source Helps to Bring Back Product ObsessionHow Open Source Helps to Bring Back Product Obsession
How Open Source Helps to Bring Back Product Obsession
 
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
Webinar: A Sneak Peek at Selenium 4 with Simon StewartWebinar: A Sneak Peek at Selenium 4 with Simon Stewart
Webinar: A Sneak Peek at Selenium 4 with Simon Stewart
 
[Deu] Test Automatisierung Mit Web Driver.io
[Deu] Test Automatisierung Mit Web Driver.io[Deu] Test Automatisierung Mit Web Driver.io
[Deu] Test Automatisierung Mit Web Driver.io
 
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
Accelerating Innovation: Leveraging Open Source to Optimize Your Shift-Left I...
 
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
Accelerating Your Digital Agenda with Continuous Testing ft. ForresterAccelerating Your Digital Agenda with Continuous Testing ft. Forrester
Accelerating Your Digital Agenda with Continuous Testing ft. Forrester
 
How to Measure Success in Continuous Testing
How to Measure Success in Continuous TestingHow to Measure Success in Continuous Testing
How to Measure Success in Continuous Testing
 
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
From Zero to 2.7 Million - How Verizon Media Embraced Open Source to Accelera...
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
 
Sauce Labs Webinar: Rising Importance of Software Testing
Sauce Labs Webinar: Rising Importance of Software TestingSauce Labs Webinar: Rising Importance of Software Testing
Sauce Labs Webinar: Rising Importance of Software Testing
 
BDD With Selenide by Hima Bindu Peteti
BDD With Selenide by Hima Bindu PetetiBDD With Selenide by Hima Bindu Peteti
BDD With Selenide by Hima Bindu Peteti
 
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
Closer To the Metal - Why and How We Use XCTest and Espresso by Mario Negro P...
 
Continuous Delivery for "Mature" Codebases by Melisa Benua
Continuous Delivery for "Mature" Codebases by Melisa BenuaContinuous Delivery for "Mature" Codebases by Melisa Benua
Continuous Delivery for "Mature" Codebases by Melisa Benua
 

Recently uploaded

Online blood donation management system project.pdf
Online blood donation management system project.pdfOnline blood donation management system project.pdf
Online blood donation management system project.pdf
Kamal Acharya
 
Fruit shop management system project report.pdf
Fruit shop management system project report.pdfFruit shop management system project report.pdf
Fruit shop management system project report.pdf
Kamal Acharya
 
Laundry management system project report.pdf
Laundry management system project report.pdfLaundry management system project report.pdf
Laundry management system project report.pdf
Kamal Acharya
 
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxCFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
R&R Consult
 
Automobile Management System Project Report.pdf
Automobile Management System Project Report.pdfAutomobile Management System Project Report.pdf
Automobile Management System Project Report.pdf
Kamal Acharya
 

Recently uploaded (20)

IT-601 Lecture Notes-UNIT-2.pdf Data Analysis
IT-601 Lecture Notes-UNIT-2.pdf Data AnalysisIT-601 Lecture Notes-UNIT-2.pdf Data Analysis
IT-601 Lecture Notes-UNIT-2.pdf Data Analysis
 
KIT-601 Lecture Notes-UNIT-5.pdf Frame Works and Visualization
KIT-601 Lecture Notes-UNIT-5.pdf Frame Works and VisualizationKIT-601 Lecture Notes-UNIT-5.pdf Frame Works and Visualization
KIT-601 Lecture Notes-UNIT-5.pdf Frame Works and Visualization
 
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical EngineeringIntroduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-4 Notes for II-II Mechanical Engineering
 
Event Management System Vb Net Project Report.pdf
Event Management System Vb Net  Project Report.pdfEvent Management System Vb Net  Project Report.pdf
Event Management System Vb Net Project Report.pdf
 
Halogenation process of chemical process industries
Halogenation process of chemical process industriesHalogenation process of chemical process industries
Halogenation process of chemical process industries
 
Introduction to Machine Learning Unit-5 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-5 Notes for II-II Mechanical EngineeringIntroduction to Machine Learning Unit-5 Notes for II-II Mechanical Engineering
Introduction to Machine Learning Unit-5 Notes for II-II Mechanical Engineering
 
ASME IX(9) 2007 Full Version .pdf
ASME IX(9)  2007 Full Version       .pdfASME IX(9)  2007 Full Version       .pdf
ASME IX(9) 2007 Full Version .pdf
 
Online blood donation management system project.pdf
Online blood donation management system project.pdfOnline blood donation management system project.pdf
Online blood donation management system project.pdf
 
Cloud-Computing_CSE311_Computer-Networking CSE GUB BD - Shahidul.pptx
Cloud-Computing_CSE311_Computer-Networking CSE GUB BD - Shahidul.pptxCloud-Computing_CSE311_Computer-Networking CSE GUB BD - Shahidul.pptx
Cloud-Computing_CSE311_Computer-Networking CSE GUB BD - Shahidul.pptx
 
Fruit shop management system project report.pdf
Fruit shop management system project report.pdfFruit shop management system project report.pdf
Fruit shop management system project report.pdf
 
Laundry management system project report.pdf
Laundry management system project report.pdfLaundry management system project report.pdf
Laundry management system project report.pdf
 
Construction method of steel structure space frame .pptx
Construction method of steel structure space frame .pptxConstruction method of steel structure space frame .pptx
Construction method of steel structure space frame .pptx
 
Peek implant persentation - Copy (1).pdf
Peek implant persentation - Copy (1).pdfPeek implant persentation - Copy (1).pdf
Peek implant persentation - Copy (1).pdf
 
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdfRESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
RESORT MANAGEMENT AND RESERVATION SYSTEM PROJECT REPORT.pdf
 
Top 13 Famous Civil Engineering Scientist
Top 13 Famous Civil Engineering ScientistTop 13 Famous Civil Engineering Scientist
Top 13 Famous Civil Engineering Scientist
 
HYDROPOWER - Hydroelectric power generation
HYDROPOWER - Hydroelectric power generationHYDROPOWER - Hydroelectric power generation
HYDROPOWER - Hydroelectric power generation
 
Democratizing Fuzzing at Scale by Abhishek Arya
Democratizing Fuzzing at Scale by Abhishek AryaDemocratizing Fuzzing at Scale by Abhishek Arya
Democratizing Fuzzing at Scale by Abhishek Arya
 
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxCFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptx
 
Automobile Management System Project Report.pdf
Automobile Management System Project Report.pdfAutomobile Management System Project Report.pdf
Automobile Management System Project Report.pdf
 
Pharmacy management system project report..pdf
Pharmacy management system project report..pdfPharmacy management system project report..pdf
Pharmacy management system project report..pdf
 

Continuous Testing Meets the Classroom at Code.org