Slides from the NYC Metro Selenium Users Meetup group: http://www.meetup.com/NYCSelenium/events/17236633/

1. Automated Regression Suites in a CI Environment
by Michelle D’Netto

2. Etsy’s ‘Ye Olde’ Automation Test Suite
Python and Selenium-RC
Random data
Non-deterministic results
5 deploys, 20 engineers

3. How Etsy does Automation Now
Automation Engineer
BDD framework
Ruby with Selenium-RC
Data-driven tests
25 deploys, 70 Engineers

4. Tiered Services
T1:
* Business Critical Features
* If these services are broken we’re screwed

5. Tiered Services
T2:
* Important but not mission critical services

6. Customer Focus...
Customer focus... is paramount to us.

7. Headless Testing
$ Xvfb :99 -ac
$ export DISPLAY=:99

8. At Etsy...
At Etsy.... everyone is responsible for
quality.

9. Cucumber
@register
Feature: Register for an Etsy account as a new buyer
As an Etsy buyer
I want to register for an account
So that I can buy items on Etsy
Scenario: I register for an Etsy account, confirm my registration and sign into Etsy
Given I am new to Etsy
When I type in the registration url
And I select my new_buyer account credentials (username, password, email)

10. Ruby Step Definition
And I select my new_buyer account credentials (username, password, email)
And /^I select my (.*) account credentials (username, password, email)$/do | role |
register eval(“username_of_#{role}”), eval(“password_of_#{role}”,
eval(“email_address_of_#{role}”)
steps %Q{Then the duplicate username message should not appear}
end

11. Signin Register Helper
module signin_register_Helper
def register_for_etsy opt
@selenium.type opt[:username_field], opt[:username]
@selenium.type opt[:password_field], opt[:password]
@selenium.type opt[:confirm_password_field],
opt[:confirm_password]
@selenium.type opt[:email_address_field], opt[:email_address]
end
def register username, password, email_address
register_for_etsy ({
:username_field => username_field_for_registration,
:password_field => password_field_for_registration,
:confirm_password_field => confirm_password_field,
:email_address_field => email_address_field,
:email_address => email_address,
:username => username,

12. Guard Assertions
steps %Q{Then the duplicate username message should not appear}
And /^the duplicate username message should not appear$/ do
@selenium.is_element_present('css=span#username-error') &&
@selenium.get_text('css=span#username-error').should_not(match(/Sorry, that username is
taken./))
end

13. Selenium Test Runs

14. Username Error

15. Data Driven Tests
module Unique_ID_Helper
def username_of_new_buyer
uid_base_new_buyer_username +
uid
end
def uid
ENV['BUILD_NUMBER']
end
def uid_base_new_buyer_username
"misspiggy"
end
end

16. Continuous Integration
Our deployment environment requires a lot of trust,
transparency, communication, coordination, and
discipline across the team.
- Chad Dickerson (CTO @ Etsy)

17. Deployinator

18. Test Slaves

19. Jenkins Projects

20. Green Test Builds

21. Cucumber Test Results reported through Jenkins

22. Challenges
We can't see any test results!

23. Advantages

24. Read More....
http://codeascraft.etsy.com/

25. Q&A