In-house web
 automation?

 Open Source Bridge
       2010


                Adam Christian
http://adamchristian.com | @admc
Talk Format



Brain Dump
Question
Tools
Infrastructure
Infrastructure

• Boxes
• VM’s
• The Cloud
Continuous Integration
Continuous Integration

• Hudson
• BuildBot
• Cruise Control
• Home brew, etc
Frameworks!
•   Windmill
    •   JS Tests!    • Twill
• WatiR              • FunkLoad
    • WatiN (Iron)   • HTML Unit
•  ...
Frameworks!
•   Windmill
    •   JS Tests!         • Twill
• WatiR                   • FunkLoad
    • WatiN (Iron)        ...
Flash?
Flash?
• FlexPilot (my story)
• FlexMonkey
• Flash-Selenium
• Selenium Flex
• Commercial (QTP)
DEMO
Workload
Maintenance
Maintenance

•   Software Updates
    • WPKG - Windows
• Purity (Dirty Machines)
• Image Management
Expansion
Expansion


• Hardware budget
• Define your OS/Version matrix
• Cloud advantages
Languages
Languages

• JavaScript!
 •   Platform of the web

•   Integration
 •   DB, Email, XDomain Services

•   Focus on producti...
Coverage
Coverage


• User base
• Browser matrix for speed
• Smoke fringes
DEMO
Methodology
Tests
Tests

•Functional
• Unit
• Separate efforts
Reporting
Reporting

•True / False
• JUnit / XUnit
• Notifications!
 • Email, SMS, Tweets,Yams
Automation
Automation

•Take the leap
• QA
• Development
• Block Releases
Money Path
Money Path


• Start with what counts
• Robust and reliable
Alternatives
Pay Someone
 (Shameless Plug)
Pay Someone
       (Shameless Plug)



•Sauce Labs
• SOASTA
• BrowserMob
DEMO
Manually Test
Conclusion
More Info

• http://adamchristian.com
• http://saucelabs.com
• #windmill, #selenium,
 #watir, #saucelabs
 on FreeNode
Thanks for attending!
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
In-house web automation?
Upcoming SlideShare
Loading in …5
×

In-house web automation?

1,451 views
1,360 views

Published on

My brain dump experience on building CI systems.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,451
On SlideShare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Introduce self, what it takes to build and maintain a web automation system.

    My story!
  • I work at Sauce Labs, we are trying to make your life less painful.
  • This is my cat, he’s a nice guy, and yeah - I am a real person.

  • How many people are doing any sort of web automation?
    Of you how many have to deal with flash?
  • Awesome red wrench.
    When designing your automation system, be very careful about the tools you choose.
  • General
    Boxes - Great starting place, don’t scale that well
    VM’s - Expensive hardware and licensing, no recurring costs ++
    Cloud - hard to test internal apps without tunnels, recurring costs, slow
  • You need it, evaluate, pick one and go from there.
  • I’m not going to go into the “best one”, they all have strengths and weaknesses and you need to decide for yourself which fits your needs.

    If you can get away with not having a JS enabled browser, run your tests without one.


  • Demo FlexPilot in Sel IDE


  • Eventually you will need to expand your setup
    Upgrading VM servers gets expensive
    You should decide what configurations you really care about, because the matrix gets immense
    The cloud stays pay per machine per minute
  • JavaScript is a great platform for testing the web
    However it has some shortfalls when it comes to integration with various services
    You need to focus on what makes your QA/Dev teams the most productive
  • What are your users using? That plays a huge role in your technology choices
    If you can test just safari, IE, chrome and FF on windows, you should focus on that
    Smoke test macosx and linux as they are probably a tiny portion of your userbase

  • Hudson running a Selenium test
  • There are a few schools of thought behind how testing should be done, lets explore those
  • Functional test your user experience
    Unit test your backend code, integrate as much as you need and run those tests often
    I would advise you to separate those efforts functional testing and unit testing are important for different reasons, and different points in your process
  • True/False can tell you if 100% of your functional tests are passing (which is important), but its also incredibly important to know exactly which aren't passing
    Know all the time the status of your system, and be dedicated to it.

  • Automation is at a point where it has proven itself
    All elements of your company should be involved
    Take it seriously, if your testing suites aren’t stabilized, releases shouldn't go out
  • Start with what matters, your money path
    Improve that small set of tests to the point where failures really mean something serious


  • Shamless
  • Running a test in the Cloud
  • This does not scale, do I really have to talk about this?
  • I have told you all the different kinds of solutions that you can chose and how to tailor each of them to your specific needs, please go make it happen.


  • In-house web automation?

    1. 1. In-house web automation? Open Source Bridge 2010 Adam Christian
    2. 2. http://adamchristian.com | @admc
    3. 3. Talk Format Brain Dump
    4. 4. Question
    5. 5. Tools
    6. 6. Infrastructure
    7. 7. Infrastructure • Boxes • VM’s • The Cloud
    8. 8. Continuous Integration
    9. 9. Continuous Integration • Hudson • BuildBot • Cruise Control • Home brew, etc
    10. 10. Frameworks! • Windmill • JS Tests! • Twill • WatiR • FunkLoad • WatiN (Iron) • HTML Unit • Selenium • Grid
    11. 11. Frameworks! • Windmill • JS Tests! • Twill • WatiR • FunkLoad • WatiN (Iron) • HTML Unit • Selenium • Grid • Web Driver Story
    12. 12. Flash?
    13. 13. Flash? • FlexPilot (my story) • FlexMonkey • Flash-Selenium • Selenium Flex • Commercial (QTP)
    14. 14. DEMO
    15. 15. Workload
    16. 16. Maintenance
    17. 17. Maintenance • Software Updates • WPKG - Windows • Purity (Dirty Machines) • Image Management
    18. 18. Expansion
    19. 19. Expansion • Hardware budget • Define your OS/Version matrix • Cloud advantages
    20. 20. Languages
    21. 21. Languages • JavaScript! • Platform of the web • Integration • DB, Email, XDomain Services • Focus on productivity
    22. 22. Coverage
    23. 23. Coverage • User base • Browser matrix for speed • Smoke fringes
    24. 24. DEMO
    25. 25. Methodology
    26. 26. Tests
    27. 27. Tests •Functional • Unit • Separate efforts
    28. 28. Reporting
    29. 29. Reporting •True / False • JUnit / XUnit • Notifications! • Email, SMS, Tweets,Yams
    30. 30. Automation
    31. 31. Automation •Take the leap • QA • Development • Block Releases
    32. 32. Money Path
    33. 33. Money Path • Start with what counts • Robust and reliable
    34. 34. Alternatives
    35. 35. Pay Someone (Shameless Plug)
    36. 36. Pay Someone (Shameless Plug) •Sauce Labs • SOASTA • BrowserMob
    37. 37. DEMO
    38. 38. Manually Test
    39. 39. Conclusion
    40. 40. More Info • http://adamchristian.com • http://saucelabs.com • #windmill, #selenium, #watir, #saucelabs on FreeNode
    41. 41. Thanks for attending!

    ×