Journey to 1000 tests a day
Upcoming SlideShare
Loading in...5
×
 

Journey to 1000 tests a day

on

  • 1,974 views

Automation has come a long way in the last 15 years. This presentation tells the story of my journey from no automation, to running in excess of 1000 tests a day. It shares my insights, failures, ...

Automation has come a long way in the last 15 years. This presentation tells the story of my journey from no automation, to running in excess of 1000 tests a day. It shares my insights, failures, successes and lessons learned, so hopefully you can get there too.

Statistics

Views

Total Views
1,974
Views on SlideShare
1,083
Embed Views
891

Actions

Likes
0
Downloads
3
Comments
0

8 Embeds 891

http://www.teknologika.com 843
http://teknologika.com 34
http://feeds.feedburner.com 5
http://www.linkedin.com 5
http://webcache.googleusercontent.com 1
http://translate.googleusercontent.com 1
http://www.newsblur.com 1
https://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Journey to 1000 tests a day Journey to 1000 tests a day Presentation Transcript

  • Journey to 1000 Tests a Day Bruce McLeodTuesday, 15 November 11
  • DISCLAIMER The views and content presented in this presentation are my own and do not represent the views of any of my employers, past or present. All content is copyright 2011 Bruce McLeod Creative commons attribution for images is given at the end of this presentationTuesday, 15 November 11
  • Tuesday, 15 November 11
  • Every journey has a first step My introduction to Test AutomationTuesday, 15 November 11
  • Record and playbackTuesday, 15 November 11
  • Huge gains compared to manual execution • Win32 desktop application • 5 different clients (95, 98 Beta, NT4, NT 351 and Citrix) • 5 days work for 3 people could be done in 4 hours • 97% efficiency improvementTuesday, 15 November 11
  • But there were problems • Tests were not maintainable • Spent days in maintenance and development • They were brittle and fragile • And when I left for my next job ...Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Kindergarten The early manual coding yearsTuesday, 15 November 11
  • Manual coding beginsTuesday, 15 November 11
  • A 2001 test automation recipe Ingredients 1 eager client 1 test application 14 test automatorsTuesday, 15 November 11
  • A 2001 test automation recipe Method Step 1 : Write an automation “framework” Step 2 : Create a gui map Step 3 : Write a test Step 4 : Repeat step 4 another 200 timesTuesday, 15 November 11
  • Lessons learnt • A Successful project • 200 tests automated in 6 months • Team was a real dev. team with all the same issues • Versioning • Merging and branching • Coding standards • Individual differences in style and designTuesday, 15 November 11
  • Tuesday, 15 November 11
  • Childhood The innocent years of keyword testingTuesday, 15 November 11
  • What if we remove the need to code?Tuesday, 15 November 11
  • Spreadsheets !!!Tuesday, 15 November 11
  • Keyword driven testing Object Action Data Textfield (domain) Enter text <domain> Textfield (username) Enter text <username> Textfield (password) Enter text <password> Button (login) Click One left click l Object - Action - data syntax l Automation libraries with excel front endTuesday, 15 November 11
  • Keyword driven testing • Advantages • Reduction in coding effort • “Anyone” can write tests!! • Challenges • “Anyone” can write tests!! • Each implementation is custom • Maintenance effort moves from code into ...Tuesday, 15 November 11
  • … Lots of Spreadsheets !!!Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Adolescence Embracing open sourceTuesday, 15 November 11
  • Manual coding returnsTuesday, 15 November 11
  • Until …Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Unexpected benefits • We could now use 1st generation developer tools • Visual Studio 2005 • Version control • C# and .net instead of VBATuesday, 15 November 11
  • Unexpected benefits • Testers were now true peers to developers • Same tools • Same development language • Same recruitment process • Object oriented designTuesday, 15 November 11
  • Tuesday, 15 November 11
  • A discovery & epiphany! The “braidy tester” blog postsTuesday, 15 November 11
  • The “sparkle” stack • Used to test Microsoft Expression • Key principles • Test cases written in customer view • Verification and data separated from tests • Application internals re-created in test codeTuesday, 15 November 11
  • Tuesday, 15 November 11
  • How can I execute 1000 tests a day ?Tuesday, 15 November 11
  • Design goals • Maintainability • Robustness • Performance • Maintainability • Rapid test developmentTuesday, 15 November 11
  • MaintainabilityTuesday, 15 November 11
  • Devs change stuff It’s their job !Tuesday, 15 November 11
  • Each action should occur once and once onlyTuesday, 15 November 11
  • Each control should be mapped once and once onlyTuesday, 15 November 11
  • Abstract the engine You shouldn’t know what’s “under the bonnet”Tuesday, 15 November 11
  • RobustnessTuesday, 15 November 11
  • Tests should fail, not your tools.Tuesday, 15 November 11
  • Embrace regexTuesday, 15 November 11
  • Leverage your vendor, and the communityTuesday, 15 November 11
  • PerformanceTuesday, 15 November 11
  • CPU should be your bottleneckTuesday, 15 November 11
  • Never wait ! Wait, don’t sleepTuesday, 15 November 11
  • Scale-outTuesday, 15 November 11
  • Rapid test developmentTuesday, 15 November 11
  • K.I.S.STuesday, 15 November 11
  • Cheat ! • Leverage an IDE • Refactoring • Use code generation • Write less code • Use design patternsTuesday, 15 November 11
  • What about the future?Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Open source frameworks will consolidate Driven by adoption of webdriverTuesday, 15 November 11
  • More focus on mobile automation iOS, Android, Windows PhoneTuesday, 15 November 11
  • Behaviour driven development Will become mainstreamTuesday, 15 November 11
  • Given I am a logged in user When I have a valid credit card And I have 1 click purchasing enabled Then I should see the message “your purchase was successful” And I should be re-directed to the home page.Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Automation is development. Hire, plan, develop and execute with that in mind.Tuesday, 15 November 11
  • Tuesday, 15 November 11
  • Image attribution • Creative commons attribution credit is required for the following users on flickr • seattlemunicipalarchives • mathias-erhart • jaffathecake • 28096801 • loty • brilliantmichaelTuesday, 15 November 11