Oscar reiken jr on our success at manheim
•Download as ODP, PDF•
0 likes•1,128 views
This document discusses test automation at Manheim, a wholesale auto auction company. It describes how test automation was implemented for three of Manheim's major applications: Ove.com, Simulcast, and Manheim.com. Regression testing times were reduced from over 160 hours to under 10 minutes for Ove.com and similar improvements for the other applications. This was achieved by converting test cases to Cucumber scenarios, prioritizing by business value, and implementing the tests in Ruby and Java using tools like Watir and Selenium. The automation allows running hundreds of tests in parallel and integration with a build pipeline.
Recommended
More Related Content
Similar to Oscar reiken jr on our success at manheim
Similar to Oscar reiken jr on our success at manheim (20)
More from Skills Matter
More from Skills Matter (20)
Recently uploaded
Recently uploaded (20)
Oscar reiken jr on our success at manheim
- 2. Hi!, I’m Oscar Coder/Tester/Hacker BDD Fan Automation enthusiast http://github.com/orieken @OscarRieken
- 3. A Bit about Manheim Whole Sale Auto Auction Mostly Agile-ish Lots of Legacy apps As400s, green screens, Chaos, Madness, and so on 3 Major applications – Ove.com – WholeSale Autos (Enterprise Rails app) – Simulcast – Live streaming Auctions (Java – Java – Swing) – Manheim.com – Account management (Java – Ruby – JRuby) And mobile Simulcast and Manheim.com
- 4. Before Regression – OVE.com – 160+ hours Manual and “Automated” (QTP) – Simulcast – ??? Manual and “Automated” (QTP) – Manheim.com – 80+ Automated ??? Manual Other big problems – Card Churn – What does the regression cover – Everyone on the team speaking different languages – Testing Data inconsistent (to long to set up)
- 5. Now Regression – OVE.com - ~ 2700 Scenarios about 7-10min (ruby – Watir-WebDriver) – Simulcast - ~150 Scenarios (Jruby – selenium + FEST) – Manheim.com – 1700 Scenarios (ruby – Watir-WebDriver)
- 6. How Dev, BA, QA identified the most important “test cases’ from QTP regression convert them to Cucumber Scenarios. (realized that app was not even really being regressed) Talked to product owners about what was important (everything as always) Got Dev, BA, QA to talk out scenarios before estimation – We write the automation for these after “estimation” and before
- 7. Some cool Things Exposing factories from testing framework in a small Sinatra app to allow “Functional” testers to create their own data. Allowing them to not be dependent on existing data Selenium Grid2 currently we have the ability to run about 400 scenarios in parallel with the ability to scale up as needed Build Pipeline – working on getting this down to under an hour. Moving towards the dream of Continuous Delivery – or as close as we can currently get.
Editor's Notes
- {"5":"I know I know it sounds like a lot to maintain, well it is kinda, \n<number>\n","6":"Probably the hardest part was getting everyone to talk together, \nOnce everyone started seeing value in how we took the scenarios and Dev’s could use them while they were building out functionality, BA and QA could use them to help sign off the cards, we still do a bit of a formal BA/QA signoff where they pair and go over the scenarios for a card and give it a once over, this for us isnt too bad usually takes less than 30 min total, and is a lot better than the days when BA’s would spend a few hours looking at a card and then testers would spend the same or more time looking at it. It also gives us a bit more confidence in what we are building.\n<number>\n","7":"Factories – Standing up Sinatra app seems like a small thing but this really helped out other teams that were not following our model, they had the ability to create states and data in our applications that they were able to use when doing these enterprise level integration tests (not all of our teams follow the same philosophies of software development) so the challenge was to give these other teams something easy for them to be able to call from their testing frameworks(mostly QTP) or for the manual testers or analysts with limited technical skills to fire off this data creation\nGrid – Supports (ie8, ie9, chrome, firefox, iOS6.1(ipad, iphone) using appium and working on getting android how we want it, and using puppet and some magic we have our grid up and running with various windows and about 20 linux vm’s and the ability to add more on the fly we are working on getting all our or teams to under 10 minute regressions, we are also experimenting with how we scale up and down \nPipeline – code check-in => unit tests => build => deploy to automation isolated => full regression => deploy to automation int => external services sanity check => full regression => ready to deploy to qa(pre prod)\nCD – I don’t know if we will actually get there for all of our products but for some we already have. and its created a buzz in all the departments and removed a lot of “Release Stress”, Everyone has noticed that it is easier when you start a\n<number>\n","2":"Hi, I’m Oscar \n<number>\n"}