Failure and Success in Software Automation by Oleksandr Reminnyi


Published on

Oleksandr Reminnyi, System Architect at SoftServe, Inc. compared seven projects for his presentation on Failure and Success in Software Automation at the 2013 IT Weekend Ukraine conference that took place on September 14, 2013, in Kyiv, Ukraine. Here`s Oleksandr`s presentation.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Failure and Success in Software Automation by Oleksandr Reminnyi

  1. 1. Failure and Success in Software Automation Comparing 7 Projects Oleksandr Reminnyi
  2. 2. Pre word ▪ The only identification of project being successful is the fact that your customer is happy. PMbook
  3. 3. Understanding the goals We want to save money and time later on manual testing We have a corporate direction… But the main is… Automation is a BUZZ word
  4. 4. Average Automation Reality
  5. 5. Automation AND Skill Set
  6. 6. HOKKU «Let`s automate everything!», the client said. And Mount Fuji trembled.
  7. 7. Customer skills Should customer be mature in automation? This is definitely not a prerequisite. But it can influence…
  8. 8. ХОККУ Читаю чужі тести і торкається моїх скронь сивина… То мозок проливається
  9. 9. Automation Dev. Team Skills Does mature mean good automation here? Not always. Practices mean more.
  10. 10. Automation AND Time line
  11. 11. Automation and SDLC When the automation might be successful? Waterfall OR Agile? Depends on automation layer; Post release period!
  12. 12. Implementation OF one TC Depends on complexity Depends on number of steps Depends on tested application instrumentation Average 4 to 20 man hours. Happens 50
  13. 13. Automation Toolset
  14. 14. Automation and Backend Unit Tests Really? Why? Component/integration testing Good job! Service layer automation Definite place to automate!
  15. 15. Automation And TOOLS Can tool influence the success of automation? YES!
  16. 16. Automation of Mobility Can mobile platform automation be buried before birth? YES! Silk Mobile $5000 per dev. per year, 20-30% failure rate only because of the tool issues MonkeyTalk, Ranorex, etc. Build in the agent, then find issues Appium maybe light in the end of the tunnel. But later Consider out of device testing!
  17. 17. Documentation
  18. 18. Can automation exist without regression test Cases suite? YES! At least they say it’s successful…
  19. 19. Different types of manual test cases being Automated - Functional Regression Acceptance … Benefit is brought at ones once by Smoke
  20. 20. Automation Metrics
  21. 21. HOKKU Sacura`s leaves are destined to fall down. Once again we`ll see the Test Report in red. Not only leaves are down.
  22. 22. What is usually Calculated? ROI? Nothing! Sometimes: Failure Rate Suite Run Duration
  23. 23. Given architecture… SUT Automation Tool UI Services … Other SUT facilities that can be hooked but not needed for automation tool access Reporting Base classes Layer Framework utilities Test scripts Test scripts UI Maps – facilities to access UI (or other services) through automation tools API Layer Etc. Loading configuration Test scripts Libraries Test scripts Functional Logging Layer DB Helpers Test scripts Test scripts Test scripts Facilities to hook into SUT through AF Multi - layered architecture Sometimes joined Layer
  24. 24. Gave us possibility to collect More metrics
  25. 25. Something obvious
  26. 26. Questions? “I see the testing standards that do not tell me how to create a test worthy of running." DeMarco