Successfully reported this slideshow.
Your SlideShare is downloading. ×

[Selenium] Effective ROI for test automation

Ad

Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Effective ROI
for getting started test automation
Woohyeok Kim
Rakuten,...

Ad

Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Woohyeok Aaron Kim
• Software Engineer
• Leading test automation team
•...

Ad

Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Why Test Automation
• The end of waterfall era
• New era satisfying wit...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Loading in …3
×

Check these out next

1 of 19 Ad
1 of 19 Ad

More Related Content

[Selenium] Effective ROI for test automation

  1. 1. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Effective ROI for getting started test automation Woohyeok Kim Rakuten, Inc. @woosyume
  2. 2. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Woohyeok Aaron Kim • Software Engineer • Leading test automation team • Scrum Master • At Rakuten Ichiba Ranking Frontend • Organizer • Japan Selenium UC Kansai branch • Korea Selenium UC (co-founder) • Speaker • SeleniumConf 2019 • Rakuten Technology Conference 2019 • Regional Scrum Gathering Tokyo 2020
  3. 3. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Why Test Automation • The end of waterfall era • New era satisfying with customer needs with agility is coming • Quality Assurance is essential Product For customer
  4. 4. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Test Automation • No one can deny the importance. • Global companies already introduced test automation • Introducing itself was done a few years ago • They’re trying to improve their test application • Performance improvement, Test Strategy etc. • Japanese companies are getting started test automation
  5. 5. Japan Selenium User Community 日本Seleniumユーザーコミュニティ UI Test Automation • UI Test Automation is so called pinnacle of all the tests
  6. 6. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Why we cannot start test automation • Engineer • Doesn’t have enough time (仕事が忙 しすぎる) • Doesn’t want to perform it manually (マニュアルテスト嫌や) • Doesn’t know how to implement it (なにで作ったら良いか分からん) • Manager doesn’t allow it (したいけど、 マネージャーが。。) • Cost issue • Resource issue • Manager • Request cost so much • Hope to concentrate on business logic more (ビジネス案件に集中したい) • Doesn’t understand what should be done (何したらいいのか分からん) • ROI is not reasonable (コスパが良く ない) • Believe we don’t need UT test
  7. 7. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Why we cannot start test automation • Manager • Request cost so much • Hope to concentrate on business logic more (ビジネス案件に集中したい) • Doesn’t understand what should be done (何したらいいのか分からん) • ROI is not reasonable (コスパが良く ない) • Believe we don’t need UT test Today’s topic is targeting THE BIG MYTH
  8. 8. Japan Selenium User Community 日本Seleniumユーザーコミュニティ 俺を説得してみろ!!
  9. 9. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Many Managers already know about merit of test automation • Benefits • Fast • Programmable • Reliable • Etc. • But they concentrate on only classic ROI
  10. 10. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Classic ROI • Cost managers see • Automation cost • Dev + Maint + Execution + Result Analysis • Manual Test cost • Design + Execution + Maint • ROI Formula they use • (Manual Test cost – Automation cost) / Automation cost • E.g. -500% → NOT reasonable They’re TOO familiar with classic ROI
  11. 11. Japan Selenium User Community 日本Seleniumユーザーコミュニティ What’s wrong? • Test Automation cannot be calculated with classic ROI • Automated Test can be repeatable • Automated Test can be reusable • Classic ROI only consider 1 on 1 comparison between AT and MT • Actual ROI formula will be (Manual Test cost – Automation cost) / Automation cost 222…
  12. 12. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Actual ROI • Cover test cases automatically • Increased productivity (生産性) • Able to restore resource for manual test • Increased test effectiveness (効率) • Consistence, Stability • Fostering team culture for Kaizen(文化作り) • Resolve bottleneck • Engineers more be happy • Reducing Time to market • Able to minimize release pace Actual ROI show the effect beyond expectation
  13. 13. Japan Selenium User Community 日本Seleniumユーザーコミュニティ How to get approval (どうしたらいいか)
  14. 14. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Appeal current bottleneck on organization • Physical bottleneck (物理的) • It takes long time to perform test manually • Time and resources are kept being wasted • The resources used for the test make development plan be tight • Psychological bottleneck (心理的) • Depress engineer’s motivation
  15. 15. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Show your business is already in the red (すでに赤字ということを見せる) • Deficit is arising per test execution (テストの時、赤字が発生している) • Productivity is down (生産性が低くなっている) • Wasted resources (無駄なリソースの消費がある) • “We’re losing minimum XX hours. It’s over XXXXXX JPY when we monetize that” • “We need to consider Re-execution cost for test”
  16. 16. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Show you can minimize automation cost • Able to develop PARALLELLY (リリースが特に必要ない) • Able to develop FAST because it has NO DEPENDENCY with other projects (他プロジェクトの依存性がない) • “I have FRIENDS having same trouble. Selenium User Community!”
  17. 17. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Killer technique (殺し文句) “It’s probable that we cannot realize trouble even if particular part is having trouble” 「今のままだと、トラブっていても、気づけない可能性があります」
  18. 18. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Then • Finally, we can start test automation getting approvals “Okay, please keep trying to improve our system” 「頑張ってください」
  19. 19. Japan Selenium User Community 日本Seleniumユーザーコミュニティ Conclusion • Value automated test has cannot be compared with it of manual test • We cannot see proper return with classic ROI • Managers are familiar with classic ROI • Test Automation can provide us with many merits beyond expectation • Productivity • Kaizen culture • Time to market • Managers think the current status is reasonable • But there is a possibility that it is NOT • Propose optimized ROI for test automation

×