Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20180828 QA meetup

233 views

Published on

Our team automation testing activity.
This year theme is to improve automation testing speed.
show issues , solution and result .

Published in: Technology
  • John Buffi is a retired police offer who lost his home to Superstorm Sandy. He now uses the "Demolisher" system to help take care of his 91-year-old father and children. John says: "My only statement is "WOW"...I thought your other systems were special but this is going to turn out to be the " Holy Grail" of all MLB systems, no doubt! ▲▲▲ http://t.cn/A6zP24pL
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

20180828 QA meetup

  1. 1. Improve Automation Testing ~ speed up deployment ~ Aug. 28, 2018 Sadaaki Emura Rakuten, Inc.
  2. 2. 2 Who am I Name : Sadaaki Emura (nickname M) join in Rakuten: 2007 Group: Service Quality Assurance in OSPD established in 2016 Role: Test Engineer Lead Hobby: jog , climbing , horse racing
  3. 3. 3 OSPD organization Product Manager Engineer QA
  4. 4. 4 Our team (QA) Scope : 5 services Members : 8 + off site (if needed) Skill : QA specialist (almost not engineer) Mission : Quality Assurance by manual & automation test
  5. 5. 5 Our Automation Testing activity
  6. 6. 6 Automation testing activity 2016 Proof of concept 2017 Expansion 2018 Improve speed
  7. 7. 7 Why is this year theme “improve speed”? Make impact = release many apps frequently photoAC
  8. 8. 8 What was bottleneck in development speed? Specification Design Development Test (E2E) Development Test Design Test case / script Implement Test Execution scope
  9. 9. 9 What was bottleneck in development speed? 1. Setup test environment takes time 2. Test feedback was late 3. Scripting for PC and SP takes time
  10. 10. 10 1. Setup test environment takes time photoAC What was bottleneck in development speed? Example Install apps (1st time) 120 min Setup devices 5 min Run apps 2 min Closing 5 min :
  11. 11. 11 2. Test feedback was late (E2E) ■One job has Mouse click : 3 sec Input form : 5 sec Page transition : 10 sec : 2-3 mins / page 10 mins / job What was bottleneck in development speed? ■Single process worsens speed Test job Test job Test job 10 mins
  12. 12. 12 For PC For SP What was bottleneck in development speed? 3. Scripting for PC and SP takes time Apps support PC and SP Need to create 2 script
  13. 13. 13 How did we resolve?
  14. 14. 14 Solution to resolve these bottleneck 1. Every time, Everywhere, Everyone can use 2. Many test run at same time 3. One script covers multiple devices Features
  15. 15. 15 Test script management Mobile Labo Architecture Chrom e FF IE iPhon e Andro id Edge Safari CI environment AppiumAPI SeleniumAPI SeleniumAPI Client side Test Environment (Mobile Labo) Clients Windows server MAC server MAC server Test script management Test script management https://jenkins.io/ Java iPhon e Andro id
  16. 16. 16 Mobile Labo image Appium Selenium Mac Windows Smart phone https://qiita.com/emurin Document ▼
  17. 17. 17 1. Every time, Everywhere, Everyone can use per person every time As Is Install apps (1st time) 120 min Setup devices 5 min Run apps 2 min Closing 5 min : To Be Connect API 1 min
  18. 18. 18 2. Many test run at same time FireFox Chrome android iOS iOS One JENKINS runs a couple of test script
  19. 19. 19 3. One script covers multiple devices PC SP Able to run test in each device ※apps rule - same function - same class name (HTML) : example : responsive site
  20. 20. 20 What’s happened?
  21. 21. 21 Improvement effect 1 Example (keiba) Betting Lite (投票Lite) in Horse Racing PC & SP are same function • Step is Choose race, Bet , Confirm , Complete • 9 betting method / 4 type • Selecting order of placing(Max 3rd) ⇒ 736 test case x 2 devices(PC , SP) PC SP
  22. 22. 22 Improvement effect 1 (initial phase) Manual testing Previous automation Mobile labo 120 hour 60 hour Method Speed 30 hour ※1 include scripting , setup task ※2 include scripting task ※1 ※2
  23. 23. 23 Improvement effect 1 (CI phase) Manual testing Previous automation Mobile labo 120 hour 1.5 hour Method Speed 0.5 hour ※1 include setup task ※1
  24. 24. 24 Improvement effect 2 Example (Wedding) Wedding Service is responsive site • Reserve wedding hall • Review wedding hall • Search wedding hall : ⇒ 302 test case x 2 devices(PC , SP) PC SP
  25. 25. 25 Improvement effect 2 (initial phase) Manual testing Previous automation Mobile labo 50 hour 20 hour Method Speed 11 hour ※1 include scripting , setup task ※2 include scripting task ※1 ※2
  26. 26. 26 Improvement effect 2 (CI phase) Manual testing Previous automation Mobile labo 50 hour 0.5 hour Method Speed 0.2 hour ※1 include setup task ※1
  27. 27. 27 Improvement effect Mobile Labo improves speed FREEIMAGES/Geo Okretic
  28. 28. 28 Next challenge
  29. 29. 29 photoAC Next challenge encounter issues
  30. 30. 30 Issue 1 : clash with others iPhon e 1 Andro id Chro me iPhon e 2 iPhon e 1 Andro id Chro me iPhon e 2 clash Able to use devices Not able to use same device at same time
  31. 31. 31 Issue 2 : need to know device location iPhon e 1 Andro id 1 Appium API MAC server 1 iPhon e 2 Andro id 2 iPhon e X Andro id X Appium API MAC server X … Clients set configuration - Appium IP address & port - Capabilities (UDID etc) Where ?
  32. 32. 32 New problem and Next step Improve speed more 1. Clash with others 2. Need to know device location FREEIMAGES/Alexander

×