Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MAD-LAB - Capabilities And Features

1,963 views

Published on

In this age of a variety of cloud-based-services for virtual Mobile Test Labs, building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it!

The slides are part of the discussion on the Why, What and How I built my own MAD-LAB (Mobile Automation Devices LAB). The discussion also includes the Automation Strategy, Tech Stack, Capabilities & Features of MAD-LAB and the learnings from successful & failed experiments in the journey.

See blog post for more details: https://essenceoftesting.blogspot.com/2018/03/mad-lab-capabilities-features-agileindia2018.html

Published in: Software
  • Be the first to comment

  • Be the first to like this

MAD-LAB - Capabilities And Features

  1. 1. MAD-LAB Mobile Automation Devices LAB Capabilities & Features Anand Bagmar Software Quality Evangelist
  2. 2. About Me @BagmarAnand Blog – essenceoftesting
  3. 3. What do you expect from this session?
  4. 4. Context
  5. 5. Ecosystem Platforms • Android (> v4.4.2) • Unique installs à 4M • Unique # of devices à 13K • iPhones (> v5) • Desktop / WAP Regions • Indonesia • Malaysia • India • Middle-East • ...
  6. 6. Dynamic Content
  7. 7. Search
  8. 8. Download
  9. 9. Offers
  10. 10. ADs
  11. 11. Consumer Engagement is the key to success
  12. 12. Automation Strategy
  13. 13. How would you approach Automation for such a product?
  14. 14. Automation Criteria • E2E tests should focus on Business Rules & User Scenarios • Reports should reflect the above • Understood by all Business & Technical stakeholders • Generated automatically • Rich information • Should run frequently • Every new change in the product-under-test • On a variety of device / OS combinations
  15. 15. Automation Tech Stack • Cucumber-JVM à For specifying Business Rules • Appium à To allow integration with Android / iOS • Gradle / Groovy à As build tool scripting language • TTA - Test Trend Analyzer à Trend & Failure Analysis
  16. 16. MVP / POC
  17. 17. Scale & Repeat!
  18. 18. Run Automation against... Usage patterns from Analytics / PlayStore / App Store • Identify devices that matter • Identify devices that are “automation-friendly” • Run frequently • Every new change in the product-under-test • Jenkins à Continuous Integration Server
  19. 19. Experiment, Learn & Evolve
  20. 20. Experiments • Run tests on Emulators • Use Cloud-based services (Local Vs Remote) • SauceLabs • TestObject • Amazon Device Farm • pCloudy • Devices Video Player Location & Tech-Stack Automation “un-friendly” Falling off! Exploding FailFast!
  21. 21. MAD-LAB
  22. 22. MAD-LAB – Mobile Automation Devices LAB
  23. 23. MAD-LAB – Mobile Automation Devices LAB
  24. 24. Automation Framework
  25. 25. gradle / groovy Manage Devices Wifi Management Optimize Device Utilization Appium Server Management Test Specifications Business Rules (cucumber-jvm) Business Flows Implement Domain functionality (actions, verifications, rules) Resources Entities / Models Pages / Screens Trigger e2e Tests in Jenkins Server - Jenkins File - Node Management Archive artefacts - logs, screenshots, videos, logcat, Appium logs, etc - Cucumber reports ‘x’ Jenkins Agents on ‘y’ Mac Minis Helpers / Utilities (Android / iOS) Driver utilities Loggers File utilities Custom Reporter Screenshot utilities ADB utilities Video capture Analytics GPU Profiling Command Executor Log files from Devices Enrich Test Execution Report Manage Failing Tests Upload to TTA
  26. 26. Lessons Learnt
  27. 27. Learnings (so far) Blue-print is important! • Stepping-stone approach
  28. 28. Learnings (so far) Blue-print is important! • Stepping-stone approach Experiment! • Know your objective(s) •Fail-fast!
  29. 29. Why is Experimentation important?
  30. 30. Challenges are Opportunities to be Creative & Innovative
  31. 31. Challenges solved Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling
  32. 32. Challenges solved Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling
  33. 33. Challenges solved Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling
  34. 34. Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling Challenges solved
  35. 35. Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling Challenges solved
  36. 36. Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling Challenges solved
  37. 37. Challenges solved Jenkins Agents on Mac Minis • Device management • Maximize device utilization • Appium server management • Adb utilities (periodic ADB server disconnects) • Rich Reports (device, build, video, etc.) using cucumber-reports • Analytics Automation • GPU Profiling
  38. 38. Next opportunities • Manage Appium Server better • Better & Intelligent RCA • Functional Performance Benchmarking • Comparing across CI Jobs (for same tests)
  39. 39. Learnings (so far) Blue-print is important! • Stepping-stone approach Experiment! • Know your objective(s) • Fail-fast! Make the Product Testable!!
  40. 40. References • Sample code • https://github.com/anandbagmar/cucumber-jvm-appium-infra • Blog posts • https://essenceoftesting.blogspot.com/search/label/madlab • TTA (Test Trend Analyzer) • https://essenceoftesting.blogspot.com/search/label/tta
  41. 41. Thank you! @BagmarAnand Blog – essenceoftesting

×