Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Testdroid: Build a Large Scale In-House Test Lab for Mobile Apps

3,229
views

Published on

Watch a live presentation at http://www2.testdroid.com/build-a-large-scale-in-house-test-lab-for-mobile-apps …

Watch a live presentation at http://www2.testdroid.com/build-a-large-scale-in-house-test-lab-for-mobile-apps

Mobile app development relies on agile methodology pretty well. This means companies building their apps and game for Android and iOS are heavily using agile methods in their daily doings, but also in technology that enables them to do daily builds and enable instant testing as part of the development. This webinar focus on in-house test labs and explains how this sort of environment can be built.

Published in: Technology

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,229
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
21
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Building a Large-Scale In-House Test Lab for Mobile Apps 4 June 2014 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com
  • 2. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 2 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 3. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 3 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 4. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 4 Manual vs. Automation Smaller coverage, More money burnt & time wasted, Error- prone Large coverage, quickly completed, Less money & time wasted, Exact results.
  • 5. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 5 Prerequisites • Testing environment as authentic as possible – Real devices <-> emulators/simulators • Test coverage needs to be adequate – Software is not identical in all Android/iOS devices – Hardware is even more fragmented landscape • Surrounding testing conditions (e.g. network) • Try not to build in repetitive, manual processes
  • 6. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 6 In-House Mobile App Testing • Continuous Testing with real devices • Cloud/Out-of-Premises vs. In-House • Improve app ratings & your brand reputation • Operational costs vs. Lost revenue? • Consequences of not-being-tested-enough?
  • 7. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 7 Reasons to Build In-House Labs • Pre-Market devices or other hardware • Corporate security policies • Availability and usage level • Special setups
  • 8. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 8 Core Benefits of CI and Testing • Version control and build revision • Build automation, instant execution & testing • All types of testing & integrations enabled • Frequent commits, Code consolidation -> Fast builds • CI/Testing is the best fit for mobile agile process
  • 9. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 9 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 10. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 10 Why Apps Fail?
  • 11. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 11 EXAMPLE: Android devices Picture by OpenSignal 2013 - http://opensignal.com/reports/fragmentation-2013/
  • 12. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 12 EXAMPLE: Android OEMs Picture by OpenSignal 2013 - http://opensignal.com/reports/fragmentation-2013/
  • 13. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 13 EXAMPLE: Android OS Picture by OpenSignal 2013 - http://opensignal.com/reports/fragmentation-2013/
  • 14. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 14 OS Versions ICS JB KK 17.7M test runs, 288 unique Android devices, 50+ configurations
  • 15. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 15 Display Resolution 17.7M test runs, 288 unique Android devices, 50+ configurations
  • 16. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 16 Memory
  • 17. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 17 Coverage? IS THAT ALL YOU NEED? Picture by Double Encore - http://www.doubleencore.com/2014/05/android-user-data-2014/
  • 18. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 18 Data, results & setup information were gather between 1 January – 31 March 2014 (Q1’2014) - 17.7M test/device runs - 288 unique devices - Thousands of userswww.testdroid.com Research
  • 19. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 19 Global Coverage
  • 20. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 20 Robust/Unstable – Q1’14 The Most Robust Devices Samsung Infuse 4G SGH-I997 Lenovo A60 Dell Streak 7 M02M001 HTC ChaCha A810e Coolpad 7295 Samsung Galaxy Europa GT-I5500 Samsung Acclaim SCH-R880 Motorola Motosmart XT389 Sharp Aquos Phone Zeta SH-02E Samsung Galaxy mini GT-S5570 The Most Unstable Devices Samsung Galaxy S IV Active SGH-I537 Sony Xperia J ST26i Samsung Galaxy S IV Altius SCH-R970 Samsung Galaxy S IV Altius SGH-M919 Samsung Galaxy Note 8.0 GT-N5110 Samsung Galaxy S IV Altius SGH-I337 Samsung Galaxy Mega GT-I9205 HTC One M7 4.3 Asus Transformer Pad TF300TL Sony Xperia Acro S LT26w 99.805 % 99.797% 99.794% 99.785% 99.783% 99.780% 99.778% 99.775% 99.771% 99.770% 29.722% 36.376% 39.007% 41.455% 43.284% 44.737% 45.536% 45.794% 47.689% 50.576 %
  • 21. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 21 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 22. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 22 Hardware – Top 3 Things • Device Control Servers – Hard disks – RAM – Energy Efficiency • USB Hubs • WiFi Infrastructure
  • 23. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 23 Software – Top 3 Things • Testdroid Enterprise • Continuous Integration – e.g. Jenkins CI • Other agile tools
  • 24. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 24 How Testdroid Helps? • Speed up dev & testing cycles significantly • Agile development & integration with mobile practices • Continuous Testing
  • 25. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 25 Simple Setup Device clusters Device Control Server(s) Cluster #1 Cluster #2 Cluster #3 USB Hubs, Cables Cluster #4 Cluster #5
  • 26. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 26 Developers Source code Repository Build failed Successful build QA & Testers Error reporting Test Cases Application Commit Users / Consumers
  • 27. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 27 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 28. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 28 1
  • 29. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 29 2
  • 30. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 30 3
  • 31. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 31 4
  • 32. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 32 5
  • 33. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 33 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 34. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 34 Testdroid Update
  • 35. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 35 Testdroid Update
  • 36. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 36 Stay tuned! A Step-by-step guide how to build an in- house test lab for mobile apps. Will be available at www.testdroid.com Coming Soon!
  • 37. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 37 Deployment Options
  • 38. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 38 Only $999! For Trial Purpose! We’re giving a 30-day trial license with Mac Mini. In worst case, you got a Mac Mini with $999 – and it the best scenario you’ll have a product that will save you a lot of money, time and results as a better application!
  • 39. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 39 Agenda • Intro & Benefits: In-House Mobile App Testing • Challenges w/Apps, Devices, Testing & Market • Infrastructure – both Hardware & Software • Tips & Tricks for Building In-House Lab • Testdroid Update • Q&A Agenda
  • 40. © Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved. 40