Which One Works You The Best: In-House or Cloud-Based Development Environment

1. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 1 8 January 2015 Ville-Veikko Helppi Technical Product Manager ville-veikko.helppi@bitbar.com Which One Works You The Best: In-House or Cloud-Based Development Environment W E B I N A R

2. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 2 Agenda •  Comparison of In-house and Cloud-based Setups •  Investments in Mobile App Development and Testing Environments •  The Characteristics to Use for Comparison of Both Development Environments •  Calculation Examples •  Testdroid Update •  Q&A

5. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 5 Why Automation? Smaller coverage, More money burnt & ?me wasted, Error-‐prone Large coverage, quickly completed, Less money & ?me wasted, Exact results. Manual Automa@on

6. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 6 •  Why Do You Need an In-House Development and Test Lab? –  Pre-market devices (OEMs, ODMs, integrators ...) –  Prototypes / parts of hardware –  Corporate security policies –  Availability and high usage level –  Special setups (integration w/other equipments) –  Large set of devices available In-house Dev&Test Labs

7. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 7 •  What is Required for an In-House Development & Test Lab? –  Server hardware / infrastructure –  Own mobile devices –  Device control servers •  Hard Disks, RAM, Energy management –  USB hubs –  WiFi / Network infrastructure –  Continuous Integration/Delivery/Testing –  Build systems / other software tools –  POWER! In-house Dev&Test Labs

9. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 9 •  Version control and build revision •  Build automation, instant execution&testing •  All types of testing & integrations enabled •  Frequent commits, Code consolidation -> Fast builds •  CI integrated with development and testing provides the best fit for mobile agile process Core Benefits of CI

10. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 10 •  How to Reach Enterprise-Grade Reliability for In-House Dev&Test Lab – Device management •  Always run on clean devices •  Create intelligent retry mechanisms •  Create fall-back mechanisms – System management •  Automatic reconnect of network and USB •  Automatic configuration changes •  Professional-level monitoring system –  Diskspace, network/connections, Power In-house Dev&Test Labs

13. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 13 •  Someone else is taking care of heavy-lifting •  Cloud-based device cloud includes ALL relevant devices (phones, tablets, cams...) •  You get everything automatically delivered – Data, Results, Logs, Screenshots, Pass/Fail Reports, Additional data-sets •  Diversity and Keeping Up-to-Date with SW – New OS versions and OEM updates •  Easiness of Adoption and Integration Why Cloud?

15. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 15 •  Authentication •  Project Management – Create, Delete, Configure, Managing Users •  Configuring Access and Assets – Configs, Upload/Download Files (App & Tests) – Device Access & Device Group Configuration •  Running Tests – Upload App + Tests <-> Download Results/Data – Notifications, Sharing, Pre-processing of Data Access via Comprehensive API

17. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 17 Investments Aspects •  What is the Most Cost- Efficient Approach? •  What is the Total Cost of Ownership? •  Capital Expenses (CapEx) •  Operation Expenses (OpEx)

18. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 18 •  CapEx is a business expense incurred to create future benefits or to add to the value of an existing asset •  OpEx refers to the expenses that the mobile app developer spends in order to get their day-to- day activities going on and to get their infrastructure fully working CapEx vs. OpEx

23. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 23 •  Corporate Policies •  Regulation •  Minimizing security breaches •  Security/Protection mechanisms •  Monitoring •  Authentication & User Access Security1

24. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 24 •  Needs to be fully configurable •  Back-ups & Retention •  Preventing non- authorized use and access •  Needs to available 24/7 •  Transferrable nature of data, source code, results Data Ownership and Retention2 In-house vs. Cloud

25. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 25 •  API access – no delays •  Critical/Medium/Low •  DevOps in crucial role •  Various data center locations •  Hardware failures? Monitoring? Scale up? •  Average Cloud Up-time: 99.5% Performance and Availability3

26. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 26 •  Support: Email, Phone, Online service systems •  Intuitive online help: Manuals, How-Tos, Instructions (self-service) •  Integration with bug- tracking systems (e.g. JIRA) •  Frequent product updates •  Needs to be available 24/7 Support4

27. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 27 •  Clear policies on updates, maintenance, monitoring •  Terms of Service + EULAs •  Manual vs. Automated enhancements/updates •  Customized features •  Product roadmaps Enhancements and Update Policy5

28. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 28 •  Agile process enables agile integrations •  Tools, Frameworks, Other complementary software •  API •  Avoid lock-ins to proprietary frameworks and other SW! •  Can define cost-efficiency of using either approach Integrations6

29. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 29 •  Intuitiveness, Easy-to- Use, Good Help Materials •  Directly related to TCO •  Live demos and free trials with the system •  Frequent webinars, Q&A, "How-To" materials Usability7

30. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 30 •  Lock-ins with in-house environments •  Term or Perpetual licensing model •  Return-on- Investment? •  Subscription models – optimal use of service Contractual Flexibility8

31. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 31 •  Own devices vs. devices from cloud farm? •  Scalability with server infrastructure •  Device control servers – Linux with Android = 15 – iOS with Mac = 3 •  Easiest scalability comes w/privately hosted cloud Equipment Scalability9

35. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 35 •  Different levels of control and abstraction •  Definitions aren't always clear – so make sure you understand what you are buying! SaaS-PaaS-IaaS vs. In-house

36. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 36 •  Assumption – You are building several apps •  No. of apps is not limited here, as all example infrastructures can handle large number of apps – You use 50 different devices for testing – You need everything up and running and someone maintaining the system 24/5 – OR – just business hours Basic Example

37. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 37 •  In-house Setup –  Devices: ($200 x 50 = $10,000) –  Server infrastructure: ($1,000 per 5 devices -> $10,000) –  Software: ($99/month x 50 = $4,950/mo -> $59,400/year) –  Setup/deployment: 3 biz days ($50/hour x 8 x 3 = $1,200) –  DevOps to manage the system 24/5: ($50/hour x 24 x 5 = $6,000/week x 50 weeks = $300,000) OR ALTERNATIVELY –  DevOps to manage the system on business hours: ($50 x 8 x 5 = $2,000/week x 50 weeks = $100,000) •  Majority of costs comes from OpEX In-House Dev & Test Lab

38. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 38 •  Public Cloud Setup –  Deployment: 1 business day ($50 x 8 = $400) –  Integration: 2 business days ($50 x 8 x 2 = $800) –  License for 50 devices: ($1499/mo -> $17,988/year) •  Private Cloud Setup –  Deployment: 1 business day ($50 x 8 = $400) –  Integration: 2 business days ($50 x 8 x 2 = $800) –  License for 50 devices: ($199 x 50 -> $9,950/month -> $119,400/year) •  Majority of costs comes from CapEx Cloud-Based Services

39. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 39 •  Head-to-Head: – In-house (24/5) costs $380KUSD vs. Private Cloud $120KUSD •  Privately hosted cloud is 31.5% of in-house system costs Cost Comparison Comparable environments: 216% Significant difference in costs between in-house and cloud

41. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 41 Testdroid PrivateCloud - Running user-selected devices at any of our hosting facilities, more cost-efficiently than your own devices!

42. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 42 Option 1: Public Device Cloud on-demand devices (multitenant) Mobile app testing on over 400+ real Android and iOS devices hosted by Bitbar Option 2: Private Device Cloud reserved devices Hosted by Bitbar in the US and/or Europe Devices chosen by and reserved only for the Customer Option 3: On-premise Device Cloud Automated mobile app testing on real Android and iOS devices hosted by the customer, usually 30-500 devices Testdroid Deployment Options

43. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 43 •  Get to know the essential facts in terms of selecting cloud-based or in-house solution for your needs: – Characteristics – Costs vs. Benefits – Calculation examples •  Available at www.testdroid.com/library New EBOOK Available!

45. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 45 Video recording & slides of this webinar will be soon available at Vimeo and Slideshare! Visit at www.testdroid.com for more information. THANK YOU! W E B I N A R

46. © Copyrights by Bitbar Technologies Ltd. 2015 All rights reserved. 46 We operate the largest global device cloud with over 400 devices instantly available for developers Devices from the US, Europe, China, Japan and Korea – global devices for global customers