The document discusses best practices for mobile continuous integration (CI), emphasizing the automation of build, test, and deployment processes to enhance development efficiency. It highlights the challenges specific to mobile development, such as infrastructure, compilation, testing, and deployment, while comparing simulators, emulators, and physical devices for testing. Additionally, it provides insights into unit and functional testing, showcasing various CI tools and services suitable for mobile applications.

1. Best Practices for Mobile CI
April 28, 2015

2. © Sauce Labs, Inc.
Presenters
Kevin Rohling
Co-founder / VP of Product at Emberlight
Founder/CEO of cisimple now Ship.io
kevin@kevinrohling.com
Kristian Meier
Sales Engineer at Sauce Labs

3. WHAT IS CONTINUOUS INTEGRATION FOR MOBILE?
AUTOMATICALLY…
Build Test Deploy
EVERY COMMIT.

4. CONTINUOUS INTEGRATION
HOW DOES IT WORK
1. Developer commits code

5. CONTINUOUS INTEGRATION
HOW DOES IT WORK
2. CI Server picks up the change.

6. CONTINUOUS INTEGRATION
HOW DOES IT WORK
3. Code is automatically compiled

7. CONTINUOUS INTEGRATION
HOW DOES IT WORK
4. Tests are run

8. CONTINUOUS INTEGRATION
HOW DOES IT WORK
5. Application is deployed

9. CONTINUOUS INTEGRATION
WHY?
• Reduce the need for human-based testing
• Faster Feedback
• Automate EVERYTHING

10. WHY DO I USE CI?
• No Touch Configuration

11. WHY DO I USE CI?
• No Touch Configuration
• Automated OTA Distribution

12. WHY DO I USE CI?
• No Touch Configuration
• Automated OTA Distribution
• Code Validation: Automated Builds and Tests

13. ONE SIZE DOES NOT FIT ALL
Business Drivers Engineering Challenges
Early Startup
Initial Traction,
Funding
Stability, UX
Consulting Firm
Client Status Updates,
Deadlines
Ad Hoc Distribution, Unit Testing
Large Product Team
Retention, Growth,
Revenue
Automation, Communication
PROCESS

14. MOBILE IS HARDER
• Infrastructure

15. MOBILE IS HARDER
• Infrastructure
• Compilation/Code Signing

16. MOBILE IS HARDER
• Infrastructure
• Compilation/Code Signing
• Testing

17. MOBILE IS HARDER
• Infrastructure
• Compilation/Code Signing
• Testing
• Deployment

18. SIMULATORS/EMULATORS vs DEVICES
SIMULATORS:
• Used by iOS, included w/ Xcode
• Execute i386 instruction set, not ARM
• Very fast compared to Emulators
• Do not have access to certain hardware functions
• (GPS, Bluetooth Radio, Accelerometers, etc)

19. SIMULATORS/EMULATORS vs DEVICES
EMULATORS:
• Used by Android, included w/ Android SDKs
• Execute ARM (or native device instruction set)
• Very slow compared to Simulators
• Do not have access to certain hardware functions
• (GPS, Bluetooth Radio, Accelerometers, etc)

20. SIMULATORS/EMULATORS vs DEVICES
DEVICES:
• Reproduces the actual performance experienced by your users
• The only way to catch manufacturer/OEM bugs
• Very expensive to configure and maintain compared
• Full access to hardware functions:
• (GPS, Bluetooth Radio, Accelerometers, etc)

21. WHEN TO USE DEVICES vs SIMULATORS/EMULATORS?
• Simulators and Emulators are an excellent solution for running automated tests
during development. They are inexpensive and will reliably catch most problems.
• Physical Devices can be used on a lower frequency (i.e. pre-release, weekly,
daily, etc.). They are the only way to catch performance problems, test certain
hardware features and find OEM issues. In the least, devices should be used
before every release.

22. UNIT TESTING
According to a recent survey:
-77% of developers said app quality is “very important or mission critical”
-80% of mobile developers test their apps manually
-Only 13% of mobile developers use automated testing

23. UNIT vs FUNCTIONAL TESTING
• Unit Testing - testing small pieces of code
• Functional Testing - testing button clicks and UI interaction

24. UNIT/FUNCTIONAL TESTING
BENEFITS:
• Instant Gratification (GREEN is good!)
• Repeatable
• Can automatically test every commit

25. UNIT/FUNCTIONAL TESTING
CHALLENGES:
• Unit Tests are not users
• LOTs of work to write and maintain them
• Which framework to use?

26. • What language/framework do your developers know?
• Open Source/Community Support
• 3rd Party Framework requirements
UNIT/FUNCTIONAL TESTING FRAMEWORKS
WHICH ONE TO USE?

27. CI TOOLS/SERVICES FOR MOBILE
Good: Open Source, Lots of plugins
Bad: Self-Hosted, DIY solution
Jenkins

28. CI TOOLS/SERVICES FOR MOBILE
Good: Hosted solution, OS X support, Lots of plugins
Bad: Tedious setup process
Travis CI

29. CI TOOLS/SERVICES FOR MOBILE
Good: Hosted solution, Designed specifically for mobile, Easy setup
Bad: Less flexible than other solutions
Ship.io

30. CI TOOLS/SERVICES FOR MOBILE
Good: Integrated w/ Xcode, Apple-Supported
Bad: Self-Hosted, iOS Only
Xcode CI

31. Using the Sauce Labs Cloud with CI

32. Sauce is the ideal Automated Testing Platform for CI

33. © Sauce Labs, Inc.
Appium
• “Selenium for mobile apps” - uses the
WebDriver spec to simulate user actions on
iOS and Android
• Leverage your existing Selenium test
architecture to help speed apps to market
• Maximize test code reuse across platforms
makes developers more productive
• Development led by Sauce Labs
Appium is an open
source framework
that automates
testing of native,
hybrid and mobile
web apps.

34. © Sauce Labs, Inc.
The Sauce Labs Platform
• Point your existing test
scripts to Sauce Labs
• We spin up a new,
pristine VM every time
• Run tests in parallel to
reduce test time
• Capture videos, logs and
test metadata
• Choose from over 500
browsers / OS
combinations

35. Demo

36. © Sauce Labs, Inc.
Learn more about Sauce Labs
• Sign up for a free trial of Sauce
Labshttps://saucelabs.com/signup/trial
• Watch a video
https://www.youtube.com/user/saucelabs