The document describes LinkedIn's approach to releasing updates for their flagship mobile app three times per day (dubbed "3x3") in order to enable faster iteration. Key aspects included distributed and parallel testing across devices and emulators, refactoring tests for speed, creating consistent test environments, static analysis of code, and feature flagging to reduce risk and enable A/B testing of changes. The approach helped reduce release cycles from monthly to daily but further automation of areas like production releases and performance testing was still needed. Distributing workload and leveraging services like TestFlight and Google Play helped scale testing across many devices.
Speakers: Chirayu Krishnappa & Daniel Liem (Uber), Christian Legnitto (Pocketship), Jacek Suliga (LinkedIn), Robbert Van Ginkel & Gautam Korlam (Uber), Rachel Brindle (Pivotal Labs)
Tools for building and releasing mobile applications provided by Google and Apple are geared towards smaller mobile application development teams. However, once development teams start to grow, the requirement from your build and release tooling change drastically. This meetup was all about building & releasing mobile applications at scale: How does your build infrastructure need to evolve to support increasingly complex code-bases? How can you build an automated release pipeline that tracks every build and rolls out your mobile applications safely? What kind of unexpected problems can arise from adopting new technologies when you have a larger team?
Covered in this webinar:
- What is Appium?
- How did Appium become so popular?
- Appium's shortcomings and how they impact business
- Intro to CQ Lab & Perfecto's approach to OSS Frameworks
- How Appium and CQ Lab work seamlessly
- Demo
- Q&A
By the end of this webinar, you'll be chomping at the bit for your next opportunity to work with a customer, Perfecto, and Appium.
Why Appium alone doesn't cut it for enterprises.
Covered in this webinar:
- Intro to Perfecto
- Intro to Appium
- How Perfecto Fits In
- Demos
- Q&A
By the end of this webinar, you'll be chomping at the bit for the next opportunity involving Appium!
Covered in this webinar:
- Perfecto 101
- Core Diffs for Insurance
- Insurance Case Studies
- Open Discussion
By the end of this webinar, you'll be chomping at the bit for your next opportunity to work with an insurance customer and Perfecto.
Speaker:
Amir Rozenberg
Director of Product Management
Perfecto
Have you been staying up to date on Perfecto's biggest product updates?
In this special Mid-Year edition, we will discuss the latest technology in test automation so you can ensure quality and agility for your application teams.
Topic will include:
- Apple’s recent change to XCUITest automation framework with iOS 11
- Fingerprint (touch ID) automation
- New network traffic
- Reporting at scale
- Live Demos
Stay until the end to participate in a live Q&A with our experts!
SwaamTech, is an independent QA and Software Testing company helping clients to bring quality in there products. Contact us for testing of your SmartPhone App testing: support@swaam.com
Speakers: Chirayu Krishnappa & Daniel Liem (Uber), Christian Legnitto (Pocketship), Jacek Suliga (LinkedIn), Robbert Van Ginkel & Gautam Korlam (Uber), Rachel Brindle (Pivotal Labs)
Tools for building and releasing mobile applications provided by Google and Apple are geared towards smaller mobile application development teams. However, once development teams start to grow, the requirement from your build and release tooling change drastically. This meetup was all about building & releasing mobile applications at scale: How does your build infrastructure need to evolve to support increasingly complex code-bases? How can you build an automated release pipeline that tracks every build and rolls out your mobile applications safely? What kind of unexpected problems can arise from adopting new technologies when you have a larger team?
Covered in this webinar:
- What is Appium?
- How did Appium become so popular?
- Appium's shortcomings and how they impact business
- Intro to CQ Lab & Perfecto's approach to OSS Frameworks
- How Appium and CQ Lab work seamlessly
- Demo
- Q&A
By the end of this webinar, you'll be chomping at the bit for your next opportunity to work with a customer, Perfecto, and Appium.
Why Appium alone doesn't cut it for enterprises.
Covered in this webinar:
- Intro to Perfecto
- Intro to Appium
- How Perfecto Fits In
- Demos
- Q&A
By the end of this webinar, you'll be chomping at the bit for the next opportunity involving Appium!
Covered in this webinar:
- Perfecto 101
- Core Diffs for Insurance
- Insurance Case Studies
- Open Discussion
By the end of this webinar, you'll be chomping at the bit for your next opportunity to work with an insurance customer and Perfecto.
Speaker:
Amir Rozenberg
Director of Product Management
Perfecto
Have you been staying up to date on Perfecto's biggest product updates?
In this special Mid-Year edition, we will discuss the latest technology in test automation so you can ensure quality and agility for your application teams.
Topic will include:
- Apple’s recent change to XCUITest automation framework with iOS 11
- Fingerprint (touch ID) automation
- New network traffic
- Reporting at scale
- Live Demos
Stay until the end to participate in a live Q&A with our experts!
SwaamTech, is an independent QA and Software Testing company helping clients to bring quality in there products. Contact us for testing of your SmartPhone App testing: support@swaam.com
The future of mobile apps is looking bright. New features and functionalities are creating better user experiences than ever before. These emerging trends and technologies are changing mobile apps as we know them, and with it, how we develop and test them.
Advanced mobile technologies, such as APKs, PWAs, App Clips, foldable phones, and emerging frameworks like Flutter are now the new normal for mobile. But what does that mean for mobile app development and testing?
In this webinar, led by Perfecto Chief Evangelist, Product Manager, & Author, Eran Kinsbruner, along with Karthik K.K., test automation expert, instructor, and founder of ExecuteAutomation, you will learn more about the changing mobile landscape and what it means for you.
Watch this webinar to:
- Understand the trends changing the mobile app landscape.
- Learn the implications of these emerging trends on development and testing.
- Watch a demo of an enterprise test automation cloud environment.
Learn more at www.perfecto.io.
Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud Applitools
Learn how you can benefit from testing your mobile app with real devices in the cloud, and how to ensure compatibility across devices, while avoiding visual and functional regressions.
Do You Enjoy Espresso in Android App Testing?Bitbar
Watch a live presentation at http://offer.bitbar.com/do-you-enjoy-espresso-in-android-app-testing
Majority of us love coffee but let's put that aside and focus on Espresso - by Google. This exciting new test automation framework just got open sourced and is available for app developers and testers to hammer their app UIs. Espresso has a small, predictable and easy to learn API - built on top of Android Instrumentation Framework - and you can very quickly write concise and reliable Android UI tests with it.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Covered in this Session:
Intro to Perfecto
Automation Testing with Appium and Perfecto
- High Level Overview of Appium & how Perfecto fits in
Mobile Application Types
Understanding the CQ Lab - Architecture/Setup
iOS, Appium and Perfecto
- As of iOS 10, Apple has announced deprecation of its legacy UI Automation framework. Official Apple testing framework for
- iOS 11 is XCUITest framework.
What is Perfecto doing and additional information
Q&A
Cloud testing has never been easier with Perfecto. We’ve rolled out new features and new public cloud packages that make testing in the cloud more accessible than ever before.
Come and see how others are leveraging the platform to scale automation, unite global and remote teams, and release faster. If you’re no longer in the office due to COVID-19, our platform is tailored to help you get through it successfully.
Join this webinar, led by Perfecto’s Director Global Engineering, Nico Kruger, to see the top five use cases for Perfecto’s public cloud, and learn how to successfully automate testing, no matter where you’re currently at.
Attend this webinar to:
- Learn how testing clouds can extend test coverage, accelerate testing, and unite global teams.
- Learn how other teams are addressing app testing challenges to help accelerate their release cycles.
- Get a guided tour of the Perfecto public cloud and watch a live demo.
Testing Techniques for Mobile ApplicationsIndicThreads
With the fantastic growth of mobile computing platforms such as the iPhone, Blackberry,Symbian, J2ME, Windows Mobile and Android environments, there has been a dramatic increase in the value of mobile applications for most companies.
However, one of the biggest challenges that one faces when developing a mobile application is how to test it. Testing Mobile applications is a very intricate and arduous undertaking. There are an enormous number of factors to consider for mobile computing which simply aren’t present for desktop or web development, including hardware/software platforms, installation the application, network type, network strength, memory & battery consumption, external interfacing through WAP and HTTP etc. Additionally, testing the application in simulator, using various debugging tools are some further adventures which the tester undergoes during testing cycles.
With a blend of these challenges as the core of the address, we would be presenting our experience from our product development cycles
Accelerating Digital Transformation With API Lifecycle & Test AutomationPerfecto by Perforce
Today, more than ever, exceptional digital experiences are critical. Many businesses are going entirely online. But ensuring high quality mobile and desktop apps is complicated.
Apps must meet a wide range of functional, nonfunctional, integration, and UX requirements. Your test strategy must cover the entire range of the testing pyramid, include unit, API, and UI/UX. This will provide the best digital experience.
In this webinar, you'll understand how to build a winning digital strategy from design, development, and testing. You'll also learn the following:
- Why APIs matter to the user experience.
- The fundamentals of API testing.
- How Perfecto can provide high quality digital apps.
Learn more at www.perfecto.io.
Automated Testing and Continuous Deployment for Mobile Apps with Jenkins CloudBees
Slides from Silicon Valley Cloud Computing Group Meetup at Yahoo! Campus 09/30/12
Presented by Kohsuke Kawaguchi and Mark Prichard (CloudBees), with Matt Solnit (SOASTA)
Covered in this Session:
- Intro to Perfecto
- Monitoring Discovery Questions
- Monitoring Key Requirements
- Real User Monitoring vs. Synthetic Monitoring
- Steps to Set Up Monitoring
- Demo
- Success Story
- Q&A
How to Test Security and Vulnerability of Your Android and iOS AppsBitbar
Watch a live presentation at http://offer.bitbar.com/how-to-test-security-and-vulnerability-of-your-android-and-ios-apps
Majority of today’s mobile apps consist of third-party code/libraries. This is a prudent and well-accepted development practice that offloads the task of developing code for non-core functions of your mobile app – or game. Identifying third-party code, its vulnerabilities and its license restrictions, is highly critical in order to understand your security exposure and your liability.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Managing Continuous Delivery of Mobile Apps - for the EnterpriseSauce Labs
Enabling CI / CD in your mobile development process means understanding the different solutions, overcoming unique challenges and ensuring the right ownership of the processes. In this webinar, you will learn the steps required to enable Continuous Delivery of Mobile Application Platforms.
Hassle-Free Continuous Integration with Real Device TestingBitbar
Watch a live presentation at http://offer.bitbar.com/hassle-free-continuous-integration-combined-with-a-real-device-testing
Thousands of mobile apps are developed and released every day. The fierce competition for attention of end-users has changed the way mobile developers build and test their apps – and how they utilize today’s mobile devops tools to gain advantage of productivity, effectiveness and speed.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Shifting landscape of mobile automation, and the future of Appium - Jonathan ...Applitools
Full recording here: https://youtu.be/AV8p2aeqsOg
Watch Jonathan Lipps' in-depth overview of the mobile test automation landscape: past, present, and future -- including an in-depth analysis of current frameworks and what's in store for Appium.
5 years ago, mobile automation was in its infancy. None of the tools that enabled testing of mobile apps was very comprehensive, but on the other hand, there were a lot of open source options. Nowadays, the players and the playing field are different, and Appium came to dominate the open source mobile testing scene.
In this talk, expert Jonathan Lipps gives an exposition of the mobile testing landscape. He talks about what writing tests looks like with each of the current tools, and discuss when each might be a good (or bad) choice. In addition, he'll share his reflections on increasingly popular modes of testing beyond functional testing (like visual testing, for example), and what challenges might lie ahead for testers.
Key takeaways from Jonathan's session:
* History of mobile automation
* In-depth overview of the current technology and trends
* Set of factors to use when picking the technology that's right for you
* All about Appium's vision for the future
The future of mobile apps is looking bright. New features and functionalities are creating better user experiences than ever before. These emerging trends and technologies are changing mobile apps as we know them, and with it, how we develop and test them.
Advanced mobile technologies, such as APKs, PWAs, App Clips, foldable phones, and emerging frameworks like Flutter are now the new normal for mobile. But what does that mean for mobile app development and testing?
In this webinar, led by Perfecto Chief Evangelist, Product Manager, & Author, Eran Kinsbruner, along with Karthik K.K., test automation expert, instructor, and founder of ExecuteAutomation, you will learn more about the changing mobile landscape and what it means for you.
Watch this webinar to:
- Understand the trends changing the mobile app landscape.
- Learn the implications of these emerging trends on development and testing.
- Watch a demo of an enterprise test automation cloud environment.
Learn more at www.perfecto.io.
Testdroid: Release Perfect Apps with Mobile Visual Testing in the Cloud Applitools
Learn how you can benefit from testing your mobile app with real devices in the cloud, and how to ensure compatibility across devices, while avoiding visual and functional regressions.
Do You Enjoy Espresso in Android App Testing?Bitbar
Watch a live presentation at http://offer.bitbar.com/do-you-enjoy-espresso-in-android-app-testing
Majority of us love coffee but let's put that aside and focus on Espresso - by Google. This exciting new test automation framework just got open sourced and is available for app developers and testers to hammer their app UIs. Espresso has a small, predictable and easy to learn API - built on top of Android Instrumentation Framework - and you can very quickly write concise and reliable Android UI tests with it.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Covered in this Session:
Intro to Perfecto
Automation Testing with Appium and Perfecto
- High Level Overview of Appium & how Perfecto fits in
Mobile Application Types
Understanding the CQ Lab - Architecture/Setup
iOS, Appium and Perfecto
- As of iOS 10, Apple has announced deprecation of its legacy UI Automation framework. Official Apple testing framework for
- iOS 11 is XCUITest framework.
What is Perfecto doing and additional information
Q&A
Cloud testing has never been easier with Perfecto. We’ve rolled out new features and new public cloud packages that make testing in the cloud more accessible than ever before.
Come and see how others are leveraging the platform to scale automation, unite global and remote teams, and release faster. If you’re no longer in the office due to COVID-19, our platform is tailored to help you get through it successfully.
Join this webinar, led by Perfecto’s Director Global Engineering, Nico Kruger, to see the top five use cases for Perfecto’s public cloud, and learn how to successfully automate testing, no matter where you’re currently at.
Attend this webinar to:
- Learn how testing clouds can extend test coverage, accelerate testing, and unite global teams.
- Learn how other teams are addressing app testing challenges to help accelerate their release cycles.
- Get a guided tour of the Perfecto public cloud and watch a live demo.
Testing Techniques for Mobile ApplicationsIndicThreads
With the fantastic growth of mobile computing platforms such as the iPhone, Blackberry,Symbian, J2ME, Windows Mobile and Android environments, there has been a dramatic increase in the value of mobile applications for most companies.
However, one of the biggest challenges that one faces when developing a mobile application is how to test it. Testing Mobile applications is a very intricate and arduous undertaking. There are an enormous number of factors to consider for mobile computing which simply aren’t present for desktop or web development, including hardware/software platforms, installation the application, network type, network strength, memory & battery consumption, external interfacing through WAP and HTTP etc. Additionally, testing the application in simulator, using various debugging tools are some further adventures which the tester undergoes during testing cycles.
With a blend of these challenges as the core of the address, we would be presenting our experience from our product development cycles
Accelerating Digital Transformation With API Lifecycle & Test AutomationPerfecto by Perforce
Today, more than ever, exceptional digital experiences are critical. Many businesses are going entirely online. But ensuring high quality mobile and desktop apps is complicated.
Apps must meet a wide range of functional, nonfunctional, integration, and UX requirements. Your test strategy must cover the entire range of the testing pyramid, include unit, API, and UI/UX. This will provide the best digital experience.
In this webinar, you'll understand how to build a winning digital strategy from design, development, and testing. You'll also learn the following:
- Why APIs matter to the user experience.
- The fundamentals of API testing.
- How Perfecto can provide high quality digital apps.
Learn more at www.perfecto.io.
Automated Testing and Continuous Deployment for Mobile Apps with Jenkins CloudBees
Slides from Silicon Valley Cloud Computing Group Meetup at Yahoo! Campus 09/30/12
Presented by Kohsuke Kawaguchi and Mark Prichard (CloudBees), with Matt Solnit (SOASTA)
Covered in this Session:
- Intro to Perfecto
- Monitoring Discovery Questions
- Monitoring Key Requirements
- Real User Monitoring vs. Synthetic Monitoring
- Steps to Set Up Monitoring
- Demo
- Success Story
- Q&A
How to Test Security and Vulnerability of Your Android and iOS AppsBitbar
Watch a live presentation at http://offer.bitbar.com/how-to-test-security-and-vulnerability-of-your-android-and-ios-apps
Majority of today’s mobile apps consist of third-party code/libraries. This is a prudent and well-accepted development practice that offloads the task of developing code for non-core functions of your mobile app – or game. Identifying third-party code, its vulnerabilities and its license restrictions, is highly critical in order to understand your security exposure and your liability.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Managing Continuous Delivery of Mobile Apps - for the EnterpriseSauce Labs
Enabling CI / CD in your mobile development process means understanding the different solutions, overcoming unique challenges and ensuring the right ownership of the processes. In this webinar, you will learn the steps required to enable Continuous Delivery of Mobile Application Platforms.
Hassle-Free Continuous Integration with Real Device TestingBitbar
Watch a live presentation at http://offer.bitbar.com/hassle-free-continuous-integration-combined-with-a-real-device-testing
Thousands of mobile apps are developed and released every day. The fierce competition for attention of end-users has changed the way mobile developers build and test their apps – and how they utilize today’s mobile devops tools to gain advantage of productivity, effectiveness and speed.
Stay tuned and join our upcoming webinars at http://bitbar.com/testing/webinars/
Shifting landscape of mobile automation, and the future of Appium - Jonathan ...Applitools
Full recording here: https://youtu.be/AV8p2aeqsOg
Watch Jonathan Lipps' in-depth overview of the mobile test automation landscape: past, present, and future -- including an in-depth analysis of current frameworks and what's in store for Appium.
5 years ago, mobile automation was in its infancy. None of the tools that enabled testing of mobile apps was very comprehensive, but on the other hand, there were a lot of open source options. Nowadays, the players and the playing field are different, and Appium came to dominate the open source mobile testing scene.
In this talk, expert Jonathan Lipps gives an exposition of the mobile testing landscape. He talks about what writing tests looks like with each of the current tools, and discuss when each might be a good (or bad) choice. In addition, he'll share his reflections on increasingly popular modes of testing beyond functional testing (like visual testing, for example), and what challenges might lie ahead for testers.
Key takeaways from Jonathan's session:
* History of mobile automation
* In-depth overview of the current technology and trends
* Set of factors to use when picking the technology that's right for you
* All about Appium's vision for the future
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with AppiumSrijan Technologies
Speaker: Justin Ison
Check out the complete session slides here: http://www.srijan.net/webinar/mobile-...
This session dives into the history of Appium, and it's pros and cons. The speaker also looks at how to write a good test setup and collect meaningful data points. We look at quick demos and comparisons of how Appium significantly reduces test times.
And you definitely should hang around till the Q&A session, where participants pitch in with their issues and queries. The speaker answers all the questions, sharing additional information and tips on Appium.
This is a further continuation or my Selenium Conference lightning talk. In this talk I provide examples of capturing video and logcat data for every test run and attaching to Allure. I also discuss how to leverage cloud test service (Sauce Labs) into your automation framework.
Cloud-powered Continuous Integration and Deployment architectures - Jinesh VariaAmazon Web Services
The presentation will discuss some architectural patterns in continuous integration, deployment and optimization and I will share some of the lessons learned from Amazon.com.
The goal of the presentation is to convince you that if you invest your time where you get the maximum learning from your customers, automate everything else in the cloud (CI + CD + CO), you get fast feedback and will be able to release early, release often and recover quickly from your mistakes. Dynamism of the cloud allows you to increase the speed of your iteration and reduce the cost of mistakes so you can continuously innovate while keeping your cost down.
Velocity Conference: Increasing Speed To Market In Mobile Development Through...Intuit Inc.
In the past few years, Intuit has put an intense focus on ramping up its innovation engine. A big part of this effort is speeding up time-to-market for its hundreds of native mobile apps, while maintaining the highest quality standards. The apps need to work across an ever-increasing number of devices and versions of the operating system.
Once our engineers had access to all possible combinations of devices and operating systems, they needed to be able to kick off automated testing whenever they checked in code. We enable mobile continuous integration by integrating our virtual device lab with our test execution platform and Jenkins system. A notification system detects the arrival of a new build of our mobile products, which kicks off execution of automated tests based on a pre-configured test matrix.
The result is a 95% reduction in the time it takes to test our mobile apps.
My final talk on the Appium mobile grid and getting started.
- Benefits of parallelization.
- The pros and cons of both cloud services and local setup.
- Getting connected devices information.
- Setting up Grid and Appium servers.
- Getting critical metadata to insert into Allure.
- The benefits of distributed tests vs parallel and when best to use them.
- Setup examples for Android and iOS.
- Leveraging cloud services by sending only selected tests to the cloud.
- Reduce service costs.
- Expose your app to a greater amount of devices and os's.
- A demo of Wunderlist's local grid.
Our DevOps Journey
Transforming 6 Month Waterfalls to 1 Hour Code Deploys
https://info.dynatrace.com/17q3_wc_from_agile_to_cloudy_devops_na_registration.html
In the 2nd part of our webinar series, Anita Engleder, DevOps Lead at Dynatrace reviews and dissects lessons learned during the transformational journey moving Dynatrace from an on-prem culture to one that is cloud native. She will lend her perspective as a key member of the team that executed on the original vision: to “implement a new cloud native offering and deploy a new feature release every 2 weeks. Additionally, be able to support a 1-hour lead time from Code Change to Production”.
On November 17th at 1pm/10am PT Anita will present the challenges she and her team faced transforming 6 Months Waterfall to 1 Hour Code Deploys.
In this webinar Anita will discuss:
How to enable a complete cultural shift across multiple teams, in terms of thought process AND execution
What the specific role of her DevOps team is and how it played into the transformation
The role of Feature teams and why continuous feedback is critical for them
How to successfully influence key stakeholders for complete alignment
Today Anita’s team runs 170 production changes every day, running across several AWS Data Centers as well as On-Premise – something that would have been thought impossible only a few years prior.
Speed and quality through Mobile Continuous Integration on Real Devices at Intuit. The presentation share about our key considerations for 3rd-party vs custom built solutions and how we created Virtual Device Lab and test automation framework to enable end-2-end Mobile continuous integration that reduce development iteration cycle from Days to Minutes
The Future of Selenium Testing for Mobile Web and Native Apps
Uber Mobility Meetup: Mobile Testing
1.
2. Evolution of Octopus
Bian Jiang, Nirav Nagda
Uber Mobility Meetup
July 7, 2016
From cross-app/cross device testing, to network
record/replay, to scenario-based testing
3. Soon after we started at Uber, we encountered a fresh
testing challenge. Can you guess what it is?
Our Uber Challenge
5. GTAC 2015
Octopus Takes on the Uber Challenge of
Cross-app/Cross-device Testing
https://www.youtube.com/watch?v=p6gsssppeT0
Uber Eng Blog
https://eng.uber.com/rescued-by-octopus/
6. EndToEnd (L)
Scenario-based
Tests (M)
Unit Tests (S)
Top: small number of large end to
end tests (dozens)
Middle: middling amount of medium
integration tests (hundreds)
Bottom: large number of small unit
tests (thousands)
PREREQUISITES: HEALTHY TEST PYRAMID
8. Record mode:
- Records the network responses
Replay mode:
- Plays back the network responses from local files
HERMETIC TESTS: USING NETWORK RECORD/REPLAY
14. ● What are rules?
● How do they help?
○ Easier test code
○ Shared setup/tearDown code logic
○ Abstract out the complicated logic from tests
UBER RULES
17. ● What is scenario-based testing?
● How does it help?
○ Speeds up the test run
○ Easier to maintain
○ Deterministic tests
SCENARIO-BASED TESTING
18. SCENARIO-BASED TESTING: ANDROID EXAMPLE
Only when
recording
Actual test
public void cancelTripRequest() {
replayScenarioManager.doWhenRecording(new Interactions() {
@Override
public void perform() {
AcceleratorsLayout.verifyLayout().clickItem(0);
ProductSelectionLayout.verifyLayout().swipeToIndex(1)
.clickProduct(R.string.uberx);
ConfirmationLayout.verifyLayout().clickGo();
}
});
// The test will start from here in replay mode.
DispatchingLayout.verifyLayout().clickCancel();
ConfirmationLayout.verifyLayout();
}
19. Special thanks to mobile-test-platform@,
octopus-eng@, mobile-eng@ and all the
amazing people at Uber :)
Uber eng blog
https://eng.uber.com/rescued-by-octopus/
GTAC talk on Octopus
https://www.youtube.com/watch?v=p6gsssppeT0
20.
21. How an "ideal" CI setup for mobile looks like
Valera Zakharov
32. Ideal Test Device
Starts reliably and quickly
Runs as fast as top-of the line phone
Allows you to scale
Provides tests with control over system settings
Test Services
Prevents UI popups
35. Vanilla Setup Problems
Device
Not test friendly
Physical devices
not designed for CI usage
hard to maintain and scale
Emulator:
hard to configure properly
system images not well maintained
37. Google’s Internal Setup
Orchestration
Custom (hooked into the internal build system)
Adb
adb.turbo ™
Device
Stock x86 emulator running in dedicated data-center machines
Properly configured/maintained
Config abstracted away by a script
38.
39. Existing services come up short
Firebase Google Cloud Test Lab
Runs all tests in one instrumentation
Virtual Devices are slow
Not debuggable
Xamarin
Doesn’t run instrumentation tests
AWS
Runs all tests in one instrumentation
41. Some thoughts...
Orchestration
Tools for discovering tests in an apk (based on dexdump)
An alternative to ./gradlew connectedCheck
android_test --apk my.apk --tests test.apk
ADB
adb.turbo (or equivalent) if planning to run at Google-scale
Device
Custom IActivityController
42.
43.
44. Faster Tests with Test Parallelization
JUSTIN MARTIN & DANIEL RIBEIRO
82. Project Voyager
New version of flagship
LinkedIn app
250+ committers across
Android & iOS
~1 year of development
Investment in mobile
infrastructure at
LinkedIn
83. Before Voyager
12 releases per year
RC build + manual regression suite
Mad dash to commit code before the RC cutoff
Missing the cutoff meant a long wait for the next release
Product & marketing plans were made around the monthly releases
Hard to iterate on member feedback
84. 3x3
Release three times per day, no more than three
hours from code commit to member availability
85. Why three hours?
Not enough time for manual testing steps
Not enough time to test everything
The goal isn’t 100% automation, it’s faster iterations
We don’t want engineers spending most of their time maintaining tests that break whenever a
design changes
UI tests are chosen based on production-critical business flows
Faster iteration helps emphasize craftsmanship
Devs can take the extra time to write quality code since the next release is soon
88. Static analysis
Compile-time contract with API server using Rest.li
Rest.li data templates are shared between API server & clients
Provides static analysis checks that guarantee backwards compatibility
Client models are code generated for further safety
Java Checkstyle
Android Lint
Over 200 checks provided by Google
Several custom checks written for LinkedIn-specific patterns
89. Building the code
Over 500k lines of code between Android & iOS
Building production binaries for a large codebase is slow
iOS & Swift
At one point early on, Swift compilation took over two hours
Refactoring into sub-projects and modules lead to a more than 50% speed up
Android Split APKs
Separate binary for each combination of screen density and CPU architecture
Distributed builds
90. What do we test?
Unit tests
Layout tests
Unit tests for views
Stress test views with long strings, short strings
Make sure views don’t overlap, and render properly in right-to-left mode
Scenario tests
Validate that key business metric flows are working properly
Usually flows that span multiple screens in the app
91. Test stability
UI tests use Android Espresso & iOS KIF frameworks
Needed to create a consistent test environment across dev & build machines
Android
Self-contained, standardized Android emulator bundle
Custom test harness that runs one test per application launch
iOS
KIF contained hard-coded waits that passed on dev boxes, but failed on slower build servers
Forked KIF to convert to a checkpoint-based system,
where the code tells the test when to proceed to the next step
92. Test speed
Android
Use Espresso’s IdlingResource API to avoid sleeps and waits
Run up to 16 Android emulators in parallel on a single build machine
Custom test harness allows optimal test parallelization
iOS
Refactoring KIF cut UI testing time by more than 80%
Distributed testing -> Shard tests across multiple machines
Significantly faster, but led to greater exposure to any tooling instability
96. Partner teams
Historically, several partner teams validated the
build before a release
For example, we needed sign off from the
localization team
Lint checks catch hardcoded or improperly
formatted strings
Layout tests catch strings that are too long and
RTL layout bugs
Semantic correctness of translations is still
validated by translators manually
97. Getting to members
Every three hours, internal alpha testers get a new build
Mainly members of the Flagship team
Product managers, devs, and execs who want to see the latest code ASAP
Every week, the rest of the company gets a new beta build
iOS build is submitted to Apple for review
After a week of beta, the build is promoted to production
Assuming Apple’s review is complete, iOS is released
Take advantage of Google Play staged rollout for Android
98. Dogfooding
Android: Google Play alpha/beta channel
Easy upgrades for employees, even while off the corporate network
Somewhat difficult to get set up, but easy once registered
iOS: TestFlight
Nice, but limited number of users
iOS: Custom enterprise distribution
Scales to our number of users, but employees must be on corporate wifi to upgrade
Splash screen in the production app encourages employees to use beta builds
99. Minimizing risk & enabling experiments
Take advantage of LinkedIn’s existing A/B testing infrastructure
New features are developed behind feature flags
Code can be ramped dynamically to different groups of members
Performance of new features or changes can be monitored
Dynamic configuration
Server-controlled kill switch
Crashing or buggy code can often be disabled without a new build
100. 3x3 after 6 months: areas to improve
Release automation
Production uploads to the app stores are still a manual process
Getting release notes & translations is painful
Automated performance testing
We can sample performance of the app in production,
but don’t have a great way of catching issues before release
Android Monkey testing
Enables wide range of API level & device coverage with very low overhead cost
iOS speed improvements
102. 3x3 blogs & videos
3x3: Speeding up mobile releases
3x3: iOS Build Speed and Stability
Test Stability - How We Make UI Tests Stable
UI Automation: Keep it Functional - and Stable!
Consistent Android Testing Environments with Gradle (slides)
Effective Layout Testing Library for iOS
Managing iOS Continuous Integration at Enterprise Scale