This document summarizes mobile testing practices at Gilt. It describes how Gilt started with an informal mobile testing process when the app was simple, but as features and the codebase grew, more structure was needed. Gilt introduced continuous integration, code conventions, and began writing tests to help scale testing and reduce issues. They chose Appium for mobile test automation since it uses the same Selenium infrastructure they had in place and allows testing on both iOS and Android without needing to compile a separate SDK. Page object modeling and explicit waits were implemented to make tests more reliable and representative of the app's state. While challenges remain around overlays and accessibility, writing tests has helped testing scale and reduced pressure on QA as the mobile business grows.

1. Mobile Testing at Gilt
Matt Isaacs
@haveahennessy
Gilt

2. What is Gilt?

3. Luxe brands up to 60% off

4. Inventory Changes Daily
Sales last 36 hours.

5. Sales start at noon ET.
Bonus sales at 9PM, Sundays and Wednesdays.

6. Gilt Mobile
The simplified story

7. iPhone • iPad • Android

8. Developers focus on building the App(s).
QA handles testing - All of it.
No automation.
Informal Process

9. When feature sets were simple.
When the team was small.
When the codebase was small.
No Big Deal

10. – Some App Store Reviews
“Perfect App !!! Well done”
“Awesome app! Love it!”
“OMG! Amazing!”
“Your app is great and keeps getting better."”

11. We added more features.
The codebase grew.
The team grew.
Some time passed

12. 5-stars become harder to get
You can’t please everyone - But still…

13. Informal process doesn't scale

14. Add some structure
CI + Nightlies with Jenkins.
Dogfooding with Hockey.
Code + git branching conventions.

15. And tests…

16. – NSHipster
“Objective-C developers have, for the most
part, remained relatively apathetic to Unit
Testing.”

17. How can this be?
Objective-C is statically typed.
The compiler is awesome.
TDD antipatterns - Cocoa makes heavy use of singletons.

18. ~40% revenue comes from native mobile.
Bad reviews hurt more than ever.
Increased pressure on QA.
Mounting cost of issues

19. So help out QA
Write some tests!

20. Bad habits are hard to break
Make time - Monthly testing hack-days.
Visibility - Make a scene when tests catch issues.
Gentle, but firm approach.

21. Integration Tests
UI + Functional tests.
Notoriously difficult.
Provides the most benefit for QA.

22. Appium
Selenium webdriver for native mobile.
Actively developed.
Open source.

23. Test
Appium
Instruments
App
HTTP
Unix Sockets
DTrace + Magic

24. Accessibility makes it work.
Elements located via accessibility labels.
Be careful with container views.
UIAccessibilityIdentifier is for tests.

25. Why we chose Appium
We’ve already built Selenium infrastructure.
We already have Selenium skills.
No SDK to compile in.

26. – Someone at Gilt
“Selenium is still too flaky. You can't trust the
results. We need more dependable tests"

27. When should I wait?
How long should I wait for?
Waiting

28. Where am I?
Is it where I should be?
When should I be checking this?
State

29. Staging or Production?
Painful issues with either.
Environment

30. Convention on top of Actions and Locators.
Page validated during construction.
Actions return new page objects.
Page Objects

31. When to wait? - Page object construction.
Page objects represent state.
Actions → State transitions
Solved!

32. Sore points.
Accessibility.
Overlays and pass-throughs.
Partially obscured controls.

34. End of the day
You’ll need some process eventually.
It’s never too early - or late.
TDD is not a religion.

35. Thanks
tech.gilt.com
@gilttech
!
gilt.com/careers