Hike is a mobile-first, messaging platform that is used by 100 million users to exchange 40 billion messages/month. Hike app is available on Android, iOS and Windows phone. On the back-end, we’ve 100+ macro-services in Java, Python, Ruby, Go and Elixir. While setting up a Continuous Delivery pipeline, we ran into a series of technical challenges. However it was more important to address the organisational/behavioural challenges to ensure a sustainable culture shift in the company. In this talk, I cover how we went about: * Setup a trunk-based development model * Decentralised our build & test environments using Docker and Jenkins * Segregated and containerised our macro-services * Refactored the mobile apps to be more container friendly * Setup a mobile device farm using STF * Improved the quality of code-reviews using PRBuilder & PRRiskAdvisor * Created different kinds of automated tests to align with our CI Pipeline and get rapid feedback * Finally how we used C3 to visualise the health of our code-base

1. S E T T I N G U P C O N T I N U O U S
D E L I V E RY C U LT U R E
F O R A L A R G E S C A L E M O B I L E A P P
Naresh Jain
@nashjain
http://nareshjain.com

2. M O B I L E C D - K E Y C H A L L E N G E S ?
• Code & Config spread across multiple dependent repos -
how to do CI?
• How do you manage access to multiple test phone?
• With lack of quality tests, how do you promote your
artifacts?
• How can we manage different kind of environments, to
support different kinds of test?
• How do we visualise your code quality?

3. F O R A N E F F E C T I V E C D C U LT U R E
Hypothesis
Validation
Build MVP
Staged Rollout
AB Exp
Monitor
Analyse
Insights
Refine
Measure
Speed
Quality

4. Commercial Break!

5. Copyright
© 2012,

6. Mumbai

10. Tech Talks!

12. 12

13. I O S A P P C D C H A L L E N G E S
• Apple takes any where between 3 Days to 1 Week to
push a new update to the app store
• Pretty complicated and cumbersome signing and
packing process (in 2012-13)
• Need to deploy to actual iOS devices to test minor
changes

14. H O W D I D W E S O LV E T H I S P R O B L E M S ?
• We decided to go down the Hybrid App Route
• Changes/Updates needed to be made at 3 levels:
• Content Changes & Logic Changes => JS/JSON
• Look and Feel Changes + Assets => HTML/CSS/JS
• Infrastructure Changes => Native Code

15. C L I E N T S I D E A R C H I T E C T U R E
JS
Native

20. Native iOS, Android and Windows App.

21. F E AT U R E B R A N C H = > T R U N K B A S E D D E V E L O P M E N T

22. Build/Release Infra
Code a
Feature
Run Static
Analysis
Run Unit &
Component Tests
Run Functional Tests
for my Feature
Local Dev
Raise a PR
Static Analysis
on Changeset
Run All
Unit Tests
Code
Review
C O N T I N U O U S I N T E G R AT I O N

23. How do we improve
the quality of the
incoming code?

24. Automated xUnit Testing

25. Licensed Under Creative Commons by Naresh Jain
S TAT I C C O D E A N A LY S I S

26. I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S

27. I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S

28. I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S

29. I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S

30. V I S U A L I S I N G C O D E H E A LT H

31. Build/Release Infra
Build
Mainline
Run Component &
Sanity Tests on
Instrumented Code
Run Workflow Tests on
Instrumented Code
Code a
Feature
Run Static
Analysis
Run Unit &
Component Tests
Run Functional Tests
for my Feature
Local Dev
Raise a PR
Consumable
Certified
Artifact
Merge
Full
Regression
Full Static
Code Analysis
Benchmark, Perf
Tests & Dynamic
Analysis
Nightly Build
Static Analysis
on Changeset
Run All
Unit Tests
Code
Review
C O N T I N U O U S I N T E G R AT I O N

32. C I F O R M O B I L E A P P V S . S E R V E R

33. TESTING STRATEGY
Unit
Tests
Integration
Tests
End-to-End
GUI Tests 80-90%
5-15%
1-5%
Manual Checking

34. Unit Tests
Component / Service (API) Tests
Integration Tests
Acceptance Tests
Workflow
Tests
GUI
Tests
1%
70%
10%
9%
6%
4%
Performance Tests
Security Tests
Epic Level
Story Level
3rd Party/Device Dependencies
Exploratory Testing
% of Tests

35. jUnit, XCTest, Mockito, OCMock, RoboElectric
jUnit, XCTest (Instrumented/In-Container) Tests
jUnit, XCTest
Cucumber,Appium
Cucumber,
Appium
Cucumber,
Appium
1%
70%
10%
9%
6%
4%
Workflow Tests
Acceptance Tests
Integration Tests
Exploratory Testing
GUI Tests
Component / Service (API) Tests
Unit Tests

36. Continuous Integration Build Pipeline
Local
Dev Build
Smoke
Build
Functional
Build
Epic
Build
Theme
Build
Product
Build
Perf
Build
Live
Build
Staging
Build
Dev Box
Team CI Server
Team CI Server
Dev Env
SIT Env
QA Env
UAT Env
Staging Env
Prod Env
~5 Min
~10 Min
~30 Min
~60 Min
~60 Min
~60 Min
~30 Min
~30 Min
~30 Min
Update
Checkin
Checkout
Publish
Artifacts
(jars)
RetrievePublishedArtifacts
Retrieve PublishedArtifacts
Publish/Promote DependentArtifacts
Retrieve
Artifacts
Retrieve
Artifacts
Retrieve Published Artifacts
Retrieve Published Artifacts
Checkout
Checkout
Checkout
Checkout
SCM
Each build will update
Task statues
Each build will broadcast build
stats and other reports to all
relevant stakeholders at each
stage of the build pipeline
Dependency
Management
Project/Task
Management
Tool
Version
Control

37. S E T T I N G U P O U R O W N M O B I L E T E S T L A B

38. S E T T I N G U P O U R O W N M O B I L E T E S T L A B

39. D E C E N T R A L I S I N G T E S T E N V I R O N M E N T S

40. D O C K E R I Z AT I O N
• All Services had to be containerised
• Registration and Discovery of Services (ZooKeeper)
• Mobile Apps need to externalise the configs
• Docker Image Repository (Artifactory)

41. Licensed Under Creative Commons by Naresh Jain
Questions?
Naresh Jain
@nashjain
naresh@agilefaqs.com
http://blogs.agilefaqs.com