S E T T I N G U P C O N T I N U O U S
D E L I V E RY C U LT U R E
F O R A L A R G E S C A L E M O B I L E A P P
Naresh Jain
@nashjain
http://nareshjain.com
M O B I L E C D - K E Y C H A L L E N G E S ?
• Code & Config spread across multiple dependent repos -
how to do CI?
• How do you manage access to multiple test phone?
• With lack of quality tests, how do you promote your
artifacts?
• How can we manage different kind of environments, to
support different kinds of test?
• How do we visualise your code quality?
F O R A N E F F E C T I V E C D C U LT U R E
Hypothesis
Validation
Build MVP
Staged Rollout
AB Exp
Monitor
Analyse
Insights
Refine
Measure
Speed
Quality
Commercial Break!
Copyright
© 2012,
Mumbai
Tech Talks!
12
I O S A P P C D C H A L L E N G E S
• Apple takes any where between 3 Days to 1 Week to
push a new update to the app store
• Pretty complicated and cumbersome signing and
packing process (in 2012-13)
• Need to deploy to actual iOS devices to test minor
changes
H O W D I D W E S O LV E T H I S P R O B L E M S ?
• We decided to go down the Hybrid App Route
• Changes/Updates needed to be made at 3 levels:
• Content Changes & Logic Changes => JS/JSON
• Look and Feel Changes + Assets => HTML/CSS/JS
• Infrastructure Changes => Native Code
C L I E N T S I D E A R C H I T E C T U R E
JS
Native
Native iOS, Android and Windows App.
F E AT U R E B R A N C H = > T R U N K B A S E D D E V E L O P M E N T
Build/Release Infra
Code a
Feature
Run Static
Analysis
Run Unit &
Component Tests
Run Functional Tests
for my Feature
Local Dev
Raise a PR
Static Analysis
on Changeset
Run All
Unit Tests
Code
Review
C O N T I N U O U S I N T E G R AT I O N
How do we improve
the quality of the
incoming code?
Automated xUnit Testing
Licensed Under Creative Commons by Naresh Jain
S TAT I C C O D E A N A LY S I S
I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S
I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S
I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S
I M P R O V I N G Q U A L I T Y O F C O D E R E V I E W S
V I S U A L I S I N G C O D E H E A LT H
Build/Release Infra
Build
Mainline
Run Component &
Sanity Tests on
Instrumented Code
Run Workflow Tests on
Instrumented Code
Code a
Feature
Run Static
Analysis
Run Unit &
Component Tests
Run Functional Tests
for my Feature
Local Dev
Raise a PR
Consumable
Certified
Artifact
Merge
Full
Regression
Full Static
Code Analysis
Benchmark, Perf
Tests & Dynamic
Analysis
Nightly Build
Static Analysis
on Changeset
Run All
Unit Tests
Code
Review
C O N T I N U O U S I N T E G R AT I O N
C I F O R M O B I L E A P P V S . S E R V E R
TESTING STRATEGY
Unit
Tests
Integration
Tests
End-to-End
GUI Tests 80-90%
5-15%
1-5%
Manual Checking
Unit Tests
Component / Service (API) Tests
Integration Tests
Acceptance Tests
Workflow
Tests
GUI
Tests
1%
70%
10%
9%
6%
4%
Performance Tests
Security Tests
Epic Level
Story Level
3rd Party/Device Dependencies
Exploratory Testing
% of Tests
jUnit, XCTest, Mockito, OCMock, RoboElectric
jUnit, XCTest (Instrumented/In-Container) Tests
jUnit, XCTest
Cucumber,Appium
Cucumber,
Appium
Cucumber,
Appium
1%
70%
10%
9%
6%
4%
Workflow Tests
Acceptance Tests
Integration Tests
Exploratory Testing
GUI Tests
Component / Service (API) Tests
Unit Tests
Continuous Integration Build Pipeline
Local
Dev Build
Smoke
Build
Functional
Build
Epic
Build
Theme
Build
Product
Build
Perf
Build
Live
Build
Staging
Build
Dev Box
Team CI Server
Team CI Server
Dev Env
SIT Env
QA Env
UAT Env
Staging Env
Prod Env
~5 Min
~10 Min
~30 Min
~60 Min
~60 Min
~60 Min
~30 Min
~30 Min
~30 Min
Update
Checkin
Checkout
Publish
Artifacts
(jars)
RetrievePublishedArtifacts
Retrieve PublishedArtifacts
Publish/Promote DependentArtifacts
Retrieve
Artifacts
Retrieve
Artifacts
Retrieve Published Artifacts
Retrieve Published Artifacts
Checkout
Checkout
Checkout
Checkout
SCM
Each build will update
Task statues
Each build will broadcast build
stats and other reports to all
relevant stakeholders at each
stage of the build pipeline
Dependency
Management
Project/Task
Management
Tool
Version
Control
S E T T I N G U P O U R O W N M O B I L E T E S T L A B
S E T T I N G U P O U R O W N M O B I L E T E S T L A B
D E C E N T R A L I S I N G T E S T E N V I R O N M E N T S
D O C K E R I Z AT I O N
• All Services had to be containerised
• Registration and Discovery of Services (ZooKeeper)
• Mobile Apps need to externalise the configs
• Docker Image Repository (Artifactory)
Licensed Under Creative Commons by Naresh Jain
Questions?
Naresh Jain
@nashjain
naresh@agilefaqs.com
http://blogs.agilefaqs.com

Setting up Continuous Delivery Culture for a Large Scale Mobile App

  • 1.
    S E TT I N G U P C O N T I N U O U S D E L I V E RY C U LT U R E F O R A L A R G E S C A L E M O B I L E A P P Naresh Jain @nashjain http://nareshjain.com
  • 2.
    M O BI L E C D - K E Y C H A L L E N G E S ? • Code & Config spread across multiple dependent repos - how to do CI? • How do you manage access to multiple test phone? • With lack of quality tests, how do you promote your artifacts? • How can we manage different kind of environments, to support different kinds of test? • How do we visualise your code quality?
  • 3.
    F O RA N E F F E C T I V E C D C U LT U R E Hypothesis Validation Build MVP Staged Rollout AB Exp Monitor Analyse Insights Refine Measure Speed Quality
  • 4.
  • 5.
  • 6.
  • 10.
  • 12.
  • 13.
    I O SA P P C D C H A L L E N G E S • Apple takes any where between 3 Days to 1 Week to push a new update to the app store • Pretty complicated and cumbersome signing and packing process (in 2012-13) • Need to deploy to actual iOS devices to test minor changes
  • 14.
    H O WD I D W E S O LV E T H I S P R O B L E M S ? • We decided to go down the Hybrid App Route • Changes/Updates needed to be made at 3 levels: • Content Changes & Logic Changes => JS/JSON • Look and Feel Changes + Assets => HTML/CSS/JS • Infrastructure Changes => Native Code
  • 15.
    C L IE N T S I D E A R C H I T E C T U R E JS Native
  • 20.
    Native iOS, Androidand Windows App.
  • 21.
    F E ATU R E B R A N C H = > T R U N K B A S E D D E V E L O P M E N T
  • 22.
    Build/Release Infra Code a Feature RunStatic Analysis Run Unit & Component Tests Run Functional Tests for my Feature Local Dev Raise a PR Static Analysis on Changeset Run All Unit Tests Code Review C O N T I N U O U S I N T E G R AT I O N
  • 23.
    How do weimprove the quality of the incoming code?
  • 24.
  • 25.
    Licensed Under CreativeCommons by Naresh Jain S TAT I C C O D E A N A LY S I S
  • 26.
    I M PR O V I N G Q U A L I T Y O F C O D E R E V I E W S
  • 27.
    I M PR O V I N G Q U A L I T Y O F C O D E R E V I E W S
  • 28.
    I M PR O V I N G Q U A L I T Y O F C O D E R E V I E W S
  • 29.
    I M PR O V I N G Q U A L I T Y O F C O D E R E V I E W S
  • 30.
    V I SU A L I S I N G C O D E H E A LT H
  • 31.
    Build/Release Infra Build Mainline Run Component& Sanity Tests on Instrumented Code Run Workflow Tests on Instrumented Code Code a Feature Run Static Analysis Run Unit & Component Tests Run Functional Tests for my Feature Local Dev Raise a PR Consumable Certified Artifact Merge Full Regression Full Static Code Analysis Benchmark, Perf Tests & Dynamic Analysis Nightly Build Static Analysis on Changeset Run All Unit Tests Code Review C O N T I N U O U S I N T E G R AT I O N
  • 32.
    C I FO R M O B I L E A P P V S . S E R V E R
  • 33.
  • 34.
    Unit Tests Component /Service (API) Tests Integration Tests Acceptance Tests Workflow Tests GUI Tests 1% 70% 10% 9% 6% 4% Performance Tests Security Tests Epic Level Story Level 3rd Party/Device Dependencies Exploratory Testing % of Tests
  • 35.
    jUnit, XCTest, Mockito,OCMock, RoboElectric jUnit, XCTest (Instrumented/In-Container) Tests jUnit, XCTest Cucumber,Appium Cucumber, Appium Cucumber, Appium 1% 70% 10% 9% 6% 4% Workflow Tests Acceptance Tests Integration Tests Exploratory Testing GUI Tests Component / Service (API) Tests Unit Tests
  • 36.
    Continuous Integration BuildPipeline Local Dev Build Smoke Build Functional Build Epic Build Theme Build Product Build Perf Build Live Build Staging Build Dev Box Team CI Server Team CI Server Dev Env SIT Env QA Env UAT Env Staging Env Prod Env ~5 Min ~10 Min ~30 Min ~60 Min ~60 Min ~60 Min ~30 Min ~30 Min ~30 Min Update Checkin Checkout Publish Artifacts (jars) RetrievePublishedArtifacts Retrieve PublishedArtifacts Publish/Promote DependentArtifacts Retrieve Artifacts Retrieve Artifacts Retrieve Published Artifacts Retrieve Published Artifacts Checkout Checkout Checkout Checkout SCM Each build will update Task statues Each build will broadcast build stats and other reports to all relevant stakeholders at each stage of the build pipeline Dependency Management Project/Task Management Tool Version Control
  • 37.
    S E TT I N G U P O U R O W N M O B I L E T E S T L A B
  • 38.
    S E TT I N G U P O U R O W N M O B I L E T E S T L A B
  • 39.
    D E CE N T R A L I S I N G T E S T E N V I R O N M E N T S
  • 40.
    D O CK E R I Z AT I O N • All Services had to be containerised • Registration and Discovery of Services (ZooKeeper) • Mobile Apps need to externalise the configs • Docker Image Repository (Artifactory)
  • 41.
    Licensed Under CreativeCommons by Naresh Jain Questions? Naresh Jain @nashjain naresh@agilefaqs.com http://blogs.agilefaqs.com