Rabobank and Perfecto Mobile show how to grow from static waterfall development to an integrated DevOps infrastructure for your mobile SDLC.

1. The journey from agile towards DevOps
April 2, 2015
Kwo Ding, Hardy Seinhorst - Rabobank
Carlo Cadet – Perfecto Mobile
Faster and more stable delivery against
reduced costs

2. Speakers
Hardy Seinhorst Kwo Ding
• Over 20 years of management and
technical experience in IT.
• Experience in application development,
running and optimizing data centers, IT
strategy and information management.
• Rabobank’s program manager for
implementing new digital engagement.
• 8+ years of software testing experience.
• Experience with onsite and offshore team
management for both waterfall and agile
methods.
• Specialized in chain integration (end-to-end)
testing, continuous integration & delivery and
test automation.
Test ManagerDigital Program Manager

3. Introducing the digital portfolio
Current online presence
- Website for desktop and for mobile
- Native app for iOS, Android and Windows Phone
Development
- 1 new hybrid environment that will replace all native apps and the
website for desktop created by 10 scrum teams
- Total of 18 scrum teams, 200 FTE
- Core of a team: Development, PO, QA, Operations
- Extended team: BA, Appl Eng, project mngt
Platform #users # login / month
Desktop 3,6M 18,6M
App iOS 1,2M 30M
App Android 1,2M 30M
App Windows Phone 0,02M 0,3M

4. The waterfall methodology of software development;
success rate as well as fit for use are low
What the Business wants from IT:
The fit for use is also low2:
Project size FTE
Time
(months)
Success
rate
<750K€ 6 6 55%
750K€ to 1.5M 12 9 33%
1.5M to 3M 25 12 25%
3M to 6M 40 18 15%
6M to 10M +250 +24 8%
Over 10M +500 +36 0%
13%
often
always
7%
rarely
19%
never 45%
sometimes
And side effects occur:In reality success rate is low1:
 Long cycle times
 Silo mentality
 Big bang delivery
 Fragile process
 Over-documentation
Stick to the plan seems
to make sense, but…
Shorter time to market
More relevant software
Lower cost to change
Higher quality
Alignment Business and IT
2) Source: CHAOS report 2004: Standish group
1) Source: The Standish Group International Inc. 2002

5. POLL:
What methodology are you using?
1. Waterfall Development
2. Agile Development
3. Continuous Integration
4. Continuous Delivery
5. DevOps
5

6. The journey ahead – Shrinking feedback cycles
6
We are about here in our journey

7. AchievingAgility Drives Serious Change
7
Complete MinimumViable
Gradual ExposureBig Bang

8. Wave 1 Wave 2 Wave 3
Collaboration T-Shaping Collective Ownership/Delivery
Time Boxed “Just In Time” Continuous Delivery
Monitoring/Tools Inception Automation Testing Continuous Deployment
 Coaching and Grooming
 Team Empowerment
 Initiate T-Shaping culture
 Integrated Team - Dev & Ops
 Improved Happiness Score
 Pick up any task outside the
comfort zone
 Integrated Team - DevOps
 Peak Happiness Score
 Shared capabilities, Pick up any task
 Integrated Team - DevOps
 Time Boxed
 Visual Board in place
 Daily Prioritization based on
shared backlog
 Cycle and Lead time
 Non Time Boxed/Continuous
 Real time usage of visual board
 “Just In Time” delivery
 Improved Cycle and Lead time
 Non Time Boxed/Continuous
 Real time usage of visual board
 Lean implementation
 Proactive maintenance
 Integrated tooling
 Application monitoring
 Start with automated testing
 Metrics Integration – holistic view
 Full use of tooling (automated in flow
of tickets)
 Continuous Integration
 Move to fully automated tests
 Business Value measurement
 Full use of tooling (complete use of reports
as well)
 Continuous Deployment
 Fully automated deployment
 Business Value measurement
Dev Ops
PeopleProcessTechnology
Measuring progress

9. Why test automation?
• Tests take less time to run
• Increase # test runs
• Improve accuracy and efficiency
• Increase test coverage
• Necessary for Continous Integration
& Delivery (Agile, DevOps)
Test Automation
Environment
Testing
Deployment

10. Tool Stack
10
Cucumber UFT

11. Test Suite: Automated vs Manual
Automated
tests
Tests not
possible to run
manually
Manual tests
automated
Tests not worth
automating
Manual tests
Manual vs Automated
Source: Dorothy Graham

12. Automation Strategy
12
Technology facing
Business facing
Criticalprojects
Supportprogramming
Functional tests
(system (integration) & chain, 200
portability testing – 10% x 10 devices)
Exploratory testing
User acceptance testing
Accessibility
Visuals (partially)
Unit testing
Unit integration testing
Non-functional testing
(performance, security, production
acceptance testing)
Regression testing is cross-category
Automated
Automated
Manual
Manual/automated

13. Test
Continuously
Continuous Quality
(as-is)
Build
Unit Test
Check-out &
Build application
Security/Perf.
code analysis
Quality code
analysis
Check-in
Check-in code &
review
Deploy
to TST
Deploy to
TST
Smoke test
on TST
Regression
Test
System Test
Deploy
to ACC
Deploy to
ACC
Smoke test
on ACC
Acceptance
Chain Test
Performance
Test
PAT
UAT
Portability
Test
Pre-PAT
Security
Production
Schedule P-
deployment
Deploy to
PROD
PROD
verification
Automated
step
Manual step
Code review
Exploratory
test
3 week
sprint
*
*
Once per Sprint

14. Test
Continuously
Continuous Quality
(to-be)
Build
Unit Test
Check-out &
Build application
Security/Perf.
code analysis
Quality code
analysis
Check-in
Check-in code &
review
Deploy
to TST
Deploy to
TST
Smoke test
on TST
Regression
Test
System Test
Deploy
to ACC
Deploy to
ACC
Smoke test
on ACC
Acceptance
Chain Test
Performance
Test
PAT
UAT
Portability
Test
Pre-PAT
Security
Production
Schedule P-
deployment
Deploy to
PROD
PROD
verification
Automated
step
Manual step
Code review
Exploratory
test
3 week
sprint
Deploy stories when done, not per sprint schedule

15. Recommendations
15
1. Just start, solve issues during the journey
2. Focused change
3. Work closely together with complete team, Behavior
Driven Development (BDD) is a great approach to
improve collaboration.
4. Perform discrete test levels