Slides voor de introductie van de cursus Continuous Delivery 3.0 van het Nederlands Instituut voor de Software Industrie. NISI is een spin off van Universiteit Utrecht. www.nisi.nl

1. 1
Continuous Delivery 3.0
Velocity Culture: “Amazon releases software every 11.6
seconds”, Amazon 2011
Session 1: Introduction
2017Q3-Q4
Jan Vlietland, Ewan Klomp, Slinger Jansen

2. Key needs of the Software Industry
• Employees, Employees, Employees
• Continuous Education
• Knowledge for Business and Product Innovation
– Generic software technology
– Generic development tooling
– Generic entrepreneurship knowledge
• Means
– National research and education initiative
– Post-graduate certification curricula
– Joint research programmes

3. Netherlands Institute for the Software Industry
• Initiative Utrecht University:
– prof. S. Brinkkemper
– Dr. S. Jansen
– Dr. J. Vlietland
– G. Lucassen MSc
• Currently three successful courses:
– Software Product Management
– Continuous Delivery 3.0
– Advanced Agile Product Ownership

4. • NISI is a spin-off of Utrecht University
• Mission: advance the software industry in the Netherlands, by
providing – Network – Innovation - Knowledge
• Research agenda:
–Continuous Delivery
–Agile Software Architectures
–Agility & Agile scaling
4
NISI mission

5. 1. NISI Academy
2. NISI Scans & Assessments
3. Knowledge & Skills for the software industry
4. Knowledge Hub towards universities
5
NISI Services

6. Post-graduate certification
• During BSc/MSc in Computer Science/Software Engineering: just 1 or 2
courses on Architecture, Requirements Engineering, Delivery,
Development
• Need for continuous education
• Companies want to invest in personnel
• Personnel certification, e.g. individuals
• Areas: Software Architect; Product Management, Delivery Management,
Software Management, Software Development

7. 7
NISI core team
Dr. Jan Vlietland has a
doctorate in Agile eco-
systems, is co-founder and
head of NISI, and has been
working in the software
industry for over 20 years.
Prof. Sjaak Brinkkemper is
head of the faculty of
Information and Computer
Science and professor at
Utrecht University, and NISI
initiator.
Dr. Slinger Jansen is assistant
professor at the faculty of
Information and Computer
Science of Utrecht University,
and heads research at large
software companies.
Garm Lucassen performs PhD
research about User Stories.

8. Continuous Delivery 3.0
• Introduction to Continuous Delivery 3.0
• Organized by:
• In collaboration with:
– NederlandICT - Software VOC - Scrum.org & ISPMA
• Powered by:
8

9. CD 3.0 lecturer Team
Jan Vlietland Arjan Franzen Maikel Meeuwse Fabiano Dalpiaz
Sjaak Brinkkemper Slinger Jansen Ewan Klomp Garm Lucassen
9

10. Course Objectives
• Help companies to cope with fast moving markets
• Advance participants knowledge about Continuous Delivery
3.0
• Help participants to implement CD 3.0 in their organizations
• Knowledge sharing between Continuous Delivery experts of
different companies
• Certification by NISI
10

11. Certification
• Continuous Delivery 3.0 exam and certification at the end of
the course
• We are currently transferring the Continuous Delivery 3.0
certification to the International Software Product
Management Association (ISPMA)

12. • What would you like to know about Continuous Delivery?
• How do you currently track usage of your software?
• How do you select the work for your developers?
• What do you hope to find in this session/course?
Participants Needs
12

13. 13
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

14. • Continuous Delivery enables fast delivery of new software to the market, with
technology that integrates, tests en distributes software in a fast pace
• But what do you need to have in place to enable such delivery?
• And how to grasp the need of your customers and market?
• How do your select the right features to deliver customer oriented software?
• This course offers answers to these questions!
• We take modern software companies as an example and go in the details of a
Continuous Delivery 3.0 cycle from concept to cash.
14
Market theme

15. 15
Agile/Scrum
Product
Backlog
Scrum
Master
Product
Owner
Development
team
Stakeholders

16. 16
Continuous Delivery 3.0 overview
Continuous
Testing
Continuous
Deployment
Continuous
Integration
Continuous
Planning
Continuous
Tracking

17. Continuous Delivery 3.0 defined
Continuous delivery (CD) is the automation of the software delivery process,
which enables teams to continuously produce software. It aims at building,
testing, and releasing software faster and more frequently.
Continuous delivery 3.0 (CD3.0) is the automation of the full software
engineering cycle, which enables teams to continuously produce software,
track usage and adapt planned engineering. It aims at tracking, planning,
building, testing, and releasing software faster and more frequently.
Sources: https://en.wikipedia.org/wiki/Continuous_delivery

18. Continuous Delivery topics
Commit Build
Check
Code
Quality
Store
in
respository
Unit
testing
Acceptance
testing
Performance
testing
Deploy
on
acceptance
Deploy
on
performance
environment
Deploy
to
production
Continuous
Testing
Continuous
Deployment
Continuous
Integration

19. 19
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

20. 20
Continuous Integration
Continuous Delivery workflow orchestrator
Commit
Build &
Document
generation
Check
Code
Quality
Store
in
respository
Unit
testing
Source
Code
Repository
Build Processor
& Document
Generator
Code Quality
Checker
Unit Test Runner
Component
Repository

21. 21
Continuous Testing
Store
in
respository
Unit
testing
Acceptance
testing
Performance
testing
Deploy
on
acceptance
Deploy
on
performance
environment
Deploy
to
production
Continuous Delivery workflow orchestrator
Unit Test Runner
System
Tests
Acceptance Tests
Performance
Tests

22. 22
Continuous Deployment
Store
in
respository
Unit
testing
Acceptance
testing
Performance
testing
Deploy
on
acceptance
Deploy
on
performance
environment
Deploy
on
production
Continuous Delivery workflow orchestrator
Deploy
on
Acceptance
Deploy
on
Performance
Environment
Deploy for
A/B Testing on
Production
Deploy on
Production

23. 23
Stack Building
Application
Configuration
Infrastructure
Application
Configuration
Host Operating System
Platform (e.g. Docker Engine)
Host Operating System
Bin/Libs Bin/Libs
Hypervisor
Application
Configuration
Application
Configuration
Bin/Libs Bin/Libs
Guest OS Guest OS

24. 24
Questions & Answers
Continuous
Testing
Continuous
Deployment
Continuous
Integration

25. 25
Break & Discussion

26. 26
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

27. 27
Continuous Delivery 3.0 overview
Continuous
Testing
Continuous
Deployment
Continuous
Integration
Continuous
Planning
Continuous
Tracking

28. Continuous
Planning
Continuous
Tracking
Continuous Tracking topics
Data
Collecting
Data
Analyzing
Result
Reporting

29. Data Collection Sources
Online sources
• App usage
• Browser usage
• Social Media
• Email
• Search Engines
• Special Interest Groups
• Press e.g.: news papers, magazines
• Radio & TV
Offline sources
• Human sources, e.g.:
– Surveys
– Focus groups
– Interviews
• Locations: shopping streets, companies
• Physically documented
– Paper
– Magazines
– Books
29
Data
Collecting

30. Basic Browser information
• Cache-Control: max-age=0
• Connection: keep-alive
• Content-Length: 32
• Content-Type: application/x-www-form-urlencoded
• Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
• Accept-Language: en-US,en;q=0.8
• Cookie: tico=vw4rzlru0yryu; pico=vw4rzlru0yryu; tpag=puqnhaw31qrpr; ppag=puqnhaw31qrpr; tcss=3gv2mqiuakctq;
pcss=3gv2mqiuakctq
• Host: www.grc.com Referer: https://www.grc.com/x/ne.dll?rh1dkyd2 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.99 Safari/537.36
• Origin: https://www.grc.com
• Upgrade-Insecure-Requests: 1
• FirstParty: https://www.grc.com
• ThirdParty: https://www.grctech.com
• Secure: https://www.grc.com
• Nonsecure: http://www.grc.com
• Session: i0yalbs05l4ld
Source : https://www.grc.com/x/ne.dll?rh1dkyd2

31. Cross Browser Fingerprinting
• Uses code that instructs browsers to perform a variety of
tasks.
• Those tasks, in turn, draw on operating-system and
hardware resources—including graphics cards, multiple
CPU cores, audio cards, and installed fonts
• Works slightly different for each computer.
• The cross-browser fingerprinting carries out 20 carefully
selected tasks that use the WebGL standard for
rendering 3D graphics in browsers. In all, 36 new
features work independently of a specific browser.
• Over time it builds consistent unique profile
identifications with a 99% success rate
Source : http://yinzhicao.org/TrackingFree/crossbrowsertracking_NDSS17.pdf: Cao, Li & Wijmans (2017)

32. • http://www.mousestats.com/sales/playbackdemo
32
Data Collection demo

33. • Linking data – linking data from various sources
• Correlation Analysis – researches the relationships between variables
• Regression Analysis – researches the causal relationship between one
or more independent variables and a dependent variable
• Monte Carlo – simulation method in which a physical process is
simulated many times, each time with different starting conditions
• Neural networks – analyzing mathematical models with a neural
network, by recognizing patterns and manipulating data
33
Data Analyzing Methods
Data
Analyzing

34. 34
Information Reporting
Result
Reporting
Tools: https://www.adpushup.com/blog/web-analytics-tools-google-analytics-alternatives/

35. Continuous
Planning
Continuous
Tracking
Continuous Planning topics
Data
Collecting
Data
Analyzing
Result
Reporting
Visualization
Agile
Portfolio
Planning

36. 36
Continuous Portfolio Planning
Large
development chunks
More traditional releases?
Sprint
Sprint
D
D
D
Epic
Story
Task
Feature

37. Example
• X-axis: Time
• Y-axis:
– Main items: Theme
– Sub items: Epics
• Cells: Features/stories
• Attributes:
– Available budget
– Deadlines

38. Continuous Planning specials
Decision
making
• Agile business cases
• Managing deadlines
• Supply based planning
• Fast laning (e.g. Incident Handling)
• Agile Scaling automation
38

39. 39
Questions & Answers
Continuous
Testing
Continuous
Deployment
Continuous
Integration

40. 40
Break & Discussion

41. 41
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

42. Continuous Delivery Demonstration

43. 44
Continuous Monitoring Tooling
SCM Build Integrate Quality Store Test DeployPlanTrack Monitor

44. 45
SaaS Continuous Delivery 3.0 Cloud Offering
√ √ √ √ √ √ √ √
√ √ √ √ √ √ √
√ √ √ √ √ √ √
SCM Build Integrate Quality Store Test DeployPlanTrack
Comparison between Cloud Providers:
• https://dataresolution.net/aws-vs-azure-vs-google-comparison/
• Cost calculation: https://calculator.unigma.com/#/instances

45. 46
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

46. • Continuous Delivery 3.0 Maturity Assessment
• Continuous Delivery 3.0 Growth Model
• Value Stream Mapping
• Monitoring, transparency & Feedback
• Engagement factors
47
Continuous Improvement
Continuous
Testing
Continuous
Deployment
Continuous
Integration
Continuous
Planning
Continuous
Tracking

47. Making Improvements
Requirements
5 days
2 : 3 days
Sign Off
10 days
1 : 9 days
Analysis
5 days
4 : 1 days
Code
2 days
2 : 0 days
Test
6 days
1 : 5 days
Review
10 days
1 : 9 days
Deploy
15 days
1 : 14 days
1 day 5 day 4 day 5 day
4 day 3 day40% rejected / Repeat 2x
20% rejected / Repeat 1x

48. Practical Sample Value Stream Map
Source: http://mediad.publicbroadcasting.net/p/kwmu/files/styles/x_large/public/201105/ED_War_Room.JPG

49. 50
Questions & Answers
Continuous
Testing
Continuous
Deployment
Continuous
Integration
Continuous
Planning
Continuous
Tracking

50. 51
Agenda
1. Introduction
2. Continuous Integration, Testing, Deployment
– Questions & Answers
– Break & Discussion
3. Continuous Tracking, Continuous Planning
– Questions & Answers
– Break & Discussion
4. Tooling & Cloud
– Demonstration
– Break & Discussion
5. Continuous Improvement
– Questions & Answers
6. Closure

51. Session Date Topic Teacher
1 06 sep Introductie en CD 3.0 Jan Vlietland, Carlo Kuip-Snelstart
2 13 sep Continuous Integration Sjaak Brinkkemper/Jan Vlietland, Arjan Franzen-Zen Software
3 20 sep Continuous Testing Jan Vlietland / Jeroen Bultje, Ewan Klomp-Universiteit Utrecht
4 27 sep Continuous Deployment Arjan Franzen-Zen Software, Dmitriy Novakovskiy-Google NL
5 04 okt Continuous Tracking Jan Vlietland
6 11 okt Continuous Planning Jan Vlietland, Ewan Klomp-Universiteit Utrecht
7 18 okt Cloud Solutions Jan Vlietland, Gerard van der Pol-Microsoft NL
8 25 okt Continuous Improvement Jan Vlietland
52
Course dates and Topics

52. • Knowledge development
• Knowledge sharing
• Connecting the Software Industry
53
Workgroup Continuous Delivery 3.0

53. Course website
http://nisi.nl/cursussen/continuousdeliverypipelines
You will receive the slides by mail or via the website
54

54. Information
For more information:
• Jan Vlietland
• j.vlietland@nisi.nl
• 06 – 2041 1834
Don’t forget to submit your homework!
www.nisi.nl
info@nisi.nl
+31(0)30 - 268 5398
Copyright © 2017 Netherlands Institute for the Software Industry (NISI) and Utrecht University
55

55. Thank you for your attention!