The methodologies to adopt for successfully
implementing DevOps in your organization
Continuous
everything
Andrea Tino (Software Development Engineer)
Our journey
The agenda for today
A broad overview about
Continuous Integration (CI)
and Continuous Delivery (CD)
methodologies as gateways to
implement DevOps.
What? The IT Industry has moved
towards different technologies
that help Teams adopt CI and
CD in well integrated
frameworks.
How?
DevOps is based on a fixed set
of principles: CI and CD are
today’s recommendation to
achieve them.
Why?
As part of your process and
daily worklife inside your
organization and business.
Where?
As soon as possible :) Changes
take time, but the sooner you
think about it the better.
When?
This is both for startups or
experienced organizations:
no matter what, this is for you!
Who?
Continuous Integration
Continuous Delivery
DevOps consists of 6 key practices.
However two of them are very
important: CI and CD. When both are
successfully implemented, one
organization is already able to reach a
very good adoption level of DevOps.
Our focus
Organizations have been
implmenting CI and CD for quite some
time, we are going to focus on the
most common and effective practices.
Best
practices
We will see that implementing CI and
CD means actually adopting different
tools. We will refer to the most
commonly used in the Industry today.
Tools
Continuous Integration
Definition
noun
The process of automating the build and testing of code every
time a team member commits changes to version control.
Sam Guckenheimer
(Microsoft Visual Studio Cloud Services)
CI encourages developers to share their code
and unit tests by merging their changes into
a shared version control repository after
every small task completion.
Key
benefits
Source Control
CI is based on following Agile
methodologies. Whatever your Agile flavor
is (XP, Scrum, Crystal, etc.), keep it and try
to improve it.
Work
Agile
Use a Source Control (like Git) to share the
code across the departments of your
organization. This will enable you to gain 2
very important achievements:
Share
code
Collaboration: developers can easily
work together while still in isolation.
Branching: Thanks to branches, you
can handle releases very well.
Commit often
Do not pile up stuff and commit large amount of code
changes. Instead, commit small changes.
What
It’s not because we like the feeling of committing stuff...
(well certainly some of us do). It’s more for the
following points:
Why
Risk of regressions: The smaller a change is, the
lower the risk to introduce regressions in the
codebase.
Bug fixing: If a commit introduces a regression, it
is easier to identify that commit. Also, the
regression can be caught up earlier and
addressed faster.
CI architecture
CONTINUOUS INTEGRATION
SOURCE CONTROL
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
CI is built on top of Source Control systems.
Typically, systems like Git will allow
developers to hook up to certain events, like
commits.
By using the Source Control’s API, it is
possible to react to these events and trigger
the build and publish process.
Stacked
CI systems are built to create a chain of
operations which typically involve the most
common steps: code is committed, then
tested and, if tests are green, finally built.
Build
chain
All generated builds are typically saved in a
secure location so they can later be picked
up for deployment. Typically, a cloud
storage is used.
Build
bucket
Don’t re-invent the wheel
Today there are many companies
which have created cloud solutions
for offering integrated CI tools.
They all can interface with the
Many
options
most common source controls (like
Git) and do provide, out of the box,
many tools for collaboration.
So no need to implement CI on
your own, even if your
requirements are very special,
think twice before playing solo :)
By using one of the
many CIs available on
the Internet today, an
organization can
avoid spending so
many resources
building and
maintaining such an
infrastructure. All of
these services have
pros and cons, so pick
the best one for your
needs and scale and
have fun by
automating your
build system.
If your application has
special requirements,
be aware that almost
all of the solutions
available today
provide high levels of
customization via rich
API you can use.
Continuous Delivery
Definition
noun
The process to build, test and release from a build to a
pre-production environment.
Sam Guckenheimer
(Microsoft Visual Studio Cloud Services)
Without Continuous Delivery, software release cycles
were previously a bottleneck for application and
operation teams. Manual processes led to unreliable
releases that produced delays and errors.
Improved
process
CD architecture
SOURCE CONTROL
CONTINUOUS INTEGRATION
CONTINUOUS DELIVERY
CD is built on top of CI. As soon as CI is
over, a new build is available. The build is
published.
Nothing happens to Production
environments, if the build is picked-up for
release, the process of approving it is fully
manual.
On top
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Stage to PPE
Build is deployed into a
pre-production environ-
ment and published for
download.
In order to test the
product E2E, it is useful to
automate the deployment
into a non-production
environment acting as a
safe playground.
Pre
production
PS1T2
Build
A new change is pushed
into the pipeline and the
updated version is built.
Stage to PPE
The build is staged to
PPE into another
separate environment
for E2E testing.
Swap slots
Traffic is redirected to
the new slot leaving the
previous one idle.
Stage to slot
A slot is created to
validate the environ-
ment and run a
warm-up test.
PT1
Build
The first iteration gets
built and executables
are available after
running tests.
Stage to PPE
Bits are deployed into a
test environment for E2E
testing.
Stage to PROD
After validating the
build, bits are deployed
into a newly created slot
in production.
Approval definition
Pipelines can include
deployment stages. Those
can be guarded by manual
approval locks which pause
the pipeline.
Release pipeline
AWS gives developers the
ability to define a task
pipeline with customizable
stages to cover the different
steps for building software.
Slot swapping
Thanks to slots, it is possible
to test a slot before turning it
into production by just
redirecting the production
traffic.
Deployment slots
Microsoft Azure has
introduced staging slots in
order to facilitate the
delivery of new versions of
your application.
Continuous Deployment
Definition
noun
The process to build, test, configure and deploy from a build to a
production environment.
Sam Guckenheimer
(Microsoft Visual Studio Cloud Services)
Though similar, very similar, the definitions of
Continuous Delivery and Continuous Deployment
do actually differs from the very last step. They are
two different practices!
Attention
CDp architecture
SOURCE CONTROL
CONTINUOUS INTEGRATION
CONTINUOUS DELIVERY
CONTINUOUS DEPLOYMENT
CDp is the last step in the release pipeline.
If present, it is built on top of CD.
The foundamental point is that the
process is fully automated. As soon as the
build is emitted and passes all tests,
customers will have immediate access to
it in their Production environments.
Final
stage
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Stage to PPE
Build is deployed into a
pre-production environ-
ment and published for
download.
Stage to PROD
Build is deployed
directly to production
into the customer’s live
environment.
Warning
Continuous Delivery
Continuous Deployment
Always remember that
the following holds:
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Stage to PPE
Build is deployed into a
pre-production environ-
ment and published for
download.
Stage to PROD
Build is deployed
directly to production
into the customer’s live
environment.
Cont. Integration Cont.
Delivery
Deployment
Continuous Delivery
The deployment must wait a
manual approval process to
grant permission.
Continuous Deployment
As soon as the build is
available, the release
definition deploys it to
Production.
Automatically
triggered as
build is over.The way the
deployment is
triggered is the key
point which
differentiates the two
practices.
In CD, deployment to
DevOps
Continuum
production is done
manually over
permission granting.
In CDp this happens
automatically and the
buid goes straight
into Production!
Fast deployment rollback
Everything can go wrong, if it does, a
process for rolling back a deployment to
the last stable build must be in place.
E2E automation
Having E2E tests is important so that
high critical user scenarios are tested.
Code coverage
Good testing is a must. Ensuring all code
paths are also covered is important to
have a good guarantee the product is
high-severity bug-free.
Continuous Deployment is a risky practice
because builds get deployed straight into
production.
An organization must be prepared to
make this step, and some key
requirements can help you understand
whether you are ready to implement CDp:
Be aware
It takes time
Cont. Delivery
Builds are published
and automatically
deployed to PPE.
Cont. Deployment
Customers receive new
deployments
automatically after
every commit.
Cont. Integration
Builds automatically
generated after every
commit.
Agile development
Engineering Teams
work with Agile
methodologies.
Think about it, CDp is cool
but is also pretty scary!
Deploying automatically to
the customer means that
every uncaught bug is
going straight to
production!
Given this high risk, CDp is
a practice that not many
can achieve and should
implement.
The point is, if you wanna
get there, be aware that
this last step in the
continuum takes long time
and much effort.
You
gotta
be
ready
Application Telemetry
Service Monitoring
There are different ways to ask for
feedback from your customers. One is
asking them, but we know it can be
bothersome for users to answer online
questions.
So we have instead our application tell
us about the user experience and how
the application is performing.
Data from
your
application
Today’s cloud systems allow
organizations to deploy solutions fast.
They also provide monitoring tools
out of the box with nice and cool
dashboards to check the state of your
services and notifications to get
alerted when something’s wrong.
Service
health
check
Why should I be so
concerned about
Telemetry?
Q
“You don’t go to the
Cloud without some
good Telemetry. It would
basically be suicide!”
A
A manager I heard in office the year we were
launching our solution to the Cloud
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Emit telemetry
The product emits telemetry
as users work on it. Services
are responsible for storing
telemetry data in the cloud.
Collect telemetry
Periodically, as part of the development
process, Teams collect telemetry and
process the results by aggregating data
and producing actionable output.
Act on telemetry
Basing on results from telemetry,
the next iteration of the product
will contain bug fixes or features
which improve the product.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Stage to PPE
Build is deployed into a
pre-production environ-
ment and published for
download.
Stage to PROD
Build is deployed
directly to production
into the customer’s live
environment.
Customer provides its feedback in a passive way! Thanks
to telemetry, it is possible to log user’s activity.
Feedback
The product undergoes development, testing and
deployment. In this phase, the direction is from Teams
to the customer.
Development
The End
Thank you
This work is licensed under a
Creative Commons
Attribution-NonCommercial-NoDerivatives
4.0 International License
Cover: Continuous Office
This work includes artworks
designed by Freepik.com.
April 2018
v1.0
Keywords
#devops #continuum #agile
#techtalk #microsoft
#continuous-integration
#continuous-delivery
#continuous-deployment
Presentation info
Duration: 30 mins.
Background: Technical
Audience: Developers and
operations
This work includes artworks
designed by Shutterstock.com.
Conference KMD Copenhagen May 2018, hosted by:
Andrea Tino
Software Development Engineer
Twitter:
E-Mail:
@_atino
andrea.tino@microsoft.com

Continuous Everything

  • 1.
    The methodologies toadopt for successfully implementing DevOps in your organization Continuous everything Andrea Tino (Software Development Engineer)
  • 2.
    Our journey The agendafor today A broad overview about Continuous Integration (CI) and Continuous Delivery (CD) methodologies as gateways to implement DevOps. What? The IT Industry has moved towards different technologies that help Teams adopt CI and CD in well integrated frameworks. How? DevOps is based on a fixed set of principles: CI and CD are today’s recommendation to achieve them. Why? As part of your process and daily worklife inside your organization and business. Where? As soon as possible :) Changes take time, but the sooner you think about it the better. When? This is both for startups or experienced organizations: no matter what, this is for you! Who?
  • 3.
    Continuous Integration Continuous Delivery DevOpsconsists of 6 key practices. However two of them are very important: CI and CD. When both are successfully implemented, one organization is already able to reach a very good adoption level of DevOps. Our focus Organizations have been implmenting CI and CD for quite some time, we are going to focus on the most common and effective practices. Best practices We will see that implementing CI and CD means actually adopting different tools. We will refer to the most commonly used in the Industry today. Tools
  • 4.
    Continuous Integration Definition noun The processof automating the build and testing of code every time a team member commits changes to version control. Sam Guckenheimer (Microsoft Visual Studio Cloud Services) CI encourages developers to share their code and unit tests by merging their changes into a shared version control repository after every small task completion. Key benefits
  • 5.
    Source Control CI isbased on following Agile methodologies. Whatever your Agile flavor is (XP, Scrum, Crystal, etc.), keep it and try to improve it. Work Agile Use a Source Control (like Git) to share the code across the departments of your organization. This will enable you to gain 2 very important achievements: Share code Collaboration: developers can easily work together while still in isolation. Branching: Thanks to branches, you can handle releases very well.
  • 6.
    Commit often Do notpile up stuff and commit large amount of code changes. Instead, commit small changes. What It’s not because we like the feeling of committing stuff... (well certainly some of us do). It’s more for the following points: Why Risk of regressions: The smaller a change is, the lower the risk to introduce regressions in the codebase. Bug fixing: If a commit introduces a regression, it is easier to identify that commit. Also, the regression can be caught up earlier and addressed faster.
  • 7.
    CI architecture CONTINUOUS INTEGRATION SOURCECONTROL Commit A change (feature or bug fix) is submitted to the repository for merge into the codebase. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. CI is built on top of Source Control systems. Typically, systems like Git will allow developers to hook up to certain events, like commits. By using the Source Control’s API, it is possible to react to these events and trigger the build and publish process. Stacked CI systems are built to create a chain of operations which typically involve the most common steps: code is committed, then tested and, if tests are green, finally built. Build chain All generated builds are typically saved in a secure location so they can later be picked up for deployment. Typically, a cloud storage is used. Build bucket
  • 8.
    Don’t re-invent thewheel Today there are many companies which have created cloud solutions for offering integrated CI tools. They all can interface with the Many options most common source controls (like Git) and do provide, out of the box, many tools for collaboration. So no need to implement CI on your own, even if your requirements are very special, think twice before playing solo :) By using one of the many CIs available on the Internet today, an organization can avoid spending so many resources building and maintaining such an infrastructure. All of these services have pros and cons, so pick the best one for your needs and scale and have fun by automating your build system. If your application has special requirements, be aware that almost all of the solutions available today provide high levels of customization via rich API you can use.
  • 9.
    Continuous Delivery Definition noun The processto build, test and release from a build to a pre-production environment. Sam Guckenheimer (Microsoft Visual Studio Cloud Services) Without Continuous Delivery, software release cycles were previously a bottleneck for application and operation teams. Manual processes led to unreliable releases that produced delays and errors. Improved process
  • 10.
    CD architecture SOURCE CONTROL CONTINUOUSINTEGRATION CONTINUOUS DELIVERY CD is built on top of CI. As soon as CI is over, a new build is available. The build is published. Nothing happens to Production environments, if the build is picked-up for release, the process of approving it is fully manual. On top Commit A change (feature or bug fix) is submitted to the repository for merge into the codebase. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Stage to PPE Build is deployed into a pre-production environ- ment and published for download. In order to test the product E2E, it is useful to automate the deployment into a non-production environment acting as a safe playground. Pre production
  • 11.
    PS1T2 Build A new changeis pushed into the pipeline and the updated version is built. Stage to PPE The build is staged to PPE into another separate environment for E2E testing. Swap slots Traffic is redirected to the new slot leaving the previous one idle. Stage to slot A slot is created to validate the environ- ment and run a warm-up test. PT1 Build The first iteration gets built and executables are available after running tests. Stage to PPE Bits are deployed into a test environment for E2E testing. Stage to PROD After validating the build, bits are deployed into a newly created slot in production. Approval definition Pipelines can include deployment stages. Those can be guarded by manual approval locks which pause the pipeline. Release pipeline AWS gives developers the ability to define a task pipeline with customizable stages to cover the different steps for building software. Slot swapping Thanks to slots, it is possible to test a slot before turning it into production by just redirecting the production traffic. Deployment slots Microsoft Azure has introduced staging slots in order to facilitate the delivery of new versions of your application.
  • 12.
    Continuous Deployment Definition noun The processto build, test, configure and deploy from a build to a production environment. Sam Guckenheimer (Microsoft Visual Studio Cloud Services) Though similar, very similar, the definitions of Continuous Delivery and Continuous Deployment do actually differs from the very last step. They are two different practices! Attention
  • 13.
    CDp architecture SOURCE CONTROL CONTINUOUSINTEGRATION CONTINUOUS DELIVERY CONTINUOUS DEPLOYMENT CDp is the last step in the release pipeline. If present, it is built on top of CD. The foundamental point is that the process is fully automated. As soon as the build is emitted and passes all tests, customers will have immediate access to it in their Production environments. Final stage Commit A change (feature or bug fix) is submitted to the repository for merge into the codebase. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Stage to PPE Build is deployed into a pre-production environ- ment and published for download. Stage to PROD Build is deployed directly to production into the customer’s live environment.
  • 14.
  • 15.
    Commit A change (featureor bug fix) is submitted to the repository for merge into the codebase. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Stage to PPE Build is deployed into a pre-production environ- ment and published for download. Stage to PROD Build is deployed directly to production into the customer’s live environment. Cont. Integration Cont. Delivery Deployment Continuous Delivery The deployment must wait a manual approval process to grant permission. Continuous Deployment As soon as the build is available, the release definition deploys it to Production. Automatically triggered as build is over.The way the deployment is triggered is the key point which differentiates the two practices. In CD, deployment to DevOps Continuum production is done manually over permission granting. In CDp this happens automatically and the buid goes straight into Production!
  • 16.
    Fast deployment rollback Everythingcan go wrong, if it does, a process for rolling back a deployment to the last stable build must be in place. E2E automation Having E2E tests is important so that high critical user scenarios are tested. Code coverage Good testing is a must. Ensuring all code paths are also covered is important to have a good guarantee the product is high-severity bug-free. Continuous Deployment is a risky practice because builds get deployed straight into production. An organization must be prepared to make this step, and some key requirements can help you understand whether you are ready to implement CDp: Be aware
  • 17.
    It takes time Cont.Delivery Builds are published and automatically deployed to PPE. Cont. Deployment Customers receive new deployments automatically after every commit. Cont. Integration Builds automatically generated after every commit. Agile development Engineering Teams work with Agile methodologies. Think about it, CDp is cool but is also pretty scary! Deploying automatically to the customer means that every uncaught bug is going straight to production! Given this high risk, CDp is a practice that not many can achieve and should implement. The point is, if you wanna get there, be aware that this last step in the continuum takes long time and much effort. You gotta be ready
  • 18.
    Application Telemetry Service Monitoring Thereare different ways to ask for feedback from your customers. One is asking them, but we know it can be bothersome for users to answer online questions. So we have instead our application tell us about the user experience and how the application is performing. Data from your application Today’s cloud systems allow organizations to deploy solutions fast. They also provide monitoring tools out of the box with nice and cool dashboards to check the state of your services and notifications to get alerted when something’s wrong. Service health check
  • 19.
    Why should Ibe so concerned about Telemetry? Q “You don’t go to the Cloud without some good Telemetry. It would basically be suicide!” A A manager I heard in office the year we were launching our solution to the Cloud
  • 20.
    Commit A change (featureor bug fix) is submitted to the repository for merge into the codebase. Emit telemetry The product emits telemetry as users work on it. Services are responsible for storing telemetry data in the cloud. Collect telemetry Periodically, as part of the development process, Teams collect telemetry and process the results by aggregating data and producing actionable output. Act on telemetry Basing on results from telemetry, the next iteration of the product will contain bug fixes or features which improve the product. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Stage to PPE Build is deployed into a pre-production environ- ment and published for download. Stage to PROD Build is deployed directly to production into the customer’s live environment. Customer provides its feedback in a passive way! Thanks to telemetry, it is possible to log user’s activity. Feedback The product undergoes development, testing and deployment. In this phase, the direction is from Teams to the customer. Development
  • 21.
    The End Thank you Thiswork is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License Cover: Continuous Office This work includes artworks designed by Freepik.com. April 2018 v1.0 Keywords #devops #continuum #agile #techtalk #microsoft #continuous-integration #continuous-delivery #continuous-deployment Presentation info Duration: 30 mins. Background: Technical Audience: Developers and operations This work includes artworks designed by Shutterstock.com. Conference KMD Copenhagen May 2018, hosted by: Andrea Tino Software Development Engineer Twitter: E-Mail: @_atino andrea.tino@microsoft.com