1
DevOps Overview
Subhendu Karmakar
What is DevOps?
2
• DevOps is a Software Development approach which involves Continuous Development, Continuous Testing, Continuous Integration, Continuous
Deployment and Continuous Monitoring.
• DevOps is a culture, method and practice shift that supports the acceleration of software delivery through Development, Testing and Production
deployment.
DevOps Goals DevOps Benefits
• Faster time to market
• Lower failure rate of new releases
• Shortened lead time between fixes
• Faster mean time to recovery
• Improved customer satisfaction
• Better product quality
• Improved productivity and efficiency
DevOps = Development + Operations
• “Dev” is used as a shorthand for developers in particular but in
practice it means all stakeholders involved in developing the
product: Developer, QA, BA etc.
• “Ops” is a shorthand of ‘Operations’. Operations team includes
system admin, DBA, operations stuff, network engineer, security
professional etc. These people are responsible for deploying the
product in an environment.
Evolution of Software Development process
3
Traditional Waterfall
Model
•Completed requirements
are clear and fixed.
•Product definition is stable
Agile Development
•Requirements change frequently
•Development and testing needs to
be fast
DevOps Approach
•Requirements change frequently
•Development needs to be agile
•Operations needs to be agile
Pre-DevOps Software Development & Deployment
4
• Development and Operations teams used to work in silos.
• Less collaboration or communications causes overhead for both teams.
• Pre-DevOps process used to take more time to send product into market.
• More failure rate in production and more time taken for recovery.
• Developers always look for
changes
• Developers try to implement
every new technique
introduced
Change
Change
Change
• Operations people do
not like changes
• Change is not reliable
and leads to instability
for Operations people
Development Operations
Pre-DevOps approach in SDLC
Why DevOps? – Scenario 1
5
• Perform coding based on requirements received
from business.
• Send out the artifacts to Operations for
deployment.
• Manually downloads the scripts.
• Make required changes in multiple configuration
files manually to reflect changes in Production
environment.
Old process of Software Development Old process of Software Deployment
In case of any issues after deployment, developers
blame Operations team that deployment was not proper.
In case of any issues after deployment, Operations team
blame developers that developers gave faulty artifacts.
Loss of Work
to find out
issues
• DevOps improves the software development and deployment process by getting Development and Operations team worked together collaboratively and collectively.
• DevOps reduces or eliminates any rework and reduces overhead by making software development and deployment process automated.
Development Operations
Why DevOps? – Scenario 2 (Facebook example)
6
Challenges faced by Facebook:
• Features released to 500 millions users that caused heavy website traffic.
• Due to heavy website traffic server got meltdown.
• Mixed feedbacks received from users which lead to no conclusion whether Facebook should keep the features or no.
Feature launched in 2011
Music, Timeline, Ticker
Worldwide deployment
After this issue Facebook came up with Dark Launching
technique:
• New features are first deployed on a smaller & specific user
base.
• They are continuously monitored and feedbacks are
continuously developed and tested.
• Once features are stable they are deployed on other user bases
in multiple releases.
• To implement Dark Launching technique, below activities are
fundamental as they lie at the heart of DevOps lifecycle:
• Continuous Development
• Continuous Testing
• Continuous Integration
• Continuous Deployment
• Continuous Monitoring
How DevOps works
7
Continuous
Deployment
Continuous
Feedback
• New Requirement
• Defect fix
• Enhancement
Production
• Deployments are more frequent in DevOps approach
• Feedbacks are received frequently from testers or actual customers
• Continuous Deployment and continuous Feedback makes software product more reliable
• Automation of overall process reduces time to market of a software
DevOps proposes two improvements:
1. Improve software product based on the feedback received from customer
2. Improve the software delivery process
Following are different phases in software development lifecycle that occurs continuously
for every small changes in software:
• Coding
• Build
• Testing
• Release
• Deployment
• Operation
• Monitoring
• Planning
How DevOps works (cont..)
8
Instead of seeing two group “Dev” and “Ops” in silos, DevOps proposes to work them together in same team and use both skills and tools to automate overall
development lifecycle.
DevOps Cycle
• An ideal DevOps cycle comprises of following activities:
• Writing code
• Building & deploying of binaries on an environment
• Executing test cases
• Deploying on to Production in one smooth integrated flow.
• Most important features of this approach are automation of build,
deployment and testing.
• Use of Continuous Integration (CI) tools, automation testing tools
have become a standard in a DevOps cycle.
How DevOps impacts Testing
9
• QA takes a critical role in today’s organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not.
• This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs.
• Today QA groups are charged with preventing defects from reaching the public site.
• QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the software
product but also in the process, and recommending changes wherever they can.
• In today’s world there are several companies that are publishing new release every day (or every hour), there is no room for manual testing. There automation test suite must be developed for
functional testing, that can ensure quality standards are maintained.
• Anything that can be automated, should be automated.
• Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.
• Beyond Functional Testing, Automation for Load Testing, Stress Testing, and Performance Testing.
DevOps Tools
10
Popular DevOps tool:
Git (GitHub): This is a SCM (Source Control Management) tool. The purpose
of Git is to manage a project, or a set of files, as they change over time.
Git stores these information in a data structure.
JIRA: It is used for bug tracking, issue tracking, and project management related
to software and mobile apps.
Eclipse: This is an Integrated Development Environment (IDE) used for
programming.
Maven: Apache Maven is a software project management and comprehension
tool. Based on the concept of a project object model (POM), Maven can manage a
project's build, reporting and documentation from a central piece of information.
Selenium: Selenium is a portable software testing framework for web
applications.
Cucumber: This is a tool used by computer programmers for testing other
software. It runs automated acceptance tests written in a behavior-driven
development (BDD) style.
Jenkins: Jenkins is a powerful application that allows continuous integration and
continuous delivery of projects, regardless of the platform.
Bamboo: Bamboo is a continuous integration and deployment tool that ties
automated builds and tests.
Splunk: This is a software for monitoring, and analyzing machine-generated big
data. It captures, indexes, and correlates real-time data in a searchable repository
from which it can generate graphs, reports, alerts, dashboards, and visualizations.
SaltStack: This is an infrastructure and application management software package that
automates the packaging and provisioning of code.
Puppet: This is a configuration management tool that is used for deploying, configuring,
and managing a server. It can automatically configure newly provisioned systems to
create a specific infrastructure.
Chef: This is a configuration management tool for dealing with machine setup on
physical servers, virtual machines and in the cloud.
Continuous Delivery (CD)
11
• Continuous Delivery is a disciplined approach of developing software that emphasizes quality at each phase of delivery pipeline that will enable automated deployment at any environment.
Continuous Delivery consists of four main components:
• Source Code Management (SCM)
• Continuous Integration (CI)
• Continuous Test (CT)
• Automated Deployment
Why Continuous Delivery?
• Working software is delivered frequently to business partner in small pieces of functionalities
• Releases are scheduled based on input and needs from business partners
• As the process matures the confidence of automation increased and ultimately it will reduce delivery time, defects, manual efforts and human errors.
• become more agile by having capability to identify risks, issues or changes in requirements earlier of development lifecycles.
High-level Overview of CD process in major organizations:
• Ensure application has been properly On-Boarded to Automated Deployment tool. E.g: Application Infrastructure Model (AIM).
• Complete Application code development and test script development for functional and integration tests.
• Check in code to approved source control repository: Git
• After each check-in Continuous Integration tool Jenkins automatically kicks off the build process and loads artifacts in FRS.
• Once tests are passed, Automated Deployment is done to higher environment.
Continuous Integration (CI)
12
• Continuous Integration (CI) is a development practice that requires developers to integrate
code into a shared repository several times a day. Each check-in is then verified by an
automated build, allowing teams to detect problems early.
• This is a development practice that calls upon development teams to ensure that a build and
subsequent testing is conducted for every code change made to a software program.
• This concept was meant to remove the problem of finding late occurrences of issues in the
build lifecycle.
• Instead of the developers working in isolation and not integrating enough, Continuous
Integration was introduced to ensure that the code changes and builds were never done in
isolation.
All source codes are kept in a repository system
(Git/SVN)
Developer checks in new code into source code
repository system (Git/SVN)
Continuous Integration server will pick up the
software and starts building the new project
(Jenkins)
If build is a failure team will be notified
accordingly. If success it will be tested for any
defects (Jenkins)
If test fails developers are notified. If test
passes the build is ready to be deployed to
higher environment. (Jenkins)
Why Continuous Integration?
The continuous Integration process helps to answer the following questions for the software
development team.
• Do all the software components work together as they should?
• Is the code too complex for integration purposes?
• Does the code adhere to the established coding standards?
• How much code is covered by automated tests?
• Were all the tests successful after the latest change?
Preferred CI tool in major organizations: Jenkins
• Jenkins provides a container for automated build and testing to support CI behavior.
• A server based system running in a servlet container Apache-Tomcat.
• Supports SCM tool like Subversion (SVN) or Git. It can execute Apache Maven based projects; shell
scripts or windows batch commands.
CI Workflow:
Continuous Testing and Automated Deployment
13
Continuous Testing (CT): Continuous testing is the process of executing automated
tests as part of the software delivery pipeline to obtain immediate feedback on the
business risks associated with a software release candidate.
Why CT?
• Traditional testing processes are too slow
• Huge testing timelines are preventing from delivering quality software at the
desired speed.
• The goal of continuous testing is to provide fast and continuous feedback in the
latest build candidate.
• Since testing begins early and is executed continuously, application risks are
exposed soon after they are introduced
• Development teams can prevent issues from progressing to the next stage of
the SDLC
Automated Deployment: Deployment automation allows applications to be deployed across
the various environments used in the development process, as well as the final production
environments. This results in more efficient, reliable, and predictable deployments.
Benefits of Deployment Automation:
• Repeatability
• Fewer errors
• Anyone can deploy
• Work on what matters
• Customer satisfaction
• Lower costs
Test Automation Code
Development Code
Git
Jenkins
Artifact Repo
Commands to
deploy code
Code commit
Build triggered
No
Dev team gets notified to fix code
Artifacts
stored
Automated deployment
of Development and
Test Code
Code deployed
Test Environment
Test
Successful?
Higher Environment Yes
14
Thank You !

DevOps Overview in my own words

  • 1.
  • 2.
    What is DevOps? 2 •DevOps is a Software Development approach which involves Continuous Development, Continuous Testing, Continuous Integration, Continuous Deployment and Continuous Monitoring. • DevOps is a culture, method and practice shift that supports the acceleration of software delivery through Development, Testing and Production deployment. DevOps Goals DevOps Benefits • Faster time to market • Lower failure rate of new releases • Shortened lead time between fixes • Faster mean time to recovery • Improved customer satisfaction • Better product quality • Improved productivity and efficiency DevOps = Development + Operations • “Dev” is used as a shorthand for developers in particular but in practice it means all stakeholders involved in developing the product: Developer, QA, BA etc. • “Ops” is a shorthand of ‘Operations’. Operations team includes system admin, DBA, operations stuff, network engineer, security professional etc. These people are responsible for deploying the product in an environment.
  • 3.
    Evolution of SoftwareDevelopment process 3 Traditional Waterfall Model •Completed requirements are clear and fixed. •Product definition is stable Agile Development •Requirements change frequently •Development and testing needs to be fast DevOps Approach •Requirements change frequently •Development needs to be agile •Operations needs to be agile
  • 4.
    Pre-DevOps Software Development& Deployment 4 • Development and Operations teams used to work in silos. • Less collaboration or communications causes overhead for both teams. • Pre-DevOps process used to take more time to send product into market. • More failure rate in production and more time taken for recovery. • Developers always look for changes • Developers try to implement every new technique introduced Change Change Change • Operations people do not like changes • Change is not reliable and leads to instability for Operations people Development Operations Pre-DevOps approach in SDLC
  • 5.
    Why DevOps? –Scenario 1 5 • Perform coding based on requirements received from business. • Send out the artifacts to Operations for deployment. • Manually downloads the scripts. • Make required changes in multiple configuration files manually to reflect changes in Production environment. Old process of Software Development Old process of Software Deployment In case of any issues after deployment, developers blame Operations team that deployment was not proper. In case of any issues after deployment, Operations team blame developers that developers gave faulty artifacts. Loss of Work to find out issues • DevOps improves the software development and deployment process by getting Development and Operations team worked together collaboratively and collectively. • DevOps reduces or eliminates any rework and reduces overhead by making software development and deployment process automated. Development Operations
  • 6.
    Why DevOps? –Scenario 2 (Facebook example) 6 Challenges faced by Facebook: • Features released to 500 millions users that caused heavy website traffic. • Due to heavy website traffic server got meltdown. • Mixed feedbacks received from users which lead to no conclusion whether Facebook should keep the features or no. Feature launched in 2011 Music, Timeline, Ticker Worldwide deployment After this issue Facebook came up with Dark Launching technique: • New features are first deployed on a smaller & specific user base. • They are continuously monitored and feedbacks are continuously developed and tested. • Once features are stable they are deployed on other user bases in multiple releases. • To implement Dark Launching technique, below activities are fundamental as they lie at the heart of DevOps lifecycle: • Continuous Development • Continuous Testing • Continuous Integration • Continuous Deployment • Continuous Monitoring
  • 7.
    How DevOps works 7 Continuous Deployment Continuous Feedback •New Requirement • Defect fix • Enhancement Production • Deployments are more frequent in DevOps approach • Feedbacks are received frequently from testers or actual customers • Continuous Deployment and continuous Feedback makes software product more reliable • Automation of overall process reduces time to market of a software DevOps proposes two improvements: 1. Improve software product based on the feedback received from customer 2. Improve the software delivery process Following are different phases in software development lifecycle that occurs continuously for every small changes in software: • Coding • Build • Testing • Release • Deployment • Operation • Monitoring • Planning
  • 8.
    How DevOps works(cont..) 8 Instead of seeing two group “Dev” and “Ops” in silos, DevOps proposes to work them together in same team and use both skills and tools to automate overall development lifecycle. DevOps Cycle • An ideal DevOps cycle comprises of following activities: • Writing code • Building & deploying of binaries on an environment • Executing test cases • Deploying on to Production in one smooth integrated flow. • Most important features of this approach are automation of build, deployment and testing. • Use of Continuous Integration (CI) tools, automation testing tools have become a standard in a DevOps cycle.
  • 9.
    How DevOps impactsTesting 9 • QA takes a critical role in today’s organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. • This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. • Today QA groups are charged with preventing defects from reaching the public site. • QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the software product but also in the process, and recommending changes wherever they can. • In today’s world there are several companies that are publishing new release every day (or every hour), there is no room for manual testing. There automation test suite must be developed for functional testing, that can ensure quality standards are maintained. • Anything that can be automated, should be automated. • Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability. • Beyond Functional Testing, Automation for Load Testing, Stress Testing, and Performance Testing.
  • 10.
    DevOps Tools 10 Popular DevOpstool: Git (GitHub): This is a SCM (Source Control Management) tool. The purpose of Git is to manage a project, or a set of files, as they change over time. Git stores these information in a data structure. JIRA: It is used for bug tracking, issue tracking, and project management related to software and mobile apps. Eclipse: This is an Integrated Development Environment (IDE) used for programming. Maven: Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information. Selenium: Selenium is a portable software testing framework for web applications. Cucumber: This is a tool used by computer programmers for testing other software. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Jenkins: Jenkins is a powerful application that allows continuous integration and continuous delivery of projects, regardless of the platform. Bamboo: Bamboo is a continuous integration and deployment tool that ties automated builds and tests. Splunk: This is a software for monitoring, and analyzing machine-generated big data. It captures, indexes, and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards, and visualizations. SaltStack: This is an infrastructure and application management software package that automates the packaging and provisioning of code. Puppet: This is a configuration management tool that is used for deploying, configuring, and managing a server. It can automatically configure newly provisioned systems to create a specific infrastructure. Chef: This is a configuration management tool for dealing with machine setup on physical servers, virtual machines and in the cloud.
  • 11.
    Continuous Delivery (CD) 11 •Continuous Delivery is a disciplined approach of developing software that emphasizes quality at each phase of delivery pipeline that will enable automated deployment at any environment. Continuous Delivery consists of four main components: • Source Code Management (SCM) • Continuous Integration (CI) • Continuous Test (CT) • Automated Deployment Why Continuous Delivery? • Working software is delivered frequently to business partner in small pieces of functionalities • Releases are scheduled based on input and needs from business partners • As the process matures the confidence of automation increased and ultimately it will reduce delivery time, defects, manual efforts and human errors. • become more agile by having capability to identify risks, issues or changes in requirements earlier of development lifecycles. High-level Overview of CD process in major organizations: • Ensure application has been properly On-Boarded to Automated Deployment tool. E.g: Application Infrastructure Model (AIM). • Complete Application code development and test script development for functional and integration tests. • Check in code to approved source control repository: Git • After each check-in Continuous Integration tool Jenkins automatically kicks off the build process and loads artifacts in FRS. • Once tests are passed, Automated Deployment is done to higher environment.
  • 12.
    Continuous Integration (CI) 12 •Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early. • This is a development practice that calls upon development teams to ensure that a build and subsequent testing is conducted for every code change made to a software program. • This concept was meant to remove the problem of finding late occurrences of issues in the build lifecycle. • Instead of the developers working in isolation and not integrating enough, Continuous Integration was introduced to ensure that the code changes and builds were never done in isolation. All source codes are kept in a repository system (Git/SVN) Developer checks in new code into source code repository system (Git/SVN) Continuous Integration server will pick up the software and starts building the new project (Jenkins) If build is a failure team will be notified accordingly. If success it will be tested for any defects (Jenkins) If test fails developers are notified. If test passes the build is ready to be deployed to higher environment. (Jenkins) Why Continuous Integration? The continuous Integration process helps to answer the following questions for the software development team. • Do all the software components work together as they should? • Is the code too complex for integration purposes? • Does the code adhere to the established coding standards? • How much code is covered by automated tests? • Were all the tests successful after the latest change? Preferred CI tool in major organizations: Jenkins • Jenkins provides a container for automated build and testing to support CI behavior. • A server based system running in a servlet container Apache-Tomcat. • Supports SCM tool like Subversion (SVN) or Git. It can execute Apache Maven based projects; shell scripts or windows batch commands. CI Workflow:
  • 13.
    Continuous Testing andAutomated Deployment 13 Continuous Testing (CT): Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. Why CT? • Traditional testing processes are too slow • Huge testing timelines are preventing from delivering quality software at the desired speed. • The goal of continuous testing is to provide fast and continuous feedback in the latest build candidate. • Since testing begins early and is executed continuously, application risks are exposed soon after they are introduced • Development teams can prevent issues from progressing to the next stage of the SDLC Automated Deployment: Deployment automation allows applications to be deployed across the various environments used in the development process, as well as the final production environments. This results in more efficient, reliable, and predictable deployments. Benefits of Deployment Automation: • Repeatability • Fewer errors • Anyone can deploy • Work on what matters • Customer satisfaction • Lower costs Test Automation Code Development Code Git Jenkins Artifact Repo Commands to deploy code Code commit Build triggered No Dev team gets notified to fix code Artifacts stored Automated deployment of Development and Test Code Code deployed Test Environment Test Successful? Higher Environment Yes
  • 14.