The white paper discusses the integration of DevOps as a solution to improve IT delivery by combining development and operations for faster and more reliable software releases. It highlights the cultural shift required for successful implementation, underlining the importance of collaboration, continuous improvement, and automated practices to enhance agility while maintaining system stability. Ultimately, organizations embracing DevOps can achieve higher performance, increased profitability, and overcome the traditional challenges of slow and costly software delivery processes.

1. White paper
The DevOps promise: IT deliverythat’s hot-off-the-catwalk
and made-to-last
Peter Shirley-Quirk
October 2014
Software delivery with fast fashion agility and couture quality?
Business users demand new features and revenue streams as soon as possible. At the same time, they
want systems that are stable and free from down-time. IT teams are challenged with delivering a
steady stream of value to the business, ensuring that software delivery is robust, well-tested and easy
to fix, whilst minimizing disruption and costs.
The problem with the traditional software delivery process is that it is not well
adapted to support these two requirements simultaneously. So companies have to
choose between either delivering changes fast and ending up with a messy
production environment or keeping a stable but outdated environment.1
These scenarios are no longer an acceptable choice for organisations who want to survive in today’s
competitive market; they must find an alternative solution or be left behind.
DevOps promises rapid delivery AND stable operations by integrating business, development, test,
deployment and operations into a cohesive workflow with a rapid feedback cycle. To achieve the
benefits from this new way of cross-functional working, organisations must embrace this cultural
change.
DevOps approaches are already delivering real value to a growing number of companies. Mature
DevOps organisations are five times more likely than their peers to be classed as high performing in
terms of agility and stability. And IT Performance matters: these organisations are twice as likely to
exceed their profitability, market share and productivity goals.2
So why is that, and what can we learn
from this approach?
Why is a new approach needed?
IT delivery is often too slow, too expensive and in many cases not producing consistent quality of
delivery. This isn’t just the business’ view but has emerged from the teams that are building and
supporting systems for them:
Agile Development – these practitioners have made strides in collaboration with the business,
continuous build and test but have not broken the log-jam of lengthy final stage test cycles and
service delivery gateway procedures. They want to extend their agile delivery practises into
system operation and control.
IT Operations – these practitioners were unhappy with the existing state of the art in operations: poor
tooling, opacity of systems, lengthy deployment and repair cycles. This led to improvements
within the discipline of enterprise systems management, and best practises such as:
configuration management, system monitoring, automated provisioning, and the tool-chain
approach.

2. DevOps brings these two strands, and their best practises, together for the purpose of delivering
business value faster, and with high quality and consistency. Communicating these shared goals will be
the basis for bringing the business on-side.
What is DevOps and how does it work?
The word DevOps itself was coined in 2009 by Patrick Debois3
, a leader in the DevOps movement. The
term was formed by combining “development” and “operations” and this provides a starting point for
understanding what it is, but first let’s say what it isn’t. It is not a process, not a set of tools or a
product, and not a job specification; but it could have an influence on all of these.
DevOps is an approach to IT delivery that is based on collaboration
between development and operations arms of IT. The result is regular
delivery of business value, while simultaneously improving the
reliability and operability of the production environment.
Practices of high performing organisations
Key cross-team practices that underpin this approach are outlined below:
 Ensure all teams are working towards the same goal and are being measured by common
business metrics.
 Maintain short development cycles that enable the business to pivot quickly with changing
requirements.
 Use deployment switches and progressive deployment strategies that make it easy to enable
or disable new features in production without re-deployments.
 Create extremely fast feedback loops that allow for almost immediate problem identification
and remediation by the appropriate teams.
 Look for continuous improvement. Reflect on how to become more effective as a team, then
tune and adjust your behaviour accordingly.4
Common business metrics
It is not uncommon for development teams to be targeted purely on delivery to project time and cost
estimates. Likewise test teams for the number of issues they find (more the merrier). And operations
(or production support) have an SLA to meet but otherwise don’t need to reduce the number of defects
or the time to fix them as this load justifies their team size.
Instead, aligning teams towards a common goal will deliver business value whilst reducing the number
and severity of incidents in live, improving the time to recovery and mitigating any impacts.
 Development would be motivated to design for resilience, improve code quality, and improve
the information available for diagnostic activity
 Test teams would be focussed on early detection, preventing defects reaching the next
environment
 Operations and Support teams would aim to prevent alerts from becoming incidents, reducing
the duration of all types of incident and preventing downstream impacts that might cause a
low priority incident to become a high priority issue.
So what metrics are needed?
 Number of incidents and their priority (exclude duplicates, and non-issues or questions)
 Mean time to recovery (MTTR)
 Business value delivered and supported (organisation–defined but often based on profitability)
The first two metrics can be multiplied to give a total time to recovery (TTTR) and from there it should
be possible to calculate a business impact in cost terms. As development and IT deliver system change

3. to support the business, we can use the business case to measure its value, and so the total business
value supported will increase. As your DevOps practises become embedded, the performance of your IT
organisation expressed as the ratio of business value : business impact will show a marked upward
trajectory.
Short development cycles
Keeping the development cycle short means faster time to market and earlier delivery of business
value. Additionally a shorter development lifecycle drives efficiency in delivery costs. As project size
increases the complexity of the delivery increases too due to the size and number of teams involved
and dependencies. More coordination is required and more effort to ensure that the requirements and
design are correctly interpreted at each stage. This increase is not linear as the additional complexity
causes higher management and delivery overheads such as test complexity, environment management
complexity. This in turn leads to longer elapsed time at each stage with increased cost burn as more
teams are involved for longer periods.
Deployment switches
Deployment switches allow you to control when a piece of new functionality is made available to end
users, allowing the business a greater level of control over the delivery of change whilst reducing the
risk.
These switches or feature flags are a special case of operational switches, which can be used to control
business logic dynamically. In a simple case, a piece of configuration can be changed dynamically to
control a business process. Deployment switches work the same way but allow you to turn off some
new piece of functionality whilst leaving the system functioning as before in other respects. They are
most commonly used in customer-facing software and can be part of A/B testing to identify which
alternative the customer prefers.
In back-end scenarios they can be used to support decoupling of delivery between elements that would
previously have been tested and released together. For example, take a change to a message flow:
Source to Gateway and Target. If the Source or Gateway change is tested it can be deployed but
disabled and then enabled once the Target change is implemented. In complex environments with
multiple systems, teams and change boards decoupling delivery is an essential tool in simplifying the
test dependencies, preventing delivery bottlenecks and allowing change to be enabled at the flick of a
switch.
Fast feedback
This principle helps to ensure that the customer is satisfied. And not just your external customers but
internal ones equally. You need to monitor their behaviour, how well do they understand the new
functionality and do they want to use it? For some organisations this will involve formal surveys, for
others monitoring social media and the bottom line will give you the answers. Another method is to
use A/B testing to see whether the customers prefer a particular approach. In a recent case on a
banking website, changing the wording on one button led to a 50% increase in conversion ensuring that
the business case could be met.
Feedback also applies to when things are not working as they should. In this case you don’t want to
wait for the customer to tell you! You need monitoring and alerting based on the observed behaviour
of the system and going further you can detect non-events: deviations from expected behaviour such as
reduced conversions from enquiry to sales.
Continuous improvement
Any process can be improved - and this is particularly true for DevOps, an approach to improving
development life-cycle processes, that is still evolving. However the process for achieving continuous
improvement is well documented and good guidance is available from LEAN, Six Sigma or elsewhere.
The key is to encourage reflection and feedback using, for instance, a Plan  Do  Check  Act cycle.

4. What does a successful DevOps partnership look like?
Capabilities
What are the capabilities that make a mature DevOps practise stand out from the norm? They may be
expressed in slightly different terms in your organisation but these are widely accepted as key
capabilities:
 Collaboration
 Automation
 Continuous Build and Test
 Regular Delivery
 Monitoring and Control
Capability Challenge/Lesson
Collaboration
While the name DevOps points to two parts of
the organisation, the collaboration needed is
much wider.
Successful DevOps requires business,
development, QA, and operations
organisations to coordinate and play
significant roles at different phases of
the application lifecycle. It may be
difficult, even impossible, to eliminate
silos, but collaboration is essential.5
One successful pattern for achieving tighter
integration between the different roles is to
create integrated support teams combining
delivery with support responsibilities. Each team
is focussed on a business function and the
systems that support it. This extends the agile
pattern of embedding business resources in the
delivery team.
Some organisations are not ready to take this
step and the practices below show how you can
improve coordination and knowledge sharing in
other ways.
Automation
DevOps relies on tools to deliver automation –
they may be home grown scripts, open source or
supported tools. Much of the success of DevOps
has come from the powerful tools that the
community has developed to make it work.
Whilst tools are important they are not the
whole story. DevOps is a primarily a culture
change.
Continuous Build and Test
This is development practice that can be used in
many methodologies but is an essential part of
Agile delivery. It ensures that changes are
integrated into source control frequently, built
and tested automatically and any defects
resolved by the developer.
System integration tests can also be automated
with the help of repeatable data set-up scripts,
and proxies for external systems.
Performance tests will rely on statistics from
operations as well as plans from the project
team.
Operational Acceptance Tests should also be
automated – proving the system will function as
expected in the scenario.
Automated tests in the development
environment are all very well but providing
automated testing in higher environments can be
a significant challenge.
If a regression cycle takes two weeks than there
is a fundamental limit to how rapidly you can
deliver change. Automating this often provides
the biggest benefit.
The payoff from continuous testing is usually
well worth the effort. The test function in a
DevOps environment helps to balance quality and
speed. Using automated tools reduces the cost of
testing and allows test engineers to use their
time effectively. Continuous testing shortens
delivery cycles by focussing the time on
functional validation.6

5. Regular Delivery
Instead of ending at the door of the development
lab, continuous Integration in DevOps extends to
the entire release chain: including QA and
operations. The result is that individual releases
are far less complex and come out much more
frequently.
The actual release frequency varies greatly
depending on the company’s legacy and goals.
For example, one Fortune 100 company improved
its release cycle from once a year to once a
quarter - a release rate that seems glacial
compared to the hundreds of releases an hour
achieved by Amazon.
Exactly what gets released varies as well. In
some organisations, QA and operations triage
potential releases: many go directly to users,
some go back to development, and a few are
simply not deployed at all. Other companies -
Flickr is a notable example - push everything
that comes from developers out to users and
count on real-time monitoring and rapid
remediation to minimize the impact of the rare
failure.7
One secret to successful implementations is
practise. The more routine the process, the
more slick the automation, the better the post-
release smoke test, the less risk you have.
Another route to success is making the release
smaller – east to test, easy to roll back. This is
fine for behind-the scenes changes but can be
unsettling for users if they observe the
functionality changing without understanding
why, or even how to use it.
Just because we can deliver more regularly it
doesn’t mean we should overwhelm our users
with frequent changes. Adherence to good
organisational change practise is essential
whether your users are the public, partners or
internal teams.
Organisations need to strike the right balance so
that significant changes are planned and
communicated in advance and the impacts on
users, work in progress, migration activity and
new operational processes can be designed,
delivered and communicated successfully.
Monitoring and Control
As the number of releases goes up the time
available for pre-release user acceptance testing
will eventually have to reduce. Some
organisations, like Flickr mentioned above, are
willing to accept a level of risk so long as they
retain control.
Monitoring is a vital part of understanding
whether what has been delivered is working
successfully. Even the most thoroughly tested
software can be caught out when your real-world
data does not meet the constraints you had
designed for. It is important therefore to have
excellent diagnostics to help you swiftly identify
the cause of an issue and the context in which it
occurred.
Control can mean being able to disable the new
function or revert to the previous functionality
using a deployment switch. This makes it
possible to revert functionality without having to
roll-back the whole release. This is particularly
useful if the releases (though regular) are still
large and combine deliveries from multiple
projects.
How will you know if the release is working? It is
wise to implement post-release smoke tests to
confirm the delivery is working as expected. But
what about when the release is complete? You
have to rely on system health monitoring and
business activity monitoring to provide a near
real-time picture of system activity that you can
compare against your expectations and against
comparable periods prior to the change.
For each technical improvement you bring under the banner of DevOps there is an organisational design
and communication element that is essential to ensure the change is embedded permanently in order
to maintain confidence in the programme.
Practices
In a mature DevOps culture a number of repeatable practices have been observed that can assist with
repeatable delivery and resilient solutions8
. These fall into four areas of which the first two have more
of a tools focus and the latter two have more implications for people and culture.

6. 1. Extend Development lifecycle practices into Production
2. Provide feedback from Production into Development
3. Embed Development into IT Operations
4. Embed IT Operations into Development
Practise Area Challenge/Lesson
Extend Development lifecycle practices into
Production
Extend the continuous integration and release
function into production, integrate QA and
release management into the work-stream to
ensure production readiness of the code and
environment.
Continuous delivery needs to be smooth and
pain-free or you will create downtime in a target
test environment. This is not about “throwing it
over the wall” but rather promoting tested
changes to the next level environment for QA
and evaluation.
This is true of test environments and more so of
production – the change needs to be easy to
validate and easy to revert.
Provide feedback from Production into
Development
Give developers access to production log-files
(where policy allows) and metrics so that they
can better learn how well the system is working.
Information Security policies should define what
data can be provided and what needs to be
sanitized (e.g. to remove Personally Identifiable
Information).
The aim should be to provide as much feedback
as possible with the least friction.
Embed Development into IT Operations
Development resources should be part of the
production support escalation and involved in
issue resolution. Equally, they should be
involved in cross-training operations to reduce
the need for escalation.
Separation of duties is an important principle in
IT Audit and control and compliance with such
standards may require additional controls, such
as controlled, audited, temporary access to
accounts in higher environments.
Supplying the project knowledge to operations
(and the wider community) is essential. The
knowledge base may be embedded in the support
system or in a general purpose store such as
SharePoint.
Embed IT Operations into Development
Involve operations in defining reusable non-
functional requirements and tailoring them
during the design phase.
Create reusable user stories for operational
concerns – with ops as the stakeholder - and add
them to the backlog.
Involve Operations users in Failure Mode Effect
Analysis workshops to provide knowledge,
revalidate the design and improve resilience.
Following these practices and providing test
results will smooth the path to delivery as
operational stakeholders will have confidence
their concerns are met.
Operational procedures should be scripted
(where possible) and tested to de-risk from
human error.

7. Overcoming the barriers to DevOps adoption
We have discussed a number of best practices and will go on to outline the benefits of DevOps; but
what are the obstacles to taking this path? In the 2012 survey by Puppet Labs some cultural barriers
emerged:
48 percent of all respondents indicated that one of the biggest difficulties in
implementing DevOps was that the value wasn't understood outside of their group.
A similar proportion (43%) cited the lack of a common management structure for
Dev and Ops teams.
Of those who had no plans to implement DevOps 49 percent identified a lack of
manager buy-in as a blocker, followed by "lack of team buy-in" (38%).9
The reasons have a lot in common: lack of awareness and understanding. The primary capability
needed to make DevOps a success is open collaboration between all the partners. This requires good
communication and a supportive attitude – the benefits come by making everyone’s life that little bit
better so start by asking what they would like to improve in the way your teams relate.
Other cultural concerns include risk aversion. A company that knows the business is highly risk averse,
for example concerned that a deployment failure might cause reputational damage, will be sceptical
about the claim that all the testing required can be completed in weeks or even days. For this case,
you should be sure to walk before you run. Similarly, there may be established governance processes
and the owners will be concerned that this is an attempt to circumvent governance.
To address these concerns, prove the value of the automated tests, ensure they cover performance,
operational, penetration and regression tests. In this way DevOps becomes part of the governance
solution rather than making governance the final bottleneck that can’t be removed10
. Show that the
process has met the quality and operational readiness stage gates, ensure the stakeholders have signed
off user acceptance tests. If your system has sufficient scale or sophistication, consider using a
limited-visibility deployment to enable UAT in the live environment before switching the functionality
on for all users.
What are the benefits?
Improving IT operations brings a range of benefits, and research has shown that this always has made a
dramatic impact on the effectiveness of IT within an organisation.
Some organisations have always stood out from their peers in their IT performance. In 2007, the IT
Process Institute benchmarked over 1,500 IT organisations and concluded that high-performing IT
organisations were on average 5-7x times more productive than their non-high performing peers. They
were making 14x more changes, with one-half the change failure rate with 4x higher first fix rates, and
10x shorter Severity 1 outages times. They had 4x fewer repeat audit findings, they were 5x more
likely to detect breaches by an automated internal control, and had 8x better project due date
performance!11
At this time the term DevOps had not yet been coined (that followed the first DevOpsDays conference
in October 2009). So our starting point is that high performing organisations by definition have found
ways to deliver successfully, reduce deployment failures, and recover quickly from failure. These
lessons have provided the background out of which DevOps guidance has emerged taking advantage of
changes in the IT landscape. So what has happened since then?
Since 2011 Puppet Labs has run a survey of IT professionals to determine the extent and impact of
DevOps practices9
. Naturally the respondents were self-selecting as interested in the topic but there
was still a good cross-section that had not “implemented DevOps practices”. In the 2012 survey, sixty-

8. three percent of respondents had implemented DevOps practices, a 26 percent increase in adoption
rate since 2011.
The respondents come from 90 countries and have an even distribution across different scales of IT
organisation. There is no ideal company size for DevOps – small nimble companies can punch above
their weight and the major players have come to rely on it to maintain their agility as they have grown
so rapidly.12
A high-performing organisation is characterized by its ability to ship business-critical applications
quickly without disrupting service. The review found that organisations that had implemented DevOps
practices were up to five times more likely to be high-performing than those that had not, and the
more mature the implementation, the higher the performance.
One consistent result across the whole survey was that all the measures indicating high performance
increase as the maturity of the DevOps practise within the organisation increases: the good continue to
get better. To highlight just one of these, the meantime to recovery (MTTR) improves considerably
with DevOps maturity. 40% of those who have not implemented DevOps have an MTTR of more than
one hour, whereas for those who have more than a year’s experience this is 10%, and half of the rest
are resolved within minutes. Think of all the saved downtime this represents.
DevOps may be in fashion now but it is set to become a classic
Adopting DevOps means embracing a change in the IT culture: improved communication between the
business and all teams involved in IT delivery; increased automation to drive out bottlenecks from the
delivery lifecycle; and rapid feedback to confirm success, adoption, benefits and to inform the next
round of improvements.
Organisations will gain measurable business benefits from adopting DevOps:
 Faster time to market – shorter development cycles and regular deployment
 Increased quality – particularly in non-functional areas such as increased availability, improved
change success rate, fewer failures taking less time to recover
 Increased efficiency – more time spent on delivering business value and less on routine
operations and repetitive testing
There are less tangible benefits too. DevOps practices promote employee satisfaction. Using tools and
automation to remove repetitive work allows people to concentrate on adding value, making decisions
and taking responsibility. Positive relationships with colleagues remove friction - and not just from the
delivery process. If this is supported by a culture of learning and high trust then you have all the top
predictors of job satisfaction in place. And job satisfaction is the #1 predictor of organisational
performance.13
0 20 40 60 80 100 120
Not implementing
Currently implementing
Implemented <12 months
Implemented >12 months
% High Performers within cohort
increases with DevOps maturity
High Performers % Rest

9. Small wonder that executives who became interested in DevOps for these bottom-line and brand-value
benefits now also value it for ability to attract and motivate top talent – which in turn improves the
business performance.
Where do I start?
If these benefits appeal to you then here are some questions and options to consider.
Are you delivering fast fashion agility to the business?
Is the IT Department meeting the needs of the business? Where can we improve? What about
internally within the department – which processes are working and which could be improved? Where
are the bottlenecks?
Is your organisation designed to support this delivery model? What changes are needed and how can
they be brought about? This is going to be fundamental to achieving the desired results. Technology,
tools and processes can help but they will not be successful unless the leadership, communication and
incentives support it.
Haut couture quality is built-in, not just window dressing
Automation of the delivery process includes automation of the test cycles that support a high quality
product. Repeatable delivery using these techniques reduces the risk and increases the control for the
business. Operational information supports the business by making the systems more transparent and
able to deliver the insight needed to control and guide the business.
If you want your IT department to excel in timely delivery and customer satisfaction you need to get
the logistics right – and the time to start is now!
© 2014 Peter Shirley-Quirk
1
Niek Bartholomeus, “My experience with introducing DevOps in a traditional enterprise”,
http://niek.bartholomeus.be/2013/01/28/introducing-a-devops-culture-in-a-traditional-enterprise, 28 Jan
2013.
2
“2014 State of DevOps Report”, Puppet Labs, http://puppetlabs.com/sites/default/files/2014-state-of-devops-
report.pdf, February 2014.
3
Damon Edwards, “The History of DevOps”, IT Revolution Press, http://itrevolution.com/the-history-of-devops.
4
New Relic, “Navigating DevOps”, http://try.newrelic.com/rs/newrelic/images/NewRelic -DevOps-Primer.pdf
5 Laurie Wurster et al, “Emerging Technology Analysis: DevOps a Culture Shift, Not a Technology”, Gartner report,
August 2013.
6
“Enterprise testing capability for continuous software delivery”,
http://www.ibm.com/ibm/devops/us/en/build/test/.
7 New Relic, “Navigating DevOps”, http://try.newrelic.com/rs/newrelic/images/NewRelic -DevOps-Primer.pdf
8
Patrick Debois, “Devops Areas - Codifying devops practices”, http://jedi.be/blog/2012/05/12/codifying-devops-
area-practices/, 12 May 2012.
9
“2013 State of DevOps Report”, Puppet Labs and IT Revolution Press, https://puppetlabs.com/wp-
content/uploads/2013/03/2013-state-of-devops-report.pdf
10
Mike Kavis, “DevOps: Are We Finally Buying into Governance?”, The Virtualization Practice,
http://www.virtualizationpractice.com/devops-finally-buying-governance-23247, October 2013.
11
Gene Kim, “Visible Ops”, http://realgenekim.squarespace.com/visible-ops/
12
James B. Brown, “5 Reasons Why DevOps is Hitting Its Stride”, Innovation Insights,
http://insights.wired.com/profiles/blogs/5-reasons-why-devops-is-hitting-its-stride, March 2014.
13
“2014 State of DevOps Report”, Puppet Labs, http://puppetlabs.com/sites/default/files/2014-state-of-devops-
report.pdf, February 2014.