Optimizing DevOps Initiatives: The View from Both Sides of the DevOps Divide

IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Optimizing DevOps Initiatives:
The View from Both Sides of the DevOps Divide
Dennis Nils Drogseth
Vice President of Research
Enterprise Management Associates
Steve Hendrick
Research Director
Enterprise Management Associates

INDUSTRY ANALYSIS & CONSULTING© 2019 Enterprise Management Associates, Inc.
Watch On-Demand Webinar
2
• Optimizing DevOps Initiatives: The View from Both Sides of the
DevOps Divide On-Demand webinar is available here:
http://info.enterprisemanagement.com/optimizing-devops-initiatives-
webinar-ws
• Check out upcoming webinars from EMA here:
http://www.enterprisemanagement.com/freeResearch

Featured Speakers
3
Dennis Nils Drogseth, Vice President, EMA
Dennis joined EMA in 1998 and brings several years of experience in various aspects
of marketing and business planning for service management solutions. He supports
EMA through leadership in IT service management (ITSM), CMDB systems, and
megatrends like advanced operations analytics, cross-domain automation systems,
IT-to-business alignment, and service-centric financial optimization. Dennis also
works over several practice areas to promote dialogue across critical areas of
technology and market interdependencies.
Steve Hendrick, Research Director, EMA
Steve has over 20 years of experience as an industry analyst and 10 years of
experience across a variety of application development roles. He specializes in
application development and deployment topics, including DevOps, lifecycle
management, application platforms, decision analytics, middleware, and the broad
spectrum of tools and technologies that support professional developers and IT
managers.

Sponsors
Platinum Sponsor Silver Sponsors
4

Agenda
• Demographics
• Strategic priorities
• Organizational and process
perspectives
• Growth of agile methods and
productivity
• Defined DevOps activities
• Application performance in production
• Application perspectives
• Technology priorities
• Obstacles and success
• Conclusions
© 2019 Enterprise Management Associates, Inc.5

Demographics
6

Optimizing DevOps Initiatives
Project Objectives
How effective are agile methods and DevOps at transforming IT and the business?
• Development teams. To establish what type of pipeline is in place, its effectiveness,
and how it has changed developer roles.
• Operations staff. To determine how DevOps has impacted IT operations, how well
DevOps is addressing IT Ops needs, and how the Dev to Ops handoff should be
improved.
• IT service management (ITSM). To understand how DevOps is supporting the ITSM
agenda, what’s working and what isn’t, and the most important open issues.
• IT executives. To determine what the IT DevOps strategy is, understand the current
and intended scope of DevOps initiatives, and the most significant challenges in
execution of this strategy.

Optimizing DevOps Initiatives
Research Parameters and Screening Criteria
Sample Size: 400
Geographic Scope: Worldwide
Survey Fielding: March 2019
Respondent: Senior IT staff
Target Respondent: Developers, IT Ops Management, ITSM, IT Executives
Enterprise size: 1,000 employees or more
Custom application development: Active enterprise involvement
Vertical industries: All other than education and IT vendors
Domain expertise: Strong familiarity and involvement with DevOps

Company Size
Please estimate how many total employees does your organization have worldwide? (single response)
9
13%
13%
13%
35%
15%
13%
0% 5% 10% 15% 20% 25% 30% 35% 40%
1,000 to 2,499
2,500 to 4,999
5,000 to 9,999
10,000 to 19,999
20,000 to 29,999
30,000 or more
Percent responding
Limits: <= 12.5% of sample
Limits: >= 30% of sample
Limits: <= 12.5% of sample
EMA Optimizing DevOps Survey, March 2019, Q3, N=400

DevOps Maturity
How would you describe your enterprise’s use of DevOps principles for custom app dev? (single response)
10
11%
27%
35%
19%
8%
0% 5% 10% 15% 20% 25% 30% 35% 40%
We have not started to automate any activities in our DevOps
pipeline
We are just beginning to automate one or two activities in our
DevOps pipeline
We have automated some of the key activities in our DevOps
pipeline
We have automated many of the activities in our DevOps pipeline
We have automated nearly all activities in our DevOps pipeline
Percent responding

Strategic Priorities
11

What’s Driving your DevOps Initiatives?
What is driving your enterprise’s DevOps initiatives? (Select all that apply)
12
60%
57%
57%
51%
36%
31%
29%
29%
29%
28%
28%
28%
17%
16%
14%
Coping with/optimizing cloud opportunities
Speed in updating existing applications
Improved OpEx efficiencies in IT
Move to a micro-services or container-based application portfolio
Speed in delivering new applications
Changes to regulatory or compliance requirements
Superior levels of innovation in application design
Improved customer satisfaction (external)
Business process optimization/efficiencies
Superior performance and reliability
Improved customer satisfaction (internal)
Requirements for overall improved employee productivity
Business competitiveness – revenue growth
Business requirements for new products or services
Business competitiveness – brand protection
0% 10% 20% 30% 40% 50% 60% 70%
Percent responding

DevOps and Top 3 DevOps Activities
What application lifecycle activities are you aware of in your enterprise? (Select all that apply)
What are the three most important lifecycle activities?
13
14%
11%
13%
13%
21%
11%
21%
6%
9%
15%
30%
27%
42%
48%
17%
26%
31%
32%
32%
34%
35%
35%
37%
38%
40%
44%
49%
55%
58%
69%
80%
0% 20% 40% 60% 80% 100%
End-user experience analysis prior to production
App open source security and compliance testing
Service level management for security
DevOps for Database
Application vulnerability testing
Service level management for performance
Application release automation
DevSecOps
Delivery to application staging (pre-production)
End-user experience management and feedback
Application performance monitoring (APM)
End-user experience analysis in production
User acceptance testing
Application design
Release and deployment planning
Requirements assessments
CI/CD
DevOps Activities
Top 3 DevOps Activities
EMA Optimizing DevOps Survey, March 2019, Q13&Q14, N=400

Organization and Process Perspectives
14

What organizations/groups are involved with DevOps initiatives?
77%
68%
65%
59%
56%
45%
32%
28%
26%
24%
23%
22%
21%
21%
18%
17%
14%
14%
14%
13%
11%
8%
0%
Application development teams specific to line of business
Executive IT
Q/A (Quality Assurance team)
Application development - central team
Site reliability engineering (SRE)
Security/compliance
Applications management/applications support
Network operations
Data center operations
An overall dedicated DevOps team
Application-specific DevOps teams (affiliated with line of business or…
Cloud
Line of business – business stakeholders
Process management and compliance
Cross-domain service management team
Dedicated cloud management team
Center of Excellence
Voice/video/UC support
Business planning/market planning
Digital experience management team
ITSM (service desk and beyond)
E-commerce or e-business
Don’t know or not sure
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
% Valid Cases(n=0), Sample Size = 400
Application development, Q/A, and executive IT led in active DevOps groups.
App development specific to LOBs was also most likely to drive the DevOps
initiative.

• On average, 6.76 different groups were involved in DevOps
• CEOs were involved in driving DevOps 33% of the time, but 19% indicated no
business involvement
• CIOs drove DevOps 33% of the time and were affiliated with success
• IT VPs drove DevOps 28% of the time
Organizational Highlights

How has your organizational model changed for DevOps initiatives?
10%
35%
31%
19%
5%
0%
0% 5% 10% 15% 20% 25% 30% 35% 40%
We are now fully decentralized, with multiple
DevOps teams aligned with specific application
and business needs
We are in the process of becoming fully
decentralized with multiple DevOps teams
Development is decentralized for business
alignment, while IT operations and ITSM are part
of a core IT team
Some development is decentralized, but most
resides within our core IT team
We are fully centralized for both operations and
development
Column %
Sample Size = 400, Valid Cases = 0, Total Mentions = 0
The move to decentralization is well underway, but still with a balance of
centralized/decentralized teams

• Success in performance, value, and relevance strongly correlated with the
move to decentralization
• Only 37% have established a separate team to evaluate cross-silo DevOps
processes
• Most of these teams are of limited reach across the enterprise
• Development to operations interactions were rated as good-to-seamless by
46%
• Rated bad-to-confrontational by 15%
Organizational Highlights

To what degree has your DevOps initiative(s) impacted security/compliance/risk
management processes?
14%
32%
38%
14%
3%
0%
0% 5% 10% 15% 20% 25% 30% 35% 40%
Security/compliance/risk management are now
thoroughly integrated into our DevOps processes
Security/compliance/risk management are partly
integrated into our DevOps processes
We are just beginning to integrate
security/compliance/risk management into our
DevOps processes
We are planning to integrate
DevOps processes
We have no current plans to integrate
DevOps processes
Column %
DevSecOps is Clearly on the Rise

• 46% viewed digital transformation as a high priority, or a very high priority
• Only 7% said digital transformation is not a focus
• Most DevOps initiatives are not yet tightly coupled with digital transformation
• Only 31% had plans for, or were actively leveraging, best practices
• Another 31% found no current best practices fit DevOps
• Top four best practices were:
 Six Sigma
 Scrum
 ITIL
 IT Balanced Scorecard
Best Practices Highlights

Growth of Agile Methods and Productivity
21

• More than half of custom enterprise applications are using agile methods in
2019
• 11% will use agile for less than 10% of their custom applications
• 11% are applying agile methodology to more than 80% of their custom applications
• The average expands across the board by about 10% by 2020
• On average, IT organizations completed 57 custom development projects last year
• 131 production releases
• In terms of production deployments
• 46% indicated an annual increase of more than 20%
• Only 19% indicated no increase, or a decrease in production deployments
Agile Methodology and Productivity Highlights

You indicated that production deployments have not changed or decreased in the past year. Please select the top 3 reasons why.
53%
39%
33%
30%
28%
27%
18%
16%
14%
9%
9%
8%
8%
6%
3%
0%
0%
DevOps is demanding changes to our culture and this is temporarily slowing…
Our DevOps initiatives are not yet far enough along to have a material impact…
The agile and microservices approach we are taking loads application…
Security/compliance issues
IT complexity has increased so much that everything takes longer
Troubleshooting apps by ops is taking longer and reduces the rate at which ops…
Negative company revenue growth is slowing application development
We don’t have effective collaboration between developers, IT operations…
Our project workload has increased faster than our staffing
DevOps is more about making dev faster and doesn’t address the challenges of…
Addressing improvements to application quality makes development take longer
Increased business stakeholder complaints about IT
The handshake with operations has become more of a barrier
The handshake with ITSM has become more of a barrier
Senior management has unrealistic expectations regarding how long projects take
Other (please specify)
0% 10% 20% 30% 40% 50% 60%
The Top Reasons for a Decrease in Productivity
• Cultural change is slowing down productivity
• DevOps initiatives are not yet far enough along to have an impact
• Moving to agile and microservices places stress on application architecture and design
% Valid Cases

You indicated that production deployments have increased in the past year. Please select the top 3 reasons why.
38%
35%
33%
31%
27%
22%
19%
17%
15%
14%
12%
11%
9%
8%
5%
4%
0%
0%
We have been effective at making dev faster and addressing the…
Our emphasis on application architecture and design makes development…
Automating how we address application quality makes development go…
We have adopted new technology to mitigate increasing IT complexity
Our DevOps initiatives are improving our productivity
Senior management has been supportive in getting us the resources we…
Better tools are helping ops troubleshoot apps and improve the rate at…
We have been effective at changing our culture to address the changes…
We have been able to increase our staffing to meet project workloads
Increased business stakeholder trust and confidence in IT
The growing use of analytics
Positive company revenue growth is accelerating application development
The handshake with ITSM has become more seamless
The handshake with operations has become more seamless
We have more effective collaboration between developers, IT operations…
Superior levels of integration with security/compliance
0% 5% 10% 15% 20% 25% 30% 35% 40%
The Top Reasons for Increase in Productivity
• We’re effective in accelerating Dev and the handoff to Ops
• An emphasis on architecture and design accelerates development and testing
• Automating our approach to application quality is accelerating development
% Valid Cases

Defined DevOps Activities
25

Current Pipeline Activities
Which activities are currently defined as part of your enterprise’s DevOps pipeline? (select all that apply)
26
66%
63%
60%
58%
57%
56%
43%
43%
42%
42%
32%
31%
30%
29%
28%
Performance testing
Application Deployment to staging
Development of test cases
Architecting applications
Application lifecycle planning and management
Application design
Application staging to production
Requirements gathering
Unit and system integration testing
Component development
DevOps for database activities
Portfolio planning and optimization for cost and value
IT project & portfolio management
Development of use cases
0% 10% 20% 30% 40% 50% 60% 70%
Percent responding

Top 3 Pipeline Activities
What are the top three activities that are most critical to your enterprise’s DevOps process? (select up to
three responses that apply) (not restricted to selecting from Q39 responses)
27
45%
36%
33%
29%
25%
20%
20%
19%
17%
15%
13%
10%
7%
7%
6%
Performance testing
Application architecture
Application design
Unit and system integration testing
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Percent responding

Which DevOps Activities are Most Problematic?
Which of these activities is the most problematic/conflicted in your enterprise’s DevOps process? (select up
to 3 responses)
28
56%
42%
32%
20%
20%
19%
17%
15%
14%
14%
13%
12%
12%
8%
7%
Application architecture
Unit testing & system/integration testing
Application design
Performance testing
0% 10% 20% 30% 40% 50% 60%
Percent responding

Application Performance in Production
29

What is the primary way that your teams solve application problems that occur in a production environment?
19%
44%
23%
14%
0%
0% 10% 20% 30% 40% 50%
Troubleshooting by "digital war room" meetings
using integrated tools
Troubleshooting primarily by domain experts using
siloed tools
Troubleshooting primarily by DevOps teams using
siloed tools
Troubleshooting primarily by DevOps teams using
integrated tools
Column %Sample Size = 400, Valid Cases = 0, Total Mentions = 0
Troubleshooting by domain experts using siloed tools led for solving
application problems in production

What is the primary way that your company finds out about custom-developed application problems once they’re in production?
28%
29%
25%
19%
0%
0%
0% 5% 10% 15% 20% 25% 30% 35%
Calls from users
Events and alerts
Trouble ticket notification
Analytics/predictive insights
Column %
Events, alerts, and calls from users led in how IT finds out about
application problems in production

• Fewer than 40% of production-level application issues
are fully remediated
• 60% are workarounds
• 29% have an integrated release management
process with ITSM
• 41% intend to have a formal release management process
• 19% have an ad hoc release management process with ITSM
Application Performance in Production Highlights

Which of the following statements describes how you are integrating with ITSM?
54%
51%
39%
38%
23%
10%
9%
9%
0%
0%
ITSM creates common workflows with development teams as they move
into production
ITSM actively provisions pre-production environments via our
CMDB/CMS
We leverage configuration automation via ITSM teams to provision our
pre-production environments
Development creates manifests that ITSM inherits and integrates with
our workflows once in production
ITSM participates in an overlay team directed at DevOps and release
management
Releases are scheduled via our ITSM platform
ITSM provides active feedback loops with development on service
quality
ITSM provides active feedback loops with development on application
usage and application requirements
0% 10% 20% 30% 40% 50% 60%
% Valid Cases
Common workflows and CMDB/CMS pre-production provisioning led for ITSM/DevOps integrations
Average Response = 2.33 ITSM/DevOps Integrations

• More than half have created both a fast track and a more traditional track
• To accommodate both agile and more established ITSM best practices
• Having both tracks correlated strongly with success
• 49% have or seek converged toolsets for both tracks
 33% have and continue to have a mix of similar and different toolsets
• 50% include ITSM for all application changes
• Which also correlates with success
Application Performance in Production Highlights

Application-Related Perspectives
35

DevOps Objectives
What statement best characterizes your objective(s) in developing custom applications using DevOps?
(select all that apply)
36
84%
76%
73%
71%
70%
61%
60%
58%
57%
55%
54%
Reduce the time it takes to release new application capabilities
Deliver a significantly higher volume of application capabilities
Integrate developer, IT ops, and ITSM activities
Ensure the seamless transition of apps from development to
production
Automate key activities of the software development lifecycle
Reduce ITSM costs
Reduce application development cost
Improve application quality
Improve developer productivity
Improve customer satisfaction
Reduce IT ops costs
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Percent responding

The Top 2 Most Important DevOps Objectives
37
0% 10% 20% 30% 40% 50% 60% 70%
Reduce IT ops costs
Improve application quality
Reduce application development cost
Reduce ITSM costs
Automate key activities of the software development lifecycle
Improve developer productivity
Integrate developer, IT ops, and ITSM activities
Ensure the seamless transition of apps from development to production
Improve customer satisfaction
Deliver a significantly higher volume of application capabilities
Reduce the time it takes to release new application capabilities
Percent responding
#1 Choice
#2 Choice

Technology Priorities
38

How has DevOps impacted the effectiveness of your company’s operational activities in production?
12%
43%
24%
10%
0%
11%
0% 10% 20% 30% 40% 50%
Operations has become significantly more agile
and automated
Operations has become more agile and automated
Operations has remained the same
Operations has declined in its efficiency and
effectiveness
Operations has declined significantly in its
efficiency and effectiveness
Column %Sample Size = 400, Valid Cases = 0, Total Mentions = 0
55% of respondents claim that DevOps has promoted a
more agile and automated operations

1. Source code control and management
2. CI/CD
3. Network management
4. Collaboration tools
5. ITSM tools
6. Software configuration and management
7. End-user experience analytics
Top Seven Tool Categories Actively Used in Support
of Lifecycle Activities

What are the three most critical tool category gaps in your DevOps activities?
49%
26%
24%
24%
22%
19%
19%
18%
16%
16%
16%
10%
9%
8%
7%
6%
5%
3%
1%
0%
0%
Application performance monitoring
End-user experience analytics
Network management
Collaboration tools
ITSM tools
Dev/test environment creation & management
Software configuration management
Collaboration tools
Source code control and management
CI/CD
Security
Incident management
Application release automation
Database automation (DevOps for Database)
IT project and portfolio management
Release orchestration
AIOps
Value stream management
Capacity planning or service optimization analytics
0% 10% 20% 30% 40% 50% 60%
% Valid Cases
Application performance monitoring and end-user experience led for most critical
tool gaps

1. Active or synthetic transactions
2. API monitoring in production
3. Transactional replay
4. Byte code instrumentation
5. Code injection at the endpoint
6. Pre-deployment functional testing
7. Passive or observed transactions
Top Seven Tool Categories for Application
Performance Monitoring

How does development participate in working with other IT professionals in
assessing application performance and usability in production?
78%
72%
63%
51%
42%
37%
31%
28%
23%
8%
0%
Functional testing based on performance management
specifications
Application design and features are informed by performance data
Lab testing for end-user experience
Analytics targeting application performance are applied prior to
deployment
Focus group feedback on end-user experience
Real user/beta testing for end-user experience
Unique toolsets acquired through the cloud
Analytics targeting user or customer experience are applied prior to
deployment
SDKs targeting performance requirements
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
% Valid Cases
Functional testing and app design informed by performance data led
for how development interacts with IT

• Top three business integrations
• Business process management systems
• Competitive analysis data
• Business-related (external/customer) social media
• 46% viewed AIOps or advanced IT analytics as “very” or “extremely
important” for DevOps
• Only 7% viewed AIOps as “not important”
• 69% felt advanced levels of automation were “very” or “extremely
important”
• Only 4% felt automation was “not important”
Other Technology Priority Highlights

What are your organization's DevOps priorities for leveraging IT analytics?
77%
70%
65%
63%
62%
56%
51%
42%
0%
0%
Optimize application performance by providing rapid feedback to
development from production
Minimizing the time developers spend troubleshooting production
performance issues
Driving improvements via user analytics
To support production/operations in handoffs from development
To support the application development process directly
To provide feedback to development to optimize application
design
To support Q/A test
Assessing application portfolio costs and relevance
Other (Please specify)
0% 20% 40% 60% 80% 100%
% Valid Cases
Optimizing app performance with feedback to development in
production led in leveraging AIOps or IT analytics

Which types of workflow and/or other types of automation are you currently using or planning to use in support of your DevOps initiatives?
66%
64%
64%
63%
62%
62%
61%
59%
51%
50%
37%
34%
32%
30%
28%
27%
25%
23%
0%
0%
IT process automation (and/or runbook)
Advanced workflow integrated with automation
Workflow automation combined with social IT
Auto-scaling/capacity optimization
Standard service desk or ITSM workflows
Configuration automation
Security-related process automation (and/or playbooks)
Integrated trouble ticketing
Automation in support of Q/A testing
Pre-production development-related process automation
Manifest-driven automation throughout the lifecycle
Security instrumentation (continuous attack testing and defense stack validation)
Automation-driven discovery/inventory for pre-production
Automation in support of business-specific outcomes
Alert-driven notification
Advanced incident management handling (beyond trouble ticketing)
Automation in support of data assimilation/data reconciliation
Automation-driven discovery/inventory for production
Other (Please specify)
0% 10% 20% 30% 40% 50% 60% 70%
% Valid Cases
Leading Automation Technology in Use for DevOps:
IT Process Automation (Runbook)

• Top four technical metrics
• Time to deliver new applications
• Acceleration in the number of lines of code delivered
• Reductions in operations-related backlogs
• Acceleration in the number of applications delivered
• Top four business metrics
• Improved IT OpEx efficiencies
• Metrics to show end-user application preferences
• Metrics to show end-user effectiveness
• Business process impact metrics
Other Technology Priority Highlights

Obstacles and Success
48

Areas where DevOps Solutions are Least Effective
Select up to four areas where your current DevOps solutions are least effective. (select up to four that apply)
49
40%
40%
39%
38%
37%
35%
32%
27%
22%
21%
12%
8%
7%
7%
7%
7%
6%
5%
4%
In supporting hybrid team environments
Communicating the business impact of IT services to LOB
In supporting migrations to private cloud/virtualized environments
Understanding digital experience across customer segments, browsers, geographies,
In promoting superior dialog and collaboration between development and operations
In supporting the transition between production and production
In supporting migrations to public cloud
Supporting mobile end users
In orchestrating requirements and processes across the full application lifecycle
In supporting the migration to micro-services and containers
In supporting enhanced levels of integrations testing
In supporting pre-production requirements
In supporting Q/A test
Measuring true end-user satisfaction
In supporting improvements in managing test data (finding, masking, creating and…
Addressing security or compliance-related issues
Capturing how often and by whom services are used
Identifying root causes of problems
Capturing and correlating how services are used in support of business outcomes
0% 10% 20% 30% 40% 50%
Percent responding

Areas where DevOps Teams are Least Effective
Select up to three areas where your current DevOps Team is least effective. (select up to three that apply)
50
50%
43%
41%
35%
31%
29%
16%
13%
12%
9%
7%
Project backlogs in operations
Portfolio planning—IT wastes time developing applications that aren’t valued
Breakage in the handoffs between development and operations/ITSM
Project backlogs in development
Poor levels of change control
Q/A Test can’t keep up with the volume of code and/or code changes
Poor requirements gathering process
Delays in deploying new or changed/code applications
Custom applications fail or perform poorly in production
Development team can’t deliver new solutions quickly enough to keep pace
Poor application performance in production
0% 10% 20% 30% 40% 50% 60%
Percent responding

Conclusions
51

• A more inclusive DevOps is a more successful DevOps
• Development + operations + ITSM
• More evolved DevOps teams have
• Higher investments in analytics
• Higher investments in automation
• More tools for lifecycle management
• More tools for application performance management
Concluding Thoughts

Top 3 DevOps Objectives
• Cost effectively increase the
speed & volume of hybrid apps
• Customer satisfaction
• Improving the transition from DEV
to PRD
Top 3 DevOps Problems
• SDLC planning & management
• App delivery & deployment
• Modern app dev
© 2019 Enterprise Management Associates, Inc.
Concluding Thoughts
53
Top 3 DevOps Solution Issues
• Support for hybrid teams
• Communicating status to the LOB
• Migration of apps to the cloud
Top 3 DevOps Team Issues
• Solving the Ops project backlog
• Effective Portfolio planning
• Dev/Ops/ITSM handoffs

INDUSTRY ANALYSIS & CONSULTING© 2019 Enterprise Management Associates, Inc.54
Check out EMA’s upcoming report,
“Optimizing DevOps Initiatives” at
http://bit.ly/2keKhBs

Optimizing DevOps Initiatives: The View from Both Sides of the DevOps Divide

More Related Content

What's hot

Similar to Optimizing DevOps Initiatives: The View from Both Sides of the DevOps Divide

More from Enterprise Management Associates

Recently uploaded

Optimizing DevOps Initiatives: The View from Both Sides of the DevOps Divide