The document outlines steps taken to measure ROI and drive adoption of Tanzu Application Services. It describes collecting data through a questionnaire about time savings across development, operations, and maintenance. Sample sizes were determined and data was collected through interviews. Results were analyzed and visualized in Tableau dashboards showing cost offsets, hours saved, and adoption metrics over time. This identified that delays in the software delivery process needed to be addressed, pointing to the next metric of focus: cycle time reduction.

1. Nuz Mahmood & Daryl Dwyer
Measuring ROI to Drive Tanzu Application Services Adoption
September 3, 2020

2. 2
Cloud Adoption Programs are Expensive and Complex
The journey:
A potentially complex one from legacy
infrastructure  n-tier infrastructure 
virtualization  public/ private/ hybrid
cloud …
The Cost:
Millions to billions of dollars…
Success:
Not guaranteed…

3. 3
What are your goals? Most likely candidates:
 Scale applications beyond what the data center or hosting provider can handle
 Reduce operating costs
 Faster time to market
 A premium experience for the cloud tenant and/ or the end-user
 An improved experience for the developer and/or the administrator
These are worthwhile goals, and it’s worth spending resources and time to help your
investment in a platform like Tanzu Application Services be successful.
Invest in getting to your strategy’s desired result

4. 4
Workstreams App Migration &
Modernization
Delivery Tools Platform
Enablement
Dev Enablement & Metrics
Mission Accelerate and
advance the adoption
of Tanzu Application
Services while re-
platforming and
modernizing our
application portfolio.
Support TAS adoption
by modernizing software
delivery pipeline to
leverage power of the
platform; create toolsets
to reduce delays in the
process including New
Application Services;
create and maintain
application inventory
system.
Provide and
operate a reliable
platform that
enables rapid
delivery of quality,
innovative
enterprise
applications.
Define and coach mindset,
behaviors, and skills for software
and testing artisanship; “how to”
modernize design of applications
so that they are right sized,
scalable, have resiliency,
leverage our investment in many
data centers, and require a lower
time to market for
delivering business value.
Drive adoption of Tanzu
Application Services using a
data-driven approach to realize
business case and program
goals.
Led By Application
Architecture
Cloud Platforms &
Strategy
Cloud Infrastructure
Engineering
Agile Transformation Office
Structuring the Tanzu Application Services program
Workstreams and Responsibilities

5. 5
First, define what’s important to measure
 Spend some time to develop your strategy
 What behavior from software delivery teams fuels your metric?
 What program approach delivers on your objective
“Give me your monoliths, your client-server apps, your legacy
SOA services yearning to be free."
 “What is the one metric that matters?”

6. 6
Next, operationalize your metrics collection
Set goals around enterprise adoption:
- Measure by team? by application?
- How many suitable teams or applications?
 Challenge #1: how do you define an application?
 Challenge #2: how do you collect the information?
 Note: Both of the above were subject to “organizational anti-bodies”,
i.e. resistance to change! “It’s too much overhead, it’s too broad a
definition, it’s not valuable, etc.”
Create Definitions

7. 7
Then, Visualize Goal and Make Progress Transparent
• Visualizing the adoption using
a stacked bar chart
• “At a glance” totals for executive
tracking.
• Total is broken down by business
unit so managing directors can
compare their progress with others.
• Measured adoption to drive “lift and
shift” approach.
• Presented weekly to our CIO
and Managing Directors.
• Celebrated apps and teams
that were “in production” while
also giving recognition to those
that were “on platform”
1. Application subtotals grouped by delivery orgs like back office, middle office, retail, etc.
0
50
100
150
200
250
FY On Platform Goal On Platform In Production
NumberofApplications
Environment
YTD Adoption Compared to Goal
Business Unit 1 Business Unit 2 Business Unit 3 Business Unit 4
Business Unit 5 Business Unit 6 Business Unit 7

8. 8
Translate Goal for Teams in Terms of ROI for Executives
Rapid adoption was the vision we wanted
our software delivery teams to embrace.
To drive engagement from executives,
translated “adoption of applications” to
hours saved.
“Return” was not in terms of revenue,
but time savings from adopting the
platform, which would increase
opportunity for productivity.

9. 9
Collect Data to Measure Time Saved
1. Created a 65 question template of where Cloud Foundry
was likely to impact our software delivery and
operations processes.
1. Agreed on a dollar cost that would be used to translate
one hour of savings into a cost-offset:
a) If activity X used to take 3 hours before Cloud Foundry,
and now takes 1 hour;
b) And if each activity hour is paid at an average cost of
$75/hr;
c) Then the cost offset is (3*$75) – (1*$75) = $150 of cost-
offset.
With help from Tanzu’s insights

10. 10
Data Collection
For the 65 question template, questions fell into these four categories:
1. Provisioning
• Time in the process of getting environments ready; Storage Setup; Build Server Setup; Firewall Configurations;
Load Balancer Configurations; Internal DNS setup; Databases; Configure access by Infra Ops; Setup for Splunk
monitoring
2. Planning
• Presentations, coordination, meetings to build consensus or gain alignment; pre-work meetings with Business
Process Lines; effort to request and implement Dev Sandbox function; Architecture Review Board; CI setup in
Jenkins
3. Development and Promotion
• Development Time; QA in Dev; Integration Testing in Dev; Deploy to integrated test environments; Deploy to
performance test environments, Deploy to Prod, deploy to disaster recovery environment, Server prep for Feature
Testing; Daily Team Meetings; issues in setup of non-prod environments.
4. Maintenance and Expansion
• Software Upgrades & Patching in Integration Testing environments; OS Patching, including scripts and server
prep; OS Upgrades including integration testing; Application Scaling including vertical scale CPU; Horizontal scale
instances; testing and server prep.

11. 11
Data Collection - From whom, how many
• Used publicly available online calculator:
www.Raosoft.com/samplesize.html
• Established thresholds for these measures acceptable to
CIO and Managing Directors:
• Margin of error (Lower margin of error requires a larger
sample size.)
• Confidence Level (Higher confidence level requires a
larger sample size.)
• Estimated population size
• Response distribution
• Pursued data collection for recommended sample size
Determined sample size & desired confidence

12. 12
Data Collection
• Completed on a rolling basis as teams adopted
Tanzu Application Services
• Interviewed two or more leads from the software
development team to gather data
• Operations questions pursued with those team
members.
• Kept the same interviewer for all data collection.
• Created Tableau dashboards to provide visibility into
progress made across the enterprise.
65 question questionnaire
How

13. 13
Data Collection
Analyze and Visualize

14. 14
Data Collection
Analyze and Visualize
“Measured apps” – numbers reflect number of interviews completed and data compiled
“Prod apps” – numbers of apps in the production environment. This is different from the adoption metrics.
The reason is that “ROI” isn’t fully realized until the app is in prod.
“Cost offset hours” calculated based on number of hours saved identified through the questionnaire.
Prod Apps
156
Measured Apps
39
Developer Efficiency
Improvement
Developer hours saved divided by
“before” developer hours X%
Cost offset hours for
prod apps
No. of apps in prod *
average cost offset
hours
Cost Offset hours for
measured apps
Sum of actual number of
cost offset hours for apps
in prod
Cost offset per measured app
Sum of cost offset hours divided
by number of measured apps.

15. 15
Data Collection
Analyze and Visualize

16. 16
Final Look and Insights Gained
• At the end of our program, we had
reached our adoption and ROI goals.
• …. And discovered in the process that
we needed to solve for delays in our
software delivery process.
• This gave rise to next “One Metric That
Matters,” i.e. Cycle Time.
• To pursue improvement in this metric,
we have started the next phase of our
journey, “Streamlining the Path to
Production.”
Application subtotals grouped by delivery orgs like back office, middle office, retail, etc.
0
100
200
300
400
500
600
700
800
On Platform In Production
NumberofApplications
Environment
Business Unit 1 Business Unit 2 Business Unit 3 Business Unit 4
Business Unit 5 Business Unit 6 Business Unit 7

17. 17

18. 18