1. s
Mantra for leveraging Emerging Trends
Namit Agrawal
Senior Program Manager
Adobe Systems
Page | 1

2. Title of the Paper
Next Generation Program and Project Managers – How the role is evolving in the era of Lean and Cloud
computing
Theme
Mantra for Innovative Project Management
Keywords
Program Management, Project Management, Roles and Responsibilities, Change
Abstract
Next Generation Project/Program Manager in a Product Development setting. With the onset of SMAC
(Social, Mobile, Analytics and Cloud) there has been a new felt need for the project and program
managers to transform themselves. Delivering projects on time, with satisfactory quality and within budget
is no longer good enough. Project Managers need to go beyond the obvious and seek new ways to add
value to project teams and to the business. There are three ways in which project and program managers
can do this 1) By being Lean and Agile Champions - Doing development iteratively is now not a new
concept but making sure what is being developed right or not is becoming more and more import. In that
sense project managers need to not only imbibe lean principles but become Lean champions 2) By being
the Voice of the Customer - Often project managers are biased by technology challenges and the
development decisions taken/inspired are more from technology challenges. Project managers need to
think more in terms of what would be in the benefit of customers 3) By bringing in Analytics - Project
cycles no longer end at acceptance testing and shipping the product. What happens to the product after
the release is more important. How is the product getting used, what features are getting used, what
issues are frequently coming up etc. need to be collected and analyzed. The feedback collected needs to
be made part of the product life cycle itself.
Authors
Namit Agrawal – Senior Program Manager, Adobe
Sandeep Grover – Senior Program Manager, Adobe
Page | 2

3. Table of Contents
Introduction ......................................................................................................................................4
Program Management Evolution at Adobe .....................................................................................4
Lean Development .......................................................................................................................4
The Lean Cycle.........................................................................................................................5
Customer Delight..........................................................................................................................6
Data Driven Analysis ....................................................................................................................7
Customer Lifecycle Funnel .......................................................................................................7
Cohort Analysis.........................................................................................................................8
Product Scorecards ..................................................................................................................9
Globalization Metrics .............................................................................................................. 10
Product Locale Analysis ......................................................................................................... 10
Forum Analytics ...................................................................................................................... 11
Support Cases by Customers ................................................................................................ 12
Trial Downloads ...................................................................................................................... 12
Net Promoter Score Surveys.................................................................................................. 13
Conclusion ..................................................................................................................................... 13
References ..................................................................................................................................... 14
Page | 3

4. Introduction
This paper covers key trends in Program Management that are evolving at Adobe as the company
transforms from a shrink wrap, product based business model to a cloud based, subscription business
model. Program Management has played its part in this transformation by reinventing the role and being
more in tune with the realities and requirements of the business. This paper context is from a software
product development company perspective but some of the takeaways can be applied to an IT service
bespoke development environment. Experiences shared here are some of the authors own experiences
or the experiences shared by other Program Managers from different parts of the business.
Please Note: Data/charts used in this paper are indicative/sample data to protect privacy of our
customers.
Program Management Evolution at Adobe
The only constant in life is change. Everything about software development has changed and changed
fast. From waterfall to agile, from shrink wrapped to cloud, from desktop to devices, from no social to
social, from multiple year release cycles to multiple times a day releases. With this rapidly changing
environment program management has to change and adapt as well.
Traditionally Program/Project Managers have been focused mainly on
1) Managing Cost , Schedule , Delivery , Quality
2) Cross Functional Team Management
3) Stakeholder Management
4) Upstream/Downstream Communication Management
5) Dependency and Risk Management
6) Coordination ( Cross Geo, Cross Team, Intra Team)
With the advent of Agile this has changed. Agile brings in principles of self-managed team and several of
these aspects have now been taken over by engineering teams themselves. Program Managers are
increasingly taking the role of resolving outside team control impediments. This has allowed Program
Managers to also look at other value adds they can bring to the teams. Based on our own experiences
and by talking to other Program Managers we have identified three main areas where we feel Program
Managers have stepped up and started to add value to the business.
Lean Development
The concept of lean comes from manufacturing. Resources are limited and they need to be optimized and
waste needs to be minimized (a la Toyota philosophy). This require making sure we are
Page | 4

5. manufacturing/developing the right things. Lean movement is geared towards that. It encourages to
develop incrementally and take frequent customer feedbacks to ensure we are on the right track. This
requires a mindset change from the days when project cycles used to be more than one year long.
Program Managers can help here by becoming lean coaches and empowering the teams to move fast,
move independently and move in the right direction by eliminating wasteful practices. Program Managers
can help create/contribute, maintain and enhance a companywide Lean framework (examples of artifacts
which are part of the framework would be training recordings, templates, samples, case studies etc.)
which new and existing teams can leverage. In essence,they can democratize the lean start up process.
Make it available to anyone, anytime, anywhere. Majority of Program Managers at Adobe today have
attended Lean workshops and are now in the process of taking their teams on “Do it right and do what is
right” path. Several product teams have been able to move to monthly and quarterly release cycles by
imbibing these principles and thus providing continuous value to the customer. A subscription based
model creates low stickiness because exit barrier for the customer is lowered. Therefore by providing
continuous value to the customer we have been able to keep our attrition rates to better than planned
numbers.
The Lean Cycle
Figure 1 illustrates the Lean Cycle some of the product teams have started to use at Adobe.
Build a Hypothesis– At the start of a release or when a new product is conceived, product team builds a
hypothesis. This hypothesis has inherent assumptions/intuitions which need to be validated by taking real
customer feedback. The hypothesis identifies the customer pain we wants to solve and how we want to
solve the pain. This can also be referred as the use cases we wanted to address .
Build Experiments – Experiments are created to validate the hypothesis. Experiments could be as
Page | 5
simple as prototypes created to demonstrate
intended user benefits. Experiments help cut the
waste as they are not effort intensive. Product
team instincts would often translate into
experiments.
Quantify –Qualitative (unstructured, anecdotal,
revealing and hard to aggregate) and
Quantitative(involves numbers and statistics and
provide hard numbers but less insight) feedback
on experiments needs to be captured. This
concept is also referred as data driven product
development where measurement aspects are
Figure 1

6. defined upfront and built into the product from the very beginning.
Analyze – this feedback should then be analyzed and the output of this phase is a set of learnings.
Mostly Program Managers have used spreadsheets to collect the data and ran specific analytical tools
like Cohort Analysis or Pivot tables to draw inferences.
Learn – Post analysis team knows if the hypothesis was correct (in this case team decides to persevere
with the hypothesis) or not (in these case team decides to pivot or change course). Any adjustments
required can be feeded into the next iteration and this whole process is repeated at regular intervals.
The Lean cycle allows teams to quickly develop a product which can be monetized rather than the
traditional approach of trying to monetize what was developed.
Customer Delight
Faster release cycles and data driven product development approach requires one to be in constant
touch with customers (greater than ever before). Product teams are busy developing the right product and
this has opened up the opportunity for Program Managers to participate and drive the customer delight
cycle (Figure 2)
Validate Often - As discussed in Lean principles, Program Managers need to ensure we are reaching out
to customers on a regular basis for validation of work being done. Ideally this frequency should be same
as the duration of one development iteration but a few iterations can also be clubbed depending upon the
nature of the project. Customers own time taken to absorb incremental builds and investment of time to
test and provide feedback needs to be factored in validate often decisions.
Satisfaction Surveys - Periodically satisfaction surveys should be rolled out to get pointed customer
feedback on areas which need improvement. In our company environment this is typically done once a
quarter.
Page | 6
Proactive Response - Program Managers
need to keep a watch on customer
communication and ensure prompt and
comprehensive response. The number of
channels via which customer can
communicate directly or indirectly with us
has increased – email, web, phone, social
media, forums etc. therefore the need to
monitor and consolidate all communication

7. channels. Often a response may require cross team, cross functional coordination where Program
Managers are best suited to get a prompt response back to the customer.
Engage - the customer in the process of development right from the start. Engagement can be by way of
Beta/Prerelease programs, newsletters, webinars, events, emails etc. Engagement drives the quality of
feedback being received from the customers and adds a sense of ownership and trust in the community.
Partner - Program Managers should treat customers as their partners to the success and thus there is
some element of relationship building/rapport building which should happen. Program Managers should
identify counterparts on customer side and build the bridges to make this happen.
Data Driven Analysis
Program Managers should enable the teams to takedata based decision and not subjective/intuition/gut
based decisions. To this extent Program Management should keep an eye on various data capture
points, get the data and build insights. Data should be sliced/diced and analyzed. Peter Drucker famously
said – if you can’t measure it you can’t manage it (or scale it)
For several product teams, analytics instrumentation is now part of user story/product backlog. Analytics
is used to corroborate key hypothesis, understanding patterns, analyzing errors and resolving workflow
issues. Measuring makes us accountable. Teams are forced to confront inconvenient truth rather than be
guided purely by vision/gut/hunch/subjective opinions.
Below are a few examples to illustrate how program management has enabled the team to usedata
driven analysis concepts.
Customer Lifecycle Funnel
Few teams have started using Customer Lifecycle funnels (Figure 3). These funnels track the numbers of
customers who move from Acquisition to Referral as a % of total customers acquired. Greater the number
of customers who are not only revenue generating but also refer products to others (a concept we also
call as Net Promoter Score) the better it is for product teams.
Page | 7

8. Page | 8
Acquisition – this is the
number of customers
‘acquired’ via customer
acquisition process
(marketing campaigns,
advertising, social media
etc.)
Activation – number of
customers who actually
sign up to use the product
and register the product
and launch it
Retention – is
determined by repeat usage. Customers who launch the product more than once, login more than once,
use more than one feature more than one time. It is not necessary that the customer is a paying customer
to qualify as a retained customer. Several products today are available in freemium model whereby the
base products could continue to be used for free.
Revenue – is determined by how many customers are paying customers
Referral – is the number of customers who have brought new customers by way of referrals. This set of
customers are the most engaged and valued customers.
Cohort Analysis
Some of the program Managers have successfully used Cohort analysis (Figure 4). A Cohort is a logical
collection of customers having the same set of characteristics. A sample Cohort analysis done on
Creative Cloud subscribers for measuring usage, conversion and retention over time.
Figure 4

9. Product Scorecards
Another example of data analysis for one of the products – A product scorecard was created to measure
most launches, active use, conversions and retention (Figure 5)
Figure 5
Scorecards - Product
Scorecards
Product ABC - App Most Launched Cohort View - Active Use, Conversion
and Retention at 7,30,60 Days
Page | 9
2013
-42
2013
-43
2013
-44
2013
-45
2013
-46
2013
-47
2013
-48
2013
-49
2013
-50
2013
-51
Free- 7 Day Funnel
Free members who
launch in 7 Days
1866
0
1909
2
1976
2
1970
6
1940
9
1998
1
1880
9
1906
4
1992
1
1930
7
% of members who
launch in 7 Days 30% 30% 30% 29% 29% 29% 29% 29% 29% 29%
Active Free Members
(2+Launch) 5594 5757 5901 5780 5790 5500 5442 5800 5507 3456
% Members who Launch
1.10
1.20
1.10
0.80
0.90
1.00
1.00
1.00
0.90
1.10
in 7 Days
%
%
%
%
%
%
%
%
%
%
Convert in 7 Days 202 234 208 165 173 207 196 184 182 213
Fee-30 Day Funnel
Free members who
launch in 7 Days
1866
0
1909
2
1976
2
1970
6
1940
9
1998
1
1880
9
1906
4
1992
1
1930
7
% of members who
launch in 7 Days 30% 30% 30% 29% 29% 29% 29% 29% 29% 29%

10. Active Free Members
(2+Launch) 5594 5757 5901 5780 5790 5500 5442 5800 5507 3456
% Members who Launch
in 7 Days
Page | 10
1.10
%
1.20
%
1.10
%
0.80
%
0.90
%
1.00
%
1.00
%
1.00
%
0.90
%
1.10
%
Convert in 7 Days 202 234 208 165 173 207 196 184 182 213
Globalization Metrics
Examples of Metrics which Adobe Globalization team captures to support product teams in their product
Globalization efforts.
 Localization Revenue by country
 Localization Revenue by app language
 English vs. non-English revenue
 Localization spend vs. revenue from
localization versions
 Downloads by app language
 Paid vs. Trial by language for Int’l
customers
 Country prioritization / product
prioritization
 Cost of Adding a new language
 Cost per word added in the UI
 Cost of re-work (string
churn/certifications)
 Localization content readership (Product
Help)
 Volume of outsourced effort (Cost)
 Volume of words localized across all
languages
 Leveraging Technology efficiency
 NPS Score
 CSAT Score
 Product specific quality measures
Product Locale Analysis
Another example of Analytics done by Globalization team (Figure 6) wasto measure locale wise number
of downloads. Such metrics is helpful to product teams to optimize spend per locale. Teams can now take
informed decisions.

11. Figure 6
In some of the product teams, Program Managers have brought in data game storming techniques to
enable product teams to define critical factors for success and create metrics dashboards . Gamification of
innovation and dashboarding makes the whole process engaging and fun. Some teams have
experimented with online mediums like www.innovationgames.com to bring in gamification to innovation
process. Program Managers have also acted as facilitators and coaches in the process.
Forum Analytics
An example from forums tracking(Figure 7). Here customer inputs are taken from forum participation in
terms of how many new threads were there, how many views were there, how many were responded.
Program Manager creates a voice of customer report where top 10 issues in terms of views is highlighted
and categorized in terms of product areas impacted. This report is then reviewed with engineering on a
monthly basis and tangible action items are discussed and updated in the form of backlog items which
are then prioritized by Product Management.
Figure 7
Page | 11

12. Discussion
Thread
Page | 12
User
Names
Total
View
helpful_answer
correct_answe
r
Total
Reply
Thread
Link
Thread Topic 1 _name 34 4 2 12 URL
Thread Topic 2 _name 45 5 3 32 URL
Thread Topic 3 _name 23 8 6 24 URL
Support Cases by Customers
Below example (Figure 8) shows how number of support cases from customers is tracked. Orange boxes
indicate customer stopped using the product. This metrics was studied along with the revenue brought in
by each of that customer and that helped the engineering team to prioritize escalations work. This also
helped forecast how many escalations the team should be factoring in their release plan. Learning here
was to not look at the number of support issues in isolat ion but also look at the revenue impact and
strategic importance of the customer to the business.
Figure 8
Company
Name
1/20
10
2/20
10
3/20
10
4/20
10
1/20
11
2/20
11
3/20
11
4/20
11
1/20
12
2/20
12
Grand
Total
A 22 10 38 12 20 15 14 5 9 1 146
C 23 25 12 23 11 8 3 7 5 117
C 49 35 4 2 3 5 3 7 2 110
D 7 6 11 19 25 14 9 17 108
E 24 9 17 21 14 4 3 1 93
F 9 10 21 7 8 7 3 6 2 2 75
G 8 22 5 8 9 4 3 4 5 3 71
H 16 7 4 8 5 5 45
I 2 3 4 8 16 9 42
J 3 3 4 7 6 2 5 8 3 41
K 8 7 5 9 5 2 1 37
L 3 4 1 4 8 8 1 2 3 34
Trial Downloads

13. This example (Figure 9) shows actual product trial downloads were captured along with number of visits
to the login page. Observation here was that that the number of actual downloads was less than 50% of
visits to the login page which sends a message to the product team for taking corrective actions.
Figure 9
Net Promoter Score Surveys
This example (Figure 10) is for the Net Promoter Score surveys rolled out to customers to determine
likelihood of them to promote our products to others
Figure 10
Page | 13
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8
Customer 1 7 5 4 3 4 5 5 4.3
Customer 2 8 4 5 4 2 4 4 3.8
Customer 3 9 5 3 5 5 5 4 4.5
Customer 4 9 4 5 5 3 4 4 4.2
Customer 5 6 4 2 3 2 3 2 2.7
Customer 6 8 4 4 4 2 2 4 3.3
Conclusion

14. Changing business and work environment has opened up a lot of new opportunities for Program
Managers to contribute beyond the regular project management & rather participate as an equal
business stakeholder. At Adobe, Program Managers are focusing on three key areas – (1) bringing in
lean development to the teams (2) bringing in customer advocacy i.e.getting the customer involved in
all aspects of product development and (3) enabling the teams to take data driven decisions.
This is how we see that the program management will evolve (or has already evolved) in coming days
& we should be ready to embrace it.
References
 The Lean StartUp – Eric Ries , Penguin Publication , 2011
 Lean Analytics – Alistair Croll and Benjamin Yoskovitz , O Reilly Publication , 2013
 Adobe Intranet ( Not accessible outside Adobe)
 www.innovationgames.com
Page | 14