This is the result of a research study conducted in Pivotal Chicago office to discover the metrics of product success, prioritization on product decisions and the overall key learnings and feedback from the user interviews conducted.

1. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0
January 2018
Pivotal Tracker Research
Analysis and Findings - Chicago office

2. Table of Contents
● Introduction
○ The Who? The What? And Why?
○ Story time
○ Pivotal Tracker Quick facts
● Methodology
○ Study Population and Sampling
○ Data Collection
○ Competition Analysis
● Results
○ Statistical Analysis
○ Top Learnings
○ Limitations
● Conclusion
○ Top Recommendations
○ Future Research
○ Reflexions

3. Introduction
PIVOTAL TRACKER RESEARCH

4. Cover w/ Image
The Who? The What? And Why?
Who?
● I'm Paulina, a product designer from Pivotal Labs
working for the Chicago Office
What?
● A quantitative and qualitative research study about
Pivotal Chicago's use of Tracker.
Why?
● To test quantitative frameworks for user research.
● To practice and polish user interviewing and research
skills.
● Identify areas for improvement
● How does Tracker stack up to similar tools in the
market?

5. Story time
● Everything started as a beach project with the personal goal of practicing and polishing
interviewing and researching skills.
● After talking with pivots and clients in the Chicago office I observed excitement to talk
about the product and that created more motivation.
● When I met the product team in Denver, I was inspired by their process, attention to
detail, passion and willingness to take action on the next version of Pivotal Tracker.
● Then my research had a new purpose: it turned into a more meaningful collaboration
with the product team where I would share my findings as an addition to their own
plans and workflows.

6. Pivotal Tracker Quick Facts
What I know
● Core project management and planning tool for products in
the Pivotal Chicago office
● Created 14 years ago to deliver agile software solutions
● Has about 100,000 active users and is supported by 30
people in Denver
● The 14-year old tech stack is difficult to maintain; it’s
challenging to release major features frequently

7. Methodology
PIVOTAL TRACKER RESEARCH

8. Study Population and Sampling
Total People interviewed: 20
+- 25% participation of the Chicago population
+- 0.7% participation of the entire organization
● Pivots: 16 people
○ 5 Designers
○ 3 Engineers
○ 8 Project Managers
● Clients: 4 people
○ 4 Project Managers
Chicago Office: 47+ labs
pivots
Pivots: 16
Clients: 4

9. Cover w/ Image
Data Collection
Artifacts and process
● Initial research on the target users,
competition analysis and some market
trends.
● Gathered information from Pivotal Tracker
product team (Lisa Doan, Matthew Gist
and Sara Mcwhorter) to get a sense on the
current status of the tool, main concerns,
next steps and context.
● 20 interviews in 7 days at the Chicago
Office.

10. Competition Analysis and Market Research
● Supports web and iphone.
● Pricing starting from $12.50/month
● 30-days free trial.
● No credit card required
● Subscription based
● Main Features: API, Agile Development,
Estimating, prioritizing, projections.

11. Competition Analysis and Market Research
● Supports web, iphone and
android.
● Pricing starting from
$10.00/month
● Free trial available
● No credit card required
● Subscription based
● Main Features: API, Activity
dashboard, Activity tracking,
estimating, project tracking,
projection, third party
integration.
● Supports web and iphone
● Pricing starting from FREE for
up to 10 users
● Free trial available
● No credit card required
● Subscription based
● Main Features: API, Agile
Development, Prioritizing,
Projections.
● Supports web, iphone and
android
● It provides a knowledge base
and video tutorials.
● Pricing starting from FREE
● Free trial available
● No credit card required
● Subscription based
● Main Features: API, Agile
Development, Activity
Dashboard, collaboration tools
Prioritizing, Projections, Third
party integration

12. Pivotal Tracker

13. Atlassian Jira

14. CA Rally

15. Atlassian Trello

16. Results
PIVOTAL TRACKER RESEARCH

17. Feedback Results: Categories
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

18. Feedback Results: Categories
Pivotal
Tracker
Feedback
337
comments
User
Education /
Training
15
comments
Usability
71
comments
New
features
requests
93
comments
General
Positive
76
comments
General
Negative
82
comments

19. Feedback Results: Themes
Notificati
ons
Presenta
tion
Mode
Search Stories
Learning
/Training
User
Mentions
Mobile
General
Interface
IceboxAnalytics

20. Feedback Results: General Negative
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

21. Feedback Results: General Negative
General
Negative
82
comments
General
Interface
43
Top 3 Themes
Stories
8
Analytics
9

22. General Negative: #1 General Interface
General
Interface
43

23. General Negative: #2 Analytics
Analytics
9

24. General Negative: #3 Stories
Stories
8

25. Feedback Results: Usability
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

26. Feedback Results: Usability
Usability
71
comments
General
Interface
24
Top Themes
Stories
24
Analytics
7
Search
7

27. Usability: #1 General Interface
General
Interface
24

28. Usability: #2 Stories
Stories
24

29. Usability: #3 Analytics
Analytics
7

30. Usability: #4 Search
Search
7

31. Feedback Results: User Education / Training
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

32. Feedback Results: User Education / Training
User
Education /
Training
15 comments
Top Theme
Learning
/Training

33. User Education / Training: #1 Training
Learning
/Training
12

34. Feedback Results: New Features Requests
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

35. Feedback Results: New Features Requests
New
features
93
comments
82
comments
General
Interface
50
Top 3 Themes
Stories
24
Analytics
14

36. New Features Requests: #1 General Interface
General
Interface
50

37. New Features Requests: #2 Stories
Stories
24

38. New Features Requests: #3 Analytics
Analytics
14

39. WE
TRACKER

40. Feedback Results: General Positive
Pivotal Tracker
Feedback
(337 comments)
User Education
/ Training
15 comments
Usability
71 comments
New features
requests
93 comments
General
Positive
76 commentsGeneral
Negative
82 comments

41. Feedback Results: General Positive
General
Positive
76
comments
Top 3 Themes
General
Interface
54
Stories
12
Analytics
5

42. General Positive: #1 General Interface
General
Interface
54

43. General Positive: #2 Stories
Stories
12

44. General Positive: #3 Analytics
Analytics
5

45. Interpretation of Results
New Features Requests
Learnin
g/Traini
ng
Training
Genera
l
Interfac
e
24
Usability
Stories
24
Analyti
cs
7
Search
7
General Negative
Genera
l
Interfac
e
50
Stories
24
Analyti
cs
14
Genera
l
Interfac
e
43
Stories
8
Analyti
cs
9
General Positive
Genera
l
Interfac
e
54
Stories
12
Analyti
cs
5

46. Feedback Results: Overall per Customer Type
● Pivots: 16 people
● Clients: 4 people

47. Feedback Results: Overall per Practice
Total: 337 comments

48. Top Learnings
● Top 3 categories for room for improvement and new features requests:
○ User Interface
○ Story module
○ Analytics section
● UI redesign and better colors. It is really hard to read.
● Chrome plugin for highlighting tags should be part of Tracker.
● Story template picker on the top of the story window.
● We need more robust velocity metrics.
● We need more flexibility to interact with dates in Analytics section.
● Markdown text is really hard to use.
● There are no instructions nor training to start using the tool.
● There is no way to deeply track stories: Which stories were released on which build?
● Search results don't give you context about where the story is located and when it was released.

49. Cover w/ Image
● These are the results for interviewing
clients and pivots only from the Chicago
office.
● Different mindsets, processes and
usage across the different practices.
● There were time constraints to have a
deeper exploration of major pain points
(10 mins limit for each interview)
Limitations

50. Conclusion
PIVOTAL TRACKER RESEARCH

51. ● Better and more friendly user interface.
● Improve colors for readability.
● Chrome plugin to highlight tags should be part of Tracker.
● Dashboard view as a landing page to have an overall view of the status of the project
● Need Story's Template selector.
● More robust analytics, better graphics and advanced filters (by date range).
● Search is not visible and more advanced functionality to find stories.
● There is no way to track stories per release or build.
● There is a need to add more training resources and online guides for non experienced
and intermediate users.
Top Recommendations

52. ● Get more feedback from pivots and clients from the other offices.
● Analyze feedback from the rest of the offices, find patterns and prioritize major pain
points.
● Create a visual and functional prototype to incorporate user feedback and adapt the
prototype accordingly.
● Best practices to define a plan for major feature releases.
Future Research

53. ● What is the goal of the product for the long term?
○ What is missing to get there?
● Robust sentiment analysis, research and feedback across the different offices.
● Minimum efforts, maximum impact: Prioritize offices that use tracker and implement
usage analytics per influence and practice.
● ROI analysis: How much money do we want to spend according to the product goals
and when can we get the profits of that investment?
○ Marketing campaigns
○ Promotions and customer offers
○ Customer service
● What would be the best place to be if Tracker improves as a product and business
unit?
Reflexions

54. Thanks

55. Transforming How The World Builds Software
© Copyright 2017 Pivotal Software, Inc. All rights Reserved.

56. APPENDIX

57. Story time
● Everything started as a beach project with the personal goal of practicing and polishing my interviewing and researching skills.
● After talking with pivots and clients in the Chicago office I observed excitement to talk about the product and took this as a good
opportunity to express their thoughts and feedback in a more direct way.
● When I met the product team in Denver, I was inspired by their process, attention to detail, passion and willingness to take
action on the next version of Pivotal Tracker.
● Then my research had a new purpose: it turned into a more meaningful collaboration with the product team where I would
share my findings with the hope that my research serves as an addition to their own plans and workflows.
● I expect to contribute with the Pivotal Tracker product team by:
﹢ How Pivotal Tracker is being measured, the process around new features implementation and how these changes take
priority over the others and general workflow processes.
﹢ Bring more ideas to the table so the product team has different perspectives and impressions on how the tool is being
used.
﹢ Patterns of how the customer service experience could improve based on people most common trends.

58. Data Collection
Interview Scripts
The interview scripts were designed to discover feedback and opinions held by
the Chicago users with mind set to answer the following questions:
● How are users using the tool?
● What are the user motivations?
● What do they find most engaging?
● What are the main pain points?
● Impressions about the overall look and feel
● What would be the next feature the Pivotal Tracker product team should
implement in the next release?
The challenge
I wanted to get answers to these
questions keeping a conversational
mood for the interview knowing that
I had to get most of the information
in a conversation no longer than 10
minutes (And we nailed it!).

59. Competition Analysis and Market Research
According to the main competitors in the market and based on customer feedback, these
are some of the most relevant competitors mentioned during the interviews in Chicago.
References
Full comparison:
https://www.getapp.com/project-management-planning-software/a/jira/compare/pivotal-tracker-vs-rally-software-agile-project-ma
nagement-vs-trello/
Rally: https://project-management.com/rally-software-review/
Rally: https://www.ca.com/us/products/agile-planning.html?intcmp=headernav
Trello: https://trello.com/about