By Mike Peach from Pendo at ProductCamp Twin Cities 2016 One of the most common questions we hear at Pendo is "What should we measure?" Everybody understands the imperative to measure product usage, but what's less clear is what, specifically, should be measured, and what do those measurements mean. This session will walk through some of the key metrics that we use internally to measure the health of our customers and our product, and how we put those measurements into action.

1. 5 Key Measurements of Product Success
Assessing the health of your customers and your products

2. 2
Introductions
5 KEY MEASUREMENTS FOR PRODUCT SUCCESS
What is Pendo? Who am I?
A venture-backed start-up based in Raleigh, NC.
Pendo extends your product to capture all user
behavior, gather feedback, and guide users
Head of Product Marketing at Pendo
Previously Product Marketing and Demand Gen
at IBM, Product Manager at Quadbase Systems

3. 3
How We Used To Measure Things
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
● Feedback
● Transactional Data
● (Some) User Observation

4. 4
Our Development Path
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
● Idea / Intuition
● Sell
● Build
● Sell
● Feedback
● Revenue?

5. 5
Transactional Data Says...
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
v1.0 Licenses sold: 1

6. 6
The World is Different Now
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
User Onboarding
Big Data SaaS Agile

7. 7
Powering a New Product Experience Cycle
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Goals
Metrics & Results
1. User Onboarding
New user becomes proficient with basic application
functions, and account is setup (if needed).
Onboarding experience is tailored based on external
customer data
2. Product Intelligence
User interactions with the application are measured,
and feedback is captured.
3. Adaptive Interface
Customized guidance based on product intelligence
helps the user get more value from the application.
The UX itself adapts to the user’s context
4. Continuous Education
The product is updated, and new learning content is
delivered allowing the user to become proficient with
the new version of the product.

8. 8
We Can Measure Everything...
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● Unique visitors
● Pageviews
● Logged-in users
● Login frequency
● Clicks, focus, or hover events
● Load times
● Time in the application
● Enter / exit points
● Task completion
● Device, OS, and browser
● Session path
● Satisfaction ratings
...But Which Ones Matter?

9. 9
It’s Ultimately About Revenue
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

10. 10
And Revenue is Driven by Churn
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

11. 11
5 Key Measurements of Product Success
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Breadth
How many active users do I
have?
Frequency
How often do user login?
Depth
How much of my product do
they use
Efficiency
How hard is it for users to
complete tasks?
Satisfaction
How do employees rate the
community experience?

12. 12
Breadth of Use
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How many total users does my product have?
● How many of them are active users?
● Which customers / accounts are they from?

13. 13
Measuring Visitors & Accounts
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Your SaaS Product
Account A Account B Account C
Visitors Visitors Visitors

14. 14
Breadth Measurements
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Healthy
> 5 Active Users
Improvement
2 - 5 Active Users
Risk
< 2 Active Users

15. 15
Frequency of Use
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How often do my users login?
● How much time do they spend in the application when they do?
● Which customers / accounts are they from?

16. 16
Frequency Measurements
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Healthy
> 10 Logins per
Month
Improvement
2 - 10 Logins per Month
Risk
< 2 Logins per
Month

17. 17
Segment Analysis
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Tracking breadth and frequency across groups of users
Account Age Account Size User Role
✓ Quarterly groups of
new customers
✓ Enterprise vs. SMB
customers
✓ Access across
departments

18. 18
Depth of Use
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● Which features get the most use, and how often are they used?
● Which are the most valuable to specific user segments?
● What percentage of the product gets regular use?

19. 19
Understanding Key Features
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Core functionality common to the most productive users

20. 20
Key Features for Pendo
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
TAG FEATURES & PAGES PUBLISH A GUIDE INTEGRATE WITH SFDC
LAUNCH A POLL CREATE A SEGMENT

21. 21
Depth Measurements
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Healthy
Used All Key
Features
Improvement
Used 2 / 5 Key Features
Risk
Used < 2 Key
Features

22. 22
Usability is Critical
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

23. 23
Efficiency of User Actions
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● How useable is the overall user experience?
● What features and functions do users struggle to understand?
● Which step(s) cause users to abandon a process?

24. Step 1
Step 2: 70%
Step 3: 50%
Step 4: 25%
24
How to Measure Efficiency
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

25. 25
Efficiency Measurements
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Very Usable
60%+ Completion
Rate
Usable
30 - 60% Completion Rate
Users Struggling
< 30% Completion
Rate

26. 26
Satisfaction and Feedback
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
● What is overall user satisfaction?
● Which is the motivation behind observed user behavior?
● How useful are specific features? How can they be improved?

27. 27
What is NPS?
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
"Net Promoter System is based on the
fundamental perspective that every company's
customers can be divided into three categories.
Customers can be categorized based on their
answer to [our] ultimate question...
“The best way to gauge the efficiency of a
company's growth engine is to take the
percentage of customers who are promoters and
subtract the percentage who are detractors. ”
~ Bain & Company
NPS was introduced in the BHR in 2003

28. 28 5 KEY MEASUREMENTS OF PRODUCT SUCCESS
“How likely are you to recommend us?”

29. 29 5 KEY MEASUREMENTS OF PRODUCT SUCCESS

30. 30
Additional Feedback - Feature Input
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

31. 31
Additional Feedback - Demographic Information
5 KEY MEASUREMENTS OF PRODUCT SUCCESS

32. 32
Sources of Product Data
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Internal objects
Key application stats such as users, licences,
etc, that is stored as part of application data
Web analytics
Page-level and user data captured by
instrumenting application pages with Google
Analytics or other web analytics software
Support cases
Current or archived support requests from
help desk (kana, zendesk) or other repository
software
User testing & surveys
Qualitative user feedback from observed user
experience (UX) testing sessions and user
surveys

33. 33
Begin Optimizing Your Product Experience
5 KEY MEASUREMENTS OF PRODUCT SUCCESS
Goals
Metrics & Results

34. www.pendo.io
Thanks!
mike@pendo.io