Successfully reported this slideshow.
Your SlideShare is downloading. ×

Measuring and Improving CX as a PM by fmr Twilio Staff PM

Ad

Measuring and Improving CX as
a PM by fmr Twilio Staff PM
productschool.com

Ad

Your Product Management
Certificate Path
Certificates
Product Manager
Certification™
Senior Product Manager
Certification™
...

Ad

Corporate Training
Level Up Your Team Management Skills
productschool.com

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 25 Ad
1 of 25 Ad

Measuring and Improving CX as a PM by fmr Twilio Staff PM

Download to read offline

As a Product Manager, you benefit from mixing anecdotes and data to have an understanding of customer needs.

- You can track the success metrics of your product launch better by defining output metrics vs intermediary signals of progress.
- Sometimes you need to drive stakeholder alignment and internal process changes to improve customer experience
- You can build your own checklist for product launch and for communicating with customers to ensure you have your metrics ready, your communication is well received, and you are driving the desired customer behavior.

As a Product Manager, you benefit from mixing anecdotes and data to have an understanding of customer needs.

- You can track the success metrics of your product launch better by defining output metrics vs intermediary signals of progress.
- Sometimes you need to drive stakeholder alignment and internal process changes to improve customer experience
- You can build your own checklist for product launch and for communicating with customers to ensure you have your metrics ready, your communication is well received, and you are driving the desired customer behavior.

More Related Content

More from Product School

Measuring and Improving CX as a PM by fmr Twilio Staff PM

  1. 1. Measuring and Improving CX as a PM by fmr Twilio Staff PM productschool.com
  2. 2. Your Product Management Certificate Path Certificates Product Manager Certification™ Senior Product Manager Certification™ Product Leader Certification™
  3. 3. Corporate Training Level Up Your Team Management Skills productschool.com
  4. 4. Free Product Management Resources Resources Events Courses Podcasts Newsletters Communities eBooks and Reports
  5. 5. Measure & Improve Customer Experience Harshal Patil, Consultant and Product Coach
  6. 6. 2/21 Where does the content (and I) come from? ● I run Spark Creative Technologies, where we provide Product Management coaching and business consultancy services. Most recently, I was a Staff Product Manager at Twilio. ● Some of my foundational experiences were in improving customer experience and different ways of measuring CX, which form the basis of today’s content. ● Questions? See follow-up and FAQs at harshalp.com/talk
  7. 7. 3/21 ● Data + Anecdotes: Merge product analytics, surveys, and interviews to understand your customers ● Metrics + Signals: Apart from output metrics, use input metrics and signals to get leading measures of customer experience (CX) ● Product + Process: Beyond product features, work on eliminating bottlenecks via process changes to improve CX ● Launch + Checklist: Boost your launch success by using an information checklist What can you take away today? 🕗
  8. 8. 4/21 ● In Customers’ KYC Registration funnel ● Normal: Analytics showed some drop-off at each step ● Surprising: But, mid-way through funnel, Analytics showed 60% drop-off of customers ● Step: Simple step that asks customers to wait for verification. Once verified, may congratulate on success... #1 Where did 60% customers drop-off? 60% ↓
  9. 9. 5/21 #1 Why did 60% customers drop-off? How to understand? ● Qualitative Data from surveys and support requests ● Customers complain about receiving reminders although they think they’ve completed the KYC registration ● Empathy through Interviews. Why dropoff? ● Customers thought the step felt (so easy that) like the last step and they were done. ● Customers thought we would notify them when the wait is done. confusions 60% drop? Empathy Quantitative Analytics sw Survey, Support Interviews Qualitative
  10. 10. 6/21 #1 Solving for 60% drop-off Solved for wait… ● Notify customers after wait time, instead of wait. ● Targeted email reminders for drop-offs. ● Notifications and emails bring customers back at that step. Solved for step sequence… ● Merge the earlier and later part of the funnel ● Clear Call-To-Action to move forward, instead of congratulations.
  11. 11. 7/21 #1 Mix anecdotes and data ● Understand and measure customer experience in 3 ways Qualitative anecdotes ● Surveys and Customer support 02 ● Patterns and high-level “why” ● Clarify hypotheses of “why” Empathy from conversations ● Interview customers 03 ● Understand their mental model ● Figure out “why” Quantitative data ● Product and Web Analytics ● Identify patterns ● Prioritize problems to solve ● Build hypotheses of “why” 01 Few Interviews Lots of Data Lots of Why Very little Why Quantitative Qualitative Empathy
  12. 12. 8/21 #1 Anecdotes + Data Merge product analytics, surveys, and interviews to understand your customers
  13. 13. 9/21 ● 100s of support requests to change billing settings. ● Metric to improve is # of support requests. ● Built and launched a self-service feature. ● Yet no improvement in metric. No reduction in requests. ● How to know whether it is working or what is failing? #2 Why didn’t the metrics improve? 🕗
  14. 14. 10/21 ● Support tickets are still high. ● Latency in benefit from launch time. ● Launch is the input and Support tickets reduction is the output. ● what are early indicators? #2 Is the launch working or failing? Launch FAQ views Support redirects Page views API usage Support reduction timeline 🕗
  15. 15. 11/21 #2 Look at Signals along with Metrics ● Track success using input and output metrics. ● You can control input metrics - launch of product. ● Find signals or early indicators of progress. NPS timeline Revenue Launches Sales Training hours Sales calls # Support shortcuts Drop-offs Page latency Support tickets Support costs Outages # Blogs Page views 🕗
  16. 16. 12/21 #2 Metrics + Signals Apart from output metrics, use input metrics and signals to get leading measures of customer experience 🕗
  17. 17. 13/21 #3 Why are 10,000 customers angry? ● Angry emails from customers that we didn’t respond to them, even though that was their first email to Customer Support. ● Reviewed few emails. Found a pattern. ● Customers were replying to emails from us. But their replies not reviewed. ● 10,000 customers emails unopened.
  18. 18. 14/21 #3 Product is good but customers are angry? ● Customers are angry although this isn’t a bug in the product. ● What to do? ● Process changes with stakeholders.
  19. 19. 15/21 #3 Process changes too, not just Product ● Improve process, not just product. Whatever is the bottleneck. ● Process improvements might be cheaper than product launches. ● Product launches also need process changes. Examples below metrics dashboard reviews beta product support Internal roadshows escalation policies manual outage monitoring support redirections
  20. 20. 16/21 #3 Product + Process Beyond product features, work on process changes to improve customer experience by eliminating bottlenecks
  21. 21. 17/21 ● Launched a product redesign - Some features removed. Visual look changed. ● “Good product doesn’t need documentation.” or does it? ● “Why inform support? Anyway there won’t be support tickets.” ● “Did the launch succeed?” No metrics before launch so no idea of improvement. ● Lack of launch hygiene. ● Software worked but customers were not aware, so no customer benefit. Launch fail. #4 Why did the product launch fail?
  22. 22. 18/21 ● Next time, when removing a feature and disrupting CX, build a checklist. ● Beyond, build a gradual release cycle. #4 How to make a disruption a success? ● Wrote escalation policy and educated Customer Support. Support ● Advance heads-up to sales team of large customers. GTM ● Added information in email, bills, and FAQs but not on web portal. In-situ ● SQL Queries to tell which customers are impacted. Identify
  23. 23. 19/21 #4 Use a Checklist to boost launch success ● Product launch involves steps beyond deploying code to production. ● But also, too many small steps. Inspiration - Checklist Manifesto by Dr. Atul Gawande ● How will customers find out about this? ● Announce in conferences, webinars, and periodic marketing emails. Announce ● What will customers ask? ● Provide information, redirection shortcuts, and escalation policies for Customer Support. Support ● How will we sell this to customers? ● Provide information to Sales to share with their customers or answer questions. GTM ● Where will customers think about this? ● Add information of a launch to appropriate parts of the product. In-situ ● Which customers are interested or impacted? ● Based on product analytics, support queries, and sales insights. Identify
  24. 24. 20/21 #4 Launch + Checklist Boost your launch success by using a information checklist
  25. 25. 21/21 Data + Anecdotes: Merge the 3 of product analytics, surveys, and interviews to understand your customers Metrics + Signals: Apart from output metrics, use input metrics and signals to get leading measures of customer experience (CX) Product + Process: Beyond product features, work on process changes to improve CX by eliminating bottlenecks Launch + Checklist: Boost your launch success by using a information checklist What can you take away today? 🕗 Questions? See follow-up and FAQs at harshalp.com/talk

×