VoC Naspers product conference oct 2018
•
1 like•123 views
This document discusses using Customer Effort Score (CES) as a key performance indicator for companies. It states that CES is stable when no changes are made but immediately reactive to changes, either positive or negative. Examples of CES reacting to issues like site speed improvements or new features being released are given. The document also mentions storing all customer feedback in a tool called NOMNOM and having teams incentivized to reduce UX debt and improve CES through a 15% rule.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to VoC Naspers product conference oct 2018
Similar to VoC Naspers product conference oct 2018 (20)
Recently uploaded
Recently uploaded (20)
VoC Naspers product conference oct 2018
- 1. Why Customer Effort Score is a wonderful company wide KPI Paying your UX debt Sep 2018
- 2. A potential dubizzler What we sell dubizzle.com + = Awesome dubizzler who can do rad shit What users buy
- 3. 3
- 4. 4 Schumpeter’s creative destruction is accelerating
- 5. 5 A dissatisfied customer doesn’t complain; he just switches. W. Edwards Deming
- 6. 6 Behaviour StrategicTactical Opinion How useful is dubizzle?How user-friendly is dubizzle? How user-friendly do users perceive dubizzle to be? How useful do users perceive dubizzle to be?
- 7. 7 Behaviour StrategicTactical Remote or in Lab Summative Usability Test Remote or In Lab Formative Usability Test Relational NPS Wufoo tickets / App Store Feedback / CS tickets Transactional NPS Customer Effort Score Survey JTBD Interviews Contextual Inquiry Opinion
- 9. Customer Effort Score = % ‘easy’ - % ‘difficult’
- 10. CES is stable If you don’t change anything the trend remains quite stable
- 11. CES is reactive If you do change something CES will react immediately > Releasing new search UX to 100%
- 12. CES is reactive If you do change something CES will react immediately > Release of improve UI
- 13. CES is reactive If you do change something CES will react immediately > Photos not loading due to ISP 8 Apr - 22 Apr > Filter reset on back issue
- 14. CES is reactive If you do change something CES will react immediately > Desktop loading speed improved > Filter bugs > Back button issues > Improve site speed
- 15. We store all user comments in NOMNOM Our one-stop-shop for all customer feedback and insights
- 16. 16 PFR/PFS Content Quality: - Fake ads (pics, Description, Price) Location Filter: - Parent Child relation (marina, sulafa tower) - No guidance for new users who aren’t familiar with neighborhoods. - Cities are not part of the filters. Its very change the city specially on mweb and apps. OLDUI (Frankenstein): - There are still more than one way for users to access the old filters and LPV on desktop and apps. Requests: - Exclude area filter. - Map search option R4R Filter requests: Users struggle to find what they are looking for because ads in r4r section are for rooms, partitions and bedspaces all together. Also most if not all ads mention specific tenant types. - Gender - Family vs Bachelor - Partition vs room vs bedspace Content Quality: - Fake listings Classifieds Keyword Search on Mweb not accessible and reliable enough Category tree is outdated/confusing only 50% found the right category in a tree test. Back button from DPV to LPV on mweb is broken. IOS filters ( you can’t remove a keyword filter unless you go to home and start over) IOS filter button doesnt work on zero results page. Content quality: - Poor description - scam ads - repetitive ads Used Cars Sorting not available on Mweb Filters on Mweb are way less than the desktop experience Content Quality: - Dealers posting as normal users - Cars no longer available - False information IOS filters ( you can’t remove a keyword filter unless you go to home and start over) Google ads & banners: are too many and distracting and are annoying on MWEB because they load slower and push the content down. Used Cars PAA Saving progress: - If a user decides to click back or get interrupted while placing an ad he/she loses all the progress and need to start all over again. Photo uploader: - Our photo uploader doesn't work 100% - It says upload main photo while its meant for uploading all photos. - It will most of the time say that the photo is too heavy. Map option on PAA: - to set your location of your ad is nearly impossible.
- 17. RICE = ( Reach * Impact * Confidence ) / Effort
- 18. 18 Our teams are incentivized to fix our UX debt by our 15% rule and by our Customer Effort Score targets Quick Wins (70%) Big Bets (15%) Thankless Tasks (0%) UX Debt (15%) High Value High EffortLow Effort Low Value VERTICAL FINDING MWEB FINDING DESKTOP POST AD USED CARS 80 80 76 PROP 4 RENT 80 70 N/A ROOMS 4 RENT 80 70 N/A GOODS 80 80 90 BUY CHAT 85 SELL CHAT 95
- 19. 19 However, tracking touchpoints is not enough, because scoring a goal does not equal winning the match Source: https://www.mckinsey.com/business-functions/operations/our-insights/the-ceo-guide-to-customer-experience
- 20. IS THIS FOR YOU? YES! Especially when you find yourself in the ‘KINDA’ stage of becoming user-centric
- 21. IS THIS FOR YOU? YES! Because it’s cheap as chips: only $ 578 per month Getsite Control Zapier Nomnom Instabug OptimizelyX $29/month $299/Month $125/Month $125/Month OLX group funded
- 22. Thank you!
Editor's Notes
- Hi, my name is Joost. I’m a user guy and a classifieds guy through & through. Been with OLX since 2011, found my destiny at dubizzle since 2016 Today I am going to give you a lightning talk on how dubizzle, like any good Lannister, pays off its UX debt. ------------------ Our road towards becoming user-centric At dubizzle we want to turn our users into supercharged versions of themselves. We show them that better picture of themselves over and over again as we help them shape their UAE life to perfection: From finding their first job to finding their apartment to upgrading to a new family car. Knowing what picture to show our users requires us to have a deep understanding of their ambitions and the obstacles that get in the way. We spend tremendous energy developing that intuition day in day out, because in today's fluid business environment, disruption has become the new norm. Creative destruction is accelerating: The average time a company spends on the Standard & Poor’s top 500 list of largest US companies continues to drop. In 1960, it was 55 years, in 2015 it was about 20 years. A big reason is that has never been easier to create a product and get it customers. This increases the pace at which insurgents replace incumbents, a process known as creative destruction. Today innovation has become nothing less than a survival strategy. To constantly reinvent ourselves we need to get the hell out of the building, because no facts live inside. Every day we crawl the sizzling hot streets of the UAE to go and talk, observe and empathize with our users. We go to their homes. We meet them in cafes and in their offices. We rapidly ideate, co-create and test our solutions with them in our mobile usability lab to ensure the minimal viable experiment fits their needs and is 100% user-friendly. We test skateboards, not cars without wheels. Usability is critical. Only then will users experience the true benefits of the experiment and will we be able to establish if our hypothesis false or true. Or to put it more eloquently: “The truth, badly expressed, is a lie.” (Stephen Fry). When our slick-to-the-bone experiments go live our Voice of the Customer team works around the clock to intercept and help users who show signs of fatigue in onsite Customer Effort Score surveys, App Store reviews, Facebook comments and our contact form. We capture their complaints, tag them neatly, follow the trend, get to the root, close the loop and fix them as soon as possible. It’s vital to be fast, because only 1 out of 26 unhappy customers bother to complain. The rest are simply lost. The UAE is a unique place: Where else in the world do you have so much diversity? Where else can you discover the true nature of so many cultures? Connecting to users is your next best alternative to walking into that Star Wars bar on Tatooine. Expect to be blown away!
- At dubizzle we want to turn our users into supercharged versions of themselves. It’s our job to show them that better picture of themselves …over and over again…as we help them shape their Dubai lives to perfection: From finding that first job, to renting a home - to upgrading the family car.
- Knowing what picture to show our user requires us to have a deep understanding of her ambitions. At dubizzle we spend tremendous energy developing that intuition day in day out. It remains our biggest challenge: Most of the time I feel like she is this unreachable lover at the end of the horizon.
- And there’s no time to waste! Creative destruction is the new norm: The time a company spends in Standard & Poor’s top 500 list continues to drop: From 55 years in 1960 to 20 years in 2015. In this world, we have to stay on top of things, especially our users! ----------------- Insurgents pop up during you’re your afternoon snooze and will have disrupted you by tea-time!
- And, like Edwards Deming says: “A dissatisfied customer doesn’t complain, he just switches.” And he’s right: Research suggests that only 1 out of 26 unhappy customers bother to complain! So we have to reach out to users who show signs of fatigue before they fire us without a word. --------------------- “Messy reality”, I hear you think: “Shareholders hate chaos! They want control. They love the crisp precision of the spreadsheet and consider quantitative data to be more ‘objective’ than qualitative. Well, first of you’ll have to make him accept that all data is built upon human bias and judgment. “How might I get my manager to accept that “objective” quantitative data to the hazy contours of a story of our user’s struggles?” How do I get them to see that the qualitative data I bring into into the team is as as objective as quantitative data? That all data is built upon human bias and judgment? https://cxm.co.uk/1-26-unhappy-customers-complain-rest-churn/
- At dubizzle we try to make sense of all of this by answering 4 key question... How user-friendly is dubizzle? How useful is dubizzle? How useful do users perceive dubizzle to be? Today we’ll focus on how we answer the last question: How user-friendly do users perceive dubizzle to be?
- To answer this question we run Customer Effort Score surveys. They capture perceived effort in the moment. Oh … and just to be clear about Effort: It is not about the number of clicks, it’s really about every click being unambiguous and mindless. So how do we intercept our users? ------------ Oh … and just to be clear about what Effort is truly about: Effort is not about the number of clicks, it’s about every click being unambiguous and mindless.
- At the moment we run 26 Customer Effort Score surveys. We only invite users when it’s least intrusive: For example when you just quit posting your ad. It simply asks ‘today was it easy or difficult’. If you had a bad experience, we invite you to tell us why. ------------------------------------ We only ask today. This tiny word ensures the feedback is as actionable as possible. This is a big pro over NPS surveys. NPS surveys usually ask you for your feedback a few or many days after you have done something on the site. It’s usually badly timed, because in classifieds we don’t know the moment you made the exchange. By then it’s probably a tainted, vague memory that is likely to be contorted by today’s shitty weather or the fact that your toast burned to a crisp this morning.
- Much like NPS, the Customer Effort Score is calculated by subtracting the % of people that say ‘difficult’ from the % of people that say ‘easy’. According to Gartner, this score is 1.8 times more predictive of loyalty than Satisfaction Scores and 2 times more predictive than Net Promoter Scores.
- That said, Customer Effort Score is not about an actual score. It’s more about the trend. Sometimes that trend can be boring, but stable. For example: This is the graph for finding used cars on dubizzle. We didn’t change our UX this year. As you can see the trend remains relatively stable. -------------------------------- Of course, it does go up and down a little, because we might not receive a stable number of completed surveys for that week. I might also posit that the Dubai summer does our score in: It’s just too bloody hot and people are just that little less happy in the scorching desert.
- But more importantly, CES is reactive: You’re looking here at the trend lines for finding property on dubizzle. In October 2017 we launched a new UI. The drop clearly shows people were having a hard time. In fact, I’m amazed they didn’t fire me! -------------------------------------- We should also keep in mind that releasing any big change will normally lead to a drop in scores until users get used to it, so we also have to keep an eye out for the comments that are coming in during that change.
- Thankfully we started to improve: We did a lot of fixes to our UI, but even then scores remained relatively low. By January we released fixes for the larger usability issues: Our users showed their appreciation instantly. As you can see since then the trend is positive: Apparently we’re doing the right stuff. -------------------------------------- We should also keep in mind that releasing any big change will normally lead to a drop in scores until users get used to it, so we also have to keep an eye out for the comments that are coming in during that change.
- But then in April 2018 one of the 2 large ISPs in Dubai refused to load photos of our properties. And again, the Customer Effort Score reacted immediately: It went down as a home sick moll. It even stayed down: Some issues seem to have a more lasting effect!.
- Thankfully it also goes up: We increased site speed in August and fixed a few small issues. Instantly we saw a positive effect! To conclude, CES has strong predictive powers: When it goes down your KPIs will follow. So better pay attention to what our users are complaining about!
- And we do pay attention. So far we received over 33 thousand raw comments and complaints. Often helpful, often emotional and sometimes straight-down abusive. We store and neatly tag all those comments in a tool called NomNom: Our one-stop-shop for all our customer feedback and user insights. ----------------- By the way, when I say one-stop-shop, I mean it. We throw all user feedback and insights in NomNom: From usability test observations to complaints to App Store feedback and even comments on social networks.
- Yet, only after tagging the real work starts: Our Customer Hero Kareem Mazhar hunts down the root causes of all usability horrors with the team. Nothing escapes him! And as a result we have a complete overview of all known UX debt at dubizzle: From large Frankenstein issues to minor Back button issues.
- Of course, only knowing our debt is not enough. We’ll actually have to fix it too. So each team makes a UX debt backlog, which they prioritize using RICE. In short RICE stands for Reach, Impact, Confidence and Effort. RICE really helps us to not over-analyze every decision. --------------- Reach represents the number of people hurt by the issue divided by a 1000. Impact is a multiplier. Effort is the number of days the team will have to invest. And confidence is… well the confidence level. --------------------
- But how do we get the product teams to actually pay off our debt in light of all the other urgent business needs? Well, first of all each team has to spend at least 15% of their time on cleaning their kitchen. Second, they are given these Customer Effort Score targets. As a result since April we already created a whopping 75 projects and fixed over 30 issues, both large and small. -------------- https://docs.google.com/spreadsheets/d/1K6SJ_DUx2pZeNVTB2io1D2xDMxBmdJPOKmfsVHNgHiA/edit?usp=sharing As a result we’re proud to share that since April we have formulated 75 fixes so far of which we have been able to fix 30. And this number is growing every day.
- But don’t think Customer Effort Score is a faultless Divine Entity. Tracking touchpoints is not enough, because it does not mean that our users are satisfied with the end result. Scoring a goal does not equal winning the match. ____________________________ So our next plan is to migrate everything to OptimizelyX, where - with a bit of luck - we will be able to track the user’s whole journey! -------------------------------- Second, we don’t yet have the capability to run them in AB tests and… Finally, we haven’t been able to track Customer Effort Score in our apps yet: A big missed opportunity! So in the coming 6 months we set out to experiment with OptimelyX to trigger the surveys and use Instabug for the apps. -------------------------- Also, the surveys are not always very well timed: popups are a bit old-fashioned after all. And think about all those annoying millennials with Adblockers installed.
- Finally, is this for you? It just might be if you answer ‘yes’ to any of the following questions: Are you in that ‘kinda’ stage of becoming user-centric? Are you frustrated about how the company solves customer issues? Is access to customer insights limited and fragmented? Do you feel you have a lot of UX debt? Yes? Then try it! ______________ Is quantitative data completely separate from qualitative analyses? Do you feel you have a lot of UX debt? If the answer is ‘yes’ to most of them then I would consider it.
- It’s cheap as chips: To be precise it will set you back a mere $ 578 per month! Getsite – only $29 NomNom – a humbling $299 Zapier – a mere $125 And Instabug for apps – a terrific $125 So give it a shot!
- But be aware: You might just learn more about local cultural nuances that you can handle: “This site sucks Camel BALLS!”