Downloaded 66 times
Uploaded byJosh Breinlinger
Marketplace quality tactics
The document provides tactics for marketplace operators to improve quality by establishing a virtuous cycle. It discusses the importance of optimizing screening, training, optimizing matches, retaining high quality participants, and expelling low quality ones. Specific tactics discussed include using data and limited upfront requirements for screening, tailoring training to expected lifetime value, using feedback combined with behavioral data to optimize matches, rewarding top performers to boost retention, and leveraging referrals to attract more high quality participants. Case studies from marketplaces like Faber, Rev, and Outlier/Masterclass are presented. The overarching goal is to design marketplaces where quality begets more quality through strategic application of these tactics.
Marketplace quality tactics
- 1. TACTICS TO IMPROVEMARKETPLACE QUALITY
- 2. Jackson Square Ventures| Confidential | 2 Hello, My Name is Josh
- 3. Jackson Square Ventures| Confidential | 3Jackson Square Ventures | Confidential | 3 Quality is a Cycle Many marketplaces fall into a death spiral Attract Screen TrainOptimize Retain Expel
- 4. Jackson Square Ventures| Confidential | 4Jackson Square Ventures | Confidential | 4 Your Goal is a Virtuous Cycle All depends on how you optimize, retain, and expel Attract Screen TrainOptimize Retain Expel Good Bad
- 5. Jackson Square Ventures| Confidential | 5Jackson Square Ventures | Confidential | 5 How to Measure Quality Ideally, focus your quality metrics on the inputs that drive positive customer outcomes. Figureout your health metrics Ideally,tie LTVto marketplaceparticipant interactions. Eg, HireJosh Low LTV If not, finda good proxy,what marketplaceparticipantsleadtogoodoutcomes forthe business. Ortie certainactions tofutureLTV, egUber drivershows upin3 min,80% likelytorepeat purchase
- 6. Jackson Square Ventures| Confidential | 6Jackson Square Ventures | Confidential | 6 Screening Lots of automation potential Best to use mostly existing data and limited new user requirements Services • Existing profiles • Automated phone screen referencechecks (bonus customer acquisition) • Custom Tests(Prelude Institute) • Existing online activity,egGithub, Dribbble,etc Goods • Content moderation • Photo quality– Airbnb • Visual / videoinspection • In-person authentication (GOAT)
- 7. Jackson Square Ventures| Confidential | 7Jackson Square Ventures | Confidential | 7 Screening – Case Study Screening should be custom tailored to the marketplace Require a simple test to gain as much signal as Marketplace that was hiringAgents • Screening forremote 1099call center agents –hugepoolofcandidates thatcan dothejob. • Screening focused onempathy,internet connectivity, demeanor • InUpworkpost,detailed requirements. Ifyougetthroughtheprocess, youwillbepartofaspecial, elite workforce. • Roleplay scenario ofahelpticket. Received anurgentmessage. Callbackandleave avoicemail tosupportthecustomer. • Test evaluation –wasthereempathy? Wasthereaclear actionplan?Wastherereassurance? Wasitaclear /articulatevoice? 0-100 rank. • Failed –because agents hadtimetoprepare. • Iteration –listen andrecordmessage w10seconds toprepare. • TypingTest –goodproxyforgeneral computer literacy. Wordsperminuteisindicative. • Acceptance Rate=300applications toget30tests completed toget5topcandidates toget2hires. • Impact? -- ~100Agents, TopQuality,EndedupBeingaCriticalBusiness Asset
- 8. Jackson Square Ventures| Confidential | 8Jackson Square Ventures | Confidential | 8 Don’t Go Too Far Be careful not to go too far. Too many upfront hurdles are a turnoff to great quality Increasing Screening Decreasing Interest=
- 9. Jackson Square Ventures| Confidential | 9Jackson Square Ventures | Confidential | 9 More is not Always Better Can go broad or deep, but not both Align the dynamics of the marketplace to your end customer needs Few, Great,Engaged Many,Varied, Low Utilization VS
- 10. Jackson Square Ventures| Confidential | 10Jackson Square Ventures | Confidential | 10 Training (*Onboarding) Training or onboarding should align with the expected LTV of users Focus on getting to productive users Services • In-personvs virtual • Familiarity with Policies and Procedures • Sample Jobs • Let yourusers train each other • How muchcan youinvest inonboarding when futureLTVis unknown? • Zeus – can invest a ton inonboarding units because of long term horizonand highLTV • Incrementaltraining. • High volume / low LTV= notraining – freefor all. • Low volume / high LTV= invest inonboarding • Next Up: Integrationof ISAswith marketplaces?
- 11. Jackson Square Ventures| Confidential | 11Jackson Square Ventures | Confidential | 11 Optimize Marketplaces have a ton of levers to pull on optimizing liquidity for quality This makes the difference between great and terrible customer Whogetswhatand why? • Who getsthe work? • Which item getsordered? • Is it buyer bewareor managed marketplace?
- 12. Jackson Square Ventures| Confidential | 12Jackson Square Ventures | Confidential | 12 Real World vs 5 Stars 5 star systems do not reflect reality Consider a different approach RealWorld 5 StarDistribution 1 2 3 4 5
- 13. Jackson Square Ventures| Confidential | 13Jackson Square Ventures | Confidential | 13 Feedback for Bad 5 stars can separate the bad sellers But do very little to identify the best 5 stars separatebad. 5 StarDistribution 1 2 3 4 5
- 14. Jackson Square Ventures| Confidential | 14Jackson Square Ventures | Confidential | 14 Feedback for Good Arpad Elo developed a system for scoring 2-person competitions It results in a more real world distribution ELO Ratingsseparate good.
- 15. Jackson Square Ventures| Confidential | 15Jackson Square Ventures | Confidential | 15 Finding Signals You won’t be able to separate the good by user feedback alone Need to use behavior and marketplace actions If I choose Jeannie, she should move upand Romany andJamieshould move down.
- 16. Jackson Square Ventures| Confidential | 16Jackson Square Ventures | Confidential | 16 Signals of Progression Look for meaningful signals User feedback is often too subjective Rapidcareeror rateprogressionis a great indicator ofquality!
- 17. Jackson Square Ventures| Confidential | 17Jackson Square Ventures | Confidential | 17 Signals of Repeat Usage Actions speak louder than words Ask your doctor what she would do rather than what she recommends Repeat purchase rateor durationis a strong indicator! “The psychologist Gerd Gigerenzer has a simple heuristic. Never ask the doctor what you should do. Ask him what he would do if he were in your place. You would be surprised at the difference” ― Nassim Nicholas Taleb, Antifragile: Things That Gain from Disorder
- 18. Jackson Square Ventures| Confidential | 18Jackson Square Ventures | Confidential | 18 Optimize – Case Study Constant iteration needed to drive results Faber – construction staffing marketplace • 15k+construction workers • Top 2-3%active each month • 3primaryfactors – Quality Score, Repeat,Location • WorkRequests in5 Tiers • Tier0– Handpicked by Company • Tier1– Quality Score >=90 OR Repeat workerfor the Company • Tier2– Quality Score >=75 OR Recent worker inlast 15days • Tier3– Quality Score >=50 OR Recent applicant inlast 90days ORRecent Workerin last 15days OR Proximity <= 5 km • Tier4– Recent applicant inlast 90days ORany past work OR Proximity <= 10km • Tier5– New workers Impact: increased workerrepeat rate10-20%,GMV per client up ~50%,noshow ratedown 30-40%. 80%of jobs filled by Tier0, 1, or 2. QualityScore • JobCompletionRate • NoShow Rate • CancellationRate • Feedback,thumbsup/thumbsdown/neutral • Positiveratingsin 90days • Hoursworked • Hoursworkedin 90days • Extensions • Extensionsin 90days
- 19. Jackson Square Ventures| Confidential | 19Jackson Square Ventures | Confidential | 19 Retain / Expel This Step is Critical to Build a Virtuous Cycle Reward great users – make their life better Whostays active on the platform? Who goes inactive? Doesyour system naturallykeepthe best andexpel the worst? Make itGreat –but only forGreat Users It’sOk tohave a BadExperience for BadUsers – Segment!
- 20. Jackson Square Ventures| Confidential | 20Jackson Square Ventures | Confidential | 20 Retain / Expel – Case Study Make Sure Great Users Stay, Kick out the Bad Ones Keep iterating on great user retention Rev • Tiered worker system- necessary because ofsupply /demand imbalance. Best workers get promoted tothe highest Tier, Revver+ • Reliability (delivering on time andworking regularly), Tenure, Quality (customer feedback, low errorrate) • Have as much workas they could possibly want –long term earning potential. • Earnmore on aper hourbasis andhave moresteady work. Lower risk. Reduced competition on supply side. • Promotions partiallybased on tenure. Very appealing to users. Feel appreciated. • Engage in the online community • Below ~4.5 stars, low utilization, high errorrates – no longer can get work. Impact: Retention –Revver+ retention is ~5X higher than regular Revver. Be willing tospend lots ofmoney on Great User retention. How not todoit? 99 Designs.
- 21. Jackson Square Ventures| Confidential | 21Jackson Square Ventures | Confidential | 21 Attract Like acquires Like Referrals from top quality users are a great signal. Qualityattractsmore quality Low Qualityattracts thesame Greatathletes wanttoplayfor thebest team Put greatnessondisplay Hidetheugliness
- 22. Jackson Square Ventures| Confidential | 22Jackson Square Ventures | Confidential | 22 Attract – Case Study Attraction is a flywheel Needs to be nurtured, but it will accelerate Outlier.org /Masterclass Going fromzero to0.1 is brutal. But the virtuous cycle ofquality is getting started. Phase 1 • Recipe forSuccess =Content Production Expertise (Academy AwardDirector) +Expert • The Expertise makes the Expert Comfortable Joining • Flywheel on Exceptional content • Casting --> Creates Initial Customer Signals. Phase 2 • James Patterson Serena Williams Usher Annie Liebovitz • Put a Stake in the Ground. We will be the best. • A-List Actor Didn’t make the Cut – Fame vs Mastery. (Screening) • Social Proof Attached Names toa Project (producer, director, editor, etc) Phase 3 • Gotta Deliver. Royalty Checks. Fuel the Flywheel • Can’t pay someone todogreat work. They’re intrinsically motivated. • Targeted Expert Acquisition. Lend Credibility to the Non-Famous Expert
- 23.
Editor's Notes
- #2 Option 1
- #7 Grade inflation Causes mismatching distribution curves Causes an inability to find high quality Means we need to use unused signals
- #8 Grade inflation Causes mismatching distribution curves Causes an inability to find high quality Means we need to use unused signals
- #9 Grade inflation Causes mismatching distribution curves Causes an inability to find high quality Means we need to use unused signals
- #10 Grade inflation Causes mismatching distribution curves Causes an inability to find high quality Means we need to use unused signals
- #24 Option 2