The Marketplace Quality Flywheel

1. TACTICS TO IMPROVE MARKETPLACE QUALITY

2. Jackson Square Ventures | Confidential | 2
Hello, My Name is Josh

3. Jackson Square Ventures | Confidential | 3Jackson Square Ventures | Confidential | 3
Quality is a Cycle
Many marketplaces fall into a death spiral
Attract
Screen
TrainOptimize
Retain
Expel

4. Jackson Square Ventures | Confidential | 4Jackson Square Ventures | Confidential | 4
Your Goal is a Virtuous Cycle
All depends on how you optimize, retain, and expel
Attract
Screen
TrainOptimize
Retain
Expel
Good
Bad

5. Jackson Square Ventures | Confidential | 5Jackson Square Ventures | Confidential | 5
How to Measure Quality
Ideally, focus your quality metrics on the inputs that
drive positive customer outcomes.
Figureout your health metrics
Ideally,tie LTVto marketplaceparticipant interactions. Eg, HireJosh Low LTV
If not, finda good proxy,what marketplaceparticipantsleadtogoodoutcomes forthe
business.
Ortie certainactions tofutureLTV, egUber drivershows upin3 min,80% likelytorepeat
purchase

6. Jackson Square Ventures | Confidential | 6Jackson Square Ventures | Confidential | 6
Screening
Lots of automation potential
Best to use mostly existing data and limited new user
requirements
Services
• Existing profiles
• Automated phone screen referencechecks
(bonus customer acquisition)
• Custom Tests(Prelude Institute)
• Existing online activity,egGithub, Dribbble,etc
Goods
• Content moderation
• Photo quality– Airbnb
• Visual / videoinspection
• In-person authentication (GOAT)

7. Jackson Square Ventures | Confidential | 7Jackson Square Ventures | Confidential | 7
Screening – Case Study
Screening should be custom tailored to the
marketplace
Require a simple test to gain as much signal as
Marketplace that was hiringAgents
• Screening forremote 1099call center agents –hugepoolofcandidates thatcan dothejob.
• Screening focused onempathy,internet connectivity, demeanor
• InUpworkpost,detailed requirements. Ifyougetthroughtheprocess, youwillbepartofaspecial, elite workforce.
• Roleplay scenario ofahelpticket. Received anurgentmessage. Callbackandleave avoicemail tosupportthecustomer.
• Test evaluation –wasthereempathy? Wasthereaclear actionplan?Wastherereassurance? Wasitaclear /articulatevoice? 0-100 rank.
• Failed –because agents hadtimetoprepare.
• Iteration –listen andrecordmessage w10seconds toprepare.
• TypingTest –goodproxyforgeneral computer literacy. Wordsperminuteisindicative.
• Acceptance Rate=300applications toget30tests completed toget5topcandidates toget2hires.
• Impact? -- ~100Agents, TopQuality,EndedupBeingaCriticalBusiness Asset

8. Jackson Square Ventures | Confidential | 8Jackson Square Ventures | Confidential | 8
Don’t Go Too Far
Be careful not to go too far.
Too many upfront hurdles are a turnoff to great quality
Increasing
Screening
Decreasing
Interest=

9. Jackson Square Ventures | Confidential | 9Jackson Square Ventures | Confidential | 9
More is not Always Better
Can go broad or deep, but not both
Align the dynamics of the marketplace to your end
customer needs
Few, Great,Engaged Many,Varied, Low
Utilization
VS

10. Jackson Square Ventures | Confidential | 10Jackson Square Ventures | Confidential | 10
Training (*Onboarding)
Training or onboarding should align with the expected LTV of users
Focus on getting to productive users
Services
• In-personvs virtual
• Familiarity with Policies and Procedures
• Sample Jobs
• Let yourusers train each other
• How muchcan youinvest inonboarding when futureLTVis unknown?
• Zeus – can invest a ton inonboarding units because of long term horizonand highLTV
• Incrementaltraining.
• High volume / low LTV= notraining – freefor all.
• Low volume / high LTV= invest inonboarding
• Next Up: Integrationof ISAswith marketplaces?

11. Jackson Square Ventures | Confidential | 11Jackson Square Ventures | Confidential | 11
Optimize
Marketplaces have a ton of levers to pull on optimizing liquidity for
quality
This makes the difference between great and terrible customer
Whogetswhatand why?
• Who getsthe work?
• Which item getsordered?
• Is it buyer bewareor managed marketplace?

12. Jackson Square Ventures | Confidential | 12Jackson Square Ventures | Confidential | 12
Real World vs 5 Stars
5 star systems do not reflect reality
Consider a different approach
RealWorld 5 StarDistribution
1 2 3 4 5

13. Jackson Square Ventures | Confidential | 13Jackson Square Ventures | Confidential | 13
Feedback for Bad
5 stars can separate the bad sellers
But do very little to identify the best
5 stars separatebad. 5 StarDistribution
1 2 3 4 5

14. Jackson Square Ventures | Confidential | 14Jackson Square Ventures | Confidential | 14
Feedback for Good
Arpad Elo developed a system for scoring 2-person competitions
It results in a more real world distribution
ELO Ratingsseparate
good.

15. Jackson Square Ventures | Confidential | 15Jackson Square Ventures | Confidential | 15
Finding Signals
You won’t be able to separate the good by user feedback alone
Need to use behavior and marketplace actions
If I choose Jeannie, she should move upand
Romany andJamieshould move down.

16. Jackson Square Ventures | Confidential | 16Jackson Square Ventures | Confidential | 16
Signals of Progression
Look for meaningful signals
User feedback is often too subjective
Rapidcareeror rateprogressionis a great
indicator ofquality!

17. Jackson Square Ventures | Confidential | 17Jackson Square Ventures | Confidential | 17
Signals of Repeat Usage
Actions speak louder than words
Ask your doctor what she would do rather than what she
recommends
Repeat purchase rateor durationis a
strong indicator!
“The psychologist Gerd Gigerenzer has a simple heuristic.
Never ask the doctor what you should do. Ask him what
he would do if he were in your place. You would be
surprised at the difference”
― Nassim Nicholas Taleb, Antifragile: Things That
Gain from Disorder

18. Jackson Square Ventures | Confidential | 18Jackson Square Ventures | Confidential | 18
Optimize – Case Study
Constant iteration needed to drive results
Faber – construction staffing marketplace
• 15k+construction workers
• Top 2-3%active each month
• 3primaryfactors – Quality Score, Repeat,Location
• WorkRequests in5 Tiers
• Tier0– Handpicked by Company
• Tier1– Quality Score >=90 OR Repeat workerfor the Company
• Tier2– Quality Score >=75 OR Recent worker inlast 15days
• Tier3– Quality Score >=50 OR Recent applicant inlast 90days ORRecent Workerin last 15days
OR Proximity <= 5 km
• Tier4– Recent applicant inlast 90days ORany past work OR Proximity <= 10km
• Tier5– New workers
Impact: increased workerrepeat rate10-20%,GMV per client up ~50%,noshow ratedown 30-40%. 80%of
jobs filled by Tier0, 1, or 2.
QualityScore
• JobCompletionRate
• NoShow Rate
• CancellationRate
• Feedback,thumbsup/thumbsdown/neutral
• Positiveratingsin 90days
• Hoursworked
• Hoursworkedin 90days
• Extensions
• Extensionsin 90days

19. Jackson Square Ventures | Confidential | 19Jackson Square Ventures | Confidential | 19
Retain / Expel
This Step is Critical to Build a Virtuous Cycle
Reward great users – make their life better
Whostays active on the platform?
Who goes inactive?
Doesyour system naturallykeepthe best andexpel the worst?
Make itGreat –but only forGreat Users
It’sOk tohave a BadExperience for BadUsers – Segment!

20. Jackson Square Ventures | Confidential | 20Jackson Square Ventures | Confidential | 20
Retain / Expel – Case Study
Make Sure Great Users Stay, Kick out the Bad Ones
Keep iterating on great user retention
Rev
• Tiered worker system- necessary because ofsupply /demand imbalance. Best workers get promoted tothe highest Tier, Revver+
• Reliability (delivering on time andworking regularly), Tenure, Quality (customer feedback, low errorrate)
• Have as much workas they could possibly want –long term earning potential.
• Earnmore on aper hourbasis andhave moresteady work. Lower risk. Reduced competition on supply side.
• Promotions partiallybased on tenure. Very appealing to users. Feel appreciated.
• Engage in the online community
• Below ~4.5 stars, low utilization, high errorrates – no longer can get work.
Impact: Retention –Revver+ retention is ~5X higher than regular Revver. Be willing tospend lots ofmoney on Great User retention.
How not todoit?
99 Designs.

21. Jackson Square Ventures | Confidential | 21Jackson Square Ventures | Confidential | 21
Attract
Like acquires Like
Referrals from top quality users are a great signal.
Qualityattractsmore quality
Low Qualityattracts thesame
Greatathletes wanttoplayfor thebest team
Put greatnessondisplay
Hidetheugliness

22. Jackson Square Ventures | Confidential | 22Jackson Square Ventures | Confidential | 22
Attract – Case Study
Attraction is a flywheel
Needs to be nurtured, but it will accelerate
Outlier.org /Masterclass
Going fromzero to0.1 is brutal. But the virtuous cycle ofquality is getting started.
Phase 1
• Recipe forSuccess =Content Production Expertise (Academy AwardDirector) +Expert
• The Expertise makes the Expert Comfortable Joining
• Flywheel on Exceptional content
• Casting --> Creates Initial Customer Signals.
Phase 2
• James Patterson Serena Williams Usher Annie Liebovitz
• Put a Stake in the Ground. We will be the best.
• A-List Actor Didn’t make the Cut – Fame vs Mastery. (Screening)
• Social Proof Attached Names toa Project (producer, director, editor, etc)
Phase 3
• Gotta Deliver. Royalty Checks. Fuel the Flywheel
• Can’t pay someone todogreat work. They’re intrinsically motivated.
• Targeted Expert Acquisition. Lend Credibility to the Non-Famous Expert

23. Josh Breinlinger
josh@jsv.com
THANK YOU
QUESTIONS?