Activate 2019 - Search and relevance at scale for online classifieds
•
0 likes•267 views
A high performing search service implies both having an effective search infrastructure and high search relevance. Seeking for a fault tolerant, self-healing and cost-effective search infrastructure at scale, we built a platform based on Apache Solr search engine with light in-memory indexes, avoiding sharding and decreasing the overall infrastructure needs. To populate the indexes, we use flexible ETL processes, keeping our product catalog and search indexes updated in a near real-time fashion and distributed across high-performant database engines. We aim at getting a high search relevance precision and recall by applying query relaxation and boost solutions on top of the optimised platform. https://www.activate-conf.com/speakers/detail/roger-rafanell
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Activate 2019 - Search and relevance at scale for online classifieds
Similar to Activate 2019 - Search and relevance at scale for online classifieds (20)
More from Roger Rafanell Mas
More from Roger Rafanell Mas (13)
Recently uploaded
Recently uploaded (20)
Activate 2019 - Search and relevance at scale for online classifieds
- 1. SEARCH AND RELEVANCE AT SCALE FOR ONLINE CLASSIFIEDS
- 2. STAY CONNECTED Twitter @activate_conf Facebook @activateconf #Activate19 Log in to wifi, follow Activate on social media, and download the event app where you can submit an evaluation after the session WIFI NETWORK: Activate2019 PASSWORD: Lucidworks DOWNLOAD THE ACTIVATE 2019 MOBILE APP Search Activate2019 in the App/Play store Or visit: http://crowd.cc/activate19
- 3. ROGER RAFANELL Senior Big Data Engineer | letgo About me
- 4. ?
- 5. letgo • Second-hand marketplace app • Founded 2015 • Main markets: US & Turkey • 5M downloads/month & 20M MAU
- 6. Agenda • Introduction to search in classifieds • search in the past • Building a new search platform at scale • Enabling data science • The future of search platform
- 7. Introduction
- 8. Introduction Search in classifieds 100K items/day Reposted We cannot cache results!!! Sold / Deleted
- 9. Introduction Hyperlocality search letgo catalog at (39.89,-77.08) letgo catalog at (39.28, -76.69) Results in Washington, D.C. ≠ Results in Baltimore
- 10. Introduction User input data example • Typos • Slang • Poor pictures • Wrong information • Ambiguity • Weirdness
- 11. Introduction Explorers vs Deal Seekers (Marco Polo vs Hernán Cortés) • Like browsing • Recall > Precision • “Cars” • Search, filter, haggle • Precision > Recall • “2015 honda civic lx”
- 12. search in the past
- 13. Search in the past Early 2015 Listings API Search API
- 14. Search in the past Late 2017 Shard 1 ... 8 replicas / shard x 3 Shard 1 Shard 5 ... 8 replicas / shard x 2 x 1 (↑nodes) (↑nodes) (↑nodes) Listings API Search API Shard 5
- 15. 24hFULL INDEXATION 8hINSTANCES RECOVERY 150msRESPONSE TIME Operation limitations • Slow full catalog imports • Slow reactivity to traffic spikes • High costs
- 16. Business limitations • No enrichment at import time • Not easy to evolve schemas • Not agile! NOA/B TESTING NODATA SCIENCE
- 17. Search API limitations • One API request -> One search query • PHP + Solarium (↓ concurrency) • High costs Search API 200rpsTHROUGHPUT 400msRESPONSE TIME 60+SERVICE INSTANCES
- 18. The platform was not scaling
- 19. Building a new search platform
- 20. • Spot oldest queries sent by search API • ↑Traffic for fresh listings • All fields were stored Building a new search platform Analysis 3 monthsCATALOG RETENTION 15 minHIGHLY REQUESTED LISTINGS
- 21. • Keep only the last 3 month listings • Index only the queried fields • Store only listings IDs Building a new search platform Looking for a strategy >100GBOLD CATALOG SIZE <4GBNEW CATALOG SIZE
- 22. Solr was used as a key-value storage NOT as a full-text search engine
- 23. Building a new search platform THE BAD • Where to store all listings fields? • Need a catalog storage (database) • Need also a fast serving layer • Near real-time indexing constraints THE GOOD • No more sharding (↓index size) • Standalone Solr instances • High bump in performance Drawing a plan
- 24. Building a new search platform Big Data to the rescue • NRT pipeline to keep the listings catalog up-to-date • Batch pipeline to fully rebuild the catalog
- 25. Building a new search platform The new architecture Self-healing
- 26. Building a new search platform The Search indexer ETL Fetch Listings Enrich Listings Fetch Verticals Features Normalize Attributes Anonymize PII Store to DB Store to Fast Layer
- 27. Building a new search platform Search engine performance Throughput Recovery time Latency ↑12x ↓8x12’
- 28. Building a new search platform Catalog performance Catalog (Fast layer) Catalog (Database) Worst Case Latency 16ms 56ms40ms
- 29. Gluing all the pieces
- 30. Building a new search platform Search API redesign x 1 x 1 x 1 Listings API Search Library Search API IDs
- 31. Building a new search platform Search library - Scala to rule them all • Wrap the search retrieval logic • One request → Multiple parallel queries to Solr • Non-blocking I/O with solrS, persistence drivers • Seamless integration with Finagle framework
- 32. Building a new search platform Search API - Scala to rule them all • Based on Finagle services framework • Finatra/Finagle = ↑concurrency & ↓resources • Enable backend driven A/B testing • Personalized search
- 33. Building a new search platform Overall performance ↑Throughput &↓ Latency Resources Cost Reduction 13x 100x↓20x
- 34. Enabling data science SEARCH & RELEVANCE
- 35. Enabling data science Unlocked data science projects • Recall – Query expansion • Precision – Learning to Rank
- 36. Enabling data science Improving recall - Query expansion Searching for: ‘mountain bike’ blue mountain bicycle → Synonyms mountain and road bike → OK mountain bike frame → Relevant? bicicleta de montaña → Language scout montain bike → Spelling mountain bike lock → Relevant? Similar Queries Cause blue mountain bicycle mountain and road bike mountain bike frame bicicleta de montaña scout montain bike mountain bike lock Expected Behavior
- 37. Enabling data science Improving precision - Learning to Rank ‘mountain bike’ Items retrieved
- 38. Enabling data science ‘mountain bike’ 2 months ago 30 miles Improving precision - Learning to Rank
- 39. Enabling data science Improving precision - Learning to Rank
- 40. Enabling data science bike 1 2 3 4 5 6 Improving precision - Learning to Rank y = 0 y = 0 y = 1
- 41. Enabling data science Conversions on query ‘bike’ Improving precision - Learning to Rank
- 42. Enabling data science Before After SEARCH CONVERSIONS Improving precision - Learning to Rank
- 43. Text score = indicator of relevance. Freshness and distance are key!
- 44. The future of search platform
- 45. Future of search platform Work in progress • Migration to Solr 8 (↓latency & better security) • Iterate Learning to Rank • Real-time personalization • Visual categorization (Reveal)
- 47. Conclusions
- 48. Conclusions Raising the bar • Indexer pipeline enables data enrichment & transformations • Simplified search architecture with lightweight in-memory indices • Fault-tolerant and self-healing infrastructure and processes • Unlock real data science in