Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Eventbrite Data Platform Talk foir SFDM

3,013 views

Published on

Slides for Eventbrite's data platform talk at SF data mining meetup.

Published in: Technology, Education

Eventbrite Data Platform Talk foir SFDM

  1. 1. Data PlatformVipul Sharma – vipul@eventbrite.com
  2. 2. A social event ticketing and discovery platform
  3. 3. $1B total sales 68M tickets sold 1.4M events hosted.5M organizers served23M attendees served 12 countries
  4. 4. Post Event ConceptionOrganization Event Lifecycle Creation Sale Discovery
  5. 5. Frictionless is the mantra!
  6. 6. Data Platform and Discovery
  7. 7. • SearchDiscovery • Recommendation • Social • Data warehouse and MetricsAnalytics • Internal and External reporting • Real Time and Batch Analytics Abuse • Spam • FraudPrevention • TOS
  8. 8. Analytics • Add–Hoc queries by Analysts
  9. 9. Fraud and Spam
  10. 10. Data Platform
  11. 11. Hadoop Cluster• 30 persistent EC2 High-Memory Instances• 30TB disk with replication factor of 2, ext3 formatted• CDH3• Fair Scheduler• HBase
  12. 12. Infrastructure• Search • Solr • Incremental updates towards event driven• Recommendation/Graph • Hadoop • Native Java MapReduce • Bash for workflow• Social • Cassandra • Denormalizedvview• Persistence • MySql • HDFS • HBase • MongoDB (Moving to Cassandra)
  13. 13. Infrastructure• Stream • RabbitMQ • Internal Fire hose • Storm• Offline • MapRedude • Streaming • Hive • Hue
  14. 14. DiscoverySocial, Interest, Local
  15. 15. Attendees EventsOrganizers
  16. 16. Categorization - Prism Tech Conference Music Sports
  17. 17. Prism - Features• Supervised Learning• Logistic Regression using MLE• Pair wise classification into 20 categories• High precision lower recall• Use mapreduce for feature extraction• Use for clustering as well
  18. 18. Prism – Training Data• Binary classification for each category• Training data needed for positive and negative • Conference and not Conference • Sports and not Sports• Samasource and Crowdflower• Stem words to create initial set• Positive, negative, negative with stem words
  19. 19. Prism - Features• Convert Event and Organizer data in feature vector• Event details, Organizer details, Ticket details• Boolean representation of predefined attributes • Words – tf-idf, dictonaries • Phrases • Domains • Rules – regular expression • Functions – business logic e.g. ticket price between $10-$20 • Compounds – boolean combination of features & and || rules – <COMPOUND1>:techcrunch& disrupt &techcrunch.com – <COMPOUND2>:COMPOUND2 && after && party
  20. 20. Prism - Features• Each feature is represented in various context • Event Title, Event Description, Organizer Title, Organizer Description• Each feature has meta info – Termclass • <LANG_EN>, <CONF_LANG_EN>,<ADULT_LANG_EN> • <SPORTS_LANG_EN>:<EVENT_TITLE>ball• Feature vector is represented as sparse vector+1 391158:1 401814:1 410526:1 411489:1 411606:2 413910:1 427659:1 438369:1 449735:1 449736:2 455478:1 456741:1 463188:1693|||||warrior spirits 3rd annual fundraising auction|||||1:<DESC>again,1:<NAME>annual,1:<DESC>annu al,2:<DESC>approaching,2:<NAME>auction,4:<DESC>auctio n,2:<DESC>auctions,2:<DESC>bring
  21. 21. Prism - Training• Binary classifier• Multiclass less accurate• Each event get classified into 20 category• MapReduce for creating sparse matrix• MapReduce for batch classification • Distributed cache for feature set and models• We can use same sparse matrix for clustering
  22. 22. Attendee• What your interests are? - Prism• Who your friends are? – Explicit and Implicit• What are the interests of your friends? - Prism• Which of your friend have your interests? – IBG• Location of users and events • Purchase events location • Facebook location • Our database • Other signals – ip, mobile app etc
  23. 23. You will like to attend this event
  24. 24. Recommendation Engines Interest Graph Based Social Graph Based (Your (Your friends who friends like Lady like rock music Collaborative Gaga so you will like you are Filtering – Item- like Lady Gaga, attending Eric Item similarity PYMK – Facebook, Clapton Event– Linkedin) Eventbrite) Collaborative (You like Filtering – User- Godfather so you User Similarity will like Scarface - Netflix) (People who Item bought camera Hierarchy also bought batteries - (You bought Amazon) camera so you need batteries - Amazon)
  25. 25. Why Interest? Events are Social Events are InterestDense Graph is Irrelevant Interest are Changing
  26. 26. How do we know your Interest?• We ask you• Based on your activity • Events Attended • Events Browsed (In Future)• Facebook Interests • User Interest has to match Event category • Static• Prism
  27. 27. Model Based vs Clustering Item-Item vs User-User Building Social Graph is Clustering StepSocial Graph Recommendation is a Ranking Problem
  28. 28. Implicit Social Graph U1 E1 E4 U2 U3 E2 E3 U4 U5
  29. 29. Mixed Social Graph U1 E1 U2 U3 E2 E3 FB U4 U5 LI
  30. 30. 23M * 260 * 260 = 1.5 Trillion Edges 6 Billion edges ranked Each node is a feature vector representing a UserEach edge is a feature vector representing a Relationship
  31. 31. Feature Generation• Mixed Features• A series of map-reduce jobs• Output on HDFS in flat files; Input to subsequent jobs• Orders = Event  Attendees • MAP: eid: uid • REDUCE: eid:[uid]• Attendees  Social Graph • Input: eid:[uid] • MAP: uidi:[uid] • REDUCE: uid:[neighbors]• Interest based features, user specific, graph mining etc• Upload feature values to HBase
  32. 32. HBase• Why Hbase? • To process 6B edges lookup features for each node and each edge • 6B/1000 /86400 = 70 days!! • 1M/sec = 1.5 hrs • Processing 1.3 TB of data with mapreduce• Collect data from multiple Map Reduce jobs • Stores entire social graph • Features for each node and edge
  33. 33. Data Model Rowkey U UUuid1 f1 f2 f3 uid2:f4 uid2:f5 uid3:f4rowid neighbors events featureX2718282 101 3 0.3678795rowid 314159:n 314159:e 314159:fx 161803:n 161803:e 161803:fx2718282 31 1 0.3183 83 2 0.618
  34. 34. U1U2 U3
  35. 35. HBase
  36. 36. Hadoop Tips & Tricks• Joins • Distributed cache • Hive map side joins• Hive • Nice set of statistical functions • Lots of hive queries• Hbase • Lots of memory • WAL • LZO • Proper configs • Avoid hot regioservers
  37. 37. Hadoop tips & tricks• Combiners did not work• Shuffle and Merge
  38. 38. More Innovation• Rethink everything• Add social to search• Add time series features• Real time updates using firehose and storm• Various sorts of data
  39. 39. Developers! Developers! Developers!• Interested in scaling, messaging, data, machine learning, mobile, services• We will continue to push the boundaries of hard problems• jobs@eventbrite.com• vipul@eventbrite.com
  40. 40. Storm at EventbriteTuesday August 21, 2012 at Eventbrite HQHow we are using Storm for real time processing of our datahttp://www.eventbrite.com/event/4010290888 Andrew Whangwhang@eventbrite.co m
  41. 41. Questions?

×