Machine Learning andBig Data at FoursquareBlake Shaw, PhDData Scientist @ Foursquare@metablake
What is foursquare?      An app that helps you      explore your city and      connect with friends      A platform for lo...
What is foursquare?      People use foursquare to:       •   check in to places       •   discover new places       •   sh...
What is foursquare?Mobile               Social          Local
Stats10,000,000+ people25,000,000+ places1,000,000,000+ check-ins10,000+ actions/second
Growth
Growth
Growth
Learning with location data• Check-ins are a rich source of data that describe human behavior• We apply machine learning a...
Recommendation engine• foursquare explore provides realtime recommendations using:  •   location  •   time of day  •   che...
Signals about places
Networks of people
Networks of peopleBrooklyn            Manha-an                                SF                Australia
Open questions• How to measure similarity between people and places?• How to determine influence in large networks of peop...
Our data stack• MongoDB• Amazon S3, Elastic Mapreduce• Hadoop• Hive• Flume• R and Matlab
Join us!foursquare is hiring!85+ people and growingfoursquare.com/jobsBlake Shaw@metablakeblake@foursquare.com
Nyas ml-4sq-2
Upcoming SlideShare
Loading in …5
×

Nyas ml-4sq-2

218
-1

Published on

Published in: Technology, Spiritual
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
218
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • At foursquare, we think there is a great opportunity to leverage massive amounts of location data to help people better understand and connect to places\n
  • So, what is foursquare? It’s an app that help you explore your city and connect with friends.\n\nIt’s also a platform for people to build location based services and collect and share location data\n
  • People on foursquare “check-in” on their phones when they get to a place, to find out more about it, share that they are there with friends etc.\n
  • Foursquare is in a unique place, sitting at the intersection between mobile, social, and geo.\n\n\n
  • Foursquare is generating a ton of data, every second 35 people check-in to a location. \n\nThis data offers an unprecedented view into the behavior of millions of people worldwide, as they move around cities.\n\n\n
  • Here we see the growth of the service over the last two years since, it started in mid 2009\n
  • \n
  • Foursquare now has data on over 25 million places all over the world\n
  • Check-ins are a rich source of information describing human behavior.\n\nWe apply machine learning algorithms to the collective movement patterns of millions of people to build exciting new services.\n\nWe use a variety of ML algorithms, collaborative filtering, pagerank, clustering, classification and regression\n\n\n
  • For example, Last year we launched foursquare explore. A recommendation engine that uses a variety of signals to recommend places in real time that a user might be interested in.\n\nExplore uses a variety of machine learning models to rank venues, we combine many signals, including:\n\nthe location of the user, and the time of day\nthe persons past check-in history,\nthe places their friends check-in\nthe similarities between different venues\n\n
  • Consider these signals about places. Each place has a different signature based on who is coming to the place, when, and for how long.\n\nThis plot shows 3 different places:\n\nGorilla Coffee, Gray’s Papaya, Amorino (a restaurant)\n\nSee how gorilla coffee is busy more in the morning, where amorino is busy in the evening.\n\nGray’s papaya clearly has a strong lunch crowd, but also a late night peak on the weekends.\n\nHow can we use machine learning to learn from these signals which places are similar?\n\n\n
  • We also have unique signals that describe people,\n\nWhich people are friends. Who is checking in together. Etc\n\nFrom checkins we can build a large colocation network that can be used to better understand how people interact with each other in the real world.\n\nHere we see an example of graph embedding to the foursquare employee network. People are placed near each other in 2D if they often colocate at similar places.\n\n\n\n
  • Different parts of this map line up to the different places in the world where foursquare employees live.\n\nThis plot was made by applying minimum volume embedding, a non-linear graph based dimensionality reduction algorithm, to the foursquare employee network.\n\nEach person on this map can be described by thousands of numbers, showing how often the visit different places. The goals is to reduce the dimensionality of this space to 2D while preserving the strong pairwise relationships.\n\n\n\n
  • We are constantly considering the best ways to address many of these questions\n\n
  • All of this is possible because of our world-class data stack. Amazon S3 and EC2 allow us access to on-demand access to huge computational resources\n\n\n
  • Thanks so much.\n\nFoursquare is hiring, if these projects seem interesting to you, please contact us at foursquare.com/jobs\n
  • afinn sentiment analysis word list\n
  • Friend graph for marriage equalitiocalypse, who are friends who checked in at this event\n
  • Nyas ml-4sq-2

    1. 1. Machine Learning andBig Data at FoursquareBlake Shaw, PhDData Scientist @ Foursquare@metablake
    2. 2. What is foursquare? An app that helps you explore your city and connect with friends A platform for location based services and
    3. 3. What is foursquare? People use foursquare to: • check in to places • discover new places • share w/ friends • get tips about places • get deals • earn points and badges
    4. 4. What is foursquare?Mobile Social Local
    5. 5. Stats10,000,000+ people25,000,000+ places1,000,000,000+ check-ins10,000+ actions/second
    6. 6. Growth
    7. 7. Growth
    8. 8. Growth
    9. 9. Learning with location data• Check-ins are a rich source of data that describe human behavior• We apply machine learning algorithms to the collective movement patterns of millions of people to build exciting new services
    10. 10. Recommendation engine• foursquare explore provides realtime recommendations using: • location • time of day • check-in history • friends preferences
    11. 11. Signals about places
    12. 12. Networks of people
    13. 13. Networks of peopleBrooklyn Manha-an SF Australia
    14. 14. Open questions• How to measure similarity between people and places?• How to determine influence in large networks of people and places?• What statistics can we use to describe people’s behavior in the real-world?• How do we predict what information will be
    15. 15. Our data stack• MongoDB• Amazon S3, Elastic Mapreduce• Hadoop• Hive• Flume• R and Matlab
    16. 16. Join us!foursquare is hiring!85+ people and growingfoursquare.com/jobsBlake Shaw@metablakeblake@foursquare.com

    ×