MongoDB Helps You Find Your Way
Upcoming SlideShare
Loading in...5
×
 

MongoDB Helps You Find Your Way

on

  • 1,455 views

Where do you get location data and why should you store it in MongoDB? Presented at Mongo Austin on 2/15/11.

Where do you get location data and why should you store it in MongoDB? Presented at Mongo Austin on 2/15/11.

Statistics

Views

Total Views
1,455
Views on SlideShare
1,455
Embed Views
0

Actions

Likes
1
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

MongoDB Helps You Find Your Way MongoDB Helps You Find Your Way Presentation Transcript

  • MongoDB Helps You Find Your Way Mongo Austin February 15, 2011 Sandeep Parikh Argia, Inc.
  • Why Is Location Data Special?• Location information can be encoded in multiple ways• Location data can be modeled in multiple ways• Challenges associated with querying and indexing
  • RegionsLabels Address, Zip, Neighborhood, Place NamePoints Types of Data
  • Data SourcesSocial Messages Checkin Providers Tagged-Photos Reviews Property Listings Crime Reports
  • How You Get It• Most services offer simple REST-ish APIs • Know your rate limits • Respect API rules and data ownership• Authentication typically via OAuth or OAuth2• Usually encoded in JSON (+1)
  • How You Use It• Pull data down, dump it directly into MongoDB• Do some initial analysis with simple queries• Once you know more about the data, update your schema
  • Considerations• Data comes in a pre-defined structure• Location information may not be in the “right” place• What’s the motivating question? • Pull data and store it to answer that question• And...
  • G Remember to phrase your question in theform of an API request
  • { Spot The Location Data "_id" : ObjectId( "4d33f3e2421aa92b1200000f" ), "activities" : [ { "/checkins/26641957" : { "activity_url" : "/checkins/26641957/activity", "_photos_count" : 0, "url" : "/checkins/26641957", "_comments_count" : 0, "spot" : { "name" : "Half Price Books", "url" : "/spots/19928", "city_state" : "Austin, TX", "image_url" : "http://static.gowalla.com/...", "lat" : "30.4201410833", "lng" : "-97.7017284167", "description" : null }, "created_at" : "2011-01-15T23:17:38Z", "item" : null, "photo_thumbnail_urls" : [], "type" : "checkin", "message" : "Selling books, VHS tapes, and vinyl" } }, ]}
  • { Spot The Location Data "_id" : ObjectId( "4d33f3e2421aa92b1200000f" ), "activities" : [ { "/checkins/26641957" : { "activity_url" : "/checkins/26641957/activity", "_photos_count" : 0, "url" : "/checkins/26641957", "_comments_count" : 0, "spot" : { "name" : "Half Price Books", "url" : "/spots/19928", Way down there... "city_state" : "Austin, TX", "image_url" : "http://static.gowalla.com/...", "lat" : "30.4201410833", "lng" : "-97.7017284167", "description" : null }, "created_at" : "2011-01-15T23:17:38Z", "item" : null, "photo_thumbnail_urls" : [], "type" : "checkin", "message" : "Selling books, VHS tapes, and vinyl" } }, ]}
  • Why MongoDB?• MongoDB natively supports 2D geospatial indexing • Find closest N items• Geospatial indexes can be combined with secondary indexes (ex. item categories) • Find closest N “museums”
  • Latitude & Longitude• As an array or dict of 2 elements• Array: store as lat/ long or long/lat (just be consistent!)• Dict: or with your own keys
  • Querying• db.collection.ensureIndex({loc:”2d”})• $near• geoNear (via runCommand) • Also returns item distance from point• $within • $box (specify lower-left, upper-right) • $circle (specify center, radius)
  • Get Shape Creative(and then vote up SERVER-772)
  • Get Shape Creative(and then vote up SERVER-772)
  • More Geo in 1.8.0• Currently in development (as 1.7.x), RC coming soon• Spherical queries which treat Earth as an ellipsoid (it is!) • Assumes you’ve stored location as [latitude,longitude] [longitude,latitude]• Support for geospatial indexes in sharded collections
  • What Can We Learn About Austin?
  • More Specifically• Where do people from Austin checkin, in Austin? • Which bars? • Which coffee shops? • Where do they shop?
  • What Do We Need?• Users, Spots and Checkins• Available from Gowalla (local!)• Python to • Pull the data down • Analyze the structure and content
  • What Did We Get?• 6,600 local spots• 36,000 users• 2,700 local users• 1,781,417 checkins• 238,000 local checkins from locals• 4GB of data
  • Top Places in Austin 4000AUS Austin-BergstromWhole FoodsAustin Convention Center 3000University of TexasAlamo Drafthouse CinemaMutual Mobile 2000The HiltonStubbs Bar-B-QThe Flying Saucer 1000The Driskill 0 238,674 local checkins by locals
  • When Do People Checkin? 50000 37500Checkins 25000 12500 0 0 2 4 6 8 10 12 14 16 18 20 22 Time of Day
  • Top Coffee Shops Spider House Halcyon 60 45Checkins 30 15 0 0 3 6 9 12 15 18 21 Time of Day
  • Top Grocery Stores Whole Foods Central Market-North Lamar 300 225Checkins 150 75 0 0 3 6 9 12 15 18 21 Time of Day
  • Questions? Sandeep Parikh, Argia, Inc @crcsmnky @argiainfosparikh@argiainc.com www.argiainc.com