Presentation from WebClusters 2012 conference

1. Geoindexing
with MongoDB
Leszek Krupiński
WebClusters 2012

2. About me

3. On-line since 1997

4. Funny times

5. 1 hr of internet for 1 USD

8. First social site:
geocities

9. My first web page

10. What do I do now

11. Day-time job
Managing team of developers for Polish Air Force

12. Side:
consulting, optimizing, de
siging

13. Buzzwords incoming!

14. The Internet
2008

15. Web 2.0

16. http://en.wikipedia.org/wiki/File:Web_2.0_Map.svg CC-BY-SA-2.5

17. Be social in your bedroom

18. alone.

19. The Internet
2012

20. Web 3.0

22. Why geospatial?

23. Needs shifted

24. Why?
Because they could.

28. How to implement?

29. Database. Duh.

30. Keep, but also query

31. Is there a person at 53.438522,14.52198?
Nope.
Is there a person at 53.438522,14.52199?
Nope.
Is there a person at 53.438522,14.52199?
Yeah, here’s Johnny!

32. Not too useful.

33. Give me nearby homies.
Within the range of 1 km
there is:
• Al Gore (53.438625,14.52103)
• Bill Clinton (53.432531,14.55127)
• Johnny Bravo (53.438286,14.52363)

34. Now that’s better.

35. Geoindexing.
Nothing new.

36. Oracle, PostreSQL,
Lucene/Solr, even MySQL
(via extensions)

37. SELECT c.holding_company, c.location
FROM competitor c,
bank b
WHERE b.site_id = 1604
AND SDO_WITHIN_DISTANCE(c.location,
b.location,
’distance=2 unit=mile’) = ’TRUE’
ORACLE

38. SQL is so last year

39. Let’s use something cool

40. MongoDB.
Because all the cool kids
use NoSQL now

42. Why MongoDB?

43. Choose your NoSQL wise.

44. NoSQL in MongoDB
• Document –based
• Queries (JS-like syntax)
• JSON-like storage

45. Why MongoDB?
Features Use Cases
• Ad hoc queries • Archiving
• Indexing • Event logging
• Replication • Document and CMS
• Load Balancing • Gaming
• File Storage • High volume sites
• Aggregation • Mobile
• Server-side JavaScript • Operational datastore
• Capped collections • Agile development
• Real-time stats
http://en.wikipedia.org/wiki/Mongodb

46. Back to geo.

47. {
loc: [ 52.0, 21.0 ],
name: ”Warsaw”,
type: ”City”
}

48. db.nodes.ensureIndex({loc: '2d'})

49. That’s it.

50. Query
• Exact
o db.places.find( { loc : [50,50] } )
• Near
o db.places.find( { loc : { $near : [50,50] } } )
• Limit
o db.places.find( { loc : { $near : [50,50] } } ).limit(20)
• Distance
o db.places.find(
{ loc : { $near : [50,50] , $maxDistance : 5 } }
).limit(20)

51. Compound index
• db.places.ensureIndex(
{ location : "2d" , category : 1 }
);
• db.places.find(
{
location : { $near : [50,50] },
category : 'coffee‚
}
);

52. Bound queries
• box = [
[40.73083, -73.99756],
[40.741404, -73.988135]
]
• db.places.find(
{"loc" :
{"$within" : {"$box" : box }}
}
)

53. Problems

54. Units

55. Coordinates in arc units
Distance in kilometers

56. In query

57. earthRadius = 6378 // km
multi = earthRadius * PI / 180.0
range = 3000 // km
… maxDistance : range * multi…

58. In results

59. pointDistance =
distances[0].dis / multi

60. Earth is not flat.

61. Problem: can’t use linear
distance

62. Earth isn’t flat too.

63. Solution?
Use approximation.

64. MongoDB has it built-in
distances = db.runCommand(
{ geoNear : "points",
near : [0, 0],
spherical : true,
maxDistance : range / earthRadius /* to radians */
} ).results

65. Focus: runCommand
distances = db.runCommand({ geoNear : "points" …

66. Sort by distance
Only with runCommand

67. Automatically sorted
• db.runCommand(
{ geoNear : "places" , near : [50,50], num : 10 }
);
• {
"ns" : "test.places",
"results" : [
{ "dis" : 69.29646421910687, "obj" : … },
{ "dis" : 69.29646421910687, "obj" : … },
…
],
…
}

68. Demo

69. OpenStreetMaps database
of Poland imported into
MongoDB

70. 14.411.552 nodes

71. 3GB of raw XML data

72. PHP in virtual machine

73. Imported about 100.000
nodes every 10s.

74. Pretty cool, eh?

75. Kudos to Derick Rethans
Part of this talk was inspired by his talk

76. Questions?

77. Thanks!
Rate me at https://joind.in/talk/view/6475

78. Geoindexing
with MongoDB
supplement
Leszek Krupiński
WebClusters 2012

79. Why MongoDB?

80. Evaluate.

81. PostGIS is cool too.
(but it’s SQL, meh)

82. Why MongoDB?
Features Use Cases
• Ad hoc queries • Archiving
• Indexing • Event logging
• Replication • Document and CMS
• Load Balancing • Gaming
• File Storage • High volume sites
• Aggregation • Mobile
• Server-side JavaScript • Operational datastore
• Capped collections • Agile development
• Real-time stats
http://en.wikipedia.org/wiki/Mongodb

83. If you need other features
of MongoDB, use it

84. If you don’t, evaluate.

85. Evaluate.

86. Demo
(hopefully)

87. Questions?

88. Please leave feedback!
Rate me at https://joind.in/6475