The Weather of the Century Part 3: Visualization
•
1 like•1,423 views
MongoDB natively supports geospatial indexing and querying, and it integrates easily with open source visualization tools. In this presentation, learn high-performance techniques for querying and retrieving geospatial data, and how to create a rich visual representation of global weather data using Python, Monary, and Matplotlib.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The Weather of the Century Part 3: Visualization
Similar to The Weather of the Century Part 3: Visualization (20)
More from MongoDB
More from MongoDB (20)
Recently uploaded
Recently uploaded (20)
The Weather of the Century Part 3: Visualization
- 1. A. Jesse Jiryu Davis #MongoDBWorld The Weather of the Century! Part III:! Visualization Senior Python Engineer, MongoDB
- 5. This Talk
- 6. Where’s the data from?
- 7. Where’s the data from?
- 8. How Much Is There?
- 9. Deployment
- 10. Visualization
- 11. Visualization Pipeline MongoDB PyMongo NumPy Matplotlib Python dicts SciPy
- 12. import numpy! import pymongo! ! data = []! db = pymongo.MongoClient().my_database! ! for doc in db.collection.find(query):! data.append((! doc['position']['coordinates'][0],! doc['position']['coordinates'][1],! doc['airTemperature']['value']))! ! arrays = numpy.array(data)!
- 13. # NumPy column access syntax.! lons = arrays[:, 0]! lats = arrays[:, 1]! temps = arrays[:, 2]!
- 15. from scipy import griddata! from matplotlib import pyplot! ! xs = numpy.linspace(-180, 180, 361)! ys = numpy.linspace(-90, 90, 181)! zs = griddata(lats, lons, temps,! (xs, ys),! method='linear')! ! pyplot.contour(xs, ys, zs)! Magic!! Also magic!!
- 17. from matplotlib import pyplot! ! xs = numpy.linspace(-180, 180, 361)! ys = numpy.linspace(-90, 90, 181)! zs = griddata(lats, lons, temps,! (xs, ys),! method='linear')! ! pyplot.contour(xs, ys, zs)!
- 18. Triangulation
- 19. Triangulation
- 21. Barycentric Interpolation What temperature? 53 48 54 Weighted Average 51.1
- 23. Interpolation
- 24. Interpolation
- 25. Contours
- 26. Contours
- 33. import numpy! import pymongo! ! data = []! db = pymongo.MongoClient().my_database! ! for doc in db.collection.find(query):! data.append((! doc['position']['coordinates'][0],! doc['position']['coordinates'][1],! doc['airTemperature']['value']))! ! arrays = numpy.array(data)! Not terrifically fast
- 34. Analyzing large datasets • Querying: 109k documents per second • (On localhost) • Can we go faster? • Enter“Monary”
- 35. MongoDB PyMongo NumPy Matplotlib Python dicts MongoDB Monary NumPy Matplotlib Monary by David Beach
- 36. import monary! ! data = []! connection = monary.Monary()! ! arrays = monary_connection.query(! db='my_database',! coll='collection',! query=query,! fields=['lon', 'lat', 'temp'],! types=[! 'float32', 'float32', 'float32'])!
- 37. Monary • PyMongo: 109k documents per second • Monary: 817k documents per second
- 38. Visualization
- 39. {! ts: ISODate("1991-01-01T00:00:00Z"),! position: {! type: "Point",! coordinates: [! -94.6,! 39.117! ]! },! airTemperature: {! value: 45,! quality: "1"! }! }! Original Schema
- 40. {! ts: ISODate("1991-01-01T00:00:00Z"),! lon: 39.117,! lat: -94.6,! temp: 45! }! Target Schema
- 41. Future of Monary • Author: David Beach • Interns: Kyle Suarez Matt Cotter • Mentors: A.Jesse Jiryu Davis Jason Carey
- 42. Future of Monary • Subdocuments: "airTemperature.value" • Aggregation cursor • Packaging • Bugfixes • Python 3
- 43. Thanks • Monary • NumPy • SciPy • Matplotlib
- 44. Thanks
- 45. Thank you #MongoDBWorld A. Jesse Jiryu Davis Senior Python Engineer, MongoDB