Your SlideShare is downloading. ×
  • Like

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Weather of the Century Part 3: Visualization

  • 307 views
Published

MongoDB natively supports geospatial indexing and querying, and it integrates easily with open source visualization tools. In this presentation, learn high-performance techniques for querying and …

MongoDB natively supports geospatial indexing and querying, and it integrates easily with open source visualization tools. In this presentation, learn high-performance techniques for querying and retrieving geospatial data, and how to create a rich visual representation of global weather data using Python, Monary, and Matplotlib.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
307
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
15
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A. Jesse Jiryu Davis #MongoDBWorld The Weather of the Century! Part III:! Visualization Senior Python Engineer, MongoDB
  • 2. Serious MongoDB Talk
  • 3. Serious MongoDB Talk Database
  • 4. Serious MongoDB Talk
  • 5. This Talk
  • 6. Where’s the data from?
  • 7. Where’s the data from?
  • 8. How Much Is There?
  • 9. Deployment
  • 10. Visualization
  • 11. Visualization Pipeline MongoDB PyMongo NumPy Matplotlib Python dicts SciPy
  • 12. import numpy! import pymongo! ! data = []! db = pymongo.MongoClient().my_database! ! for doc in db.collection.find(query):! data.append((! doc['position']['coordinates'][0],! doc['position']['coordinates'][1],! doc['airTemperature']['value']))! ! arrays = numpy.array(data)!
  • 13. # NumPy column access syntax.! lons = arrays[:, 0]! lats = arrays[:, 1]! temps = arrays[:, 2]!
  • 14. from scipy import griddata! from matplotlib import pyplot! ! xs = numpy.linspace(-180, 180, 361)! ys = numpy.linspace(-90, 90, 181)! zs = griddata(lats, lons, temps,! (xs, ys),! method='linear')! ! pyplot.contour(xs, ys, zs)! Magic!! Also magic!!
  • 15. from matplotlib import pyplot! ! xs = numpy.linspace(-180, 180, 361)! ys = numpy.linspace(-90, 90, 181)! zs = griddata(lats, lons, temps,! (xs, ys),! method='linear')! ! pyplot.contour(xs, ys, zs)!
  • 16. Triangulation
  • 17. Triangulation
  • 18. What temperature? Triangulation
  • 19. Barycentric Interpolation What temperature? 53 48 54 Weighted Average 51.1
  • 20. Interpolation 51.1
  • 21. Interpolation
  • 22. Interpolation
  • 23. Contours
  • 24. Contours
  • 25. import numpy! import pymongo! ! data = []! db = pymongo.MongoClient().my_database! ! for doc in db.collection.find(query):! data.append((! doc['position']['coordinates'][0],! doc['position']['coordinates'][1],! doc['airTemperature']['value']))! ! arrays = numpy.array(data)! Not terrifically fast
  • 26. Analyzing large datasets • Querying: 109k documents per second • (On localhost) • Can we go faster? • Enter“Monary”
  • 27. MongoDB PyMongo NumPy Matplotlib Python dicts MongoDB Monary NumPy Matplotlib Monary by David Beach
  • 28. import monary! ! data = []! connection = monary.Monary()! ! arrays = monary_connection.query(! db='my_database',! coll='collection',! query=query,! fields=['lon', 'lat', 'temp'],! types=[! 'float32', 'float32', 'float32'])!
  • 29. Monary • PyMongo: 109k documents per second • Monary: 817k documents per second
  • 30. Visualization
  • 31. {! ts: ISODate("1991-01-01T00:00:00Z"),! position: {! type: "Point",! coordinates: [! -94.6,! 39.117! ]! },! airTemperature: {! value: 45,! quality: "1"! }! }! Original Schema
  • 32. {! ts: ISODate("1991-01-01T00:00:00Z"),! lon: 39.117,! lat: -94.6,! temp: 45! }! Target Schema
  • 33. Future of Monary • Author:
 David Beach • Interns:
 Kyle Suarez
 Matt Cotter • Mentors:
 A.Jesse Jiryu Davis
 Jason Carey
  • 34. Future of Monary • Subdocuments: "airTemperature.value" • Aggregation cursor • Packaging • Bugfixes • Python 3
  • 35. Thanks • Monary • NumPy • SciPy • Matplotlib
  • 36. Thanks
  • 37. Thank you #MongoDBWorld A. Jesse Jiryu Davis Senior Python Engineer, MongoDB