This document discusses using PixieDust to create data visualizations and maps in Jupyter notebooks. PixieDust allows users to easily create visualizations with a simple display() function call and integrate data from notebooks with cloud services. Features highlighted include the package manager, various visualization options, integration with Scala, custom visualization creation, and embedding apps in notebooks. The document concludes with references for further information.

1. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Mapping Data in Jupyter
Notebooks with PixieDust
Margriet Groenendijk & Raj Singh
GeoPython - 10 May 2017 - Basel, Switzerland

2. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
from mpl_toolkits.basemap import Basemap
from matplotlib.offsetbox import AnnotationBbox, OffsetImage
from matplotlib._png import read_png
from itertools import izip
matplotlib.style.use('bmh')
fig, axes = plt.subplots(nrows=1, ncols=2,
figsize=(10, 12))
# background maps
m1 = Basemap(projection='mill',resolution=None,
llcrnrlon=-7.5, llcrnrlat=49.84,urcrnrlon=2.5,
urcrnrlat=59,ax=axes[0])
m1.drawlsmask(land_color='dimgrey',
ocean_color='dodgerBlue',lakes=True)
# temperature map
for [temp,city] in izip(temps,cities):
lat = city[1]
lon = city[2]
if temp>8:
col='indigo'
...
elif temp>4:
col='turquoise'
x1, y1 = m2(lon,lat)
bbox_props = dict(boxstyle="round,pad=0.3",
fc=col, ec=col, lw=2)
axes[1].text(x1, y1, temp, ha="center", va="center",
size=11,bbox=bbox_props)
plt.tight_layout()
https://www.flickr.com/photos/mjtmail/2170840714

3. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
PixieDust with Mapbox

4. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Watson Data Platform
Weather Machine Learning Watson APIsTwitter Geo

5. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

6. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

7. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
IBM Data Science Experience + Project
Jupyter

8. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

9. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr https://medium.com/ibm-watson-data-lab

10. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Jupyter + Pixiedust =
1. PackageManager
2. Visualizations
3. Cloud Integration
4. Scala Bridge
5. Extensibility
6. Embedded Apps

11. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Package Manager
Uses the GraphFrame Python APIs
Install GraphFrames Spark Package
Install Spark packages or plain jars

12. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Visualizations One simple API: display()
Call the Options dialog
Panning/Zooming
options
Performance statistics

13. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Cloud Integration
Easily export your data to csv, json, html, etc. locally on your laptop
or into a cloud-based service like Cloudant or Object Storage

14. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Scala Bridge
Execute Scala code directly from your python Notebook
Define a Python variable
Use the Python var in Scala
Define a Scala variable
Use the Scala var in Python

15. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Extensibility
Easily extend PixieDust to create your own visualizations using
HTML/CSS/JavaScript
Customized
visualisation for
GraphFrame
Graphs

16. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Embed Apps in Notebooks
PixieApps encapsulate analytics into lightweight HTML UIs

17. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
PixieDust demo
https://apsportal.ibm.com/analytics/notebooks/
e76cad7a-30f7-493f-9ea0-7e14be178053/view?
access_token=acc140772a62646df3bd2987ca4a8463c0f8169a
02c9fe6bafc0311fadca85f2

18. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

19. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
How it all works
• Spark DataFrame -> GeoJSON
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• Get bin cutoff points for quantiles
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• Create choropleth styling JSON
• /display/chart/renderers/mapbox/mapBoxMapDisplay.py
• GeoJSON data and styling JSON => Jinja2 template
• /display/chart/renderers/mapbox/templates/mapView.html
• Render template inside an <iframe> inside the cell
• /display/chart/renderers/mapbox/templates/iframesrcdoc.html
• Call Mapbox basemapping service for streets underlay
• /display/chart/renderers/mapbox/templates/mapView.html

20. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

21. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
Where next with PixieDust?
• Polygon support?
• Grid/raster support?
• More cartographic options?
• Animated temporal visualization?
• arcgis?
• Bokeh maps?
• Folium?
• … any visualisation you want!

22. Register now!
seti.org/ML4SETI
from The SETI Institute
Hackathon
Galvanize, San Francisco
June 10-11, 2017
Code Challenge
June/July, 2017

23. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr

24. © 2017 IBM Corp.
Watson Data
Platform@MargrietGr
References
• IBM Data Science Experience - https://datascience.ibm.com
• Pixiedust - https://github.com/ibm-cds-labs/pixiedust
• Project Jupyter - http://jupyter.org/
• Slides - https://www.slideshare.net/MargrietGroenendijk/presentations
• Blog - https://medium.com/ibm-watson-data-lab
• Me - mgroenen@uk.ibm.com - @MargrietGr