PixieDust overview

1. ©2016 IBM Corporation IBM Data Science Experience
PixieDust: an Open Source
Library that simplifies and
improves Jupyter Python
Notebooks

2. ©2016 IBM Corporation IBM Data Science Experience
PixieDust: an Open Source Library that simplifies and
improves Jupyter Python Notebooks
Jupyter + Pixiedust =
1. PackageManager
2. Visualizations
3. Cloud Integration
4. Scala Bridge
5. Extensibility
6. Embedded Apps
https://github.com/ibm-cds-labs/pixiedust

3. ©2016 IBM Corporation IBM Data Science Experience
1/6 - Package Manager
Install Spark packages or plain jars in your
Notebook Python kernel without the need to
modify configuration file
Install GraphFrames Spark Package
Uses the GraphFrame Python APIs

4. ©2016 IBM Corporation IBM Data Science Experience
2/6 - Visualizations
Call the Options dialog
Performance statistics
Panning/Zooming
options
One simple API: display()

5. ©2016 IBM Corporation IBM Data Science Experience
3/6 - Cloud Integration
Easily export your data to csv, json, html, etc.
locally on your laptop or into a cloud-based
service like Cloudant or Object Storage

6. ©2016 IBM Corporation IBM Data Science Experience
4/6 - Scala Bridge
Execute Scala code directly from your python Notebook
%%scala
val demo = com.ibm.cds.spark.samples.StreamingTwitter
demo.setConfig("twitter4j.oauth.consumerKey",”XXXXX")
demo.setConfig("twitter4j.oauth.consumerSecret",”XXXXX")
demo.setConfig("twitter4j.oauth.accessToken",”XXXXX")
demo.setConfig("twitter4j.oauth.accessTokenSecret",”XXXXX")
demo.setConfig("watson.tone.url","https://watsonplatform.net/tone-analyzer/api")
demo.setConfig("watson.tone.password",”XXXXX")
demo.setConfig("watson.tone.username",”XXXX”)
import org.apache.spark.streaming._
demo.startTwitterStreaming(sc, Seconds(10))
pythonVar = “pixiedust”
Define Python variable
println(pythonVar) Use the python var in Scala
val __fromScalaVar = “Hello from Scala” Define scala variable
print(__fromScalaVar) Use the scala var in Python

7. ©2016 IBM Corporation IBM Data Science Experience
5/6 - Extensibility
Easily extend PixieDust to create your own
visualizations using HTML/CSS/JavaScript
Customized
Visualization for
GraphFrame Graphs

8. ©2016 IBM Corporation IBM Data Science Experience
6/6 - Embed Apps in Notebooks
Encapsulate your analytics into compelling User
Interfaces better suited for Line of Business Users
from pixiedust_twitterdemo import *
twitterDemo()

9. ©2016 IBM Corporation IBM Data Science Experience
https://github.com/ibm-cds-labs/pixiedust