Andrew Odewahn CTO, O'Reilly Media The Jupyter Notebook is a key part of every data scientist's tool chest, but it's often seen as just an incremental tool in a process, a scratchpad that is discarded once the real work is done. But, what if instead the notebook was itself the final product that communicated the results of a data science team's efforts in a lasting format? In this talk, I'll show how O'Reilly Media is using Jupyter to create a variety of media (books, articles, and new kinds of code+video+text hybrids) and cover some of the lessons learned.

1. Making Media
with Jupyter
Andrew Odewahn, O’Reilly Media
@odewahn

3. • First Commercial
Website
• Web 2.0
• Open Source
• Make Magazine

6. “My experience with Jupyter notebooks
is that students engage with the
material in an almost tactile way.
Manipulating a piece of code,
experimenting with parameters in a
problem, then reflecting and discussing
the output makes for an active learning
experience that works. The best results
occur when students start creating their
own notebooks, either documenting a
mini-project or for note-taking and
experimenting with code. I encourage
them to do it, and it takes a lot of
cajoling, but when they start doing this,
I know I’ve reeled them in and they will
succeed in the course.”
Lorena Barba, GWU
http://bit.ly/2q3spX3

7. Computable Content
“Computable content: educational content
made powerfully interactive via compute
engines in the learning platform.”*
— Lorena Barba
* https://bids.berkeley.edu/resources/videos/computational-thinking-and-pedagogy-computable-content

8. Physical to Digital

9. Video + Text + Assessment

10. New Interaction Models
http://worrydream.com/ExplorableExplanations/

11. Data + Content

12. Preservation

13. Phase 1
Shifting from Physical to
Digital Publishing

14. 2010
2011
2013

15. https://medium.com/@cshirky/last-call-c682f6471c70#.hgj47w2xq
http://mjperry.blogspot.hk/2012/09/freefall-adjusted-for-inflation-print.html

16. Digital Publishing
Structured
Markup
Transformation
Engines
Version
Control

17. Structured Markup
• Markdown
• AsciiDoc
• HTML
• DocBook
• etc.
<section data-type="chapter">
<h1>Chapter 1</h1>
<p>
It was a dark
and stormy night
</p>
<section data-type="sect1">
...
</section>
</section>

18. Transformation Engines
• Converts source into
distribution ready
• PDF
• ePub
• Mobi (for Kindle)
• Web
• Separates content from
styling, like on the web
• Similar to static site
generators (Jekyll, Hugo,
etc)

19. Version Control with Git
• Stores content, data,
and assets (can
include git-lfs)
• Enables collaboration
through “branch and
merge” workflow
• Tap into broad
community via
GitHub

20. 80%

21. Phase 2
Jupyter as a platform for
computable content.

22. Jupyter Notebooks
• Open Source tool for
“narrated computation”
• Created by Fernando
Perez and Brian
Granger
• Estimated 2-3 million
users
http://jupyter.org

23. Markdown-based Authoring

24. Coding Environment
50+ language “kernels”

25. Inline Charting

26. Interactive Charting

27. Powerful Ecosystem
• NBconvert to convert notebook into static formats
• JupyterHub for running large, centralized installs
• NBViewer (among others!) for publishing static versions
• Charting, graphing, and visualization (Plotly, etc)
• NBGrader for automating grading and assignments
• HTML
• LaTeX
• PDF
• Slideshow
• Markdown
• reStructuredText
• Executable script
• Notebook

28. Phase 3
Stubbing our toes.

29. Dear Editor or Reviewer,
Check out this great Jupyter project by
Jake VanderPlas and send me a few
thoughts. To use it, first set up and
configure git and then drop into the
command line to clone the repo. Then,
depending on your setup, you’ll need to
install conda.
Once you do that, then you’ll need to
install all the dependencies, which you
do with this pip or conda. I *think* I
might have gotten everything set up in a
requirements.txt file, so you can just
run “conda install -r requirements.txt”
and that should all work.
Let me know how it goes!

30. How do I get to the
“command line?”

31. Then it begins…

32. How do I set up git?
$ git config --global user.email “andrew@odewahn.com”
$ git config --global user.name “Andrew Odewahn”

33. How do I create a directory?
$ cd ~/Desktop
$ mkdir my-git-projects
$ cd my-git-projects

34. How do I clone it again?
$ git clone https://github.com/AllenDowney/ThinkDSP

35. How do I run it?
$ jupyter notebook

36. I got this…
-bash: jupyter: command not found

37. HELP!
Installing collected packages: six, singledispatch, certifi, backports-abc, tornado, jupyter-core, pyzmq, jupyter-client,
functools32, jsonschema, nbformat, pygments, mistune, MarkupSafe, jinja2, nbconvert, path.py, pickleshare, simplegeneric,
setuptools, gnureadline, appnope, ptyprocess, pexpect, ipython, ipykernel, terminado, notebook, ipywidgets, jupyter-console,
qtconsole, jupyter
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This
is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 726, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 746, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)

38. It starts, but…

39. SUCCESS!

40. How do I commit again?
$ git add .
$ git commit -a -m”edited chapter 1”
$ git push origin chap1Edits

41. How do I share this?
Dear Andrew,
Wow! It’s a bit rough to get set up,
but once I did, Jupyter is really cool.
There were a lot of applications for
what Jake is doing, so I sent your
instructions to a colleague.
Could you help him out?

42. How do I get to the
“command line?”

43. Phase 4
From computable
content to computational
publishing.

44. High Barriers to Entry
• Requires familiarity with an entire software lifecycle tools
that are largely invisible to developers but are huge
roadblocks to others
• Git and GitHub literacy
• clone -> branch -> commit -> push
• diffing / merging
• Package management
• Need to learn and understand markdown

45. Computational Publishing
Source
Content
Content
Dependencies
Jupyter
Dependencies
Machine Image
Runtime
Engine
File Sharing

46. Source Content
• Git Repo
• ipynb files
• Dockerfile
• Data
https://launchbot.io

47. Machine Image
• A snapshot of a filesystem that specifies the entire
environment
• Jupyter Dependencies
• Content Dependencies
• Community converging around Docker

48. Jupyter Dependencies

49. Content Dependencies
• Allow you to specify
the packages specific
to your content
• NumPy, SciPy, etc
• AstroPy, BioPy, etc
Domain
Package
Manager
OS apt, yum, apk,
brew, choco
Python pip, conda
R cran
Node npm, yarn
Java maven
… …

50. Docker Stacks
https://github.com/jupyter/docker-stacks
pandas, matplotlib, scipy, seaborn,
scikit-learn, scikit-image, sympy,
cython, patsy, statsmodel,
cloudpickle, dill, numba, bokeh,
vincent, beautifulsoup, xlrd

51. Dockerfile
Specify a
base image
Add content
specific libraries

52. Runtime Engine
Single
User
Multi
User
VM/Cloud
Your
Infrastructure
1 2
3 4