Project Jupyter, evolved from the IPython environment, provides a platform for interactive computing that is widely used today in research, education, journalism and industry. The core premise of the Jupyter architecture is to design tools around the experience of interactive computing, building an environment, protocol, file format and libraries optimized for the computational process when there is a human in the loop, in a live iteration with ideas and data assisted by the computer. In this talk, I will discuss what are the basic ideas that underpin Jupyter, and how they provide "lego blocks" that enable the project team, and the broader community, to develop a variety of tools and approaches to problems in interactive computing, data science, visualization and more.

1. Fernando Pérez
(@fperez_org & fperez@lbl.gov)
LBL & UC Berkeley
interactive data
exploration and
visualization across
languages
The Architecture of
Jupyter

2. Mandatory COI disclosure

3. –Hamming'62
“The purpose of computing is insight,
not numbers”

4. Every research discipline is now awash in data
Physics: LHC
Sociology: The Web
Biology: Sequencing
Economics: POS
terminals
Neuroscience: EEG, fMRI
Astronomy: LSST
Personalized, data-driven
medicine

6. IPython: Interactive Python, 2001
❖ Object Introspection (TAB!)
❖ OS Integration
❖ Rich terminal client
❖ GUI support (plots, …)
❖ %magic commands
❖ Embeddable

7. Team today: where all the credit goes
Plus ~ 500 more Open source contributors!

8. Funding and partnerships

9. Pragmatic abstractions: vocabulary

10. Interactivity as a protocol
❖ The REPL as a network protocol
❖ Kernels
❖ execute code
❖ Clients
❖ Read input
❖ Present output
Simple abstractions enable rich,
sophisticated clients

11. The IPython/Jupyter Notebook
❖ Rich web client
❖ Text & math
❖ Code
❖ Results
❖ Share, reproduce.

12. From IPython to Project Jupyter

13. IPython … Jupyter
❖ Interactive Python shell at the
terminal
❖ Kernel for this protocol in
Python
❖ Tools for Interactive Parallel
computing
❖ Network protocol for
interactive computing
❖ Clients for protocol
❖ Console
❖ Qt Console
❖ Notebook
❖ Notebook file format & tools
(nbconvert...)
❖ Nbviewer
Language Agnostic

14. Protocols: kernels & clients

15. Jupyter Protocol
is language agnostic
u alj i
~75 different kernels: https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages

16. Alternate clients (nb): nteract
❖ Local desktop application
❖ Written in node.js (uses React)
❖ Uses:
❖ Jupyter messaging protocols
❖ Notebook file format.
❖ https://github.com/nteract/nteract
Kyle Kelley Safia Abdalla

17. Alternate clients (editor): hydrogen
Lukas Geiger

18. Alternate clients (IDE): spyder
https://github.com/spyder-ide/spyder

19. Jupyter Kernel Gateway
❖ Web server for spawning and communicating with
kernels over HTTP/Websocket
❖ Defaults to letting web clients talk the Jupyter protocol
❖ Extensible with other modes / personalities
Slides/credit: Peter Parente, @parente

20. Notebooks: structured files
(metadata!)

21. nbgrader: notebook homework
B. Granger
Cal Poly
J. Hamrick
UC Berkeley

22. Notebooks as dashboards
❖ Same file format
❖ Metadata based
❖ Live dashboard with
Jupyter kernel
❖ Web view with hidden
details (code, setup,
etc)
https://github.com/jupyter-incubator/dashboards

23. By Cameron Davidson-PilonBy Matthew Russell
By Jose Unpingco
By Cyrille Rossant
“Executable books”

24. “The Notebook”:
reusable web application

25. mybinder.org
github.com/freeman-lab
Andrew Osheroff’s SciPy’16 talk:
https://www.youtube.com/watch?v=OK6M4w7LYIc
github.com/andrewosh

26. Oriole: executable, video-narrated tutorials
Paco Nathan
Andrew Odewahn
Taylor Martin
oreilly.com/learning/regex-golf-with-
peter-norvig

27. Microsoft, IBM, Google, Continuum…

28. JupyterHub: multiuser support

29. Berkeley’s Foundations of Data Science
❖ New curriculum aimed at all
freshmen at UC Berkeley
❖ Interactive textbook is
Jupyter Notebooks
❖ Course deployment is
JupyterHub
❖ Off Jess Hamrick’s work
data.berkeley.edu, data8.org

30. Data Science: Connector Courses

31. JupyterHub: interactive HPC
Cori @ NERSC: Department of Energy
Supercomputing Center (LBNL)
Shreyas Cholia
Rollin Thomas

32. JupyterLab: the notebook,
evolved…

33. The “Notebook”?

34. JupyterLab: unifying these ideas
Jason Grout
(here)
+Brian, Steven, Darian,
Sylvain, Carol, Cameron,
Farica, Paul, Reese, Kyle,
Chris, Ian, Matthias, …

35. Live Demo!
Demo credits/thank you:
Brian Granger (Cal Poly SLO)
Jason Grout (Bloomberg)

36. Thank You