This document discusses using Jupyter Notebooks and Ansible together for literate computing and reproducible infrastructure. It introduces Jupyter Notebooks and their architecture including notebook extensions and server extensions. It then provides examples of specific notebook extensions developed at NII including code_cell_status and multi_outputs that add additional functionality. The document concludes by discussing how Ansible playbooks can be executed from Jupyter Notebooks to automate infrastructure configuration.

1. Jupyter Notebook Ansible

2. ◦
◦ GitHub/Twitter: @yacchin1205
:
:
: ( )
(NII)
( )

3. ◦ /

4.
is

6. …
1867 1930
…
– – , 1978

7. …
…
– – , 1978

8. /
/

10. Which type are you?
ANACONDA EMPOWERS THE ENTIRE DATA SCIENCE TEAM
<https: //www.continuum.io/open-data-science>
as of July 1, 2016

11. ( )
Moving Target
◦
◦
◦
◦
◦
→

12. NII
• …
•
• Hadoop …
• …
•
•
•
•

13. Hardiman, General Electric in 1965
https://www.engadget.com/2014/01/26/ge-man-amplifying-robots/

14. Literate
Computing
for
Reproducible
Infrastructure

15. Jupyter: Literate Computing

16. Jupyter
Data Scientist
Jupyter Notebook
https://tmpnb.org

17. Jupyter( IPython) + Ansible
◆
◆
◆
Notebook
Literate Computing for Reproducible Infrastructure

21. “Literate Computing for Reproducible Infrastructure”
/
= computation = communication
◦ data live-code
◦

22. Automation .. Communication
:
: Communication
◦
◦
◦
(Code )
◦ /

23. … Traceability
… Reusability
… Reproducibility

24. “ ”Notebook
https://github.com/NII-cloud-operation/Literate-computing-Basics

26. Hadoop

28. Notebook

29. Ansible Playbook

30. Ansible Playbook

31. …

32. …

34. Update

37. •
• 1
• Notebook
• Notebook
• Notebook
…

39. • https://github.com/ansible/ansible/issues/13485
Not only the handler is not triggered,
it does not throw any error if it does not exist.

40. • …

41. Notebook

42. Jupyter + Ansible

43. Jupyter Notebook
http://qiita.com/yacchin1205/items/efaa498ca68e7c169015

44. Ansible
• Ansible
•
• / sudo
•
• Notebook
•
Notebook
•

45. Jupyter

46. N.
)
takasix
2010 SIer SaaS
2011 ( )
2012 ( ) Microsoft Kinect Gescha
2013 2015 SIer OSS”CloudConductor-Patterns”
2016

47. Jupyter Notebooks Architecture
0MQ
HTTP &
Websokets

48. notebook extension system
0MQ
HTTP &
Websokets
nbextensions serverextensions

49. nbextensions
• JavaScript
• UI, Note
•
•

50. server extension
• 4.2
• Notebook server
• HTTP
• (?)

51. Jupyter Notebook

52. nb extension
• Jupyter_code_cell_status
• https://github.com/NII-cloud-operation/Jupyter-code_cell_status
• Jupyter-multi_outputs
• https://github.com/NII-cloud-operation/Jupyter-multi_outputs

53. Jupyter-code_cell_status

54. Jupyter_code_cell_status -
•

55. Jupyter_code_cell_status -

56. Jupyter_code_cell_status -
•

57. Jupyter_code_cell_status -
• nb extension

58. Jupyter-multi_outputs

59. Jupyter-multi_outputs -
•
•
• ”Trial and Error”
•
• :
• :

60. Jupyter-multi_outputs -

61. Jupyter-multi_outputs - (1)
•

62. Jupyter-multi_outputs - (2)
•

63. Jupyter-multi_outputs - (3)
•

64. Jupyter-multi_outputs - (4)
• .ipynb
• Extension
( )

65. Jupyter-multi_outputs -
• diff
• ……

66. Jupyter-multi_outputs -
• 1 stream

67. Jupyter-multi_outputs -
%matplotlib inline
import numpy
from matplotlib import pyplot
x = numpy.arange(0, 10, 0.2)
y = numpy.cos(x)
pyplot.plot(x,y)
display_data:[<matplotlib.lines.Line2D at 0x7eff779f56d0>]
execute_result:data:image/png;base64,iVBORw0KGgoA…

68. notebook extension system

69. notebook extension system (1)
• nb extensions
• Extension
• JavaScript prototype Extension
✴ code_cell_status multi_outputs
1 Extension

70. notebook extension system (2)
• server extensions
•
• notebook extension

71.
Jupyter notebook
LiterateComputing
for Reproducible Infrastructure
PullRequest

72. Jupyter Notebook Ansible