Download as PDF, PPTX
Uploaded byWey-Han Liaw
Jupyter Kernel: How to Speak in Another Language
The document discusses how to create a Jupyter kernel. It explains that kernels use ZeroMQ sockets to communicate with clients via the Jupyter messaging protocol. Native kernels are implemented from scratch while wrapper kernels are built from an existing interpreter using Python. The document provides examples of existing kernels like IJulia and the Python ipykernel. It also outlines the steps to build a wrapper kernel and mentions several other kernel types.
More Related Content
Jupyter Kernel: How to Speak in Another Language
- 1. Jupyter Kernel How toSpeak in Another Language
- 2. How to Makea Jupyter Kernel:
- 3. 1: 5 ømqsockets
- 4.
- 5. 3. Holy Shit, YOURUINED EVERYTHING, LIKE EVERYTHING YOU DO
- 6. 4. This Kernel IsLike Your Life
- 7. 5. Begin aLot of Things, Left Undone
- 8. 6. And NobodyLoves You
- 9.
- 10. Agenda • Jupyter, IPython,notebook, console, client, kernel • The Interactive Computing Protocol • Implementing a Kernel • Live Demo
- 11. About this guy •廖偉涵 Adrian Liaw • github.com/adrianliaw • High School Homeschooler • A Pythonista • Student of Udacity’s Data Analyst Nanodegree
- 12. About this guy •Amateur Pianist • Classical Music Lover • Coffee Shop Lover
- 14.
- 15. IPYTHON NOTEBOOK WEB-BASED NOTEBOOKENVIRONMENT
- 16.
- 18. Jupyter Notebook (With RKernel)
- 19. -Project Jupyter “Project Jupyterwas born out of the IPython Project in 2014 as it evolved to support interactive data science and scientific computing across all programming languages.”
- 22.
- 23.
- 24.
- 25.
- 26. What’s Inside IPython •The Interactive IPython Shell (No ØMQ) • Magic Commands • Auto Word Completer • Beautiful Traceback • Shell History Management
- 27.
- 28. IPyKernel • Jupyter’s “Kernel0” • ØMQ Handlers • github.com/ipython/ipykernel
- 29. What’s Inside Jupyter •Web-based Notebook Interface & nbconvert • Interactive Qt Console • Terminal-based Console • Spec for Notebook Document Format • Spec for Interactive Computing Protocol
- 30.
- 31. Interactive Computing Protocol • i.e.Jupyter Messaging Protocol • Communication Between Kernel and Client • Based on ØMQ and JSON
- 32. DEALER ROUTER CLIENT KERNEL Shell SUBPUBIOPub DEALER ROUTER stdin DEALER ROUTER Control REQ REP Heartbeat
- 33. Connection File { "transport": "tcp", "ip":"127.0.0.1", "shell_port": 56166, "iopub_port": 56167, "stdin_port": 56168, "control_port": 56169, "hb_port": 56170, "signature_scheme": "hmac-sha256", "key": "<hashed key>" }
- 34. Message Format { 'header' :{ 'msg_id' : str, 'username' : str, 'session' : str, 'date': str, 'msg_type' : str, 'version' : '5.0', }, 'parent_header' : dict, 'metadata' : dict, 'content' : dict, }
- 35.
- 36. { "header": { "msg_type": "kernel_info_request", ... }, "content":{}, } CLIENT KERNEL Shell
- 37. CLIENT KERNEL Shell { "header": { "msg_type":"kernel_info_reply", ... }, "parent_header": {...}, "content": { "banner": "Python 3.5.1 ... IPython 4.0.0 ...", "implementation": "ipython", "implementation_version": "4.0.0", "language_info": { "name": "python", "version": "3.5.1", ... }, }, }
- 38. CLIENT KERNELShell { "header": { "msg_type":"execute_request", ... }, "content": { "code": "print('HAI')", "silent": False, "store_history": True, }, }
- 39. CLIENT KERNELIOPub { "header": {"msg_type":"stream", ...}, "parent_header": {...}, "content": { "name": "stdout", "text": "HAIn", }, }
- 40. CLIENT KERNELShell { "header": { "msg_type":"execute_reply", ... }, "parent_header": {...}, "content": { "execution_count": 1, "status": "ok", "user_expressions": {}, }, }
- 41. CLIENT KERNELShell { "header": {"msg_type":"execute_request", ...}, "content": { "code": "pd.read_csv('foo.csv')", "silent": False, "store_history": True, }, }
- 42. CLIENT KERNELIOPub { "header": {"msg_type":"execute_result", ...}, "parent_header": {...}, "content": { "execution_count": 5, "data": { "text/plain": " col1 col2n A 3n....", "text/html": "<div><table>...</table></div>", }, }, }
- 43. CLIENT KERNELShell { "header": { "msg_type":"execute_reply", ... }, "parent_header": {...}, "content": { "execution_count": 5, "status": "ok", "user_expressions": {}, }, }
- 46. CLIENT KERNELShell { "header": {"msg_type":"execute_request", ...}, "content": { "code": "read.csv('table.csv')", "silent": False, "store_history": True, }, }
- 47. CLIENT KERNELIOPub { "header": {"msg_type":"execute_result", ...}, "parent_header": {...}, "content": { "execution_count": 1, "data": { "text/plain": " col1 col2n A 3n....", "text/html": "<table>...</table>", }, }, }
- 48. CLIENT KERNELShell { "header": { "msg_type":"execute_reply", ... }, "parent_header": {...}, "content": { "execution_count": 1, "status": "ok", "user_expressions": {}, }, }
- 49.
- 50. Kernel Types • NativeKernel • Python Wrapper Kernel • REPL Wrapper Kernel
- 51. IRKernel 11. zmqctx <<-zmq.ctx.new() 12. sockets <<- list( 13. hb = zmq.socket(zmqctx, .pbd_env$ZMQ.ST$REP), 14. iopub = zmq.socket(zmqctx, .pbd_env$ZMQ.ST$PUB), 15. control = zmq.socket(zmqctx, .pbd_env$ZMQ.ST$ROUTER), 16. stdin = zmq.socket(zmqctx, .pbd_env$ZMQ.ST$ROUTER), 17. shell = zmq.socket(zmqctx, .pbd_env$ZMQ.ST$ROUTER))
- 52.
- 53. Wrapper Kernel • Python-driven •ipykernel.kernelbase.Kernel • http://bit.ly/WrapperKernel
- 54. from ipykernel.kernelbase importKernel class MyKernel(Kernel): def do_execute def do_complete def do_history def do_shutdown
- 55. Hy Kernel Redis Kernel Xonsh CKernel Matlab Kernel SAS Kernel
- 56. REPL Wrapper from pexpect.replwrap importREPLWrapper
- 57. Octave Kernel Jupyter Powershell BashKernel Mongo Kernel Scilab Kernel
- 58.
- 59.
- 60. Reference • P.3, P.4,P.17, P.20, P.50 images: Project Jupyter • P.6, P.7, P.8 images: Molg H. • P.16: Project Jupyter • P.11 image: Toomore Chiang • P.12 image: Virginia Cheng • P.13 image: ipython • P.45 image: R