Vicky Close, Second Measure @ PyData Paris 2016

1. PYTHON TO REPORT IN
ONE COMMAND
Vicky Close, Second Measure

2. Talk overview
• Communicating insights
• Google Apps Script
• Coffee

3. About me

4. http://blog.secondmeasure.com/2016/04/26/chipotle-recovering-from-food-safety-scare/

5. http://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-
data-science-task-survey-says/#581e921a7f75

6. My motivation
ipython
notebook
for analysis
Report in
Google Doc
Dashboards
(interactive?)
Excel
spreadsheets

8. Existing tools for creating reports
• There are lots of options!

9. Coffee prices data set
■ http://www.economicswebinstitute.org/ecdata.htm
■ Prices paid to growers in exporting ICO countries for
two coffee brands (Arabica and Robusta).
■ Retail price paid by consumer to commercial
distributer in importing ICO countries.
■ US cents per lb per month between 1982 and 2001.

12. nbconvert
ipython nbconvert --to html ipython nbconvert --to latex

13. Plotly
• Online data visualisation platform.
• Written using Python, Django, JavaScript,
D3.js.
• API library for Python.

14. Creating a map using Plotly

15. Prices paid to Arabica coffee growers

16. HTML reports using Plotly
http://moderndata.plot.ly/generate-html-reports-with-python-pandas-and-plotly/

17. Jinja2
• Template engine for Python.
http://jinja.pocoo.org/
http://pbpython.com/pdf-reports.html

23. ipython
notebook
Report
(html, pdf, …)

24. But…
• Not easy for other’s to add comments and
edits.

25. Google Drive API
• One of many Google APIs
• Has a python client library
• Upload and download files
• Convert files
https://developers.google.com/drive/v3/web/quickstart/python#prerequisites

26. Workflow
ipython
notebook
html
Google
Doc
ipython nbconvert
--to html
Google Drive API

29. Uploading files
http://wescpy.blogspot.co.uk/2015/12/google-drive-uploading-downloading.html

33. Google Apps Script
• Scripting language for application
development in the Google Apps platform.
• Create custom sidebars/menus in docs,
sheets and forms.
• Based on Javascript.

34. Workflow
ipython
notebook
json
Google
Doc
Google
Apps Script
Python
script

36. What is the underlying structure of an ipython notebook?
• Four different types of cells
• Heading
• Markdown
• Raw
• Code (output)

39. Markdown and heading cells

40. Code cells: Tables
‘#% table’ at beginning of cell
Use output field (html) + beautiful soup.

41. Code cells: Images
‘#% figure’ at beginning of cell
Extract name of figure

42. Json

43. Workflow
ipython
notebook
json
Google
Doc
Google
Apps Script
Python
script

53. Python to report in one command?
Step 1: Google Drive
API to upload images
to Google Drive
ipynb
Google
Drive
Step 3: Run Google
Apps Script using the
Python Execution API.
json
Google
Doc
Step 2: Python script
to convert ipynb to
json.

54. What I’ve learnt and what’s next?
• Automate report writing
• Harder project that I anticipated
• Google Apps Script is very versatile
• Comments, updates/edits
• Spreadsheets