Talk at Paris.Py, with captures from demo

Romain Dorgueil
Romain DorgueilMaker & Technical Advisor at MakerSquad
Bonobo
Lightweight
Extract Transform Load
Python 3.5+
Introduction
Extract Transform Load
foo
bar
baz
Extract Transform Load
Extract Transform Load
foo
bar
baz
Extract
Transform Load
Transform

more
Join


DB
HTTP POST
log?
Bonobo ETL
• It’s just Plain Old Python (3) Objects
• Streaming data through them
• Write / Configure / Execute / Monitor
Code first. Eventually a GUI will come.
Bonobo is not…
• Data Science & Analytics (see pandas)
• Workflow Scheduler & Monitor (see airflow)
• Big Data (see hadoop, big query, dataflow, spark, …)
• A monkey (spoiler : it’s an ape)
Action
Quick Demo
or brown fox?
pip install bonobo then import it
Let’s show what we can do with simple callable (generators, here).
Adding them to a graph show the data flow
Run them, see what happens real time.
Use external services, implementation resolution is done at runtime.
Talk at Paris.Py, with captures from demo
Provide implementations at runtime, easy to be 12factor-compliant.
Also easy to read/write from files.
Simplest cases can be run without any code, from CLI
$ bonobo convert orders.csv orders.json
- CsvReader in=1 out=29 [done]
- JsonWriter in=29 out=29 [done]
Even with custom transformations in the middle of the chain
$ bonobo convert orders.csv -t transform:append_isbn orders.json
- CsvReader in=1 out=29 [done]
- append_isbn in=29 out=29 [done]
- JsonWriter in=29 out=29 [done]
Epilogue
Young
• First commit : December 2016
• 23 releases, ~420 commits, 4 contributors
• 25 releases, ~565 commits, 12 contributors
• 34 releases, ~774 commits, 21 contributors
• Current : 0.4.3 0.5.1 0.6.1
…
• Open-Source.
• Light & Focused.
• Very few dependencies.
• Comprehensive standard library.
Integrations
• Django
• SQLAlchemy
• Jupyter
• Docker
• Google APIs, OpenDataSoft, Cheese Shop :)
www.bonobo-project.org
Thank you!
@bonobo_etl @rdorgueil
https://goo.gl/e25eoa
1 of 24

Recommended

Business Dashboards using Bonobo ETL, Grafana and Apache Airflow by
Business Dashboards using Bonobo ETL, Grafana and Apache AirflowBusiness Dashboards using Bonobo ETL, Grafana and Apache Airflow
Business Dashboards using Bonobo ETL, Grafana and Apache AirflowRomain Dorgueil
2.8K views91 slides
Simple Data Engineering in Python 3.5+ — Pycon.DE 2017 Karlsruhe — Bonobo ETL by
Simple Data Engineering in Python 3.5+ — Pycon.DE 2017 Karlsruhe — Bonobo ETLSimple Data Engineering in Python 3.5+ — Pycon.DE 2017 Karlsruhe — Bonobo ETL
Simple Data Engineering in Python 3.5+ — Pycon.DE 2017 Karlsruhe — Bonobo ETLRomain Dorgueil
617 views45 slides
EuroPython 2017 - Bonono - Simple ETL in python 3.5+ by
EuroPython 2017 - Bonono - Simple ETL in python 3.5+EuroPython 2017 - Bonono - Simple ETL in python 3.5+
EuroPython 2017 - Bonono - Simple ETL in python 3.5+Romain Dorgueil
1.5K views82 slides
Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes) by
Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes)Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes)
Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes)Romain Dorgueil
410 views37 slides
Simple ETL in python 3.5+ with Bonobo - PyParis 2017 by
Simple ETL in python 3.5+ with Bonobo - PyParis 2017Simple ETL in python 3.5+ with Bonobo - PyParis 2017
Simple ETL in python 3.5+ with Bonobo - PyParis 2017Romain Dorgueil
1.7K views82 slides
Docker and the Container Revolution by
Docker and the Container RevolutionDocker and the Container Revolution
Docker and the Container RevolutionRomain Dorgueil
1.3K views46 slides

More Related Content

Recently uploaded

Marketing and Community Building in Web3 by
Marketing and Community Building in Web3Marketing and Community Building in Web3
Marketing and Community Building in Web3Federico Ast
12 views64 slides
UiPath Document Understanding_Day 3.pptx by
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptxUiPathCommunity
103 views25 slides
IETF 118: Starlink Protocol Performance by
IETF 118: Starlink Protocol PerformanceIETF 118: Starlink Protocol Performance
IETF 118: Starlink Protocol PerformanceAPNIC
244 views22 slides
PORTFOLIO 1 (Bret Michael Pepito).pdf by
PORTFOLIO 1 (Bret Michael Pepito).pdfPORTFOLIO 1 (Bret Michael Pepito).pdf
PORTFOLIO 1 (Bret Michael Pepito).pdfbrejess0410
8 views6 slides
Is Entireweb better than Google by
Is Entireweb better than GoogleIs Entireweb better than Google
Is Entireweb better than Googlesebastianthomasbejan
12 views1 slide
How to think like a threat actor for Kubernetes.pptx by
How to think like a threat actor for Kubernetes.pptxHow to think like a threat actor for Kubernetes.pptx
How to think like a threat actor for Kubernetes.pptxLibbySchulze1
5 views33 slides

Recently uploaded(12)

Marketing and Community Building in Web3 by Federico Ast
Marketing and Community Building in Web3Marketing and Community Building in Web3
Marketing and Community Building in Web3
Federico Ast12 views
UiPath Document Understanding_Day 3.pptx by UiPathCommunity
UiPath Document Understanding_Day 3.pptxUiPath Document Understanding_Day 3.pptx
UiPath Document Understanding_Day 3.pptx
UiPathCommunity103 views
IETF 118: Starlink Protocol Performance by APNIC
IETF 118: Starlink Protocol PerformanceIETF 118: Starlink Protocol Performance
IETF 118: Starlink Protocol Performance
APNIC244 views
PORTFOLIO 1 (Bret Michael Pepito).pdf by brejess0410
PORTFOLIO 1 (Bret Michael Pepito).pdfPORTFOLIO 1 (Bret Michael Pepito).pdf
PORTFOLIO 1 (Bret Michael Pepito).pdf
brejess04108 views
How to think like a threat actor for Kubernetes.pptx by LibbySchulze1
How to think like a threat actor for Kubernetes.pptxHow to think like a threat actor for Kubernetes.pptx
How to think like a threat actor for Kubernetes.pptx
LibbySchulze15 views
𝐒𝐨𝐥𝐚𝐫𝐖𝐢𝐧𝐝𝐬 𝐂𝐚𝐬𝐞 𝐒𝐭𝐮𝐝𝐲 by Infosec train
𝐒𝐨𝐥𝐚𝐫𝐖𝐢𝐧𝐝𝐬 𝐂𝐚𝐬𝐞 𝐒𝐭𝐮𝐝𝐲𝐒𝐨𝐥𝐚𝐫𝐖𝐢𝐧𝐝𝐬 𝐂𝐚𝐬𝐞 𝐒𝐭𝐮𝐝𝐲
𝐒𝐨𝐥𝐚𝐫𝐖𝐢𝐧𝐝𝐬 𝐂𝐚𝐬𝐞 𝐒𝐭𝐮𝐝𝐲
Infosec train9 views
Building trust in our information ecosystem: who do we trust in an emergency by Tina Purnat
Building trust in our information ecosystem: who do we trust in an emergencyBuilding trust in our information ecosystem: who do we trust in an emergency
Building trust in our information ecosystem: who do we trust in an emergency
Tina Purnat98 views
We see everywhere that many people are talking about technology.docx by ssuserc5935b
We see everywhere that many people are talking about technology.docxWe see everywhere that many people are talking about technology.docx
We see everywhere that many people are talking about technology.docx
ssuserc5935b6 views

Featured

Google's Just Not That Into You: Understanding Core Updates & Search Intent by
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
6.1K views99 slides
How to have difficult conversations by
How to have difficult conversations How to have difficult conversations
How to have difficult conversations Rajiv Jayarajah, MAppComm, ACC
4.6K views19 slides
Introduction to Data Science by
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data ScienceChristy Abraham Joy
82.2K views51 slides
Time Management & Productivity - Best Practices by
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
169.7K views42 slides
The six step guide to practical project management by
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
36.6K views27 slides
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright... by
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
12.6K views21 slides

Featured(20)

Google's Just Not That Into You: Understanding Core Updates & Search Intent by Lily Ray
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray6.1K views
Time Management & Productivity - Best Practices by Vit Horky
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
Vit Horky169.7K views
The six step guide to practical project management by MindGenius
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
MindGenius36.6K views
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright... by RachelPearson36
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson3612.6K views
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present... by Applitools
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Applitools55.4K views
12 Ways to Increase Your Influence at Work by GetSmarter
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
GetSmarter401.6K views
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G... by DevGAMM Conference
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
DevGAMM Conference3.6K views
Barbie - Brand Strategy Presentation by Erica Santiago
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy Presentation
Erica Santiago25.1K views
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well by Saba Software
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Saba Software25.2K views
Introduction to C Programming Language by Simplilearn
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming Language
Simplilearn8.4K views
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr... by Palo Alto Software
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
Palo Alto Software88.3K views
9 Tips for a Work-free Vacation by Weekdone.com
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation
Weekdone.com7.2K views
How to Map Your Future by SlideShop.com
How to Map Your FutureHow to Map Your Future
How to Map Your Future
SlideShop.com275.1K views
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -... by AccuraCast
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...
Beyond Pride: Making Digital Marketing & SEO Authentically LGBTQ+ Inclusive -...
AccuraCast3.4K views

Talk at Paris.Py, with captures from demo