The document discusses the use of dbt (data build tool) to improve analytics workflows by allowing users who know SQL to create their own data pipelines. It highlights industry trends such as the shift from ETL to ELT and the challenges faced by data teams, including data silos and dashboard issues. The presentation includes a demo of dbt in action using a Google merchandising dataset to showcase its capabilities.

1. Transforming your analytics
workflow with dbt
#Siligong.Data May 2021 Meetup
Jon Su
Jon Su @ Internetrix

2. Who knows what SQL is?
(pronounced sequel or ss-cue-l depending on who you ask)
https://www.commitstrip.com/en/?
Jon Su @ Internetrix

3. Tonight’s Talk: dbt (data build tool)
● A History Lesson
● What is dbt?
● Demo: dbt in action
Jon Su @ Internetrix

5. ETL was all the craze!
https://blog.bismart.com/en/what-do-we-do-etl
Jon Su @ Internetrix

6. ● How to build a data
infrastructure that can
scale
● Controlling storage
costs
● Performance Tuning
The focus for data teams was on extraction
Jon Su @ Internetrix

7. But then came the “Cloud”...
This is how the Cloud
actually looks like in real-
life ;)
Jon Su @ Internetrix

8. Jon Su @ Internetrix

9. Industry Trend #1: Move away from “do-it-all-in-1-tool”
https://lakefs.io/the-state-of-data-engineering-in-2021/
Jon Su @ Internetrix

10. Industry Trend #2: Shift from ETL to ELT
https://www.striim.com/etl-vs-elt/
Jon Su @ Internetrix

11. But there are still problems...
https://www.striim.com/etl-vs-elt/
Data Science
BI Tools
Jon Su @ Internetrix

12. Analytics workflow problems...
● Data consumers don’t have the data when they need it
○ Silos between different members of a traditional data team
Data Analyst
Data Engineer Business
Triangle of
Madness
Jon Su @ Internetrix

13. Analytics workflow problems...
● Beautiful dashboards that suddenly break when something upstream goes wrong /
source schema changes
● Having to rewrite and rewrite the same piece of SQL again & again….
○ Not sharing analytics code in a team
○ Analysts work in isolation, knowledge isn’t shared
○ Different definitions of a shared metric
● Hard for a business to adopt using BI easily
○ time + $
○ Low BI adoption
Jon Su @ Internetrix

14. Does this have to be the way?
Jon Su @ Internetrix

15. Jon Su @ Internetrix

16. dbt (data build tool) lets anyone who knows SQL author their
analytics workflow and make their own data pipelines.
If you know SQL, you can use it - essentially no barrier to entry
Supports a large number of warehouse through adapters
● Can build your own adapter
“
Jon Su @ Internetrix

17. Introduces basic software engineering principles to
solve the workflow problems we mentioned!
Jon Su @ Internetrix

18. Version
Control
Quality
Assurance
Modularity
Multiple
Environments
Documentation
Automated
Tools
Code
Maintainability
https://docs.getdbt.com/docs/about/viewpoint
Jon Su @ Internetrix

19. How does dbt work?
https://www.getdbt.com/product/
Jon Su @ Internetrix

20. How does dbt work?
A dbt project consists of .sql and .yml files:
1. Write dbt code (SQL + Jinja templating)
2. Run dbt command from the CLI or dbt Cloud
3. dbt compiles your dbt code into raw SQL and executes that code against your warehouse.
4. Data is transformed and then created as tables/views back in the data warehouse
Jon Su @ Internetrix

21. dbt Core dbt Cloud
Jon Su @ Internetrix

22. Demo
Scenario:
● Google Merchandise Store
Dataset:
● Google Analytics sample dataset for
BigQuery
Goal:
● Find all Purchases made in Feb 2017
by Users who previously visited the site
using a Chrome browser in Jan 2017
Jon Su @ Internetrix

23. ● dbt website: https://getdbt.com
● Demo Source Code:
https://github.com/jkersu/dbt-basic-demo
● Get in touch by email at:
jon@irx.io
The End.
Jon Su @ Internetrix