In this webinar you'll learn about the best practices for Google BigQuery—and how Matillion ETL makes loading your data faster and easier. Find out from our experts how to leverage one of the largest, fastest, and most capable cloud data warehouses to improve your business and save money. In this webinar: - Discover how to work fast and efficiently with Google BigQuery - Find out the best ways to monitor and control costs - Learn to leverage Matillion ETL and optimize Google BigQuery - Get tips and tricks for better performance

1. 1
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks or
service marks are property of their respective owners. 10/10/2017
Google BigQuery Best Practices
Tuesday, February 27th, 11am EST
James Johnson and Laura Malins

2. 2
www.matillion.com
© 2017 Matillion. All rights reserved.
Agenda
• Introduction – James Johnson
• Google BigQuery Architecture
• Data Loading
• Query Optimisation
• Cost Optimisation

3. 3
www.matillion.com
© 2017 Matillion. All rights reserved.
Architecture
• Serverless – all managed by
Google Project and Dataset
• Data physically organised by
column
• Dremel Execution Engine
• ‘Workers’ and ‘Slots’

4. 4
www.matillion.com
© 2017 Matillion. All rights reserved.
Data Loading
• Federated Data Source Load
• Many sources
• Google Analytics
• Google Sheets
• Files held in Cloud Storage buckets
• BigQuery Data Transfer Service

5. 5
www.matillion.com
© 2017 Matillion. All rights reserved.
Demonstration

6. 6
www.matillion.com
© 2017 Matillion. All rights reserved.
Streaming
3 options for streaming:
• Directly with Python
• Via Pub/Sub using Python
• Cloud DataFlow

7. 7
www.matillion.com
© 2017 Matillion. All rights reserved.
Query Optimization
• Avoid SELECT *
• LIMIT and WHERE clauses
• Use intermediary tables
• Partition data
• Avoid WITH clauses
• Preview Data
• Early Filtering
• Materialise Common Joins into intermediate table

8. 8
www.matillion.com
© 2017 Matillion. All rights reserved.
Query Optimization
• Analytic Functions
• Avoid OLTP Practices
• Avoid User Defined Functions
• Avoid High Compute Queries e.g. multiple joins
• Avoid Single Row Data Manipulation Language (DML)

9. 9
www.matillion.com
© 2017 Matillion. All rights reserved.
Demonstration

10. 10
www.matillion.com
© 2017 Matillion. All rights reserved.
Cost Optimization
• Pricing
• Expiration
• Partitioning
• Monitoring

11. 11
www.matillion.com
© 2017 Matillion. All rights reserved.
Demonstration

12. 12
www.matillion.com
© 2017 Matillion. All rights reserved.
Case Study

13. 13
www.matillion.com
© 2017 Matillion. All rights reserved.
Matillion ETL for BigQuery
• Built for BigQuery, ELT Architecture
• Intuitive UX, scalable, 5 minute purchase
process
How to engage
• Available only on Cloud Launcher on GCP
• SA team to support PoCs, demos and
training
• Support@matillion.com
• Youtube channel
Summary and how to engage

14. 14
www.matillion.com
© 2017 Matillion. All rights reserved.

15. 15
www.matillion.com
© 2017 Matillion. All rights reserved.
Presented by:
Copyright © 2017. All rights reserved. Matillion, trademarks, registered trademarks or
service marks are property of their respective owners. 10/10/2017
Thank You
James Johnson and Laura Malins