In this practical webinar, we'll walk through some of the key location intelligence functions that are now available in BigQuery with our Spatial Extension.
Determinants of health, dimensions of health, positive health and spectrum of...
CARTO Cloud Native – An Introduction to the Spatial Extension for BigQuery
1. CARTO Cloud Native – An
introduction to the Spatial
Extension for BigQuery
Follow @CARTO on Twitter
2. CARTO — Unlock the power of spatial analysis
Introductions
Margara Tejera
Product Manager at CARTO
Miguel Álvarez
Data Scientist at CARTO
3. CARTO — Unlock the power of spatial analysis
Agenda
● Introduction to the Spatial Extension and vision
● Overview of the functions available in the Spatial Extension for BigQuery
● Demo: cluster, polygonize and spatial indexes
● Tiler + feature preview.
● Demo: solving a complete spatial analysis
● Spatial Extension for Snowflake
● Q&A
4. CARTO — Unlock the power of spatial analysis
POLL 1
Data Analyst ………..…………………………………………………………….
Data Scientist ………………….……….………………………………………..
Developer …..…………………………………...………………………………..
What’s your current role?
Other …..…………………………………………………………………………….
6. CARTO — Unlock the power of spatial analysis
CARTO is going cloud native
7. CARTO — Unlock the power of spatial analysis
The CARTO Spatial Extension is
a suite of user defined
functions (UDFs) and
procedures to perform spatial
analysis natively within the
cloud data warehouse
environment.
What’s the Spatial
Extension?
9. CARTO — Unlock the power of spatial analysis
● Scalable and serverless spatial analytics
● No more ETLs: everything happens within the data
warehouse
● Security compliance!
Some key benefits
10.
11. CARTO — Unlock the power of spatial analysis
Status in BigQuery
12. ● 60+ core functions
● 12 advanced
functions and
procedures
* Full list in the Documentation
13. CARTO — Unlock the power of spatial analysis
● Spatial Extension: BigQuery
project called bqcarto.
● Module: BigQuery dataset, e.g. h3,
quadkey, tiler, etc. Organises the
functionality of the Spatial Extension in
categories.
● Function / procedure:
BigQuery UDF functions and
procedures.
14. CARTO — Unlock the power of spatial analysis
POLL 2
Only CARTO ……………………………………………………………………….
Only BigQuery ………………………….………………………………………..
Both CARTO & BigQuery…..………………………………………………..
Do you use CARTO and/or BigQuery ?
15. CARTO — Unlock the power of spatial analysis
Examples & new
CARTO Cloud Native
16. CARTO — Unlock the power of spatial analysis
CARTO Cloud Native
Workspace
17.
18. CARTO — Unlock the power of spatial analysis
Cluster and polygonize
19. CARTO — Unlock the power of spatial analysis
Weather stations
coverage using Voronoi
diagrams
New police stations based
on Chicago Crime
locations
20. CARTO — Unlock the power of spatial analysis
Spatial Indexes
H3 · Quadkey · S2
21. CARTO — Unlock the power of spatial analysis
What are they?
22. CARTO — Unlock the power of spatial analysis
Spatial indices = scalable spatial analysis
Enable to partition the space at different resolutions [viz]
23. CARTO — Unlock the power of spatial analysis
Spatial indices = scalable spatial analysis
Expensive spatial operations like intersections become table JOINS
26. CARTO — Unlock the power of spatial analysis
BigQuery Tiler is a solution to visualise
massive datasets hosted in BigQuery
through the transformation of the original
data into vector tiles.
What is it?
27. CARTO — Unlock the power of spatial analysis
How it works
● Converts tables into tilesets: z, x, y, data (MVT), carto_partition
● Metadata stored in z/x/y -1/NULL/NULL (JSON)
● 4000 partitions using carto_partition: improve speed
● Uses procedures: reads query twice (get metadata, generate tileset)
z x y carto_partition data
16 45340 24576 3605 H4sIAAAAAAAA/5Py52JPdt3eyCLEwM (…)
28. CARTO — Unlock the power of spatial analysis
1. World Population (Worldpop)
2. New American Atlas (CDRC)
3. Night time lights (Colorado School of
Mines)
Examples
Publicly available in BigQuery
project carto-do-public-tilesets
29. CARTO — Unlock the power of spatial analysis
● No configuration needed
● Smart tile memory
management
Feature preview:
Smart Tiler
ST_GENERATETILESET(
input_sql STRING,
output_table_name STRING,
--OPTIONAL
options)
30. CARTO — Unlock the power of spatial analysis
CARTO offers a complete toolset
1. Create tilesets with BigQuery Tiler
2. Serve tiles through the Maps API
3. Visualise tilesets
○ In the Dashboard / Cloud Native Workspace
○ In custom apps through CARTO for deck.gl and CARTO for
React
31. CARTO — Unlock the power of spatial analysis
Point Aggregation
Tileset
● Encodes the original features
(points, lines, polygons) and
their attributes.
● Only for points or data that can be
converted to points.
● Encodes the aggregation of the
points’ attributes.
Simple Tileset
32. CARTO — Unlock the power of spatial analysis
It’s time for a real world example!
33. CARTO — Unlock the power of spatial analysis
Getting access
34. CARTO — Unlock the power of spatial analysis
1. Create BigQuery connection, OAuth type.
2. Choose any billing project
3. Check that you have access running this in BQ:
bqcarto.tiler.VERSION()
35. CARTO — Unlock the power of spatial analysis
Other data
warehouses
36. CARTO — Unlock the power of spatial analysis
Status in
Snowflake
● Same functionality as
in BigQuery, except
Tiler and Data module.
● Installation through the
Snowflake Data
Marketplace.
* Full list in the Documentation
Coming soon!
38. Thanks for listening!
Any questions?
Request a demo at CARTO.COM
Miguel Álvarez
Data Scientist at CARTO // malvarez@cartodb.com
Margara Tejera
Product Manager at CARTO // mtejera@cartodb.com