1© 2018 Snowflake Computing Inc. All Rights Reserved.
Y O U R D A T A , N O L I M I T S
Master the Multi-Clustered Data
Warehouse
Alex Letterese, Account Executive
Matillion
2© 2018 Snowflake Computing Inc. All Rights Reserved.
Our presenters
Alex Letterese
Account Executive
Craig Collier
Global Solutions Architect
Kalyan Arangam
Solution Architect
3© 2018 Snowflake Computing Inc. All Rights Reserved.
Uniting the Data Nation
© 2018 Snowflake Computing Inc. All Rights Reserved.
DataPeople Applications
What is Data Nation?
© 2018 Snowflake Computing Inc. All Rights Reserved.
The people in data nation are divided
People who
analyze data
People who
store and curate data
People who
secure the data
People who have to
get new data
People who want to
upgrade the data platform
The people who want
to share the data
People who need new
data immediately
People who pay for the
data platform
© 2018 Snowflake Computing Inc. All Rights Reserved.
Their data is disparate
Structured
data
Semi-
structured
data
Data mart
for sales
Data mart
for ops
Corporate
data
warehouse
Customer
data
Partner data
Internal
data
Streaming
data
Static data
© 2018 Snowflake Computing Inc. All Rights Reserved.
The apps are the source of many problems
Concurrency problems when too
many people use the database
Database that can only
support a defined amount
of structured data
Unsecure and challenging
methods for sharing data
Manual, maintenance
intensive tools
Expensive up-front database
purchases
Specialized skills needed
for different tools
© 2018 Snowflake Computing Inc. All Rights Reserved.
There are fresh challenges on the horizon
Need it all in real
time for up to the
minute insight
Expanding to
new markets with
different
regulations
Applications
in cloud and
on-prem
© 2018 Snowflake Computing Inc. All Rights Reserved.
Data
People
Applications
WE NEED A
TECHNOLOGY
THAT CONNECTS
THE DATA NATION
GLOBALLY
© 2017 Snowflake Computing Inc. All Rights Reserved.
Data
People
Applications
THE CLOUD IS A
PERFECT CANVAS TO
MAKE THOSE
CONNECTIONS
Snowflake + Matillion help Unite the Data Nation
Data Integration
Business Intelligence &
Analytics
Data Warehouse
Enterprise
apps
Data Sources
Corporate
Web
Mobile
IoT
© 2018 Snowflake Computing Inc. All Rights Reserved.
Complete
SQL Database
Minimal
Management
All of
Your Data
All of
Your Users
How can Snowflake help Unite the Data Nation?
Pay Only for
What You Use
Live Data
Sharing
© 2018 Snowflake Computing Inc. All Rights Reserved.
Complete SQL Data Warehouse
STANDARD SQL
Full DML (read/write/modify),
analytic SQL
BROAD ECOSYSTEM
Leverage the skills and tools
you already have – BI, ETL…
AVAILABILITY
Automatic data replication, online recovery, high
availability and geo redundancy
SECURITY
Automatic encryption,
third-party certification
© 2018 Snowflake Computing Inc. All Rights Reserved. 14
Minimal Management
NO Infrastructure
NO Tuning
NO Optimization
NO Indexing
NO Storage worries
NO Vacuuming
NO Partitioning
NO Required sorting
NO Workload mgmt.
NO Manual backups
It Just Works!
© 2018 Snowflake Computing Inc. All Rights Reserved.
asdfAll your data
Structured Semi-structuredCSV JSON
Avro
XML
Parquet
© 2018 Snowflake Computing Inc. All Rights Reserved.
asdf
Data Science
SQL Analysts
ETL & Processing
BI & Analytics Tools
16
All Your Users
Database Services
© 2018 Snowflake Computing Inc. All Rights Reserved.
asdf
Traditional databases
are inflexible
Snowflake uses the cloud
to enable elasticity
Usage
varies
Pay for only
what you use with no
overprovisioning
Eliminate
overbuy
Instant Elasticity No need for
capacity planning,
make capacity
decisions on the fly
17
Only pay for what you use
© 2018 Snowflake Computing Inc. All Rights Reserved.
18
Seamless data sharing
Data Provider
Snowflake Account
Data to Share
Share Live Data Instantly
Grant Secure Row Level
Access
Use One Share for Many
NO FTP or EDI
NO Duplicate Files or
Storage
NO Additional Charge
Secure Data
Share
Data
Science
WH
Consumer 2
Snowflake Account
Wants provider
data for data
scientists
Consumer 1
Snowflake Account
SQL
Analysts
WH
Wants provider
data for SQL
analysts
Live Data Sharing
© 2018 Snowflake Computing Inc. All Rights Reserved.
Database Services
Data
Science (L)
Data
Transformation
(M)
SQL Queries
(S)
How is this possible? A Brand New Architecture
Easy-to-use Service with No Management
Unlimited Compute Clusters to Serve Every Use
Flexible Cloud Storage For All Kinds of
Data
Managemen
t
Optimization Securit
y
Availabilit
y
Transaction
s
Metadata
BI/Reporting
(Multi-Cluster M)
Structured
Semi-Structured
19
Infinite
Concurrency
© 2017 Snowflake Computing Inc. All Rights Reserved.
Uniting your data nation with Snowflake and Matillion?
+
© 2018 Snowflake Computing Inc. All Rights Reserved.
Matillion ETL for Snowflake
• Cloud native Data integration tool built specifically
for AWS and Snowflake
• Push-down ELT architecture
• Intuitive browser-based UX
• Retail like acquisition through AWS Marketplace –
pay by hour
About Matillion
Customer - Upside Travel
Matt Boegner
Data Engineer, Upside Travel
Scenario
Upside ingests over 3TB of raw flight/hotel
inventory on a daily basis, in addition to website,
mobile app, and marketing data streams. A lot of
data and a lot of data processing!
Negative Consequences
Existing data warehouse couldn’t keep up
Degraded performance due to concurrent loading
Expensive to maintain
After
Matillion ETL for Snowflake made their migration to
Snowflake simple and quick, and Upside was able
to actualize the benefits of Snowflake within 2
weeks, at production workloads.
“Use Snowflake if you have a cloud data
warehouse; use Matillion if you use
Snowflake.”
Snowflake
Flight/ Hotel data
Website data
Mobile data
Marketing data
Looker – BI &
Visualization
Better performance, mitigation of
concurrency issues and improved
scalability
• Migration to production live in under 2 weeks
• Scalability (up and down) >> Cost Efficiency
(Transform applied directly)
Matillion ETL for Snowflake
• 5 minute launch process on Marketplace
• 14 day Free Trial
• From $1.37 per hr
How to engage
• SA team to support PoCs, demos and training
• Support Portal
• Tutorials & Customer references on YouTube
How to Engage
Demonstration
Use bulk loading to get the data to the cloud
Leverage the scalable compute layer to do the bulk of
the data processing
Isolate load and transform jobs from queries to
prevent resource contention
Focus on organizing the data to support business
users
© 2018 Snowflake | All Rights Reserved.
MASTER THE MULTI-CLUSTERED DATA WAREHOUSE
Best Practices
27© 2018 Snowflake Computing Inc. All Rights Reserved.
Q & A
© 2017 Snowflake Computing Inc. All Rights Reserved.
How will you unite your data nation with Snowflake and Matillion?
Ask us how to start today.
+
#BetterTogether #UniteTheDataNation

Master the Multi-Clustered Data Warehouse - Snowflake

  • 1.
    1© 2018 SnowflakeComputing Inc. All Rights Reserved. Y O U R D A T A , N O L I M I T S Master the Multi-Clustered Data Warehouse Alex Letterese, Account Executive Matillion
  • 2.
    2© 2018 SnowflakeComputing Inc. All Rights Reserved. Our presenters Alex Letterese Account Executive Craig Collier Global Solutions Architect Kalyan Arangam Solution Architect
  • 3.
    3© 2018 SnowflakeComputing Inc. All Rights Reserved. Uniting the Data Nation
  • 4.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. DataPeople Applications What is Data Nation?
  • 5.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. The people in data nation are divided People who analyze data People who store and curate data People who secure the data People who have to get new data People who want to upgrade the data platform The people who want to share the data People who need new data immediately People who pay for the data platform
  • 6.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Their data is disparate Structured data Semi- structured data Data mart for sales Data mart for ops Corporate data warehouse Customer data Partner data Internal data Streaming data Static data
  • 7.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. The apps are the source of many problems Concurrency problems when too many people use the database Database that can only support a defined amount of structured data Unsecure and challenging methods for sharing data Manual, maintenance intensive tools Expensive up-front database purchases Specialized skills needed for different tools
  • 8.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. There are fresh challenges on the horizon Need it all in real time for up to the minute insight Expanding to new markets with different regulations Applications in cloud and on-prem
  • 9.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Data People Applications WE NEED A TECHNOLOGY THAT CONNECTS THE DATA NATION GLOBALLY
  • 10.
    © 2017 SnowflakeComputing Inc. All Rights Reserved. Data People Applications THE CLOUD IS A PERFECT CANVAS TO MAKE THOSE CONNECTIONS
  • 11.
    Snowflake + Matillionhelp Unite the Data Nation Data Integration Business Intelligence & Analytics Data Warehouse Enterprise apps Data Sources Corporate Web Mobile IoT
  • 12.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Complete SQL Database Minimal Management All of Your Data All of Your Users How can Snowflake help Unite the Data Nation? Pay Only for What You Use Live Data Sharing
  • 13.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Complete SQL Data Warehouse STANDARD SQL Full DML (read/write/modify), analytic SQL BROAD ECOSYSTEM Leverage the skills and tools you already have – BI, ETL… AVAILABILITY Automatic data replication, online recovery, high availability and geo redundancy SECURITY Automatic encryption, third-party certification
  • 14.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. 14 Minimal Management NO Infrastructure NO Tuning NO Optimization NO Indexing NO Storage worries NO Vacuuming NO Partitioning NO Required sorting NO Workload mgmt. NO Manual backups It Just Works!
  • 15.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. asdfAll your data Structured Semi-structuredCSV JSON Avro XML Parquet
  • 16.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. asdf Data Science SQL Analysts ETL & Processing BI & Analytics Tools 16 All Your Users Database Services
  • 17.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. asdf Traditional databases are inflexible Snowflake uses the cloud to enable elasticity Usage varies Pay for only what you use with no overprovisioning Eliminate overbuy Instant Elasticity No need for capacity planning, make capacity decisions on the fly 17 Only pay for what you use
  • 18.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. 18 Seamless data sharing Data Provider Snowflake Account Data to Share Share Live Data Instantly Grant Secure Row Level Access Use One Share for Many NO FTP or EDI NO Duplicate Files or Storage NO Additional Charge Secure Data Share Data Science WH Consumer 2 Snowflake Account Wants provider data for data scientists Consumer 1 Snowflake Account SQL Analysts WH Wants provider data for SQL analysts Live Data Sharing
  • 19.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Database Services Data Science (L) Data Transformation (M) SQL Queries (S) How is this possible? A Brand New Architecture Easy-to-use Service with No Management Unlimited Compute Clusters to Serve Every Use Flexible Cloud Storage For All Kinds of Data Managemen t Optimization Securit y Availabilit y Transaction s Metadata BI/Reporting (Multi-Cluster M) Structured Semi-Structured 19 Infinite Concurrency
  • 20.
    © 2017 SnowflakeComputing Inc. All Rights Reserved. Uniting your data nation with Snowflake and Matillion? +
  • 21.
    © 2018 SnowflakeComputing Inc. All Rights Reserved. Matillion ETL for Snowflake • Cloud native Data integration tool built specifically for AWS and Snowflake • Push-down ELT architecture • Intuitive browser-based UX • Retail like acquisition through AWS Marketplace – pay by hour About Matillion
  • 22.
    Customer - UpsideTravel Matt Boegner Data Engineer, Upside Travel Scenario Upside ingests over 3TB of raw flight/hotel inventory on a daily basis, in addition to website, mobile app, and marketing data streams. A lot of data and a lot of data processing! Negative Consequences Existing data warehouse couldn’t keep up Degraded performance due to concurrent loading Expensive to maintain After Matillion ETL for Snowflake made their migration to Snowflake simple and quick, and Upside was able to actualize the benefits of Snowflake within 2 weeks, at production workloads. “Use Snowflake if you have a cloud data warehouse; use Matillion if you use Snowflake.”
  • 23.
    Snowflake Flight/ Hotel data Websitedata Mobile data Marketing data Looker – BI & Visualization Better performance, mitigation of concurrency issues and improved scalability • Migration to production live in under 2 weeks • Scalability (up and down) >> Cost Efficiency (Transform applied directly)
  • 24.
    Matillion ETL forSnowflake • 5 minute launch process on Marketplace • 14 day Free Trial • From $1.37 per hr How to engage • SA team to support PoCs, demos and training • Support Portal • Tutorials & Customer references on YouTube How to Engage
  • 25.
  • 26.
    Use bulk loadingto get the data to the cloud Leverage the scalable compute layer to do the bulk of the data processing Isolate load and transform jobs from queries to prevent resource contention Focus on organizing the data to support business users © 2018 Snowflake | All Rights Reserved. MASTER THE MULTI-CLUSTERED DATA WAREHOUSE Best Practices
  • 27.
    27© 2018 SnowflakeComputing Inc. All Rights Reserved. Q & A
  • 28.
    © 2017 SnowflakeComputing Inc. All Rights Reserved. How will you unite your data nation with Snowflake and Matillion? Ask us how to start today. + #BetterTogether #UniteTheDataNation

Editor's Notes

  • #3 <Alex> Introduce our speakers – pass it over to them to do brief intros.
  • #5 It’s all of us Everyone who cares about working with data Better answers to drive the business forward Whether you are small or big, data nation is for you
  • #6 Data is critical, but it’s a challenging space. Every org struggling to get value from data. Divide: People who are responsible for the data, and the people who are using the data Irordinate amount of time wrangling data
  • #7 Different types It’s all over the place and difficult to bring together Data isn’t just about whats in front of you
  • #9 Connecting different applications GDPR and other regulations Things that used to take 24 hours, now we need under a minute or less
  • #11 Effectively infinite resources - Serve every use case with unbounded compute Connect all of your apps with limitless connectivity Unite your data in one place with infinite storage
  • #13 Basically skip this slide
  • #14 Familiar! The language of analytics. Name other products Not a query tool ETL can work with Snowflake, BI works with Snowflake -The difference is that it SCALES.
  • #15 A huge difference: built as a cloud saas service You load the data, you run the queries, and the rest we take care for you Hard to grasp… But it just works!
  • #16 Many different types of data Complete picture requires multiple variations of data Different environments (SQL Hadoop) Transformation… requires effort Machine generated data has structure! Has a schema. -Heirarchical, repetitive aptterns -Just changing… -Analytics team unaware -Can you find that schema? We find it for you. Virtual column, access with SQL with dot notation
  • #17 Anyone has the rights can gain insight Make decisions and share Ability to allow all of your end users when they want it -Always a food chain – who is on top? -Don’t touch the data warehouse!
  • #18 Monday morning dashboard rush Thousands of nodes at once With Snowflake you only pay for what you use. Compute is separate from storage so you pay for things separately. Snowflake is a fully elastic cloud data warehouse—scale up and down at any time. The cost of storage is comparable to the price of Amazon S3—with a one-year contract your cost of storage is only $23 / compressed TB In other words, you can effectively store data in the world’s most powerful data warehouse for the same price as Amazon S3.
  • #19 Leverage data as a business asset Companies need data to gain insight into their own company -Drive revenue -Competitive insight -Closer partnerships Snowflake enables any data stored in Snowflake, Securely share with anyone -Your account or other Snowflake customers Your data, you have complete ownership, pay for storage Enable other organizations access to the data through secure view When they run queries, they do that in their own account We don’t participate
  • #20 That’s not possible! Multi cluster shared data At the heart of the architecture is the data itself -Blobs within S3 (storage engine) Immutability is a tremendous advantage. When you load its loaded into S3 as a set of files -As the data is brought in, we extract metadata and put it into columns and micropartions for pruning -Don’t declare partition keys -Tens of millions of partitions To do work, you need a virtual warehouse. -Cluster of compute, EC2 nodes on AWS -They do all the processing -Different sizes -Multi-cluster – more than one cluster at a time. Concurrency. -Servcices, the layer that makes this all possible When you login online, you authenticate and it passes through services When you issue a query, we parse your query, perform the right prune, and efficiently scan the data for you When a multi statement transaction starts, every other query will see the data as it was before it began… until the query is done then they see the updated version It all works because S3 is immutable!