SlideShare a Scribd company logo
1 of 79
Download to read offline
1Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Y O U R D A T A , N O L I M I T S
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
@KentGraziano
KENT GRAZIANO
Chief Technical Evangelist
Snowflake Computing
Demystifying
Data Warehousing as a Service
(DWaaS)
2Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Chief Technical Evangelist, Snowflake Computing
ā€¢ Oracle ACE Director (DW/BI)
ā€¢ OakTable Network
ā€¢ Blogger ā€“ The Data Warrior
ā€¢ Certified Data Vault Master and DV 2.0 Practitioner
ā€¢ Former Member: Boulder BI Brain Trust (#BBBT)
ā€¢ Member: DAMA Houston & DAMA International
ā€¢ Data Architecture and Data Warehouse Specialist
ā€¢ 30+ years in IT
ā€¢ 25+ years of Oracle-related work
ā€¢ 20+ years of data warehousing experience
ā€¢ Author & Co-Author of a bunch of books (Amazon)
ā€¢ Past-President of ODTUG and Rocky Mountain Oracle
User Group
My Bio
3Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
About Snowflake
Experienced,
accomplished
leadership team
2012
Founded by
industry veterans
with over 120
database patents
Vision:
A world with
no limits on data
First data
warehouse
built for the
cloud
Over 1000
customers
since GA
4Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Data Challenges
ā€¢ What is a Data Warehouse as a Service?
ā€¢ What can a Cloud DWaaS do for me?
ā€¢ Features of a Cloud DWaaS
ā€¢ Top 10 (or so) Cool Features of Snowflake
ā€¢ Becoming more Agile with DWaaS
ā€¢ Where Snowflake fits
ā€¢ Spark Integration
ā€¢ Continuous Loading
ā€¢ Conclusion
Agenda
5Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Data challenges today
6Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Scenarios Ā with Ā affinity Ā for Ā cloud
Gartner 2016
Predictions:
By 2018, six
billion connected
things will be
requesting
support.
Connecting applications, devices, and
ā€œthingsā€
Reaching employees, business partners,
and consumers
Anytime, anywhere mobility
On demand, unlimited scale
Understanding behavior; generating,
retaining, and analyzing data
7Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
40 Zettabytes by 2020
Web ERP3rd party apps Enterprise apps IoTMobile
8Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Itā€™s not the data itself
itā€™s how you take full advantage of the insight it provides
Web ERP3rd party apps Enterprise apps IoTMobile
9Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Most firms donā€™t consistently turn data into action
Source: Forrester
All Possible data All Possible Action
of firms
aspire to be
data-driven.
73%
of firms are
good at
turning data
into action.
29%
10Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
The data struggle
Data is stale
Hard to incorporate
new data sources
Too much time
waiting for data
Too much time spent
on manual
administration
Reports are too slow
whenever the system
is busy
Hard to experiment
with data
11Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Symptoms of fundamental challenges
Data silos
Data locked into
separate
databases, big
data systems, and
applications
Inflexibility
Slow, cumbersome
scaling and limited
support for diverse
data
Complexity
Multiple systems
to integrate and
manage requiring
specialized skills
and tools
Performance
Contention for
limited resources
resulting in latency
and delays
Cost
Painful upfront
costs and
overprovisioned
capacity
12Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
The evolution of data platforms
Data warehouse
& platform
software
Vertica,
Greenplum,
Paraccel, Hadoop
Data
warehouse
appliance
Teradata
1990s 2000s 2010s
Cloud DWaaS
Snowflake
1980s
Relational
database
Oracle, DB2,
SQL Server
13Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What is a Cloud DWaaS?
DW- Data Warehouse
ā€¢ Relational database
ā€¢ Uses standard SQL
ā€¢ Optimized for fast loads and
analytic queries
aaS ā€“ As a Service
ā€¢ Like SaaS (e.g. SalesForce.com)
ā€¢ No infrastructure set up
ā€¢ Minimal to no administration
ā€¢ Managed for you by the vendor
ā€¢ Pay as you go, for what you use
14Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Goals of a Cloud DWaaS
Make your life easier
ā€¢ So you can load and use your data
faster
Support business
ā€¢ Make data accessible to more people
ā€¢ Reduce time to insights
Handle big data too!
ā€¢ Schema-less ingestion
15Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What to Expect from a
Cloud DWaaS It should support standard SQL
(natively)
ā€¢ It should support standard ETL, BI
& data science tools
ā€¢ ODBC or JDBC connectivity
It should be infinitly scalable
(cloud)
ā€¢ Handle huge amounts of data
ā€¢ Handle large number of concurrent
queries without performance
degradation
It should handle flexible schema
data types
ā€¢ No sharding or ETL required
16Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What to Expect from a
Cloud DWaaS
It should be secure
ā€¢ Built in encryption?
It shoud be stable
ā€¢ Resiliancy and availability should be
easy to configure and manage
It should be easy to configure and
manage
It should provide a lower TCO
ā€¢ Cloud scale pricing
17Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Common customer scenarios
Data Ā warehouse Ā 
for Ā SaaS Ā offerings
Use Ā Cloud Ā DW as Ā back-Ā­
end Ā data Ā warehouse Ā 
supporting Ā data-Ā­driven Ā 
SaaS Ā products
noSQL Ā replacement
Replace Ā use Ā of Ā noSQL Ā 
system Ā (e.g. Ā Hadoop) Ā for Ā 
transformation Ā and Ā SQL Ā 
analytics Ā of Ā multi-Ā­
structured Ā data Ā 
Data Ā warehouse Ā 
modernization
Consolidate Ā legacy Ā 
datamarts Ā and Ā support Ā 
new Ā projects
18Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Enabling key use cases
Datamart & data silo consolidation
Consolidate legacy datamarts to eliminate silos and support new projects
Integrated data analytics
Directly load structured + semi-structured data into Snowflake for reporting
& analytics
Data Science, Exploratory & ad hoc analytics
Direct access to data for SQL analysts and data scientists to explore data,
identify correlations, build & test models
19Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Introducing Snowflake
20Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflake:
1st Data Warehouse Built for the Cloud
SQL relational database
Optimized storage & processing
Standard connectivity ā€“ BI, ETL, ā€¦
Data Warehousingā€¦
Existing SQL skills and tools
ā€œLoad and goā€ ease of use
Cloud-based elasticity to fit any scale
Data
scientists
SQL
users &
tools
ā€¦for Everyone
21Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What is Snowflake?
Built for the
cloud
SQL Data
Warehouse
Our Vision
Provide all users anytime, anywhere
insights so they can make actionable
decisions based on data
Our Solution
Next-generation data warehouse built
from the ground up for the cloud and
for todayā€™s data and analytics
Delivered as
a service
22Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflakeā€™s differentiating technology
Unique architecture: Multi-cluster, shared data
Single place for
data
z
Instant, unlimited
scalability
Zero
management
Instant, live data
sharing
23Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
The Data Warriorā€™s
Top 10+ Cool Things About Snowflake
(A Data Geeks Guide to Cloud-native DW)
24Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#10 ā€“ Persistent Result Sets
ā€¢ No setup
ā€¢ In Query History
ā€¢ By Query ID
ā€¢ 24 Hours
ā€¢ No re-execution
ā€¢ No Cost for Compute
25Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#9 -Connect w/JDBC & ODBC to the cloud
Data Sources
Custom & Packaged
Applications & Data
Science
ODBC WEB UIJDBC
Interfaces
Java
>_
Scripting
Reporting
& Analytics
Data Modeling,
Management &
Transformation
SDDM
26Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#8 - UNDROP
UNDROP TABLE <table name>
UNDROP SCHEMA <schema name>
UNDROP DATABASE <db name>
Part of Time Travel feature: AWESOME!
27Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#7 Fast Clone (Zero-Copy)
Instant copy of table, schema, or database:
CREATE OR REPLACE TABLE MyTable_V2
CLONE MyTable;
With Time Travel:
CREATE SCHEMA mytestschema_clone_restore
CLONE testschema
BEFORE (TIMESTAMP =>
TO_TIMESTAMP(40*365*86400));
PROD
PUBLIC
Table A Table B
Table C
DEV
PUBLIC
Table A Table B
Table C
PUBLIC
Table A Table B
Table C
INT
28Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#6 ā€“ JSON Support with SQL
Apple 101.12 250 FIH-2316
Pear 56.22 202 IHO-6912
Orange 98.21 600 WHQ-6090
Structured data
(e.g. CSV)
Semi-structured data
(e.g. JSON, Avro, XML)
{ "firstName": "John",
"lastName": "Smith",
"height_cm": 167.64,
"address": {
"streetAddress": "212nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{ "type": "home", "number":"212555-1234"},
{ "type": "office", "number": "646 555-4567"}
]
}
Optimized storage
Flexible schema - Native
Relational processing
select v:lastName::string as last_name
from json_demo;
All Your Data!
29Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#5 ā€“ Standard SQL w/Analytic Functions
select Nation, Customer, Total
from (select
n.n_name Nation,
c.c_name Customer,
sum(o.o_totalprice) Total,
rank() over (partition by n.n_name
order by sum(o.o_totalprice) desc)
customer_rank
from orders o,
customer c,
nation n
where o.o_custkey = c.c_custkey
and c.c_nationkey = n.n_nationkey
group by 1, 2)
where customer_rank <= 3
order by 1, customer_rank
SQL
Complete SQL database
ā€¢ Data definition language (DDLs)
ā€¢ Query (SELECT)
ā€¢ Updates, inserts and deletes (DML)
ā€¢ Role based security
ā€¢ Multi-statement transactions
New partner post: https://sonra.io/2018/02/04/create-custom-aggregate-udaf-window-functions-snowflake/
30Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#4 ā€“ Separation of Storage & Compute
Snowflakeā€™s multi-cluster, shared data architecture
Centralized storage
Instant, automatic scalability & elasticity
Service
Compute
Storage
31Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#3 ā€“ Support Multiple Workloads
Accelerate the data pipeline
Run loading & analytics at any time, concurrently, to get
data to users faster
Scale compute to support any workload
Scale processing horsepower up and down on-the-fly,
with zero downtime or disruption
Scale concurrency without performance impact
Multi-cluster ā€œvirtual warehouseā€ architecture scales
concurrent users & workloads without contention
Deliver faster analytics
at any scale
Loading
Marketing
Finance
32Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Elastic scaling for storage
Low-cost cloud storage, fully
replicated and resilient
ā€¢ Elastic scaling for compute
Virtual warehouses scale up &
down on the fly to support
workload needs
ā€¢ Elastic scaling for concurrency
Automatically scale
concurrency using multi-
cluster virtual warehouses
Instant, unlimited scalability
33Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#2 Ā ā€“ Secure Ā by Ā Design Ā with Ā Automatic Ā Encryption Ā of Ā Data!
Embedded
multi-factor authentication
Federated authentication
available
Certified against enterprise-
class requirements
HIPPA Certified!
PCI Certified!
All data encrypted, always,
end-to-end
Encryption keys managed
automatically
NEW: Tri-secret security
Role-based access
control model
Granular privileges on all
objects & actions
Authentication Access control Data encryption External validation
34Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
#1 - Automatic Query Optimization
Zero Management
Fully managed with no knobs or tuning required
No indexes, distribution keys, partitioning,
vacuuming,ā€¦
Zero infrastructure costs
Zero admin costs
35Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
New #1 Fav ā€“ Data Sharing (The Data ā€œSharehouseā€)
Data
Consumers
Data
Providers
No data movement
Share with unlimited
number of consumers
Live access
Data consumers
immediately see all updates
Ready to use
Consumers can immediately
start querying
36Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Data Sharing - Configuring access
create share s1; --empty share
grant usage on database sales to share s1; -- add database
grant usage on schema sales.east to share s1; -- add schema
grant usage on view sales.east.accts to share s1; -- add view
alter share s1 add accounts=a1, a2, a3; -- add accounts
s1
Provider Database
Data Share Object
Schema List
View List
Account List
a3
a1
a2
Data provider Data Consumers
create database sales from share p1.s1;
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Enabling Agile DW in the Cloud
Examples
38Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢Agile Warehouse Scaling
ā€¢ Separation of Workloads
ā€¢ Virtual WH Scaling Techniques
ā€¢Agile Data Lifecycle
ā€¢ Cloning
Enabling the Agile Data Warehouse
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Agile Ā Warehouse Ā Scaling
Separation Ā of Ā Workloads
40Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ DWaaS - Snowflake solution
ā€¢ Assign each workload its own Virtual WH
ā€¢ At a minimum, two WH: ETL and Business
ā€¢ ETL can run continuously if it makes sense for the
business
ā€¢ ETL / Business workload contention is eliminated
ā€¢ Further subdivide business workloads into
own clusters as needed
ā€¢ Eliminate contention between Sales, Marketing,
Finance, Data Science workloads
ā€¢ International groups can operate clusters on their
own local time schedules
ā€¢ Easy to add new workloads on demand!
Separation of Workloads
Virtual
Warehouse
Databases
Virtual
Warehouse
ETL & Data Loading
Business Workloads
Finance
Virtual
Warehouse
Test/Dev
Virtual
Warehouse
S
Marketing
Virtual
Warehouse
Sales
Virtual
Warehouse
S
Research
41Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Increase cluster size ā€“ on the fly!
ā€¢ More data being analyzed
ā€¢ More complex queries
ā€¢ Get some concurrency boost
ā€¢ No need to start, stop, or reboot!
ā€¢ Automatic scale out!
ā€¢ Multi-cluster warehouse concurrency
ā€¢ Workload queries have usual weight
ā€¢ But more of them, e.g. 20 dashboard users rather than the
usual 5
ā€¢ Automatically scales back after the load
drops
Other Agile Warehouse Scaling Techniques
42Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Anticipated surges
ā€¢ Explicitly increase WH nodes (T-shirt size) when expecting more data
ā€¢ Explicitly increase MCWH minimum clusters when expecting more queries
ā€¢ Can do both at once with ALTER WAREHOUSE
ā€¢ Use cron or other scheduling/orchestration tool
ā€¢ Unanticipated surges
ā€¢ Rely on MCWH maximum clusters for some extra headroom
ā€¢ Maximize Business Agility!
ā€¢ Responsiveness for users
ā€¢ Throughput and value extracted from variable compute power
ā€¢ Minimize
ā€¢ Cost and administrative overhead
Agile Warehouse Scaling ā€“ Best Practices
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Agile Ā Data Ā Lifecycle
44Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Separation of Workloads
ā€¢ Individual virtual warehouse for each dev/test/prod functional area
ā€¢ CLONE for dev/test or Data Science Sandbox ā€“ on demand!
ā€¢ Full logical copy of the data, but uses no extra storage
ā€¢ Test/dev operations against clone have no effect on original data
ā€¢ Security
ā€¢ RBAC limits dev/test access to clone and not production data
ā€¢ Secure Views permit role- or user-based obfuscation / masking / projection
ā€¢ Business Impact ā€“ better quality code
ā€¢ Dev and test teams are working on data at scale, see true app performance
ā€¢ Full range of values means fewer surprises when app encounters live data
Agile Data Lifecycle
45Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Create development (DEV) and integration (INT) databases from production
(PROD)
Scenario 1
PROD
PUBLIC
Table
A
Table B
INT
PUBLIC
Table
A
Table B
DEV
PUBLIC
Table
A
Table B
CLONE
CLONE
CREATE OR REPLACE DATABASE DEV
CLONE PROD
CREATE OR REPLACE DATABASE INT
CLONE PROD
46Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Create two new tables, C and D, in the development (DEV) database
Scenario 2: new development
PROD
PUBLIC
Table
A
Table B
INT
PUBLIC
Table
A
Table B
DEV
PUBLIC
Table
A
Table B
Table
C
Table
D
47Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Mini-release: promote table C for integration testing
Scenario 2: new development
PROD
PUBLIC
Table
A
Table B
INT
PUBLIC
Table
A
Table B
DEV
PUBLIC
Table
A
Table B
Table
C
Table
D
Table
C
CREATE
TABLE
LIKE
48Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Deploy to production: promote table C to PROD database
Scenario 2: new development
PROD
PUBLIC
Table
A
Table B
INT
PUBLIC
Table
A
Table B
DEV
PUBLIC
Table
A
Table B
Table
C
Table
D
Table
C
CREATE
TABLE
LIKE
Table
C
49Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Refresh dev: get latest PROD data into DEV and INT
Scenario 2: new development
PROD
PUBLIC
Table
A
Table B
INT
PUBLIC
Table
A
Table B
DEV
PUBLIC
Table
A
Table B
Table
C
Table
D
Table
C
Table
C
CLONE
DEV
PUBLIC
Table
A
Table B
Table
C
Table
D
DEV2
CLONE
CREATE OR REPLACE
TABLE Dev2.TableD
CLONE Dev.TableD
CREATE OR REPLACE
DATABASE DEV2
CLONE PROD
50Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ CLONE for Data Scientists
ā€¢ Quick and Safe sandbox for discovery and testing
ā€¢ Experiment and explore, and write back!
ā€¢ Test hypothisis and keep iterating ā€“ drop and reclone as needed
ā€¢ No impact to production data
ā€¢ Combine with own virtual warehouse for complete isolation
ā€¢ Business Impact ā€“ better data science
ā€¢ More fine-grained data over longer time intervals
ā€¢ Deeper insights, better forecasting, more monetizable results
ā€¢ CLONE for Compliance
ā€¢ Monthly, quarterly, annual clones ā€“ financial reporting, auditing requirements
ā€¢ Business Impact ā€“ simpler compliance
ā€¢ Your "backups" are live and immediately available
Agile Data Lifecycle
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflake & Spark
52Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Where Snowflake fits with complementary solutions
Vast majority, >> 80% or
more, is semi-/structured
>> 20% or less is unstructured
Data
Sources
for
Analytics
(Audio, video, text, etc.)
(Tables, CSV, JSON, Avro,
Parquet, XML, flat files, etc.)
Complementary
Special data processing
(Spark ML, Hadoop lakes, NoSQL
front-ends, In-memory DBs,
non-relational exploring, et al.)
Built for the cloud, relational
data warehouse/DB & analytics
delivered as a service
52
Use Cases
(Tables, metadata)
ā€¢ Live analytics
ā€¢ Operational DW
ā€¢ Real-time dashboards
ā€¢ Continuous data loading
ā€¢ Change data capture
ā€¢ Traditional DW/BI
ā€¢ Consolidation of data silos
ā€¢ Relational data exploration
ā€¢ Interactive queries
ā€¢ Data science/statistics (Python, R)
ā€¢ And more
ā€¢ Spark machine learning
ā€¢ Unstructureddata mining
ā€¢ Streaming analytics
ā€¢ Non-relational data exploration
ā€¢ Non-relational data science
ā€¢ And more
53Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Modern Ā Data Ā Architectures
Data Production Data Lake Data
Consumption
Web
Servers
Data
Bases
App
Metrics
Snowflake
Business
Intelligence
Data Science:
Exploration
Data Science:
Machine Learning
Model Training
Data
Augmentation &
Loading
Data
Augmentation &
Loading
Model-driven
Apps
Online
Recommendations
& Behavioral
Targeting on Web
Pages
Offline Churn
Prediction
Other Machine
Learning Apps
54Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Modern Ā Data Ā Architectures: Ā 
Data Ā Loading Ā & Ā Augmentation
Web
Servers
App
Metrics
Data
Bases
Data ProductionCustomers &
End Users
External Data
Augmentation
Spark
Cluster
Kafka
SnowSQL
COPY
Kinesis
Transformations and
Augmentations in e.g.
Python, Scala,
SparkSQL, Spark
Streaming etc.
Snowflake
Warehouse
Snowflake
Spark
Connector
Full/Pruned
Data Lake with
Augmentation
Snowflake
Spark
Connector
Source/Raw
Data
Augmented
Data
Source/Raw
Data
Augmented
Data
Transformations and
Augmentations using Snowflake
SQL
Augmentation
Augmentation
CDC -
FiveTran
55Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Modern Ā Data Ā Architectures: Ā Data Ā Science
Jupyter
Notebook
using e.g.
ML
Data Exploration
Model Design
Spark app
using
Spark ML
Model
(Re-)Training
Operationalized
Model (Code + Params)
External Data
Augmentation
Spark
Cluster
Snowflake
Warehouse
Snowflake
Spark
Connector
Data Lake with
Augmentation
Snowflake
Spark
Connector
Source/Raw
Data
Augmented
Data
Source/Raw
Data
Augmented
Data
Augmentation
Augmentation
Data Exploration
using Snowflake SQL
56Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflake WarehouseSpark Cluster (e.g. in AWS EMR)
AWS S3
3 2
Data Flow:
Data Frames
Data Flow:
Snowflake Tables
Master Node
Slave Nodes
Query
(SQL)
Cloud Services
Virtual Warehouse
1
Architecture: Ā Data Ā Interchange Ā (Read)
57Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflake WarehouseSpark Cluster (e.g. in AWS EMR)
AWS S3
1 3
Data Flow:
Data Frames
Data Flow:
Snowflake Tables
Master Node
Slave Nodes
ControlFlow
(SQL)
Cloud Services
Virtual Warehouse
2
Architecture: Ā Data Ā Interchange Ā (Write)
58Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Spark Cluster
Snowflake Virtual Warehouse
Super-Charge Spark Processing with Snowflake
ā€¢ Spark optimizer extension
automatically identifies Spark
operations with corresponding
Snowflake implementations
ā€¢ Spark connector pushes these
operations into Snowflake SQL
ā€¢ Pushed operations include: project,
filter, join, aggregation, limit
ā€¢ Optimized Snowflake SQL applied to
all Spark data frames backed by
Snowflake tables
ā€¢ Transparent to Spark language choice:
No need to rewrite Spark (Python,
Scala, Java, ā€¦) or SparkSQL code
df1 = spark.loadā€¦(ā€˜T1ā€™)
df2 = spark.loadā€¦(ā€˜T2ā€™)
df1
.join(df2, ā€˜idā€™)
.groupBy(ā€˜regionā€™)
.count()
.collect()
SELECT region, count(*)
FROM T1
JOIN T2
ON T1.id = T2.id
GROUPBY region
Snowflake SQL
AWS
S3
Results
59Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Reduced complexity, better manageability
ā€¢ Same data store for curated and raw data
ā€¢ Same data store for data science and BI data sets
ā€¢ Fully transactional data store even for raw data when needed
ā€¢ Performance benefits
ā€¢ Leveraging structure of the data even in semi-structured data formats such as JSON as compared to flat file
storage
ā€¢ Metadata for pruning based on queries
ā€¢ Easy to integrate with Spark as compute and application platform
ā€¢ Overall compute capacity and cost savings due to performance benefits with
Snowflake
ā€¢ Blog posts:
ā€¢ https://www.snowflake.net/snowflake-and-spark-part-1-why-spark/
ā€¢ https://www.snowflake.net/snowflake-spark-part-2-pushing-query-processing/
Snowflake + Spark Benefits
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
NEW
Continous Data Loading with Snowpipe
61Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Data is created constantly.Data accumulated over time.
Data is Being Generated Faster than Ever
Before
Infrequent
Now
Constant
Web
data
Corp
data
App
data
Thereā€™s an enormous opportunity to use
continuously generated data in analysis.
62Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowpipe is an automated service that
asynchronously listens for new data as it arrives in
Amazon Web Servicesā€™ S3 cloud storage service
and continuously loads that data into Snowflake.
Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake
Confidential ā€“ under NDA only.
63Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
6
Data Ingress Approaches & Snowpipe Support
Approach Approach Definition Snowpipe
Batch Data accumulates over time (hours,
days) and is then loaded periodically
Option 1:
Point at an S3 bucket and a destination table in
your warehouse at which point new data is
automatically uploaded.
Option 2:
Technical resources can interface directly with the
programmatic REST API along with Java and Python
SDKs to enable highly customized loading use cases
Micro-batch Data accumulates over small time
windows (minutes) and is then loaded
Continuous Every data item is loaded individually as
it arrives
64Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowpipe Scenario: Automatic Loading from S3
Snowflake
Database
External
S3
SnowPipe Service
Server-less
Loader
S3 notification
File data
65Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowpipe ā€“ Current Preview
Snowflake
Database
REST Call
{file names}
S3
Application
SnowPipe Service
REST Endpoint
Server-less
Loader
66Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowpipe Auto-Ingest: Fully Automatic Loading from S3
Snowflake
Database
External
S3
SnowPipe Service
Server-less
Loader
S3 notification
File data
Use simple Snowflake DDL statements to configure bucket for automatic load with Snowpipe
See the video here: https://www.youtube.com/embed/rwZWmcy0aBU?autoplay=1&rel=0
Blog posts:
https://www.snowflake.net/your-first-steps-with-snowpipe/
https://www.snowflake.net/snowpipe-serverless-loading-for-streaming-data-2/
67Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Key Snowpipe Benefits
You only pay
for the compute
time you use to
load data
Avoid repeated
manual COPY
commands
Continuously
generated data is
available for
analysis in seconds
0 management. No
indexing, tuning,
partitioning or
vacuuming on load
Full support for
semi-structured
data on load
Server-less: No
servers to manage or
concurrency to worry
about.
68Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
ā€¢ Server-less billing model: utilization-
based billing
ā€¢ No warehouse to manage for load
ā€¢ Per core, per second granularity
ā€¢ Charged as Snowflake credits
ā€¢ New line item on Snowflake bill
ā€¢ Components of the Snowpipe
charges:
ā€¢ Loading work ā€“ idle times are not charged
ā€¢ Time spent in metadata management for file
loading
6
Pricing
69Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Snowflake in Action Today
70Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What customers are doing with Snowflake
DATA Ā MARTS Ā 
& Ā EXTRACTS
Market research
company consolidated
data marts to reduce
costs and data silos
Gaming company
replaced Hadoop +
SQL database with
Snowflake
STAGING
DATA Ā LAKE
DATA Ā 
WAREHOUSE
Consumer retailer
modernizing DW by
replacing legacy
appliance with
Snowflake
Mobile analytics
company shares live
data with clients
REPORTING, Ā ANALYTICS Ā 
& Ā APPLICATIONS
DATA Ā SOURCES
71Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Delivering compelling results
Simpler data pipeline
Replace noSQL database with Snowflake
for storing & transforming JSON event
data
Faster analytics
Replace on-premises data warehouse with
Snowflake for analytics workload
Significantly lower cost
Improved performance while adding new
workloads--at a fraction of the cost
noSQL data base:
8 hours to prepare data
Snowflake:
1.5 minutes
Data warehouse appliance:
20+ hours
Snowflake:
45 minutes
Data warehouse appliance:
$5M + to expand
Snowflake:
added 2 new workloads for $50K
72Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Over 1000 customers demonstrating whatā€™s possible
Up to 200x faster reports that enable analysts to make
decisions in minutes rather than days
Ability to load and update data in near real time by replacing
legacy data warehouse + Hadoop cluster
New applications that provide secure access to analytics to
11,000+ pharmacies
73Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Chosen by leading enterprises
Built advertising analytics
platform on Snowflake
Moving enterprise reporting to
Snowflake
Deploying BI solution stack on
Snowflake
Built new embedded analytics
application on Snowflake
Moved audience reporting
infrastructure to Snowflake
Replaced Hadoop + legacy data
warehouse with Snowflake
74Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Ranked #1 Cloud Data Warehouse!
ā€œSnowflake Hits All the Marksā€ ā€“ Gigaom
4,85
4,50
4,45
3,75
3,75
3,35
3,20
3,15
2,60
Cloud
Analytics
Database
Distruption Vectors
AWS Redshift
Oracle Database Exdata Cloud Service
SAP HANA Cloud Platform
Azure Data Warehouse
Vertica
DashDB (IBM)
Teradata
Google Big Query
Snowflake
RobustnessofSQL15%
Built-inOptimization15%
On-the-flyElasticity25%
DynamicEnvironment
Adaption20%
SeparationofCompute
fromstorage15%
SupportforDiversedata
10%
Score
ā€œYou can tell the data warehouse
pedigree from the developmentā€¦
With superior performance and
the most hands-off model of
ownership, Snowflake is the
epitome of data warehouse as a
service. The model, cost, features
and scalability have already
caused some to postpone
Hadoop adoption.ā€
William McKnight
Gigaom
Disruption
Vectors
Gigaom Analyst Report: Sector Roadmap: Cloud Analytic
Databases 2017
Read the full report on snowflake.net
75Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
What does a Cloud-native
DWaaS Provide?
Cost effective storage and analysis of
GBs, TBs, or even PBā€™s
Lightning fast query performance
Continuous data loading without
impacting query performance
Unlimited user concurrency
Full SQL relational support of both
structured and semi-structured data
Support for the tools and languages you
already useODBC WEB UIJDBCJava
>_
Scripting
76Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Big Data does not have to
equal Big Effort
Web ERP3rd party
apps
Enterprise
apps
IoTMobile
77Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Discover the performance, concurrency, and simplicity of
Snowflake
As easy as 1-2-3!
01 Visit Snowflake.net
02 Click ā€œTry for Freeā€
03 Sign up & register
Snowflake is the only data warehouse built for the
cloud. You can automatically scale compute up,
out, or downā€”independent of storage. Plus, you
have the power of a complete SQL database, with
zero management, that can grow with you to
support all of your data and all of your users. With
Snowflake On Demandā„¢, pay only for what you
use.
Sign up and receive
$400 worth of free
usage for 30 days!
Kent Graziano
Snowflake Computing
Kent.graziano@snowflake.net
On Twitter @KentGraziano
More info at
http://snowflake.net
Visit my blog at
http://kentgraziano.com
Contact Information
79Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
YOUR DATA, NO LIMITS
Ā© 2017 Snowflake Computing Inc. All Rights Reserved.
Thank You!

More Related Content

What's hot

How to Take Advantage of an Enterprise Data Warehouse in the Cloud
How to Take Advantage of an Enterprise Data Warehouse in the CloudHow to Take Advantage of an Enterprise Data Warehouse in the Cloud
How to Take Advantage of an Enterprise Data Warehouse in the CloudDenodo
Ā 
Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglyTyler Wishnoff
Ā 
Letā€™s get to know Snowflake
Letā€™s get to know SnowflakeLetā€™s get to know Snowflake
Letā€™s get to know SnowflakeKnoldus Inc.
Ā 
Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation Brett VanderPlaats
Ā 
Snowflake Architecture.pptx
Snowflake Architecture.pptxSnowflake Architecture.pptx
Snowflake Architecture.pptxchennakesava44
Ā 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMatillion
Ā 
Data Sharing with Snowflake
Data Sharing with SnowflakeData Sharing with Snowflake
Data Sharing with SnowflakeSnowflake Computing
Ā 
Introduction to snowflake
Introduction to snowflakeIntroduction to snowflake
Introduction to snowflakeSunil Gurav
Ā 
Actionable Insights with AI - Snowflake for Data Science
Actionable Insights with AI - Snowflake for Data ScienceActionable Insights with AI - Snowflake for Data Science
Actionable Insights with AI - Snowflake for Data ScienceHarald Erb
Ā 
The Marriage of the Data Lake and the Data Warehouse and Why You Need Both
The Marriage of the Data Lake and the Data Warehouse and Why You Need BothThe Marriage of the Data Lake and the Data Warehouse and Why You Need Both
The Marriage of the Data Lake and the Data Warehouse and Why You Need BothAdaryl "Bob" Wakefield, MBA
Ā 
Lakehouse Analytics with Dremio
Lakehouse Analytics with DremioLakehouse Analytics with Dremio
Lakehouse Analytics with DremioDimitarMitov4
Ā 
An overview of snowflake
An overview of snowflakeAn overview of snowflake
An overview of snowflakeSivakumar Ramar
Ā 
KSnow: Getting started with Snowflake
KSnow: Getting started with SnowflakeKSnow: Getting started with Snowflake
KSnow: Getting started with SnowflakeKnoldus Inc.
Ā 
Changing the game with cloud dw
Changing the game with cloud dwChanging the game with cloud dw
Changing the game with cloud dwelephantscale
Ā 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
Ā 
Snowflake Best Practices for Elastic Data Warehousing
Snowflake Best Practices for Elastic Data WarehousingSnowflake Best Practices for Elastic Data Warehousing
Snowflake Best Practices for Elastic Data WarehousingAmazon Web Services
Ā 

What's hot (20)

How to Take Advantage of an Enterprise Data Warehouse in the Cloud
How to Take Advantage of an Enterprise Data Warehouse in the CloudHow to Take Advantage of an Enterprise Data Warehouse in the Cloud
How to Take Advantage of an Enterprise Data Warehouse in the Cloud
Ā 
Elastic Data Warehousing
Elastic Data WarehousingElastic Data Warehousing
Elastic Data Warehousing
Ā 
Snowflake Overview
Snowflake OverviewSnowflake Overview
Snowflake Overview
Ā 
Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the Ugly
Ā 
Letā€™s get to know Snowflake
Letā€™s get to know SnowflakeLetā€™s get to know Snowflake
Letā€™s get to know Snowflake
Ā 
Zero to Snowflake Presentation
Zero to Snowflake Presentation Zero to Snowflake Presentation
Zero to Snowflake Presentation
Ā 
Snowflake Architecture.pptx
Snowflake Architecture.pptxSnowflake Architecture.pptx
Snowflake Architecture.pptx
Ā 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
Ā 
Data Sharing with Snowflake
Data Sharing with SnowflakeData Sharing with Snowflake
Data Sharing with Snowflake
Ā 
Introduction to snowflake
Introduction to snowflakeIntroduction to snowflake
Introduction to snowflake
Ā 
Actionable Insights with AI - Snowflake for Data Science
Actionable Insights with AI - Snowflake for Data ScienceActionable Insights with AI - Snowflake for Data Science
Actionable Insights with AI - Snowflake for Data Science
Ā 
The Marriage of the Data Lake and the Data Warehouse and Why You Need Both
The Marriage of the Data Lake and the Data Warehouse and Why You Need BothThe Marriage of the Data Lake and the Data Warehouse and Why You Need Both
The Marriage of the Data Lake and the Data Warehouse and Why You Need Both
Ā 
Lakehouse Analytics with Dremio
Lakehouse Analytics with DremioLakehouse Analytics with Dremio
Lakehouse Analytics with Dremio
Ā 
snowpro (1).pdf
snowpro (1).pdfsnowpro (1).pdf
snowpro (1).pdf
Ā 
An overview of snowflake
An overview of snowflakeAn overview of snowflake
An overview of snowflake
Ā 
KSnow: Getting started with Snowflake
KSnow: Getting started with SnowflakeKSnow: Getting started with Snowflake
KSnow: Getting started with Snowflake
Ā 
How to build a successful Data Lake
How to build a successful Data LakeHow to build a successful Data Lake
How to build a successful Data Lake
Ā 
Changing the game with cloud dw
Changing the game with cloud dwChanging the game with cloud dw
Changing the game with cloud dw
Ā 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Ā 
Snowflake Best Practices for Elastic Data Warehousing
Snowflake Best Practices for Elastic Data WarehousingSnowflake Best Practices for Elastic Data Warehousing
Snowflake Best Practices for Elastic Data Warehousing
Ā 

Similar to Demystifying Data Warehousing as a Service - DFW

Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeKent Graziano
Ā 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database RoundtableEric Kavanagh
Ā 
Demystifying Data Warehousing as a Service (GLOC 2019)
Demystifying Data Warehousing as a Service (GLOC 2019)Demystifying Data Warehousing as a Service (GLOC 2019)
Demystifying Data Warehousing as a Service (GLOC 2019)Kent Graziano
Ā 
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...Matillion
Ā 
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Denodo
Ā 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubCloudera, Inc.
Ā 
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...DataStax Academy
Ā 
Demystifying Data Warehouse as a Service (DWaaS)
Demystifying Data Warehouse as a Service (DWaaS)Demystifying Data Warehouse as a Service (DWaaS)
Demystifying Data Warehouse as a Service (DWaaS)Kent Graziano
Ā 
Insights into Real-world Data Management Challenges
Insights into Real-world Data Management ChallengesInsights into Real-world Data Management Challenges
Insights into Real-world Data Management ChallengesDataWorks Summit
Ā 
Architecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseArchitecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseAmazon Web Services
Ā 
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataBig Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataMatt Stubbs
Ā 
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataBig Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataMatt Stubbs
Ā 
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?Denodo
Ā 
Snowflakeā€™s Cloud Data Platform and Modern Analytics
Snowflakeā€™s Cloud Data Platform and Modern AnalyticsSnowflakeā€™s Cloud Data Platform and Modern Analytics
Snowflakeā€™s Cloud Data Platform and Modern AnalyticsSenturus
Ā 
The new dominant companies are running on data
The new dominant companies are running on data The new dominant companies are running on data
The new dominant companies are running on data SnapLogic
Ā 
Introduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph DatabaseIntroduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph DatabaseDataStax Academy
Ā 
Insights into Real World Data Management Challenges
Insights into Real World Data Management ChallengesInsights into Real World Data Management Challenges
Insights into Real World Data Management ChallengesDataWorks Summit
Ā 
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...Cloudera, Inc.
Ā 
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016StampedeCon
Ā 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Databricks
Ā 

Similar to Demystifying Data Warehousing as a Service - DFW (20)

Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with Snowflake
Ā 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database Roundtable
Ā 
Demystifying Data Warehousing as a Service (GLOC 2019)
Demystifying Data Warehousing as a Service (GLOC 2019)Demystifying Data Warehousing as a Service (GLOC 2019)
Demystifying Data Warehousing as a Service (GLOC 2019)
Ā 
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...
Simplifying Your Journey to the Cloud: The Benefits of a Cloud-Based Data War...
Ā 
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Self Service Analytics and a Modern Data Architecture with Data Virtualizatio...
Ā 
The Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data HubThe Future of Data Management: The Enterprise Data Hub
The Future of Data Management: The Enterprise Data Hub
Ā 
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Cassandra Day SV 2014: Apache Cassandra at Equinix for High Performance, Scal...
Ā 
Demystifying Data Warehouse as a Service (DWaaS)
Demystifying Data Warehouse as a Service (DWaaS)Demystifying Data Warehouse as a Service (DWaaS)
Demystifying Data Warehouse as a Service (DWaaS)
Ā 
Insights into Real-world Data Management Challenges
Insights into Real-world Data Management ChallengesInsights into Real-world Data Management Challenges
Insights into Real-world Data Management Challenges
Ā 
Architecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the EnterpriseArchitecting an Open Data Lake for the Enterprise
Architecting an Open Data Lake for the Enterprise
Ā 
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataBig Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Ā 
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on DataBig Data LDN 2017: The New Dominant Companies Are Running on Data
Big Data LDN 2017: The New Dominant Companies Are Running on Data
Ā 
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?
SAP Analytics Cloud: Haben Sie schon alle Datenquellen im Live-Zugriff?
Ā 
Snowflakeā€™s Cloud Data Platform and Modern Analytics
Snowflakeā€™s Cloud Data Platform and Modern AnalyticsSnowflakeā€™s Cloud Data Platform and Modern Analytics
Snowflakeā€™s Cloud Data Platform and Modern Analytics
Ā 
The new dominant companies are running on data
The new dominant companies are running on data The new dominant companies are running on data
The new dominant companies are running on data
Ā 
Introduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph DatabaseIntroduction to DataStax Enterprise Graph Database
Introduction to DataStax Enterprise Graph Database
Ā 
Insights into Real World Data Management Challenges
Insights into Real World Data Management ChallengesInsights into Real World Data Management Challenges
Insights into Real World Data Management Challenges
Ā 
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...
Turning Petabytes of Data into Profit with Hadoop for the Worldā€™s Biggest Ret...
Ā 
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Ā 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Ā 

More from Kent Graziano

Balance agility and governance with #TrueDataOps and The Data Cloud
Balance agility and governance with #TrueDataOps and The Data CloudBalance agility and governance with #TrueDataOps and The Data Cloud
Balance agility and governance with #TrueDataOps and The Data CloudKent Graziano
Ā 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for DinnerKent Graziano
Ā 
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...Kent Graziano
Ā 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeKent Graziano
Ā 
Rise of the Data Cloud
Rise of the Data CloudRise of the Data Cloud
Rise of the Data CloudKent Graziano
Ā 
Making Sense of Schema on Read
Making Sense of Schema on ReadMaking Sense of Schema on Read
Making Sense of Schema on ReadKent Graziano
Ā 
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)Agile Data Engineering: Introduction to Data Vault 2.0 (2018)
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)Kent Graziano
Ā 
Extreme BI: Creating Virtualized Hybrid Type 1+2 Dimensions
Extreme BI: Creating Virtualized Hybrid Type 1+2 DimensionsExtreme BI: Creating Virtualized Hybrid Type 1+2 Dimensions
Extreme BI: Creating Virtualized Hybrid Type 1+2 DimensionsKent Graziano
Ā 
Agile Data Warehousing: Using SDDM to Build a Virtualized ODS
Agile Data Warehousing: Using SDDM to Build a Virtualized ODSAgile Data Warehousing: Using SDDM to Build a Virtualized ODS
Agile Data Warehousing: Using SDDM to Build a Virtualized ODSKent Graziano
Ā 
Agile Data Engineering - Intro to Data Vault Modeling (2016)
Agile Data Engineering - Intro to Data Vault Modeling (2016)Agile Data Engineering - Intro to Data Vault Modeling (2016)
Agile Data Engineering - Intro to Data Vault Modeling (2016)Kent Graziano
Ā 
Agile Methods and Data Warehousing (2016 update)
Agile Methods and Data Warehousing (2016 update)Agile Methods and Data Warehousing (2016 update)
Agile Methods and Data Warehousing (2016 update)Kent Graziano
Ā 
Data Warehousing 2016
Data Warehousing 2016Data Warehousing 2016
Data Warehousing 2016Kent Graziano
Ā 
Worst Practices in Data Warehouse Design
Worst Practices in Data Warehouse DesignWorst Practices in Data Warehouse Design
Worst Practices in Data Warehouse DesignKent Graziano
Ā 
Data Vault 2.0: Using MD5 Hashes for Change Data Capture
Data Vault 2.0: Using MD5 Hashes for Change Data CaptureData Vault 2.0: Using MD5 Hashes for Change Data Capture
Data Vault 2.0: Using MD5 Hashes for Change Data CaptureKent Graziano
Ā 
Agile Methods and Data Warehousing
Agile Methods and Data WarehousingAgile Methods and Data Warehousing
Agile Methods and Data WarehousingKent Graziano
Ā 
Agile Data Warehouse Modeling: Introduction to Data Vault Data Modeling
Agile Data Warehouse Modeling: Introduction to Data Vault Data ModelingAgile Data Warehouse Modeling: Introduction to Data Vault Data Modeling
Agile Data Warehouse Modeling: Introduction to Data Vault Data ModelingKent Graziano
Ā 
Top Five Cool Features in Oracle SQL Developer Data Modeler
Top Five Cool Features in Oracle SQL Developer Data ModelerTop Five Cool Features in Oracle SQL Developer Data Modeler
Top Five Cool Features in Oracle SQL Developer Data ModelerKent Graziano
Ā 
(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling
(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling
(OTW13) Agile Data Warehousing: Introduction to Data Vault ModelingKent Graziano
Ā 
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile ApproachUsing OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile ApproachKent Graziano
Ā 
Why Data Vault?
Why Data Vault? Why Data Vault?
Why Data Vault? Kent Graziano
Ā 

More from Kent Graziano (20)

Balance agility and governance with #TrueDataOps and The Data Cloud
Balance agility and governance with #TrueDataOps and The Data CloudBalance agility and governance with #TrueDataOps and The Data Cloud
Balance agility and governance with #TrueDataOps and The Data Cloud
Ā 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for Dinner
Ā 
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
Ā 
Intro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on SnowflakeIntro to Data Vault 2.0 on Snowflake
Intro to Data Vault 2.0 on Snowflake
Ā 
Rise of the Data Cloud
Rise of the Data CloudRise of the Data Cloud
Rise of the Data Cloud
Ā 
Making Sense of Schema on Read
Making Sense of Schema on ReadMaking Sense of Schema on Read
Making Sense of Schema on Read
Ā 
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)Agile Data Engineering: Introduction to Data Vault 2.0 (2018)
Agile Data Engineering: Introduction to Data Vault 2.0 (2018)
Ā 
Extreme BI: Creating Virtualized Hybrid Type 1+2 Dimensions
Extreme BI: Creating Virtualized Hybrid Type 1+2 DimensionsExtreme BI: Creating Virtualized Hybrid Type 1+2 Dimensions
Extreme BI: Creating Virtualized Hybrid Type 1+2 Dimensions
Ā 
Agile Data Warehousing: Using SDDM to Build a Virtualized ODS
Agile Data Warehousing: Using SDDM to Build a Virtualized ODSAgile Data Warehousing: Using SDDM to Build a Virtualized ODS
Agile Data Warehousing: Using SDDM to Build a Virtualized ODS
Ā 
Agile Data Engineering - Intro to Data Vault Modeling (2016)
Agile Data Engineering - Intro to Data Vault Modeling (2016)Agile Data Engineering - Intro to Data Vault Modeling (2016)
Agile Data Engineering - Intro to Data Vault Modeling (2016)
Ā 
Agile Methods and Data Warehousing (2016 update)
Agile Methods and Data Warehousing (2016 update)Agile Methods and Data Warehousing (2016 update)
Agile Methods and Data Warehousing (2016 update)
Ā 
Data Warehousing 2016
Data Warehousing 2016Data Warehousing 2016
Data Warehousing 2016
Ā 
Worst Practices in Data Warehouse Design
Worst Practices in Data Warehouse DesignWorst Practices in Data Warehouse Design
Worst Practices in Data Warehouse Design
Ā 
Data Vault 2.0: Using MD5 Hashes for Change Data Capture
Data Vault 2.0: Using MD5 Hashes for Change Data CaptureData Vault 2.0: Using MD5 Hashes for Change Data Capture
Data Vault 2.0: Using MD5 Hashes for Change Data Capture
Ā 
Agile Methods and Data Warehousing
Agile Methods and Data WarehousingAgile Methods and Data Warehousing
Agile Methods and Data Warehousing
Ā 
Agile Data Warehouse Modeling: Introduction to Data Vault Data Modeling
Agile Data Warehouse Modeling: Introduction to Data Vault Data ModelingAgile Data Warehouse Modeling: Introduction to Data Vault Data Modeling
Agile Data Warehouse Modeling: Introduction to Data Vault Data Modeling
Ā 
Top Five Cool Features in Oracle SQL Developer Data Modeler
Top Five Cool Features in Oracle SQL Developer Data ModelerTop Five Cool Features in Oracle SQL Developer Data Modeler
Top Five Cool Features in Oracle SQL Developer Data Modeler
Ā 
(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling
(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling
(OTW13) Agile Data Warehousing: Introduction to Data Vault Modeling
Ā 
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile ApproachUsing OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Using OBIEE and Data Vault to Virtualize Your BI Environment: An Agile Approach
Ā 
Why Data Vault?
Why Data Vault? Why Data Vault?
Why Data Vault?
Ā 

Recently uploaded

Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxEmmanuel Dauda
Ā 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfMarinCaroMartnezBerg
Ā 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxthyngster
Ā 
Ukraine War presentation: KNOW THE BASICS
Ukraine War presentation: KNOW THE BASICSUkraine War presentation: KNOW THE BASICS
Ukraine War presentation: KNOW THE BASICSAishani27
Ā 
Invezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz1
Ā 
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...Suhani Kapoor
Ā 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfLars Albertsson
Ā 
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...shivangimorya083
Ā 
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiLow Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiSuhani Kapoor
Ā 
Aminabad Call Girl Agent 9548273370 , Call Girls Service Lucknow
Aminabad Call Girl Agent 9548273370 , Call Girls Service LucknowAminabad Call Girl Agent 9548273370 , Call Girls Service Lucknow
Aminabad Call Girl Agent 9548273370 , Call Girls Service Lucknowmakika9823
Ā 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubaihf8803863
Ā 
Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxJohnnyPlasten
Ā 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...Suhani Kapoor
Ā 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998YohFuh
Ā 
Predicting Employee Churn: A Data-Driven Approach Project Presentation
Predicting Employee Churn: A Data-Driven Approach Project PresentationPredicting Employee Churn: A Data-Driven Approach Project Presentation
Predicting Employee Churn: A Data-Driven Approach Project PresentationBoston Institute of Analytics
Ā 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptSonatrach
Ā 
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Sapana Sha
Ā 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationshipsccctableauusergroup
Ā 

Recently uploaded (20)

Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptx
Ā 
FESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdfFESE Capital Markets Fact Sheet 2024 Q1.pdf
FESE Capital Markets Fact Sheet 2024 Q1.pdf
Ā 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM  TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
Ā 
Ukraine War presentation: KNOW THE BASICS
Ukraine War presentation: KNOW THE BASICSUkraine War presentation: KNOW THE BASICS
Ukraine War presentation: KNOW THE BASICS
Ā 
Invezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signalsInvezz.com - Grow your wealth with trading signals
Invezz.com - Grow your wealth with trading signals
Ā 
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
Ā 
Schema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdfSchema on read is obsolete. Welcome metaprogramming..pdf
Schema on read is obsolete. Welcome metaprogramming..pdf
Ā 
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...
Full night šŸ„µ Call Girls Delhi New Friends Colony {9711199171} Sanya Reddy āœŒļøo...
Ā 
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service BhilaiLow Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Low Rate Call Girls Bhilai Anika 8250192130 Independent Escort Service Bhilai
Ā 
Aminabad Call Girl Agent 9548273370 , Call Girls Service Lucknow
Aminabad Call Girl Agent 9548273370 , Call Girls Service LucknowAminabad Call Girl Agent 9548273370 , Call Girls Service Lucknow
Aminabad Call Girl Agent 9548273370 , Call Girls Service Lucknow
Ā 
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls DubaiDubai Call Girls Wifey O52&786472 Call Girls Dubai
Dubai Call Girls Wifey O52&786472 Call Girls Dubai
Ā 
Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptx
Ā 
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
VIP Call Girls Service Charbagh { Lucknow Call Girls Service 9548273370 } Boo...
Ā 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
Ā 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998
Ā 
Predicting Employee Churn: A Data-Driven Approach Project Presentation
Predicting Employee Churn: A Data-Driven Approach Project PresentationPredicting Employee Churn: A Data-Driven Approach Project Presentation
Predicting Employee Churn: A Data-Driven Approach Project Presentation
Ā 
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.pptdokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
dokumen.tips_chapter-4-transient-heat-conduction-mehmet-kanoglu.ppt
Ā 
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in  KishangarhDelhi 99530 vip 56974 Genuine Escort Service Call Girls in  Kishangarh
Delhi 99530 vip 56974 Genuine Escort Service Call Girls in Kishangarh
Ā 
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Saket, (-DELHI )+91-9654467111-(=)CHEAP Call Girls in Escorts Service Saket C...
Ā 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships
Ā 

Demystifying Data Warehousing as a Service - DFW

  • 1. 1Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Y O U R D A T A , N O L I M I T S Ā© 2017 Snowflake Computing Inc. All Rights Reserved. @KentGraziano KENT GRAZIANO Chief Technical Evangelist Snowflake Computing Demystifying Data Warehousing as a Service (DWaaS)
  • 2. 2Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Chief Technical Evangelist, Snowflake Computing ā€¢ Oracle ACE Director (DW/BI) ā€¢ OakTable Network ā€¢ Blogger ā€“ The Data Warrior ā€¢ Certified Data Vault Master and DV 2.0 Practitioner ā€¢ Former Member: Boulder BI Brain Trust (#BBBT) ā€¢ Member: DAMA Houston & DAMA International ā€¢ Data Architecture and Data Warehouse Specialist ā€¢ 30+ years in IT ā€¢ 25+ years of Oracle-related work ā€¢ 20+ years of data warehousing experience ā€¢ Author & Co-Author of a bunch of books (Amazon) ā€¢ Past-President of ODTUG and Rocky Mountain Oracle User Group My Bio
  • 3. 3Ā© 2017 Snowflake Computing Inc. All Rights Reserved. About Snowflake Experienced, accomplished leadership team 2012 Founded by industry veterans with over 120 database patents Vision: A world with no limits on data First data warehouse built for the cloud Over 1000 customers since GA
  • 4. 4Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Data Challenges ā€¢ What is a Data Warehouse as a Service? ā€¢ What can a Cloud DWaaS do for me? ā€¢ Features of a Cloud DWaaS ā€¢ Top 10 (or so) Cool Features of Snowflake ā€¢ Becoming more Agile with DWaaS ā€¢ Where Snowflake fits ā€¢ Spark Integration ā€¢ Continuous Loading ā€¢ Conclusion Agenda
  • 5. 5Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Data challenges today
  • 6. 6Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Scenarios Ā with Ā affinity Ā for Ā cloud Gartner 2016 Predictions: By 2018, six billion connected things will be requesting support. Connecting applications, devices, and ā€œthingsā€ Reaching employees, business partners, and consumers Anytime, anywhere mobility On demand, unlimited scale Understanding behavior; generating, retaining, and analyzing data
  • 7. 7Ā© 2017 Snowflake Computing Inc. All Rights Reserved. 40 Zettabytes by 2020 Web ERP3rd party apps Enterprise apps IoTMobile
  • 8. 8Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Itā€™s not the data itself itā€™s how you take full advantage of the insight it provides Web ERP3rd party apps Enterprise apps IoTMobile
  • 9. 9Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Most firms donā€™t consistently turn data into action Source: Forrester All Possible data All Possible Action of firms aspire to be data-driven. 73% of firms are good at turning data into action. 29%
  • 10. 10Ā© 2017 Snowflake Computing Inc. All Rights Reserved. The data struggle Data is stale Hard to incorporate new data sources Too much time waiting for data Too much time spent on manual administration Reports are too slow whenever the system is busy Hard to experiment with data
  • 11. 11Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Symptoms of fundamental challenges Data silos Data locked into separate databases, big data systems, and applications Inflexibility Slow, cumbersome scaling and limited support for diverse data Complexity Multiple systems to integrate and manage requiring specialized skills and tools Performance Contention for limited resources resulting in latency and delays Cost Painful upfront costs and overprovisioned capacity
  • 12. 12Ā© 2017 Snowflake Computing Inc. All Rights Reserved. The evolution of data platforms Data warehouse & platform software Vertica, Greenplum, Paraccel, Hadoop Data warehouse appliance Teradata 1990s 2000s 2010s Cloud DWaaS Snowflake 1980s Relational database Oracle, DB2, SQL Server
  • 13. 13Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What is a Cloud DWaaS? DW- Data Warehouse ā€¢ Relational database ā€¢ Uses standard SQL ā€¢ Optimized for fast loads and analytic queries aaS ā€“ As a Service ā€¢ Like SaaS (e.g. SalesForce.com) ā€¢ No infrastructure set up ā€¢ Minimal to no administration ā€¢ Managed for you by the vendor ā€¢ Pay as you go, for what you use
  • 14. 14Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Goals of a Cloud DWaaS Make your life easier ā€¢ So you can load and use your data faster Support business ā€¢ Make data accessible to more people ā€¢ Reduce time to insights Handle big data too! ā€¢ Schema-less ingestion
  • 15. 15Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What to Expect from a Cloud DWaaS It should support standard SQL (natively) ā€¢ It should support standard ETL, BI & data science tools ā€¢ ODBC or JDBC connectivity It should be infinitly scalable (cloud) ā€¢ Handle huge amounts of data ā€¢ Handle large number of concurrent queries without performance degradation It should handle flexible schema data types ā€¢ No sharding or ETL required
  • 16. 16Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What to Expect from a Cloud DWaaS It should be secure ā€¢ Built in encryption? It shoud be stable ā€¢ Resiliancy and availability should be easy to configure and manage It should be easy to configure and manage It should provide a lower TCO ā€¢ Cloud scale pricing
  • 17. 17Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Common customer scenarios Data Ā warehouse Ā  for Ā SaaS Ā offerings Use Ā Cloud Ā DW as Ā back-Ā­ end Ā data Ā warehouse Ā  supporting Ā data-Ā­driven Ā  SaaS Ā products noSQL Ā replacement Replace Ā use Ā of Ā noSQL Ā  system Ā (e.g. Ā Hadoop) Ā for Ā  transformation Ā and Ā SQL Ā  analytics Ā of Ā multi-Ā­ structured Ā data Ā  Data Ā warehouse Ā  modernization Consolidate Ā legacy Ā  datamarts Ā and Ā support Ā  new Ā projects
  • 18. 18Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Enabling key use cases Datamart & data silo consolidation Consolidate legacy datamarts to eliminate silos and support new projects Integrated data analytics Directly load structured + semi-structured data into Snowflake for reporting & analytics Data Science, Exploratory & ad hoc analytics Direct access to data for SQL analysts and data scientists to explore data, identify correlations, build & test models
  • 19. 19Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Introducing Snowflake
  • 20. 20Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake: 1st Data Warehouse Built for the Cloud SQL relational database Optimized storage & processing Standard connectivity ā€“ BI, ETL, ā€¦ Data Warehousingā€¦ Existing SQL skills and tools ā€œLoad and goā€ ease of use Cloud-based elasticity to fit any scale Data scientists SQL users & tools ā€¦for Everyone
  • 21. 21Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What is Snowflake? Built for the cloud SQL Data Warehouse Our Vision Provide all users anytime, anywhere insights so they can make actionable decisions based on data Our Solution Next-generation data warehouse built from the ground up for the cloud and for todayā€™s data and analytics Delivered as a service
  • 22. 22Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflakeā€™s differentiating technology Unique architecture: Multi-cluster, shared data Single place for data z Instant, unlimited scalability Zero management Instant, live data sharing
  • 23. 23Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved. The Data Warriorā€™s Top 10+ Cool Things About Snowflake (A Data Geeks Guide to Cloud-native DW)
  • 24. 24Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #10 ā€“ Persistent Result Sets ā€¢ No setup ā€¢ In Query History ā€¢ By Query ID ā€¢ 24 Hours ā€¢ No re-execution ā€¢ No Cost for Compute
  • 25. 25Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #9 -Connect w/JDBC & ODBC to the cloud Data Sources Custom & Packaged Applications & Data Science ODBC WEB UIJDBC Interfaces Java >_ Scripting Reporting & Analytics Data Modeling, Management & Transformation SDDM
  • 26. 26Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #8 - UNDROP UNDROP TABLE <table name> UNDROP SCHEMA <schema name> UNDROP DATABASE <db name> Part of Time Travel feature: AWESOME!
  • 27. 27Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #7 Fast Clone (Zero-Copy) Instant copy of table, schema, or database: CREATE OR REPLACE TABLE MyTable_V2 CLONE MyTable; With Time Travel: CREATE SCHEMA mytestschema_clone_restore CLONE testschema BEFORE (TIMESTAMP => TO_TIMESTAMP(40*365*86400)); PROD PUBLIC Table A Table B Table C DEV PUBLIC Table A Table B Table C PUBLIC Table A Table B Table C INT
  • 28. 28Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #6 ā€“ JSON Support with SQL Apple 101.12 250 FIH-2316 Pear 56.22 202 IHO-6912 Orange 98.21 600 WHQ-6090 Structured data (e.g. CSV) Semi-structured data (e.g. JSON, Avro, XML) { "firstName": "John", "lastName": "Smith", "height_cm": 167.64, "address": { "streetAddress": "212nd Street", "city": "New York", "state": "NY", "postalCode": "10021-3100" }, "phoneNumbers": [ { "type": "home", "number":"212555-1234"}, { "type": "office", "number": "646 555-4567"} ] } Optimized storage Flexible schema - Native Relational processing select v:lastName::string as last_name from json_demo; All Your Data!
  • 29. 29Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #5 ā€“ Standard SQL w/Analytic Functions select Nation, Customer, Total from (select n.n_name Nation, c.c_name Customer, sum(o.o_totalprice) Total, rank() over (partition by n.n_name order by sum(o.o_totalprice) desc) customer_rank from orders o, customer c, nation n where o.o_custkey = c.c_custkey and c.c_nationkey = n.n_nationkey group by 1, 2) where customer_rank <= 3 order by 1, customer_rank SQL Complete SQL database ā€¢ Data definition language (DDLs) ā€¢ Query (SELECT) ā€¢ Updates, inserts and deletes (DML) ā€¢ Role based security ā€¢ Multi-statement transactions New partner post: https://sonra.io/2018/02/04/create-custom-aggregate-udaf-window-functions-snowflake/
  • 30. 30Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #4 ā€“ Separation of Storage & Compute Snowflakeā€™s multi-cluster, shared data architecture Centralized storage Instant, automatic scalability & elasticity Service Compute Storage
  • 31. 31Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #3 ā€“ Support Multiple Workloads Accelerate the data pipeline Run loading & analytics at any time, concurrently, to get data to users faster Scale compute to support any workload Scale processing horsepower up and down on-the-fly, with zero downtime or disruption Scale concurrency without performance impact Multi-cluster ā€œvirtual warehouseā€ architecture scales concurrent users & workloads without contention Deliver faster analytics at any scale Loading Marketing Finance
  • 32. 32Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Elastic scaling for storage Low-cost cloud storage, fully replicated and resilient ā€¢ Elastic scaling for compute Virtual warehouses scale up & down on the fly to support workload needs ā€¢ Elastic scaling for concurrency Automatically scale concurrency using multi- cluster virtual warehouses Instant, unlimited scalability
  • 33. 33Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #2 Ā ā€“ Secure Ā by Ā Design Ā with Ā Automatic Ā Encryption Ā of Ā Data! Embedded multi-factor authentication Federated authentication available Certified against enterprise- class requirements HIPPA Certified! PCI Certified! All data encrypted, always, end-to-end Encryption keys managed automatically NEW: Tri-secret security Role-based access control model Granular privileges on all objects & actions Authentication Access control Data encryption External validation
  • 34. 34Ā© 2017 Snowflake Computing Inc. All Rights Reserved. #1 - Automatic Query Optimization Zero Management Fully managed with no knobs or tuning required No indexes, distribution keys, partitioning, vacuuming,ā€¦ Zero infrastructure costs Zero admin costs
  • 35. 35Ā© 2017 Snowflake Computing Inc. All Rights Reserved. New #1 Fav ā€“ Data Sharing (The Data ā€œSharehouseā€) Data Consumers Data Providers No data movement Share with unlimited number of consumers Live access Data consumers immediately see all updates Ready to use Consumers can immediately start querying
  • 36. 36Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Data Sharing - Configuring access create share s1; --empty share grant usage on database sales to share s1; -- add database grant usage on schema sales.east to share s1; -- add schema grant usage on view sales.east.accts to share s1; -- add view alter share s1 add accounts=a1, a2, a3; -- add accounts s1 Provider Database Data Share Object Schema List View List Account List a3 a1 a2 Data provider Data Consumers create database sales from share p1.s1;
  • 37. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Enabling Agile DW in the Cloud Examples
  • 38. 38Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢Agile Warehouse Scaling ā€¢ Separation of Workloads ā€¢ Virtual WH Scaling Techniques ā€¢Agile Data Lifecycle ā€¢ Cloning Enabling the Agile Data Warehouse
  • 39. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Agile Ā Warehouse Ā Scaling Separation Ā of Ā Workloads
  • 40. 40Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ DWaaS - Snowflake solution ā€¢ Assign each workload its own Virtual WH ā€¢ At a minimum, two WH: ETL and Business ā€¢ ETL can run continuously if it makes sense for the business ā€¢ ETL / Business workload contention is eliminated ā€¢ Further subdivide business workloads into own clusters as needed ā€¢ Eliminate contention between Sales, Marketing, Finance, Data Science workloads ā€¢ International groups can operate clusters on their own local time schedules ā€¢ Easy to add new workloads on demand! Separation of Workloads Virtual Warehouse Databases Virtual Warehouse ETL & Data Loading Business Workloads Finance Virtual Warehouse Test/Dev Virtual Warehouse S Marketing Virtual Warehouse Sales Virtual Warehouse S Research
  • 41. 41Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Increase cluster size ā€“ on the fly! ā€¢ More data being analyzed ā€¢ More complex queries ā€¢ Get some concurrency boost ā€¢ No need to start, stop, or reboot! ā€¢ Automatic scale out! ā€¢ Multi-cluster warehouse concurrency ā€¢ Workload queries have usual weight ā€¢ But more of them, e.g. 20 dashboard users rather than the usual 5 ā€¢ Automatically scales back after the load drops Other Agile Warehouse Scaling Techniques
  • 42. 42Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Anticipated surges ā€¢ Explicitly increase WH nodes (T-shirt size) when expecting more data ā€¢ Explicitly increase MCWH minimum clusters when expecting more queries ā€¢ Can do both at once with ALTER WAREHOUSE ā€¢ Use cron or other scheduling/orchestration tool ā€¢ Unanticipated surges ā€¢ Rely on MCWH maximum clusters for some extra headroom ā€¢ Maximize Business Agility! ā€¢ Responsiveness for users ā€¢ Throughput and value extracted from variable compute power ā€¢ Minimize ā€¢ Cost and administrative overhead Agile Warehouse Scaling ā€“ Best Practices
  • 43. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Agile Ā Data Ā Lifecycle
  • 44. 44Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Separation of Workloads ā€¢ Individual virtual warehouse for each dev/test/prod functional area ā€¢ CLONE for dev/test or Data Science Sandbox ā€“ on demand! ā€¢ Full logical copy of the data, but uses no extra storage ā€¢ Test/dev operations against clone have no effect on original data ā€¢ Security ā€¢ RBAC limits dev/test access to clone and not production data ā€¢ Secure Views permit role- or user-based obfuscation / masking / projection ā€¢ Business Impact ā€“ better quality code ā€¢ Dev and test teams are working on data at scale, see true app performance ā€¢ Full range of values means fewer surprises when app encounters live data Agile Data Lifecycle
  • 45. 45Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Create development (DEV) and integration (INT) databases from production (PROD) Scenario 1 PROD PUBLIC Table A Table B INT PUBLIC Table A Table B DEV PUBLIC Table A Table B CLONE CLONE CREATE OR REPLACE DATABASE DEV CLONE PROD CREATE OR REPLACE DATABASE INT CLONE PROD
  • 46. 46Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Create two new tables, C and D, in the development (DEV) database Scenario 2: new development PROD PUBLIC Table A Table B INT PUBLIC Table A Table B DEV PUBLIC Table A Table B Table C Table D
  • 47. 47Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Mini-release: promote table C for integration testing Scenario 2: new development PROD PUBLIC Table A Table B INT PUBLIC Table A Table B DEV PUBLIC Table A Table B Table C Table D Table C CREATE TABLE LIKE
  • 48. 48Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Deploy to production: promote table C to PROD database Scenario 2: new development PROD PUBLIC Table A Table B INT PUBLIC Table A Table B DEV PUBLIC Table A Table B Table C Table D Table C CREATE TABLE LIKE Table C
  • 49. 49Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Refresh dev: get latest PROD data into DEV and INT Scenario 2: new development PROD PUBLIC Table A Table B INT PUBLIC Table A Table B DEV PUBLIC Table A Table B Table C Table D Table C Table C CLONE DEV PUBLIC Table A Table B Table C Table D DEV2 CLONE CREATE OR REPLACE TABLE Dev2.TableD CLONE Dev.TableD CREATE OR REPLACE DATABASE DEV2 CLONE PROD
  • 50. 50Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ CLONE for Data Scientists ā€¢ Quick and Safe sandbox for discovery and testing ā€¢ Experiment and explore, and write back! ā€¢ Test hypothisis and keep iterating ā€“ drop and reclone as needed ā€¢ No impact to production data ā€¢ Combine with own virtual warehouse for complete isolation ā€¢ Business Impact ā€“ better data science ā€¢ More fine-grained data over longer time intervals ā€¢ Deeper insights, better forecasting, more monetizable results ā€¢ CLONE for Compliance ā€¢ Monthly, quarterly, annual clones ā€“ financial reporting, auditing requirements ā€¢ Business Impact ā€“ simpler compliance ā€¢ Your "backups" are live and immediately available Agile Data Lifecycle
  • 51. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake & Spark
  • 52. 52Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Where Snowflake fits with complementary solutions Vast majority, >> 80% or more, is semi-/structured >> 20% or less is unstructured Data Sources for Analytics (Audio, video, text, etc.) (Tables, CSV, JSON, Avro, Parquet, XML, flat files, etc.) Complementary Special data processing (Spark ML, Hadoop lakes, NoSQL front-ends, In-memory DBs, non-relational exploring, et al.) Built for the cloud, relational data warehouse/DB & analytics delivered as a service 52 Use Cases (Tables, metadata) ā€¢ Live analytics ā€¢ Operational DW ā€¢ Real-time dashboards ā€¢ Continuous data loading ā€¢ Change data capture ā€¢ Traditional DW/BI ā€¢ Consolidation of data silos ā€¢ Relational data exploration ā€¢ Interactive queries ā€¢ Data science/statistics (Python, R) ā€¢ And more ā€¢ Spark machine learning ā€¢ Unstructureddata mining ā€¢ Streaming analytics ā€¢ Non-relational data exploration ā€¢ Non-relational data science ā€¢ And more
  • 53. 53Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Modern Ā Data Ā Architectures Data Production Data Lake Data Consumption Web Servers Data Bases App Metrics Snowflake Business Intelligence Data Science: Exploration Data Science: Machine Learning Model Training Data Augmentation & Loading Data Augmentation & Loading Model-driven Apps Online Recommendations & Behavioral Targeting on Web Pages Offline Churn Prediction Other Machine Learning Apps
  • 54. 54Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Modern Ā Data Ā Architectures: Ā  Data Ā Loading Ā & Ā Augmentation Web Servers App Metrics Data Bases Data ProductionCustomers & End Users External Data Augmentation Spark Cluster Kafka SnowSQL COPY Kinesis Transformations and Augmentations in e.g. Python, Scala, SparkSQL, Spark Streaming etc. Snowflake Warehouse Snowflake Spark Connector Full/Pruned Data Lake with Augmentation Snowflake Spark Connector Source/Raw Data Augmented Data Source/Raw Data Augmented Data Transformations and Augmentations using Snowflake SQL Augmentation Augmentation CDC - FiveTran
  • 55. 55Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Modern Ā Data Ā Architectures: Ā Data Ā Science Jupyter Notebook using e.g. ML Data Exploration Model Design Spark app using Spark ML Model (Re-)Training Operationalized Model (Code + Params) External Data Augmentation Spark Cluster Snowflake Warehouse Snowflake Spark Connector Data Lake with Augmentation Snowflake Spark Connector Source/Raw Data Augmented Data Source/Raw Data Augmented Data Augmentation Augmentation Data Exploration using Snowflake SQL
  • 56. 56Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake WarehouseSpark Cluster (e.g. in AWS EMR) AWS S3 3 2 Data Flow: Data Frames Data Flow: Snowflake Tables Master Node Slave Nodes Query (SQL) Cloud Services Virtual Warehouse 1 Architecture: Ā Data Ā Interchange Ā (Read)
  • 57. 57Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake WarehouseSpark Cluster (e.g. in AWS EMR) AWS S3 1 3 Data Flow: Data Frames Data Flow: Snowflake Tables Master Node Slave Nodes ControlFlow (SQL) Cloud Services Virtual Warehouse 2 Architecture: Ā Data Ā Interchange Ā (Write)
  • 58. 58Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Spark Cluster Snowflake Virtual Warehouse Super-Charge Spark Processing with Snowflake ā€¢ Spark optimizer extension automatically identifies Spark operations with corresponding Snowflake implementations ā€¢ Spark connector pushes these operations into Snowflake SQL ā€¢ Pushed operations include: project, filter, join, aggregation, limit ā€¢ Optimized Snowflake SQL applied to all Spark data frames backed by Snowflake tables ā€¢ Transparent to Spark language choice: No need to rewrite Spark (Python, Scala, Java, ā€¦) or SparkSQL code df1 = spark.loadā€¦(ā€˜T1ā€™) df2 = spark.loadā€¦(ā€˜T2ā€™) df1 .join(df2, ā€˜idā€™) .groupBy(ā€˜regionā€™) .count() .collect() SELECT region, count(*) FROM T1 JOIN T2 ON T1.id = T2.id GROUPBY region Snowflake SQL AWS S3 Results
  • 59. 59Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Reduced complexity, better manageability ā€¢ Same data store for curated and raw data ā€¢ Same data store for data science and BI data sets ā€¢ Fully transactional data store even for raw data when needed ā€¢ Performance benefits ā€¢ Leveraging structure of the data even in semi-structured data formats such as JSON as compared to flat file storage ā€¢ Metadata for pruning based on queries ā€¢ Easy to integrate with Spark as compute and application platform ā€¢ Overall compute capacity and cost savings due to performance benefits with Snowflake ā€¢ Blog posts: ā€¢ https://www.snowflake.net/snowflake-and-spark-part-1-why-spark/ ā€¢ https://www.snowflake.net/snowflake-spark-part-2-pushing-query-processing/ Snowflake + Spark Benefits
  • 60. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. NEW Continous Data Loading with Snowpipe
  • 61. 61Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Data is created constantly.Data accumulated over time. Data is Being Generated Faster than Ever Before Infrequent Now Constant Web data Corp data App data Thereā€™s an enormous opportunity to use continuously generated data in analysis.
  • 62. 62Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowpipe is an automated service that asynchronously listens for new data as it arrives in Amazon Web Servicesā€™ S3 cloud storage service and continuously loads that data into Snowflake. Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake Confidential ā€“ under NDA only.
  • 63. 63Ā© 2017 Snowflake Computing Inc. All Rights Reserved. 6 Data Ingress Approaches & Snowpipe Support Approach Approach Definition Snowpipe Batch Data accumulates over time (hours, days) and is then loaded periodically Option 1: Point at an S3 bucket and a destination table in your warehouse at which point new data is automatically uploaded. Option 2: Technical resources can interface directly with the programmatic REST API along with Java and Python SDKs to enable highly customized loading use cases Micro-batch Data accumulates over small time windows (minutes) and is then loaded Continuous Every data item is loaded individually as it arrives
  • 64. 64Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowpipe Scenario: Automatic Loading from S3 Snowflake Database External S3 SnowPipe Service Server-less Loader S3 notification File data
  • 65. 65Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowpipe ā€“ Current Preview Snowflake Database REST Call {file names} S3 Application SnowPipe Service REST Endpoint Server-less Loader
  • 66. 66Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowpipe Auto-Ingest: Fully Automatic Loading from S3 Snowflake Database External S3 SnowPipe Service Server-less Loader S3 notification File data Use simple Snowflake DDL statements to configure bucket for automatic load with Snowpipe See the video here: https://www.youtube.com/embed/rwZWmcy0aBU?autoplay=1&rel=0 Blog posts: https://www.snowflake.net/your-first-steps-with-snowpipe/ https://www.snowflake.net/snowpipe-serverless-loading-for-streaming-data-2/
  • 67. 67Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Key Snowpipe Benefits You only pay for the compute time you use to load data Avoid repeated manual COPY commands Continuously generated data is available for analysis in seconds 0 management. No indexing, tuning, partitioning or vacuuming on load Full support for semi-structured data on load Server-less: No servers to manage or concurrency to worry about.
  • 68. 68Ā© 2017 Snowflake Computing Inc. All Rights Reserved. ā€¢ Server-less billing model: utilization- based billing ā€¢ No warehouse to manage for load ā€¢ Per core, per second granularity ā€¢ Charged as Snowflake credits ā€¢ New line item on Snowflake bill ā€¢ Components of the Snowpipe charges: ā€¢ Loading work ā€“ idle times are not charged ā€¢ Time spent in metadata management for file loading 6 Pricing
  • 69. 69Ā© 2017 Snowflake Computing Inc. All Rights Reserved.Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Snowflake in Action Today
  • 70. 70Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What customers are doing with Snowflake DATA Ā MARTS Ā  & Ā EXTRACTS Market research company consolidated data marts to reduce costs and data silos Gaming company replaced Hadoop + SQL database with Snowflake STAGING DATA Ā LAKE DATA Ā  WAREHOUSE Consumer retailer modernizing DW by replacing legacy appliance with Snowflake Mobile analytics company shares live data with clients REPORTING, Ā ANALYTICS Ā  & Ā APPLICATIONS DATA Ā SOURCES
  • 71. 71Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Delivering compelling results Simpler data pipeline Replace noSQL database with Snowflake for storing & transforming JSON event data Faster analytics Replace on-premises data warehouse with Snowflake for analytics workload Significantly lower cost Improved performance while adding new workloads--at a fraction of the cost noSQL data base: 8 hours to prepare data Snowflake: 1.5 minutes Data warehouse appliance: 20+ hours Snowflake: 45 minutes Data warehouse appliance: $5M + to expand Snowflake: added 2 new workloads for $50K
  • 72. 72Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Over 1000 customers demonstrating whatā€™s possible Up to 200x faster reports that enable analysts to make decisions in minutes rather than days Ability to load and update data in near real time by replacing legacy data warehouse + Hadoop cluster New applications that provide secure access to analytics to 11,000+ pharmacies
  • 73. 73Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Chosen by leading enterprises Built advertising analytics platform on Snowflake Moving enterprise reporting to Snowflake Deploying BI solution stack on Snowflake Built new embedded analytics application on Snowflake Moved audience reporting infrastructure to Snowflake Replaced Hadoop + legacy data warehouse with Snowflake
  • 74. 74Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Ranked #1 Cloud Data Warehouse! ā€œSnowflake Hits All the Marksā€ ā€“ Gigaom 4,85 4,50 4,45 3,75 3,75 3,35 3,20 3,15 2,60 Cloud Analytics Database Distruption Vectors AWS Redshift Oracle Database Exdata Cloud Service SAP HANA Cloud Platform Azure Data Warehouse Vertica DashDB (IBM) Teradata Google Big Query Snowflake RobustnessofSQL15% Built-inOptimization15% On-the-flyElasticity25% DynamicEnvironment Adaption20% SeparationofCompute fromstorage15% SupportforDiversedata 10% Score ā€œYou can tell the data warehouse pedigree from the developmentā€¦ With superior performance and the most hands-off model of ownership, Snowflake is the epitome of data warehouse as a service. The model, cost, features and scalability have already caused some to postpone Hadoop adoption.ā€ William McKnight Gigaom Disruption Vectors Gigaom Analyst Report: Sector Roadmap: Cloud Analytic Databases 2017 Read the full report on snowflake.net
  • 75. 75Ā© 2017 Snowflake Computing Inc. All Rights Reserved. What does a Cloud-native DWaaS Provide? Cost effective storage and analysis of GBs, TBs, or even PBā€™s Lightning fast query performance Continuous data loading without impacting query performance Unlimited user concurrency Full SQL relational support of both structured and semi-structured data Support for the tools and languages you already useODBC WEB UIJDBCJava >_ Scripting
  • 76. 76Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Big Data does not have to equal Big Effort Web ERP3rd party apps Enterprise apps IoTMobile
  • 77. 77Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Discover the performance, concurrency, and simplicity of Snowflake As easy as 1-2-3! 01 Visit Snowflake.net 02 Click ā€œTry for Freeā€ 03 Sign up & register Snowflake is the only data warehouse built for the cloud. You can automatically scale compute up, out, or downā€”independent of storage. Plus, you have the power of a complete SQL database, with zero management, that can grow with you to support all of your data and all of your users. With Snowflake On Demandā„¢, pay only for what you use. Sign up and receive $400 worth of free usage for 30 days!
  • 78. Kent Graziano Snowflake Computing Kent.graziano@snowflake.net On Twitter @KentGraziano More info at http://snowflake.net Visit my blog at http://kentgraziano.com Contact Information
  • 79. 79Ā© 2017 Snowflake Computing Inc. All Rights Reserved. YOUR DATA, NO LIMITS Ā© 2017 Snowflake Computing Inc. All Rights Reserved. Thank You!