SlideShare a Scribd company logo
1 of 27
Copyright © 2019 Impetus Technologies, Inc.
You are prohibited from making a copy or modification of, or from redistributing, rebroadcasting, or
re-encoding of this content without the prior written consent of Impetus.
This presentation may include images from other products and services. These images are used for
illustrative purposes only. Unless explicitly stated there is no implied endorsement or sponsorship of
these products by Impetus. All copyrights and trademarks are property of their respective owners.
Automate and Optimize
Data Warehouse Migration
to Snowflake
IMP
Our Mission
Enabling a unified, clear, and present view of your business
Contents
Cloud migration strategy
How Impetus can help
Automated workload transformation to Snowflake
Demo
Q&A
Poll
Speakers
Barry Tuthill
VP Field Operations
Impetus Technologies
Sanjay Sharma
Principal Architect
Impetus Technologies
Cloud Migration Strategy
Key Considerations
Business
• BAU
• Existing investments
• Risk management
• Cost and scale
• People, processes, etc.
• Interoperability
Key Considerations
Technical
• Architecture (Tech stack)
• Performance
• Migration blueprint
• Automation
• Parallel-run period
Top 5 Concerns
Existing investments
Migrate as is vs. total re-engineering
Extent of automation
Risk and uncertainty
Performance and cost
Typical Migration Checklist
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
How Impetus Can Help
End-to-end Automated Migration
Assessment and recommendations for migration
Optimized Schema (DDL) transformation
Data migration – one time and incremental pipelines setup
Workload logic (DML) conversion to SnowSQL
Stored Procedure conversion to Snow JavaScript procedures, Spark or Python wrappers
Scheduling/ orchestration logic conversion to Shell or Python or Cloud native wrappers
Agile DevOps, Continuous integration and delivery process setup
Validation and certification
Automating the Complete Migration Lifecycle
1
2
3
4
Assess
Transform
Validate
Execute
Assessment
Automated profiling
Identification of workloads and dependencies
Creation of optimized schema
Grouping of workloads
Transformation
90% automated code conversion to SnowSQL
Automated data migration to an optimized schema
Automated handling of data types, views, UDFs and stored procedures
Creation of patterns for the target platform
Auto-generation of patterns for newer migrations
Query-editing for optimized fixes and performance tuning
Validation and Execution
Auto-generation of validation/ reconciliation scripts
Execution through cloud ready orchestration/ execution engine
Integrated DevOps and Agile processes setup
Accelerated decommissioning methodology to retire legacy footprint
Multi cloud ready with security and governance
Automated Workload Transformation to Snowflake
ENTERPRISE
DATA WAREHOUSE
VALIDATE
Query Logs
Script Files
Data
Query
BIG DATA
WAREHOUSE
ASSESS EXECUTETRANSFORM
Data - S3/Spark, Snowpipe, ETL
SQL,
Stored
Procedures,
Scheduling/
Workflow
LOGIC TRANSFORMATION
SnowSQL
Snow
JavaScript
procedures/
Spark/ Python /
Wrappers
Ingestion,
Scheduler/
workflow setup
Demo
Business Benefits
Accelerate time-to-value
Meet performance SLAs
Decrease risk and uncertainty
Automate end-to-end migration
Save up to 80% in cost, time, and effort
Maximize ROI
Poll Results and Feedback
Q&A
Thank You
Visit www.impetus.com or get in touch with us at inquiry@impetus.com

More Related Content

What's hot

Snowflake: The most cost-effective agile and scalable data warehouse ever!
Snowflake: The most cost-effective agile and scalable data warehouse ever!Snowflake: The most cost-effective agile and scalable data warehouse ever!
Snowflake: The most cost-effective agile and scalable data warehouse ever!Visual_BI
 
Considerations for Data Access in the Lakehouse
Considerations for Data Access in the LakehouseConsiderations for Data Access in the Lakehouse
Considerations for Data Access in the LakehouseDatabricks
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lakeJames Serra
 
Snowflake essentials
Snowflake essentialsSnowflake essentials
Snowflake essentialsqureshihamid
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Amazon Web Services
 
How to migrate workloads to the google cloud platform
How to migrate workloads to the google cloud platformHow to migrate workloads to the google cloud platform
How to migrate workloads to the google cloud platformactualtechmedia
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta LakeDatabricks
 
Oracle IT Financial Management
Oracle IT Financial ManagementOracle IT Financial Management
Oracle IT Financial ManagementJoseph Alaimo Jr
 
Snowflake Architecture.pptx
Snowflake Architecture.pptxSnowflake Architecture.pptx
Snowflake Architecture.pptxchennakesava44
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleAdam Doyle
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Cathrine Wilhelmsen
 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for DinnerKent Graziano
 
Free Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseFree Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseDatabricks
 
Migration to Databricks - On-prem HDFS.pptx
Migration to Databricks - On-prem HDFS.pptxMigration to Databricks - On-prem HDFS.pptx
Migration to Databricks - On-prem HDFS.pptxKshitija(KJ) Gupte
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data MeshLibbySchulze
 
Data Platform Architecture Principles and Evaluation Criteria
Data Platform Architecture Principles and Evaluation CriteriaData Platform Architecture Principles and Evaluation Criteria
Data Platform Architecture Principles and Evaluation CriteriaScyllaDB
 

What's hot (20)

Snowflake: The most cost-effective agile and scalable data warehouse ever!
Snowflake: The most cost-effective agile and scalable data warehouse ever!Snowflake: The most cost-effective agile and scalable data warehouse ever!
Snowflake: The most cost-effective agile and scalable data warehouse ever!
 
Considerations for Data Access in the Lakehouse
Considerations for Data Access in the LakehouseConsiderations for Data Access in the Lakehouse
Considerations for Data Access in the Lakehouse
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
Snowflake Architecture
Snowflake ArchitectureSnowflake Architecture
Snowflake Architecture
 
Snowflake essentials
Snowflake essentialsSnowflake essentials
Snowflake essentials
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
Snowflake: Your Data. No Limits (Session sponsored by Snowflake) - AWS Summit...
 
How to migrate workloads to the google cloud platform
How to migrate workloads to the google cloud platformHow to migrate workloads to the google cloud platform
How to migrate workloads to the google cloud platform
 
Intro to Delta Lake
Intro to Delta LakeIntro to Delta Lake
Intro to Delta Lake
 
Oracle IT Financial Management
Oracle IT Financial ManagementOracle IT Financial Management
Oracle IT Financial Management
 
Snowflake Architecture.pptx
Snowflake Architecture.pptxSnowflake Architecture.pptx
Snowflake Architecture.pptx
 
Snowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at ScaleSnowflake Data Science and AI/ML at Scale
Snowflake Data Science and AI/ML at Scale
 
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse A...
 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for Dinner
 
Free Training: How to Build a Lakehouse
Free Training: How to Build a LakehouseFree Training: How to Build a Lakehouse
Free Training: How to Build a Lakehouse
 
Migration to Databricks - On-prem HDFS.pptx
Migration to Databricks - On-prem HDFS.pptxMigration to Databricks - On-prem HDFS.pptx
Migration to Databricks - On-prem HDFS.pptx
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
 
Data Platform Architecture Principles and Evaluation Criteria
Data Platform Architecture Principles and Evaluation CriteriaData Platform Architecture Principles and Evaluation Criteria
Data Platform Architecture Principles and Evaluation Criteria
 
Introduction to Data Engineering
Introduction to Data EngineeringIntroduction to Data Engineering
Introduction to Data Engineering
 

Similar to Automate and Optimize Data Warehouse Migration to Snowflake

The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...Impetus Technologies
 
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarKeys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarImpetus Technologies
 
AWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAmazon Web Services
 
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationLeverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationCapgemini
 
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...Amazon Web Services
 
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Amazon Web Services
 
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarAutomated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarImpetus Technologies
 
tero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppttero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.pptTero Peltola
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationAmazon Web Services
 
Deep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudDeep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudNGA Human Resources
 
Creating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationCreating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationTom Laszewski
 
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Amazon Web Services
 
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014Amazon Web Services
 
Digital transformation slideshare
Digital transformation   slideshareDigital transformation   slideshare
Digital transformation slideshareShivamPatsariya1
 
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesSuccessful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesCprime
 
Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Oliver Wells
 
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceAWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceTom Laszewski
 
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionAccenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionTom Laszewski
 

Similar to Automate and Optimize Data Warehouse Migration to Snowflake (20)

The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
 
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarKeys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
 
AWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud Migrations
 
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationLeverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
 
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
 
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
 
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarAutomated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
 
tero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppttero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppt
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio Migration
 
Deep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudDeep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR Cloud
 
Creating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationCreating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organization
 
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
 
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
 
original.pptx
original.pptxoriginal.pptx
original.pptx
 
Digital transformation slideshare
Digital transformation   slideshareDigital transformation   slideshare
Digital transformation slideshare
 
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesSuccessful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
 
Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS)
 
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceAWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
 
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionAccenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
 
Migration Planning
Migration PlanningMigration Planning
Migration Planning
 

More from Impetus Technologies

Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Impetus Technologies
 
Building a mature foundation for life in the cloud
Building a mature foundation for life in the cloudBuilding a mature foundation for life in the cloud
Building a mature foundation for life in the cloudImpetus Technologies
 
Best practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarBest practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarImpetus Technologies
 
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarInstantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarImpetus Technologies
 
Solving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarSolving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarImpetus Technologies
 
Anomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleAnomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleImpetus Technologies
 
Keys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataKeys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataImpetus Technologies
 
Build Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesBuild Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesImpetus Technologies
 
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Impetus Technologies
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Impetus Technologies
 
Streaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkStreaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkImpetus Technologies
 
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationImpetus Technologies
 
The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...Impetus Technologies
 
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Impetus Technologies
 
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxAnomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxImpetus Technologies
 

More from Impetus Technologies (16)

Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...
 
Building a mature foundation for life in the cloud
Building a mature foundation for life in the cloudBuilding a mature foundation for life in the cloud
Building a mature foundation for life in the cloud
 
Best practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarBest practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus Webinar
 
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarInstantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
 
Solving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarSolving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinar
 
Anomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleAnomaly detection with machine learning at scale
Anomaly detection with machine learning at scale
 
Keys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataKeys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of Data
 
Build Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesBuild Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in Minutes
 
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
 
Streaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkStreaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache Spark
 
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
 
The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...
 
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
 
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxAnomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
 
Importance of Big Data Analytics
Importance of Big Data AnalyticsImportance of Big Data Analytics
Importance of Big Data Analytics
 

Recently uploaded

Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 

Recently uploaded (20)

Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 

Automate and Optimize Data Warehouse Migration to Snowflake

  • 1. Copyright © 2019 Impetus Technologies, Inc. You are prohibited from making a copy or modification of, or from redistributing, rebroadcasting, or re-encoding of this content without the prior written consent of Impetus. This presentation may include images from other products and services. These images are used for illustrative purposes only. Unless explicitly stated there is no implied endorsement or sponsorship of these products by Impetus. All copyrights and trademarks are property of their respective owners.
  • 2. Automate and Optimize Data Warehouse Migration to Snowflake IMP
  • 3. Our Mission Enabling a unified, clear, and present view of your business
  • 4. Contents Cloud migration strategy How Impetus can help Automated workload transformation to Snowflake Demo Q&A
  • 6. Speakers Barry Tuthill VP Field Operations Impetus Technologies Sanjay Sharma Principal Architect Impetus Technologies
  • 8. Key Considerations Business • BAU • Existing investments • Risk management • Cost and scale • People, processes, etc. • Interoperability
  • 9. Key Considerations Technical • Architecture (Tech stack) • Performance • Migration blueprint • Automation • Parallel-run period
  • 10. Top 5 Concerns Existing investments Migrate as is vs. total re-engineering Extent of automation Risk and uncertainty Performance and cost
  • 12. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 13. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 14. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 15. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 17. End-to-end Automated Migration Assessment and recommendations for migration Optimized Schema (DDL) transformation Data migration – one time and incremental pipelines setup Workload logic (DML) conversion to SnowSQL Stored Procedure conversion to Snow JavaScript procedures, Spark or Python wrappers Scheduling/ orchestration logic conversion to Shell or Python or Cloud native wrappers Agile DevOps, Continuous integration and delivery process setup Validation and certification
  • 18. Automating the Complete Migration Lifecycle 1 2 3 4 Assess Transform Validate Execute
  • 19. Assessment Automated profiling Identification of workloads and dependencies Creation of optimized schema Grouping of workloads
  • 20. Transformation 90% automated code conversion to SnowSQL Automated data migration to an optimized schema Automated handling of data types, views, UDFs and stored procedures Creation of patterns for the target platform Auto-generation of patterns for newer migrations Query-editing for optimized fixes and performance tuning
  • 21. Validation and Execution Auto-generation of validation/ reconciliation scripts Execution through cloud ready orchestration/ execution engine Integrated DevOps and Agile processes setup Accelerated decommissioning methodology to retire legacy footprint Multi cloud ready with security and governance
  • 22. Automated Workload Transformation to Snowflake ENTERPRISE DATA WAREHOUSE VALIDATE Query Logs Script Files Data Query BIG DATA WAREHOUSE ASSESS EXECUTETRANSFORM Data - S3/Spark, Snowpipe, ETL SQL, Stored Procedures, Scheduling/ Workflow LOGIC TRANSFORMATION SnowSQL Snow JavaScript procedures/ Spark/ Python / Wrappers Ingestion, Scheduler/ workflow setup
  • 23. Demo
  • 24. Business Benefits Accelerate time-to-value Meet performance SLAs Decrease risk and uncertainty Automate end-to-end migration Save up to 80% in cost, time, and effort Maximize ROI
  • 25. Poll Results and Feedback
  • 26. Q&A
  • 27. Thank You Visit www.impetus.com or get in touch with us at inquiry@impetus.com

Editor's Notes

  1. This could be the poll questions How to leverage existing investments? Low hanging fruits Interdependent workloads
  2. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  3. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  4. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  5. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  6. Data types – configuration driven mapping from source data warehouse to Snowflake data types
  7. Automated legacy data warehouse inventory and profiling Identification of workloads (metadata, data, etc.) and dependencies Creation of optimized schema (clustering keys, Parquet format/file size for S3 uploads, etc.) Grouping of workloads into migration units
  8. Up to 90% automated code conversion to SnowSQL Automated data migration to an optimized schema Automated handling of data types, views, intervals, loops, UDFs, procedures Creation of patterns for the target platform (ingestion, data sync, recon, lineage, security, orchestration, etc.) Auto-generation of patterns for newer migrations Query-editing for optimized fixes and performance tuning
  9. Data types – configuration driven mapping from source data warehouse to Snowflake data types