SlideShare a Scribd company logo
p. 11
1
Chapter: SQL Server 2012 Integration Services
Course: SQL Server 2012 - A Comprehensive Introduction
Course ID: 170
Instructor: Scott Whigham
Chapter 16: Video # 2
The Importance
of the ETL
Process
p. 22
2
SQL Server 2012 Integration
Services (SSIS) is Microsoft’s ETL
tool
– Extract, Transform, and Load
p. 33
3
Most businesses have data in more
than one format
– How does one business happen to use so
many different databases?
p. 44
4
Let’s walk through a likely scenario
and see how this happens:
– 2001: The “AdventureWorks” company
launches a web store to complement its
brick-and-mortar stores
• ASP-based website
• SQL Server 2000 backend
• Customers are encouraged to phone questions in or
to send an email
p. 55
5
Things change...
– 2001: Launch with SQL 2000
– 2003: AdventureWorks buys a competitor
• Competitor used a PHP/MySQL ticketing system
• AW mgmt chooses to adopt this system for customer
ticketing rather than build/buy an alternative
p. 66
6
AdventureWorks timeline:
Year Usage Data Source
2001 Website MS SQL Server 2000
2003 Customer Ticket System MySQL 3.23
p. 77
7
Needs change...
– 2001: Launch with SQL 2000
– 2003: PHP/MySQL 3.23 ticketing system
– 2004: The company is growing – time for
more “stuff”:
• A PHP/MySQL project management system is
installed
• A marketing mailer application with contact mgmt
is purchased
p. 88
8
AdventureWorks timeline:
Year Usage Data Source
2001 Website MS SQL Server 2000
2003 Customer Ticket System MySQL 3.23
2004 Project Management MySQL 4.0
2004 Marketing mailer MS Access
p. 99
9
Markets change...
– 2001: Launch with SQL 2000
– 2003: PHP/MySQL 3.23 ticketing system
– 2004: PHP/MySQL 4.0 project management
– 2005: A new ASP.NET website is rolled out
with a SQL Server 2005 backend
• Major upgrade from SQL Server 2000 -> 2005
p. 1010
10
AdventureWorks timeline:
Year Usage Data Source
2001 Website MS SQL Server 2000
2003 Customer Ticket System MySQL 3.23
2004 Project Management MySQL 4.0
2004 Marketing mailer MS Access
2005 Website upgrade MS SQL Server 2005
p. 1111
11
Trends change...
– 2001: Launch with SQL 2000
– 2003: PHP/MySQL 3.23 ticketing system
– 2004: PHP/MySQL 4.0 project management
– 2005: Upgraded website to SQL 2005
– 2008: Website sales popularity causes
“growing pains”
• A new supply chain management app purchased
• A new employee management/HR/payroll package
is purchased
p. 1212
12
AdventureWorks timeline:
Year Usage Data Source
2001 Website MS SQL Server 2000
2003 Customer Ticket System MySQL 3.23
2004 Project Management MySQL 4.0
2004 Marketing mailer MS Access
2005 Website upgrade MS SQL Server 2005
2008 Supply chain mgmt MS SQL Server 2008
2008 Employee/HR/Payroll DB2
p. 1313
13
The world grows smaller...
– 2001: Launch with SQL 2000
– 2003: PHP/MySQL 3.23 ticketing system
– 2004: PHP/MySQL 4.0 project management
– 2005: Upgraded website to SQL 2005
– 2008: Added supply chain mgmt and HR/payroll packages
– 2010: Website sales continue to gain
popularity, particularly overseas
• A new shipping database is purchased
• Employee expenses are now tracked in custom MS
Excel spreadsheets
p. 1414
14
AdventureWorks timeline:
Year Usage Data Source
2001 Website MS SQL Server 2000
2003 Customer Ticket System MySQL 3.23
2004 Project Management MySQL 4.0
2004 Marketing mailer MS Access
2005 Website upgrade MS SQL Server 2005
2008 Supply chain mgmt MS SQL Server 2008
2008 Employee/HR/Payroll DB2
2010 Shipping *.csv file downloaded monthly
2010 Employee expense tracking MS Excel
p. 1515
15
It’s 2012 and company executives +
management have been playing a
game lately...
– You know this one, don’t you?
p. 1616
16
p. 1717
17
The world grows smaller...
– 2001: Launch with SQL 2000
– 2003: PHP/MySQL 3.23 ticketing system
– 2004: PHP/MySQL 4.0 project management
– 2005: Upgraded website to SQL 2005
– 2008: Added supply chain mgmt and HR/payroll packages
– 2010: New shipping database, employee expense tracking
– 2012: Executives want a B.I. solution
• You name it, they want it
• But...
– there’s no budget for software purchases...
p. 1818
18
No budget for new software = more
opportunities for you!
– You decide:
• ... to create a relational OLAP data warehouse to store all the
company’s historic data in a unified way
• ... to create a multidimensional database with multiple cubes (to
facilitate fast browsing of analytics)
• ... to install Excel 2013 on all CxO and management machines, and
to teach them how to build pivot tables and pivot charts
• ... to investigate Reporting Services as a way to build internal web
dashboards and subscription-based reporting
– On-the-job experience, here we come!
p. 1919
19
The company data is all “loosely
connected”
– A customer makes a small order via the website
– The same customer submits a “Help!” ticket
– Customer rep. has to make an order for a
replacement part
– Sales person takes customer to an entertainment
event
– Customer now makes a large order
– Key question: how did we acquire this customer?
p. 2020
20
Integration Services is your ETL tool
1. You Extract the data from the source to a
staging area
• Optional, but typically an MS SQL Server relational
database
2. You make any changes to the data (a.k.a. a
Transformation)
• Either in motion or in the staging area
3. You Load the data into the relational data
warehouse
4. You process the cube(s)
– SSIS is your “one stop shop” for all of this!
p. 2121
21
Your final step is to build a
dashboard
– Reporting Services or PowerPivot?
– Power View or Excel?
– SharePoint or email?
– On-demand or subscription-based?
p. 2222
22
Your dashboard is a hit!
p. 2323
23
In the next video…
–How to Install and Configure SSIS
2012
“A painter paints pictures on canvas. But musicians paint their
pictures on silence.”
- Leopold Stokowski

More Related Content

Viewers also liked

Informatica session
Informatica sessionInformatica session
Informatica session
vinuthanallam
 
Sql server 2012 tutorials reporting services
Sql server 2012 tutorials   reporting servicesSql server 2012 tutorials   reporting services
Sql server 2012 tutorials reporting servicesSteve Xu
 
A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0 A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0
DataWorks Summit
 
Large scale ETL with Hadoop
Large scale ETL with HadoopLarge scale ETL with Hadoop
Large scale ETL with Hadoop
OReillyStrata
 
Introduction to ETL and Data Integration
Introduction to ETL and Data IntegrationIntroduction to ETL and Data Integration
Introduction to ETL and Data Integration
CloverDX (formerly known as CloverETL)
 
ETL Process
ETL ProcessETL Process
ETL Process
Karthik Selvaraj
 

Viewers also liked (6)

Informatica session
Informatica sessionInformatica session
Informatica session
 
Sql server 2012 tutorials reporting services
Sql server 2012 tutorials   reporting servicesSql server 2012 tutorials   reporting services
Sql server 2012 tutorials reporting services
 
A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0 A Reference Architecture for ETL 2.0
A Reference Architecture for ETL 2.0
 
Large scale ETL with Hadoop
Large scale ETL with HadoopLarge scale ETL with Hadoop
Large scale ETL with Hadoop
 
Introduction to ETL and Data Integration
Introduction to ETL and Data IntegrationIntroduction to ETL and Data Integration
Introduction to ETL and Data Integration
 
ETL Process
ETL ProcessETL Process
ETL Process
 

Similar to The Importance of the ETL Process

Make Your Decisions Smarter With Msbi
Make Your Decisions Smarter With MsbiMake Your Decisions Smarter With Msbi
Make Your Decisions Smarter With Msbi
Edureka!
 
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
Veselin Georgiev
 
Introduction to MSBI
Introduction to MSBIIntroduction to MSBI
Introduction to MSBIEdureka!
 
BI SQL Server2008R2 Portfolio
BI SQL Server2008R2 PortfolioBI SQL Server2008R2 Portfolio
BI SQL Server2008R2 Portfolio
dlinder25
 
BI Portfolio
BI PortfolioBI Portfolio
BI Portfoliotcomeaux
 
Create a Heidelpay plugin for Sitecore Commerce 9
Create a Heidelpay plugin for Sitecore Commerce 9Create a Heidelpay plugin for Sitecore Commerce 9
Create a Heidelpay plugin for Sitecore Commerce 9
Jonne Kats
 
NZSPC 2013 - Upgrading to SharePoint 2013
NZSPC 2013 - Upgrading to SharePoint 2013NZSPC 2013 - Upgrading to SharePoint 2013
NZSPC 2013 - Upgrading to SharePoint 2013Michael Noel
 
SPCSEA 2013 - Upgrading to SharePoint 2013
SPCSEA 2013 - Upgrading to SharePoint 2013SPCSEA 2013 - Upgrading to SharePoint 2013
SPCSEA 2013 - Upgrading to SharePoint 2013Michael Noel
 
SAP BW Reports - Copy
SAP BW Reports - CopySAP BW Reports - Copy
SAP BW Reports - CopyAby m
 
Heavy Metal PowerPivot Remastered
Heavy Metal PowerPivot RemasteredHeavy Metal PowerPivot Remastered
Heavy Metal PowerPivot RemasteredJason Himmelstein
 
Sukhwant resume
Sukhwant resumeSukhwant resume
Sukhwant resume
Sukhwant Singh
 
Sukhwant resume
Sukhwant resumeSukhwant resume
Sukhwant resume
Sukhwant Singh
 
Mai Nguyen Profile
Mai Nguyen ProfileMai Nguyen Profile
Mai Nguyen ProfileMai Nguyen
 
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
p6academy
 
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned ApproachSPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
NCCOMMS
 
Bi developer gary thompson
Bi developer   gary thompsonBi developer   gary thompson
Bi developer gary thompsonGary Thompson
 
Bi developer gary thompson
Bi developer   gary thompsonBi developer   gary thompson
Bi developer gary thompsonGary Thompson
 
Unlock The Value Of Your Microsoft and SAP Investments
Unlock The Value Of Your Microsoft and SAP InvestmentsUnlock The Value Of Your Microsoft and SAP Investments
Unlock The Value Of Your Microsoft and SAP Investments
SAP Technology
 

Similar to The Importance of the ETL Process (20)

Make Your Decisions Smarter With Msbi
Make Your Decisions Smarter With MsbiMake Your Decisions Smarter With Msbi
Make Your Decisions Smarter With Msbi
 
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
Abilitics-Microsoft Days 11-Case Ctudy: Where SharePoint, Silverlight & BI me...
 
Introduction to MSBI
Introduction to MSBIIntroduction to MSBI
Introduction to MSBI
 
BI SQL Server2008R2 Portfolio
BI SQL Server2008R2 PortfolioBI SQL Server2008R2 Portfolio
BI SQL Server2008R2 Portfolio
 
BI Portfolio
BI PortfolioBI Portfolio
BI Portfolio
 
Create a Heidelpay plugin for Sitecore Commerce 9
Create a Heidelpay plugin for Sitecore Commerce 9Create a Heidelpay plugin for Sitecore Commerce 9
Create a Heidelpay plugin for Sitecore Commerce 9
 
NZSPC 2013 - Upgrading to SharePoint 2013
NZSPC 2013 - Upgrading to SharePoint 2013NZSPC 2013 - Upgrading to SharePoint 2013
NZSPC 2013 - Upgrading to SharePoint 2013
 
My SQL Portfolio
My SQL PortfolioMy SQL Portfolio
My SQL Portfolio
 
SPCSEA 2013 - Upgrading to SharePoint 2013
SPCSEA 2013 - Upgrading to SharePoint 2013SPCSEA 2013 - Upgrading to SharePoint 2013
SPCSEA 2013 - Upgrading to SharePoint 2013
 
SAP BW Reports - Copy
SAP BW Reports - CopySAP BW Reports - Copy
SAP BW Reports - Copy
 
Heavy Metal PowerPivot Remastered
Heavy Metal PowerPivot RemasteredHeavy Metal PowerPivot Remastered
Heavy Metal PowerPivot Remastered
 
Sukhwant resume
Sukhwant resumeSukhwant resume
Sukhwant resume
 
Sukhwant resume
Sukhwant resumeSukhwant resume
Sukhwant resume
 
Mai Nguyen Profile
Mai Nguyen ProfileMai Nguyen Profile
Mai Nguyen Profile
 
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
Case study migration from cm13 to cm14 - Oracle Primavera P6 Collaborate 14
 
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned ApproachSPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
SPCA2013 - Upgrade to SharePoint 2013 - A Cautioned Approach
 
Bi developer gary thompson
Bi developer   gary thompsonBi developer   gary thompson
Bi developer gary thompson
 
Bi developer gary thompson
Bi developer   gary thompsonBi developer   gary thompson
Bi developer gary thompson
 
Unlock The Value Of Your Microsoft and SAP Investments
Unlock The Value Of Your Microsoft and SAP InvestmentsUnlock The Value Of Your Microsoft and SAP Investments
Unlock The Value Of Your Microsoft and SAP Investments
 
Bi developer gary t
Bi developer   gary tBi developer   gary t
Bi developer gary t
 

More from LearnItFirst.com

What Types of Reports Can You Create?
What Types of Reports Can You Create? What Types of Reports Can You Create?
What Types of Reports Can You Create?
LearnItFirst.com
 
What Is SQL Server 2012?
What Is SQL Server 2012? What Is SQL Server 2012?
What Is SQL Server 2012?
LearnItFirst.com
 
Course 170: SQL Server 2012: A Comprehensive Introduction
Course 170: SQL Server 2012: A Comprehensive Introduction Course 170: SQL Server 2012: A Comprehensive Introduction
Course 170: SQL Server 2012: A Comprehensive Introduction
LearnItFirst.com
 
How To Become A Good C# Programmer
How To Become A Good C# ProgrammerHow To Become A Good C# Programmer
How To Become A Good C# Programmer
LearnItFirst.com
 
Windows Server 2003 Administration
Windows Server 2003 AdministrationWindows Server 2003 Administration
Windows Server 2003 Administration
LearnItFirst.com
 
ISA Server 2006 Administration
ISA Server 2006 AdministrationISA Server 2006 Administration
ISA Server 2006 Administration
LearnItFirst.com
 
Exchange Server 2007 Administration
Exchange Server 2007 AdministrationExchange Server 2007 Administration
Exchange Server 2007 Administration
LearnItFirst.com
 
Exchange Server 2003 Administration
Exchange Server 2003 AdministrationExchange Server 2003 Administration
Exchange Server 2003 Administration
LearnItFirst.com
 
C# Programming with Visual Studio 2005
C# Programming with Visual Studio 2005C# Programming with Visual Studio 2005
C# Programming with Visual Studio 2005
LearnItFirst.com
 
SQL Server 2008 Database Administration
SQL Server 2008 Database AdministrationSQL Server 2008 Database Administration
SQL Server 2008 Database Administration
LearnItFirst.com
 
SQL Server 2005 Database Administration
SQL Server 2005 Database AdministrationSQL Server 2005 Database Administration
SQL Server 2005 Database Administration
LearnItFirst.com
 
SQL Server 2000 Database Administration
SQL Server 2000 Database AdministrationSQL Server 2000 Database Administration
SQL Server 2000 Database Administration
LearnItFirst.com
 

More from LearnItFirst.com (12)

What Types of Reports Can You Create?
What Types of Reports Can You Create? What Types of Reports Can You Create?
What Types of Reports Can You Create?
 
What Is SQL Server 2012?
What Is SQL Server 2012? What Is SQL Server 2012?
What Is SQL Server 2012?
 
Course 170: SQL Server 2012: A Comprehensive Introduction
Course 170: SQL Server 2012: A Comprehensive Introduction Course 170: SQL Server 2012: A Comprehensive Introduction
Course 170: SQL Server 2012: A Comprehensive Introduction
 
How To Become A Good C# Programmer
How To Become A Good C# ProgrammerHow To Become A Good C# Programmer
How To Become A Good C# Programmer
 
Windows Server 2003 Administration
Windows Server 2003 AdministrationWindows Server 2003 Administration
Windows Server 2003 Administration
 
ISA Server 2006 Administration
ISA Server 2006 AdministrationISA Server 2006 Administration
ISA Server 2006 Administration
 
Exchange Server 2007 Administration
Exchange Server 2007 AdministrationExchange Server 2007 Administration
Exchange Server 2007 Administration
 
Exchange Server 2003 Administration
Exchange Server 2003 AdministrationExchange Server 2003 Administration
Exchange Server 2003 Administration
 
C# Programming with Visual Studio 2005
C# Programming with Visual Studio 2005C# Programming with Visual Studio 2005
C# Programming with Visual Studio 2005
 
SQL Server 2008 Database Administration
SQL Server 2008 Database AdministrationSQL Server 2008 Database Administration
SQL Server 2008 Database Administration
 
SQL Server 2005 Database Administration
SQL Server 2005 Database AdministrationSQL Server 2005 Database Administration
SQL Server 2005 Database Administration
 
SQL Server 2000 Database Administration
SQL Server 2000 Database AdministrationSQL Server 2000 Database Administration
SQL Server 2000 Database Administration
 

Recently uploaded

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 

Recently uploaded (20)

Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 

The Importance of the ETL Process

  • 1. p. 11 1 Chapter: SQL Server 2012 Integration Services Course: SQL Server 2012 - A Comprehensive Introduction Course ID: 170 Instructor: Scott Whigham Chapter 16: Video # 2 The Importance of the ETL Process
  • 2. p. 22 2 SQL Server 2012 Integration Services (SSIS) is Microsoft’s ETL tool – Extract, Transform, and Load
  • 3. p. 33 3 Most businesses have data in more than one format – How does one business happen to use so many different databases?
  • 4. p. 44 4 Let’s walk through a likely scenario and see how this happens: – 2001: The “AdventureWorks” company launches a web store to complement its brick-and-mortar stores • ASP-based website • SQL Server 2000 backend • Customers are encouraged to phone questions in or to send an email
  • 5. p. 55 5 Things change... – 2001: Launch with SQL 2000 – 2003: AdventureWorks buys a competitor • Competitor used a PHP/MySQL ticketing system • AW mgmt chooses to adopt this system for customer ticketing rather than build/buy an alternative
  • 6. p. 66 6 AdventureWorks timeline: Year Usage Data Source 2001 Website MS SQL Server 2000 2003 Customer Ticket System MySQL 3.23
  • 7. p. 77 7 Needs change... – 2001: Launch with SQL 2000 – 2003: PHP/MySQL 3.23 ticketing system – 2004: The company is growing – time for more “stuff”: • A PHP/MySQL project management system is installed • A marketing mailer application with contact mgmt is purchased
  • 8. p. 88 8 AdventureWorks timeline: Year Usage Data Source 2001 Website MS SQL Server 2000 2003 Customer Ticket System MySQL 3.23 2004 Project Management MySQL 4.0 2004 Marketing mailer MS Access
  • 9. p. 99 9 Markets change... – 2001: Launch with SQL 2000 – 2003: PHP/MySQL 3.23 ticketing system – 2004: PHP/MySQL 4.0 project management – 2005: A new ASP.NET website is rolled out with a SQL Server 2005 backend • Major upgrade from SQL Server 2000 -> 2005
  • 10. p. 1010 10 AdventureWorks timeline: Year Usage Data Source 2001 Website MS SQL Server 2000 2003 Customer Ticket System MySQL 3.23 2004 Project Management MySQL 4.0 2004 Marketing mailer MS Access 2005 Website upgrade MS SQL Server 2005
  • 11. p. 1111 11 Trends change... – 2001: Launch with SQL 2000 – 2003: PHP/MySQL 3.23 ticketing system – 2004: PHP/MySQL 4.0 project management – 2005: Upgraded website to SQL 2005 – 2008: Website sales popularity causes “growing pains” • A new supply chain management app purchased • A new employee management/HR/payroll package is purchased
  • 12. p. 1212 12 AdventureWorks timeline: Year Usage Data Source 2001 Website MS SQL Server 2000 2003 Customer Ticket System MySQL 3.23 2004 Project Management MySQL 4.0 2004 Marketing mailer MS Access 2005 Website upgrade MS SQL Server 2005 2008 Supply chain mgmt MS SQL Server 2008 2008 Employee/HR/Payroll DB2
  • 13. p. 1313 13 The world grows smaller... – 2001: Launch with SQL 2000 – 2003: PHP/MySQL 3.23 ticketing system – 2004: PHP/MySQL 4.0 project management – 2005: Upgraded website to SQL 2005 – 2008: Added supply chain mgmt and HR/payroll packages – 2010: Website sales continue to gain popularity, particularly overseas • A new shipping database is purchased • Employee expenses are now tracked in custom MS Excel spreadsheets
  • 14. p. 1414 14 AdventureWorks timeline: Year Usage Data Source 2001 Website MS SQL Server 2000 2003 Customer Ticket System MySQL 3.23 2004 Project Management MySQL 4.0 2004 Marketing mailer MS Access 2005 Website upgrade MS SQL Server 2005 2008 Supply chain mgmt MS SQL Server 2008 2008 Employee/HR/Payroll DB2 2010 Shipping *.csv file downloaded monthly 2010 Employee expense tracking MS Excel
  • 15. p. 1515 15 It’s 2012 and company executives + management have been playing a game lately... – You know this one, don’t you?
  • 17. p. 1717 17 The world grows smaller... – 2001: Launch with SQL 2000 – 2003: PHP/MySQL 3.23 ticketing system – 2004: PHP/MySQL 4.0 project management – 2005: Upgraded website to SQL 2005 – 2008: Added supply chain mgmt and HR/payroll packages – 2010: New shipping database, employee expense tracking – 2012: Executives want a B.I. solution • You name it, they want it • But... – there’s no budget for software purchases...
  • 18. p. 1818 18 No budget for new software = more opportunities for you! – You decide: • ... to create a relational OLAP data warehouse to store all the company’s historic data in a unified way • ... to create a multidimensional database with multiple cubes (to facilitate fast browsing of analytics) • ... to install Excel 2013 on all CxO and management machines, and to teach them how to build pivot tables and pivot charts • ... to investigate Reporting Services as a way to build internal web dashboards and subscription-based reporting – On-the-job experience, here we come!
  • 19. p. 1919 19 The company data is all “loosely connected” – A customer makes a small order via the website – The same customer submits a “Help!” ticket – Customer rep. has to make an order for a replacement part – Sales person takes customer to an entertainment event – Customer now makes a large order – Key question: how did we acquire this customer?
  • 20. p. 2020 20 Integration Services is your ETL tool 1. You Extract the data from the source to a staging area • Optional, but typically an MS SQL Server relational database 2. You make any changes to the data (a.k.a. a Transformation) • Either in motion or in the staging area 3. You Load the data into the relational data warehouse 4. You process the cube(s) – SSIS is your “one stop shop” for all of this!
  • 21. p. 2121 21 Your final step is to build a dashboard – Reporting Services or PowerPivot? – Power View or Excel? – SharePoint or email? – On-demand or subscription-based?
  • 23. p. 2323 23 In the next video… –How to Install and Configure SSIS 2012 “A painter paints pictures on canvas. But musicians paint their pictures on silence.” - Leopold Stokowski