4. Old Process
~ 1500 Lines of SQL
No Variables
No Functions
Code Duplicated
Everything Hard-coded
SQL Scripts
5. Old Process – Phase 1, Day 1 of 3
Developer Creates Initial SQL Scripts For
Monthly We Want You Back Campaign
• ~ 2 Thousand Of Lines Of Sql
• Hard-coded Criteria
• Customized For Business Rules, Which
Change Every Month
• ~ 4 Hours
Developer
SQL Scripts
6. Old Process – Phase 1, Day 1 of 3
Developer Runs SQL To Generate Customer
Records
• Each Script Takes Several Minutes To Run
• SQL Runs Against Production DB, Impacting
Other Applications
• Risk: DB Locks, Server Crashes, Etc.
• ~ 1 Hour
PROD DB
Developer SQL Scripts
7. Old Process – Phase 1, Day 1 of 3
Developer:
• Runs Scripts
• Gets Results
• Creates Excel Files
• Waits For VP To Verify
Customer Counts
• ~ 1- 4 Hours
PROD
DB
LAF VP, Risk
Results
Problem With Customer Counts?
Repeat Process With New Criteria
Customer Counts
OK? Proceed to
Next Step
Developer
Copy and Paste
SQL Scripts
8. Old Process – Phase 1, Day 1 of 3
• Developer Updates SQL To Be
Run With Hot Fix
• Copies Script Files To Deployment
Folder, Submits Hot Fix
• ~ 1 Hour To Prepare Hot Fix
• ~ 1 Hour To Approve Hot Fix
• > 1 Day To Run All Scripts In
Prod (Overnight)
• ~ 4 Hours
PM
Approves Hot Fix
Hot Fix
Director
Approves Hot Fix
Developer
DBA Runs SQL on PROD DB
PROD DB
SQL Scripts
SQL Scripts
9. Old Process – Phase 1, Day 2 of 3
• Developer Runs SQL To Create CSV
Files, Adds Additional Criteria If Needed
Because Of Business Rules (Club
Closures, Etc.)
• Copies And Pastes Data Into Excel
• Uploads Smaller Data Files To
WhatCounts During The Day
• ~ 4 Hours
• Scripts To Create Csvs Run In
Production, Impacting Other Apps
• Millions Of Records Used To Create
Several Csv Files
• ~8 Hours
Developer
Developer
Developer
Copy and Paste
PROD DB
SQL Scripts
10. Old Process – Phase 1, Day 2 of 3
Regular EFT Extract/Upload
• Developer Confirms That
Overnight Hot Fix Was
Completed
• Starts Upload Of ~2
Millions Records To What
Counts
• ~ 2 Hours
Developer PROD DB
Copy and Paste
Developer
SQL Scripts
11. Old Process – Phase 1, Day 3 of 3
• Developer confirms all
customer data has been
uploaded to
WhatCounts overnight
• Developer kicks off
email campaign for
Regular EFT
• ~2 Hours
Developer
Former Member Gets Email
12. Old Process – Phase 2, Day 1 of 2
WWYB Resend
• ~ 2 Million Emails Need To Be
Resent To Former Members
Who Received Email From
Phase 1
• Developer Runs SQL Scripts
To Extract Member Data For
Emails Sent In Phase 1
• Developer Starts Upload Of
~2 Millions Records To What
Counts To Run Overnight
• ~8 Hours
Developer
Developer
Copy and PastePROD DB
SQL Scripts
13. Old Process – Phase 2, Day 2 of 2
• Developer Confirms All
Customer Data Has
Been Uploaded To
Whatcounts Overnight
• Developer Kicks Off
Email Resend Campaign
• ~1 Hour
Developer
Former Member Gets Email
14. New Process – ETL Workflow
Extract
Transform
Load
PROD
DB
Email
DB
15. New Process – Phase 1, Day 1 of 1
• Developer Runs SSIS Package To
Create CSV Files
• Confirms Counts With VP
• Admin Assistant Starts Upload Of ~2
Millions Records To What Counts
• Admin Assistant Confirms All
Customer Data Has Been Uploaded
To Whatcounts
• Admin Assistant Kicks Off Email
Campaign
• ~ 3 Hours From Start To Finish
Developer
IRV-SQL-
003
(Email DB)
PROD DB
Admin.
Assistant
Former Member Gets Email
SSIS
SSIS
LAF VP, Risk
Problem With Customer Counts?
Update Tables, Repeat Process With New Criteria
16. New Process – Phase 2, Day 1 of 1
WWYB Resend
• ~ 2 Million Emails Need To Be
Resent To Former Members
Who Received Email From
Phase 1
• Admin Assistant Runs Script
To Create Suppression Lists
• Admin Assistant Uploads Data
To WhatCounts
• Admin Assistant Runs
Campaign
• ~1 Hour From Start To Finish
Admin.
Assistant
Admin.
Assistant
Suppression ListsPROD DB
Former Member Gets Email
Admin.
Assistant
17. Old Process vs. New Process
Old Process New Process
Duration ~20 Hours over 5 Days, every month (240-hours/year) ~3 Hours over 2 Days, every month (36-hours/year)
Resources Involved 1. Developer (75%)
2. PM (2%)*
3. Director (2%)*
4. DBA (6%)*
5. Admin. Assistant (15%)
1. Developer (2%)
2. Admin. Assistant (98%)
Process 1. Hard-coded
2. SQL Scripts
3. Manual
4. Hi-Risk/Hi-Stress
5. Slow Process: Days, not Hours
6. Most Work Done By Expensive Resources (Dev, DBA)
1. Table-Driven
2. SSIS Packages, Stored Procs, Functions, Parallel Processing
3. Automated
4. Low-Risk/Low-Stress
5. Fast Process: Hours, not Days
6. Low-cost Resources (mostly done by Admin. Assistant)
Maintainability Poor Trivial
Cost $18,000/year (240 hours@$75/hr) $1800/year (36 hours@50/hr)
10% of Old Process
Points of Failure 1. Emails sent to members of closed/sold clubs
2. Date Criteria incorrect
3. Impact on Production DB (SQL must be run at night)
4. No time for QA
None
*HotFix Process
