Kyle Hailey, profile picture
Uploaded byKyle Hailey
PPTX, PDF796 views

Agile Data: revolutionizing data and database cloning

This document discusses how data constraints can strain IT departments and be costly for businesses. It presents the following key points: 1. Moving and managing data is difficult and expensive for IT, consuming resources like storage, personnel time, and infrastructure costs. This strains IT departments and slows application development. 2. The total cost of data constraints for businesses, including capital, operations, development delays, and lost revenue opportunities, can be huge - in the billions of dollars for large companies. 3. Many companies are unaware of how much their current data management practices cost them and constrain their businesses because IT departments think their current processes are sufficient. The document proposes using a data virtualization solution called Delphix

Embed presentation

Downloaded 35 times
1 / 141
2 / 141
3 / 141
4 / 141
5 / 141
6 / 141
7 / 141
8 / 141
9 / 141
10 / 141
11 / 141
12 / 141
13 / 141
14 / 141
15 / 141
16 / 141
17 / 141
18 / 141
19 / 141
20 / 141
21 / 141
22 / 141
23 / 141
24 / 141
25 / 141
26 / 141
27 / 141
28 / 141
29 / 141
30 / 141
31 / 141
32 / 141
33 / 141
34 / 141
35 / 141
36 / 141
37 / 141
38 / 141
39 / 141
40 / 141
41 / 141
42 / 141
43 / 141
44 / 141
45 / 141
46 / 141
47 / 141
48 / 141
49 / 141
50 / 141
51 / 141
52 / 141
53 / 141
54 / 141
55 / 141
56 / 141
57 / 141
58 / 141
59 / 141
60 / 141
61 / 141
62 / 141
63 / 141
64 / 141
65 / 141
66 / 141
67 / 141
68 / 141
69 / 141
70 / 141
71 / 141
72 / 141
73 / 141
74 / 141
75 / 141
76 / 141
77 / 141
78 / 141
79 / 141
80 / 141
81 / 141
82 / 141
83 / 141
84 / 141
85 / 141
86 / 141
87 / 141
88 / 141
89 / 141
90 / 141
91 / 141
92 / 141
93 / 141
94 / 141
95 / 141
96 / 141
97 / 141
98 / 141
99 / 141
100 / 141
101 / 141
102 / 141
103 / 141
104 / 141
105 / 141
106 / 141
107 / 141
108 / 141
109 / 141
110 / 141
111 / 141
112 / 141
113 / 141
114 / 141
115 / 141
116 / 141
117 / 141
118 / 141
119 / 141
120 / 141
121 / 141
122 / 141
123 / 141
124 / 141
125 / 141
126 / 141
127 / 141
128 / 141
129 / 141
130 / 141
131 / 141
132 / 141
133 / 141
134 / 141
135 / 141
136 / 141
137 / 141
138 / 141
139 / 141
140 / 141
141 / 141
Agile Data : Virtual Data Revolution Kyle@delphix.com kylehailey.com slideshare.com/khailey
The Goal - Theory of Constraints (TOC) Improvement not made at the constraint is an illusion
Factory floor : straight forward
Factory floor : straight forward constraint
Factory floor : optimize at the constraint constraint Tuning here Stock piling
Factory floor : optimize at the constraint constraint Tuning here Stock piling Tuning here Starvation
Factory floor : straight forward constraint
Factory Floor Optimization
Drum, Rope, Buffer (DBR) 45 75 305080 constraint 25
For IT does the Theory of Constraints work ?
The Phoenix Project • Constraints Identify • Metrics Define • Priorities Set • Goals Clarify • Iterations Fast • CI continuous integration • Cloud • Agile • Kanban
The Phoenix Project What is the constraint in IT ? “One of the most powerful things that IT can do is get environments to development and QA when they need it” 2x throughput increase with agile data
In this presentation : • Data Constraint • Solution • Use Cases
In this presentation : • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
Data is the constraint 60% Projects Over Schedule 85% delayed waiting for data Data is the Constraint CIO Magazine Survey: Current situation: only getting worse … Data Doomsday
In this presentation : • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
I. Data Constraint strains IT If you can’t satisfy the business demands then your process is broken.
I. Data Constraint : moving data is hard – Storage & Systems – Personnel – Time
Typical Architecture Production Instance File system Database
Typical Architecture Production Instance Backup File system Database File system Database
Typical Architecture Production Instance Reporting Backup File system Database Instance File system Database File system Database
Typical Architecture Production Instance File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database Dev, QA, UAT Reporting Backup Triple Tax
Typical Architecture Production Instance File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database
In this presentation : • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
II. Data Constraint price is huge
I. Data constraint: Data floods company infrastructure 92% of the cost of business , in financial services business , is “data” www.wsta.org/resources/industry-articles Most companies have 2-9% IT spending http://uclue.com/?xq=1133 Data management is the largest Part of IT expense Gartner: Data Doomsday
II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$
II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
II. Data constraint price is huge : 1. IT Capital • Hardware –Servers –Storage –Network –Data center floor space, power, cooling
II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
II. Data constraint price is huge : 2. IT Operations • People – DBAs – SYS Admin – Storage Admin – Backup Admin – Network Admin • Hours : 1000s just for DBAs • $100s Millions for data center modernizations
II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
II. Data constraint price is Huge : 3. App Dev • Inefficient QA: Higher costs of QA • QA Delays : Greater re-work of code • Sharing DB Environments : Bottlenecks • Using DB Subsets: More bugs in Prod • Slow Environment Builds: Delays “if you can't measure it you can’t manage it”
II. Data Tax is Huge : 3. App Dev Slow Environment Builds Never enough environments
Part II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
II. Data constraint price is Huge : 4. Business Ability to capture revenue • Business Intelligence – Old data = less intelligence • Business Applications – Delays cause => Lost Revenue
II. Data constraint price is Huge : 4. Business
II. Data constraint price is Huge : 4. Business 0 5 10 15 20 25 30 Storage IT Ops Dev Revenue Billion $
II. Data constraint price is Huge • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ Review
In this presentation : • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
Part III. Data Constraint companies unaware
III. Data Constraint companies unaware DBA Developer
III. Data Constraint companies unaware #1 Biggest Enemy : IT departments believe – best processes – greatest technology – Just the way it is
There are always new and better ways to do things
III. Data Constraint companies unaware Why do I need an iPhone ? Don’t we already do that ? SQL scripts Alter database begin backup Back up datafiles Redo Archive Alter database end backup RMAN
III. Data Constraint companies unaware • Ask Questions – me: we provision environments in minutes for almost not extra storage. – Customer: We already do that – me: How long does it take a developer to get an environment after they ask ? – Customer: 2-3 weeks – me: we do it in 2-3 minutes
III. Data Constraint companies unaware How to enlighten? Ask for metrics – How long does it take a developer to get a DB copy? • QA? – How old is data ? • BI and DW : ETL batch windows • QA and Dev : how often refreshed – How much storage used for copies? • How much DBA time?
In this presentation : • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
Clone 1 Clone 3Clone 2 99% of blocks are identical
Solution
Clone 1 Clone 2 Clone 3 Thin Clone
Technology Core : file system snapshots • Vmware Linked Clones – Not supported for Oracle • EMC – 16 snapshots – Write performance impact • Netapp – 255 snapshots • ZFS – Unlimited snapshots
Engine not equal car
Three Core Parts Production File System Instance DevelopmentStorage 21 3 Copy Sync Snapshots Time Flow Purge Clone (snapshot) Compress Share Cache Storage Mount, recover, rename Self Service, Roles & Security Rollback & Refresh Branch & Tag Instance
Database Virtualization
Three Physical Copies Three Virtual Copies Data Virtualization Appliance
Install Delphix on x86 hardware Intel hardware
Allocate Any Storage to Delphix Allocate Storage Any type Pure Storage + Delphix Better Performance for 1/10 the cost
One time backup of source database Database Production File systemFile system Upcoming Supports InstanceInstanceInstance Application Stack Data
DxFS (Delphix) Compress Data Database Production Data is compressed typically 1/3 size File system InstanceInstanceInstance
Incremental forever change collection Database Production File system Changes • Collected incrementally forever • Old data purged File system Time Window Production InstanceInstanceInstance
Virtual DB 62 / 30 Jonathan Lewis © 2013 Snapshot 1 – full backup once only at link time a b c d e f g h i We start with a full backup - analogous to a level 0 rman backup. Includes the archived redo log files needed for recovery. Run in archivelog mode.
Virtual DB 63 / 30 Jonathan Lewis © 2013 Snapshot 2 (from SCN) b' c' a b c d e f g h i The "backup from SCN" is analogous to a level 1 incremental backup (which includes the relevant archived redo logs). Sensible to enable BCT. Delphix executes standard rman scripts
Virtual DB 64 / 30 Jonathan Lewis © 2013 a b c d e f g h i Apply Snapshot 2 b' c' The Delphix appliance unpacks the rman backup and "overwrites" the initial backup with the changed blocks - but DxFS makes new copies of the blocks
Virtual DB 65 / 30 Jonathan Lewis © 2013 Drop Snapshot 1 b' c'a d e f g h i The call to rman leaves us with a new level 0 backup, waiting for recovery. But we can pick the snapshot root block. We have EVERY level 0 backup
Virtual DB 66 / 30 Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) b' c'a d e f g h i
Virtual DB 67 / 30 Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) i’b' c'a d e f g h ib' c'a d e f g h i
Before Delphix Production Dev, QA, UAT Instance Reporting Backup File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database “triple data tax”
With Delphix Production Instance Database Dev & QA Instance Database Reporting Instance Database Backup Instance Instance Instance Database InstanceInstance Database InstanceInstance File system Database
In this presentation : • Problem in the Industry • Solution • Use Cases
Use Cases 1. Development 2. QA 3. Recovery 4. Business Intelligence 5. Modernization
Use Cases 1. Development 2. QA 3. Recovery 4. Business Intelligence 5. Modernization
Development • Parallelized Environments • Full size environments • Self Service Development
Development without Agile Data: bottlenecks Frustration Waiting Old Unrepresentative Data
Development without Agile Data: subsets of DB
Development without Agile Data: bugs
Development with Agile Data: Parallelize Environments gif by Steve Karam
Development with Agile Data: Full size copies
Development without Agile Data: slow env build times Developer Asks for DB Get Access Manager approves DBA Request system Setup DB System Admin Request storage Setup machine Storage Admin Allocate storage (take snapshot) 3-6 Months to Deliver Data
Development without Agile Data: slow env build times Why are hand offs so expensive? 1hour 1 day 9 days
http://martinfowler.com/bliki/NoDBA.html Development without Agile Data: slow env build times
Development with Agile Data: Self Service
Use Cases 1. Development 2. QA 3. Recovery 4. Business Intelligence 5. Modernization
QA • Fast • Parallel • Rollback • A/B testing
QA without Agile Data : Long Build times Build Time 96% of QA time was building environment $.04/$1.00 actual testing vs. setup Build Build Time QA Test QA Test Build
QA without Agile Data : slow QA Build QA Env QA Build QA Env QA Sprint 1 Sprint 2 Sprint 3 Bug CodeX 0 10 20 30 40 50 60 70 1 2 3 4 5 6 7 Delay in Fixing the bug Cost To Correct Software Engineering Economics – Barry Boehm (1981)
QA with Agile Data : Fast environments with Branching Instance Instance Instance Source Dev QA branched from Dev Source dev QA
QA with Agile Data: Fast environments with Branching B u i l d T i m e QA Test 1% of QA time was building environment $.99/$1.00 actual testing vs. setup Build Time QA Test Build
QA with Agile Data: bugs found fast Sprint 1 Sprint 2 Sprint 3 Bug CodeX QA QA Build QA Env Q A Build QA Env Q A Sprint 1 Sprint 2 Sprint 3 Bug Cod e X
QA with Agile Data: Parallel environments Instance Instance Instance Instance Source
QA with Agile Data: Rewind for patch and QA testing Instance Instance Development Time Window Prod
QA with Agile Data: A/B testing Instance Instance Instance Index 1 Index 2
Use Cases 1. Development 2. QA 3. Quality 4. Business Intelligence 5. Modernization
Quality • Prod & Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics
Quality : 50 days of backup in size of production
Quality : Surgical recovery Instance Instance Development Time Window Before dropDrop Source
Quality: recovery of development Instance Instance Dev1 VDB Time Window Time Window Dev1 VDB Instance Source Source Dev2 VDB Branched
Quality : recovery of production Instance Instance VDBSource Time Window Corruption
Quality : Forensics - Investigate Production Bugs Instance Time Window Instance Development Bug Yesterday Yesterday
Use Cases 1. Development 2. QA 3. Quality 4. Business Intelligence 5. Modernization
Business Intelligence • 24x7 Batches • Low Bandwidth • Temporal Data • Confidence Testing
Business Intelligence: ETL and Refresh Windows 1pm 10pm 8am noon
Business Intelligence: ETL and DW refreshes taking longer 1pm 10pm 8am noon 2011 2012 2013 2014 2015
Business Intelligence ETL and Refresh Windows 2011 2012 2013 2014 2015 1pm 10pm 8am noon 10pm 8am noon 9pm 6am 8am 10pm
Business Intelligence: ETL and DW Refreshes Instance Prod Instance DW & BI Data Guard – requires full refresh if used Active Data Guard – read only, most reports don’t work
Business Intelligence: Fast Refreshes • Collect only Changes • Refresh in minutes Instance Instance Prod Instance BI and DW ETL 24x7
Business Intelligence: Temporal Data
Business Intelligence a) 24x7 Batches & Refreshes a) Temporal queries b) Confidence testing
Use Cases 1. Development 2. QA 3. Quality 4. Business Intelligence 5. Modernization
Modernization 1. Federated 2. Consolidation 3. Migration 4. Auditing
Modernization: Federated Instance Instance Instance Instance Source1 Source2 Source1
Modernization: Federated
“I looked like a hero” Tony Young, CIO Informatica Modernization: Federated
Data movement required for 1 source DB and 4 clones 5x Source Data Copy < 1 x Source Data Copy S SC C C C V V V V Without Delphix (c = clone) With Delphix (v = virtual DB)
Modernization: Consolidation Without Delphix With Delphix
Dev QA UAT Dev QA UAT 2.6 2.7 Dev QA UAT 2.8 Data Control = Source Control for the Database Production Time Flow Modernization: Auditing & Version Control CIO Insurance 600 Applications CIO Investment Banking 180 Applications CIO South America 65 Applications
Use Cases 1. Development • Parallelized Environments • Full size environments • Self Service 2. QA • Fast • Parallel • Rollback • A/B testing 3. Recovery • Prod & Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics 4. Business Intelligence • 24x7 Batches • Low Bandwidth • Temporal Data • Confidence Testing 5. Modernization • Federated • Consolidation • Migration • Auditing
Use Case Summary 1. Development 2. QA 3. Quality 4. Business Intelligence 5. Modernization
How expensive is the Data Constraint? Before and after Delphix w/ Fortune 500 : Dev throughput increase by 2x
How expensive is the Data Constraint? • 10 x Faster Financial Close – 21 days down to 2 • 9x Faster BI refreshes – 3 weeks per refresh to 3x a week • 2x faster Projects • 20 % less bugs
Agile Data Quotes • “Allowed us to shrink our project schedule from 12 months to 6 months.” – BA Scott, NYL VP App Dev • "It used to take 50-some-odd days to develop an insurance product, … Now we can get a product to the customer in about 23 days.” – Presbyterian Health • “Can't imagine working without it” – Ramesh Shrinivasan CA Department of General Services
Summary • Problem: Data is the constraint • Solution: Agile data is small & fast • Results: Deliver projects – Half the Time – Higher Quality – Increase Revenue Kyle@delphix.com kylehailey.com slideshare.net/khailey
Oracle 12c
80MB buffer cache ?
200GB Cache
5000 Tnxs/minLatency 300 ms 1 5 10 20 30 60 100 200 with 1 5 10 20 30 60 100 200 Users
8000 Tnxs/minLatency 600 ms 1 5 10 20 30 60 100 200 Users 1 5 10 20 30 60 100 200
$1,000,000 1TB cache on SAN $6,000 200GB shared cache on Delphix Five 200GB database copies are cached with :
Use Cases 1. Development • Parallelized Environments • Full size environments • Self Service 2. QA • Fast • Parallel • Rollback • A/B testing 3. Recovery • Prod & Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics 4. Business Intelligence • 24x7 Batches • Low Bandwidth • Temporal Data • Confidence Testing 5. Modernization • Federated • Consolidation • Migration • Auditing
•END
Source Full Copy Source backup from SCN 1
Snapshot 1 Snapshot 2
Snapshot 1 Snapshot 2 Backup from SCN
Snapshot 1 Snapshot 2 Snapshot 3
Drop Snapshot Snapshot 1 Snapshot 2 Snapshot 3 Snapshot 2 Snapshot 3 Drop Snapshot 1
Agile Data: revolutionizing data and database cloning

More Related Content

PDF
S de2784 footprint-reduction-edge2015-v2
byTony Pearson
 
PDF
S016828 storage-tiering-nola-v1710b
byTony Pearson
 
PDF
S100294 bcdr-seven-tiers-orlando-v1804a
byTony Pearson
 
PDF
S100293 hybrid-cloud-orlando-v1804a
byTony Pearson
 
PDF
S100296 data-footprint-orlando-v1804a
byTony Pearson
 
PPTX
EMC config Hadoop
bysolarisyougood
 
PDF
S100298 pendulum-swings-orlando-v1804a
byTony Pearson
 
PDF
Hadoop Analytics + Enterprise Class Storage: One-Stop Solution From EMC for H...
byEMC
 
S de2784 footprint-reduction-edge2015-v2
byTony Pearson
 
S016828 storage-tiering-nola-v1710b
byTony Pearson
 
S100294 bcdr-seven-tiers-orlando-v1804a
byTony Pearson
 
S100293 hybrid-cloud-orlando-v1804a
byTony Pearson
 
S100296 data-footprint-orlando-v1804a
byTony Pearson
 
EMC config Hadoop
bysolarisyougood
 
S100298 pendulum-swings-orlando-v1804a
byTony Pearson
 
Hadoop Analytics + Enterprise Class Storage: One-Stop Solution From EMC for H...
byEMC
 

What's hot

PPTX
Audax Group: CIO Perspectives - Managing The Copy Data Explosion
byactifio
 
PDF
S cv3179 spectrum-integration-openstack-edge2015-v5
byTony Pearson
 
PDF
S016389 ibm-cos-brazil-v1708b
byTony Pearson
 
PDF
IBM Object Storage and Software Defined Solutions - Cleversafe
byDiego Alberto Tamayo
 
PDF
S016579 data-optimization-spectrum-control-brazil-v2
byTony Pearson
 
PDF
Druva In Sync Product Overview
byrammotive
 
PDF
S016386 business-continuity-melbourne-v1708c
byTony Pearson
 
PPTX
In-Place analytics with Unified Data Access
byDataWorks Summit
 
PDF
EMC Starter Kit - IBM BigInsights - EMC Isilon
byBoni Bruno
 
PPTX
IT Backup & Restoration: Never Worry about a Late Backup Again
byHelpSystems
 
PDF
S sy0883 smarter-storage-strategy-edge2015-v4
byTony Pearson
 
PDF
S016825 ibm-cos-nola-v1710d
byTony Pearson
 
PPTX
Emc vi pr software defined storage
bysolarisyougood
 
PDF
S100295 reporting-monitoring-orlando-v1804a
byTony Pearson
 
PDF
S100297 ilm-archive-orlando-v1804c
byTony Pearson
 
PDF
S016826 cloud-storage-nola-v1710d
byTony Pearson
 
PDF
cleversafe_definitive_guide_white_paper
byChris Woeppel
 
PDF
IBM Cloud Storage Options
byTony Pearson
 
PDF
IBM Social Media Birds of a Feather v5
byTony Pearson
 
PDF
Data Footprint Reduction: Understanding IBM Storage Options
byTony Pearson
 
Audax Group: CIO Perspectives - Managing The Copy Data Explosion
byactifio
 
S cv3179 spectrum-integration-openstack-edge2015-v5
byTony Pearson
 
S016389 ibm-cos-brazil-v1708b
byTony Pearson
 
IBM Object Storage and Software Defined Solutions - Cleversafe
byDiego Alberto Tamayo
 
S016579 data-optimization-spectrum-control-brazil-v2
byTony Pearson
 
Druva In Sync Product Overview
byrammotive
 
S016386 business-continuity-melbourne-v1708c
byTony Pearson
 
In-Place analytics with Unified Data Access
byDataWorks Summit
 
EMC Starter Kit - IBM BigInsights - EMC Isilon
byBoni Bruno
 
IT Backup & Restoration: Never Worry about a Late Backup Again
byHelpSystems
 
S sy0883 smarter-storage-strategy-edge2015-v4
byTony Pearson
 
S016825 ibm-cos-nola-v1710d
byTony Pearson
 
Emc vi pr software defined storage
bysolarisyougood
 
S100295 reporting-monitoring-orlando-v1804a
byTony Pearson
 
S100297 ilm-archive-orlando-v1804c
byTony Pearson
 
S016826 cloud-storage-nola-v1710d
byTony Pearson
 
cleversafe_definitive_guide_white_paper
byChris Woeppel
 
IBM Cloud Storage Options
byTony Pearson
 
IBM Social Media Birds of a Feather v5
byTony Pearson
 
Data Footprint Reduction: Understanding IBM Storage Options
byTony Pearson
 

Similar to Agile Data: revolutionizing data and database cloning

PPTX
Delphix Platform Overview
byFranco_Dagosto
 
PPTX
Data Virtualization: revolutionizing database cloning
byKyle Hailey
 
PPTX
Kscope 14 Presentation : Virtual Data Platform
byKyle Hailey
 
PPTX
Nyoug delphix slideshare
byKyle Hailey
 
PPTX
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
byKyle Hailey
 
PPTX
Denver devops : enabling DevOps with data virtualization
byKyle Hailey
 
PPTX
BGOUG "Agile Data: revolutionizing database cloning'
byKyle Hailey
 
PPTX
Virtual Data : Eliminating the data constraint in Application Development
byKyle Hailey
 
PPTX
Data Virtualization: Revolutionizing data cloning
byKyle Hailey
 
PPTX
Accelerate Develoment with VIrtual Data
byKyle Hailey
 
PPTX
Data is the Constraint
byKyle Hailey
 
PPTX
Data as a Service
byKyle Hailey
 
PPTX
Kscope 2013 delphix
byKyle Hailey
 
PPTX
DBTA Data Summit : Eliminating the data constraint in Application Development
byKyle Hailey
 
PPTX
Why 2015 is the Year of Copy Data - What are the requirements?
byStorage Switzerland
 
PDF
e-IT exec lunch - "It's all about data" - 25 May '16
byDevin Deen
 
PPTX
SQL Saturday San Diego
byKellyn Pot'Vin-Gorman
 
PPTX
Version Control meets Database Control
byDBmaestro - Database DevOps
 
PDF
Backing Up Mountains of Data to Disk
byIT Brand Pulse
 
PDF
Maximizing the Benefits of Virtualization with Real-­time Compression
byIBM India Smarter Computing
 
Delphix Platform Overview
byFranco_Dagosto
 
Data Virtualization: revolutionizing database cloning
byKyle Hailey
 
Kscope 14 Presentation : Virtual Data Platform
byKyle Hailey
 
Nyoug delphix slideshare
byKyle Hailey
 
DevOps, Databases and The Phoenix Project UGF4042 from OOW14
byKyle Hailey
 
Denver devops : enabling DevOps with data virtualization
byKyle Hailey
 
BGOUG "Agile Data: revolutionizing database cloning'
byKyle Hailey
 
Virtual Data : Eliminating the data constraint in Application Development
byKyle Hailey
 
Data Virtualization: Revolutionizing data cloning
byKyle Hailey
 
Accelerate Develoment with VIrtual Data
byKyle Hailey
 
Data is the Constraint
byKyle Hailey
 
Data as a Service
byKyle Hailey
 
Kscope 2013 delphix
byKyle Hailey
 
DBTA Data Summit : Eliminating the data constraint in Application Development
byKyle Hailey
 
Why 2015 is the Year of Copy Data - What are the requirements?
byStorage Switzerland
 
e-IT exec lunch - "It's all about data" - 25 May '16
byDevin Deen
 
SQL Saturday San Diego
byKellyn Pot'Vin-Gorman
 
Version Control meets Database Control
byDBmaestro - Database DevOps
 
Backing Up Mountains of Data to Disk
byIT Brand Pulse
 
Maximizing the Benefits of Virtualization with Real-­time Compression
byIBM India Smarter Computing
 

More from Kyle Hailey

PPTX
Delphix and Pure Storage partner
byKyle Hailey
 
PDF
Oaktable World 2014 Kevin Closson: SLOB – For More Than I/O!
byKyle Hailey
 
PPT
Ash masters : advanced ash analytics on Oracle
byKyle Hailey
 
PPT
Delphix for DBAs by Jonathan Lewis
byKyle Hailey
 
PDF
Oaktable World 2014 Toon Koppelaars: database constraints polite excuse
byKyle Hailey
 
PPTX
Hooks in postgresql by Guillaume Lelarge
byKyle Hailey
 
PPTX
What is DevOps
byKyle Hailey
 
PPT
Jonathan Lewis explains Delphix
byKyle Hailey
 
PPTX
Delphix and DBmaestro
byKyle Hailey
 
PPTX
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
byKyle Hailey
 
PDF
Dan Norris: Exadata security
byKyle Hailey
 
PPTX
Mark Farnam : Minimizing the Concurrency Footprint of Transactions
byKyle Hailey
 
PPTX
Performance insights twitch
byKyle Hailey
 
PPT
History of database monitoring
byKyle Hailey
 
PPT
Oracle Open World Thursday 230 ashmasters
byKyle Hailey
 
PDF
Profiling the logwriter and database writer
byKyle Hailey
 
PPT
Big data big_skills_data_visualization
byKyle Hailey
 
PDF
Martin Klier : Volkswagen for Oracle Guys
byKyle Hailey
 
PPT
Successfully convince people with data visualization
byKyle Hailey
 
Delphix and Pure Storage partner
byKyle Hailey
 
Oaktable World 2014 Kevin Closson: SLOB – For More Than I/O!
byKyle Hailey
 
Ash masters : advanced ash analytics on Oracle
byKyle Hailey
 
Delphix for DBAs by Jonathan Lewis
byKyle Hailey
 
Oaktable World 2014 Toon Koppelaars: database constraints polite excuse
byKyle Hailey
 
Hooks in postgresql by Guillaume Lelarge
byKyle Hailey
 
What is DevOps
byKyle Hailey
 
Jonathan Lewis explains Delphix
byKyle Hailey
 
Delphix and DBmaestro
byKyle Hailey
 
Oracle Open World 2014: Lies, Damned Lies, and I/O Statistics [ CON3671]
byKyle Hailey
 
Dan Norris: Exadata security
byKyle Hailey
 
Mark Farnam : Minimizing the Concurrency Footprint of Transactions
byKyle Hailey
 
Performance insights twitch
byKyle Hailey
 
History of database monitoring
byKyle Hailey
 
Oracle Open World Thursday 230 ashmasters
byKyle Hailey
 
Profiling the logwriter and database writer
byKyle Hailey
 
Big data big_skills_data_visualization
byKyle Hailey
 
Martin Klier : Volkswagen for Oracle Guys
byKyle Hailey
 
Successfully convince people with data visualization
byKyle Hailey
 

Recently uploaded

PDF
Louisville User Group: Transforming Technical Debt into Technical Wealth
byLynda Kane
 
PPTX
Design Flaws and Known Attacks in Agentic AI - ITI Business Session
byInformation Technology Institute
 
PDF
Building Software in the AI Era: Spec-Driven Development with Kiro IDE
byHaim Michael
 
PDF
6 Insights from the Patron Saint of AI Failure
byScott M. Graffius
 
PDF
Intro to CrafterQ - AI Agent Platform for the Enterprise.pdf
byCrafterQ
 
PDF
TrustArc Webinar - Unpacking India’s DPDP Act: What You Should Know
byTrustArc
 
PDF
Chapter 2 Computer Threat .pdf
byGetnet Tigabie Askale -(GM)
 
PDF
Master the FME Connector for ArcGIS: Streamlined Data Management & Robust Met...
bySafe Software
 
PDF
Requestly or Postman: What’s the Right API Tool for Your Team?
byhenrycavill30521
 
PDF
SAP WalkMe Overview.pdf SAP WalkMe Overview.pdf
byMurniatiSimbolon2
 
PDF
Best Bank Statement Converter Software - A Documentation-Based Meta-Analysis ...
bylewisconrada
 
PPTX
Building Real-Time Voice AI — Udaiappa Ramachandran
byUdaiappa Ramachandran
 
PPTX
On-Device AI with Gemma 3n and PaliGemma 2 Session Slides - GDG VESIT
bygdgcodecellcmpn
 
PPTX
02_Extended Warehouse Management Functions and Features.pptx
byUdayakumar218983
 
PPTX
Lecture 05. API Security for DevSecOps Eng.pptx
byvinocol222
 
PPTX
Visual Foxpro-VFP9-Access-Report-Variable-Outside-the-report.pptx
bymanojbkalla
 
PPTX
KTU-PEMET413 -MODULE 2-POLYMER MATRIX COMPOSITES - LECTURE 1.pptx
byVINAY B
 
PDF
Effective Ai powered mock interview .pdf
byamazingnishusingh766
 
PDF
Josh Chu Boston - An Innovative Technology Executive
byJosh Chu Boston
 
PPTX
IBM_NEXA_DAC2021 NEXA, a cloud-native platform for collaborative hardware log...
byArun Joseph
 
Louisville User Group: Transforming Technical Debt into Technical Wealth
byLynda Kane
 
Design Flaws and Known Attacks in Agentic AI - ITI Business Session
byInformation Technology Institute
 
Building Software in the AI Era: Spec-Driven Development with Kiro IDE
byHaim Michael
 
6 Insights from the Patron Saint of AI Failure
byScott M. Graffius
 
Intro to CrafterQ - AI Agent Platform for the Enterprise.pdf
byCrafterQ
 
TrustArc Webinar - Unpacking India’s DPDP Act: What You Should Know
byTrustArc
 
Chapter 2 Computer Threat .pdf
byGetnet Tigabie Askale -(GM)
 
Master the FME Connector for ArcGIS: Streamlined Data Management & Robust Met...
bySafe Software
 
Requestly or Postman: What’s the Right API Tool for Your Team?
byhenrycavill30521
 
SAP WalkMe Overview.pdf SAP WalkMe Overview.pdf
byMurniatiSimbolon2
 
Best Bank Statement Converter Software - A Documentation-Based Meta-Analysis ...
bylewisconrada
 
Building Real-Time Voice AI — Udaiappa Ramachandran
byUdaiappa Ramachandran
 
On-Device AI with Gemma 3n and PaliGemma 2 Session Slides - GDG VESIT
bygdgcodecellcmpn
 
02_Extended Warehouse Management Functions and Features.pptx
byUdayakumar218983
 
Lecture 05. API Security for DevSecOps Eng.pptx
byvinocol222
 
Visual Foxpro-VFP9-Access-Report-Variable-Outside-the-report.pptx
bymanojbkalla
 
KTU-PEMET413 -MODULE 2-POLYMER MATRIX COMPOSITES - LECTURE 1.pptx
byVINAY B
 
Effective Ai powered mock interview .pdf
byamazingnishusingh766
 
Josh Chu Boston - An Innovative Technology Executive
byJosh Chu Boston
 
IBM_NEXA_DAC2021 NEXA, a cloud-native platform for collaborative hardware log...
byArun Joseph
 

Agile Data: revolutionizing data and database cloning

  • 1.
    Agile Data : VirtualData Revolution Kyle@delphix.com kylehailey.com slideshare.com/khailey
  • 2.
    The Goal -Theory of Constraints (TOC) Improvement not made at the constraint is an illusion
  • 3.
    Factory floor :straight forward
  • 4.
    Factory floor :straight forward constraint
  • 5.
    Factory floor :optimize at the constraint constraint Tuning here Stock piling
  • 6.
    Factory floor :optimize at the constraint constraint Tuning here Stock piling Tuning here Starvation
  • 7.
    Factory floor :straight forward constraint
  • 8.
  • 9.
    Drum, Rope, Buffer(DBR) 45 75 305080 constraint 25
  • 10.
    For IT doesthe Theory of Constraints work ?
  • 11.
    The Phoenix Project •Constraints Identify • Metrics Define • Priorities Set • Goals Clarify • Iterations Fast • CI continuous integration • Cloud • Agile • Kanban
  • 12.
    The Phoenix Project Whatis the constraint in IT ? “One of the most powerful things that IT can do is get environments to development and QA when they need it” 2x throughput increase with agile data
  • 13.
    In this presentation: • Data Constraint • Solution • Use Cases
  • 14.
    In this presentation: • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
  • 15.
    Data is theconstraint 60% Projects Over Schedule 85% delayed waiting for data Data is the Constraint CIO Magazine Survey: Current situation: only getting worse … Data Doomsday
  • 16.
    In this presentation: • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
  • 17.
    I. Data Constraintstrains IT If you can’t satisfy the business demands then your process is broken.
  • 18.
    I. Data Constraint: moving data is hard – Storage & Systems – Personnel – Time
  • 19.
  • 20.
  • 21.
    Typical Architecture Production Instance Reporting Backup Filesystem Database Instance File system Database File system Database
  • 22.
    Typical Architecture Production Instance File system Database Instance Filesystem Database File system Database File system Database Instance Instance Instance File system Database File system Database Dev, QA, UAT Reporting Backup Triple Tax
  • 23.
    Typical Architecture Production Instance File system Database Instance Filesystem Database File system Database File system Database Instance Instance Instance File system Database File system Database
  • 24.
    In this presentation: • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
  • 25.
    II. Data Constraintprice is huge
  • 26.
    I. Data constraint:Data floods company infrastructure 92% of the cost of business , in financial services business , is “data” www.wsta.org/resources/industry-articles Most companies have 2-9% IT spending http://uclue.com/?xq=1133 Data management is the largest Part of IT expense Gartner: Data Doomsday
  • 27.
    II. Data constraintprice is Huge • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$
  • 28.
    II. Data constraintprice is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
  • 29.
    II. Data constraintprice is huge : 1. IT Capital • Hardware –Servers –Storage –Network –Data center floor space, power, cooling
  • 30.
    II. Data constraintprice is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
  • 31.
    II. Data constraintprice is huge : 2. IT Operations • People – DBAs – SYS Admin – Storage Admin – Backup Admin – Network Admin • Hours : 1000s just for DBAs • $100s Millions for data center modernizations
  • 32.
    II. Data constraintprice is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
  • 33.
    II. Data constraintprice is Huge : 3. App Dev • Inefficient QA: Higher costs of QA • QA Delays : Greater re-work of code • Sharing DB Environments : Bottlenecks • Using DB Subsets: More bugs in Prod • Slow Environment Builds: Delays “if you can't measure it you can’t manage it”
  • 34.
    II. Data Taxis Huge : 3. App Dev Slow Environment Builds Never enough environments
  • 35.
    Part II. Dataconstraint price is Huge • Four Areas data tax hits 1. IT Capital resources 2. IT Operations personnel 3. Application Development 4. Business
  • 36.
    II. Data constraintprice is Huge : 4. Business Ability to capture revenue • Business Intelligence – Old data = less intelligence • Business Applications – Delays cause => Lost Revenue
  • 37.
    II. Data constraintprice is Huge : 4. Business
  • 38.
    II. Data constraintprice is Huge : 4. Business 0 5 10 15 20 25 30 Storage IT Ops Dev Revenue Billion $
  • 39.
    II. Data constraintprice is Huge • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ Review
  • 40.
    In this presentation: • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
  • 41.
    Part III. DataConstraint companies unaware
  • 42.
    III. Data Constraintcompanies unaware DBA Developer
  • 43.
    III. Data Constraintcompanies unaware #1 Biggest Enemy : IT departments believe – best processes – greatest technology – Just the way it is
  • 44.
    There are alwaysnew and better ways to do things
  • 45.
    III. Data Constraintcompanies unaware Why do I need an iPhone ? Don’t we already do that ? SQL scripts Alter database begin backup Back up datafiles Redo Archive Alter database end backup RMAN
  • 46.
    III. Data Constraintcompanies unaware • Ask Questions – me: we provision environments in minutes for almost not extra storage. – Customer: We already do that – me: How long does it take a developer to get an environment after they ask ? – Customer: 2-3 weeks – me: we do it in 2-3 minutes
  • 47.
    III. Data Constraintcompanies unaware How to enlighten? Ask for metrics – How long does it take a developer to get a DB copy? • QA? – How old is data ? • BI and DW : ETL batch windows • QA and Dev : how often refreshed – How much storage used for copies? • How much DBA time?
  • 48.
    In this presentation: • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases
  • 49.
    Clone 1 Clone3Clone 2 99% of blocks are identical
  • 50.
  • 51.
    Clone 1 Clone2 Clone 3 Thin Clone
  • 52.
    Technology Core :file system snapshots • Vmware Linked Clones – Not supported for Oracle • EMC – 16 snapshots – Write performance impact • Netapp – 255 snapshots • ZFS – Unlimited snapshots
  • 53.
  • 54.
    Three Core Parts Production FileSystem Instance DevelopmentStorage 21 3 Copy Sync Snapshots Time Flow Purge Clone (snapshot) Compress Share Cache Storage Mount, recover, rename Self Service, Roles & Security Rollback & Refresh Branch & Tag Instance
  • 55.
  • 56.
    Three Physical Copies ThreeVirtual Copies Data Virtualization Appliance
  • 57.
    Install Delphix onx86 hardware Intel hardware
  • 58.
    Allocate Any Storageto Delphix Allocate Storage Any type Pure Storage + Delphix Better Performance for 1/10 the cost
  • 59.
    One time backupof source database Database Production File systemFile system Upcoming Supports InstanceInstanceInstance Application Stack Data
  • 60.
    DxFS (Delphix) CompressData Database Production Data is compressed typically 1/3 size File system InstanceInstanceInstance
  • 61.
    Incremental forever changecollection Database Production File system Changes • Collected incrementally forever • Old data purged File system Time Window Production InstanceInstanceInstance
  • 62.
    Virtual DB 62 /30 Jonathan Lewis © 2013 Snapshot 1 – full backup once only at link time a b c d e f g h i We start with a full backup - analogous to a level 0 rman backup. Includes the archived redo log files needed for recovery. Run in archivelog mode.
  • 63.
    Virtual DB 63 /30 Jonathan Lewis © 2013 Snapshot 2 (from SCN) b' c' a b c d e f g h i The "backup from SCN" is analogous to a level 1 incremental backup (which includes the relevant archived redo logs). Sensible to enable BCT. Delphix executes standard rman scripts
  • 64.
    Virtual DB 64 /30 Jonathan Lewis © 2013 a b c d e f g h i Apply Snapshot 2 b' c' The Delphix appliance unpacks the rman backup and "overwrites" the initial backup with the changed blocks - but DxFS makes new copies of the blocks
  • 65.
    Virtual DB 65 /30 Jonathan Lewis © 2013 Drop Snapshot 1 b' c'a d e f g h i The call to rman leaves us with a new level 0 backup, waiting for recovery. But we can pick the snapshot root block. We have EVERY level 0 backup
  • 66.
    Virtual DB 66 /30 Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) b' c'a d e f g h i
  • 67.
    Virtual DB 67 /30 Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) i’b' c'a d e f g h ib' c'a d e f g h i
  • 68.
    Before Delphix Production Dev,QA, UAT Instance Reporting Backup File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database “triple data tax”
  • 69.
    With Delphix Production Instance Database Dev &QA Instance Database Reporting Instance Database Backup Instance Instance Instance Database InstanceInstance Database InstanceInstance File system Database
  • 70.
    In this presentation: • Problem in the Industry • Solution • Use Cases
  • 71.
    Use Cases 1. Development 2.QA 3. Recovery 4. Business Intelligence 5. Modernization
  • 72.
    Use Cases 1. Development 2.QA 3. Recovery 4. Business Intelligence 5. Modernization
  • 73.
    Development • Parallelized Environments •Full size environments • Self Service Development
  • 74.
    Development without AgileData: bottlenecks Frustration Waiting Old Unrepresentative Data
  • 75.
    Development without AgileData: subsets of DB
  • 76.
  • 77.
    Development with AgileData: Parallelize Environments gif by Steve Karam
  • 78.
    Development with AgileData: Full size copies
  • 79.
    Development without AgileData: slow env build times Developer Asks for DB Get Access Manager approves DBA Request system Setup DB System Admin Request storage Setup machine Storage Admin Allocate storage (take snapshot) 3-6 Months to Deliver Data
  • 80.
    Development without AgileData: slow env build times Why are hand offs so expensive? 1hour 1 day 9 days
  • 81.
  • 82.
    Development with AgileData: Self Service
  • 83.
    Use Cases 1. Development 2.QA 3. Recovery 4. Business Intelligence 5. Modernization
  • 84.
    QA • Fast • Parallel •Rollback • A/B testing
  • 85.
    QA without AgileData : Long Build times Build Time 96% of QA time was building environment $.04/$1.00 actual testing vs. setup Build Build Time QA Test QA Test Build
  • 86.
    QA without AgileData : slow QA Build QA Env QA Build QA Env QA Sprint 1 Sprint 2 Sprint 3 Bug CodeX 0 10 20 30 40 50 60 70 1 2 3 4 5 6 7 Delay in Fixing the bug Cost To Correct Software Engineering Economics – Barry Boehm (1981)
  • 87.
    QA with AgileData : Fast environments with Branching Instance Instance Instance Source Dev QA branched from Dev Source dev QA
  • 88.
    QA with AgileData: Fast environments with Branching B u i l d T i m e QA Test 1% of QA time was building environment $.99/$1.00 actual testing vs. setup Build Time QA Test Build
  • 89.
    QA with AgileData: bugs found fast Sprint 1 Sprint 2 Sprint 3 Bug CodeX QA QA Build QA Env Q A Build QA Env Q A Sprint 1 Sprint 2 Sprint 3 Bug Cod e X
  • 90.
    QA with AgileData: Parallel environments Instance Instance Instance Instance Source
  • 91.
    QA with AgileData: Rewind for patch and QA testing Instance Instance Development Time Window Prod
  • 92.
    QA with AgileData: A/B testing Instance Instance Instance Index 1 Index 2
  • 93.
    Use Cases 1. Development 2.QA 3. Quality 4. Business Intelligence 5. Modernization
  • 94.
    Quality • Prod &Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics
  • 95.
    Quality : 50days of backup in size of production
  • 96.
    Quality : Surgicalrecovery Instance Instance Development Time Window Before dropDrop Source
  • 97.
    Quality: recovery ofdevelopment Instance Instance Dev1 VDB Time Window Time Window Dev1 VDB Instance Source Source Dev2 VDB Branched
  • 98.
    Quality : recoveryof production Instance Instance VDBSource Time Window Corruption
  • 99.
    Quality : Forensics- Investigate Production Bugs Instance Time Window Instance Development Bug Yesterday Yesterday
  • 100.
    Use Cases 1. Development 2.QA 3. Quality 4. Business Intelligence 5. Modernization
  • 101.
    Business Intelligence • 24x7Batches • Low Bandwidth • Temporal Data • Confidence Testing
  • 102.
    Business Intelligence: ETLand Refresh Windows 1pm 10pm 8am noon
  • 103.
    Business Intelligence: ETLand DW refreshes taking longer 1pm 10pm 8am noon 2011 2012 2013 2014 2015
  • 104.
    Business Intelligence ETLand Refresh Windows 2011 2012 2013 2014 2015 1pm 10pm 8am noon 10pm 8am noon 9pm 6am 8am 10pm
  • 105.
    Business Intelligence: ETLand DW Refreshes Instance Prod Instance DW & BI Data Guard – requires full refresh if used Active Data Guard – read only, most reports don’t work
  • 106.
    Business Intelligence: FastRefreshes • Collect only Changes • Refresh in minutes Instance Instance Prod Instance BI and DW ETL 24x7
  • 107.
  • 108.
    Business Intelligence a) 24x7Batches & Refreshes a) Temporal queries b) Confidence testing
  • 109.
    Use Cases 1. Development 2.QA 3. Quality 4. Business Intelligence 5. Modernization
  • 110.
  • 111.
  • 112.
  • 113.
    “I looked likea hero” Tony Young, CIO Informatica Modernization: Federated
  • 114.
    Data movement requiredfor 1 source DB and 4 clones 5x Source Data Copy < 1 x Source Data Copy S SC C C C V V V V Without Delphix (c = clone) With Delphix (v = virtual DB)
  • 115.
  • 116.
    Dev QA UAT Dev QA UAT 2.6 2.7 Dev QA UAT 2.8 Data Control =Source Control for the Database Production Time Flow Modernization: Auditing & Version Control CIO Insurance 600 Applications CIO Investment Banking 180 Applications CIO South America 65 Applications
  • 117.
    Use Cases 1. Development •Parallelized Environments • Full size environments • Self Service 2. QA • Fast • Parallel • Rollback • A/B testing 3. Recovery • Prod & Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics 4. Business Intelligence • 24x7 Batches • Low Bandwidth • Temporal Data • Confidence Testing 5. Modernization • Federated • Consolidation • Migration • Auditing
  • 118.
    Use Case Summary 1.Development 2. QA 3. Quality 4. Business Intelligence 5. Modernization
  • 119.
    How expensive isthe Data Constraint? Before and after Delphix w/ Fortune 500 : Dev throughput increase by 2x
  • 120.
    How expensive isthe Data Constraint? • 10 x Faster Financial Close – 21 days down to 2 • 9x Faster BI refreshes – 3 weeks per refresh to 3x a week • 2x faster Projects • 20 % less bugs
  • 121.
    Agile Data Quotes •“Allowed us to shrink our project schedule from 12 months to 6 months.” – BA Scott, NYL VP App Dev • "It used to take 50-some-odd days to develop an insurance product, … Now we can get a product to the customer in about 23 days.” – Presbyterian Health • “Can't imagine working without it” – Ramesh Shrinivasan CA Department of General Services
  • 123.
    Summary • Problem: Datais the constraint • Solution: Agile data is small & fast • Results: Deliver projects – Half the Time – Higher Quality – Increase Revenue Kyle@delphix.com kylehailey.com slideshare.net/khailey
  • 124.
  • 125.
  • 126.
  • 127.
    5000 Tnxs/minLatency 300 ms 1 5 1020 30 60 100 200 with 1 5 10 20 30 60 100 200 Users
  • 128.
    8000 Tnxs/minLatency 600 ms 1 5 1020 30 60 100 200 Users 1 5 10 20 30 60 100 200
  • 129.
    $1,000,000 1TB cache onSAN $6,000 200GB shared cache on Delphix Five 200GB database copies are cached with :
  • 130.
    Use Cases 1. Development •Parallelized Environments • Full size environments • Self Service 2. QA • Fast • Parallel • Rollback • A/B testing 3. Recovery • Prod & Dev Backups • Surgical recovery • Recovery of Production • Recovery of Development • Bug Forensics 4. Business Intelligence • 24x7 Batches • Low Bandwidth • Temporal Data • Confidence Testing 5. Modernization • Federated • Consolidation • Migration • Auditing
  • 131.
  • 136.
    Source Full CopySource backup from SCN 1
  • 137.
  • 138.
  • 139.
  • 140.
    Drop Snapshot Snapshot 1 Snapshot2 Snapshot 3 Snapshot 2 Snapshot 3 Drop Snapshot 1