Data Virtualization Platform
revolutionizing database cloning
How can the DBA make the biggest
impact on the company
6/18/...
The Goal
Theory of Constraints
Improvement
not made
at the constraint
is an illusion
factory floor optimization
Factory floor :
straight forward
Factory floor :
straight forward
constraint
Factory floor :
optimize at the constraint
constraint
Tuning here
Stock piling
Factory floor :
optimize at the constraint
constraint
Tuning here
Stock piling
Tuning here
Starvation
Factory floor :
straight forward
constraint
Factory Floor
Optimization
Theory of Constraints
work for IT ?
• Goals Clarify
• Metrics Define
• Constraints Identify
• Priorities Set
• Iterations ...
The Phoenix Project
What is the
constraint
in IT ?
“One of the most powerful things that organizations
can do is to enable...
What is the constraint in IT
If you can’t satisfy
the business demands
then your process is broken.
Data is the constraint
60% Projects Over Schedule
85% delayed waiting for data
Data is the Constraint
CIO Magazine Survey:...
• Data Constraint
I. strains IT
II. price is huge
III. companies unaware
• Solution
• Use Cases
In this presentation :
• Data Constraint
I. strains IT
II. price is huge
III. companies unaware
• Solution
• Use Cases
In this presentation :
– Storage & Systems
– Personnel
– Time
I. Data Constraint :
moving data is hard
Typical Architecture
Production
Instance
File system
Database
Typical Architecture
Production
Instance
Backup
File system
Database
File system
Database
Typical Architecture
Production
Instance
Reporting Backup
File system
Database
Instance
File system
Database
File system
D...
Typical Architecture
Production
Instance
File system
Database
Instance
File system
Database
File system
Database
File syst...
Typical Architecture
Production
Instance
File system
Database
Instance
File system
Database
File system
Database
File syst...
I. Data constraint:
Data floods infrastructure
92% of the cost of business,
in financial services business , is “data”
www...
• Data Constraint
I. strains IT
II. price is huge
III. companies unaware
• Solution
• Use Cases
In this presentation :
• Four Areas data tax hits
1. IT Capital resources $
2. IT Operations personnel $
3. Application Development $$$
4. Busine...
• Four Areas data tax hits
1. IT Capital resources $
2. IT Operations personnel $
3. Application Development $$$
4. Busine...
• Hardware
–Servers
–Storage
–Network
–Data center floor space, power, cooling
II. Data constraint price is huge :
IT Capi...
• Four Areas data tax hits
1. IT Capital resources $
2. IT Operations personnel $
3. Application Development $$$
4. Busine...
• People
– DBAs
– SYS Admin
– Storage Admin
– Backup Admin
– Network Admin
• Hours : 1000s just for DBAs
• $100s Millions ...
• Four Areas data tax hits
1. IT Capital resources $
2. IT Operations personnel $
3. Application Development $$$
4. Busine...
• Inefficient QA: Higher costs of QA
• QA Delays : Greater re-work of code
• Sharing DB Environments : Bottlenecks
• Using...
• Four Areas data tax hits
1. IT Capital resources $
2. IT Operations personnel $
3. Application Development $$$
4. Busine...
Ability to capture revenue
• Business Applications
– Delays cause lost revenue
• Business Intelligence
– Old data = less i...
• Data Constraint
I. strains IT
II. price is huge
III. companies unaware
• Solution
• Use Cases
In this presentation :
Part III. Data Constraint
companies unaware
III. Data Constraint
companies unaware
DeveloperDBA
Metrics
–Time
–Old Data
–Storage
–Analysts
–Audits
III. Data Constraint
companies unaware
• Data Constraint
I. strains IT
II. price is huge
III. companies unaware
• Solution
• Use Cases
In this presentation :
Clone 1 Clone 3Clone 2
99% of blocks are
identical
Solution
Clone 1 Clone 2 Clone 3
Thin Clone
• Vmware Data Director
– Linked clones not supported for Oracle
• EMC
– 16 snapshots on Symmetrix
– Write performance impa...
Fuel not equal car
Challenges
1. Technical
2. Bureaucracy
1. Technical Challenge
Database
Luns
Production Filer
Target A
Target B
Target C
snapshot
clones
InstanceInstance
Instance...
Database
LUNs
snapshot
clonesProduction Filer
Development Filer
1. Technical Challenge
Instance
Target A
Target B
Target C...
1. Technical Challenge
Copy
Time Flow
Purge
Production
File System Instance
DevelopmentStorage
21 3
Clone (snapshot)
Compr...
2. Bureaucracy Challenge
Developer Asks for DB Get Access
Manager approves
DBA Request
system
Setup DB
System
Admin
Reques...
Data Virtualization
How to get a Data Virtualization?
– EMC + SRDF + scripting
– Netapp + SMO + scripting
– Oracle EM 12c ...
Data Supply Chain
6/18/2014 47
Data Supply Chain
• Security
• Masking
• Chain of custody
• Self Service
• Roles
• Restrict...
Install Delphix on x86
hardware
Intel hardware
Application Stack Data
Allocate Any Storage to
Delphix
Allocate Storage
Any type
Pure Storage + Delphix
Better Performance for
1/10 the cost
One time backup of
source database
Database
Production
File systemFile system
InstanceInstanceInstance
DxFS (Delphix) Compress
Data
Database
Production
Data is
compressed
typically 1/3
size
File system
InstanceInstanceInstance
Incremental forever
change collection
Database
Production
File system
Changes
• Collected incrementally forever
• Old data...
Virtual DB
53 / 30Jonathan Lewis
© 2013
Snapshot 1 – full backup
once only at link time
a b c d e f g h i
We start with a ...
Virtual DB
54 / 30Jonathan Lewis
© 2013
Snapshot 2 (from SCN)
b' c'
a b c d e f g h i
The "backup from SCN" is analogous t...
Virtual DB
55 / 30Jonathan Lewis
© 2013
a b c d e f g h i
Apply Snapshot 2
b' c'
The Delphix appliance unpacks the rman ba...
Virtual DB
56 / 30Jonathan Lewis
© 2013
Drop Snapshot 1
b' c'a d e f g h i
The call to rman leaves us with a new level 0 b...
Virtual DB
57 / 30Jonathan Lewis
© 2013
Creating a vDB
b' c'a d e f g h i
The first step in creating a vDB is to take a sn...
Virtual DB
58 / 30Jonathan Lewis
© 2013
Creating a vDB
b' c'a d e f g h i
The first step in creating a vDB is to take a sn...
Database Virtualization
Three Physical Copies
Three Virtual Copies
Data
Virtualization
Appliance
Before Virtual Data
Production Dev, QA, UAT
Instance
Reporting Backup
File system
Database
Instance
File system
Database
F...
With Virtual Data
Production
Instance
Database
Dev & QA
Instance
Database
Reporting
Instance
Database
Backup
Instance Inst...
• Problem in the Industry
• Solution
• Use Cases
In this presentation :
1. Development and QA
2. Recovery
3. Business
Use Cases
1. Development and QA
2. Recovery
3. Business
Use Cases
Development : bottlenecks
Frustration Waiting
Old Unrepresentative Data
Development : subsets
Development : bugs
Development : slow
Developer Asks for DB Get Access
Manager approves
DBA Request
system
Setup DB
System
Admin
Request
stor...
Development without Virtual Data: slow env build
times
Why are hand offs so expensive?
1hour
1 day
9 days
Development :
Lack of resources
Never enough environments
http://martinfowler.com/bliki/NoDBA.html
Development without Virtual Data: slow env build
times
Development: Virtual Data
• Unlimited
• Full size
• Self Service
Development:
Virtual Data
Development
Virtual Data:
Easy
Instance
Instance
Instance
Instance
Source
Data
Virtualization
Appliance
DVA
Parallel
Environments
• QA...
Development Virtual Data:
Parallelize
gif by Steve Karam
Development Virtual
Data: Full size
Development Virtual
Data: Self Service
Dev2
Dev1Merge to dev1
Trunk
DB
VC
Fork
Fork
Fork
Fork
DBmaestro
QA : Virtual Data
• Fast
• Parallel
• Rollback
• A/B testing
QA
Virtual Data
QA : Long Build times
96% of QA time was building environment
$.04/$1.00 actual testing vs. setup
QA Build QA
QA Build QA
...
QA : slow
Bug CodeX
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7
Delay in Fixing the bug
Cost
To
Correct
Software Engineering Econ...
QA Virtual Data : Fast
83
Dev
QA
Instance
Prod
DVA
Time Flow
1% of QA time was building environment
$.99/$1.00 actual test...
QA Virtual Data : Fast
Dev
QA
Instance
Prod
DVA
Time Flow
Bugs found quickly QA
Build QA
Sprint 1 Sprint 2
QA
Build QA
X
Q...
QA with Virtual Data:
Rewind
Instance
Instance
Development
Prod
QA with Virtual Data: A/B
Instance
Instance
Instance
Index 1
Index 2
Data Version Control
6/18/2014 87
Dev
QA
2.1
Dev
QA
2.2
2.1 2.2
Instance
Prod
DVA
1. Development and QA
2. Recovery
3. Business
Use Cases
• Backups
• Recovery
• Forensics
Recovery
Recovery: Backups
Recovery: scenarios
Instance
Instance
Recover VDB
Drop
Source
DVA
Recovery: Forensics
Instance
Instance
DevelopmentDVA
Source
Recovery: Development
Instance
Instance
Development
DVA
Source
Instance
Development
1. Development and QA
2. Recovery
3. Business Intelligence
Use Cases
Business Intelligence
Business Intelligence:
ETL and Refresh Windows
1pm 10pm 8am
noon
Business Intelligence:
batch taking too long
1pm 10pm 8am
noon
2011
2012
2013
2014
2015
2011
2012
2013
2014
2015
1pm 10pm 8am
noon
10pm 8am noon 9pm
6am 8am 10pm
Business Intelligence:
batch taking too long
Business Intelligence:
ETL and DW Refreshes
Instance
Prod
Instance
DW & BI
Before Virtual: limited, slow ETL and DW refres...
• Collect only Changes
• Refresh in minutes
Virtual Data:
Fast Refreshes
Instance Instance
Prod BI and DW
ETL
24x7
DVA Ins...
Temporal Data
Confidence testing
1. Federated
2. Migration
3. Auditing
Modernization
Modernization:
Federated
Instance
Instance
Instance
Instance
Source1
Source2
DVA
Modernization:
Federated
“I looked like a hero”
Tony Young, CIO Informatica
Modernization:
Federated
Modernization:
Migration
Audit
6/18/2014 108
Instance
Prod
DVA
Live Archive
Consolidation
1. Development & QA
2. Recovery
3. Business
Use Case Summary
How expensive is the
Data Constraint?
DVA at Fortune 500 :
Dev throughput increase by 2x
• 10 x Faster Financial Close
• 9x Faster BI refreshes
• 8x Faster surgical recovery
• 3x Project tracks
• 2x Faster Proje...
• Projects “12 months to 6 months.”
– New York Life
• Insurance product “about 50 days ... to about 23 days”
– Presbyteria...
• Problem: Data is the constraint
• Solution: Virtual Data
• Results:
– Half the time for projects
– Higher quality
– Incr...
Thank you!
• Kyle Hailey| Oracle ACE and Technical
Evangelist, Delphix
– Kyle@delphix.com
– kylehailey.com
– slideshare.ne...
Oracle 12c
80MB buffer cache ?
200GB
Cache
5000
Tnxs/minLatency
300
ms
1 5 10 20 30 60 100 200
with
1 5 10 20 30 60 100 200
Users
8000
Tnxs/minLatency
600
ms
1 5 10 20 30 60 100 200
Users
1 5 10 20 30 60 100 200
$1,000,000
1TB cache on SAN
$6,000
200GB shared cache on Delphix
Five 200GB database copies are
cached with :
6/18/2014 123
6/18/2014 124
Business Intelligence
a) 24x7 Batches
b) Temporal queries
c) Confidence testing
Thin Cloning
Snap
Manager
SnapManager
Repository
Protection
Manager
Snap Drive
Snap
Manager
Snap Mirror
Flex Clone
RMAN
Repository
Prod...
NetApp Filer - DevelopmentNetApp Filer - Production
Database
Luns
Snap
mirror
Snapshot Manager
for Oracle
Flexclone
Reposi...
Where we want to be
Database
File system
Production
Instance
Database
Development
Instance
Database
QA
Instance
Database
U...
EM 12c: Snap Clone
Production Development
Flexclone Flexclone
Netapp
Snap Manager for Oracle
II. Data constraint price is Huge : 4. Business
II. Data constraint price is
Huge : 4. Business
0 5 10 15 20 25 30
Storage
IT Ops
Dev
Revenue
Billion $
III. Data Constraint
companies unaware
#1 Biggest Enemy :
IT departments believe
– best processes
– greatest technology
– ...
There are always new
and better ways to do
things
III. Data Constraint
companies unaware
Why do I need an iPhone ?
Don’t we already do that ?
SQL scripts
Alter database beg...
Data Virtualization: revolutionizing database cloning
Data Virtualization: revolutionizing database cloning
Data Virtualization: revolutionizing database cloning
Data Virtualization: revolutionizing database cloning
Upcoming SlideShare
Loading in …5
×

Data Virtualization: revolutionizing database cloning

1,255 views

Published on

Published in: Technology, Business
  • Be the first to comment

Data Virtualization: revolutionizing database cloning

  1. 1. Data Virtualization Platform revolutionizing database cloning How can the DBA make the biggest impact on the company 6/18/2014 1 http://kylehailey.com kyle@delphix.com
  2. 2. The Goal Theory of Constraints Improvement not made at the constraint is an illusion factory floor optimization
  3. 3. Factory floor : straight forward
  4. 4. Factory floor : straight forward constraint
  5. 5. Factory floor : optimize at the constraint constraint Tuning here Stock piling
  6. 6. Factory floor : optimize at the constraint constraint Tuning here Stock piling Tuning here Starvation
  7. 7. Factory floor : straight forward constraint
  8. 8. Factory Floor Optimization
  9. 9. Theory of Constraints work for IT ? • Goals Clarify • Metrics Define • Constraints Identify • Priorities Set • Iterations Fast • CI • Cloud • Agile • Kanban “IT is the factory floor of this century”
  10. 10. The Phoenix Project What is the constraint in IT ? “One of the most powerful things that organizations can do is to enable development and testing to get environment they need when they need it“
  11. 11. What is the constraint in IT If you can’t satisfy the business demands then your process is broken.
  12. 12. Data is the constraint 60% Projects Over Schedule 85% delayed waiting for data Data is the Constraint CIO Magazine Survey: only getting worse
  13. 13. • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases In this presentation :
  14. 14. • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases In this presentation :
  15. 15. – Storage & Systems – Personnel – Time I. Data Constraint : moving data is hard
  16. 16. Typical Architecture Production Instance File system Database
  17. 17. Typical Architecture Production Instance Backup File system Database File system Database
  18. 18. Typical Architecture Production Instance Reporting Backup File system Database Instance File system Database File system Database
  19. 19. Typical Architecture Production Instance File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database Dev, QA, UAT Reporting Backup Triple Tax
  20. 20. Typical Architecture Production Instance File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database
  21. 21. I. Data constraint: Data floods infrastructure 92% of the cost of business, in financial services business , is “data” www.wsta.org/resources/industry-articles Most companies have 2-9% IT spending , ½ on “data” http://uclue.com/?xq=1133 Gartner: Data Doomsday
  22. 22. • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases In this presentation :
  23. 23. • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ II. Data constraint: price is Huge
  24. 24. • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ II. Data constraint: price is Huge
  25. 25. • Hardware –Servers –Storage –Network –Data center floor space, power, cooling II. Data constraint price is huge : IT Capital
  26. 26. • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ II. Data constraint: price is Huge
  27. 27. • People – DBAs – SYS Admin – Storage Admin – Backup Admin – Network Admin • Hours : 1000s just for DBAs • $100s Millions for data center modernizations II. Data constraint price is huge: IT Operations
  28. 28. • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ II. Data constraint: price is Huge “One of the most powerful things that IT can do is get environments to development and QA when they need it” - Gene Kim author of The Phoenix Project
  29. 29. • Inefficient QA: Higher costs of QA • QA Delays : Greater re-work of code • Sharing DB Environments : Bottlenecks • Using DB Subsets: More bugs in Prod • Slow Environment Builds: Delays II. Data constraint price is Huge : Application Development
  30. 30. • Four Areas data tax hits 1. IT Capital resources $ 2. IT Operations personnel $ 3. Application Development $$$ 4. Business $$$$$$$ Part II. Data constraint: price is Huge
  31. 31. Ability to capture revenue • Business Applications – Delays cause lost revenue • Business Intelligence – Old data = less intelligence II. Data constraint price is Huge : Business
  32. 32. • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases In this presentation :
  33. 33. Part III. Data Constraint companies unaware
  34. 34. III. Data Constraint companies unaware DeveloperDBA
  35. 35. Metrics –Time –Old Data –Storage –Analysts –Audits III. Data Constraint companies unaware
  36. 36. • Data Constraint I. strains IT II. price is huge III. companies unaware • Solution • Use Cases In this presentation :
  37. 37. Clone 1 Clone 3Clone 2 99% of blocks are identical
  38. 38. Solution
  39. 39. Clone 1 Clone 2 Clone 3 Thin Clone
  40. 40. • Vmware Data Director – Linked clones not supported for Oracle • EMC – 16 snapshots on Symmetrix – Write performance impact – No snapshots of snapshots • Netapp – 255 snapshots • ZFS – Compression – Unlimited snapshots – Snapshots of Snapshots • DxFS – Compression – Unlimited snapshots – Snapshots of Snapshots – Storage agnostic – Shared cache in memory Technology Core : file system snapshots Also check out new SSD storage such as: Pure Storage, EMC XtremIO
  41. 41. Fuel not equal car Challenges 1. Technical 2. Bureaucracy
  42. 42. 1. Technical Challenge Database Luns Production Filer Target A Target B Target C snapshot clones InstanceInstance InstanceInstance InstanceInstance InstanceInstance Instance Source
  43. 43. Database LUNs snapshot clonesProduction Filer Development Filer 1. Technical Challenge Instance Target A Target B Target C InstanceInstance InstanceInstance InstanceInstance Instance
  44. 44. 1. Technical Challenge Copy Time Flow Purge Production File System Instance DevelopmentStorage 21 3 Clone (snapshot) Compress Share Cache Provision Mount, recover, rename Self Service, Roles & Security Instance
  45. 45. 2. Bureaucracy Challenge Developer Asks for DB Get Access Manager approves DBA Request system Setup DB System Admin Request storage Setup machine Storage Admin Allocate storage (take snapshot)
  46. 46. Data Virtualization How to get a Data Virtualization? – EMC + SRDF + scripting – Netapp + SMO + scripting – Oracle EM 12c DBaaS + data guard + Netapp /ZFS + scripting – Delphix 2 31 Production DevelopmentStorage 21 3 2 31 23 1 2 31
  47. 47. Data Supply Chain 6/18/2014 47 Data Supply Chain • Security • Masking • Chain of custody • Self Service • Roles • Restrictions • Developer • Data Versioning • Refresh, Rollback • Audit: • Live Archive Snap Shots Thin Cloning Data Virtualization Data Supply Chain
  48. 48. Install Delphix on x86 hardware Intel hardware Application Stack Data
  49. 49. Allocate Any Storage to Delphix Allocate Storage Any type Pure Storage + Delphix Better Performance for 1/10 the cost
  50. 50. One time backup of source database Database Production File systemFile system InstanceInstanceInstance
  51. 51. DxFS (Delphix) Compress Data Database Production Data is compressed typically 1/3 size File system InstanceInstanceInstance
  52. 52. Incremental forever change collection Database Production File system Changes • Collected incrementally forever • Old data purged File system Time Window Production InstanceInstanceInstance
  53. 53. Virtual DB 53 / 30Jonathan Lewis © 2013 Snapshot 1 – full backup once only at link time a b c d e f g h i We start with a full backup - analogous to a level 0 rman backup. Includes the archived redo log files needed for recovery. Run in archivelog mode.
  54. 54. Virtual DB 54 / 30Jonathan Lewis © 2013 Snapshot 2 (from SCN) b' c' a b c d e f g h i The "backup from SCN" is analogous to a level 1 incremental backup (which includes the relevant archived redo logs). Sensible to enable BCT. Delphix executes standard rman scripts
  55. 55. Virtual DB 55 / 30Jonathan Lewis © 2013 a b c d e f g h i Apply Snapshot 2 b' c' The Delphix appliance unpacks the rman backup and "overwrites" the initial backup with the changed blocks - but DxFS makes new copies of the blocks
  56. 56. Virtual DB 56 / 30Jonathan Lewis © 2013 Drop Snapshot 1 b' c'a d e f g h i The call to rman leaves us with a new level 0 backup, waiting for recovery. But we can pick the snapshot root block. We have EVERY level 0 backup
  57. 57. Virtual DB 57 / 30Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) b' c'a d e f g h i
  58. 58. Virtual DB 58 / 30Jonathan Lewis © 2013 Creating a vDB b' c'a d e f g h i The first step in creating a vDB is to take a snapshot of the filesystem as at the backup you want (then roll it forward) My vDB (filesystem) Your vDB (filesystem) i’b' c'a d e f g h ib' c'a d e f g h i
  59. 59. Database Virtualization
  60. 60. Three Physical Copies Three Virtual Copies Data Virtualization Appliance
  61. 61. Before Virtual Data Production Dev, QA, UAT Instance Reporting Backup File system Database Instance File system Database File system Database File system Database Instance Instance Instance File system Database File system Database “triple data tax”
  62. 62. With Virtual Data Production Instance Database Dev & QA Instance Database Reporting Instance Database Backup Instance Instance Instance Database InstanceInstance Database InstanceInstance File system Database Data Virtualization Appliance
  63. 63. • Problem in the Industry • Solution • Use Cases In this presentation :
  64. 64. 1. Development and QA 2. Recovery 3. Business Use Cases
  65. 65. 1. Development and QA 2. Recovery 3. Business Use Cases
  66. 66. Development : bottlenecks Frustration Waiting Old Unrepresentative Data
  67. 67. Development : subsets
  68. 68. Development : bugs
  69. 69. Development : slow Developer Asks for DB Get Access Manager approves DBA Request system Setup DB System Admin Request storage Setup machine Storage Admin Allocate storage (take snapshot) Weeks to Months to Deliver Data
  70. 70. Development without Virtual Data: slow env build times Why are hand offs so expensive? 1hour 1 day 9 days
  71. 71. Development : Lack of resources Never enough environments
  72. 72. http://martinfowler.com/bliki/NoDBA.html Development without Virtual Data: slow env build times
  73. 73. Development: Virtual Data • Unlimited • Full size • Self Service Development: Virtual Data Development
  74. 74. Virtual Data: Easy Instance Instance Instance Instance Source Data Virtualization Appliance DVA Parallel Environments • QA • Dev
  75. 75. Development Virtual Data: Parallelize gif by Steve Karam
  76. 76. Development Virtual Data: Full size
  77. 77. Development Virtual Data: Self Service
  78. 78. Dev2 Dev1Merge to dev1 Trunk DB VC Fork Fork Fork Fork DBmaestro
  79. 79. QA : Virtual Data • Fast • Parallel • Rollback • A/B testing QA Virtual Data
  80. 80. QA : Long Build times 96% of QA time was building environment $.04/$1.00 actual testing vs. setup QA Build QA QA Build QA QA before virtual : resource expensive
  81. 81. QA : slow Bug CodeX 0 10 20 30 40 50 60 70 1 2 3 4 5 6 7 Delay in Fixing the bug Cost To Correct Software Engineering Economics – Barry Boehm (1981) Sprint 1 Sprint 2 QA Build QA QA Build QA Sprint 1 Sprint 2 QA before virtual : slow
  82. 82. QA Virtual Data : Fast 83 Dev QA Instance Prod DVA Time Flow 1% of QA time was building environment $.99/$1.00 actual testing vs. setup QA Build QA QA Build QA QA Virtual Data : Fast
  83. 83. QA Virtual Data : Fast Dev QA Instance Prod DVA Time Flow Bugs found quickly QA Build QA Sprint 1 Sprint 2 QA Build QA X QA Virtual Data : Fast with Branching
  84. 84. QA with Virtual Data: Rewind Instance Instance Development Prod
  85. 85. QA with Virtual Data: A/B Instance Instance Instance Index 1 Index 2
  86. 86. Data Version Control 6/18/2014 87 Dev QA 2.1 Dev QA 2.2 2.1 2.2 Instance Prod DVA
  87. 87. 1. Development and QA 2. Recovery 3. Business Use Cases
  88. 88. • Backups • Recovery • Forensics Recovery
  89. 89. Recovery: Backups
  90. 90. Recovery: scenarios Instance Instance Recover VDB Drop Source DVA
  91. 91. Recovery: Forensics Instance Instance DevelopmentDVA Source
  92. 92. Recovery: Development Instance Instance Development DVA Source Instance Development
  93. 93. 1. Development and QA 2. Recovery 3. Business Intelligence Use Cases
  94. 94. Business Intelligence
  95. 95. Business Intelligence: ETL and Refresh Windows 1pm 10pm 8am noon
  96. 96. Business Intelligence: batch taking too long 1pm 10pm 8am noon 2011 2012 2013 2014 2015
  97. 97. 2011 2012 2013 2014 2015 1pm 10pm 8am noon 10pm 8am noon 9pm 6am 8am 10pm Business Intelligence: batch taking too long
  98. 98. Business Intelligence: ETL and DW Refreshes Instance Prod Instance DW & BI Before Virtual: limited, slow ETL and DW refreshes
  99. 99. • Collect only Changes • Refresh in minutes Virtual Data: Fast Refreshes Instance Instance Prod BI and DW ETL 24x7 DVA Instance Virtual Data: Fast Refreshes
  100. 100. Temporal Data
  101. 101. Confidence testing
  102. 102. 1. Federated 2. Migration 3. Auditing Modernization
  103. 103. Modernization: Federated Instance Instance Instance Instance Source1 Source2 DVA
  104. 104. Modernization: Federated
  105. 105. “I looked like a hero” Tony Young, CIO Informatica Modernization: Federated
  106. 106. Modernization: Migration
  107. 107. Audit 6/18/2014 108 Instance Prod DVA Live Archive
  108. 108. Consolidation
  109. 109. 1. Development & QA 2. Recovery 3. Business Use Case Summary
  110. 110. How expensive is the Data Constraint? DVA at Fortune 500 : Dev throughput increase by 2x
  111. 111. • 10 x Faster Financial Close • 9x Faster BI refreshes • 8x Faster surgical recovery • 3x Project tracks • 2x Faster Projects How expensive is the Data Constraint?
  112. 112. • Projects “12 months to 6 months.” – New York Life • Insurance product “about 50 days ... to about 23 days” – Presbyterian Health • “Can't imagine working without it” – State of California Virtual Data Quotes
  113. 113. • Problem: Data is the constraint • Solution: Virtual Data • Results: – Half the time for projects – Higher quality – Increase revenue Summary
  114. 114. Thank you! • Kyle Hailey| Oracle ACE and Technical Evangelist, Delphix – Kyle@delphix.com – kylehailey.com – slideshare.net/khailey
  115. 115. Oracle 12c
  116. 116. 80MB buffer cache ?
  117. 117. 200GB Cache
  118. 118. 5000 Tnxs/minLatency 300 ms 1 5 10 20 30 60 100 200 with 1 5 10 20 30 60 100 200 Users
  119. 119. 8000 Tnxs/minLatency 600 ms 1 5 10 20 30 60 100 200 Users 1 5 10 20 30 60 100 200
  120. 120. $1,000,000 1TB cache on SAN $6,000 200GB shared cache on Delphix Five 200GB database copies are cached with :
  121. 121. 6/18/2014 123
  122. 122. 6/18/2014 124
  123. 123. Business Intelligence a) 24x7 Batches b) Temporal queries c) Confidence testing
  124. 124. Thin Cloning
  125. 125. Snap Manager SnapManager Repository Protection Manager Snap Drive Snap Manager Snap Mirror Flex Clone RMAN Repository Production Development DBA Storage Admin 1 tr-3761.pdf Netapp
  126. 126. NetApp Filer - DevelopmentNetApp Filer - Production Database Luns Snap mirror Snapshot Manager for Oracle Flexclone Repository Database Snap Drive Protection Manage Production Development 1 Netapp Target A Target B Target C InstanceInstance InstanceInstance InstanceInstance Instance
  127. 127. Where we want to be Database File system Production Instance Database Development Instance Database QA Instance Database UAT Instance Snapshots Instance Instance Instance Instance
  128. 128. EM 12c: Snap Clone Production Development Flexclone Flexclone Netapp Snap Manager for Oracle
  129. 129. II. Data constraint price is Huge : 4. Business
  130. 130. II. Data constraint price is Huge : 4. Business 0 5 10 15 20 25 30 Storage IT Ops Dev Revenue Billion $
  131. 131. III. Data Constraint companies unaware #1 Biggest Enemy : IT departments believe – best processes – greatest technology – Just the way it is
  132. 132. There are always new and better ways to do things
  133. 133. III. Data Constraint companies unaware Why do I need an iPhone ? Don’t we already do that ? SQL scripts Alter database begin backup Back up datafiles Redo Archive Alter database end backup RMAN

×