Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Data chat with Kyle Hailey and Tim Gorman
Similar to Data chat with Kyle Hailey and Tim Gorman (20)
Recently uploaded
Recently uploaded (20)
Data chat with Kyle Hailey and Tim Gorman
- 1. Data chat with Kyle Hailey and Tim Gorman © 2014 Delphix. All Rights Reserved Private and confidential
- 2. Virtualized storage is fast becoming the new norm. Nobody can justify provisioning non-production environments the way they did up to now. © 2014 Delphix. All Rights Reserved Private and confidential
- 3. The problem • PROD is what we care most about… PROD © 2014 Delphix. All Rights Reserved Private and confidential 3
- 4. The problem • PROD is what we care most about… – But behind every PROD… • There is QATEST… PROD QATEST © 2014 Delphix. All Rights Reserved Private and confidential 4
- 5. The problem • PROD is what we care most about… – But behind every PROD… • There is QATEST, DEV… PROD QATEST DEV © 2014 Delphix. All Rights Reserved Private and confidential 5
- 6. The problem • PROD is what we care most about… – But behind every PROD… • There is QATEST, DEV… • …and other project-specific environments to support PROD QATEST DEV © 2014 Delphix. All Rights Reserved Private and confidential 6
- 7. The problem • Also, as PROD grows… PROD PROD © 2014 Delphix. All Rights Reserved Private and confidential 7
- 8. The problem – Then so do the supporting non-PROD environments… QATEST DEV • Also, as PROD grows… PROD © 2014 Delphix. All Rights Reserved Private and confidential 8
- 9. The problem • And so inevitably the situation shifts from this… CEO COO CFO CMO CSO CIO © 2014 Delphix. All Rights Reserved Private and confidential 9
- 10. The problem Storage Administrator CEO COO • To this… CFO CMO CSO CIO © 2014 Delphix. All Rights Reserved Private and confidential 10
- 11. Attempting to solve the problem… • Some try to mitigate the problem by copying subsets of PROD… PROD QATEST DEV UAT © 2014 Delphix. All Rights Reserved Private and confidential 11
- 12. Attempting to solve the problem… • Some try to mitigate the problem by copying subsets of PROD… PROD QATEST DEV UAT • …but it is difficult and time-consuming to create referentially-correct subsets of databases. Most give up… © 2014 Delphix. All Rights Reserved Private and confidential 12
- 13. Attempting to solve the problem… • What inevitably happens is that cloning is performed less frequently… PROD QATEST DEV UAT • …meaning that schemas and data are older, dirtier due to previous testing, and too small to accurately represent reality © 2014 Delphix. All Rights Reserved Private and confidential 13
- 14. Attempting to solve the problem… PROD DEV for Tom DEV for Dick DEV for Sally DEV for Harry DEV for Becky And while in a perfect world what we’d like to see is something like this… © 2014 Delphix. All Rights Reserved Private and confidential 14
- 15. Attempting to solve the problem… PROD DEV …in the real world we see this… Tom Dick Sally Harry Beck y © 2014 Delphix. All Rights Reserved Private and confidential 15
- 16. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously © 2014 Delphix. All Rights Reserved Private and confidential 16
- 17. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another © 2014 Delphix. All Rights Reserved Private and confidential 17
- 18. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another • So it becomes necessary to serialize access to the limited number of environments © 2014 Delphix. All Rights Reserved Private and confidential 18
- 19. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another • So it becomes necessary to serialize access to the limited number of environments – Each user queues at the resource, so their project takes more time elapsed © 2014 Delphix. All Rights Reserved Private and confidential 19
- 20. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another • So it becomes necessary to serialize access to the limited number of environments – Each user queues at the resource, so their project takes more time elapsed – Each user destructively changes the resource © 2014 Delphix. All Rights Reserved Private and confidential 20
- 21. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another • So it becomes necessary to serialize access to the limited number of environments – Each user queues at the resource, so their project takes more time elapsed – Each user destructively changes the resource • And now we need a way to reset or clean up the environment between each use © 2014 Delphix. All Rights Reserved Private and confidential 21
- 22. Attempting to solve the problem… • It’s not easy for developers or testers to share environments simultaneously – Changes made by each user conflict with one another • So it becomes necessary to serialize access to the limited number of environments – Each user queues at the resource, so their project takes more time elapsed – Each user destructively changes the resource • And now we need a way to reset or clean up the environment between each use – Reset or clean up takes time, which causes further delays between uses © 2014 Delphix. All Rights Reserved Private and confidential 22
- 23. All kidding aside… You are here © 2014 Delphix. All Rights Reserved Private and confidential 23
- 24. A solution… • The biggest problem with cloning environments and databases is all the duplication © 2014 Delphix. All Rights Reserved Private and confidential 24
- 25. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same © 2014 Delphix. All Rights Reserved Private and confidential 25
- 26. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? © 2014 Delphix. All Rights Reserved Private and confidential 26
- 27. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? – Copy-on-write (COW) • Never modify a block, but insert a new version instead © 2014 Delphix. All Rights Reserved Private and confidential 27
- 28. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? – Copy-on-write (COW) • Never modify a block, but insert a new version instead • Then let the storage figure out what we’re supposed to be seeing? © 2014 Delphix. All Rights Reserved Private and confidential 28
- 29. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? – Copy-on-write (COW) • Never modify a block, but insert a new version instead • Then let the storage figure out what we’re supposed to be seeing? – The storage maintains the relationships and versions on retrieval © 2014 Delphix. All Rights Reserved Private and confidential 29
- 30. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? – Copy-on-write (COW) • Never modify a block, but insert a new version instead • Then let the storage figure out what we’re supposed to be seeing? – The storage maintains the relationships and versions on retrieval • And then make it portable so that ANY platform (Unix/Linux, Windows) can use it? © 2014 Delphix. All Rights Reserved Private and confidential 30
- 31. A solution… • The biggest problem with cloning environments and databases is all the duplication – Most (if not all) of the blocks comprising the database or file-system exactly the same • What if we store only one copy of each version of a block? – Copy-on-write (COW) • Never modify a block, but insert a new version instead • Then let the storage figure out what we’re supposed to be seeing? – The storage maintains the relationships and versions on retrieval • And then make it portable so that ANY platform (Unix/Linux, Windows) can use it? – NFS (network file system) © 2014 Delphix. All Rights Reserved Private and confidential 31
- 32. A solution… • Cloning the old way… PROD DEV © 2014 Delphix. All Rights Reserved Private and confidential 32
- 33. What happens when ask for a copy ? Boss, Storage Admin, DBA Developer or Analyst © 2014 Delphix. All Rights Reserved Private and confidential 33
- 34. A solution… • Cloning the smart way… PROD DEV © 2014 Delphix. All Rights Reserved Private and confidential 34
- 35. A solution… • Cloning the smart way… – As changes are made to the clone, only those changed blocks are stored… PROD DEV © 2014 Delphix. All Rights Reserved Private and confidential 35
- 36. A solution… • Cloning the smart way… – As changes are made to the clone, only those changed blocks are stored… PROD DEV • Takes less storage, faster to copy • The cost of the clone has gone way down in terms of storage and time © 2014 Delphix. All Rights Reserved Private and confidential 36
- 37. A solution… • Cloning the smart way… – As changes are made to the clone, only those changed blocks are stored… PROD DEV • Takes less storage, faster to copy • The cost of the clone has gone way down in terms of storage and time CapEx © 2014 Delphix. All Rights Reserved Private and confidential 37
- 38. A solution… • Cloning the smart way… – As changes are made to the clone, only those changed blocks are stored… PROD DEV • Takes less storage, faster to copy • The cost of the clone has gone way down in terms of storage and time CapEx OpEx © 2014 Delphix. All Rights Reserved Private and confidential 38
- 39. Key Terms Terms for Production Environment Terms for the Delphix Appliance Terms for Pre-Production Environment Source host Delphix Server Target host Source database dSource VDB © 2014 Delphix. All Rights Reserved Private and confidential 39
- 40. Key Terms: Source Host Production Development The source host is the host on which the instance for the source database runs. Typically this is the production host. © 2014 Delphix. All Rights Reserved Private and confidential 40
- 41. Key Terms: Source Database Production Development 1 TB Storage The source database is copied into the Delphix Server via standard Oracle RMAN APIs. Delphix Server then stays in sync via incremental RMAN backups and by collecting redo. © 2014 Delphix. All Rights Reserved Private and confidential 41
- 42. Key Terms: Delphix Server Production Development 1 TB Storage The Delphix Server acts as a backup destination for the source databases, and as network attached storage for VDBs. It also hosts the Delphix management GUI. © 2014 Delphix. All Rights Reserved Private and confidential 42
- 43. Key Terms: dSource Production Development 1 TB Storage ~256 GB Storage for dSource The dSource is the Delphix Server’s compressed representation of the source database. It is the basis for the storage that is allocated to a VDB. © 2014 Delphix. All Rights Reserved Private and confidential 43
- 44. Key Terms: Target Host Production Development 1 TB Storage ~256 GB Storage for dSource The target host is the host on which the instance for the VDB runs. Typically this is wherever the pre-prod database runs today. © 2014 Delphix. All Rights Reserved Private and confidential 44
- 45. Key Terms: VDB Production Development 1 TB Storage ~256 GB Storage for dSource No additional storage for VDB outside of Delphix Only changes stored for VDB The VDB is a fully functional database running on a target host. The storage for a VDB is managed by Delphix, and provided to the target host via NFS. Changes are stored in the Delphix Server. © 2014 Delphix. All Rights Reserved Private and confidential 45
- 46. Delphix Overview Production Development 1 TB Storage ~256 GB Storage for dSource No additional storage for VDB outside of Delphix Only changes stored for VDB © 2014 Delphix. All Rights Reserved Private and confidential 46
- 47. Delphix Server 4.1 • Platforms Vmware Amazon EC2 • Databases Oracle SQL Server Postgres Sybase • App Data • Data Masking • Replication
- 48. How can Delphix help DBAs? • Scratch environments Testing one-off patches, patchset updates (PSU’s), and critical-patch upgrades (CPU’s) • SQL tuning When tuning a specific SQL statement, how can you effectively test the impact of… - adding or dropping an index? - gathering CBO statistics a bit differently? …without getting an act of Congress/Parliament?
- 49. Virtualized storage is fast becoming the new norm. Nobody can justify provisioning non-production environments the way they did up to now. © 2014 Delphix. All Rights Reserved Private and confidential
Editor's Notes
- Some bio, 30 years in IT, 25 years in Oracle, 20 years as a performance-tuning DBA and as on-off Apps SysAdmin, books, ACED, OakTable, joined Delphix last May. Why?
- DevOps. Increasing the tempo of application development. Supporting agile development methods. Shorter delivery cycles and continuous delivery. It isn’t just people working harder, it needs some magic.
- Between a rock and a hard place
- Due to the constraints of building clone copy database environments one ends up in the “culture of no” Where developers stop asking for a copy of a production database because the answer is “no” If the developers need to debug an anomaly seen on production or if they need to write a custom module which requires a copy of production they know not to even ask and just give up.
- Before we proceed, there are a few terms that will come up frequently that we should define. Within our production environment, we will frequently refer to the “source host” or “source database.” We will also refer frequently to the Delphix Server, and to something called a dSource. Finally in our pre-production environments we will talk about the “target host” and “VDBs.”
- [read the slide]
- [read the slide]
- [read the slide] Note that no Oracle processes run on the Delphix server.
- [read the slide] The dSource is typically a fraction of the size of it’s production database, due to several storage optimizations made by Delphix.
- [read the slide]
- [read the slide] I should emphasize the first sentence again: VDBs are fully functional databases. They look like databases, their data files appear to be the same size as they would be physically. They can be backed up. They can be restored to. To end users, and to DBAs alike, they should be indistinguishable from any other database whose data files and logs happen to be stored on NFS.
- With Delphix, the refresh process changes. The refresh is accomplished by a few clicks in the Delphix GUI, via a command line interface, or via an automatic policy. Instead of requiring a full terabyte of storage for the development server, we now require a fraction of that to store a compressed copy of the production database within Delphix. This becomes the basis of the storage for our new Virtual Database (VDB) in the development environment, and is provided to the development host via NFS. As the development users begin to change their virtual database, only the changed blocks need be stored back within the Delphix server. To summarize, the storage requirements to make a single copy have dropped from “as large as production” to “a fraction of production,” and the provisioning process is simple. Even better, additional copies require only a few changed blocks to be stored, so the more copies we make, the better the space savings become.
- DevOps. Increasing the tempo of application development. Supporting agile development methods. Shorter delivery cycles and continuous delivery. It isn’t just people working harder, it needs some magic.