Delphix database virtualization v1.0


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Delphix database virtualization v1.0

  1. 1. Delphix Database Virtualization DataManage Ltd. 20 Yad Hamaavir St. Tel Aviv Tel. +972-52-6665638 Fax. +972-3- 6446995
  2. 2. Delphix Core Features Delphix is a software appliance that can be deployed on any ESX Virtual Machine infrastructure, a task that usually takes just a few minutes. Once installed, the primary interface to the software is accessed through your web browser. From this interface, a technologist can instantly begin choosing and setting up source databases within Delphix and choosing systems that will be used as deployment targets. Installing the software and beginning setup is a painless procedure with minimal configuration requirements. Figure 1. The Delphix Web GUI Easy Graphical Management Nearly everything in Delphix is done via the web interface. While there is a command line interface as well with some advanced functionality, the act of snapshotting and provisioning databases can be done with a few clicks by technical and non-technical staff. Versatile Source DB Compatibility Delphix is extremely versatile in that it allows a wide range of database versions both for Oracle and SQL Server on nearly any storage architecture to act as source systems. Delphix will fully automate linking the source database to the appliance. Incremental snapshots are also automated and managed by the software without the need for complex cleanup and maintenance processes.
  3. 3. TimeFlow Point-In-Time Provisioning TimeFlow is a feature that takes and stores changes on a source database for a defined recovery window. The benefits of this feature are twofold. It keeps the source database represented in Delphix up to date, which is important for guaranteeing fresh data when provisioning or refreshing a target system. But just as importantly, it allows target databases to be provisioned from any point in time during the recovery window, down to the second. This means that if you choose to keep 30 days of TimeFlow data, you can provision a Virtual Database to a target environment containing the data as it was 25 days ago, and another Virtual Database with the data as it was 20 days ago, and another as it was 15 days ago, and so on. The multiple copies and the multiple times from which they are created incur no additional space. Figure 2. Delphix is made with DBMS software in mind. By taking a single backup of a production (source) database, And propagating transaction history, Many virtual databases can be provisioned from many Points in time using its SCN aware TimeFlow feature.
  4. 4. Huge Storage Savings In addition to the storage savings achieved by compression of the source snapshot, the fact that provisioned databases all utilize the same source blocks means further storage savings. The initial snapshot is compressed, the TimeFlow incremental snapshots are compressed, and the target virtual databases are all using that same compressed space; the only extra space consumption is the changes made on each target virtual database. The storage space required for virtual snapshots is far less than the incrementally growing storage space necessary for physical backups. Additionally, in order to provision a clone from a physical backup it will be necessary to duplicate the storage space. In a virtual database environment, the target systems will all use the same space for unchanged blocks. For example, imagine a 9TB source database. The initial link to Delphix takes 3TB stored with compression. That now leaves 6TB to save changes from the source database on the Delphix but those 6TB are 6TB compressed which is actually 18TB of changes from the source database uncompressed, so Delphix can save 18TB of changes in the size of one normal backup of the database. Those 18 TB can represent, months of changes depending on the change rate of the database.
  5. 5. High Performance Block Sharing in Cache The average organization provisions multiple copies of a single production database which can include multiple copies of development, QA, UAT, stress testing, maintenance QA, and others. Each of these copies requires time to build and provision and the end result is several identical instances with their own copies of largely identical data blocks across several shared memory regions. Delphix achieves a great reduction is disk usage by only saving identical blocks to disk once. But more importantly, Delphix also only caches unique data blocks in memory. This means that read performance across multiple systems that are provisioned from a single source can show dramatic improvements and scalability. As the number of concurrent users (x-axis) increase, TPMs of a physical database reaches a limit when I/O subsystem saturates, but with a virtual database once Delphix can cache the blocks for one virtual database then they are also cached for the other virtual database(s) so TPM continues to increase and I/O latency stays low as most I/Os are being satisfied by the cache on Delphix. The above example represents a 200GB swing bench database with 200GB of memory on Delphix being accessed by 2 concurrent clones. TPM is cumulative and latency is averaged. As the number of databases provisioned from a single source and the number of concurrent users increase, performance in a Delphix environment can grow linearly. These factors would usually destroy performance on a physical database; however, shared block caching means increased TPM and low latency on virtual databases when more concurrent users are added Delphix is the perfect match for Oracle’s plugable databases (PDB). A core use case for PDBs is cloning but each clone takes up more disk space and each clone needs it’s own memory for it’s data block buffer cache, where as with PDBs on Delphix all duplicate data blocks are stored in disk and all duplicate datablocks are shared in the cache on Delphix minimizing the need for a large buffer cache memory on the container database (CDB).
  6. 6. Version Control Application development generally would not be manageable without solid version control. Popular packages like SVN, Git, Mercurial, and CVS are used by project managers, development teams, and QA groups to maintain and keep track of software revisions, bug fixes, and major releases. Yet for all the version control capabilities available to software developers, there are not many viable options for version control of the data in your database. Oracle’s built-in capabilities like Flashback Query can be useful, but require a rewrite of every query to include the AS OF syntax. Flashback Database is a viable option for rolling back an entire database to a point in time, but it requires a full copy of the database along with extra storage for flashback logs. Even more importantly, it all has to be manually managed and orchestrated by DBA staff. Virtual Database points-in-time can be lined up to code releases, and tagging allows the DBA to easily keep track of the snapshots that correspond to an application patch or release. Delphix is capable of temporal data provisioning, meaning that it can provision a virtual database from any time point in the TimeFlow retention window. Further, it can provision multiple target systems from multiple points on the same timeline without any extra storage or orchestration requirements. The Delphix software also allows tags to be created for different points in time for semantic recognition; for example, you can tag a point such as “July 25, 2012 13:00” with the words “Version 1.3.5 Bugfix Release”. Snapshots can also be “kept” as well, ensuring that a specific snapshot never ages out of the retention window.
  7. 7. The Technology Behind Delphix Delphix is shipped and deployed as an OVA file that is placed in an ESX environment. While there are no strict requirements for installing Delphix, it is recommended that the Virtual Machine hosting the software appliance be configured with a large amount of RAM for shared caching and enough disk for your source databases and their incremental snapshots. Once the Delphix Engine is installed, there are three main steps in order to create a new virtual database:  Register host machines  Source machine running a master database  Target machine with database binaries installed  Link a source database to Delphix  Provision a virtual database from the master database onto a target machine Delphix Product Demo (Click Image)
  8. 8. Delphix is made up of three main components: DxFS – The Delphix File System DxFS is the file system used by the Delphix host for storage of snapshots. It includes features for block caching, filtering, compression, and the block mapping that forms the secret sauce behind the provisioning of many environments from a single backup source. Snapshots and changes that are collected from the source database and managed by Delphix are filtered and compressed, eliminating empty and temporary blocks and freeing up large amounts of storage. Block mapping built into the file system keeps track of data blocks for multi-versioning over time and works together with the DataVisor tier to virtualize “target-ready” data files that the target virtual database can use to spin up in full read/write mode. DataVisor – Data Orchestration The DataVisor tier of Delphix orchestrates the flow of snapshots for all source databases on the Delphix system. Once a database is linked, DataVisor monitors and maintains the incremental-forever synchronization strategy used in Delphix to save time and resources over the life of your provisioning architecture. DataVisor also manages the provisioned targets and changes that are made on those systems. The synthesis of the snapshots from the source database is what builds the TimeFlow feature. Once a virtual database is deployed from a point in TimeFlow, the DDL and DML performed on that target virtual database (private only to that virtual database) is also synthesized and managed by DataVisor. In conjunction with DxFS, Delphix can store roughly 50 days of continuous recovery points (any point-in-time) in the space of 1 full normal data copy. DataVisor keeps track of all of the work behind the scenes. Self Service Management – Policies and UI The Management tier of Delphix powers the three main interfaces:  The web-based GUI for human interface  A web services API to build provisioning into your existing code  A command line interface (CLI) for scripts In addition to providing the UI components, the Management layer also includes a policy framework for users and the data they can access, provision, refresh, etc. Retention policies are also defined here and used by the DataVisor tier in the management of TimeFlow snapshots. Lastly is the automation framework. Refreshes can be scheduled to occur on a specified basis; for instance, the beginning of each month or end of the year. During the VDB creation process, pre- and post-provisioning scripts can be identified to execute security procedures, handle specific QA team requests, or whatever else might be necessary to have the provisioned environment in a ready-for-use state.
  9. 9. Typical Environment with Multiple Copies of Production Database before Delphix Typical Environment with Multiple Copies of Production Database with Delphix