Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Delphix for DBAs by Jonathan Lewis

2,006 views

Published on

Jonathan Lewis explains his impressions of Delphix, how Delphix works and what use cases Delphix solves

Published in: Software
  • Be the first to comment

Delphix for DBAs by Jonathan Lewis

  1. 1. Saving time and reducing risk with Delphix Jonathan Lewis jonathanlewis.wordpress.com www.jlcomp.demon.co.uk
  2. 2. Delphix for DBAs 2 / 36 Who am I ? Independent Consultant 31+ years in IT 26+ using Oracle Strategy, Design, Review, Briefings, Educational, Trouble-shooting Oracle author of the year 2006 Select Editor’s choice 2007 UKOUG Inspiring Presenter 2011 ODTUG 2012 Best Presenter (d/b) UKOUG Inspiring Presenter 2012 UKOUG Lifetime Award (IPA) 2013 Member of the Oak Table Network Oracle ACE Director O1 visa for USA Jonathan Lewis © 2013 - 2014
  3. 3. Delphix for DBAs 3 / 36 Topics • Testing requirements • Virtual Databases • Basic DxFS • Oracle Backups • Possible uses Jonathan Lewis © 2013 - 2014
  4. 4. Testing is difficult (a) Last night this report took 45 minutes instead of 3. Why ? Make sure it doesn't do it again tonight. It seems to be running as expected right now Maybe it was resource contention - any clues in AWR. Maybe it was a change in plan - how to prove it. What's the fastest way to eliminate possibilities ? Delphix for DBAs 4 / 36 It's not instrumented properly. It didn't get captured in any AWR snapshots. There are fragments in historic ASH Jonathan Lewis © 2013 - 2014
  5. 5. Testing is difficult (b) I believe I can make this part of the batch run six times faster - when can I test it ? You can have pre-prod at 3:00 am the day after tomorrow. But it's going to take 2 hours to prime the data correctly (provided nothing goes wrong while we're priming it) Give me 5 minutes and you can have a copy of prod as it was last night at the correct point in the batch … and the same for the previous night … and the night before Delphix for DBAs 5 / 36 Jonathan Lewis © 2013 - 2014
  6. 6. Testing is difficult (c) The best place to test is on production Right data, right stats, right hardware, right activity End-users may disagree. If nothing else competition for resources is a threat Next best is a "good" copy of production Still unlikely to emulate performance characteristics Usually adequate for "sanity" checks How "good", how often, how quickly, how private ? Delphix for DBAs 6 / 36 Jonathan Lewis © 2013 - 2014 The best way to avoid big mistakes in development is to test on a carefully timed full-scale copy of the production data.
  7. 7. Testing is difficult (d) Warning: Even if you had a disk-based clone of production on exactly the same hardware across the entire stack you still have to worry about the impact of: The target is to maximise your level of confidence Delphix for DBAs 7 / 36 Basic concurrency effects Oracle-related resource contention Read-consistency Historic caching effects Other people's tests Jonathan Lewis © 2013 - 2014 No matter how good your test system there are always imponderables that require some careful thought when you try to interpret test results.
  8. 8. Could be a "logical snapshot" of production Probably better to use a copy Plus N modifying instances "local" copies of blocks the instance has modified Delphix for DBAs 8 / 36 Virtual DB Avoids duplicating data storage One full, read-only copy Jonathan Lewis © 2013 - 2014 If you have only one development project a very recent backup may be adequate, but if you have many projects it's best to keep them separate.
  9. 9. Delphix for DBAs 9 / 36 Options (a) Storage device Jonathan Lewis © 2013 - 2014 Production Server Development Production DB Server "Logical copy" Dev Delta It is possible to have a solution that takes a "logical copy" of the production system on the same SAN, and then writes modified blocks to a separate area
  10. 10. Delphix for DBAs 10 / 36 Options (b) Production DB You could use "split mirror" technology to make a fast, cheap, local copy of Production and associate multiple deltas with it. Jonathan Lewis © 2013 - 2014 Split Mirror Dev1 Delta Dev2 Delta
  11. 11. Delphix for DBAs 11 / 36 Options (c) Production DB You could take a copy of production to another storage device, and then use the "delta" technology there. This helps to protect your production system. Jonathan Lewis © 2013 - 2014 Backup Copy Dev1 Delta Dev2 Delta
  12. 12. Considerations Minimum impact on production system Ease of creating/discarding a virtual database Speed of refreshing "primary" copy Delphix for DBAs 12 / 36 Delphix uses rman backups with log transport Very simple once target machine is setup The ideal would be a "rolling-history" mechanism Jonathan Lewis © 2013 - 2014
  13. 13. Windows 7, VMWare Server 2.0 - 16GB RAM, 4 CPU, 2 x 500GB drives Delphix for DBAs 13 / 36 My Sandbox OEL 5.6 Oracle 11.2.0.4 Production: s/w and database with "delphix" a/c Jonathan Lewis © 2013 - 2014 Delphix 3.0 Backups and virtual databases OEL 5.6 Oracle 11.2.0.4 Development: s/w and config files with "delphix" a/c
  14. 14. Basic architecture Independent system - running DxFS. Acts as a network file server for vDBs Registers source and destination machines Takes rman backups from source machines Delphix for DBAs 14 / 36 Uses transparent compression to save space Keeps rolling backups using minimum space Mounts backups on destination machines Uses DxFS technology to present virtual DBs Best effect - when a small fraction of the database changes Jonathan Lewis © 2013 - 2014
  15. 15. t0 Metadata (associates logical block with physical location) a b c d e f g h i Data (i.e. filesystem blocks) Delphix for DBAs 15 / 36 DxFS Jonathan Lewis © 2013 - 2014 128 entries per block Variable number of sectors per block Set the DxFS block size to match the database block size. DxFS blocks are transparently compressed to the smallest number of sectors possible.
  16. 16. "Updates" t0 t1 a b c d e f g h i Delphix for DBAs 16 / 36 Jonathan Lewis © 2013 - 2014 b' c' Data only becomes visible when root block is written You don't overwrite blocks on disc with a new version, you write a new copy and maintain the index (and you don't overwrite index blocks either).
  17. 17. Multi-version t1 a d e f g h i Delphix for DBAs 17 / 36 Jonathan Lewis © 2013 - 2014 b' c' This is the latest state of our file system - but we don't have to release the old versions of the blocks we "changed".
  18. 18. t1 t0 a d e f g h i Delphix for DBAs 18 / 36 Aging data Blocks that can be freed when we don’t want the older snapshot any more Jonathan Lewis © 2013 - 2014 b c b' c' Over time we may "overwrite" many of the most recent blocks, and be able to free old copies of those blocks, keeping a lot of the oldest original blocks
  19. 19. Working with Oracle • Start with an rman backup • Take regular incremental backups – Delphix has custom code to apply them efficiently • For a vDB pick a recovery point – Delphix code mounts the file system, configures the Delphix for DBAs 19 / 36 target, and sets up the instance. Jonathan Lewis © 2013 - 2014 Time for a demonstration: (identify source and target machines, identify source d/b, create first backup, create vDB).
  20. 20. Delphix for DBAs 20 / 36 Traditional Backups Level 0 Level 1 Jonathan Lewis © 2013 - 2014 Data Redo Level 1 Data Redo Level 1 Data Redo Redo
  21. 21. Delphix for DBAs 21 / 36 Delphix calls rman CONNECT TARGET / SET ECHO ON; SET ENCRYPTION OFF; RUN { SET NOCFAU; ALLOCATE CHANNEL DLPX_CHANNEL_0 DEVICE TYPE sbt … ; ALLOCATE CHANNEL DLPX_CHANNEL_1 DEVICE TYPE sbt PARMS=' Jonathan Lewis © 2013 - 2014 SBT_LIBRARY=/home/oracle/toolkit/Delphix_.../libobk.so, ENV=( DELPHIX_CONFIG_DIR=/home/oracle/toolkit/Delphix_.../lib64, DELPHIX_URL=http://192.168.91.167:8341/context/servlet, DELPHIX_HID=SNL.1.17.F76CB2C54939EE8FF470FA ) ' ; SET COMMAND ID TO 'SNL.1.17.F76CB2C54939EE8FF470FA'; BACKUP NOEXCLUDE FORCE COPIES 1 AS BACKUPSET FILESPERSET 5 TAG 'DLPX201306181730521301FE51F5B5' ARCHIVELOG FROM SCN 6726134 ; } exit
  22. 22. Delphix for DBAs 22 / 36 Delphix Backups Level 0 Level 1 Jonathan Lewis © 2013 - 2014 Data Redo Level 1 Data Redo Level 1 Data Redo Redo Delphix unpacks the rman backups as they are generated and applies them (excluding redo) to the level 0 backup.
  23. 23. Delphix for DBAs 23 / 36 Delphix Backups Jonathan Lewis © 2013 - 2014 DxFS doesn't overwrite changed blocks, though, it writes them to a separate location and creates some new index entries.
  24. 24. Delphix for DBAs 24 / 36 Delphix Backups Jonathan Lewis © 2013 - 2014
  25. 25. Delphix for DBAs 25 / 36 Delphix Backups Jonathan Lewis © 2013 - 2014
  26. 26. Redo Rolling forward creates some new (f/s) blocks and pointers, and a new root block. Every new vDB gets its own "index root block". Delphix for DBAs 26 / 36 Pick your version Pick an index to create your vDB Jonathan Lewis © 2013 - 2014 Redo Redo Redo Delphix rolls forward (and back) from the associated redo
  27. 27. Delphix for DBAs 27 / 36 Rolling history Jonathan Lewis © 2013 - 2014 Redo To delete history Delphix can do an optimised walk of "old" indexes and delete any blocks for which a newer block has been created.
  28. 28. – Take copies of archived redo log – vDBs roll forward and open at any desired SCN – Frequent snapsync operations reduce time to roll forward – Can do "pre-provisioning" (continuous rollforward) Delphix for DBAs 28 / 36 Options • Snapsync – Take regular rman backups "from SCN" – vDBs open at granularity of snapshots • Logsync Jonathan Lewis © 2013 - 2014
  29. 29. Last night this report took 45 minutes instead of 3. Why ? Make sure it doesn't do it again tonight. Delphix for DBAs 29 / 36 Solutions (a) Create a vDB as at report start time last night. It's got yesterday's data and yesterday's stats Run the report. Does it take 2 hours or 3 minutes ? Jonathan Lewis © 2013 - 2014
  30. 30. Solutions (b) I believe I can make this part of the batch run six times faster - when can I test it ? Create a vDB as at the right moment from last night. Run the old code Create a vDB as at the right moment from last night. Run the new code Check results match, check workload Delphix for DBAs 30 / 36 Rinse and repeat for every night last week / month. Jonathan Lewis © 2013 - 2014
  31. 31. – Create a vDB to gauge the side effects of index changes – Some idea of time to build. Delphix for DBAs 31 / 36 Ideas (a) • Stats Tests – Create a vDB to generate / fake different statistics • E.g. histograms, global partition code strategies – Run test-suite – Transfer stats to production if appropriate • Index tests: Jonathan Lewis © 2013 - 2014
  32. 32. Delphix for DBAs 32 / 36 Ideas (b) • Upgrades / Patches: – Create a vDB from production – Create a new Oracle Home for new s/w version – Run upgrade • Single instance conversion to RAC – you install the clusterware etc. – Delphix will give you the database and instances Jonathan Lewis © 2013 - 2014
  33. 33. Delphix for DBAs 33 / 36 Ideas (c) • Hand-off to QA – Create a vDB – Apply and test development changes – Create vDB from development vDB for QA to test – Continue development • Rinse and repeat • Drop "spare" QA vDBs Jonathan Lewis © 2013 - 2014
  34. 34. – But time for obfuscation needed only once per "generation" – (obfuscate once, use many) Delphix for DBAs 34 / 36 Ideas (d) • Obfuscation: – Create a vDB from production – Run obfusaction code. – Create vDB(s) from obfuscated vDB. • Probably more restrictive Jonathan Lewis © 2013 - 2014
  35. 35. Performance • Under any circumstances, changes in discs, CPU, memory, and users make it difficult to compare performance. – Check statistics - SQL by reads, segments by reads • Sharing (cached) blocks on the Delphix may make some I/O faster. • CoW may make tablescans / IFFS slower • Effects on writes may vary with scatter Delphix for DBAs 35 / 36 Jonathan Lewis © 2013 - 2014
  36. 36. Conclusion • Questions to ask about virtual databases Delphix for DBAs 36 / 36 – How much extra workload (at source) – How much space needed (and where) – Simplicity of generation – Simplicity / overhead of refresh • Versatility – One big backup is good for everything ! • Interpretation of performance Jonathan Lewis © 2013 - 2014

×