Collaborate instant cloning_kyle

1,535 views
1,458 views

Published on

Published in: Technology, Spiritual
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,535
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Kyle HaileyWork for a company called DelphixWe write software that enables companies toCopy their databases in 2 minutes with almost no storage overheadWe accomplish that by taking one initial copy and sharing the duplicate blocks Across all the clones
  • want data now.don’t understand DBAs.Db bigger and harder to copy.Devswant more copies.Reporting wants more copies.Everyone has storage constraints.If you can’t satisfy the business demands your process is broken
  • What are these technologiesWhat benefits and drawbacksTechnology is awesomeTechnology coming of age Coming of ageClonedb 3 pres @ OOWSMU just release month ago12c “clone” pluggable databases
  • Databases are largeMoving data around is hard work.Moving them takestime, resources, equipment and experience1) 10 years in support - No backups, hospital companies still don’t back up dev often dev is the new prod2) Full time DBA, half time copying3) OEM wanted me to do full phys cloning4) Was a consultant always wanted database to play on How many of you copy databases ? How much time does you spend on it?
  • Prod critical for businessPerformance of prod is top priorityProtect prod from load
  • 2 options to create enough copies
  • Xxx spends 50% of time copying databases have to subset because not enough storagesubseting process constantly needs fixing modificationWhat happens when developers use subsets -- ****** -----
  • Stubhub (ebay) estimates that 20% of there production bugs arise from testing onSubsets instead of full database copies.
  • Wait orebay till next slide
  • Example at Ebay2 dozen developers have a massive shared copy of productionExample at DB3 development teams agree between themselves who is doing what testing this week as some runs destroy other teams data,If the database is shared it’s hard to get opportunity to refresh and a data get’s old
  • Only having enough equipment to support 2 or 3 environments causes massive delaysState of Colorado has a 100 projectsAnd they can only support 3 at a timeKLA tencor can only support 2 projects of a dozen
  • DB had databases which were not refreshed for 6 months+ due to refresh time and size
  • Slow downs mean bottlenecksThese bottlenecks cause failures in IT projectsI’m into eliminating bottlenecks (whether it is wait events, tuning sql or provisioning copies of dbs)
  • Development asks for a database it takes days or weeks.
  • 90% of lost developer days at customer was due to waiting for environment builds
  • Happens both for dev and QA
  • Tightening constraining resourcesCascading affect on companies.The business doesn’t know or understand this DBA workDBAs are often the hardest resource for IT to justify because they are invisibleDBAs are already being asked to do a tremendous amountDBAs are often on call 24x7DBAs are foundational.
  • http://mobile.stufffundieslike.com/2010/02/upping-the-ante-on-doctrinal-disagreements/
  • Delays cause failures*http://www.pcworld.com/article/246647/10_biggest_erp_software_failures_of_2011.html
  • MisguidedattributingRelax the constraints http://martinfowler.com/bliki/NoDBA.html
  • Fastest query is the query not run
  • Performance issuesSingle point in time
  • Performance issuesSingle point in time
  • Physical File System:Performance issues Multiple points in timeOccasional rebuild
  • Physical File System:Performance issues Multiple points in timeOccasional rebuild
  • Buisness Continuance Volume
  • Business Continuance Volume
  • Symetric Remote Data Facility
  • Physical File System:Performance issues Multiple points in timeOccasional rebuild
  • Oracle Database Cloning Solution Using Oracle Recovery Manager and Sun ZFS Storage Appliancehttp://www.oracle.com/technetwork/articles/systems-hardware-architecture/cloning-solution-353626.pdf
  • Oracle Database Cloning Solution Using Oracle Recovery Manager and Sun ZFS Storage Appliancehttp://www.oracle.com/technetwork/articles/systems-hardware-architecture/cloning-solution-353626.pdf
  • OverlayClonedb - 1 copy, performance issuesCopy on writeEMC - 16 points in time performance issuesVMware – not support with OracleAllocate on WriteZFS – manually config, performance issuesNetApp – 255 points in time , rolling window
  • Technology has existed 15+ years Why hasn’t there been more adoption ??
  • http://partners.netapp.com/go/techontap/empower-dba.html?fmt=printCreate Luns, aggr, snapshots, clonesMirroring filesystemsExporting file systemsMounting file systems
  • Requires expert storage admins specialized equipment scripting2-7 Days or 2-8 hours if everyone togetherCERN recently gave a presentation where they wrote almost 30,000 lines of code13k lines & 15k lines of PHP
  • Like the internetThin provisioning ultimately led to db virtualization, goes beyond thin provisioning eliminate the overhead of managingproviding significant agility gains
  • Like the internet
  • Automate and simplify
  • Saw it at OOW nothing till couple weeks agoCustomer had beta but wouldn’t allow us to take screen shots
  • I like at DelphixFrustratedSteve and Larry gave aweseom presentationsSteve jobs and Ellison ui combined forces now I have itDelphix GUI is what Oracle Enterprise Manager would look like if Apple had designed it
  • Development Provision and RefreshFullFreshFrequent (Many) Source control for code, data control for the database Data version per release version Federated cloning QA fork copies off to QA QA fork copies back to Dev Instant replay – set up and run destructive tests performance A/B Upgrade patching Recovery Backup 50 days in size of 1 copy, continuous data protection (use recent slide ob backup schedules full, incr,inrc,inrc, full) Restore logical recovery on prod logical recovery on Dev Debugging debug on clone instead of prod debug on data at the time of a problem Validate physical integrity (test for physical corruption)
  • In physical worldIf 3 Copies of a database
  • Fast = Fresh Full = Quality Many = jet pack on development-
  • Change mentality from few as possible to as many as accelerates the businessRemember Jinga ?
  • Self Service
  • Source Control for the database data
  • Physically independent but logically correlatedCloning multiple source databases at the same time can be a daunting task
  • One example with our customers is InformaticaWho had a project to integrate 6 databases into one central databaseThe time of the project was estimated at 12 monthsWith much of that coming from trying to orchestratingGetting copies of the 6 databases at the same point in timeLike herding cats
  • Informatical had a 12 month project to integrate 6 databases.After installing Delphix they did it in 6 months.I delivered this earlyI generated more revenueI freed up money and put it into innovationwon an award with Ventana Research for this project
  • Multiple scripted dumps or RMAN backups are used to move data today. With application awareness, we only request change blocks—dramatically reducing production loads by as much as 80%. We also eliminate the need for DBAs to manage custom scripts, which are expensive to maintain and support over time.
  • 50 days of backups in the size of one backup
  • Development Provision and RefreshFullFreshFrequent (Many) Source control for code, data control for the database Data version per release version Federated cloning QA fork copies off to QA QA fork copies back to Dev Instant replay – set up and run destructive tests performance A/B Upgrade patching Recovery Backup 50 days in size of 1 copy, continuous data protection (use recent slide ob backup schedules full, incr,inrc,inrc, full) Restore logical recovery on prod logical recovery on Dev Debugging debug on clone instead of prod debug on data at the time of a problem Validate physical integrity (test for physical corruption)
  • If every MB was an Inch 300,000 customers 12 copies on average 100 GB avg size PB TB GB 300000*12*100 = 360,000,000 300000*1*.3*100 = 9,000,000 351 PB e p t g 1,191,290,000 feet to moon, 132,000,000 feet around the earthe p t g m k b 15,133,979,520 inches to the moone p t g m k b 351,000,000,00015,133,979,520 inches to the moone p t g m k b 35100000000015133979520 inches to the moon
  •  HD  720TB down to   8TB     ( create 19 x 36TB VDBs )
  • Informatica – finished 2x fasterStubhub -   2 x as many releases a yearKLA-Tencore-  5 x as many projectsQA/QualityStubhub - 20% less bugs in production, found full table scan that would have been missed on subsets
  • Moral of this storyInstead of dragging behind enormous amounts of infrastructureand bureaucracy  required to provide database copiesUses db virteliminates the drag and provides power and acceleration To your companyDefining moment CompetitorsServices
  • Once Last Thinghttp://www.dadbm.com/wp-content/uploads/2013/01/12c_pluggable_database_vs_separate_database.png
  • 250 pdb x 200 GB = 50 TBEMC sells 1GB$1000Dell sells 32GB $1,000.terabyte of RAM on a Dell costs around $32,000terabyte of RAM on a VMAX 40k costs around $1,000,000.
  • http://www.emc.com/collateral/emcwsca/master-price-list.pdf    These prices obtain on pages 897/898:Storage engine for VMAX 40k with 256 GB RAM is around $393,000Storage engine for VMAX 40k with  48 GB RAM is around $200,000So, the cost of RAM here is 193,000 / 208 = $927 a gigabyte.   That seems like a good deal for EMC, as Dell sells 32 GB RAM DIMMs for just over $1,000.    So, a terabyte of RAM on a Dell costs around $32,000, and a terabyte of RAM on a VMAX 40k costs around $1,000,000.2) Most DBs have a buffer cache that is less than 0.5% (not 5%, 0.5%) of the datafile size.
  • reduces storagealleviates DBA of repetitive focus on innovationAccelerates DevelopmentEliminate bottleneck more code faster and of better quality
  • VMWAREhttp://www.virtuallyghetto.com/2010/10/how-to-control-maximum-number-of-vmware.htmlhttp://communities.vmware.com/thread/303689EMChttp://www.lascon.co.uk/snap-timefinder.phphttps://community.emc.com/servlet/JiveServlet/previewBody/11789-102-1-45992/h8728-snapsure-oracle-dnfs-wp.pdf
  • http://media.netapp.com/documents/tr-3761.pdf
  • http://media.netapp.com/documents/tr-3761.pdf
  • http://www.redbooks.ibm.com/redpapers/pdfs/redp4133.pdf
  • http://oraclestorageguy.typepad.com/oraclestorageguy/2007/07/oracle-backup-1.html
  • incremental forever collectionClones: fast to create, small foot print, can create from any point in time
  • Batch window under pressureProduction Support environments need to be refreshed daily.Support environments needs to be available from start of business day Refresh overruns impact Production support analystsTraining & Education environments need to be ‘rolled back’ after training course.Storage constraintsNot enough storage space to hold multiple copies on-line. Tape restores required for older clones.
  • Storage infrastructure and technical resources constraints directly impact project delivery. Storage availability, procurement and operational costs traditionally limits the number of OEBS clones possible. This impacts the number of parallel activities possible, resulting in extended project timescales.Availability of data center technical & DBA resources to provide ‘On-Demand’ clones Resulting in project delays whilst environments are being sourced.Technical teams need representative data volumes & Application set-up to undertake performance testing e.g.ASCP (Advance Supply Chain Planning), Period End / Year End processing etc..
  • Collaborate instant cloning_kyle

    1. 1. Database VirtualizationandInstant CloningKyle Haileyhttp://dboptimizer.com
    2. 2. Database Cloning ChallengeBusiness want data now.Business don’t understand DBAs.Databases getting bigger & harder to copy.Developers want more copies.Reporting wants more copies.Everyone has storage constraints.If you can’t satisfy the businessdemands your process is broken.
    3. 3. Two PartsI. Cloning TechnologyII. Accelerate your business
    4. 4. Part I : Cloning Technology3. Virtual2. Thin Provision1. Physical
    5. 5. =database1. Physical Cloning
    6. 6. ProblemDevelopersQA and UATReportsFirstcopyProduction• CERN - European Organization for NuclearResearch• 145 TB database• 75 TB growth each year• Dozens of developers want copies.
    7. 7. workaroundsDevelopersQA and UATReportsSharedSub set copyProductionMany copies
    8. 8. Physical ClonesDatabase SubsetsShared Databases
    9. 9. Subsets
    10. 10. The Production‘Wall’Classic problem is that queries thatrun fast on subsets hit the wall inproduction.Developers are unable to test againstall data
    11. 11. Shared Full
    12. 12. Shared access = Poor ProductivityDevelopers andtester get frustratedDatabases become oldand unrepresentativeof production.Requires complexscheduling andmanagement
    13. 13. Never enough environmentsAverage customer makes 12 copies of production- Charles Garry, Database Product Manager Oracle
    14. 14. Physical Copies – Time & SpaceconsumingTime consumingTime to make copies, days to weeksRMAN backup, archive logs, copy data over, recoverMeetings , days to weeksSystem, Storage ,Database ,Network Admins, manager coordinationSpace consuming12 devs x 10TB production = 120TB20 report DBs x 40 TB = 800TBSlow Clones mean bottlenecks
    15. 15. Setup Develop
    16. 16. Setup
    17. 17. Setup DevelopQA
    18. 18. $40M$75M$850M$27,000MStorageITDevelopBusiness
    19. 19. ERP Project Failures 2011• NYC CityTime : delays $63 M => $760 M• Montclair Uni: delays sues PeopleSoft• Idaho : delays ERP cost millionsStandish : IT Project Failure Rate1994 1996 1998 2000 2002 2004 200931% 40% 28% 23% 15% 18% 24%★http://www.galorath.com/wp/software-project-failure-costs-billions-better-estimation-planning-can-help.php*http://www.pcworld.com/article/246647/10_biggest_erp_software_failures_of_2011.html
    20. 20. Clone 1 Clone 399% of blocks are IdenticalClone 2
    21. 21. 2. Thin Provision Cloning
    22. 22. Clone 1 Clone 2 Clone 3Thin Provision
    23. 23. 2. Thin Provision CloningRequires Cornerstone Technology
    24. 24. I. clonedbII. Copy on Write Snapshotsa) EMC BCVb) EMC SRDF or Recover Pointc) VmwareIII. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS2. Thin Provision Cloning
    25. 25. RMANbackupdNFSsparse fileI. clonedb
    26. 26. RMANbackupdNFSsparse fileI. clonedb
    27. 27. I. CloneDB• dNFS 11.2.0.2+– cd $ORACLE_HOME/rdbms/lib– make -f ins_rdbms.mk dnfs_on• Clonedb.pl initSOURCE.ora output.sql– MASTER_COPY_DIR="/rman_backup”– CLONE_FILE_CREATE_DEST="/nfs_mount”– CLONEDB_NAME="clone"• sqlplus / as sysdba @output.sql– startup nomount PFILE=initclone.ora– Create control file backup location– dbms_dnfs.clonedb_renamefile(/backup/file.dbf , /clone/file.dbf);– alter database open resetlogs;Tim Hallwww.oracle-base.com/articles/11g/clonedb-11gr2.php
    28. 28. I. CloneDBRMANphysicalTarget A NFS ServerClone 1Clone 2Read only Clone 3Target BClone 1Clone 2Clone 3
    29. 29. I. clonedbII. Copy on Write + snapshotsa) EMC BCVb) EMC SRDF or Recover Pointc) VmwareIII. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS2. Thin Provision Cloning
    30. 30. DActiveFileSystemSnapshotCBAFile System constant - EMCII. Copy on Write a) EMC snapshots
    31. 31. DActiveFileSystemSnapshotDCBAChanges written in place - EMCChanges to new area - VMwareII. Copy on Write a) EMC snapshots
    32. 32. • Goal: backup– Create BCV , take snaps of BCV– Zone and mask LUN to target host– Full copy of disk, snapshot, recover• Snapshots– 16 snapshots– Zero or one snapshots of snapshots• “Golden Copy”– EMC uses a save area, area for changes to the snapshot– Initial snapshot has to stayII. Copy on Write a) EMC BCV
    33. 33. Non-Prod FilerEMC FilerProductionDatabaseDatabaseLunsTarget ATarget BTarget CClone 1Clone 2Clone 3Clone 4SnapshotsMax 16a) EMCBCVIII. Allocate on WriteBCVDay 1snapshotSnapshotor breakDay 2Timefinder needed for snapshotson across multiple LUNs
    34. 34. • SRDF – Symetrix Remote Data Facility– Stream Changes to a remote machine• Recover Point– Capture changes on wire– Split changes send to remote siteII. Copy on Write b) EMC SRDF
    35. 35. Non-Prod FilerEMC FilerProductionDatabaseDatabaseLunsTarget ATarget BTarget CClone 1Clone 2Clone 3Clone 4File system levelSRDF orRecover PointSnapshotday 1Snapshotday 1Snapshotday 2Snapshotday 2II. Copy on Write b) EMC SRDF
    36. 36. DActiveFileSystemSnapshotDCBAChanges written to a new locationChanges to new area - VMwareII. Copy on Write, c) VMware
    37. 37. II Copy on Write c) VMwareData Director : Linked Clones• Not support for Oracle databases• Golden Copy : rebuild after 32 snapshots• x86 host databases only• Performance issues– “Having several linked clones can affect the performance of the source database and theperformance of the linked clones.”http://bit.ly/QOXbyE (on http://pubs.vmware.com )– “If you are focused on performance, you should prefer a full clone over a linked clone.”http://www.vmware.com/support/ws5/doc/ws_clone_typeofclone.html– Performance worse with more snapshots– Performance worse with more concurrent users
    38. 38. I. clonedbII. Copy on Writea) EMC BCVb) EMC SRDF or Recover Pointc) VmwareIII. Allocate on Writea) ZFSb) Netapp (EMC VNX)c) DxFS2. Thin Provision Cloning
    39. 39. Data Blocksroota.ZFSb.Netapp (EMC VNX)c. DxfsIII. Allocate on Write
    40. 40. a) ZFS• 1 disk = 1 filesystem• ~1990: N disks = 1 FS• 2001: ZFS starts• 2005: ZFS ships• 2008: ZFS storage appliance ships• 2010: Delphix heads ZFS open sourceIII. Allocate on Write
    41. 41. FS vs. ZFS• FS per Volume• FS limited bandwidth• Storage stranded• Many FS in a pool• Grow automatically• All bandwidthStorage PoolVolumeFSVolumeFSVolumeFS ZFS ZFS ZFS3a) ZFSIII. Allocate on Write
    42. 42. a) ZFSSnapshot rootLiverootDelphix Proprietary and ConfidentialZilIntent LogIII. Allocate on Write
    43. 43. ZFS Appliance + RMAN1. ZFS Appliance– Project db_master– Project db_clone– 4 file systems: datafile, redo, archive, alerts2. Source Database– NFS Mount ZFS Appliance– Backup with RMAN3. ZFS Appliance• Select db_master• Snapshots• Then each filesystem on db_master clone it onto db_clone4. Target Host– NFS Mount db_clone– recover clonecloning-solution-353626.pdfa) ZFSIII. Allocate on Write
    44. 44. Oracle ZFS Appliance + RMAN1. physicalZFS Storage ApplianceRMANcopyRMAN CopytoNFS mountTarget AClone 1Clone 1SnapshotNFSa) ZFSIII. Allocate on Writecloning-solution-353626.pdf
    45. 45. Oracle ZFS Appliance + DataguardDataguardZFS Storage AppliancedatafilesNFSTarget AClone 1Clone 1SnapshotNFSa) ZFSIII. Allocate on WriteDatafiles viaNFSmaa-db-clone-szfssa-172997.pdf
    46. 46. NetApp FilerNetApp FilerProductionDatabaseDatabaseLunssnapshotTarget ATarget BTarget CClone 1Clone 2Clone 3Clone 4File system levelsnapshotclonesSnap mirrorSnapshotManager forOracleFlexcloneb) NetappIII. Allocate on WriteRequires adatabase
    47. 47. NetApp FilerNetApp FilerProductionDatabaseDatabaseLunssnapshotTarget ATarget BClone 1Clone 2snapshotsnapshotsnapshotb) NetappIII. Allocate on Write
    48. 48. NetApp FilerNetApp FilerPhysicalDatabaseDatabaseLunssnapshotTarget ATarget BClone 1Clone 2snapshotsnapshotsnapshotb) NetappIII. Allocate on Write
    49. 49. • Compression• typically 3x• Block sharing• DxFS optimized for databases• Write optimizations• Space allocation and destroy• Shared blocks in memoryc) DxFSIII. Allocate on Write
    50. 50. I. clonedbII. Copy on Write Snapshotsa) EMC BCVb) EMC SRDF or Recover Pointc) VmwareIII. Allocate on Writea) Netapp (EMC VNX)b) ZFSc) DxFS2. Thin Provision Cloning
    51. 51. • Clonedb• EMC BCV• Netapp• EMC SRDF• EMC VNX2. Review: Thin Provision Cloning
    52. 52. 2. Thin Provision Cloning
    53. 53. 2. Thin Provision Cloning
    54. 54. 3. Database Virtualization• Oracle SMU• Delphix
    55. 55. VirtualizationLayerVirtualizationSMU
    56. 56. a) Oracle 12c SMUOracle Snap Management Utility for ZFS Appliance• Requires ZFS Appliance• Supports Linux , Solaris 10+, Windows 2008+• GUI– snapshot source databases– provision virtual databases3. Database Virtualization
    57. 57. Virtualization Layerx86 hardwareAllocateStorageAny typeSMUChoose your virtualization Layer:• Delphix and Oracle SMU automated out of box• Netapp and EMC SRDF/Recover Point require massive scripting• Delphix and possibly SMU share blocks in memoryZFS Storage Appliance
    58. 58. One time backup of source databaseDatabaseProductionInstanceFile systemRMAN APIs
    59. 59. Delphix Compress DataDatabaseProductionInstanceFile systemData iscompressedtypically 1/3size
    60. 60. Incremental forever change collectionDatabaseProductionInstanceFile systemChanges are collectedautomatically foreverData older than retentionwidow freed
    61. 61. Typical ArchitectureDatabaseFile systemProductionInstanceDatabaseFile systemDevelopmentInstanceDatabaseFile systemQAInstanceDatabaseUATInstanceFile system
    62. 62. Clones share duplicate blocksDevelopmentInstanceDatabaseProductionInstanceFile systemvDatabaseQAInstanceUATInstancevDatabase vDatabaseNFSSource Database Clone Copies of Source DatabaseFiber Channel
    63. 63. Part II: Virtualization Case Studies1. Development2. QA3. Recovery
    64. 64. 1. Development Acceleration
    65. 65. DatabaseVirtualizationlayer3 clones of same source database 3 virtual clones of same source databaseVirtualization layer orchestrates I/Oaccess between database and storage
    66. 66. Source TimeflowSource TimeflowSource DatabaseTarget HostVirtual DatabaseNFSFiberFiberRMAN overTCP
    67. 67. SourceFreshVDB
    68. 68. SourceVDBVDBVDBFrequent
    69. 69. Fresh dataFull clonesFrequent (many)fast
    70. 70. Self Service
    71. 71. Devv2.6 v2.6v2.6QA UATv2.6Productionv2.6 v2.6v2.6v2.7v2.6 v2.6v2.6v2.8Source Control for the database datav2.6v2.6v2.6v2.6v2.6v2.7v2.6v2.7v2.6v2.8v2.6v2.8
    72. 72. DevProd2.6
    73. 73. DevQAProd2.6
    74. 74. DevQAUATProd2.6
    75. 75. DevQAUATProdDevQAUAT2.62.7
    76. 76. DevQAUATProdDevQAUAT2.62.7DevQAUAT2.8
    77. 77. DevQAUATProdDevQAUAT2.62.7DevQAUAT2.8Data Control = Source Control for the Database
    78. 78. Collaborative Provisioning
    79. 79. VDB2Source2Source3Source1VDB3VDB1Collaborative sources
    80. 80. “I looked like a hero”Tony Young, CIO Informatica
    81. 81. 1: Review of Development• Provision , Self Service– Fast– Full– Fresh– Frequent (Many)• Source control for the database• Federated cloning
    82. 82. 2. Rapid QA
    83. 83. dSourcePass to QADeveloper VDBQA VDB
    84. 84. SourcePunt to DevDeveloper VDBQA VDBDeveloper VDB
    85. 85. dSourceForensic AnalysisVDB
    86. 86. dSourceUpgrades, Patches, RATVDB
    87. 87. A/B Testing• Production vs Virtual– invisible index on Prod– Creating index on virtual• Flashback vs Virtual• Confidence testing– change reporting SQL– compare new code over last 30 days
    88. 88. 2: review QA• Fork copies off of Dev to QA• QA fork copies back to Dev• Instant replay– performance A/B– Upgrade, patching– Forensic analysis of Prod
    89. 89. 3. Backup and Recovery
    90. 90. MONSUN WEDTUE THU SATFRI MONSUN TUEMONSUN WEDTUE THU SATFRI MONSUN TUEDataProcessingRequiredCurrent Volumeof Work
    91. 91. SourceRollback / Time machineVDB
    92. 92. SourceRollback / Time machineVDBVDB Rolled back
    93. 93. SourceRecoveryVDBV2P
    94. 94. 3: review Backup• Backup 50 days in size of 1 copy• logical recovery on prod, dev• Restore virtual to production
    95. 95. Review Part II: Data Virtualization1. Development: Full, Fresh, Fast, Data SCCS = SDCS2. QA : forking, replay, forensics3. Recovery: 50:1, time machine, V2PReview Part I: Technology1. Full Clones2. Thin Provision : dbclone, EMC, Netapp3. Database Virtualization : SMU, Delphix
    96. 96. over 10 times"perhaps the single largest storage consolidationopportunity history“
    97. 97. Oracle 12c
    98. 98. 80MB buffer cache ?
    99. 99. 5000Tnxs/minLatency300ms1 5 10 20 30 60 100 200with1 5 10 20 30 60 100 200Users
    100. 100. 200GBCache
    101. 101. 5000Tnxs/minLatency300ms1 5 10 20 30 60 100 200Users1 5 10 20 30 60 100 200
    102. 102. 200GBCache
    103. 103. 8000Tnxs/minLatency600ms1 5 10 20 30 60 100 200Users1 5 10 20 30 60 100 200
    104. 104. Database Virtualization
    105. 105. Matrix of featuresCloneDB ZFSApplianceDelphix DataDirectorNetApp EMCTime Flow No Yes Yes No Yes NoHardwareAgnosticYes No Yes Yes No NoSnapshots No Unlimited Unlimited 31 255 16(96 read only)Snapshots ofsnapshotsNo Unlimited Unlimited 30 255 1AutomatedSnapshotsNo No Yes No Yes NoAutomatedProvisioningNo No Yes No No NoAny O/S Yes Yes Yes Nox86 onlyYes YesMax size None None None ? 16-100TB?
    106. 106. Conclusion : Enterprise Solutions• VMware Data Director• EMC Timefinder– offer limited ability to benefit from cloning• Clonedb ***– fast easy way to create many clones of the same copy– limited to 11.2.0.2 and systems with sparse file system capability– suffers the golden image problem and performance• NetApp Flexclone, Snap Manager for Oracle– offers a rolling solution– limited database awareness– file system clones– limited snapshots– Vendor lock-in• Oracle ZFS Appliance– Vendor Lock-in• Delphix– Agility : Automation, unlimited snapshots, clones, multi-database
    107. 107. Appendix• CloneDB– http://www.oracle-base.com/articles/11g/clonedb-11gr2.php• ZFS– http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf• ZFS Appliance– ZFS setup for Oracle http://www.oracle.com/technetwork/server-storage/solaris/config-solaris-zfs-wp-167894.pdf– using RMAN http://www.oracle.com/technetwork/articles/systems-hardware-architecture/cloning-solution-353626.pdf– using dataguard http://www.oracle.com/technetwork/database/features/availability/maa-db-clone-szfssa-172997.pdf• Data Director– http://pubs.vmware.com/datadirector-25/topic/com.vmware.ICbase/PDF/vfabric-data-director-25-administration-guide.pdfp107 linked clone not support for Oracle• EMC– https://community.emc.com/servlet/JiveServlet/previewBody/11789-102-1-45992/h8728-snapsure-oracle-dnfs-wp.pdf– http://www.centroid.com/knowledgebase/blog/cloning-oracle-databases-with-emc-snapview-and-recoverpoint– http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:39960485246162 BCV and SRDF• NetApp– http://www.redbooks.ibm.com/redpapers/pdfs/redp4133.pdf– OEM for Netapp http://www.youtube.com/watch?v=J7fnfLS5Dxg&feature=youtu.be• Oracle SMU– http://www.oracle.com/us/products/servers-storage/storage/nas/smu-bus-wp-final-1857065.pdf– http://www.oracle.com/us/products/servers-storage/storage/nas/snap/smu-datasheet-1903178.pdf– http://www.oracle.com/us/products/servers-storage/storage/nas/snap/overview/index.html• Oracle OEM Snapshot Manager for Netapp– http://www.youtube.com/watch?v=J7fnfLS5Dxg&feature=youtu.be– Demo starts at 3:10 goes to 19:43
    108. 108. Other• Can’t run multiple clones on same host withASMhttps://forums.oracle.com/forums/thread.jspa?messageID=1810251• EMC VNXhttp://virtualgeek.typepad.com/virtual_geek/2012/05/vnx-inyo-is-going-to-blow-some-minds.html
    109. 109. NetApp LimitsController Size Limit32 bit controllers 16TBFAS3140/FAS3040/FAS3050 40TBFAS3160/FAS3070 50TBFAS6040/FAS3170 70TBFAS6080 100TBAll sources have to be in the same aggregate to besnapshot together.Limit of 255 snapshotssnaps are limited to the same aggregate (storage pool)Aggregates have size limits depending on controller
    110. 110. b) NetappIII. Allocate on Write
    111. 111. b) NetappIII. Allocate on Write
    112. 112. b) NetappIII. Allocate on Write
    113. 113. NetappvsEMC
    114. 114. ProductionInitialIncremental IncrementalRedoClonesRedoa) ZFSIII. Allocate on WriteGoal - How Can we do it?
    115. 115. Data DirectorCloneDB NetAppEMCDelphixSMUZFSFlawedLimited UseManuallow touch: virtualizationAutomated (easy)Manual (Hard)FullFunctionalityDoesn’twork
    116. 116. Application Life Cycle ManagementNumber of required clones change during Project and Extended Project Life CycleProjects with phased Go-Lives can have Implementation, Handover ‘Cut-Over’ andOperations ‘RUN’ all running concurrently.
    117. 117. Development & maintenance is hardUserAcceptanceTestTestDevelopmentProductionTraining &EducationProductionSupportNew release / upgrades lifecycle…Bespoke, enhancements, configuration changes etc…Production Support lifecycle…Internal bug fixes, 3rd Party Patches, Data fixes etc..UserAcceptanceTestTestDevelopmentTraining &EducationProductionSupportMultiple Clones used by Production SupportAnalysts, DBA team and for internal trainingDBA/Note: All environments periodically refreshed from production
    118. 118. Database’s not just used by developersData Migration TeamData Mapping & CleansingDevelopment of ETL(Extraction, Transformation& Load) Routines…UserAcceptanceTestUserAcceptanceTestSystemValidationTestingDataMigrationDataMigrationDataMigrationTechnical TeamsSystem IntegrationEnterprise SchedulingAutomated TestingPerformance Testing…Application FunctionalSpecialistsDevelopment of SystemSet-upBusiness ProcessConfigurationSandbox…Business Process OwnersApplication FunctionalProcess validationCRP (Conference RoomPilot)…UserAcceptanceTestUserAcceptanceTestUserAcceptanceTestAutomatedTestingAutomatedTestingAutomatedTesting
    119. 119. Source TimeflowSource TimeflowSource DatabaseTarget HostVirtual DatabaseNFSFiberFiberRMAN overTCP

    ×