Uploaded byConnor McDonald
OakTable World Sep14 clonedb
This document discusses using Oracle Database's block change tracking and direct NFS features to enable fast cloning of databases for development and testing purposes at low cost. Block change tracking allows incremental backups to be performed quickly, while direct NFS allows database files to be copied over the network efficiently to create clones that only require storage for changed blocks. Examples are provided demonstrating how this can be used to regularly clone a production database to multiple developer environments.
More Related Content
Similar to OakTable World Sep14 clonedb
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
OakTable World Sep14 clonedb
- 1. @connor_mc_d Connor McDonald OracleDBA co.uk connormcdonald.wordpress.com
- 4.
- 5.
- 6.
- 7.
- 8.
- 10.
- 11. everyone wants 11 goodata
- 12. 12 SQL> select* from CUSTOMERS; CID FORENAME SURNAME DOB -------- --------------- ----------------- --------- 101230 Sarah Johnson 01-JUN-75 101233 Michael Smith 13-FEB-69 101237 Peter Brown 24-OCT-82 SQL> select * from CUSTOMERS; CID FORENAME SURNAME DOB -------- --------------- ----------------- --------- 101230 Sarah Johnson 01-JUN-75 101252 MyTestGuy DoesümlautWork 01-JAN-01 101268 Spaces 01-JAN-01 prod dev
- 13. easy to do 13
- 14. 14 RMAN> DUPLICATEDATABASE TO myclone 2 FROM ACTIVE DATABASE; prod
- 15.
- 16.
- 17.
- 18.
- 19. clone your database.... 19 once
- 20. everyone wants 20 currednatta
- 21. 21 prod dev nightly prod dev weekly prod dev annually
- 22.
- 23. everyone wants 23 full sdiazetda
- 24. 24 prod dev annually
- 25.
- 26. You are here Hard Place 26
- 27.
- 28.
- 29. 29 procedure MY_LATEST_PROCis begin for i in ( select * from EVERY_PIECE_OF_DATA_WE_HAVE where UNINDEXED_COLUMN = 10 ) loop ... end; SQL> exec MY_LATEST_PROC PL/SQL Procedure completed successfully. Elapsed: 00:00:00.03
- 30.
- 31. everyone wants 31 their doawtna
- 32.
- 33. 33 prod MikeSue Tom Michelle
- 34. what typically happens... 34
- 35. 35 Mike SueTom Michelle dev
- 36. From: Mike To: EVERYBODY !! Which *@&@# changed my code and broke the build ?!?!? 36
- 37. "Database environments areconstantly getting bigger and bigger, and they are increasingly the bottleneck" -Tim Campos, Facebook CIO 37
- 38. a solution isat hand 38
- 39.
- 40.
- 41. let 41 me repeat
- 42. oracle feature thatis free ! 42
- 43.
- 44.
- 45.
- 46. lets talk aboutNFS ... 46
- 47.
- 48. a little backstory... 48
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56. local disk causes... 56
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63. "10 SCSI disksplease" 63
- 64.
- 65.
- 66.
- 67.
- 68.
- 69. ultra 2 SCSI 69
- 70. ultra 2 wideSCSI 70
- 71. ultra wide ubermega super duper SCSI 71
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80. fine in the90s... 80
- 81. 81 FC 1Gb/sec Ethernet 10Mb/sec
- 82. "disks are slow... 82
- 83. ... and fibreis ... 83
- 84.
- 85.
- 86.
- 87.
- 88.
- 89. 2Gb / sec= 256MB / sec 89
- 90. 2 disks ! 90
- 91.
- 92.
- 93. Like I alwayssay, "Everything is a CPU problem." - Kevin Closson 93
- 94.
- 95. "the SAN can'tbe slow" 95
- 96.
- 97.
- 98.
- 99. we need GBper second 99
- 100. for disks ! 100
- 101. flash etc even"worse" 101
- 102. separating server fromdisk 102
- 103. ... can bean expensive mistake 103
- 104.
- 105.
- 106.
- 107.
- 108.
- 109.
- 110.
- 111. "move to codenear to the data" 111
- 112.
- 113. consolidation makes senseif ... 113
- 114. ... storage tieroffers something 114
- 115. business continuance volume WAFL clustering 115 snapshots iSCSI site mirroring
- 116. successful vendors didthis 116
- 117.
- 118. back to NFS 118
- 119. does it "giveme something" ? 119
- 120.
- 121. 1) shared filesystem 121
- 122. RAC / TTS/ parallelism 122
- 123.
- 124.
- 125. FC HBA FCswitch 125
- 126. so why isn'teveryone using it ? 126
- 127.
- 128. 128 1 Gb4 Gb
- 129.
- 130.
- 131.
- 132.
- 133.
- 134.
- 135.
- 136. why direct NFS? 136
- 137. "NFS is slow" 137
- 138.
- 139. 139 oracle OS NFS client cache
- 140.
- 141.
- 142. 142 oracle NFS OS NFS client cache
- 143. direct NFS rocks! 143
- 144. 144 Source: RedHat Enterprise Linux Protocol Performance Comparison with Oracle Database 11g Release 2 NetApp, TR-3932, June 2011
- 145. even if nointerest in cloning... 145
- 146. ... think aboutdirect NFS 146
- 147.
- 148. back to clonedb 148
- 149. direct NFS means... 149
- 150. Oracle controls theI/O 150
- 151. controls where writesgo 151
- 152. and voila !clone db 152
- 153. RMAN imagecopy clonevia direct NFS 153 prod
- 154. "um ..... and?" 154
- 155. 155 RMAN imagecopy clone database prod dNFS COW
- 156. COW = copyon write 156
- 157. clone is theimagecopy 157
- 158. ... but isread write ! 158
- 159. deltas written asCOW 159
- 160. 160 SQL> connect/@myclone Connected. SQL> update EMP 2 set ENAME = 'JOE' 3 where ENAME = 'SUE';
- 161. 161 READ ONLY Ename: SUE JOE clone db
- 162. clone initial size= 162
- 163.
- 164. 164 SQL> select* from EMP;
- 165. 165 block1 block2block3 block4 block5 block1 block3 block5 read only clone
- 166. clone 5 166 clone 2 clone 1 clone 3 clone 4
- 167. how to doit ? 167
- 168. "under the covers"worry 168
- 169. Doc ID 1210656.1 169 perl clonedb.pl
- 170. its not ! 170
- 171. time for ademo 171
- 172.
- 173.
- 174. so what does"clonedb.pl" do ? 174
- 175. generates SQL ...(poorly) 175
- 176. flexible point intime 176 demo2
- 177. and its fast demo3 / demo4 177
- 178.
- 179. 5 x ~20GBdatabases 179 (plus 12c + VM)
- 180.
- 181. still can clone! 181
- 182. just COW dataon dNFS 182
- 183. 183 copy clone prod SAN SAN redo SAN COW
- 184. "but we're onwindows" 184
- 185. still can clone! 185
- 186. direct NFS workson windows 186 as you've just seen
- 187.
- 188.
- 189. dev cdev nightly AIX / RAM 189
- 190.
- 191.
- 192. available disk 192 ( small )
- 193. available licenses 193 ( additional servers )
- 194. x86 the realwinner 194
- 195. every developer ...own database 195 VM on developer PC
- 196.
- 197. 197 LICENSE RIGHTS We grant you a nonexclusive, nontransferable limited license to use the programs only for the purpose of developing, testing, prototyping and demonstrating your application, and not for any other purpose. If you use the application you develop under this license for any internal data processing or for any commercial or production purposes, or you want to use the programs for any purpose other than as permitted under this agreement, you must obtain a production release version of the program by contacting us or an Oracle reseller to obtain the appropriate license.
- 198. seems to beok 198
- 199. cloning ... in12c 199
- 200.
- 201. ... but potentiallyspace hungry 201
- 202.
- 203.
- 204. SQL> create pluggabledatabase MY_CLONE_PDB 204 2 from MY_SOURCE_PDB; Pluggable database created.
- 205. you're done ! 205
- 206. > 1 pluggabledatabase 206
- 207.
- 208.
- 209.
- 210.
- 211.
- 212.
- 213. ZFS appliance NetAppappliance ACFS 213
- 214.
- 215. 215 With theinitialization parameter CLONEDB set to true, snapshot clones of a pluggable database are supported on any ... file systems with Oracle Direct NFS (dNFS) enabled (yet to verify)
- 216. "2 minute clone.... ... 10 hour imagecopy" 216
- 217.
- 218. 1 ) blockchange tracking 218
- 219. full backup =slow 219
- 220.
- 221.
- 222. read every file... every block 222
- 223. "has SCN advanced?" 223 write to backup
- 224.
- 225. 225 SQL> alterdatabase enable block change tracking 2 using file 'c:oracledb112bct.dat' reuse; Database altered.
- 226. very low risk 226
- 227.
- 228. cost = zero 228
- 229. cost = "zero" 229
- 230.
- 231. lots of blockchanges ... 231
- 232. .. lots ofredo 232
- 233.
- 234. load CTWR versusLGWR 234
- 235.
- 236.
- 237. CTWR and CKPT 237
- 238. 238 you buffercache ckpt ctwr bct.dat dbwr
- 239.
- 240. 240 SQL> select* from v$sgastat 2 where name like '%change%'; POOL NAME BYTES ------------ -------------------------- ---------- shared pool change notification obj m 16416 shared pool change tracking state cha 2584 shared pool change notification regis 16416 shared pool change tracking recovery 262144
- 241. x$krc... 241 X$KRCFH X$KRCEXT X$KRCCDE X$KRCCDS X$KRCCDR X$KRCGFE X$KRCFDE X$KRCFBH X$KRCBIT X$KRCSTAT
- 242.
- 243. 243 datafile 1 datafile 2 datafile 3 BCT 32k 32k 32k
- 244.
- 245. 245 SQL> SELECT(count(distinct b.fno||' '||bno) * 32)/1024 MB 2 FROM x$krcbit b, 3 (SELECT MIN(ver) min_ver, fno 4 FROM 5 (SELECT curr_vercnt ver,curr_highscn high, 6 curr_lowscn low,fno 7 FROM x$krcfde 8 UNION ALL 9 SELECT vercnt ver, high, 10 low, fno 11 FROM x$krcfbh 12 ) 13 WHERE 14 (SELECT MAX(bd.checkpoint_change#) 15 FROM v$backup_datafile bd 16 WHERE bd.incremental_level <= 1) between low and high 17 GROUP BY fno ) sub 18 WHERE b.fno = sub.fno 19 AND b.vercnt >= sub.min_ver; MB ---------- 46.34375
- 246. detailed info... 246 http://www.pythian.com/documents/Pythian-oracle-block-change.pdf
- 247.
- 248. 248 C:oracle11.2.0.3BIN>rman target/ Recovery Manager: Release 11.2.0.3.0 - Production on ... Copyright (c) 1982, 2011, Oracle and/or its affiliates.. connected to target database: DB112 (DBID=730732064) RMAN> RUN { 2> allocate channel c1 type disk format 'c:oraclebkp%N.lev0'; 3> 4> BACKUP INCREMENTAL LEVEL 0 DATABASE; 5> 6> } using target database control file instead of recovery catalog allocated channel: c1 channel c1: SID=190 device type=DISK
- 249. 249 Starting backupat 09/11/2014 13:58:15 channel c1: starting datafile copy input datafile file number=00005 name=C:ORACLEORADATADB112DATAFILEO1_MF_DEMO_95F7ZWNP_.DBF output file name=C:ORACLEBKPDEMO.LEV0 ... channel c1: datafile copy complete, elapsed time: 00:03:15 ... Finished backup at 09/11/2014 14:02:41 released channel: c1 RMAN>
- 250. 4 minutes 36seconds 250 20GB
- 251.
- 252. 2 ) rollingrecovery 252
- 253.
- 254. ... but whatabout recovery 254
- 255. 255 20GB MonTue Wed Thu Fri
- 256.
- 257.
- 258. recover from abackup 258 10g+
- 259. 259 RMAN> RECOVERCOPY OF DATABASE WITH TAG 'rolling';
- 260. 260 20GB MonTue Wed Thu Fri
- 261.
- 262. 262 RMAN> RECOVERCOPY OF DATABASE WITH TAG 'rolling' UNTIL TIME 'sysdate-3';
- 263. 263 RMAN> RUN{ RECOVER COPY OF DATABASE WITH TAG 'rolling'; RECOVER COPY OF DATABASE WITH TAG 'day3' UNTIL TIME 'sysdate-3'; RECOVER COPY OF DATABASE WITH TAG 'day7' UNTIL TIME 'sysdate-7'; }
- 264. bringing it alltogether 264
- 265. prod only once! bkp 3day 7day incrementals 265
- 266. 266 prod bkp3day 7day
- 267. easy point intime copies 267 fast backup unlimited clones
- 268.
- 269. let 269 me repeat
- 270. oracle feature thatis free ! 270
- 271. once you seethe benefits 271
- 272.
- 273.
- 274.
- 275. Connor McDonald OracleDBA co.uk 275
- 276. 276 @connor_mc_d ORA-03113 connormcdonald.wordpress.com