"RAC 11g, Virtualization and More..."


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

"RAC 11g, Virtualization and More..."

  1. 1. Virtualization, Oracle RAC 11g and more.. Vijay Ragunathan Guidewire Software Inc [email_address]
  2. 2. Agenda <ul><li>Introduction </li></ul><ul><li>Set up VM and RAC </li></ul><ul><li>Take a look at a few 11g RAC/ASM features </li></ul><ul><li>Introduce Cache fusion and trace Cache Fusion </li></ul>
  3. 3. Virtualization <ul><li>Abstraction layer between applications and hardware </li></ul><ul><li>Methodology to divide the resources of a computer hardware into multiple operating environments </li></ul><ul><li>Partition at various levels like Storage, Network, Server, Application </li></ul>
  4. 4. Why Virtualize <ul><li>Consolidate IT infrastructure </li></ul><ul><li>Improve QoS </li></ul><ul><li>Reduce cost </li></ul><ul><li>Provide more reliable working environment for development and testing </li></ul><ul><li>Ease replication of environments </li></ul><ul><li>Partition applications to get security, reliability and flexibility </li></ul>
  5. 5. Where can we Virtualize <ul><li>Development and Test </li></ul><ul><li>Virtual Desktops </li></ul><ul><li>PoC deployments </li></ul><ul><li>Disaster Recovery and Business continuity </li></ul><ul><li>Technology refresh </li></ul>
  6. 6. Evolution <ul><li>Virtualization in place for three decades </li></ul><ul><li>Mainframes </li></ul><ul><li>RISC based systems </li></ul><ul><li>And now commodity hardware </li></ul>
  7. 7. Commodity Virtualization Software <ul><li>VmWare </li></ul><ul><li>Xen </li></ul><ul><li>Oracle VM (Xen) </li></ul><ul><li>Microsoft Virtual PC </li></ul><ul><li>And probably many more… </li></ul>
  8. 8. Oracle Real Application Cluster <ul><li>Cluster is two or more nodes sharing some common resources </li></ul><ul><li>They communicate with each other to check health and guarantee data integrity </li></ul><ul><li>For Oracle, it is two or more Oracle instances sharing a physical database </li></ul><ul><li>Each instance use their own SGA and background processes </li></ul><ul><li>Database components like Datafiles, control files and redo log files are shared </li></ul>
  9. 9. Real World Oracle RAC Implementations <ul><li>RISC or CISC based processers on two or more servers </li></ul><ul><li>A SAN or NAS for storing database </li></ul><ul><li>Interconnect mechanisms like Infiniband, Gigabit Ethernet, Memory channel </li></ul>
  10. 10. Node1 Node2 Storage Real Application Clusters
  11. 11. VM and Oracle RAC <ul><li>Hardware for clustering too costly </li></ul><ul><li>Difficult to always build dev and test environments with clustering </li></ul><ul><li>Typically Applications built on single instance and tested and deployed on RAC </li></ul><ul><li>VM is Beneficial for small, medium and large companies and cost effective </li></ul>
  12. 12. Virtual Node1 VirtualNode2 Same Disks in Host Real Application Clusters Physical Machine
  13. 13. Setup Xen
  14. 14. Xen Architecture <ul><li>Xen hypervisor has management and virtual hardware API </li></ul><ul><li>Dom0 hosts the management code and has control interfaces to manage other domains </li></ul><ul><li>All guest Virtual machines are referred as domU </li></ul>
  15. 15. Packages <ul><li>xen-3.0.3-41.el5 </li></ul><ul><li>kernel-xen-2.6.18-53.el5 </li></ul><ul><li>xen-libs-3.0.3-41.el5 </li></ul><ul><li>kernel-xen-devel-2.6.18-53.el5 </li></ul>
  16. 16. Dom0 Host <ul><li>Default Centos 5.1 install </li></ul><ul><li>Virtual Machine Manager GUI </li></ul><ul><li>Command line tool virt-install </li></ul>
  17. 17. Virtual Machine Manager <ul><li>The GUI VMM displays all the domains and their resource usage </li></ul>
  18. 18. Create a VM <ul><li>File -> New Machine </li></ul>
  19. 19. Name the VM <ul><li>We name the first VM as dtrac41 </li></ul>
  20. 20. Virtualization Method
  21. 21. ParaVirtualization <ul><li>ParaVirtualization uses a modified kernel </li></ul><ul><li>Not very portable </li></ul><ul><li>Not all hardware is simulated </li></ul><ul><li>Best performing for Network and IO </li></ul><ul><li>Unique address map for each VM </li></ul><ul><li>Easy to implement </li></ul>
  22. 22. Fully Virtualized <ul><li>Complete Hardware simulation </li></ul><ul><li>Any supported guest OS can be used </li></ul><ul><li>Not supported by all hardware architecture </li></ul><ul><li>VT support may need to be enabled in BIOS for some hardware </li></ul><ul><li>Use techniques to trap and emulate instructions </li></ul>
  23. 23. Install using PXE <ul><li>PXE server enables storing images in central location and install large number of machines </li></ul><ul><li>Using centos 5.1 for guestVM </li></ul><ul><li>Kickstart Template file instructs what to install </li></ul>
  24. 24. OS and Swap for GuestVM <ul><li>Using simple files </li></ul><ul><li>Default path /var/lib/xen/images </li></ul>
  25. 25. Network for the GuestVM
  26. 26. Memory and CPU <ul><li>Do not over allocate as we need resources for the dom0 Host and other GuestVMs </li></ul>
  27. 27. Summary of the Options and Resources
  28. 28. Vm Installation operations <ul><li>VMM will start allotting the disk storage </li></ul><ul><li>Create virtual network </li></ul><ul><li>Map memory to the Dom0 Virtual memory map </li></ul><ul><li>Will start installing the Guest OS </li></ul>
  29. 29. PXE install looks at dhcp server
  30. 30. Xen Utilities <ul><li>xm utility </li></ul><ul><li>- can be used to create domains </li></ul><ul><li>- useful for cloning </li></ul><ul><li>virsh </li></ul><ul><li>- can be used to change CPU, memory </li></ul><ul><li>- define config, dump config in xml form for </li></ul><ul><li> cloning </li></ul><ul><li>- start, shutdown and reboot domains </li></ul>
  31. 31. Xen Utilities <ul><li>xentop </li></ul><ul><li>- similar to top but customized to list all virtual resources and domains </li></ul><ul><li>xentop - 21:55:32 Xen 3.1.0-53.el5 </li></ul><ul><li>3 domains: 2 running, 1 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown </li></ul><ul><li>Mem: 15723456k total, 15721284k used, 2172k free CPUs: 8 @ 2992MHz </li></ul><ul><li>NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD_WR SSID </li></ul><ul><li>Domain-0 -----r 9576 2.9 13498460 85.8 no limit n/a 8 4 398061 38880 0 0 0 0 0 </li></ul><ul><li>dtrac41 -----r 2755 2.1 1048152 6.7 1048576 6.7 1 2 883749 857104 0 0 0 0 0 </li></ul><ul><li>dtrac42 --b--- 1302 8.1 1048080 6.7 1048576 6.7 1 2 850349 813954 0 0 0 0 0 </li></ul>
  32. 32. XenStore <ul><li>Configuration database for all domain </li></ul><ul><li>Multiple commands to manage this database </li></ul><ul><li>xenstore-ls </li></ul><ul><li>- can be used to list details of all hardware components in each domain </li></ul>
  33. 33. Add Virtual Hardware <ul><li>Add more storage for Oracle clusterware </li></ul>
  34. 34. Add Virtual Storage <ul><li>Specify Simple file or disk partition (if available) </li></ul><ul><li>Min size of 256Mb for Clusterware </li></ul>
  35. 35. Add virtual network interface <ul><li>For Interconnect use </li></ul>
  36. 36. Add virtual network interface <ul><li>Choose virtual network as this interconnect does not need to talk to the external world </li></ul>
  37. 37. Virtual hardware of one guestVM
  38. 38. Oracle Install Preparation <ul><li>Update sysctl.conf with the new kernel parameters </li></ul><ul><li>rmem_default and wmem_default should be 4MB min </li></ul><ul><li>Update limits.conf </li></ul><ul><li>Install any additional RPMs needed as listed in Oracle Install documentation </li></ul><ul><li>Edit /etc/hosts and add localhost and 6 other addresses to indicate public, private and virtual networks for both guestVMs </li></ul>
  39. 39. Create users and groups <ul><li>groupadd –g 10000 oinstall </li></ul><ul><li>groupadd –g 10001 dba </li></ul><ul><li>groupadd –g 10002 asmdba </li></ul><ul><li>useradd –g oinstall –Goinstall,dba,asmdba –d /home/oracle –u 10000 oracle </li></ul><ul><li>passwd oracle </li></ul>
  40. 40. Mapping raw devices <ul><li>Create something like /etc/rc3.d/S90raw and/or /etc/rc5.d/S90raw based on boot run level with contents like: </li></ul><ul><li>raw /dev/raw/raw1 /dev/xvdb </li></ul><ul><li>raw /dev/raw/raw2 /dev/xvdc </li></ul><ul><li>raw /dev/raw/raw3 /dev/xvdd </li></ul><ul><li>raw /dev/raw/raw4 /dev/xvde </li></ul><ul><li>chown root:dba /dev/raw/raw2 </li></ul><ul><li>chmod 660 /dev/raw/raw2 </li></ul><ul><li>chown oracle:dba /dev/raw/raw1 </li></ul><ul><li>chmod 660 /dev/raw/raw1 </li></ul><ul><li>chown oracle /dev/raw/raw3 </li></ul><ul><li>chown oracle /dev/raw/raw4 </li></ul><ul><li>chmod 660 /dev/raw/raw3 </li></ul><ul><li>chmod 660 /dev/raw/raw4 </li></ul>
  41. 41. Cloning GuestVM <ul><li>In dom0, /etc/xen/dtrac41 has list of devices created for this guest VM </li></ul><ul><li>Copy /etc/xen/dtrac41 to /etc/xen/dtrac42 </li></ul><ul><li>Change the mac address of the two virtual network interfaces in dtrac42 and edit the primary disk </li></ul><ul><li>Modify the name and uuid for the new VM </li></ul><ul><li>Ensure the order of the network is right in the config files </li></ul><ul><li>Copy the virtual disk of dtrac41(dtrac41.img) to /var/lib/xen/images/dtrac42.img </li></ul>
  42. 42. Config file for guestVM <ul><li>name = &quot;dtrac41&quot; </li></ul><ul><li>uuid = &quot;3656ca7c24e378bd2c63d380206a5f8d&quot; </li></ul><ul><li>maxmem = 1024 </li></ul><ul><li>memory = 1024 </li></ul><ul><li>vcpus = 1 </li></ul><ul><li>bootloader = &quot;/usr/bin/pygrub&quot; </li></ul><ul><li>kernel = &quot;/var/lib/xen/boot_kernel.qFE_y1&quot; </li></ul><ul><li>ramdisk = &quot;/var/lib/xen/boot_ramdisk.HlhabB&quot; </li></ul><ul><li>extra = &quot;ro root=/dev/vgSys/lvRoot rhgb quiet console=tty0 console=ttyS1,57600n8&quot; </li></ul><ul><li>on_poweroff = &quot;destroy&quot; </li></ul><ul><li>on_reboot = &quot;restart&quot; </li></ul><ul><li>on_crash = &quot;restart&quot; </li></ul><ul><li>vfb = [ &quot;type=vnc,vncdisplay=0,keymap=en-us&quot; ] </li></ul><ul><li>disk = [ &quot;tap:aio:/var/lib/xen/images/dtrac41.img,xvda,w&quot;, &quot;tap:aio:/var/lib/xen/images/dtrac4x-ocr11g-shared.img,xvdb,w&quot;, &quot;tap:aio:/var/lib/xen/images/dtrac4x-vote-shared.img,xvdc,w&quot;, &quot;tap:aio:/var/lib/xen/images/dtrac4x-asm1-shared.img,xvdd,w&quot;, &quot;tap:aio:/var/lib/xen/images/dtrac4x-asm2-shared.img,xvde,w&quot; ] </li></ul><ul><li>vif = [ &quot;mac=00:16:3e:0c:81:f5,bridge=xenbr0,script=vif-bridge&quot;, &quot;mac=00:16:3e:75:41:84,bridge=virbr0,script=vif-bridge&quot; ] </li></ul>
  43. 43. Cloning GuestVM <ul><li>Use xm to create the domain </li></ul><ul><li>xm create dtrac42 </li></ul><ul><li>Start the new domain </li></ul><ul><li>xm start dtrac42 </li></ul><ul><li>You should see both the GuestVMs in the Virtual Machine Manager </li></ul>
  44. 44. Setup GuestVMs <ul><li>Reboot both the virtual machines </li></ul><ul><li>Ensure fdisk –l is showing all the disks </li></ul><ul><li>Edit the network configuration files to setup the IP addresses and hostname and start the network </li></ul><ul><li>Ping the IP addresses of the other node and confirm that the network works fine </li></ul>
  45. 45. User equivalence Setup <ul><li>Use ssh-keygen and generate keys </li></ul><ul><li>Concatenate the public keys for both machines and store as ~/.ssh/authorized_keys </li></ul><ul><li>Check you can login to all combinations without prompts. For e.g., </li></ul><ul><li>ssh dtrac41 date </li></ul><ul><li>ssh dtrac42 date </li></ul><ul><li>ssh dtrac41p date </li></ul><ul><li>ssh dtrac42p date </li></ul><ul><li>Run cluster verify tool and do pre CRS inst check </li></ul>
  46. 46. Vmware <ul><li>Install Vmware server </li></ul><ul><li>Create two virtual machines using Vmware Server Console </li></ul><ul><li>For the shared disks, edit the config files (suffixed vmx) and add: </li></ul><ul><li>disk.locking = &quot;false“ </li></ul><ul><li>scsi0:1.shared = &quot;true&quot; </li></ul><ul><li>scsi0:2.shared = &quot;true&quot; </li></ul><ul><li>scsi0:3.shared = &quot;true&quot; </li></ul><ul><li>scsi0:4.shared = &quot;true&quot; </li></ul>
  47. 47. Oracle install media <ul><li>Download the clusterware and database software </li></ul><ul><li>Unzip the files </li></ul><ul><li>unzip linux_11gR1_clusterware.zip </li></ul><ul><li>unzip linux_11gR1_database.zip </li></ul><ul><li>If using cdrom/dvd, mount the clusterware media first </li></ul><ul><li>Start the Installer </li></ul><ul><li>cd clusterware </li></ul><ul><li>./runInstaller </li></ul>
  48. 48. Clusterware Software Home
  49. 49. Cluster Configuration <ul><li>Cluster name is dtrac4_cluster </li></ul><ul><li>Edit details of dtrac41 </li></ul><ul><li>Add details of dtrac42 </li></ul>
  50. 50. Network Interface Usage <ul><li>Select the right Interface for use as Public and Private network </li></ul><ul><li>Here eth0 is chosen as Public and eth1 as private </li></ul>
  51. 51. OCR and Voting disk <ul><li>Both OCR in the previous installer screen and voting disk here should have min of 256MB space </li></ul><ul><li>Specify mirror copies – here we use single voting disk location </li></ul>
  52. 52. Clusterware Install Summary <ul><li>Ensure all cluster nodes are seen </li></ul>
  53. 53. Root Scripts <ul><li>Run orainstRoot.sh and root.sh on all nodes (one at a time) </li></ul>
  54. 54. Root.sh on node1 <ul><li>Configures OCR and Voting Disk and starts CRS stack on node1 </li></ul>
  55. 55. Root.sh on node2 <ul><li>Starts CRS stack on node2 and creates cluster resources </li></ul>
  56. 56. Config Assistants for Clusterware
  57. 57. Clusterware Install Completion <ul><li>That completes the clusterware install </li></ul><ul><li>OCR and Voting disk are configured </li></ul><ul><li>Resources like ONS and VIP should be created by now and can be seen using crs_stat </li></ul><ul><li>Processes like evmd, crsd, ocssd and oprocd should be running now </li></ul><ul><li>Ready to Install RAC software </li></ul><ul><li>cd database </li></ul><ul><li>./runInstaller </li></ul>
  58. 58. Installation Type
  59. 59. Node Selection <ul><li>Choose all nodes in the cluster – both VMs </li></ul>
  60. 60. OS Authentication
  61. 61. RAC Install Summary <ul><li>Check for selection of all Cluster nodes </li></ul>
  62. 62. Create RAC database
  63. 63. Create ASM Disk Group <ul><li>Change disk discovery path and check permissions if disks are not seen </li></ul>
  64. 64. Starts Cluster database <ul><li>Database Creation is complete </li></ul>
  65. 65. CRS status <ul><li>crs_stat -t </li></ul><ul><li>Name Type Target State Host </li></ul><ul><li>------------------------------------------------------------ </li></ul><ul><li>ora....SM1.asm application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....41.lsnr application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....c41.gsd application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....c41.ons application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....c41.vip application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....SM2.asm application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora....42.lsnr application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora....c42.gsd application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora....c42.ons application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora....c42.vip application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora.xenrac.db application ONLINE ONLINE dtrac42 </li></ul><ul><li>ora....c1.inst application ONLINE ONLINE dtrac41 </li></ul><ul><li>ora....c2.inst application ONLINE ONLINE dtrac42 </li></ul>
  66. 66. RAC and ASM 11g New Features
  67. 67. 11g New Features <ul><li>Automatic Diagnostic Repository should be placed in shared disk if using shared oracle home </li></ul><ul><li>Kill sessions from anywhere in the cluster </li></ul><ul><li>ADDM for RAC – dbms_addm.analyze_db procedure and dbms_addm.get_report function </li></ul><ul><li>Inter-node parallelism can use services to determine the instances that it can use </li></ul><ul><li>Grid provisioning and EM changes </li></ul><ul><li>Clone ASM and RAC homes – clone.pl </li></ul>
  68. 68. 11g ASM New Features <ul><li>asm_preferred_read_failure_groups parameter </li></ul><ul><li>ASM rolling migration </li></ul><ul><li>– alter system start rolling migration to <pnumber>; </li></ul><ul><li>Convert single instance ASM to cluster using rconfig/EM </li></ul><ul><li>ASM compatibility attributes </li></ul><ul><li>Variable size extents </li></ul><ul><li>Higher AU values for Coarse striping 1M – 64M </li></ul>
  69. 69. 11g ASM New features <ul><li>Ability to drop ASM instances using dbca </li></ul><ul><li>dbca –silent –deleteasm –nodelist dtrac41,dtrac42 </li></ul><ul><li>ASM fast mirror resync </li></ul><ul><li>SYSASM and OSASM </li></ul><ul><li>ASMcmd command additions – cp, md_backup, lsdsk, remap </li></ul><ul><li>Mount/drop diskgroup with Force option </li></ul>
  70. 70. Cache Fusion
  71. 71. Cache Fusion <ul><li>Shared cache architecture </li></ul><ul><li>Cache coherency – preserve integrity of data across different caches for the same data </li></ul><ul><li>Employs high-speed Interconnects </li></ul><ul><li>Created in Oracle OPS 8i to reduce pings </li></ul><ul><li>8i handled read/write contention </li></ul><ul><li>9i RAC handles other contentions like write-write </li></ul>
  72. 72. Cache Fusion <ul><li>Locks – mode, role, past image </li></ul><ul><li>E.g lock : SG1 </li></ul><ul><li>Modes: Null, Shared, Exclusive </li></ul><ul><li>Role: Local, Global </li></ul><ul><li>Past Image component: 0 or 1 </li></ul><ul><li>Maximum 3-way protocol (master, holder and requestor) </li></ul>
  73. 73. Events <ul><li>Event 10432 – for diagnosing unexpected GCS events and lock requests </li></ul><ul><li>Event 10708 – shows block request and grants (level 7 or 103 is common) </li></ul><ul><li>Event 10046 – common sql trace and combines well with event 10708 </li></ul>
  74. 74. Trace commands <ul><li>SQL> alter system set events '10432 trace name context forever, level 15:10708 trace name context forever, level 103'; </li></ul><ul><li>System altered. </li></ul><ul><li>SQL> conn scott/tiger </li></ul><ul><li>Connected. </li></ul><ul><li>SQL> alter session set events '10046 trace name context forever, level 8'; </li></ul><ul><li>Session altered. </li></ul>
  75. 75. First Read session <ul><li>Select on node 1 </li></ul><ul><li>*** 2008-05-15 02:40:36.228 </li></ul><ul><li>kclscrs: req=0 block=5/36 </li></ul><ul><li>*** 2008-05-15 02:40:36.228 kclscrs: req=0 typ=immediate mode=S </li></ul><ul><li>WAIT #3: nam='db file sequential read' ela= 786 file#=5 block#=36 blocks=1 obj#=16215 tim=1210844436231630 </li></ul><ul><li>*** 2008-05-15 02:40:36.231 KCL: P02: cr prewarm </li></ul><ul><li>*** 2008-05-15 02:40:36.231 </li></ul><ul><li>kclscrs: req=0 block=5/28 </li></ul><ul><li>*** 2008-05-15 02:40:36.231 kclscrs: req=0 typ=immediate mode=S </li></ul><ul><li>*** 2008-05-15 02:40:36.231 </li></ul><ul><li>kclscrs: req=1 block=5/29 </li></ul><ul><li>*** 2008-05-15 02:40:36.231 kclscrs: req=1 typ=immediate mode=S </li></ul><ul><li>… </li></ul><ul><li>*** 2008-05-15 02:40:36.232 </li></ul><ul><li>kclscrs: req=7 block=5/25 </li></ul><ul><li>*** 2008-05-15 02:40:36.232 </li></ul><ul><li>kclscrs: req=7 typ=immediate mode=S </li></ul><ul><li>WAIT #3: nam='db file scattered read' ela= 7342 file#=5 block#=25 blocks=8 obj#=16214 tim=1210844436239474 </li></ul>
  76. 76. Second read from same node <ul><li>PARSE #9:c=0,e=1478,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=1,tim=1210847692139409 </li></ul><ul><li>EXEC #9:c=0,e=37,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=1210847692139492 </li></ul><ul><li>WAIT #9: nam='SQL*Net message to client' ela= 1 driver id=1650815232 #bytes=1 p3=0 obj#=523 tim=1210847692139514 </li></ul><ul><li>FETCH #9:c=0,e=29,p=0,cr=2,cu=0,mis=0,r=1,dep=0,og=1,tim=1210847692139568 </li></ul><ul><li> No communication to the other node </li></ul>
  77. 77. Write Operation Tracing.. <ul><li>update emp set sal=sal+100 where empno = 7654 </li></ul><ul><li>END OF STMT </li></ul><ul><li>PARSE #3:c=20001,e=44395,p=1,cr=55,cu=0,mis=1,r=0,dep=0,og=1,tim=1210848457075954 </li></ul><ul><li>WAIT #3: nam='enq: TM - contention' ela= 226 name|mode=1414332419 object #=16214 table/partition=0 obj#=-1 tim=1210848457076262 </li></ul><ul><li>*** 2008-05-15 03:47:37.076 </li></ul><ul><li>kclscrs: req=0 block=5/36 </li></ul><ul><li>*** 2008-05-15 03:47:37.076 </li></ul><ul><li>kclscrs: bid=1:3:2:0:2:c:0:0:2:0:0.. </li></ul>
  78. 78. Tracing Contd.. <ul><li>Lms trace of remote node </li></ul><ul><li>*** 2008-05-15 03:47:37.077 </li></ul><ul><li>kcl_snd_cur: block=5/36 time=54 </li></ul><ul><li>Back to update session trace </li></ul><ul><li>*** 2008-05-15 03:47:37.077 </li></ul><ul><li>kclwcrs: got 1 blocks from ksxprcvimd </li></ul><ul><li>WAIT #3: nam='gc current block 2-way' ela= 413 p1=5 p2=36 p3=1 obj#=16215 tim=1210848457077694 </li></ul><ul><li>*** 2008-05-15 03:47:37.077 </li></ul><ul><li>kclwcrs: req=0 typ=cur wtyp=2hop tm=489 </li></ul>
  79. 79. Tracing Contd.. <ul><li>*** 2008-05-15 03:47:37.140 </li></ul><ul><li>kcllkopb: req=7 block=5/25 </li></ul><ul><li>*** 2008-05-15 03:47:37.140 </li></ul><ul><li>kcllkopb: bid=1:3:2:0:2:14:0.. </li></ul><ul><li>WAIT #3: nam='gc current multi block request' ela= 27265 file#=5 block#=25 id#=33554446 obj#=16214 tim=1210848457226586 </li></ul><ul><li>Nothing in the LMS trace </li></ul><ul><li>Back to sessions trace </li></ul><ul><li>*** 2008-05-15 03:47:37.254 </li></ul><ul><li>kcllkopb: req=0 typ=cur wtyp=2hop tm=174615 </li></ul><ul><li>WAIT #3: nam='db file scattered read' ela= 83230 file#=5 block#=25 blocks=3 obj#=16214 tim=1210848457337661 </li></ul>
  80. 80. Causing Write-Write Contention <ul><li>update emp set sal=sal+100 where empno != 7654 </li></ul><ul><li>PARSE #5:c=4000,e=49748,p=1,cr=9,cu=0,mis=1,r=0,dep=0,og=1,tim=1210848625816852 </li></ul><ul><li>*** 2008-05-15 03:50:25.817 </li></ul><ul><li>kclscrs: req=0 block=5/28 </li></ul><ul><li>*** 2008-05-15 03:50:25.817 </li></ul><ul><li>kclscrs: bid=1:3:1:0:2:5:0:0:4:0:.. </li></ul>
  81. 81. Write-Write Contention Tracing.. <ul><li>Remote LMS </li></ul><ul><li>*** 2008-05-15 03:50:25.816 </li></ul><ul><li>kclgcr: req=231 block=5/28 </li></ul><ul><li>*** 2008-05-15 03:50:25.816 </li></ul><ul><li>kclgcr: bid=1:3:1:0:2:5:0:0.. </li></ul><ul><li>*** 2008-05-15 03:50:25.825 </li></ul><ul><li>kclqsnd: req=231 status=2 fscn=0:b49c2 time=8523 </li></ul><ul><li>*** 2008-05-15 03:50:25.826 </li></ul><ul><li>kclpdc: le=1c/50000 rls=49 </li></ul>
  82. 82. Write-Write Contention Tracing.. <ul><li>Back to local session </li></ul><ul><li>*** 2008-05-15 03:50:25.825 </li></ul><ul><li>kclwcrs: got 1 blocks from ksxprcvimd </li></ul><ul><li>WAIT #5: nam='gc cr block busy' ela= 8846 p1=5 p2=28 p3=1 obj#=16214 tim=1210848625825992 </li></ul><ul><li>*** 2008-05-15 03:50:25.825 </li></ul><ul><li>kclwcrs: got 1 blocks from ksxprcvimd </li></ul><ul><li>WAIT #5: nam='gc cr block busy' ela= 8846 p1=5 p2=28 p3=1 obj#=16214 tim=1210848625825992 </li></ul><ul><li>*** 2008-05-15 03:50:25.826 </li></ul><ul><li>kclwcrs: req=0 typ=cr wtyp=busy tm=9178 </li></ul>
  83. 83. Write - Write contention tracing <ul><li>*** 2008-05-15 03:50:25.826 </li></ul><ul><li>kcllkopb: req=0 block=5/28 </li></ul><ul><li>*** 2008-05-15 03:50:25.826 </li></ul><ul><li>kcllkopb: bid=1:3:1:0:2:6:0:0: </li></ul><ul><li>LMS trace </li></ul><ul><li>*** 2008-05-15 03:50:25.826 </li></ul><ul><li>kcl_snd_cur: block=5/28 time=64 </li></ul><ul><li>Local Session Trace </li></ul><ul><li>*** 2008-05-15 03:50:25.827 </li></ul><ul><li>kcllkopb: got 1 blocks from ksxprcvimd </li></ul><ul><li>WAIT #5: nam='gc current block 2-way' ela= 270 p1=5 p2=28 p3=33554433 obj#=16214 tim=1210848625827163 </li></ul><ul><li>*** 2008-05-15 03:50:25.827 </li></ul><ul><li>kcllkopb: req=0 typ=cur wtyp=2hop tm=417 </li></ul>
  84. 84. Q& A