Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair

1. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Oracle RAC 12c Release 2 and Cluster Architecture Internals Anil Nair, Ian Cookson -‐ Oracle RAC Product Management -‐ Oracle Dave Hickson – Database Architect – BT October 3, 2017

2. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direcRon. It is intended for informaRon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcRonality, and should not be relied upon in making purchasing decisions. The development, release, and Rming of any features or funcRonality described for Oracle’s products remains at the sole discreRon of Oracle. 3

5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle RAC 12c Rel. 2 Cluster Domain •  Simplified Management – Fleet Management for installaRon, update, patching and maintenance •  Reduced Local Overhead – Member Clusters benefit from the consolidaRon of common services on the Domain Services Cluster •  Improved IO Performance – URlizing consolidated shared storage 6 Centralized Management for Cluster Estates “too big to manage” otherwise

6. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | 7 Cluster Domain Applica@on Member Cluster Uses ASM Database Member Cluster Uses local ASM Database Member Cluster Uses ASM Service Database Member Cluster Uses IO Service Domain Services Cluster (DSC) Shared ASM IO Service ACFS Service ASM Service TFA Service Management Service RHP Service

7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 8 The DSC – The Heart of the Cluster Domain Domain Services Cluster (DSC) IO Service ASM Service TFA Service Management Service Shared ASM ACFS Service RHP Service •  The DSC hosts services that are consumed by Member Clusters, including: –  Management Service for centralized and simplified management –  Trace File Analyzer (TFA) for centralized diagnosRcs –  Rapid Home Provisioning (RHP) for so`ware fleet management –  Storage Services (ACFS, ASM direct or indirect over IO Service)

8. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Domain Services Cluster (DSC) IO Service ASM Service Shared ASM ACFS Service RHP Service 9 The DSC Management Service TFA Service Management Service Applied Machine Learning for Database Diagnos@cs •  Eﬃcient diagnosis using Machine Learning •  AutomaRcally performs correcRve acRons to prevent possible issues •  Provides simple alerts & recommendaRons for issues that require manual intervenRon Subject MaOer Expert ASH ML Knowledge Extrac@on Model Genera@on Human Supervision Applica@on Op@mized Models Feedback

9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10 Rapid Home Provisioning Service Domain Services Cluster (DSC) IO Service ASM Service Shared ASM ACFS Service RHP Service TFA Service Management Service •  Provision new pools onto base machines •  DB and GI: provision, scale, patch, upgrade •  Custom workflow framework •  NoRficaRon model •  Audit capabiliRes RHP Service Member Cluster Member Cluster Member Cluster Fleet Management installa@on, update, patching and maintenance

10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11 Rapid Home Provisioning Service Domain Services Cluster (DSC) IO Service ASM Service Shared ASM ACFS Service RHP Service TFA Service Management Service •  Provision new pools onto base machines •  DB and GI: provision, scale, patch, upgrade •  Custom workflow framework •  NoRficaRon model •  Audit capabiliRes RHP Service Member Cluster Member Cluster Member Cluster Fleet Management in the Oracle Cloud and On-‐Premise

11. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Zero Impact Patching •  Zero Impact Patching enables patching of the Oracle Grid Infrastructure without interrupRng database operaRons. •  Patches are applied out-‐of-‐place and in a rolling fashion with one node being patched at a Rme while the database instance(s) on that node remain up and running. •  Zero Impact Patching supports Oracle Real ApplicaRon Clusters (RAC) databases on clusters with two or more nodes. 12 Never take down a database instance to patch Grid Infrastructure

12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 13 Domain Services Cluster (DSC) Availability •  Services provided by the DSC are unaffected by nodes joining/ leaving the DSC cluster •  DSC can be patched and upgraded independently without affecRng the services provided by DSC •  Use Member Clusters for user databases (not the DSC)

14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | •  A Member Cluster is a Standalone Cluster uRlizing shared services on the Domain Services Cluster •  It automaRcally benefits from the management, TFA & RHP service. – ASM services are opRonal – and can be uRlized as needed 15 Member Clusters = Standalone Cluster + Benefits Database Member Cluster Uses local ASM Database Member Cluster Uses ASM Service of DSC Database Member Cluster Uses ASM IO Service

15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 16 Same Tools and Commands for all Types of Deployments [GRID]> crsctl get cluster name CRS-‐6724: Current cluster name is 'SolarCluster' [GRID]> crsctl get cluster class CRS-‐41008: Cluster class is 'Standalone Cluster' [GRID]> crsctl get cluster type CRS-‐6539: The cluster type is 'flex'. [GRID]> crsctl get cluster name CRS-‐6724: Current cluster name is 'SalesCluster' [GRID]> crsctl get cluster class CRS-‐41008: Cluster class is ‘Database Member Cluster' [GRID]> crsctl get cluster type CRS-‐6539: The cluster type is 'flex'.

16. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | •  Easy conversion from Standalone to Member Clusters •  Member Cluster GI version can be higher or equal to the GI version on the Domain Services Cluster 17 Convert a Standalone Cluster to Member Cluster Convert Database Member Cluster Uses local ASM

19. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Database Member Cluster with Local ASM •  For databases requiring – Full IsolaRon and performance stability – That can beneﬁt from the centralized Management Service on the DSC •  ParRcularly suitable for unpredictable workloads, or highly variable workloads •  Examples include – Business Intelligence and AnalyRcs systems – Batch processing systems – Response-‐criRcal, user-‐facing systems 20 Standalone isola@on with reduced local overhead Database Member Cluster Uses local ASM

20. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | 21 Cluster Domain Applica@on Member Cluster Uses ASM Database Member Cluster Uses ASM Service Database Member Cluster Uses IO Service Domain Services Cluster (DSC) Shared ASM IO Service ACFS Service ASM Service TFA Service Management Service RHP Service Database Member Cluster Uses local ASM

21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Database Member Cluster Uses ASM Service of DSC Database Member Cluster Using ASM Service •  For databases requiring –  IsolaRon and performance stability –  That can benefit from the centralized Management Service on the DSC –  And the centralized ASM Storage Management Service on the DSC •  Best suited for workloads for which IO stability is important, but benefit from the centralized ASM Services on the DSC •  Examples include –  OLTP systems –  ReporRng systems 22 Standalone isola@on benefihng from consolidated shared storage ASM Service

23. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Database Member Cluster Uses ASM IO Service Database Member Cluster Using the IO Service •  For databases – That need to be deployed quickly – or cloned or duplicated frequently •  Ideal for databases that can allow for IO path sharing with other Member Clusters, while maintaining control of local cluster processing, workload and resources •  Examples include – Test, integraRon, development systems •  For which maintaining addiRonal hardware (e.g. SAN networks and storage) is ineﬃcient 24 Consolida@on at its best u@lizing full resource sharing IO Service ASM Service

25. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | 1010101000010011100101001 1011100101001110010100101 0010010100001111010100101 26 •  Using Oracle MulRtenant, PDBs can be opened as singletons (in one database instance only), in a subset of instances or all in instances at once. •  If certain PDBs are only opened on some instances, Pluggable Database Isola/on –  improves performance by •  Reducing DLM operaRons for PDBs not open in all instances. •  OpRmizing block operaRons based on in-‐memory block separaRon. –  improves availability by •  Ensuring that instance failures of instances only hosRng singleton PDBs will not impact other instances of the same RAC-‐based CDB. Pluggable Database and Service IsolaRon NEW IN 12.2 Inter Cluster Messages

26. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Database Member Cluster Uses ASM Service 27 Pluggable Database and Service IsolaRon Provides higher IsolaRon in Member Cluster Consolidated Environments

27. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Hang Manager • Hang Manager “manages” database process(es) that are not progressing – Manage = detec@on and resolu@on – Considers cross-‐layer hangs •  I.E: Hangs caused by a blocked ASM resource. • In addiRon, deadlocks are now resolved by Hang Manager • User deﬁned control via PL/SQL • Early Warning exposed via (V$ view) 28 Resolving cross-‐layer hangs in the Cluster Domain Database Member Cluster Uses ASM IO Service IO Service ASM Service

28. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | 29 Resolving cross-‐layer hangs in the Cluster Domain Provides higher Availability in Member Cluster Consolidated Environments Database Member Cluster Uses ASM Service W W W B

29. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Oracle RAC Always Chooses the “Right Path” 30 •  RAC determines the opRmal path to serve blocks – network or disk •  SSDs and NVMe storage technology conRnue to drive down latency •  e.g. ﬂash storage may provide beser access Rmes to data than the private network under high load •  RAC takes those staRsRcs into account Query Block Block Network congesRon

30. Copyright © 2017, Oracle and/or its aﬃliates. All rights reserved. | Database Member Cluster Uses ASM Service 31 The “Right Path” Ensures Performance on Member Clusters EﬀecRvely reducing the impact of compeRng workloads overwhelming the I/O on the DSC B u s y

32. BT: Cluster Domains in AcRon Dave Hickson – Database Architect BT October 3, 2017

33. © BriRsh TelecommunicaRons plc About BT •  The UK’s largest broadband provider •  The UK’s largest last-‐mile network provider •  The UK’s largest wide area network provider •  In EE, the UK’s best largest and best mobile network provider •  A global footprint operaRng in 180 countries •  BT Sport delivering Premiership and UEFA football •  All underpinned by Technology, Service and OperaRons

34. © BriRsh TelecommunicaRons plc Context – ExisRng Database and RAC Services in BT •  Large scale, on-‐premise Enterprise Cloud •  Thousands of databases, hundreds of RACs •  Increasing rate of growth •  Lots of automaRon but … •  We need smarter ways to –  Deliver RAC clusters more quickly –  Administrate more eﬃciently –  Enable customer self-‐service •  What we need is a more Cloud-‐oriented RAC architecture

35. © BriRsh TelecommunicaRons plc Cluster Domains – What we’ve been doing •  Test environment on BT Enterprise Cloud –  Four node Domain Services Cluster –  8 Member Clusters –  OEM 13.2 –  VMware-‐based infrastructure •  Key Features we’re interested in (in no par@cular order!) –  IO Server –  Rapid Home Provisioning –  Autonomous Health Framework –  ApplicaRon Containers

36. © BriRsh TelecommunicaRons plc Cluster Domains – Why is this architecture asracRve to us? •  I/O Server –  Replace hundreds of independent pools of storage with centralised pools –  Increase storage on member clusters without infrastructure changes •  Rapid Home Provisioning –  Centralised management of Oracle so`ware for patching and upgrading •  Autonomous Health Framework –  Replace many independent management repositories with one Goal for BT is simple: Reduce Overhead of Many RAC Clusters

37. © BriRsh TelecommunicaRons plc Cluster Domains – our overall impression •  Architecturally this is the right direc@on –  Database servers run databases without having to administer infrastructure –  “Infrastructure” tasks such as storage, performance, so`ware managed centrally •  Applica@on Containers on Member clusters enable customer self-‐service of new databases without overheads of infrastructure management on each cluster •  Organisa@onal implica@ons: –  Infrastructure Team to manage Cluster Domains –  Database Team to manage Member clusters

38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle RAC Cluster Domain Architecture •  Benefit from the Centralized Manageability services •  Benefit from a centralized pool of consolidated shared storage •  Benefit from a centralized console to provision, patch, upgrade 39 The new standard in consolida@on

Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair

Markus Michalewicz

Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair