Oracle 11G SCAN: Concepts and Implementation Experience Sharing

10,252 views

Published on

Single Client Access Name is a concept that makes database deployment easy on Oracle Database 11g R2 grids and complete the level of abstraction from the application perspective. Starting with the understanding of why this component is vital in the 11g grid infrastructure, this presentation walks through the main concepts of SCAN. You will learn how to plan and implement SCAN, what areas to be careful with, and how to monitor SCAN infrastructure and make sure it works as expected.

Published in: Technology, Business
1 Comment
12 Likes
Statistics
Notes
  • can we have audio for this presentation?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
10,252
On SlideShare
0
From Embeds
0
Number of Embeds
96
Actions
Shares
0
Downloads
732
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

Oracle 11G SCAN: Concepts and Implementation Experience Sharing

  1. 1. Oracle 11G SCAN: Concepts and implementation experience sharingYury Velikanov Todd CarlsonSenior Oracle DBA Sr Manager, DBA Team
  2. 2. SCAN Agenda • Background • Introduction • SCAN Infrastructure Main Components • SCAN troubleshooting • Advanced points • Q&A2 © 2009/2010 Pythian
  3. 3. Few words about Yury • Oracle ACE and RAC SIG regional leader - @yvelikanov - http://www.pythian.com/news/author/velikanov/ • Started as Oracle DBA - with 7.2 (in 1997, 14+) • First international appearance - 2005 - Hotsos Symposium 2005 • First RAC experience - 2000 FIFA - Oracle Parallel Server • Education (Master Degree in Computer science) - OCP 7/8/8i/9/10 + OCM 9i/10g/11g • Several 11GR2 RAC projects in production - Including GNS implementation Google: Oracle Yury Blog, Twitter, Linkedin, ACE … email, phone number3 © 2009/2010 Pythian
  4. 4. Few words about Todd • Have been a DBA for 12 years - 7.3.4 • Currently manage a team of - 7 DBAs covering Oracle, SQL Server and eBusiness Suite (11.5.10.2) - 3 Data Warehouse Developers • First RAC experience - 2006 – 10g R2 • Education (Master Degree in Business) - OCP 8/8i/9/10g/11g • 11GR2 RAC in production - 2/18/2011 – Migrated 1.3 TB EBS Database from 10g R2 Solaris 9 Single Instance to 11g R2 3-node cluster on RHEL 5.5 via Cross Platform Transportable Tablespaces4 © 2009/2010 Pythian
  5. 5. World Wide Technology Industry Leading Systems Integrator Providing Technology & Supply Chain Services to Customers Around the World➭ Privately Held with Revenue Over $4.5 Billion➭ Over 1,600 Employees Across the Country and Around The Globe➭ Global Strategy in Three Key Markets:  Value Added Reseller  Data Center  Supply Chain © 2009/2010 Pythian
  6. 6. WWT’s SCAN Business Drivers• Cluster managed server-side load balancing• Simple connection strings with failover• Prepare for the future• Adding Nodes with SCAN is masked from the clients• SCAN with Services completely abstracts the underlying complexities from the applications PRODRACDB1 PRODRACDB2 PRODRACDB3 S Cluster Ready Services C Datafil Redo Archive Voting Control A es Logs Logs Disk All System Log Files for Operations (alert, trace, core) Files N Shared Storage via Automatic Storage Management © 2009/2010 Pythian
  7. 7. WWT’s Services Business Drivers• Need application activity visibility at the database level beyond what Application & Module offered• Use Resource Manager to control workloads by Service• Pin Services to specific nodes• Provide specific failover options• Tuning via Services © 2009/2010 Pythian
  8. 8. WWT Environment - Physical F5 EBS Tier: O O O OracleFi re X4200 r a OracleFi re X4200 r a OracleFi re X4200 r a Oracle x4170s - 24 gig RAM c l c l c l 2 x Intel Xeon X5570 Quad-Core e e e 2.93GHz O O O OracleFi re X4600 r OracleFi re X4600 r OracleFi re X4600 r a a a c l c l c l DB Tier: e e e Oracle X4270 - 96 gig RAM 2 x Intel Xeon X5570 Quad-Core 2.93GHz Interconnect: Oracle Datacenter Infiniband Switch Storage Tier: EMC DMX 3500 RAID 1+0 73GB fiber drives Database © 2009/2010 Pythian
  9. 9. WWT Environment - Database• 4 Production Databases run on the same cluster • Each runs as a separate OS user with non-shared homes • EBS database supports 11.5.10.2 & over 25 custom apps • Ave ~2000 concurrent users • 1.4 TB • Ave 220+ DML transactions per second (tps) and peak at 2500 tps• WWT applications running from • WebLogic • WebMethods Service PRODERP_IBI Application Reporting via IBI • APEX PRODERP_BI PRODERP_WEBM BI Processing Middleware • Shell PRODERP_WWT_B2B PRODERP_10g OS File Processing Reporting via 10g • Misc. platforms PRODERP_GENERAL PRODERP_APEX Developers & DBAs APEX• Each database has up to 7 services © 2009/2010 Pythian
  10. 10. WWT SCAN Results• With SCAN, the Users don’t even know what database, much less what server, their Service is running on. • 7/23/11 – Node 3 crashed due to a PCI Driver issue • All applications reconnected to Nodes 1 & 2 automatically • Load was evenly distributed • Only got 1 ticket because the offending system was hardcoded • Priceless • Mission critical batch load failed from Node 2 to Node 1 • Failed the batch load between nodes seamlessly for troubleshooting • The batch load completed successfully © 2009/2010 Pythian
  11. 11. Single [Client Access] Name scan.clustgrid-prod.yourdomain.com + service11 © 2009/2010 Pythian
  12. 12. There are Two SCAN related news • Good • SCAN is based on known components you worked for years now • Other news • SCAN uses those components in different way RAC: Frequently Asked Questions [ID 220970.1] How to Troubleshoot Connectivity Issue with 11gR2 SCAN Name [ID 975457.1] 11gR2 Grid Infrastructure Single Client Access Name (SCAN) Explained [ID 887522.1] SCAN & EBS 11i [ID 823581.1 ] R12 [823587.1]12 © 2009/2010 Pythian
  13. 13. SCAN Introduction • Single Client Access Name • Addresses the TNSNAMES multi address issue • Old - 10G FAILOVER • Complex TNS entries • Complex to manage (add a node) • Previous Oracle Clients support • New - 11GR2 • One Simple TNS entry on client side • Easy to add nodes • Transparent to Oracle Client versions (!DNS issue!) • No static listener.ora file13 © 2009/2010 Pythian
  14. 14. SCAN and PREV tnsnames.ora PROD _HR.yourdomain.com = (DESCRIPTION = (ADDRESS_LIST = (FAILOVER=on) (LOAD_BALANCE=TRUE) (ADDRESS = (PROTOCOL = TCP)(HOST = vip.node1)(PORT = 1523)) (ADDRESS = (PROTOCOL = TCP)(HOST = vip.node2)(PORT = 1523)) (ADDRESS= (PROTOCOL = TCP)(HOST = vip.node3)(PORT = 1523)) ) (CONNECT_DATA = (SERVICE_NAME = HR) ) ) scan.clustgrid-prod.yourdomain.com:1523/HR14 © 2009/2010 Pythian
  15. 15. Ora*Net: Easy Connect scan.clustgrid-prod.yourdomain.com scan.clustgrid-prod.yourdomain.com:1521 scan.clustgrid-prod.yourdomain.com:1521/HR scan.clustgrid-prod.yourdomain.com:1521/HR:dedicated/ERP1 PROD _HR.yourdomain.com = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = scan.clustgrid-prod)(PORT = 1521)) (CONNECT_DATA = (SERVICE_NAME = HR) ) ) Oracle® Database Net Services Administrators Guide 11g Release 2 (11.2) Part Number E10836-06 http://download.oracle.com/docs/cd/E11882_01/network.112/e10836/naming.htm#BABJBFHJ15 © 2009/2010 Pythian
  16. 16. 16 © 2009/2010 Pythian
  17. 17. SCAN Infrastructure Main Components • Single Client Access Name + Oracle Services (Definitions) • DNS – resolving SCAN to 3 IP addresses (Round Robin) • Primary / Secondary • NameServer configuration (client side) • SCAN Listeners • Keeps records on available Local Listeners and Services those serve • Forwards connections to less loaded Local Listener • Local (VIP) Listeners • Creates foreground processes • Manages sockets • RAC (SCAN / VIP / Interconnect) ip addresses • Grid Name Service • registers and resolves RAC ip addresses • DHCP • Assign dynamically IP addresses17 © 2009/2010 Pythian
  18. 18. 18 © 2009/2010 Pythian
  19. 19. SCAN troubleshooting• Service Names • DO NOT MODIFY init.ora:service_name • USE srvctl to configure and manage services srvctl config service -d <DB Name> … Service name: DEVERP_APEX.GGT.COM Service is enabled Failover type: NONE Preferred instances: DEVERP1 Available instances: DEVERP1,DEVERP2,DEVERP3,DEVERP4,DEVERP5,DEVERP6 … show parameter service_name SQL> show parameter service_name NAME TYPE VALUE -------------------- ----------- -------------------------------------------------- service_names string DEVERP_CDC.GGT.COM, SYS$APPLSYS.WF_CONTROL.DEVERP. WORLD, SYS$STREAMS_ADMIN.CDC$Q_ERP.DEVERP.WORLD, D EVERP_WEBM.GGT.COM, DEVERP_WWT_B2B.GGT.COM, DEVERP _RFUI.GGT.COM, DEVERP_IBI.GGT.COM, DEVERP_GENERAL. WWT.COM, DEVERP_BI.GGT.COM, DEVERP_APEX.GGT.COM, D EVERP_10g, DEVERP1, DEVERP SQL>19 © 2009/2010 Pythian
  20. 20. SCAN troubleshooting• Oracle Listeners • Running under grid OS user • Don’t start it under ORACLE user (DB OH) • If you do you end up with a mess • Manage (start/stop) by srvctl • Be careful with manual start/stop (TNS_ADMIN) • listener.ora is dynamic configuration file by default • [All] parameters managed by Cluster • Use srvctl to configure • Make sure listeners listen on corresponding IPs >lsnrctl status LISTENER_SCAN2 … (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN2))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.2.9.122)(PORT=1523))) … • LISTENER_SCAN1/2/3 on SCAN IPs • LISTENER on VIP and Public IPs20 © 2009/2010 Pythian
  21. 21. SCAN troubleshooting• init.ora:local_listener • It is an OLD good parameter • The same rules applies • Specify LOCAL listener only! • Cant stress enough !!! NO SCAN !!! • You can use TNS address directly or TNS alias • !!! If can’t resolve an instance wont start !!!SQL> show parameter local_listenerNAME TYPE VALUE-------------------- ----------- --------------------------------------------------local_listener string (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP) (HOST=devracdb1-vip)(PORT=1534))(ADDRESS=(PROTOCOL=TCP) (HOST=devracdb1-vip)(PORT=1521))))SQL> show parameter local_listenerNAME TYPE VALUE-------------------- ----------- --------------------------------------------------local_listener string devracdb1-viptnsping devracdb1-vip21 © 2009/2010 Pythian
  22. 22. SCAN troubleshooting• init.ora:remote_listener SQL> show parameter remote_listener NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ remote_listener string scan.clustgrid-prod.yourdomain.com SQL> • The same management principals apply • Make SURE it points to SCAN IP addresses only • Cant stress enough !!! NO VIP !!! • Any valid TNS config is acceptable • tnsnames alias • sqlnet.ora • NAMES.DIRECTORY_PATH=(TNSNAMES, EZCONNECT) • Use SCAN or IPs (for static SCAN conf only) !!! If can’t resolve an instance wont start !!!22 © 2009/2010 Pythian
  23. 23. SCAN troubleshooting• DNS • dig (Linux os command) • nslookup <scan> (run several times) [oracle@host01 admin]$ dig scan.clustgrid-prod.yourdomain.com ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5 <<>> scan.clustgrid-prod.yourdomain.com | tail -1 [oracle@host01 admin]$ nslookup scan.clustgrid-prod.yourdomain.com | grep Address ;;Address: 172.30.193.218 global options: printcmd • check primary and secondary name servers ;;[oracle@host01 admin]$ nslookup scan.clustgrid-prod.yourdomain.com | grep Address | tail -1 Got answer: ;;Address: 172.30.193.216QUERY, status: NOERROR, id: 15137 ->>HEADER<<- opcode: ;;[oracle@host01 admin]$1, ANSWER: 3, AUTHORITY: 1, ADDITIONAL: |0 flags: qr rd ra; QUERY: nslookup scan.clustgrid-prod.yourdomain.com grep Address | tail -1 Address: 172.30.193.218 ;; QUESTION SECTION: • Make 200% sure ;scan.clustgrid-prod.yourdomain.com. IN A • SCAN doesn’t172.30.193.218 VIPs ANSWER SECTION: scan.clustgrid-prod.yourdomain.com. 58 IN A contain ;;[oracle@host01 admin]$ dig scan.clustgrid-prod.yourdomain.com ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5 <<>> 172.30.193.216 scan.clustgrid-prod.yourdomain.com ;; global options: printcmd don’t contain SCAN IPs • VIPs scan.clustgrid-prod.yourdomain.com. 58 IN A scan.clustgrid-prod.yourdomain.com. 58 IN A 172.30.193.217 ;; Got answer: ;;;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15137 AUTHORITY SECTION: clustgrid-prod.yourdomain.com. 86400 IN AUTHORITY: 1, ADDITIONAL: 0 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, NS gns.clustgrid-prod.yourdomain.com. ;;;; QUESTION SECTION: Query time: 0 msec ;;;scan.clustgrid-prod.yourdomain.com. IN SERVER: 172.30.192.82#53(172.30.192.82) A ;; WHEN: Wed Jun 15 18:21:03 2011 ;;;; ANSWER SECTION: MSG SIZE rcvd: 114 scan.clustgrid-prod.yourdomain.com. 58 IN A 172.30.193.218 scan.clustgrid-prod.yourdomain.com. 58 IN [oracle@host01 admin]$ A 172.30.193.216 scan.clustgrid-prod.yourdomain.com. 58 IN A 172.30.193.21723 © 2009/2010 Pythian
  24. 24. GNS Advanced points • SCAN + GNS implementation • Most probably you do not need it • Makes the configuration 100% dynamic • Unlimited number of nodes with simple Oracle Client Configuration • Oracle retrieves new IPs from DHCP for SCAN / VIP / [ Interconnect ] components at startup time • The only static RAC IP is GNS IP24 © 2009/2010 Pythian
  25. 25. GNS Advanced points • Additional components • Grid Name Service • DNS and GNS integration (SCAN/VIP) • Dedicated DHCP service • Separate Network Segment • DHCP redundancy could be an issue • RAC and DHCP integration • Make DHCP assigning the same IPs (or range) each time per RAC process (Joseph Griffiths) • http://blog.jgriffiths.org/?p=24 • DHCPDISCOVER from 00:00:00:00:00:00 via eth0 • Many things could go wrong !!! • GNS Troublesooting – see my blog25 © 2009/2010 Pythian
  26. 26. There are Two SCAN related news • Good • SCAN is based on known components you worked for years now • Other news • SCAN uses those components in different way26 © 2009/2010 Pythian
  27. 27. Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 6 Oracle ACEs/ACE Directors. • Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RAC. • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response27 © 2011 Pythian - Confidential
  28. 28. RAC: Frequently Asked Questions [ID 220970.1]How to Troubleshoot Connectivity Issue with 11gR2 SCAN Name [ID 975457.1] Additional Resources11gR2 Grid Infrastructure Single Client Access Name (SCAN) Explained [ID 887522.1]SCAN & EBS 11i [ID 823581.1 ] R12 [823587.1] • www.oracle.com/scan • www.pythian.com/exadata • www.pythian.com/news/tag/exadata - Exadata Blog • www.pythian.com/news_and_events/in_the_news Article: “Making the Most of Oracle Exadata” My Oracle Support notes 888828.1 and 757552.1 Thank you! Google: Oracle Yury Blog, Twitter, Linkedin, ACE … email, phone number 28 © 2009/2010 Pythian

×