SlideShare a Scribd company logo
1 of 25
Download to read offline
01.11.2017
PRESENTED BY:
DNS Troubleshooting
Introduction to BIGIP DNS
DNS Hierarchy
Sample of a Zone File
$TTL 86400 ; 24 hours could have been written as 24h or 1d
; $TTL used for all RRs without explicit TTL value
$ORIGIN example.com.
@ 1D IN SOA ns1.example.com. hostmaster.example.com. (
2002022401 ; serial
3H ; refresh
15 ; retry
1w ; expire
3h ; nxdomain ttl
)
IN NS ns1.example.com. ; in the domain
IN NS ns2.smokeyjoe.com. ; external to domain
IN MX 10 mail.another.com. ; external mail provider
; server host definitions
ns1 IN A 192.168.0.1 ;name server definition
www IN A 192.168.0.2 ;web server definition
ftp IN CNAME www.example.com. ;ftp server definition
; non server domain hosts
bill IN A 192.168.0.3
fred IN A 192.168.0.4
BIG-IP DNS (formerly Global Traffic Manager or GTM)
It’s a load balancer for DNS queries (caching, traffic management), decision making to load
balance between datacenters.
Terms
• Wide IP - Maps FQDN to one or more pools of virtual servers that host content of the domain.
It responds to listener requests. This will auto create a zone that matches the Wide IP
• Server object - Server defined in BIG-IP DNS is either a BIG-IP or other 3rd party system
responsible for owning one or more virtual server service.
i. BIGIP devices (LTM/ASM/APM/etc) – Standalone/Redudant-Pair
ii. Generic LB/Host (3rd party system) – Citrix LB, Cisco CSS, Centos machine
• Listener - BIG-IP uses TCP/UDP listeners to respond to DNS queries.
• Pool - In BIG-IP DNS a pool contains one or more virtual servers.
BIGIP Resolution
Hierarchy
Listener
Wide IP
- Maps FQDN to set of Virtual Server that host the domain content
- Uses Pool to organize Virtual Server
iQuery communication and troubleshooting
Purpose
Establishing communication between GTM and other system to be in a same Sync Group
Requirement
1) DNS members must be running on same version (source: K13703)
i. BIG-IP DNS synchronization group communication
ii. Monitored BIG-IP systems must run the same or newer big3d version as the DNS / GTM
that are monitoring them
2) Sync parameter must defined properly
3) NTP in sync
4) Port lockdown allowing 4353 and 443
5) Compatible big3d version
iQuery
Virtual Server/Link Autodiscovery (K13994)
1) Virtual server and link auto-discovery feature allows:
2) BIG-IP DNS and BIG-IP Link Controller systems to automatically discover virtual servers and
links that are associated with defined BIG-IP systems.
3) Uses iQuery protocol to automatically discover objects on the remote BIG-IP system if enabled.
• The BIG-IP DNS configuration contains one or more BIG-IP server objects
• TCP port 4353 is allowed between the BIG-IP DNS system and target BIG-IP systems
• The target BIG-IP system's virtual server addresses must not employ network address
translation
Important Note:
K9138: The BIG-IP GTM system disables virtual server auto-discovery for BIG-IP systems
that use translated virtual server addresses
K14106: Troubleshooting virtual server and link auto-discovery (11.x - 13.x)
- telnet <remote_bigip_selfip> 4353
- iqdump
K13312: Overview of the BIG-IP GTM big3d_install, bigip_add, and
gtm_add utilities (11.x - 13.x)
Requirement of iQuery
- TCP port 4353
- SSH port 22 (for initial certificate transfer/copy)
bigip_add
- Exchanges iQuery SSL certificate with remove BIGIP
- Append Local GTM system’s certificate to remote BIGIP authorized cert (stored in /config/big3d/client.crt)
- Append remote iQuery cert to local GTM list of authenticate iQuery (stored in .config/gtm/server.crt)
bigip_install (K13703)
- Similar to bigip_add but install the big3d version if its older than the local GTM F5 system
- To check: run:
# /usr/sbin/big3d -v (default big3d agent)
#/shared/bin/big3d –v (executable file)
gtm_add
- Integrate new GTM system into existing sync group
- Replace current config (bigip_gtm.conf, named.conf and the name zone files)
Troubleshoot iQuery
1. Config Utility
- Check the status of the server object (Global Traffic -> Server -> Server List
- iQuery Stat (Statistic -> Global Traffic -> Statistic Type -> iQuery)
- Summary Stat (Statistic -> Global Traffic)
2. TMSH
- Server (tmsh show /gtm server all)
- iQuery (tmsh show /gtm iquery all)
- GTM (tmsh show /gtm)
3. /var/log/gtm
4. Verify the big3d version
# /usr/sbin/big3d -v (default big3d agent)
#/shared/bin/big3d –v (executable file)
5. Check the iQuery processes
# netstat –nap | grep 4353
Cont*
6. Iqdump utility (run from the GTM)
iqdump 10.10.10.20 <sync_group_name>
• If the iQuery channel is not established, it will prompt error
46947856243768:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1168:
• If the iQuery channel is established, iqdump returns XML similar to the following example:
<!-- Local hostname: lc1.example.com -->
<!-- Connected to big3d at: ::ffff:10.10.10.10:4353 -->
<!-- Subscribing to syncgroup: default -->
<!-- Tue May 6 09:55:43 2014 -->
<xml_connection>
<version>11.5.1</version>
<big3d>big3d Version 11.5.1.0.0.110</big3d>
7. Verify device Certificate
openssl x509 -noout -text -in /config/httpd/conf/ssl.crt/server.crt
- Verify the certificate validity date and confirm whether the certificate is expired.
- If necessary, renew the certificate. To do so, refer to K6353: Updating an SSL device certificate on a BIG-IP system.
Prober pool
Collection of device that perform monitor probes of servers to gather data about the health and performance
of the resources on the servers
By default, the members of a GTM sync group dynamically determine the best BIG-IP device within the sync
group configuration to use as the prober for the non-BIG-IP device server objects. Devices defined within the
same data center as the server object to be probed are preferred. If no local prober is available, a remote
prober is used.
Debugging
To enable debugging
tmsh modify /sys db log.gtm.level value debug
tmsh modify /sys db log.big3d.level value debug
tmsh modify /sys db gtm.debugprobelogging value enable
To disable debugging
tmsh modify /sys db log.gtm.level value debug
tmsh modify /sys db log.big3d.level value debug
tmsh modify /sys db gtm.debugprobelogging value enable
Collect qkview and full tar ball for review.
#qkview –s0
#tar -czvpf /var/tmp/$HOSTNAME-logs.tgz /var/log
Configure Decision Logging
https://devcentral.f5.com/articles/configuring-decision-logging-for-the-f5-big-ip-global-traffic-
manager
• Modify WIP(s) to enable LB Decision Logging
• Log Publisher
• DNS Logging Profile
• Custom DNS Profile and attach the logging profile
• Apply the DNS Profile to the Listener
DNS Express
DNS Express
Alows the BIG-IP to perform zone transfers from multiple primary DNS servers that are responsible
for different zones, perform a zone transfer from the local BIND server on the BIG-IP, and serve
DNS records faster than the primary DNS servers and the local BIND server.
• Perform zone transfers from multiple primary DNS servers that are responsible for different
zones.
• Perform a zone transfer from the local BIND server on the BIG-IP system.
• Serve DNS records faster than the primary DNS servers
K15298: Overview of the dnsxdump utility
You can use the dnsxdump utility to view the DNS Express database information, which includes zone
information and statistics.
• The DB Dump section of the dnsxdump utility output displays the zone information for all configured DNS
Express zones.
• The DB Stats section of the dnsxdump utility output displays a cumulative count of records for all configured
DNS Express zones.
dnsxdump > /var/tmp/my_zones.txt
Zonerunner
Managing the BIG-IP BIND configuration file
ZoneRunner utility is used to manage both DNS zone files and the BIND configuration file on the BIG-IP
GTM system
• Import and transfer DNS zone files
• Manage zone resource records
• Manage views
• Manage a local nameserver and the associated configuration file, named.conf
• Transfer zone files to a nameserver
• Import only primary zone files from a nameserver
Cont*
By default, BIG-IP GTM is configured to secure BIND to not allow zone transfers except from the
localhost. Modify the allow-transfer statement to include the IP address of the GTM. You can modify the
following allow-transfer statement to use the IP address of the GTM.
DNS > Zones > ZoneRunner > named Configuration.
allow-transfer { localhost; 192.168.10.105; }
To verify zone transfers are working properly
# dig @<IP address> es.net. axfr
Directory of the zone file stored
# cd /var/named/config/namedb/
Check the named configuration
K7032: Freezing zone files to allow manual update to ZoneRunner-
managed zone files
All changes made to a zone using dynamic update are written to the zone's journal file.
When the BIG-IP DNS system restarts after a shutdown, the system replays the journal file to incorporate any
updates that took place after the last zone file update into the zone.
Dynamic update periodically flushes the complete contents of the updated zone to its zone file and
automatically deletes the journal file.
i. cd /var/named/config/namedb
ii. cp <zone_filename> <zone_filename>.original
iii. bigstart stop zrd
iv. rndc freeze <zone name> <class> <view>
v. Manually edit the zone for any changes
vi. rndc sync -clean
vii. Run the named-checkzone command to check the file for any syntax errors
• named-checkzone askf5.net db.external.askf5.net
• named-checkconf -t /var/named -z -j /config/named.conf
viii. rndc thaw <zone name> <class> <view>
ix. bigstart start zrd
.
Behaviour of zrd
- When a new dns express zone is added , it writes the data to zxfrd.bin
- It then copies from zxfrd.bin to zxfrd-tmp.bin (15sec timer)
- Rename the zxfrd-tmp.bin to tmmdns.bin (database)
TMM then reload the database from tmmdns.bin
- For VIPRION, csyncd monitors tmmdns.bin for any changes
- Csyncd trigger tmm to reload on primary blades then populate to other blades
# bigstart stop
# rm -rf /var/db/{tmmdns.bin,zxfrd.bin}
# bigstart start
tmsh modify sys db log.zxfrd.level value debug
Collect qkview and full tar ball for review.
#qkview –s0
#tar -czvpf /var/tmp/$HOSTNAME-logs.tgz /var/log
DNS Troubleshooting.pdf

More Related Content

What's hot

Introduction to DIAMETER
Introduction to DIAMETERIntroduction to DIAMETER
Introduction to DIAMETERHossein Yavari
 
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹InfraEngineer
 
Volte troubleshooting
Volte troubleshootingVolte troubleshooting
Volte troubleshootingJamil Awan
 
Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Ray Jenkins
 
ims registration call flow procedure volte sip
ims registration call flow procedure volte sipims registration call flow procedure volte sip
ims registration call flow procedure volte sipVikas Shokeen
 
Vxlan deep dive session rev0.5 final
Vxlan deep dive session rev0.5   finalVxlan deep dive session rev0.5   final
Vxlan deep dive session rev0.5 finalKwonSun Bae
 
Type of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleType of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleHimani Singh
 
Chapter 17 : static routing
Chapter 17 : static routingChapter 17 : static routing
Chapter 17 : static routingteknetir
 
5.5.1.2 packet tracer configure ios intrusion prevention system (ips) using...
5.5.1.2 packet tracer   configure ios intrusion prevention system (ips) using...5.5.1.2 packet tracer   configure ios intrusion prevention system (ips) using...
5.5.1.2 packet tracer configure ios intrusion prevention system (ips) using...Salem Trabelsi
 
Linux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF SuperpowersLinux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF SuperpowersBrendan Gregg
 
volte ims network architecture
volte ims network architecturevolte ims network architecture
volte ims network architectureVikas Shokeen
 
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124  | Las Vegas 2017Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124  | Las Vegas 2017
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017Bruno Teixeira
 
Cisco switch commands cheat sheet
Cisco switch commands cheat sheetCisco switch commands cheat sheet
Cisco switch commands cheat sheet3Anetwork com
 
NSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
NSO: Network Service Orchestrator enabled by Tail-f Hands-on LabNSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
NSO: Network Service Orchestrator enabled by Tail-f Hands-on LabCisco Canada
 
Tutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerTutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerShu Sugimoto
 
Initial LTE call Setup Flow
Initial LTE call Setup FlowInitial LTE call Setup Flow
Initial LTE call Setup Flowassinha
 

What's hot (20)

Introduction to DIAMETER
Introduction to DIAMETERIntroduction to DIAMETER
Introduction to DIAMETER
 
Ethernet VPN (EVPN) EVerything Provider Needs
Ethernet VPN (EVPN) EVerything Provider NeedsEthernet VPN (EVPN) EVerything Provider Needs
Ethernet VPN (EVPN) EVerything Provider Needs
 
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹
[MeetUp][1st] 오리뎅이의_쿠버네티스_네트워킹
 
Volte troubleshooting
Volte troubleshootingVolte troubleshooting
Volte troubleshooting
 
Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!Understanding eBPF in a Hurry!
Understanding eBPF in a Hurry!
 
Paging in LTE
Paging in LTEPaging in LTE
Paging in LTE
 
ims registration call flow procedure volte sip
ims registration call flow procedure volte sipims registration call flow procedure volte sip
ims registration call flow procedure volte sip
 
Vxlan deep dive session rev0.5 final
Vxlan deep dive session rev0.5   finalVxlan deep dive session rev0.5   final
Vxlan deep dive session rev0.5 final
 
Type of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 exampleType of DDoS attacks with hping3 example
Type of DDoS attacks with hping3 example
 
5gc call flow
5gc call flow5gc call flow
5gc call flow
 
Chapter 17 : static routing
Chapter 17 : static routingChapter 17 : static routing
Chapter 17 : static routing
 
5.5.1.2 packet tracer configure ios intrusion prevention system (ips) using...
5.5.1.2 packet tracer   configure ios intrusion prevention system (ips) using...5.5.1.2 packet tracer   configure ios intrusion prevention system (ips) using...
5.5.1.2 packet tracer configure ios intrusion prevention system (ips) using...
 
Linux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF SuperpowersLinux 4.x Tracing Tools: Using BPF Superpowers
Linux 4.x Tracing Tools: Using BPF Superpowers
 
volte ims network architecture
volte ims network architecturevolte ims network architecture
volte ims network architecture
 
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124  | Las Vegas 2017Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124  | Las Vegas 2017
Cisco Live! :: Introduction to Segment Routing :: BRKRST-2124 | Las Vegas 2017
 
Cisco switch commands cheat sheet
Cisco switch commands cheat sheetCisco switch commands cheat sheet
Cisco switch commands cheat sheet
 
NSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
NSO: Network Service Orchestrator enabled by Tail-f Hands-on LabNSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
NSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
 
Tutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerTutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting router
 
3GPP IMS
3GPP IMS3GPP IMS
3GPP IMS
 
Initial LTE call Setup Flow
Initial LTE call Setup FlowInitial LTE call Setup Flow
Initial LTE call Setup Flow
 

Similar to DNS Troubleshooting.pdf

Building a Linux IPv6 DNS Server Project review PPT v3.0 First review
Building a Linux IPv6 DNS Server Project review PPT v3.0 First reviewBuilding a Linux IPv6 DNS Server Project review PPT v3.0 First review
Building a Linux IPv6 DNS Server Project review PPT v3.0 First reviewHari
 
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2Upgrading AD from Windows Server 2003 to Windows Server 2008 R2
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2Amit Gatenyo
 
Building Linux IPv6 DNS Server (Complete Presentation)
Building Linux IPv6 DNS Server (Complete Presentation)Building Linux IPv6 DNS Server (Complete Presentation)
Building Linux IPv6 DNS Server (Complete Presentation)Hari
 
Oracle Real Application Cluster ( RAC )
Oracle Real Application Cluster ( RAC )Oracle Real Application Cluster ( RAC )
Oracle Real Application Cluster ( RAC )varasteh65
 
Implementation of DNS Anycast - a case study
Implementation of DNS Anycast - a case studyImplementation of DNS Anycast - a case study
Implementation of DNS Anycast - a case studyA. S. M. Shamim Reza
 
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 20161049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016panagenda
 
Microsoft SharePoint Disaster Recovery to Azure
Microsoft SharePoint Disaster Recovery to AzureMicrosoft SharePoint Disaster Recovery to Azure
Microsoft SharePoint Disaster Recovery to AzureDavid J Rosenthal
 
Dg broker &amp; client connectivity - High Availability Day 2015
Dg broker &amp; client connectivity -  High Availability Day 2015Dg broker &amp; client connectivity -  High Availability Day 2015
Dg broker &amp; client connectivity - High Availability Day 2015aioughydchapter
 
Integration of neutron, nova and designate how to use it and how to configur...
Integration of neutron, nova and designate  how to use it and how to configur...Integration of neutron, nova and designate  how to use it and how to configur...
Integration of neutron, nova and designate how to use it and how to configur...Miguel Lavalle
 
Active Directory Security Assessment ADSA
Active Directory Security Assessment ADSAActive Directory Security Assessment ADSA
Active Directory Security Assessment ADSACarrie Tran
 
Moving to ws2003
Moving to ws2003Moving to ws2003
Moving to ws2003Sumit Tambe
 
Sharing-Knowledge-OAM-3G-Ericsson .ppt
Sharing-Knowledge-OAM-3G-Ericsson   .pptSharing-Knowledge-OAM-3G-Ericsson   .ppt
Sharing-Knowledge-OAM-3G-Ericsson .pptwafawafa52
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobus
 
BIND 9 logging best practices
BIND 9 logging best practicesBIND 9 logging best practices
BIND 9 logging best practicesMen and Mice
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 

Similar to DNS Troubleshooting.pdf (20)

Building a Linux IPv6 DNS Server Project review PPT v3.0 First review
Building a Linux IPv6 DNS Server Project review PPT v3.0 First reviewBuilding a Linux IPv6 DNS Server Project review PPT v3.0 First review
Building a Linux IPv6 DNS Server Project review PPT v3.0 First review
 
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2Upgrading AD from Windows Server 2003 to Windows Server 2008 R2
Upgrading AD from Windows Server 2003 to Windows Server 2008 R2
 
Building Linux IPv6 DNS Server (Complete Presentation)
Building Linux IPv6 DNS Server (Complete Presentation)Building Linux IPv6 DNS Server (Complete Presentation)
Building Linux IPv6 DNS Server (Complete Presentation)
 
Oracle Real Application Cluster ( RAC )
Oracle Real Application Cluster ( RAC )Oracle Real Application Cluster ( RAC )
Oracle Real Application Cluster ( RAC )
 
Implementation of DNS Anycast - a case study
Implementation of DNS Anycast - a case studyImplementation of DNS Anycast - a case study
Implementation of DNS Anycast - a case study
 
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 20161049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
1049: Best and Worst Practices for Deploying IBM Connections - IBM Connect 2016
 
DNSSEC - WHAT IS IT ? INSTALL AND CONFIGURE IN CHROOT JAIL
DNSSEC - WHAT IS IT ? INSTALL AND CONFIGURE IN CHROOT JAILDNSSEC - WHAT IS IT ? INSTALL AND CONFIGURE IN CHROOT JAIL
DNSSEC - WHAT IS IT ? INSTALL AND CONFIGURE IN CHROOT JAIL
 
Microsoft SharePoint Disaster Recovery to Azure
Microsoft SharePoint Disaster Recovery to AzureMicrosoft SharePoint Disaster Recovery to Azure
Microsoft SharePoint Disaster Recovery to Azure
 
Lksn2017 itnsa modul2
Lksn2017 itnsa modul2Lksn2017 itnsa modul2
Lksn2017 itnsa modul2
 
Dg broker &amp; client connectivity - High Availability Day 2015
Dg broker &amp; client connectivity -  High Availability Day 2015Dg broker &amp; client connectivity -  High Availability Day 2015
Dg broker &amp; client connectivity - High Availability Day 2015
 
70 640
70 64070 640
70 640
 
module B.docx
module B.docxmodule B.docx
module B.docx
 
Integration of neutron, nova and designate how to use it and how to configur...
Integration of neutron, nova and designate  how to use it and how to configur...Integration of neutron, nova and designate  how to use it and how to configur...
Integration of neutron, nova and designate how to use it and how to configur...
 
Active Directory Security Assessment ADSA
Active Directory Security Assessment ADSAActive Directory Security Assessment ADSA
Active Directory Security Assessment ADSA
 
Moving to ws2003
Moving to ws2003Moving to ws2003
Moving to ws2003
 
Sharing-Knowledge-OAM-3G-Ericsson .ppt
Sharing-Knowledge-OAM-3G-Ericsson   .pptSharing-Knowledge-OAM-3G-Ericsson   .ppt
Sharing-Knowledge-OAM-3G-Ericsson .ppt
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System Administrators
 
BIND 9 logging best practices
BIND 9 logging best practicesBIND 9 logging best practices
BIND 9 logging best practices
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 

Recently uploaded

Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Next-generation AAM aircraft unveiled by Supernal, S-A2
Next-generation AAM aircraft unveiled by Supernal, S-A2Next-generation AAM aircraft unveiled by Supernal, S-A2
Next-generation AAM aircraft unveiled by Supernal, S-A2Hyundai Motor Group
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 

Recently uploaded (20)

Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Next-generation AAM aircraft unveiled by Supernal, S-A2
Next-generation AAM aircraft unveiled by Supernal, S-A2Next-generation AAM aircraft unveiled by Supernal, S-A2
Next-generation AAM aircraft unveiled by Supernal, S-A2
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 

DNS Troubleshooting.pdf

  • 3. DNS Hierarchy Sample of a Zone File $TTL 86400 ; 24 hours could have been written as 24h or 1d ; $TTL used for all RRs without explicit TTL value $ORIGIN example.com. @ 1D IN SOA ns1.example.com. hostmaster.example.com. ( 2002022401 ; serial 3H ; refresh 15 ; retry 1w ; expire 3h ; nxdomain ttl ) IN NS ns1.example.com. ; in the domain IN NS ns2.smokeyjoe.com. ; external to domain IN MX 10 mail.another.com. ; external mail provider ; server host definitions ns1 IN A 192.168.0.1 ;name server definition www IN A 192.168.0.2 ;web server definition ftp IN CNAME www.example.com. ;ftp server definition ; non server domain hosts bill IN A 192.168.0.3 fred IN A 192.168.0.4
  • 4. BIG-IP DNS (formerly Global Traffic Manager or GTM) It’s a load balancer for DNS queries (caching, traffic management), decision making to load balance between datacenters. Terms • Wide IP - Maps FQDN to one or more pools of virtual servers that host content of the domain. It responds to listener requests. This will auto create a zone that matches the Wide IP • Server object - Server defined in BIG-IP DNS is either a BIG-IP or other 3rd party system responsible for owning one or more virtual server service. i. BIGIP devices (LTM/ASM/APM/etc) – Standalone/Redudant-Pair ii. Generic LB/Host (3rd party system) – Citrix LB, Cisco CSS, Centos machine • Listener - BIG-IP uses TCP/UDP listeners to respond to DNS queries. • Pool - In BIG-IP DNS a pool contains one or more virtual servers.
  • 7. Wide IP - Maps FQDN to set of Virtual Server that host the domain content - Uses Pool to organize Virtual Server
  • 8. iQuery communication and troubleshooting
  • 9. Purpose Establishing communication between GTM and other system to be in a same Sync Group Requirement 1) DNS members must be running on same version (source: K13703) i. BIG-IP DNS synchronization group communication ii. Monitored BIG-IP systems must run the same or newer big3d version as the DNS / GTM that are monitoring them 2) Sync parameter must defined properly 3) NTP in sync 4) Port lockdown allowing 4353 and 443 5) Compatible big3d version iQuery
  • 10. Virtual Server/Link Autodiscovery (K13994) 1) Virtual server and link auto-discovery feature allows: 2) BIG-IP DNS and BIG-IP Link Controller systems to automatically discover virtual servers and links that are associated with defined BIG-IP systems. 3) Uses iQuery protocol to automatically discover objects on the remote BIG-IP system if enabled. • The BIG-IP DNS configuration contains one or more BIG-IP server objects • TCP port 4353 is allowed between the BIG-IP DNS system and target BIG-IP systems • The target BIG-IP system's virtual server addresses must not employ network address translation Important Note: K9138: The BIG-IP GTM system disables virtual server auto-discovery for BIG-IP systems that use translated virtual server addresses K14106: Troubleshooting virtual server and link auto-discovery (11.x - 13.x) - telnet <remote_bigip_selfip> 4353 - iqdump
  • 11. K13312: Overview of the BIG-IP GTM big3d_install, bigip_add, and gtm_add utilities (11.x - 13.x) Requirement of iQuery - TCP port 4353 - SSH port 22 (for initial certificate transfer/copy) bigip_add - Exchanges iQuery SSL certificate with remove BIGIP - Append Local GTM system’s certificate to remote BIGIP authorized cert (stored in /config/big3d/client.crt) - Append remote iQuery cert to local GTM list of authenticate iQuery (stored in .config/gtm/server.crt) bigip_install (K13703) - Similar to bigip_add but install the big3d version if its older than the local GTM F5 system - To check: run: # /usr/sbin/big3d -v (default big3d agent) #/shared/bin/big3d –v (executable file) gtm_add - Integrate new GTM system into existing sync group - Replace current config (bigip_gtm.conf, named.conf and the name zone files)
  • 12. Troubleshoot iQuery 1. Config Utility - Check the status of the server object (Global Traffic -> Server -> Server List - iQuery Stat (Statistic -> Global Traffic -> Statistic Type -> iQuery) - Summary Stat (Statistic -> Global Traffic) 2. TMSH - Server (tmsh show /gtm server all) - iQuery (tmsh show /gtm iquery all) - GTM (tmsh show /gtm) 3. /var/log/gtm 4. Verify the big3d version # /usr/sbin/big3d -v (default big3d agent) #/shared/bin/big3d –v (executable file) 5. Check the iQuery processes # netstat –nap | grep 4353
  • 13. Cont* 6. Iqdump utility (run from the GTM) iqdump 10.10.10.20 <sync_group_name> • If the iQuery channel is not established, it will prompt error 46947856243768:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1168: • If the iQuery channel is established, iqdump returns XML similar to the following example: <!-- Local hostname: lc1.example.com --> <!-- Connected to big3d at: ::ffff:10.10.10.10:4353 --> <!-- Subscribing to syncgroup: default --> <!-- Tue May 6 09:55:43 2014 --> <xml_connection> <version>11.5.1</version> <big3d>big3d Version 11.5.1.0.0.110</big3d> 7. Verify device Certificate openssl x509 -noout -text -in /config/httpd/conf/ssl.crt/server.crt - Verify the certificate validity date and confirm whether the certificate is expired. - If necessary, renew the certificate. To do so, refer to K6353: Updating an SSL device certificate on a BIG-IP system.
  • 14. Prober pool Collection of device that perform monitor probes of servers to gather data about the health and performance of the resources on the servers By default, the members of a GTM sync group dynamically determine the best BIG-IP device within the sync group configuration to use as the prober for the non-BIG-IP device server objects. Devices defined within the same data center as the server object to be probed are preferred. If no local prober is available, a remote prober is used.
  • 15. Debugging To enable debugging tmsh modify /sys db log.gtm.level value debug tmsh modify /sys db log.big3d.level value debug tmsh modify /sys db gtm.debugprobelogging value enable To disable debugging tmsh modify /sys db log.gtm.level value debug tmsh modify /sys db log.big3d.level value debug tmsh modify /sys db gtm.debugprobelogging value enable Collect qkview and full tar ball for review. #qkview –s0 #tar -czvpf /var/tmp/$HOSTNAME-logs.tgz /var/log
  • 16. Configure Decision Logging https://devcentral.f5.com/articles/configuring-decision-logging-for-the-f5-big-ip-global-traffic- manager • Modify WIP(s) to enable LB Decision Logging • Log Publisher • DNS Logging Profile • Custom DNS Profile and attach the logging profile • Apply the DNS Profile to the Listener
  • 18. DNS Express Alows the BIG-IP to perform zone transfers from multiple primary DNS servers that are responsible for different zones, perform a zone transfer from the local BIND server on the BIG-IP, and serve DNS records faster than the primary DNS servers and the local BIND server. • Perform zone transfers from multiple primary DNS servers that are responsible for different zones. • Perform a zone transfer from the local BIND server on the BIG-IP system. • Serve DNS records faster than the primary DNS servers
  • 19. K15298: Overview of the dnsxdump utility You can use the dnsxdump utility to view the DNS Express database information, which includes zone information and statistics. • The DB Dump section of the dnsxdump utility output displays the zone information for all configured DNS Express zones. • The DB Stats section of the dnsxdump utility output displays a cumulative count of records for all configured DNS Express zones. dnsxdump > /var/tmp/my_zones.txt
  • 21. Managing the BIG-IP BIND configuration file ZoneRunner utility is used to manage both DNS zone files and the BIND configuration file on the BIG-IP GTM system • Import and transfer DNS zone files • Manage zone resource records • Manage views • Manage a local nameserver and the associated configuration file, named.conf • Transfer zone files to a nameserver • Import only primary zone files from a nameserver
  • 22. Cont* By default, BIG-IP GTM is configured to secure BIND to not allow zone transfers except from the localhost. Modify the allow-transfer statement to include the IP address of the GTM. You can modify the following allow-transfer statement to use the IP address of the GTM. DNS > Zones > ZoneRunner > named Configuration. allow-transfer { localhost; 192.168.10.105; } To verify zone transfers are working properly # dig @<IP address> es.net. axfr Directory of the zone file stored # cd /var/named/config/namedb/ Check the named configuration
  • 23. K7032: Freezing zone files to allow manual update to ZoneRunner- managed zone files All changes made to a zone using dynamic update are written to the zone's journal file. When the BIG-IP DNS system restarts after a shutdown, the system replays the journal file to incorporate any updates that took place after the last zone file update into the zone. Dynamic update periodically flushes the complete contents of the updated zone to its zone file and automatically deletes the journal file. i. cd /var/named/config/namedb ii. cp <zone_filename> <zone_filename>.original iii. bigstart stop zrd iv. rndc freeze <zone name> <class> <view> v. Manually edit the zone for any changes vi. rndc sync -clean vii. Run the named-checkzone command to check the file for any syntax errors • named-checkzone askf5.net db.external.askf5.net • named-checkconf -t /var/named -z -j /config/named.conf viii. rndc thaw <zone name> <class> <view> ix. bigstart start zrd .
  • 24. Behaviour of zrd - When a new dns express zone is added , it writes the data to zxfrd.bin - It then copies from zxfrd.bin to zxfrd-tmp.bin (15sec timer) - Rename the zxfrd-tmp.bin to tmmdns.bin (database) TMM then reload the database from tmmdns.bin - For VIPRION, csyncd monitors tmmdns.bin for any changes - Csyncd trigger tmm to reload on primary blades then populate to other blades # bigstart stop # rm -rf /var/db/{tmmdns.bin,zxfrd.bin} # bigstart start tmsh modify sys db log.zxfrd.level value debug Collect qkview and full tar ball for review. #qkview –s0 #tar -czvpf /var/tmp/$HOSTNAME-logs.tgz /var/log