Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Troubleshooting Provisioning Services Target Boot Processes

47,765 views

Published on

Citrix Provisioning Services (PVS) functionality targets network client machines that boot and connect to provisioning services servers under the guidelines specified in the PXE 2.1 specification. This in-depth technical session will provide a step-by-step detailed overview of the target boot process used by provisioning services versions 5.6.x and 6.x.

In this session you will learn about:
• Components and architecture of the PVS boot process
• Troubleshooting PVS boot issues
• Load balancing and failover technology

Published in: Technology

Troubleshooting Provisioning Services Target Boot Processes

  1. 1. Citrix Support Secrets Webinar Series Troubleshooting Provisioning Services Target Boot Processes Konstantin Cvetanov– PVS Escalation Engineer December 5th 2013
  2. 2. PVS Boot Process Demystified! PVS is not magic PVS does not use alien technology PVS is cool technology 2 © 2013 Citrix
  3. 3. Agenda • PXE Specification • PVS Boot Process ᵒIP Acquisition ᵒBootstrap Download ᵒPVS Logon Process ᵒSingle read Mode ᵒBNISTACK / MIO Read Mode • Resources • Questions 3 © 2013 Citrix
  4. 4. PXE Specification
  5. 5. PXE (Pre-boot Execution Environment) DHCP Server 1 2 3 TFTP Server 4 5 PXE Client 6 5 © 2013 Citrix NFS Server
  6. 6. Normal PC Boot Process (abridged) Target Device PC starts up BIOS  POST Identify boot device Load operating system 6 © 2013 Citrix
  7. 7. Normal PC Boot vs. PXE Boot Target Device PC starts up PVS Server BIOS  POST Identify boot device PXE boot: Load Get IP address operating system Get NBP name / location Local HDD Get NBP vDisk NBP initializes Connect to network resource Load operating system 7 © 2013 Citrix
  8. 8. PVS Boot Process
  9. 9. PVS Boot Process • The following five steps make up the PVS Boot Process: ᵒIP Acquisition ᵒBootstrap Download ᵒPVS Logon Process ᵒSingle Read Mode ᵒBNISTACK / MIO Read Mode 9 © 2013 Citrix
  10. 10. IP Acquisition
  11. 11. DHCP – Using Scope Options DHCP Server PVS Server DHCP ACK DHCPDISCOVER DHCP REQUEST DHCPOFFER TFTP Server Name Bootfile Name DHCP Client 11 © 2013 Citrix
  12. 12. DHCP – with PXE Service DHCP Server PVS Server DHCP ACK DHCPDISCOVER SERVICE DHCP REQUEST PVS DHCPOFFERPXE TFTP Server Name Bootfile Name DHCP Client 12 © 2013 Citrix
  13. 13. Citrix Boot ISO with Statically Assigned IPs 13 © 2013 Citrix
  14. 14. IP Acquisition - Troubleshooting Spanning Tree Protocol 14 © 2013 Citrix
  15. 15. Capture Software 15 © 2013 Citrix
  16. 16. Capture Guidelines • Whenever possible capture both a server-side and target side trace • Try to capture a good vs. bad trace • Get all of the IP/MAC information pertaining to a trace 16 © 2013 Citrix
  17. 17. Target Side capture methods • XenServer ᵒTCP Dump ᵒhttp://support.citrix.com/article/CTX120869 • VMware ᵒSet V-Switch into promiscuous mode ᵒhttp://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1004 099 • Hyper-V ᵒHost OS tracing ᵒOnly works if the traffic leaves the Hyper-V Machine 17 © 2013 Citrix
  18. 18. Target Side capture methods – cont. • Physical ᵒPort Mirroring ᵒUsing a Tap 18 © 2013 Citrix
  19. 19. Bootstrap Download
  20. 20. Bootstrap Download ARDBP32.BIN – UDP Port 69 Citrix PVS TFTP Service TSBBDM.BIN – UDP Port 6969 Citrix PVS Two-Stage Boot Service TSBOROM.BIN – UDP Port 6969 20 © 2013 Citrix Citrix PVS Two-Stage Boot Service
  21. 21. Configuring the Bootstrap File 21 © 2013 Citrix
  22. 22. Bootstrap Download - Troubleshooting 22 © 2013 Citrix
  23. 23. Packet Capture PXE TFTP download Provisioning Server 10.1.1.1 TFTP DHCP PXE Target 10.1.1.100 23 © 2013 Citrix
  24. 24. Common Issues During DHCP TFTP Download • Target does not receive IP address • Target gets IP but times out to TFTP • PXE Errors No bootstrap found Provisioning Server 10.1.1.1 TFTP DHCP PXE Target 10.1.1.100 24 © 2013 Citrix
  25. 25. PVS Logon Process
  26. 26. PVS Protocol • UDP based • Layer 7 additions – BNISTACK.SYS ᵒSequencing/ Fragments ᵒAcknowledgements ᵒRetry mechanism ᵒHeartbeat 26 © 2013 Citrix
  27. 27. PVS Login Sequence Target Device What Login IP:Port can I use? Use this Login IP:Port Login Server – Log me in; here’s my MAC address Use this least busy server Least busy server – what IO port should I use? Use this IO port Service my request for this disk on this device All set – begin streaming 27 © 2013 Citrix PVS Farm
  28. 28. Login Process Traffic Provisioning Server 10.1.1.1 TFTP Stream Process 28 © 2013 Citrix TFTP - Login Target 10.1.1.100
  29. 29. PVS Login – Load Balancing 29 © 2013 Citrix
  30. 30. PVS Login - Troubleshooting • Incorrect bootstrap information • Stream service not started • Target device does not exist in database • UDP port 6910 is blocked 30 © 2013 Citrix
  31. 31. Single Read Mode
  32. 32. Single Read Mode 32 © 2013 Citrix
  33. 33. Single Read Mode streaming Stream Process 33 © 2013 Citrix OS Booting UNDI Driver Target 10.1.1.100
  34. 34. Single Read Mode - troubleshooting 34 © 2013 Citrix
  35. 35. Slow Boot & Performance troubleshooting • Network or Storage ᵒSeparate the two ᵒMove vdisk local to PVS server • Storage related ᵒTry different LUN or different NAS device ᵒTry using locally attached LUN vs CIFS share • Network related ᵒBidirectional network trace ᵒCheck all network devices for old or faulty connections 35 © 2013 Citrix
  36. 36. BNISTACK / MIO Read Mode
  37. 37. BNISTACK / MIO Read Mode BNISTACK • • • • • 37 © 2013 Citrix Image Mode Active Directory Write Cache Client Name Licensing
  38. 38. BNISTACK / MIO Read Mode 38 © 2013 Citrix
  39. 39. MIO Read Mode Provisioning Server 10.1.1.1 Stream Process 39 © 2013 Citrix Target 10.1.1.100
  40. 40. BNISTACK / MIO - Troubleshooting Provisioning Services Antivirus Best Practices http://support.citrix.com/article/CTX124185 40 © 2013 Citrix
  41. 41. Retry Mechanism and HA Failover • Target requests data block from vdisk ᵒIf the server does not respond completely within 1 second timeout a retry is attempted • During retry the timeout is doubled to 2 seconds ᵒIf the server does not respond completely within 2 seconds the retry process repeats • Each retry doubles the timeout to a maximum of 10 seconds • After 10 consecutive retries for the same data an HA retry occurs ᵒTarget device refers to the bootstrap for login info 41 © 2013 Citrix
  42. 42. PVS Login Sequence – HA Reconnect Target Device What Login IP:Port can I use? PVS Farm Use this Login IP:Port HA reconnect begins here Login Server – Log me in; here’s my MAC address Use this least busy server Least busy server – what IO port should I use? Use this IO port Service my request for this disk on this device All set – begin streaming 42 © 2013 Citrix Non-HA reconnect begins here
  43. 43. PVS Boot Process Summary • Here is a summary of the PVS Boot Process: ᵒIP Acquisition – The Target Device acquires an IP address. ᵒBootstrap Download – The bootstrap file is downloaded. ᵒPVS Logon Process – The Target Device logs on to PVS. ᵒSingle Read Mode – Single read mode communication is established between the Target Device and the PVS Server. ᵒBNISTACK / MIO Read Mode – The BNISTACK driver on the Target Device takes over communications with the PVS Server and Multiple I/O occurs. 43 © 2013 Citrix
  44. 44. PVS Boot Process Diagram
  45. 45. Citrix Provisioning Services (PVS) Boot Process 1 2 IP Acquisition The PVS target device acquires an IP address using the DORA process. The following configurations are supported: 4 Bootstrap Download An IP Address is obtained using the following Discover, Offer, Request and Acknowledge (DORA) process. BNISTACK / MIO During the final phase, the BNISTACK Driver is loaded and multiple I/O (MIO) begins BNISTACK DHCP The TFTP Server Name is obtained from DHCP option 66 and the filename is obtained from DHCP option 67. 1. The Target Device broadcasts DHCP Discover packets. Simple Requests & Replies DHCP Discover Broadcast Target Device 5 After a target device has logged into PVS and has been directed to a PVS Server for streaming, the bootstrap file (ARDBP32.bin, TSBBDM.bin or TSBOROM.bin) will then intercept any requests (i.e. hard disk requests) made to Interrupt 13. The bootstrap file is downloaded from the TFTP Server to the Target Device using the TFTP Server from DHCP option 66 and filename (ARDBP32.bin) from DHCP option 67. TSBBDM.bin is downloaded if Boot Device Manager (BDM) is used and TSBOROM.bin is used if the target devices use option ROM (OROM). DHCP Single Read Mode v1.4 ARDBP32.bin 2. The DHCP Server sends a DHCP Offer packet to the DHCP Target Device with the IP address, Subnet Mask, lease time. Default Gateway and DNS Server Domain Name information Server to the Target Device. 3. The Target Device sends a unicast message to the DHCP Server requesting the offered IP address. A Transaction ID is used to track the accepted offer. The Target Device will send a broadcast message notifying other DHCP Servers that the offer from another DHCP Server was accepted. DHCP Request Target Device PVS Server Target Device DHCP Offer Packet Target Device The Target Device and the PVS Server will continue to communicate exchanging vDisk data until the Microsoft PVS Windows Operating Systems starts loading drivers and Server BNISTACK is successfully loaded. MIO Boot Device Manager (BDM) During the MIO stage, the Target Device is operation and read /write requests occur as follows: Typical Read Requests BDM uses a two-stage boot process where the PVS location is hardcoded into the bootstrap generated by BDM. The rest of the required boot information like the low level PVS device drivers is downloaded from the PVS Server using a proprietary download protocol based on TFTP that uses UDP port 6969. 4. The DHCP Server sends a DHCPACK packet to the Target Device. DHCP Acknowledgement Single Read Request Packet M ul PVS ti Server pl e R Typical Write Requests* e a M d u R l e t pl i PVS Target y p Server Single Write Reply P Confirmation Packet l Device ac e ke W ts r *MIO writes occur when the vDisk is in private mode or the write i cache is configured for server side cache. t e TSBBDM.bin Network Booting – w/DHCP Options (no PXE Service) In this configuration, the DORA process explained above is used in addition to the following: Target Device PVS Server Target Device DHCP Offer Packet Target Device The TFTP Server Name and Bootfile Name (ardbp32.bin) is provided using options 66 & 67 The Target Device initiates a handshake with the PVS Server stating the BNISTACK driver is up. BNISTACK is loaded into memory and takes over for the bootstrap PVS Server managing the MIO communication. At this point , the following information is exchanged: • vDisk name • Image Mode • Active Directory Password Management Option • Write Cache Type and Size • Client Name • Licensing DHCP Server Option ROM (OROM) On Target Devices that use option ROM, the bootstrap file name is configured on the network interface card (NIC) . Network Booting – w/PXE Service (no DHCP Options) TSBOROM.bin In this scenario, the Target Device is configured for network boot. Target Device 1. The firmware of the Target Device adds option 60 to the DHCP Discover packet being broadcast. This option is used with devices from Dell and others that use the Intel Desktop Board. Further information can be found at http://download.intel.com/products/motherb oard/DQ45CB/323251.pdf PVS Server R e q u e s t DHCP Discover Broadcast Target Device 2. DHCP Server responds with IP Address, Gateway and Subnet information. DHCP Server Reply 3. The PXE Server replies with the TFTP Server address and bootstrap file name. PXE & DHCP Servers PXE Server Reply 4. The Target Device sends a request to the TFTP Server for the bootstrap file. 3 PVS Logon Process Bootstrap File Reply When using a Citrix Boot ISO to download the bootstrap the Target Device gets the IP address information directly from the boot ISO and downloads the bootstrap manager. As configured in the Boot Device Management (BDM) configuration. Citrix Boot ISO – DHCP In this scenario, BDM is configured for the Target Device to use the DORA process to get the IP address information. Then the two stage bootstrap download will start. Get IO Port Get Login Port TFTP Server Citrix Boot ISO – Static IP 45 Load Balancing Algorithm After the Target Device gets an IP address and downloads the bootstrap file it proceeds to login to a PVS Server to start streaming the vDisk image. This steps are as follows: Bootstrap File Request 5. The TFTP Server replies with the bootstrap file name. • Target Device • The Target Device contacts the PVS Server specified in the bootstrap file using the default UDP port 6910. PVS Server The server responds with the IP address and port to Continue the login process. The Target Device requests the IP address and port used for the single read mode. PVS Server The Target Device requests the PVS Server start the I/O thread and requests information on which vDisk to use. Target Device Get IO Service Login Start • The Target Device identifies itself by its MAC address and specifies the type of login being requested. Login Complete • • Transferred to IO • The PVS Server moves the Target Device from the login thread to the I/O thread. The PVS Server replies with all disk, client and policy information needed and sent to the Target Device. P a c k e t PVS Boot Process s Summary The PVS Server grants access for the I/O operation to the Target Device. Additional configuration is sent specifying the boot device. Get vDisk Information • • The Target Device requests which vDisk will be streamed. The PVS Server replies with the vDisk information including write cache location if the Target is in standard mode. © 2013 Citrix © 2013 Citrix. Citrix Provisioning Services is either a registered trademark or a trademark of Citrix in the United States and/or other countries. All rights reserved. Other trademarks or trade names mentioned herein are the property of their respective owners. All PVS servers are capable of acting as both a login server and an I/O server. A PVS login server normally attempts to load balance devices between all servers that have access to a given vDisk when the device initially logs in. The login server only bypasses load balancing if the server override property is set for the vDisk locator. The load balancing algorithm provides simple connection count balancing. (i.e. the login server attempts to place the same number of devices on each server that has access to a given vDisk.) The following summarizes the diagram: detailed on the 1. 2. 3. 4. IP Acquisition – The Target Device acquires an IP address. Bootstrap Download – The bootstrap file is downloaded. PVS Logon Process – The Target Device logs on to PVS. Single Read Mode – Single read mode communication is established between the Target Device and the PVS Server. 5. BNISTACK / MIO – The BNISTACK driver on the Target Device takes over communications with the PVS Server and Multiple I/OCitrix Technical Support/Provisioning Services Escalation Authors: occurs. Team Please submit feedback to http://support.citrix.com/article/CTX136378
  46. 46. Resources • Provisioning Services Boot Process Diagram http://support.citrix.com/article/CTX136378 • Provisioning Services Antivirus Best Practices http://support.citrix.com/article/CTX124185 • Dynamic Host Configuration Protocol (DHCP) and Bootstrap Protocol (BOOTP) Parameters - http://www.iana.org/assignments/bootp-dhcp-parameters/bootpdhcp-parameters.xml • Preboot Execution Environment (PXE) Specification Version 2.1 – http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf • Provisioning Services eDocs – http://support.citrix.com/proddocs/topic/technologies/pvs-provisioning.html 46 © 2013 Citrix
  47. 47. About Citrix Services Citrix Services make sure you succeed with your virtualization programs. Educate | Guide | Support | Succeed How we can help Citrix Education – The fastest, most efficient way to get your team the virtualization skills they need. Online, on-site or in class. citrix.com/training Citrix Consulting – Intensive engagements for complex, critical or just plain massive projects. citrix.com/consulting Citrix Support – Always-on support services that leverage everything we know about best-practice deployment and maintenance. citrix.com/support © 2013 Citrix
  48. 48. Secrets of the Citrix Support Ninjas • 40 insider troubleshooting tips • Covering XenDesktop, XenServer, XenApp and NetScaler • Citrix Support top engineers • FREE eBook • Citrix Auto Support • Now available! © 2013 Citrix
  49. 49. Premier Support Calculator Check it out © 2013 Citrix
  50. 50. Next Webinar: January 2014 Title: 10 Tips Every XenDesktop Admin Should Know Description: As a XenDesktop administrator, it’s your responsibility to make sure your XenDesktop infrastructure is running smoothly and your users are working efficiently. In this webinar our very own XenDesktop master, Michael Glover will guide you in utilizing our award winning solution for virtual desktop delivery, XenDesktop, with some really key tips and time savers to help you make the very most of your investment. Michael has been involved in XenDesktop from day one and has gathered a huge amount of knowledge which he would love to share with you. • When: Jan 30th • Registration Now! © 2013 Citrix
  51. 51. Work better. Live better.

×