• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Platform Disk Support 2
 

Platform Disk Support 2

on

  • 3,841 views

 

Statistics

Views

Total Views
3,841
Views on SlideShare
3,841
Embed Views
0

Actions

Likes
0
Downloads
23
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Slide 34 - Sun SAS/SATA Platforms Now you understand the technology lets talk about the implementation of SAS and SATA in the Sun product range. What you will see from this table is that many platforms share a common or similar SAS or SATA controller. This is deliberate and makes driver development and maintenance more manageable. From the left, we have the product marketing name, internal product code, device storage type, controller, RAID levels supported and the drive form factor. DISCLAIMER: Some platforms are still under development so specifications may have changed since the platform documentation was created. Blade servers as an example change design very frequently due to market requirements. At the bottom of the slide the legend details marketing name and ASIC name for the Nvidia class MCP's otherwise known as Media Communications Processors. Media Communications Processors are the central bus generators for Nvidia chip sets. These extended bridge chips generate most platform buses which include the SATA controller. What is important to note with the LSI controllers is the end letter or to be exact the lack of a letter. Typically, the PCI express version of the controller ends with an “e”. The PCI and PCI-X version of the HBA does not have a letter but is more commonly referred to as the “x” version.
  • Slide 34 - Sun SAS/SATA Platforms Now you understand the technology lets talk about the implementation of SAS and SATA in the Sun product range. What you will see from this table is that many platforms share a common or similar SAS or SATA controller. This is deliberate and makes driver development and maintenance more manageable. From the left, we have the product marketing name, internal product code, device storage type, controller, RAID levels supported and the drive form factor. DISCLAIMER: Some platforms are still under development so specifications may have changed since the platform documentation was created. Blade servers as an example change design very frequently due to market requirements. At the bottom of the slide the legend details marketing name and ASIC name for the Nvidia class MCP's otherwise known as Media Communications Processors. Media Communications Processors are the central bus generators for Nvidia chip sets. These extended bridge chips generate most platform buses which include the SATA controller. What is important to note with the LSI controllers is the end letter or to be exact the lack of a letter. Typically, the PCI express version of the controller ends with an “e”. The PCI and PCI-X version of the HBA does not have a letter but is more commonly referred to as the “x” version.
  • Slide 35 – LSI 1064x The LSI 1064 or 1064x controller is the most common amongst x64 and SPARC platforms. The 1064 is a member of the MPT Fusion family of HBAs which were first seen in Sun on the v20z and v440 platforms. The v20z had a single bus LSI1020 controller and the v440 had a dual channel LSI 1030 controller. The MPT Fusion range of HBAs fuse an ARM compliant processor with memory and the physical disk interface. ARM compliant processors are commonly found in PDAs, cell phones and set top boxes around the home. There are two variants of the 4 port LSI 1064 that we at Sun use. The 1064 and 1064e. The 1064 usually sits on a digital PCI-X bus but in some implementations we use the HBA on a standard PCI bus. The 1064e is mounted on a serial PCI express bus. Time to market of the LSI 1064e meant that most products used the 1064 on a PCI-X bus. This was due to a few bugs on the PCI express version that were resolved late 2005. Products like the T2000 Ontario had to use a 1064 on a PCI-X card while the PCI express version was being fixed. Here is an overview of the LSI 1064 specifications .
  • Slide 35 – LSI 1064x The LSI 1064 or 1064x controller is the most common amongst x64 and SPARC platforms. The 1064 is a member of the MPT Fusion family of HBAs which were first seen in Sun on the v20z and v440 platforms. The v20z had a single bus LSI1020 controller and the v440 had a dual channel LSI 1030 controller. The MPT Fusion range of HBAs fuse an ARM compliant processor with memory and the physical disk interface. ARM compliant processors are commonly found in PDAs, cell phones and set top boxes around the home. There are two variants of the 4 port LSI 1064 that we at Sun use. The 1064 and 1064e. The 1064 usually sits on a digital PCI-X bus but in some implementations we use the HBA on a standard PCI bus. The 1064e is mounted on a serial PCI express bus. Time to market of the LSI 1064e meant that most products used the 1064 on a PCI-X bus. This was due to a few bugs on the PCI express version that were resolved late 2005. Products like the T2000 Ontario had to use a 1064 on a PCI-X card while the PCI express version was being fixed. Here is an overview of the LSI 1064 specifications .
  • Slide 35 – LSI 1064x The LSI 1064 or 1064x controller is the most common amongst x64 and SPARC platforms. The 1064 is a member of the MPT Fusion family of HBAs which were first seen in Sun on the v20z and v440 platforms. The v20z had a single bus LSI1020 controller and the v440 had a dual channel LSI 1030 controller. The MPT Fusion range of HBAs fuse an ARM compliant processor with memory and the physical disk interface. ARM compliant processors are commonly found in PDAs, cell phones and set top boxes around the home. There are two variants of the 4 port LSI 1064 that we at Sun use. The 1064 and 1064e. The 1064 usually sits on a digital PCI-X bus but in some implementations we use the HBA on a standard PCI bus. The 1064e is mounted on a serial PCI express bus. Time to market of the LSI 1064e meant that most products used the 1064 on a PCI-X bus. This was due to a few bugs on the PCI express version that were resolved late 2005. Products like the T2000 Ontario had to use a 1064 on a PCI-X card while the PCI express version was being fixed. Here is an overview of the LSI 1064 specifications .
  • Slide 36 – LSI 1064e The LSI 1064e is similar to the 1064 but interfaces with the host computer using the serial PCI express protocol. The 1064e can be connected to 1, 4 or 8 PCI express lanes from the NVIDIA or NEC PCI express bridge chip bus generator. Speeds and model number or the ARM CPU differ slightly but the overall performance is similar. The PCI express implementation is point to point which makes up in performance for the higher latency serial bus link. It is important to note that the firmware must be updated when available as recognition of new disks and their capacities is required, as well as generic bug fixes.
  • Slide 39 – LSI 1068x and e The LSI 1068 variant of the LSI HBA is basically a 1064 controller with 8 PHYs rather than 4 in the form of 2 transport modules. The core logic ARM cpu is as in the 1064 controller but the 1068 adds a second 4 port transport module in the mix. Specificaitons are similar for both the PCI-X digital bus and PCI express serial bus versions. As before, the v440 family of platforms is first to implement this new HBA with the v445 server. Raidctl is still used to create and administer RAID arrays but remember to patch Raidctl as it is common that the base operating system's revision of raidctl often does not support the latest LSI ASIC.
  • Slide 40 – LSI 1078x and e The LSI 1078 HBA is being marketed as a ROC design meaning RAID on chip. This controller is an option on the V445 platform and adds new support for RAID 5. System administrators using this controller will have to use raidctl with the switch -r 5 to generate a RAID 5 disk array.
  • Slide 37 – Solaris Patches Although the LSI SAS controllers are supported on Solaris 10 and some Solaris 9 platforms, LSI MPT Fusion patches and firmware do exist for Solaris 8 also. This is because the v440 supports Solaris 8. Solaris 9 versions of the x64 and SPARC patches are available on SunSolve and an ever increasing number of new platforms are being qualified for Solaris 9 due to customer pressure based on slow migration to Solaris 10. The LSI SAS and SATA drivers have now become part of the jumbo kernel update for Solaris 10 x64. It is expected that SPARC platform patches will follow. The above patch revisions were correct at the time of presentation creation. Patch increments may have changed since.
  • Slide 41 – Nvidia NF2050 and 2200 The Nvidia HBA is not actually a single ASIC. The SATA controller is actually built into the Nforce 2200 media communications processor which is also responsible for generating PCI, PCIe, USB and legacy buses. The Nforce 2050 companion chip is similar in design to the 2200 but with reduced functionality that provides another 4 PHYs and a fixed PCI express lane configuration. RAID 0 and 1 are available with this HBA however the RAID levels provided by this HBA are more of a software implementation rather than a hardware RAID solution. When the NVRAID BIOS is enabled, this RAID array looks for and executes a special boot sector which includes detailed disk member information. This boot block contains the RAID configuration which is later read by the special Nvidia storage driver. Without the storage driver, RAID will not correctly work on the platform and the individual disks will be seen rather than the group array.
  • Slide 42 – Nvidia NF3050/3400 The Nforce 3400 media communications processor and its companion the Nforce 3050 I/O are the new chipset found in Opteron AM2 and Socket F platforms. The chips contain an updated set of features and more functionality over the original NF2200 and NF2050 MCP's RAID 5 is added as an array option for this controller.
  • Slide 37 – Solaris Patches Although the LSI SAS controllers are supported on Solaris 10 and some Solaris 9 platforms, LSI MPT Fusion patches and firmware do exist for Solaris 8 also. This is because the v440 supports Solaris 8. Solaris 9 versions of the x64 and SPARC patches are available on SunSolve and an ever increasing number of new platforms are being qualified for Solaris 9 due to customer pressure based on slow migration to Solaris 10. The LSI SAS and SATA drivers have now become part of the jumbo kernel update for Solaris 10 x64. It is expected that SPARC platform patches will follow. The above patch revisions were correct at the time of presentation creation. Patch increments may have changed since.
  • Slide 43 – Marvell 88SX6081 The Marvell 88SX6081 controller is a low cost ASIC built into the Thumper x4500 platform. The x4500 was originally ear marked to use the LSI 1068 however time to market and cost of 6 full SAS/SATA ASIC's meant that the Marvell SATA only controller was a better choice for Thumber. This SATA controller does not incorporate any hardware RAID functions. The controller was selected for Thumper because Thumper uses ZFS for RAID functions.
  • Slide 37 – Solaris Patches Although the LSI SAS controllers are supported on Solaris 10 and some Solaris 9 platforms, LSI MPT Fusion patches and firmware do exist for Solaris 8 also. This is because the v440 supports Solaris 8. Solaris 9 versions of the x64 and SPARC patches are available on SunSolve and an ever increasing number of new platforms are being qualified for Solaris 9 due to customer pressure based on slow migration to Solaris 10. The LSI SAS and SATA drivers have now become part of the jumbo kernel update for Solaris 10 x64. It is expected that SPARC platform patches will follow. The above patch revisions were correct at the time of presentation creation. Patch increments may have changed since.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 44 – Uli M1575 The Uli M1575 PCI express bridge is similar in design to the Nvidia Nforce 2200 MCP. This is a low cost bridge chip comes with 4 SATA 2 ports and 2 IDE ports as well as a host of other buses. RAID levels include 0, 1, 0+1 and 5 however Sun does not use the RAID features of this controller. The Netra CT 900 platform's CP3060 Montoya blade uses this controller as a low cost alternative to adding a separate LSI 1064 controller since the Uli already contains SATA logic.
  • Slide 45 – Solaris SATA Driver Solaris 10 Update 2 included SATA driver 1.3 which replaces the traditional ata driver for connecting to SATA disks. Improvements include correct device number addressing, Improved DMA. Support for SATA features like native command queueing. Support for the full SATA 2 specification.
  • Slide 46 – Solaris “raidctl” Usage As you can see from this slide, the new raidctl command includes support for a new switch -r. The -r switch allows the user to define the RAID level to be created. Depending on platform specifics, -r 0 can be used to create a stripe. -r 1 can be used to create a mirror and -r 5 can be used to create a distributed parity array. Firmware of the HBA can be upgraded by using the -F switch and then by specifying the firmware file and location.
  • iSCSI - The Internet Small Computer Systems Interface (iSCSI) protocol defines the rules and processes to transmit and receive block storage applications over TCP/IP networks by encapsulating SCSI commands into TCP and transporting them over the network via IP. iSCSI describes: * Transport protocol for SCSI which operates on top of TCP * New mechanism for encapsulating SCSI commands on an IP network * Protocol for a new generation of data storage systems that natively use TCP/IP
  • An architecture of a pure SCSI is based on the client/server model. A client, for example, server or workstation, initiates requests for data reading or recording from a target - server, for example, a data storage system. Commands which are sent by the client and processed by the server are put into the Command Descriptor Block (CDB). The server executes a command which completion is indicated by a special signal alert. Encapsulation and reliable delivery of CDB transactions between initiators and targets through the TCP/IP network is the main function of the iSCSI, which is due to be implemented in the medium untypical of SCSI, potentially unreliable medium of IP networks. The Diagram depicts a model of the iSCSI protocol levels which allows us to get an idea of an encapsulation order of SCSI commands for their delivery through a physical carrier. The iSCSI protocol controls data block transfer and confirms that I/O operations are truly completed. In its turn, it is provided via one or several TCP connections.
  • i Benefits of IP storage * IP storage leverages the large installed base of Ethernet-TCP/IP networks and enables storage to be accessed over LAN, MAN, or WAN environments, without needing to alter storage applications. * It also lets IT managers use the existing Ethernet/IP knowledge base and management tools. * It provides for consolidation of data storage systems · Data backup · Server clusterization · Replication · Recovery in emergency conditions * To transfer data to storage devices with the iSCSI interface it's possible to use not only data carriers, communicators and routers of existent LAN/WAN but also usual network cards on the client's side. * The conception of building the World Wide Storage Area Network excellently fits in the development of modern IP Storage technologies. * Maximize storage resources to be available to more applications; * Use existing storage applications (backup, disaster recovery, and mirroring) without modification; and * Manage IP-based storage networks with existing tools and IT expertise.
  • How does it work? How iSCSI works iSCSI defines the rules and processes to transmit and receive block storage applications over TCP/IP networks. At the physical layer, iSCSI supports a Gigabit Ethernet interface so that systems supporting iSCSI interfaces can be directly connected to standard Gigabit Ethernet switches and/or IP routers. The iSCSI protocol sits above the physical and data-link layers and interfaces to the operating system's standard SCSI Access Method command set. iSCSI enables SCSI-3 commands to be encapsulated in TCP/IP packets and delivered reliably over IP networks. iSCSI can be supported over any physical media that supports TCP/IP as a transport, but today's iSCSI implementations are on Gigabit Ethernet. The iSCSI protocol runs on the host initiator and the receiving target device. iSCSI can run in software over a standard Gigabit Ethernet network interface card (NIC) or can be optimized in hardware for better performance on an iSCSI host bus adapter (HBA). iSCSI also enables the access of block-level storage that resides on Fibre Channel SANs over an IP network via iSCSI-to-Fibre Channel gateways such as storage routers and switches. In the diagram, each server, workstation and storage device support the Ethernet interface and a stack of the iSCSI protocol. IP routers and Ethernet switches are used for network connections.
  • i Limitations of ISCSI * In IP, packets are delivered without a strict order, it is also in charge of data recovery, which takes more resources. At the same time, in SCSI, as a channel interface, all packets must be delivered one after another without delay, and breach of the order may result in data losses. (iSCSI has managed to solve this problem to some degree requiring a longer packet's head). The head includes additional information which speeds up packet assembling by a great margin. * Considerable expenses of processor power on the client's side which uses such card. According to the developers, the software iSCSI realization can reach data rates of Gigabit Ethernet at a significant, about 100%, CPU load. That is why it is recommended using special network cards which support mechanisms of CPU unload before TCP stack processing. * latency issues - Although there are a lot of means developed to reduce influence of parameters which cause delays in processing of IP packets, the iSCSI technology is positioned for middle-level systems.
  • Address and Naming Conventions As the iSCSI devices are participants of an IP network they have individual Network Entities. Such Network Entity can have one or several iSCSI nodes. An iSCSI node is an identifier of SCSI devices (in a network entity) available through the network. Each iSCSI node has a unique iSCSI name (up to 255 bytes) which is formed according to the rules adopted for Internet nodes. For example, fqn.com.ustar.storage.itdepartment.161. Such name has an easy-to-perceive form and can be processed by the Domain Name System (DNS). An iSCSI name provides a correct identification of an iSCSI device irrespective of its physical location. At the same time in course of handling data transfer between devices it's more convenient to use a combination of an IP address and a TCP port which are provided by a Network Portal. The iSCSI protocol together with iSCSI names provides a support for aliases which are reflected in the administration systems for better identification and management by system administrators.
  • Session Management The iSCSI session consists of a Login Phase and a Full Feature Phase which is completed with a special command. The Login Phase of the iSCSI is identical to the Fibre Channel Port Login process (PLOGI). It is used to adjust various parameters between two network entities and confirm an access right of an initiator. If the iSCSI Login Phase is completed successfully the target confirms the login for the initiator; otherwise, the login is not confirmed and a TCP connection breaks. As soon as the login is confirmed the iSCSI session turns to the FULL Feature Phase. If more than one TCP connection was established the iSCSI requires that each command/response pair goes through one TCP connection. Thus, each separate read or write command will be carried out without a necessity to trace each request for passing different flows. However, different transactions can be delivered through different TCP connections within one session. At the end of a transaction the initiator sends/receives last data and the target sends a response which confirms that data are transferred successfully. The iSCSI logout command is used to complete a session - it delivers information on reasons of its completion. It can also send information on what connection should be interrupted in case of a connection error, in order to close troublesome TCP connections.
  • Error Handling Because of a high probability of errors in data delivery in some IP networks, especially WAN, where the iSCSI can work, the protocol provides a great deal of measures for handling errors. So that error handling and recovery can work correctly both the initiator and the target must be able to buffer commands before they are confirmed. Each terminal must have a possibility to recover selectively a lost or damaged PDU within a transaction for recovery of data transfer. Here is the hierarchy of the error handling and recovery after failures in the iSCSI: 1. The lowest level - identification of an error and data recovery on the SCSI task level, for example, repeated transfer of a lost or damaged PDU. 2. Next level - a TCP connection which transfers a SCSI task can have errors. In this case there is an attempt to recover the connection. 3. At last, the iSCSI session can be damaged. Termination and recovery of a session are usually not required if recovery is implemented correctly on other levels, but the opposite can happen. Such situation requires that all TCP connections be closed, all tasks, underfulfilled SCSI commands be completed, and the session be restarted via the repeated login.
  • Security As the iSCSI can be used in networks where data can be accessed illegally, the specification allows fpr different security methods. Such encoding means as IPSec which use lower levels do not require additional matching because they are transparent for higher levels, and for the iSCSI as well. Various solutions can be used for authentication, for example, Kerberos or Private Keys Exchange, an iSNS server can be used as a repository of keys.
  • Use 'iscsitadm' to set up iSCSI target devices. You'll need to provide an equivalently sized ZFS or UFS file system as the backing store for the iSCSI daemon. Use 'iscsiadm' to identify your iSCSI targets, which will discover and use the iSCSI target device.
  • How does it work? How iSCSI works iSCSI defines the rules and processes to transmit and receive block storage applications over TCP/IP networks. At the physical layer, iSCSI supports a Gigabit Ethernet interface so that systems supporting iSCSI interfaces can be directly connected to standard Gigabit Ethernet switches and/or IP routers. The iSCSI protocol sits above the physical and data-link layers and interfaces to the operating system's standard SCSI Access Method command set. iSCSI enables SCSI-3 commands to be encapsulated in TCP/IP packets and delivered reliably over IP networks. iSCSI can be supported over any physical media that supports TCP/IP as a transport, but today's iSCSI implementations are on Gigabit Ethernet. The iSCSI protocol runs on the host initiator and the receiving target device. iSCSI can run in software over a standard Gigabit Ethernet network interface card (NIC) or can be optimized in hardware for better performance on an iSCSI host bus adapter (HBA). iSCSI also enables the access of block-level storage that resides on Fibre Channel SANs over an IP network via iSCSI-to-Fibre Channel gateways such as storage routers and switches. Fig. 1. IP network with iSCSI devices used Here, each server, workstation and storage device support the Ethernet interface and a stack of the iSCSI protocol. IP routers and Ethernet switches are used for network connections.
  • Slide 47 – Sun Fire x4500 Prtdiag Output. Solaris 10 Update 2 x64 included a much improved ACPI layer which is similar in design to the PICL platform libraries found in the SPARC implementation of the O/S. This allows for the x64 version of prtdiag to be included. This slide shows an example of the x4500 prtdiag and as you can see the 6 Marvell controllers are listed towards the back end of the output.
  • Slide 47 – Sun Fire x4500 Prtdiag Output. Solaris 10 Update 2 x64 included a much improved ACPI layer which is similar in design to the PICL platform libraries found in the SPARC implementation of the O/S. This allows for the x64 version of prtdiag to be included. This slide shows an example of the x4500 prtdiag and as you can see the 6 Marvell controllers are listed towards the back end of the output.
  • Slide 47 – Sun Fire x4500 Prtdiag Output. Solaris 10 Update 2 x64 included a much improved ACPI layer which is similar in design to the PICL platform libraries found in the SPARC implementation of the O/S. This allows for the x64 version of prtdiag to be included. This slide shows an example of the x4500 prtdiag and as you can see the 6 Marvell controllers are listed towards the back end of the output.
  • Slide 47 – Sun Fire x4500 Prtdiag Output. Solaris 10 Update 2 x64 included a much improved ACPI layer which is similar in design to the PICL platform libraries found in the SPARC implementation of the O/S. This allows for the x64 version of prtdiag to be included. This slide shows an example of the x4500 prtdiag and as you can see the 6 Marvell controllers are listed towards the back end of the output.
  • Slide 47 – Sun Fire x4500 Prtdiag Output. Solaris 10 Update 2 x64 included a much improved ACPI layer which is similar in design to the PICL platform libraries found in the SPARC implementation of the O/S. This allows for the x64 version of prtdiag to be included. This slide shows an example of the x4500 prtdiag and as you can see the 6 Marvell controllers are listed towards the back end of the output.

Platform Disk Support 2 Platform Disk Support 2 Presentation Transcript

    • Anthony McNamara & William Watson
    • Systems TSC-VSP
    • Sun Microsystems
    Platform Storage & iSCSI Support
  • 1 – What is not covered by this TOI 2 – Controller types and functionality 3 – Controller firmware and patches 4 – Operating system connectivity 5 – Disk size and geometry 6 – RAID levels, building and recovery 7 – iSCSI concepts 8 – iSCSI requirements and building 9 – Abbreviations, issues & useful links
  • What is Not Covered by this TOI
    • PCI host bus adapters as this is embedded storage
    • Software RAID implementations
    • Solaris ZFS
    • Disk locator tools
    • Products that are not public knowledge or are under specific NDA with third parties
  • Controller Types and Functionality
  • Controller Types and Functionality
  • Controller Types and Functionality
  • Controller Performance
    • CF (DMA) - 33 MB per second
    • IDE DVD - 33 MB per second
    • IDE HDD - 133 MB per second
    • FC-AL - 120 MB per second
    • SATA 1 - 150 MB per second
    • SATA 2 - 300 MB per second
    • SAS - 300 MB per second
    • SCSI - 320 MB per second
  • LSI 1020/1030
    • 1020 is single channel, 1030 is dual channel
    • Common in x64 & SPARC platforms
    • Administered by /usr/sbin/raidctl in SPARC
    • Administered by BIOS & raidctl in x64
    • Runs on 64-bit PCI-X bus up to133MHz
    • Can run on 32-bit PCI bus at 33MHz (5v tolerant)
    • 2 x ARM966E-S processors on die
    • SCSI320 LVD parallel interface
  • LSI 1064x
    • Common in new x64 & SPARC platforms
    • Administered by /usr/sbin/raidctl in SPARC (init S)
    • Administered by BIOS & raidctl in x64
    • Runs on 64-bit PCI-X bus up to133MHz
    • Can run on 32-bit PCI bus at 33MHz (5v tolerant)
    • ARM 926 CPU at bus speed
  • LSI 1064e
    • Common in new x64 & SPARC platforms
    • Administered by /usr/sbin/raidctl in SPARC (init S)
    • Administered by BIOS & raidctl in x64
    • Runs on PCI express 1, 4 or 8 lane full duplex bus
    • ARM 966 CPU at 225 MHz
  • LSI 1068x and e
    • Original implemented for use on the Sun Fire X4500 but the ASIC was not available in time so not used
    • Administered by /usr/sbin/raidctl (init S)
    • 1068x runs on 66-133MHz PCI & PCI-X bus
      • (Not 5v tolerant so 3.3v only)
    • 1068e runs on PCIe 1, 4 or 8 lane full duplex bus
    • ARM 926 CPU at bus speed (x version)
    • ARM 966 CPU at 225MHz (e version)
    • 1068 is the same as 1064 but with 2 PHY modules
  • LSI 1078x and e
    • Single IC ROC design (RAID on Chip)
    • Option module for Sun Fire V445 platform (PCIe)
    • Administered by /usr/sbin/raidctl -r
    • 1078x runs on 66-133MHz PCI & PCI-X bus
      • (Not 5v tolerant so 3.3v only)
    • 1078e runs on PCIe 1, 4 or 8 lane, full duplex bus
    • Adds RAID 5 support
  • LSI Solaris Patches
    • Solaris 8 SPARC patch 115274-xx
    • Solaris 8 x64 patch (not supported)
    • Solaris 9 SPARC patch 115667-xx
    • Solaris 9 x64 patch 118559-xx (kernel)
    • Solaris 10 SPARC patch 119850-xx
    • Solaris 10 x64 patch 118855-xx (kernel)
  • NVidia NF2050/2200
    • NF2200 HT to PCIe bridge chip with MCP
      • (Media and Communications Processor)
    • NF2050 HT to PCIe bridge chip for I/O
      • (Reduced function NF2200 IC)
    • 4 PHY SATA support only, not SAS
    • RAID 0, 1 & 5 support via software driver
      • (RAID 5 on platforms with more than 2 x disks)
    • RAID currently supported under Windows only
  • NVidia NF3050/3400
    • NF3400 HT to PCIe bridge chip with MCP
      • (Media and Communications Processor)
    • NF3050 HT to PCIe bridge chip for I/O
      • (Reduced function NF3400 IC)
    • 4 PHY SATA support only, not SAS
    • RAID 0, 1 & 5 support via software driver
      • (RAID 5 on platforms with more than 2 x disks)
    • RAID currently supported under Windows only
  • Nvidia Solaris Patches
    • Solaris 8 SPARC patch (not applicable)
    • Solaris 8 x64 patch (not supported)
    • Solaris 9 SPARC patch (not applicable)
    • Solaris 9 x64 patch (not supported)
    • Solaris 10 SPARC patch (not applicable)
    • Solaris 10 x64 patch 118855-xx (kernel)
  • Marvell 88SX6081
    • Low cost SATA IC (Excludes SAS & RAID)
    • PCI and PCI-X signalling
    • 32-bit and 64-bit bus up to 133MHz
    • 8 x Gen2i programmable PHYs
    • Solaris 10 Update 2 support and above
    • Solaris Marvell driver marvell88sx (v 1.3)
    • Solaris sata driver (v 1.3) via SUNWckr
  • Marvell Solaris Patches
    • Solaris 8 SPARC patch (not applicable)
    • Solaris 8 x64 patch (not supported)
    • Solaris 9 SPARC patch (not applicable)
    • Solaris 9 x64 patch (not supported)
    • Solaris 10 SPARC patch (not applicable)
    • Solaris 10 x64 patch 118855-xx (kernel)
    • Solaris 10 x64 patch 119247-xx (man page update)
    • Solaris Generic Hitachi 500GB fw patch 124945-xx
    • Windows Hitachi 500GB fw patch 124955-xx
  • Uli M1575
    • Low cost 4 lane PCIe connected south bridge chip
    • Common on UltraSPARC T1 blade platforms
    • 4-port SATA 3Gbps with NCQ
    • 2 x ATA-133
    • Hosts RAID 0, RAID 1, RAID 0+1, RAID 5, & JBOD
    • 8-port USB 2.0
    • 10/100 Ethernet PHY
    • Sun do not utilize RAID or LAN on this controller
  • O/S Connectivity - LSI1020
    • Solaris
      • The raidctl command
    • Red Hat Enterprise Linux 3/4/5
      • CIM provider daemon and CIM Java browser v3.06.00
      • mpt-status
    • SuSE Enterprise Linux 9/10
      • CIM provider daemon and CIM Java browser v3.06.00
      • mpt-status
  • O/S Connectivity - LSI1020
    • Windows 32-bit
      • CIM browser for Windows v3.06.00
    • Windows 64-bit
      • Driver only
    • Windows Vista
      • Driver only
    • Windows Server 2008
      • Driver only
    • Download via:
      • http://www.lsilogic.com/cm/DownloadSearch.do
  • O/S Connectivity - LSI106x
    • Solaris
      • The raidctl command
    • Red Hat Enterprise Linux 3/4/5
      • LSI MyStorage
      • mpt-status
      • MSM-IR (Supplied on supplement CD 1.3 and above)
    • SuSE Enterprise Linux 9/10
      • LSI MyStorage
      • mpt-status
      • MSM-IR (Supplied on supplement CD 1.3 and above)
  • O/S Connectivity - LSI106x
    • Windows 32-bit
      • MSM-IR (Supplied on supplement CD 1.3 and above)
    • Windows 64-bit
      • MSM-IR (Supplied on supplement CD 1.3 and above)
    • Windows Vista
      • Driver only at present
    • Windows Server 2008
      • Driver only at present
  • O/S Connectivity - NVidia
    • Linux
      • Not supported but the Linux volume management “dmraid” can provide pseudo RAID connectivity
    • Solaris
      • Not supported currently
    • Windows 32-bit
      • Nvidia Media Shield
    • Windows 64-bit
      • Nvidia Media Shield
    • Windows Vista / Windows Server 2008
      • Nvidia Media Shield
  • O/S Connectivity – LSI Installation
    • Installation of the LSI CIM Browser:
    • 1. Go to the web site "http://www.lsilogic.com/downloads/downloads.do?"
    • 2. Select Miscellaneous and click the "GO" button
    • 3. Find the "CIM_linuxxxxxxxx.zip" file and download it
    • 4. Copy this file onto the V20z/V40z and put "CIM_linuxXXXXXXXXX.zip" in to "/tmp"
    • 5. Connect via ssh as root onto the V20x/V40z and "cd /tmp"
    • 6. Unzip the archive using unzip "CIM_linuxXXXXXXXXX.zip"
    • 7. Change directory to the install binary path "cd InstData/Linux/VM/"
    • 8. Ensure your DISPLAY variable is correct for the root login "echo $DISPLAY"
    • 9. Add execute permissions to the install binary "chmod 755 install.bin"
    • 10. Execute the install binary "./install.bin" and an installer window appears
                    • Cont ...
  • O/S Connectivity – LSI Installation
    • 11. Answer all questions in the window to complete the installation
    • 12. When finished, exit the window
    • 13. Check CIM provider is running with:
      • # ps -aef | grep -i LSIProvider (and check for output similar to that below)
      • root 2120 1 0 15:37 pts/0 00:00:04 /usr/local/bin/LSICim/jre/bin/
      • java -Djava.library.path=/usr/lib/ -classpath /usr/local/bin/LSICim/xerces.jar
      • :/usr/local/bin/LSICim/CIMOM.jar:/usr/local/bin/LSICim/HTTPClient.jar:/usr/loc
      • al/bin/LSICim/CIMLSIProvider.jar org.snia.wbemcmd.cimom.CIMServer
      • root 3722 1703 0 16:17 pts/0 00:00:00 grep -i cimlsiprovider
    • If the provider is not running start the provider daemon with the following command:
      • “ /etc/init.d/lsicim start”
    • 14. Start the CIM Browser “ /usr/local/bin/LSICim/CIMLSIBrowser”
    • 15. The CIM Browser appears
  • O/S Connectivity – MPT Installation
    • Installing the mpt-status utility:
    • 1. Make sure dev-utils are installed i.e. Make, kernel-source, glibc etc
    • 2. Download the mpt-status package for your particular Linux distribution:
    • SUSE http://www.novell.com/products/linuxpackages/suselinux/mpt-status.html
    • RedHat External http://www.drugphish.ch/~ratz/mpt-status/
    • Ubuntu http://packages.qa.debian.org/m/mpt-status.html
    • 3. If you downloaded an RPM then issue “rpm -U mpt-status*”
    • 4. If you downloaded source, un-archive the download and change directory to it
    • 5. Execute make ; make install (This will only work if the kernel-source etc are installed)
    • 6. Execute make un-install to remove the package if no longer required
    • 7. Execute “/usr/sbin/mpt-status” to view the current disk configuration
  • O/S Connectivity – MSM Installation
    • Installing the MSM-IR package in Windows:
    • 1. Locate the MSM-IR utility installation files before proceeding
    • The MSM-IR installation files are provided in the “InstallPack.exe” or the optional install pack “OptPack.zip” on the Tools and Drivers CD-ROM you received with your server You can also locate the MSM-IR installation files in the these packages on the download page for your product in the Tools and Drivers CD image in “windowsw2k3packages”
    • If you are unsure of the file locations refer to your product Windows Operating System Installation Guide for more information
    • 2. If you received the Tools and Drivers CD-ROM with your server insert it into the CD ROM drive connected to your server
    • a. If you do not have the Tools and Drivers CD-ROM, copy the “InstallPack.exe” file from a remote computer to a hard disk drive (HDD) in your server via the JavaRConsole remote media utility provided within iLOM
    • b. Start the “InstallPack.exe” application
  • O/S Connectivity – MSM Installation
    • Installing the MSM-IR package in Windows:
    • 3. Click on the “Optional Components” check box
    • 4. Click “Next” to accept the settings
    • 5. Review the important note and then click “Next”
    • The Welcome to the Sun Fire Installation Wizard displays
    • 6. Click “Next”
    • The End User License Agreement dialog box displays
    • 7. Select “I accept this agreement” and then click “Next”
    • 8. Click “Finish”
    • 9. Click “Yes” to restart your system to complete the installation
    • For full text of this installation and screen shots, refer to:
    • http://docs.sun.com/source/819-5039-12/Chap8F.html on the docs.sun.com web site
  • O/S Connectivity – Nvidia Installation
    • Installing the Nvidia Media Shield Application:
    • 1. http://www.nvidia.com/object/nforce_nf_pro_winxp_6.70.html for 32-bit Windows
    • http://www.nvidia.com/object/nforce_nf_pro_winxp64_6.69.html for 64-bit Windows
    • 2. Download the WHQL certified driver and select “Run” in the download dialog box *
    • 3. Follow the instructions to install chipset drivers if you have not already done so
    • 4. Select both Nvidia IDE and RAID driver if applicable and follow on screen instructions
    • 5. Complete install
    • * Available on the
    • supplement CD
  • O/S Connectivity - Monitoring
    • MyStorage output not available but similar to MSM-IR
    • Output of BIOS:
      • HBA ID LUN VENDOR PRODUCT REV SYNC WIDE CAPACITY
      • --------------------------------------------------------------------
      • 0 0 0 LSILOGIC 1030 IM IM 1000 16 73277 MB
      • 0 7 0 LSILogic LSI1030[ 402] 1032300 320.0 16
      • LSI Logic Corp. MPT boot ROM successfully installed!
    • Output of mpt-status:
      • [root@va64-v20zj-gmp03 bin]# /usr/sbin/mpt-status
      • ioc0 vol_id 0 type IM, 2 phy, 68 GB, state OPTIMAL, flags ENABLED
      • ioc0 phy 0 scsi_id 0 FUJITSU MAT3073N SUN72G 0602, 68 GB, state ONLINE, flags
      • ioc0 phy 1 scsi_id 1 SEAGATE ST373307LC 0007, 68 GB, state ONLINE, flags
  • O/S Connectivity - Monitoring
    • Output of LSI CIM Browser:
  • O/S Connectivity - Monitoring
    • Output of MSM-IR:
  • O/S Connectivity - Monitoring
    • Output of Nvidia Media Shield:
  • Size and Geometry
    • Size is everything!
    • Disk geometry is organized in a layout called CHS or Cylinders, Heads and Sectors
    • There are 2 types of firmware on a hard disk drive:
      • Disk vendor firmware & Sun firmware
      • Different firmware causes mismatches in size
      • Different vendors causes mismatches in size
    • LSI 1020 can now handle differences in size with IM before a partition layout is created (disk label)
    • LSI 106x unaffected as all disks are Sun firmware
  • Expanding And Limitations
    • Expanders are a little like network hubs for storage:
    • A 4 port SAS card can be expanded to connect to an 8 drive enclosure etc.
    • SAS expanders work with LSI or Intel/Adaptec cards
    • The X4240 has a SAS expander built into the chassis
    • You don’t need an extra card to address all 16 disks
    • Both the LSI and the Adaptec card can see all of the disks because of the expander but LSI limitation of 2 volumes of max RAID 1E limits expander use
    • Sun supports one SAS HBA for internal storage only
  • What is RAID
    • Redundant Array of Independent/Inexpensive Disks
    • Available with hardware controllers or software O/S
    • RAID 0 – Stripe / span
    • RAID 1 – Mirror / duplex
    • RAID 0+1 – A mirror of stripes
    • RAID 1+0 – A stripe of mirrors
    • RAID 1 E – Enhanced mirroring
    • RAID 5 – Distributed parity
  • What is RAID
    • RAID 0 (IS)
    • 2 disk minimum
    • Low cost
    • Odd or even disks
    • High performance by storing blocks across multiple spindles on different disks
    • Single disk failure results in full data loss
  • What is RAID
    • RAID 1 (IM)
    • 2 disk minimum
    • Even disks only
    • 50 % storage lost (high cost)
    • Improved read performance
    • Reduced write speed
    • Can recover from a single disk failures
  • What is RAID
    • RAID 0+1
    • 4 disk minimum
    • Even disks only
    • 50% storage lost (high cost)
    • Improved read performance
    • Improved write performance
    • Can recover from a single disk failure
  • What is RAID
    • RAID 1+0 (10)
    • 4 disk minimum
    • Even disks only
    • 50% storage lost (high cost)
    • Improved read performance
    • Improved write performance
    • Can recover from a single disk failure
    • Can recover from a dual disk failure if the failures occur on different mirrors
  • What is RAID
    • RAID 1E
    • 4+ disk for tolerance
    • Even & odd disks
    • Improved read and write performance
    • Same as RAID 10 when using even number of disks
    • Can recover from multiple disk failures if even number of disks are used in the array
    • Recovery from an odd number disk array is less than an even number
  • What is RAID
    • RAID 5
    • 3 disk minimum
    • High CPU usage
    • Slower read/write performance due to parity check
    • Traditionally, uses odd number of disks however partition implementations can use even disks also
    • Recovers from disk failures by using the missing parity bit rather than having a copy of the data on an alternate disk
  • Building RAID - LSI1020/1030
    • Power on platform
    • Press <CTRL> <C> during boot
    • Select controller (listed as 1030)
    • Select “RAID Properties”
    • Highlight 1 st disk and click on “+” to add to the array
    • Choose to preserve data or delete data on disk
    • Highlight 2 nd disk and click on “+” to add to the array
    • Press <ESC> to exit “RAID Properties”
    • Select “Save changes and exit this menu”
  • Building RAID - LSI1020/1030 Select Controller Select RAID Properties Add Primary Disk Add Members Exit Configuration Disk Now Resyncing
  • Building RAID - LSI1020/1030
    • LSI 1020/1030 RAID 1 Rules:
      • Resync time is asynchronous in OBP/BIOS mode
        • This is approximately 1 hour per gigabyte
      • Resync is synchronous when the driver is loaded
        • This is approximately 5 minutes per gigabyte
        • Resync time (Hours) = (Vol * 1024)/ 3 MB/sec)/ 3600
      • Replacement drives should either be empty or not contain a TOC (table of contents) – F4 Diagnostics
        • Introducing drives with an invalid partition layout will result in the controller refusing to create a RAID or resync existing RAIDs
      • The LSI 1020/1030 controller associates array member information with the physical slot as well as the disk
        • Mix sizes only with new RAID arrays
  • Building RAID - LSI106x
    • Power on platform
    • Press <CTRL> <C> to enter LSI setup when prompted
    • Select the primary on board controller (LSI1064)
    • Select RAID Properties
    • Select first drive (primary)
    • Select &quot;D&quot; to destroy existing drive data
    • Select second drive (secondary)
    • Select IM to create a RAID1 (IS to create RAID0)
    • Press <ESC> to go back a screen and save settings
  • Building RAID - Nvidia MCP's
    • Power on platform
    • Press <F2> during boot to enter BIOS
    • Select Advanced >IDE Configuration > Nvidia RAID
    • Enable RAID on each SATA channel and <F10>
  • Building RAID - Nvidia MCP's
    • Power off and on platform again
    • Press <F10> during boot to enter Nvidia RAID BIOS
    • Select RAID mode and then add disk members
    • Press <F7> to finish Y/N to clear existing disk data
    • Press <ESC> and then <CTRL> <X> to exit
  • Building RAID - Nvidia MCP's
    • Reboot the platform
    • Boot the Windows installation CD
    • Press <F6> to add additional storage controllers
    • Insert USB floppy disk or CD-ROM
    • Browse inserted drive
    • Select both Nvidia class drivers
    • Allow installation to continue
    • Install the Nvidia MediaShield package in the GUI once Windows has finished installation & rebooted
  • Recovery RAID - LSI Logic
    • RAID Recovery is in many ways totally automatic
      • Hot spare will automatically sync (v40z only)
      • Replacement drives will automatically sync if the new disk is ether empty or does not contain a TOC
        • Introducing different size disks or drives with an invalid partition layout will result in the controller refusing to re-sync. Pressing 'F4' in the LSI setup tool will display a diagnostics code which can be used to determine the problem. See next page for list;
      • The LSI 1020/1030 controller associates array member information with the physical slot as well as the disk. Therefore re-sync requires the replacement disk to be added to the same slot as the old damaged drive
  • Recovery RAID - LSI Logic
    • LSI RAID failure diagnostics codes:
      • 01. Problem with reading the disk serial number
      • 02. Disk does not support SMART
      • 04. Disk does not support wide data, sync mode or queue tags
      • 05. User disabled disconnects or queue tags for device
      • 07. Disk not big enough to mirror primary disk
      • 10. Disk does not have 512-byte sector sizes (check jumpers)
      • 11. Incorrect device type
      • 12. Hot spare selection not big enough to be used in the array
      • 13. Maximum disks already specified or array size exceeded
      • 03,06,08 & 09 are unused & should be regarded as unknown
  • Recovery RAID - Nvidia MCP's
    • When a Nvidia RAID array is detached/degraded:
      • Execute the Media-Shield raidtool in Windows
      • Browse the disks
      • If a disk has detached, right click on the second/lower degraded array and select delete to remove this disk
        • After a few seconds, Windows will prompt to say “Found New Hardware”
        • The disk will appear as a normal disk in Device Manager
        • Select the option “Rebuild Array” in the Media-Shield raidtool
      • If an array is degraded, select the option “Synchronize an Array”
        • This will force a complete rebuild of redundancy in RAID 1 or parity in RAID 5 arrays
  • Recovery RAID - Nvidia MCP's
    • When a Nvidia RAID disk has failed:
      • Physically replace the failed disk taking all precautions for the server/desktop environment you are working with
        • On selected systems, this could mean a power down and the opening of a platforms side or top cover
      • Power on the platform if applicable
      • After boot up, Windows will prompt to say “Found New Hardware”
      • Execute the Media-Shield raidtool in Windows
      • Browse the degraded array
      • Select the option “Rebuild array”
        • This will force a complete rebuild of redundancy in RAID 1 or parity in RAID 5 arrays
  • Solaris SATA Driver
    • First shipped in Solaris 10 Update 2 for Thumper
    • Partly ported to the generic hardware in Update 4
    • Delivered via package SUNWckr
    • /kernel/misc/sata 32-bit ELF driver
    • /kernel/misc/amd64/sata 64-bit ELF driver
    • Man page sata(7D)
    • Supports SATA and SATAII 1.5/3.0gb/sec
  • New Solaris “raidctl” Usage (May 2007)
    • raidctl -a {set | unset} -g disk {volume | controller}
    • raidctl -c [-f] [-r raid_level] disk1 disk2 [disk3...]
    • raidctl -d [-f] volume
    • raidctl -h
    • raidctl -l -g disk controller
    • raidctl -l volume
    • raidctl -l controller
    • raidctl -p &quot;param=value&quot; [-f] volume
    • raidctl -C &quot;disks&quot; [-r raid_level] [-z capacity] [-s stripe_size] [-f]
    • controller
    • raidctl -F filename [-f] controller
    • raidctl -S [volume | controller]
    • raidctl -S -g disk controller
  • What is iSCSI?
    • Enables the transport of Block I/O data over IP networks.
    • Operates on top of TCP through encapsulation of SCSI commands in a TCP/IP data stream
    • Transport of iSCSI mainly over Ethernet
  • What is iSCSI?
    • A transport for SCSI commands
    • An end to end protocol
    • Seamless implementation
      • On workstations and laptops
      • With current TCP/IP stacks
      • In a HBA
      • Uses existing routers/switches without changes
    • Has the concept of human readable SCSI device (node) naming
    • Transport includes security as a base concept
      • Authentication at the node level
      • Enabled for IPSec and other Security Techniques
  • Benefits of IP Storage
    • Leverages Ethernet-TCP/IP networks and enables storage to be accessed over LAN/WAN environments without altering storage applications.
    • Uses existing Ethernet/IP knowledge base and management tools
    • Provides consolidation of storage systems (Data backup, Server cluster, Replication, Business Continuity and Disaster Recovery)
    • Uses existing Network Infrastructure
    • Building WWSAN fits in the development of modern IP Storage technologies
    • Storage resources are now available to more applications
    • Manage IP-based storage networks with existing tools and IT expertise
  • How Does it Work?
    • The iSCSI protocol sits above the physical and data-link layers and interfaces to the operating system's standard SCSI Access Method command set
    • iSCSI enables SCSI-3 commands to be encapsulated in TCP/IP packets and delivered reliably over IP networks
    • The iSCSI protocol runs on the host initiator and the receiving target device
    • iSCSI also enables the access of block-level storage that resides on Fibre Channel SANs over an IP network via iSCSI-to-Fibre Channel gateways such as storage routers and switches
  • Limitations of iSCSI
    • IP packets are delivered without a strict order. SCSI packets must be delivered one after another without delay, and breach of the order may result in data losses
    • Processor power on the client's side which uses such card
      • Recommended using special network cards which support mechanisms of CPU unload before TCP stack processing
    • Latency issues -
      • Although there are a lot of means developed to reduce influence of parameters which cause delays in processing of IP packets, the iSCSI technology is positioned for middle-level systems
  • Four Basic iSCSI Components
    • iSCSI Address and Naming Conventions
    • iSCSI Session Management
    • iSCSI Error Handling
    • iSCSI Security
  • Address and Naming Component
    • It's more convenient to use a combination of an IP address and a TCP port which are provided by a Network Portal. For example, Sun.com.ustar.storage.itdepartment.161.
    • Such name has an easy-to-perceive form and can be processed DNS. An iSCSI name provides a correct identification of an iSCSI device irrespective of its physical location.
  • Session Management Component
    • The iSCSI session consists of a Login Phase and a Full Feature Phase which is completed with a special command
    • The “Login Phase” is used to adjust various parameters between two network entities and confirm an access right of an initiator
    • When login is confirmed, iSCSI turns to the FULL Feature Phase. If more than one TCP connection was established iSCSI requires that each command/response pair goes through one TCP connection
  • Error Handling Component
    • For iSCSI error handling & recovery to work correctly, both the initiator and the target must be able to buffer commands before confirmed. Each terminal must have a possibility to recover selectively a lost or damaged PDU within a transaction for recovery of data transfer
    • Hierarchy of iSCSI error handling and recovery after failures:
      • Lowest level - identification of an error & data recovery on the SCSI task level, for example, repeated transfer of a lost or damaged PDU
      • Next level - a TCP connection which transfers an iSCSI task can have errors. In this case we attempt to recover the connection
      • At last, the iSCSI session can be damaged. Termination and recovery of a session are usually not required if recovery is implemented correctly on other levels, but the opposite can happen. Such situation requires that all TCP connections be closed, all tasks, under-fulfilled SCSI commands be completed, and the session be restarted via the repeated login
  • Security Component
    • As the iSCSI can be used in networks where data can be accessed illegally, the specification allows fpr different security methods. Such encoding means as IPSec which use lower levels do not require additional matching because they are transparent for higher levels, and for the iSCSI as well
    • Various solutions can be used for authentication, for example, Kerberos or Private Keys Exchange, an iSNS server can be used as a repository of keys
  • Solaris iSCSI Software Requirements
    • The Solaris 10 1/06 or later release for Solaris iSCSI initiator software
    • The Solaris 10 8/07 or later release for Solaris iSCSI target software
    • The following software packages:
      • SUNWiscsir - Sun iSCSI Device Driver (root)
      • SUNWiscsiu - Sun iSCSI Management Utilities (usr)
      • SUNWiscsitgtr - Sun iSCSI Target Device Driver (root)
      • SUNWiscsitgtu - Sun iSCSI Target Management Utilities (usr)
    • S10 pkg available at: /net/iscsisupport.singapore/docs/pkgs/iscsi-initiator/S10u2
    • Patch available at: /net/iscsisupport.singapore/docs/patch/119090-23
    • Sun does not support S9 iSCSI
  • Troubleshooting iSCSI Commands
    • # iscsiadm list discovery
    • # iscsiadm list initiator-node
    • # iscsiadm list isns-server -v
    • # iscsiadm list static-config
    • # iscsiadm list target -v
    • # iscsiadm list discovery-address -v
    • # iscsiadm list isns-server -v
    • This iscsiadm(1M) command is only available on systems running Solaris 10 ( iSCSI is not supported on earlier versions)
    • Sun Explorer version 5.7 and later will gather essential iSCSI data. Refer to InfoDoc 82329 : Sun[TM] Explorer 5.7 Data Collector
  • Creating an iSCSI Device for Windows
    • Part i - From the NAS head
    • 0. Create or use an existing sfs2 file system
    • Part ii
    • 1. Configure the iSNS server
    • 2. Configure a iSCSI access group
    • 3. Create a iSCSI device
    • Note: This document assumes the Windows iSNS server and iSCSI initiator are installed on the Microsoft Windows platform
    • Information obtained from InfoDoc 85873 iSCSI basics created by Sushil Shirke
    • Part iii - From the Windows platform
    • 4. View and scan the iSCSI Initiator for the new device
    • 5. Log into the iSCSI device
    • Part iv
    • 6. Go to Disk Management
    • 7. Scan for new disks
    • 8. Create a Windows label
    • 9. Create a Windows partition
    • 10.Format Windows file system
    • 11.View in My Computer
  • Part I Create a NAS File System
        • Note: If a sfs2 file system is already created proceed to Part ii
    • 1. Telnet into the NAS head / # telnet 129.153.118.65
    • 2. Log in as admin followed by return / connect to (? for list) ? [menu] admin
    • 3. Type “menu” at the NAS prompt and the menu will appear / nas5x05 > menu
      • nas5x05 StorageTek 5320 NAS Menu
      • ------------------------------------------------------------------------------
      • Operations | Configuration | Access Control
      • 1. Activity Monitor | A. Host Name & Network | K. Admin Access
      • 2. Show Log | B. Timezone, Time, Date | L. Volume Access
      • 3. Lock Console | C. Drive Letters | M. Trusted Hosts
      • 4. Licenses | D. Disks & Volumes |
      • | E. Users | Extensions
      • | F. Hosts | U. Language Selection
      • | | V. EMAIL Configuration
      • | H. DNS & SYSLOGD | W. ADS Setup
      • | I. NIS & NIS+ | X. CIFS/SMB Configuration
      • 0. Shutdown | J. NS Lookup Order | Y. RDATE time update
  • Part I Create a NAS File System
    • 4. Type the letter “D” to Enter Disks & Volumes
      • D. Disks & Volumes
      • Drive Volume(s) Available
      • A. ide1d1 /cvol /dvol 0B
      • B. isp1d040 /iscsiemea 499.1GB
      • C. isp1d041 /logs /v2 555.6GB
      • The right side shows the available blank space
    • 5. Select a “isp” device above and create a sfs2 file system
  • Part I Create a NAS File System
    • 6. Option B - isp1d040 was selected for this example
      • nas5x05 StorageTek 5320 NAS Configure Disk
      • Disk isp1d040 Size MB 571179 SUN CSM200_R
      • # START SEC SIZE SEC TYPE C OWNER USE% FREE SIZE REQS ACTIVE
      • 1 240 122880000 sfs2 /iscsiemea 1% 57.574G/57.574G 6787+1
      • 2 122880240 1046896399 -- 1046896399 sectors (499.1GB) free
      • 3 1169776639 0 --
      • 4 1169776639 0 --
      • 5 1169776639 0 --
      • 6 1169776639 0 --
      • 7 1169776639 0 --
      • 8 1169776639 0 --
      • +------------------------------------------------------------------------+
      • | 1. Edit |
      • | |
      • | SPACE page display |
      • +------------------------------------------------------------------------+
  • Part I Create a NAS File System
    • 7. Type “1” Edit
    • 8. Use the error keys and go to slice number “2”
    • 9. Type “1” Create partition
    • 10.Type “1” sfs2
    • 11.Create a file system name. (May want use a name that helps with tracing)
    • 12.If compliance is installed a choice of “Advisory” or “Mandatory”
    • 13. Type in the desired capacity
    • 14. Press <ENTER>
    • 15. Type “7” Proceed with create
  • Part II Create an iSCSI Device
    • 1. Navigate the iSCSI menu by typing the corresponding letter to the left. Use the <SPC> to scroll through extensions in order help find the menu choice
    • 2. The iSCSI main menu appears:
        • A. Configure iSCSI LUN
        • B. Configure Access List
        • C. Configure iSNS Server
    • 3. Select “C” Configure iSNS Server and type “1” to edit the fields
    • 4. Enter the IP of the server iSNS Microsoft server, currently (129.153.118.69)
    • 5. Type &quot;7&quot; Save Changes and press <ESC> to return to the iSCSI menu
    • 6. Type option &quot;B&quot; Configure Access List.
    • 7. Type “7” to add to a list and the acces menu comes up
    • 8. Type &quot;1&quot; Edit Fields (In each follow fields below are an example)
        • Name nas5x05access
        • CHAP Initiator Name test
        • CHAP Initiator Secret ************
        • Initiator IQN Name
        • Initiator IQN Name
  • Part II Create an iSCSI Device
    • 9. Type “7” Save changes
    • 10. Push <ESC> twice to go back to iSCSI menu
    • 11. Type “A” Configure iSCSI LUN
    • 12. Type “1” Add a device
    • 13. Type “7” Add a LUN
    • 14. Type “1” Edit fields
    • 15. Enter an iSCSI device name. Alias is optional and sfs2 file system capacity of the iSCSI device must be no larger then the file system
      • Name nas5x05iscsidisk1 Alias Volume /iscsidisk1 Capacity 10g Thin No Access
      • ENTER to move through fields Save changes after last field +------------------------------------------------------------------------+ | Select or Add an Access for the iSCSI LUN. | | 1. Select/Add | +------------------------------------------------------------------------+ ESC for Menu
  • Part II Create an iSCSI Device
    • 16. Type “1” Select/Add
    • 17. Type “A” nas5x05access
      • Note: nas5x05access is the example iSCSI access group created Part ii
    • 18. Type “7” Save changes
    • 19. The following output will appear
      • Initialization in progress....
      • Elapsed time 0:07 6% done
      • A. iqn.1986-03.com.sun:01:00144f0f8322.44884716.nas5x05iscsidisk1
      • This will appear in the &quot;Configure iSCSI LUN&quot; menu once complete
    • 20. Once complete the iSCSI device should be available to the initiator on the Windows platform
  • Part III Windows Initiator (Example Using Windows Advanced Server)
    • 1. On a Windows platform, select the following:
      • Start > Programs > Microsoft iSCSI Initiator > Microsoft iSCSI Initiator
    • 2. When the iSCSI Initiator GUI displays, click on the “Targets” tab
    • 3. A new device should be in the list of targets
      • Below is the device in this example:
      • iqn.1986-03.com.sun:01:00144f0f83322.44884716.nas5x05iscsidisk1 Inactive
    • 4. Click the “Log On” button
    • 5. Check the tick box “Automatically restore this connection when the system boots”
    • 6. Check the tick box “Enable multi-path” if desired
    • 7. Click on the “Advanced” button
    • 8. Verify the source IP is 129.153.118.69 for this example. The IP if the iSNS server configured on the NAS head
  • Part III Windows Initiator (Example Using Windows Advanced Server)
    • 9. Click the tick box “CHAP login information”
    • 10. Back space the name automatically entered in the “User name” field
    • 11. Re-enter in the name &quot;test&quot; used in this example when creating the iSCSI access group. As per the CHAP Initiator Name test that was entered above in Part II of this example
    • 12. Enter in the password created in the access group
    • 13.Click on the “OK” button
    • 14.The device should now show a “Connected status”:
      • iqn.1986-03.com.sun:01:00144f0f83322.44884716.nas5x05iscsidisk1 Connected
    • 15. The device is now available to the Windows platform as locally attached storage device but communicating via IP. The new device will need to be installed as per a normal Windows storage module.
  • Part IV Create a Windows Partition
    • 1. Click on the &quot;Disk Management&quot; folder located “Computer Management”
    • 2. Once the Disk Management folder is clicked Windows will scan for devices and detect and new device is available and requires a write signature
    • 3. Click “Cancel”. The reason is the disk management GUI will load up and show all the local disks
    • 4. Right mouse click on the new disk
    • 5. Click the “OK” button
    • 6. Right mouse click on the disk and select “Create a Partition”
    • 7. Click the “Next” button
    • 8. Click “Primary Partition”
    • 9. Click the “Next” button for full capacity
    • 10. Click the “Next” button for the default assigned drive letter
    • 11. Verify the summary and then click the “Next” button
    • 12. Click “Finish” to start formatting the drive
  • Terms / Abbreviations
    • SAN - Storage Area Network
    • CDB - command descriptor block
    • PDU - Protocol Data Unit
    • QoS - Quality of Service
      • (usually describes a network through latency and band of a signal)
    • SNIA - Storage Networking Industry Association
    • DNS - Domain Name Server
    • PLOGI - Fibre Channel Port Login
    • iSCSI - Internet Small Computer Systems Interface
    • FCIP - Fibre Channel over TCP/IP
    • iFCP - Internet Fibre Channel Protocol
    • iSNS - Internet Storage Name Service
    • WWSAN - World Wide Storage Area Network
  • Current Issues – Hot Bugs
            • Solaris iSCSI Initiator - Hot Bugs (P1-3 - All releases)
      • 6436879 iscsi panic when target violates protocol after successful target/lun reset
      • 6488627 isns-client: DevAttrReg: pg object: non-key attribute precedes key attributes
      • 6549867 kernel/driver-i iSCSI initiator panic during iSNS discovery
      • 6555580 i system panic [cpu1]/thread=ffffff0007ec5c80: assertion failed: 0, file: src/iscsi_conn.c, line: 956
      • 6559145 i Upon reboot iscsi based filesystem fails to unmount with network connection issue
      • 6559860 iSystem crashed while running snv_63 on V445
      • 6568295 i iSCSI MPxIO failback does not happen with EMC CLARiiON
      • 6580820 i iscsi panic while logining to new target under isns discovery
      • 6586114 i iscsi initiator server panic while running ifconfig up/down test
      • 6598503 i isns client: DevDereg deregister the iscsi storage node only
      • 6601828 Solaris 10 (SPARC and x86) IPv6 iSCSI &quot;discovery&quot; level (initial setup) not finding target
      • 6602016 i panics while isns discovery is being enabled or disabled
      • 6606807 Panic while running iSCSI I/O
      • 6608820 i mount at boot fails :System goes to maintenance mode if storage is not accessible
  • Current Issues – RFE's
            • Solaris iSCSI Initiator - Hot RFEs (P1-3 - All releases)
      • 6292475 3 acc 27M kernel/driver-i persistent store should use NV_ENCODE_XDR
      • 6334890 3 dis 24M kernel/driver-i Need sysevents for property and visibility changes for all IMA object types
      • 6354411 3 dis 22M kernel/driver-i iSNS - add support for other iSNS discovery methods (DHCP/Heartbeat)
      • 6354413 3 dis 22M kernel/driver-i iSNS - add support for iSNS security
      • 6394307 2 dis 19M sajid.zia kernel/driver-i iSCSI software boot support
      • 6394308 2 dis 19M kernel/driver-i MC/S and EL1/2 support
      • 6397032 3 dis 18M kernel/driver-i bring discovered targets on in parallel
      • 6401562 3 acc 18M kernel/driver-i iSCSI security updates - addition of SASL support
      • 6425406 3 dis 16M kernel/driver-i RFE: MC/S and ERL1/2
      • 6457694 3 acc 13M kernel/driver-i adjust solaris initiator login parameters for better default performance
      • 6457702 3 dis 13M kernel/driver-i slow link iscsi throttle
      • 6497777 3 dis 43W kernel/driver-i iscsi initiator - make default connection retry duration configuration
      • 6558203 3 acc 19W kernel/driver-i iscsiadm should be able to deal with DNS names
  • Current Issues – Product Pages
      • Current issues are listed under each product page etc:
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesUltra20
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesUltra20M2
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesUltra40
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesUltra40M2
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesW1100Z
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesV20z
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX2100
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX2100M2
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX2200M2
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX4100
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX4500
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunFireX4600
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunBlade6000
        • http://systems-tsc/twiki/bin/view/Products/ProdIssuesSunBlade8000
  • Helpful Links
    • Monitoring MegaRAID under Windows:
      • Managing the Sun Fire[TM] V20z/ V40z server with an LSI MegaRAID card in Windows
      • Document ID: 82356 Aug 31, 2005 Info Docs
    • Monitoring MegaRAID under Linux:
      • Tools and characteristic of MegaRaid in Linux
      • Document ID: 85638 May 25, 2006 Info Docs
    • Sun Supported MegaRAID tool for Linux/Windows:
      • http://www.sun.com/download/products.xml?id=45b94409
      • (This is now included in Sun supplement CD's 1.3 and above)
    • MSM-IR TOI:
      • http://systems-tsc/twiki/pub/Products/SunFireX4100ToiRef/G4F-Storage-subsystem-TOI.pdf
  • Helpful Links and Aliases
    • Aliases for iSCSI:
      • iscsi-interest - Internal iSCSI Discussions
      • iscsi-iteam - iSCSI Initiator Development and Test Group
    • Initiator project page:
      • http://dhs.central/index.php/ISCSI
    • Target project page:
      • http://dhs.central/index.php/ISCSI_Target
    • TSC-X64 home page:
      • http://systems-tsc/twiki/bin/view/Teams/GlobalX64FocusTeam
    • This and Systems-TSC CEC presentations:
      • http://systems-tsc/twiki/bin/view/Teams/ESGcec07
    • Anthony McNamara & William Watson
    • [email_address]
    • [email_address]