More Related Content
Similar to VMware EMC Service Talk
Similar to VMware EMC Service Talk (20)
More from Yunchao (Kevin) Wang
More from Yunchao (Kevin) Wang (8)
VMware EMC Service Talk
- 1. Copyright © 2013 EMC Corporation. All Rights Reserved.
VMware EMC Service
Talk
May 2014
Kevin Wang
- 2. Copyright © 2013 EMC Corporation. All Rights Reserved.
Question 1: The detailed procedure for HBA
initiates login to SAN to complete the connection to
the SP (Answers cover both FC and FCoE)
- 3. Copyright © 2013 EMC Corporation. All Rights Reserved.
This lesson covers the following topics:
• FC protocol make-up
• FC components
Lesson 1: Fibre Channel Protocol Overview
Fibre Channel and FCoE Technology 3
- 4. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Layered Architecture: FC-0
ULP
Upper Layer Protocol
SCSI-3, IP, ESCON/FIPS, etc.
FC-4
FCP (SCSI-3 Fibre Channel Link Encapsulation),
Single-Byte Command Code Sets (FICON), etc.
FC-3 Common Services (Not Used)
FC-2
Exchange and Sequence Management,
Frame Structure ,Flow Control
FC-1
Encode/Decode, Link Control Protocols,
Ordered Sets
FC-0
Physical Interface, Optical and Electrical Interfaces, Cables,
Connectors, etc.
Fibre Channel and FCoE Technology 4
- 5. Copyright © 2013 EMC Corporation. All Rights Reserved.
Transceivers Cable Connectors Comments
SFP, SFP+ LC
• Most popular for Fibre Channel
today
• SFPs up to 8 Gbps
• SFP+ > 8Gbps
• versions for different distances
mSFP mLC
• mini-SFP and mini-LC
• Designed by Brocade for 64-port
blade
GBIC SC
• Technology found in older 1 Gbps
equipment
• Replaced by LC technology
Optical Transceivers and Cable Connectors
Fibre Channel and FCoE Technology 5
- 6. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Cables and Distances
Transceive
r Type
Speed
Multimode media maximum distance
62.5 um
(OM1)
50 um
(OM2)
50 um
(OM3)
50 um
(OM4)
SW
2 Gbps 150m 300m 500m 500m
4 Gbps 70m 150m 380m 400m
8 Gbps 21m 50m 150m 190m
10 Gbps 33m 82m 300m 550m
16 Gbps 15m 35m 100m 125m
Single-mode media maximum distance (9 um core optic fiber)
LWL
4 Gbps
10 km to 40 km depending on vendor
8 Gbps
10 Gbps
16 Gbps
Note: For greater distances, you may connect using DWDM / CWDM between switches .
Fibre Channel and FCoE Technology 6
- 7. Copyright © 2013 EMC Corporation. All Rights Reserved.
Basic Port Types
Node Port: Transmits and receives Fibre Channel data frames in a
Switched Fabric environment (N_Port)
Fabric Port: Port on a switch that N_Port connects to. (F_Port)
Expansion Port: Port on a switch that has another switch plugged into
it (two switches that are merged) (E_Port)
Node Loop Port: Transmits and receives Fibre Channel frames in an
Arbitrated Loop (private) environment (NL_Port)
Fabric Loop Port: Allows a Fabric to communicate with an Arbitrated
Loop (public) (FL_Port)
Generic Port: Port on a switch that detects the attached port type and
auto-configures to match that port type (G_Port)
Fibre Channel and FCoE Technology 7
- 8. Copyright © 2013 EMC Corporation. All Rights Reserved.
The Host Bus Adapter
Fabric
Target
Storage Array
Server
HBA
NIC
LAN
See EMC Support Matrix at https://elabnavigator.emc.com
Fibre Channel and FCoE Technology 8
- 9. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Layered Architecture: FC-1
ULP
FC-4
FC-3
FC-2
FC-1
Encode/Decode, Link Control Protocols,
Ordered Sets
FC-0
Data encoding/decoding
• Less than 10 Gbps (20% overhead for encoding)
8-bits 10-bits
• 10 Gbps and 16 Gbps (3% overhead for encoding)
64-bits 66-bits
• Special Characters
Used with data characters to create ordered sets
Ordered sets include: Idle, SOF, EOF, R_RDY
Fibre Channel and FCoE Technology 9
- 10. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Layered Architecture: FC-2
ULP
FC-4
FC-3
FC-2
Exchange and Sequence Management,
Frame Structure ,Flow Control
FC-1
FC-0
• Defines structure of Fibre Channel frame
Frames, Sequences, Exchanges
• Flow control mechanisms
• Error handling and recovery
• Provides class of services
Class 1, Class 2, Class 3, Class 4, Class 6, and Class F
Fibre Channel and FCoE Technology 10
- 11. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC-2
• Defines the structure of the Fibre channel frame
How information to large to fit into a single frame is transferred by
using a sequence of frames
How operations that consist of exchanges of sequences of frames
are managed
• Describes how the rate of frame transmission is controlled by the
flow control mechanisms
• Error handling and recovery
• Class of Service
Class 1, Class 2, Class 3, Class 4, Class 6, and Class F
Fibre Channel and FCoE Technology 11
- 12. Copyright © 2013 EMC Corporation. All Rights Reserved.
Exchange, Sequence, Frame
• Exchange is an operation manager
SCSI task, an IP connection
Has an Originator
Has a Responder
Has one or more Sequences
• Sequence is a ULP object to be transmitted in one direction that
may be one frame or longer
• Frame is the unit of information transfer
• Manages flow control (buffer-to-buffer and end-to-end)
Fibre Channel and FCoE Technology 12
- 13. Copyright © 2013 EMC Corporation. All Rights Reserved.
Conversations
Session
Exchange Exchange Exchange
Seq Sequence Sequence SequenceSeqSeq
Frame #1 Frame #1
Frame #2
Frame #3
Frame #1
Frame #2
Frame #3
Frame #4
Frame #5
Frame #6
Frame #1
Frame #2
Frame #3
Frame #4
Frame #1
Frame #2
Frame #1
Fibre Channel and FCoE Technology 13
- 14. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Frame
Data Field
End-of-Frame
CRC
Start-of-Frame
Header
Optional
Header
Fill
Bytes
Payload
(information being transported)
Optional
Header
R_CTL
CS_CTL
TYPE
SEQ_ID DF_CTL SEQ_CNT
Frame Control (F_CNT)
Source Address (S_ID)
Destination Address (D_ID)
OX_ID RX_ID
Parameter Field (PARM)
Fibre Channel and FCoE Technology 14
- 15. Copyright © 2013 EMC Corporation. All Rights Reserved.
Credit-Based Flow Control
Fibre Channel
Switch
Physical ISL
FC Frame R_RDY
Credit = 161516
Fibre Channel
Switch
Long Distance links may require additional BB_Credit
Fibre Channel and FCoE Technology 15
- 16. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Layered Architecture: FC-4
ULP
FC-4
FCP (SCSI-3 Fibre Channel Protocol) mapping,
Single-Byte Command Code Sets (used for FICON) mapping
FC-3 Common Services (Not Used)
FC-2
FC-1
FC-0
• Mapping Layer
Maps Upper Level Protocol to lower levels
SAN typically uses SCSI FCP mapping
Fibre Channel and FCoE Technology 16
- 17. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC-4 and Upper Level Protocol
Exchange, Sequence, and Frame
...Information Unit # 1 Information Unit # 3Information Unit # 2...FC-4
FC-2
Sequence # 1 Sequence # 3Sequence # 2... ...
Frame 1
...
Frame 1 Frame 2 Frame N Frame 1
Payload: SCSI CDB DATA Payload: Status
Exchange (Operation)
DATA DATA
SCSI Command SCSI ResponseSCSI DATA
ULP
SCSI-3:
One SCSI Read I/O Operation
Fibre Channel and FCoE Technology 17
- 18. Copyright © 2013 EMC Corporation. All Rights Reserved.
EMC Fibre Channel Implementations
• Point-to-Point – Link between two
ports using Arbitrated Loop
• Arbitrated Loop (FC-AL) – Shared
interconnect between 2 to 126 nodes
• Switched Fabric (FC-SW) – Dynamic
connectivity support for more than 15
million nodes
Fibre Channel and FCoE Technology 18
- 19. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Fabric
Target
Storage Array
Fabric
Server
HBA
Server
HBA
ISL
Fibre Channel and FCoE Technology 19
- 20. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Address (FCID)
• Required to route frames from source to target
• 24-bits (three-bytes) - specified in Hex
Assigned at Fabric Login (FLOGI)
• Divided into three parts:
Domain_ID Area_ID Port_ID
Domain (01-EF)
(1-239 in decimal)
00-FF 00-FF
Example: 6b 04 00
Fibre Channel and FCoE Technology 20
- 21. Copyright © 2013 EMC Corporation. All Rights Reserved.
Common Fabric Services: Well Known Addresses
Service Name
Well Known
Address
Description
Login Service
(F_Port Server)
FFFFFE
All nodes register in login service when
performing FLOGI
Name Service FFFFFC
Stores information about nodes attached
to fabric during the Fabric Login process
Fabric Controller FFFFFD
State change notification to nodes
registered in the fabric and inter-switch
communications
Fibre Channel and FCoE Technology 21
- 22. Copyright © 2013 EMC Corporation. All Rights Reserved.
World Wide Name
• A World Wide Name (WWN) is a 64-bit value used in Fibre
Channel networks to uniquely identify each element in the
network
• Usually assigned to a Host Bus Adapter (HBA), switch port, or
storage port by the vendor at the time of manufacture
• Often linked within a component, such as an HBA
Node WWN – A name for all ports within the single HBA
Port WWN – A name for each individual port
Fibre Channel and FCoE Technology 22
- 23. Copyright © 2013 EMC Corporation. All Rights Reserved.
WWPN Examples
EMC VNX World Wide Port Name Example
5 0 0 6 0 1 6 0 4 7 2 0 5 5 9 0
EMC Company ID (24-bits) Port VNX Seed (from SPE/DPE midplane)
EMC VMAX World Wide Port Name Example
5 0 0 0 0 9 7 2 0 8 1 3 4 9 a d
EMC Company ID for VMAX (24-bits) VMAX Serial Number - Decodes to country
where manufactured/Model / etc.
(HK192601234)
CPU/Dir/P
(12g port B)
Emulex World Wide Port Name Example
1 0 0 0 0 0 0 0 c 9 2 0 d c 4 0
Reserved (12-bits) Company OUI (24-bits) Company Specific (24-bits)
Fibre Channel and FCoE Technology 23
- 24. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 11 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 Port Login to Target port
8 Process Login to Target port
9 SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = ??????
6
Fibre Channel and FCoE Technology 24
- 25. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 21 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
FLOGI
ACC
Accept frame
includes FCID
for HBA 030500
FLOGI Frame
Destination Address =
FFFFFE
Source Address =
000000
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = ??????
6
Fibre Channel and FCoE Technology 25
- 26. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 31 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
5
1 2
3 4
HBA
PLOGI
ACC
PLOGI Frame
Destination Address =
FFFFFC
Source Address =
030500
6
Fibre Channel and FCoE Technology 26
- 27. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 41 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
Fibre Channel Services Frames
• Register Class of Service
• Register FC-4 Type
• Register Port type
• Register other (vendor dependent)
FC
Serv
ACC
FC
Serv
ACC
Fibre Channel and FCoE Technology 27
- 28. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 51 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
Fibre Channel Services Frames
• Query: Get PIDs
• Query: Get Port WWN for PID1
• Query: Get Port WWN for PIDn
FC
Serv
ACC
FC
Serv
ACC
Name Server Database:
• PID = 040600
• FC-4 = SCSI FCP
• WWPN = 50:06:04:8a:d5:2e:76:c4
• . . .
• PID = 030100 . . .
• PID = 040400 . . .
Fibre Channel and FCoE Technology 28
- 29. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 61 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
SCR
ACC
SCR Frame
Destination Address =
FFFFFD
Source Address =
030500
Fibre Channel and FCoE Technology 29
- 30. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 71 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
Fibre Channel and FCoE Technology 30
- 31. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 81 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
SCSI
Process
SCSI
Process
Fibre Channel and FCoE Technology 31
- 32. Copyright © 2013 EMC Corporation. All Rights Reserved.
FC Port Initialization Process - Step 91 - Link online
2 - Fabric Login - get FCID
3 - Port Login to fabric name server
4 - Registrations with name server
5 - Query Name server for FC-4 devices
6 - State Change Registration
Repeat for each Target:
7 - Port Login to Target port
8 - Process Login to Target port
9 - SCSI INQ
Fabric 1
5
1 2
3 4
HBA
WWN = 50:06:04:8a:d5:2e:76:c4
FCID = 040600
WWN = 10:00:00:00:c9:48:f1:c7
FCID = 030500
6
SCSI
Process
SCSI
Process
Fibre Channel and FCoE Technology 32
- 33. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Login at a Glance
Process FLOGI PLOGI PRLI
Requirements - Link initialization
- Cable
- HBA and driver
- Switch port
- FLOGI
- Zoning
- Persistent binding
- Driver setting
- PLOGI
- Device masking (Target)
- Device mapping (Initiator)
- Driver setting (Initiator)
Information passed - WWN
- S_ID
- Protocol
- Class
- Zoning
- WWN
- S_ID
- ULP
- Class
- BB Credit
- LUN
Session members - N_Port to F_Port - N_Port to N_Port - ULP (SCSI-3 to SCSI-3)
Verification UNIX
WINDOWS
- Syslog
- Switch Utilities
- Syslog
- Driver Utilities
- Syslog
- Host based volume management
- Event Viewer
- Switch Viewer
- Driver utilities - Driver utilities
- Host based volume management
- Device manager
Fibre Channel and FCoE Technology - 33
- 34. Copyright © 2013 EMC Corporation. All Rights Reserved.
Zone Set: Fabric 1
Zone A: Initiator A; Target w
Zone B: Initiator B; Target x
Zone C: Initiator C; Target y; Target z
. . .
Zoning Overview
Storage
(Targets)
Hosts
(Initiators)
I
T
Initiator (HBA) Target
Zone A
Fabric 1
Fibre Channel and FCoE Technology 34
- 35. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fabric 1
How to Define Zone Members
I
T
Target Port WWPN:
50:06:04:8a:d5:2e:76:c4
Zone A
Initiator Port WWPN:
10:00:00:00:c9:48:f1:c7
5 3
1 2
3 4
Zone A
10:00:00:00:c9:48:f1:c7;
50:06:04:8a:d5:2e:76:c4
Zone Members Defined by WWPN
Zone A
0305ef; (Domain_ID 03, port_05)
0403ef; (Domain_ID 04, port_03)
Zone Members Defined by Domain/Port
Fibre Channel and FCoE Technology 35
- 36. Copyright © 2013 EMC Corporation. All Rights Reserved.
This lesson covers the following topics:
• Architecture of an Ethernet solution
Lesson 2: Ethernet Overview
Fibre Channel and FCoE Technology 36
- 37. Copyright © 2013 EMC Corporation. All Rights Reserved.
Ethernet and the OSI Model
7 - Application
6 - Presentation
5 - Session
4 - Transport
3 - Network
2 - Data Link
1 - Physical
LLC - Logical Link Control
MAC - Media Access Control
Ethernet is defined at
Layer 2 of the OSI model
Fibre Channel and FCoE Technology 37
- 38. Copyright © 2013 EMC Corporation. All Rights Reserved.
Ethernet PDU
Destination
MAC
6 bytes
Source
MAC
6 bytes
802.1Q
tag
(optional)
4 bytes
Type
2 bytes
Data Payload
variable # bytes
FCS
4 bytes
Ethernet_II Frame Format
Header
Fibre Channel and FCoE Technology 38
- 39. Copyright © 2013 EMC Corporation. All Rights Reserved.
Ethernet MAC Address
I
G
G
L
Organizationally Unique
Identifier (OUI)
(Assigned by IEEE)
Vendor assigned
24 bits 24 bits
Examples:
Broadcast address: ff:ff:ff:ff:ff:ff
Physical address of port on PC: 00-26-5E-82-3E-6E
47 46 45 24 23 0
Fibre Channel and FCoE Technology 39
- 40. Copyright © 2013 EMC Corporation. All Rights Reserved.
MAC Address Used to Direct Frame Through
Network
MAC: 00-26-5E-82-3E-6E MAC: 00-26-5E-77-2F-58
…2F:58 …3E:6E 08-00 Payload FCS
Ethernet Frame
This is the source address This is the destination address
Fibre Channel and FCoE Technology 40
- 41. Copyright © 2013 EMC Corporation. All Rights Reserved.
Switching Frames Using MAC Addresses - Step 1
MAC Table
Port 1 -Empty
Port 2 - Empty
Port 3 - Empty
Port 4 - Empty
PC A PC C
PC B
PC D
Port 1
Port 2
Port 3
Port 4
Frame
Dest Src
PC A
Fibre Channel and FCoE Technology 41
- 42. Copyright © 2013 EMC Corporation. All Rights Reserved.
Switching Frames Using MAC Addresses - Step 2
MAC Table
Port 1 -PC A
Port 2 - Empty
Port 3 - Empty
Port 4 - Empty
PC C
PC B
PC D
Port 1
Port 2
Port 3
Port 4
No entry for PC
D, so flood the
frame out all
ports
PC A
Fibre Channel and FCoE Technology 42
- 43. Copyright © 2013 EMC Corporation. All Rights Reserved.
Switching Frames Using MAC Addresses - Step 3
MAC Table
Port 1 -PC A
Port 2 - Empty
Port 3 - Empty
Port 4 - Empty
PC A PC C
PC B
PC D
Port 1
Port 2
Port 3
Port 4
Frame
Dest Src
PC A PC D
PC D
Fibre Channel and FCoE Technology 43
- 44. Copyright © 2013 EMC Corporation. All Rights Reserved.
The Problem of Loops and Broadcast Storms
Switch A
Switch C
Switch B
Broadcast
Fibre Channel and FCoE Technology 44
- 45. Copyright © 2013 EMC Corporation. All Rights Reserved.
How STP Solves the Problem
• STP is a loop-prevention
protocol
• Uses spanning tree
algorithm to elect a
Root Bridge and block
certain ports.
Switch A
Switch C
Switch B
Block
Fibre Channel and FCoE Technology 45
- 46. Copyright © 2013 EMC Corporation. All Rights Reserved.
How Does STP Protocol Elect a Root Bridge?
Switch A
Switch C
Switch B
Priority = 32768
MAC = 0000.1111.2222
Priority = 32768
MAC = 0000.2222.3333
Priority = 32768
MAC = 0000.2222.2222
• Switch with lowest
Bridge ID (BID) wins
election
• If all switches have
equal priority, then
switch with lowest MAC
address wins
Since all switches are set to
default priority, this switch
will be elected Root Bridge
because it has the lowest
MAC address
Bridge ID
Fibre Channel and FCoE Technology 46
- 47. Copyright © 2013 EMC Corporation. All Rights Reserved.
Result of running STP Protocol
Switch A
Switch C
Switch B
Root
Bridge
Designated
Port (F)
Root Port (F)
Alternate
Port (D)
Nonroot
Bridge
Nonroot
Bridge
Root Port (F)
Designated
Port (F)
Designated
Port (F)
• Forwarding ports
Root ports
Designated ports
• Discarding port
Alternate
Fibre Channel and FCoE Technology 47
- 48. Copyright © 2013 EMC Corporation. All Rights Reserved.
Ethernet Hardware Overview
• Ethernet hardware supports the following speeds
10 Mbps
100 Mbps
1 Gbps
10 Gbps
40 Gbps
100 Gbps
Hardware Popular Implementations
NIC
(Network Interface
Controller)
• Stand-alone card
• Built into host motherboard
• Wireless
Cables
• Copper UTP (Unshielded Twisted Pair)
• Copper Twinax
• Optic
Connectors
• RJ45 (for copper UTP cables)
• SFP+ (for copper twinax cables)
• LC (for fiber optic cables)
Most common
Data Center Core (Backbone)
Fibre Channel and FCoE Technology 48
- 49. Copyright © 2013 EMC Corporation. All Rights Reserved.
Network Interface Controller - NIC
• Provides access to the network
• Different implementations include:
Stand-alone card (could have multiple ports)
Built into motherboard (most common now)
Wireless
• Ports can be “teamed” to act as a single logical interface for
failover
NIC LAN
Fibre Channel and FCoE Technology 49
- 50. Copyright © 2013 EMC Corporation. All Rights Reserved.
Example Ethernet Cabling
UTP
(Unshielded Twisted
Pair)
Twinax with SFP+
Fiber Optic Cable
with LC Connector
Media Copper Copper Optic
IEEE Standard 1000BaseT (802.3ab)
10GSFP+CU (Direct
Attach)
1000BaseLX (802.3z)
Distance/speed 100 meters @1 Gbps 5 meters @ 10 Gbps 10 km @ 1 Gbps
Cat 5
RJ45
Twinax / SFP+ 9/125 SM Fiber
LC Connector
Fibre Channel and FCoE Technology 50
- 51. Copyright © 2013 EMC Corporation. All Rights Reserved.
Issues With a Flat Switched Network
• Single broadcast
domain
• No Security between
users
Broadcast
Fibre Channel and FCoE Technology 51
- 52. Copyright © 2013 EMC Corporation. All Rights Reserved.
VLAN
200
VLAN
200
VLAN
100
VLAN
100
VLAN Overview
• Each VLAN is a separate
broadcast domain
• Each switch port is assigned to a
single VLAN
• Devices in different VLANs must
use a router to communicate
• VLAN Trunks allow the data
from multiple VLANs to pass on
a single link between switches
Fibre Channel and FCoE Technology 52
- 53. Copyright © 2013 EMC Corporation. All Rights Reserved.
How a VLAN Solves the Problems of the Flat
Network
• Smaller broadcast
domains
• Security between VLANs
VLAN
100
VLAN
200
VLAN
100
VLAN
200
VLAN
100
VLAN
200
Trunk
Broadcast Broadcast
Fibre Channel and FCoE Technology 53
- 54. Copyright © 2013 EMC Corporation. All Rights Reserved.
How it Works: VLAN Tagging
• 802.1Q - IEEE Standard
Each port on switch assigned to specific VLAN ID
Unassigned ports automatically belong to Default VLAN (VLAN ID = 0001)
• When port receives frame:
Compare VLAN ID to filter table.
If valid, and MAC on another switch - forward to trunk port
If destined to exit an access port on switch, remove VLAN ID
• Trunk carries both tagged and untagged traffic
Destination
MAC
6 bytes
Source
MAC
6 bytes
802.1Q
tag
(optional)
4 bytes
Type
2 bytes
Data Payload
variable # bytes
FCS
4 bytes
Ethernet_II Frame Format
VLAN Tag
Fibre Channel and FCoE Technology 54
- 55. Copyright © 2013 EMC Corporation. All Rights Reserved.
This lesson covers the following topics:
• FCoE Protocol and FCoE Fabric design basics
Lesson 3: FCoE Protocol Overview
Fibre Channel and FCoE Technology 55
- 56. Copyright © 2013 EMC Corporation. All Rights Reserved.
FCoE Protocol Stack
Volume managers, File systems, etc.
SCSI Command Descriptor Blocks, data, and
responses
FCoE frame encapsulation / de-capsulation,
FCoE entities
Frame switching, MAC address, transport
Connection to the Physical Layer
Fibre Channel Protocol frame construction
FC-4, FC-3, FC-2
Application
SCSI
FCP
FCoE
Enhanced
Ethernet
FCoE Protocol Stack
Fibre Channel and FCoE Technology 56
- 57. Copyright © 2013 EMC Corporation. All Rights Reserved.
FCoE Encapsulation
SOF
EOF
Ethernet
Header
FCoE
Header
Encapsulated FC Frame
(including FC-CRC)
FCS
FC
Header
Payload
Fill
Dest.
MAC
Address
Source
MAC
Address
Version Reserved
VLAN
TAG
CRC
Ether
Type
Fibre Channel Frame
Ethernet Frame
FCoE PDU
Fibre Channel and FCoE Technology 57
- 58. Copyright © 2013 EMC Corporation. All Rights Reserved.
FCoE Frames
• Frame Efficiency
iSCSI, FCIP, iFCP rely on IP over Ethernet which result in additional
overhead in frames
FCoE frames only use Ethernet protocol
Efficiency can be calculated by the formula
• Jumbo Frames
Ethernet frames with more than 1500 bytes of payload
Considered unreliable on regular Ethernet
CEE allows reliable use of Jumbo Frames
FCoE must use jumbo frames (2240 bytes)
Number of Data Bytes
Total Bytes Sent
Fibre Channel and FCoE Technology 58
- 59. Copyright © 2013 EMC Corporation. All Rights Reserved.
Converged Network Adapter (CNA) Overview
CNA
CNA = HBA + NIC + FCoE
Host
Fibre Channel and FCoE Technology 59
- 60. Copyright © 2013 EMC Corporation. All Rights Reserved.
• CNA is both a
Network Adapter
Fibre Channel adapter
• Host sends I/O to appropriate
adapter:
NIC – NFS, Web, Email, etc.
HBA – SCSI block data
• Host receives data through
lossless Ethernet MAC
Directs I/O to appropriate adapter
Based on Ethernet “Type”
Converged Network Adapter Architecture
2 x 10 Gbps Ethernet Ports
Ethernet NIC
Host Networking
Drivers
FCoE Encapsulation
Existing FC
Host Drivers
CNA Adapter
Lossless Ethernet MAC
Is it FCoE?NO YES
HBA Architecture
Fibre Channel and FCoE Technology 60
- 61. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fibre Channel Forwarder – FCF
• FCF function provided by FCoE switch or FCoE blade
Services FCoE Login requests
Provides services associated with a Fibre Channel switch
Stand-alone
FCoE Switch
Fibre Channel Switch
with FCoE blade
Fibre Channel and FCoE Technology 61
- 62. Copyright © 2013 EMC Corporation. All Rights Reserved.
• Lossless Ethernet:
FCoE requires reliable frame delivery
Can’t drop frames from buffer overflow
• Improves regular Ethernet:
Eliminates Ethernet loss, therefore more
reliable
Speed increased to 10 Gbps
• Additions to Ethernet:
Priority-based Flow Control (PFC)
Enhanced Transmission Selection (ETS)
DCB Capability Exchange (DCBX)
Notify neighbor node of DCB support
Data Center Bridging (DCB)
Fibre Channel and FCoE Technology 62
- 63. Copyright © 2013 EMC Corporation. All Rights Reserved.
Enhanced Ethernet Features
Feature Enhancement
Priority Flow Control
IEEE 802.1Qbb – class of service flow control by
enabling PAUSE functionality on IEEE 802.1p
lanes
Data Center Bridging Exchange
Auto-negotiation of Enhanced Ethernet
capabilities DCBX (switch to NIC)
Bandwidth Management
IEEE 802.1Qaz Enhanced Transmission Selection –
manage bandwidth and assign priorities to
groups of IEEE 802.1p lanes based on class of
traffic.
Fibre Channel and FCoE Technology 63
- 64. Copyright © 2013 EMC Corporation. All Rights Reserved.
Priority-based Flow Control
STOP
Transmit Queues Receive Buffers
0
1
2
3
4
5
6
7
Buffers are
almost full -
send “PAUSE”
Priority
Fibre Channel and FCoE Technology 64
- 65. Copyright © 2013 EMC Corporation. All Rights Reserved.
Enhanced Ethernet - DCBX
• DCBX runs between the switch and
CNA
• DCBX negotiates capabilities
• Switch distributes configuration
to all attached adapters
• Devices need to discover the
capabilities of its peers.
• DCBCX utilizes the link-layer
discovery protocol and handles
local operational configuration
for each feature
Priority Flow Control
Data Center Bridging Exchange
Bandwidth Management
Rules for Capabilities Result
Capability and configuration
match
Enabled
Adapter configured to accept
switch config.
Enabled
Adapter not configured to
accept switch config.
Disabled
Adapter does not
support/implement DCBX
Disabled
Fibre Channel and FCoE Technology 65
- 66. Copyright © 2013 EMC Corporation. All Rights Reserved.
Enhanced Ethernet – Bandwidth Management
• Enables consistent
management of QoS
• Provides consistent scheduling
of different traffic types
Priority Flow Control
Data Center Bridging Exchange
Bandwidth Management
Fibre Channel and FCoE Technology 66
- 67. Copyright © 2013 EMC Corporation. All Rights Reserved.
Addressing
MAC = A
FCID = 050100
MAC = B
Domain ID = 05 Domain ID = 01
FCID = 011C00
FC SAN
Converged
Network
FC Frame
D_ID = 011C00
S_ID = 050100
Dest = MAC B
Source = MAC A
D_ID = 011C00
S_ID = 050100
Ethernet Frame
Fibre Channel and FCoE Technology 67
- 68. Copyright © 2013 EMC Corporation. All Rights Reserved.
A Note About CNA MAC Addresses
FCoE
Controller
Lossless Ethernet MAC
FCoE_LEP
VN_Port
FC-3 / FC-4
Layers
Ethernet_Port
Ethernet NIC
Host Networking
Drivers
Existing FC
Host Drivers
Physical MAC
ENode MAC
FPMA MAC
CNA
Fibre Channel and FCoE Technology 68
- 69. Copyright © 2013 EMC Corporation. All Rights Reserved.
How are MAC Addresses Assigned to the VN_Port?
• Server-provided MAC Address (SPMA):
Uses single MAC Address for all FCoE traffic
FCoE has MAC address independent of other protocols
Support for different MAC addresses for each Virtual Port
This method isn’t used
• Fabric-provided MAC Address (FPMA):
Uses virtual port MAC addresses
Each virtual port MAC address is supplied by the fabric
Most common method
Have the following format:
Fibre Channel and FCoE Technology 69
- 70. Copyright © 2013 EMC Corporation. All Rights Reserved.
Server Provided MAC Addresses (SPMA)
• Adapter uses burned-in or configured MAC address
Consistent with the Ethernet model
• FCF needs a table to map between MAC addresses and FC_IDs
FCoE
Node Port
MAC
00.05.69.00.00.03
MAC Address
Fibre Channel and FCoE Technology 70
- 71. Copyright © 2013 EMC Corporation. All Rights Reserved.
Fabric Provided MAC Addresses (FPMA)
• MAC address assigned for each FC_ID:
Consistent with the Fibre Channel model
• Uses OUIs with U/L = 1 FC-MAP (Local addressing)
• No table needed for encapsulation
FC-MAP
(0E-FC-00)
FCID
07.08.09
24 bits 24 bits
48 bit MAC Address
FC-MAP
(0E-FC-00)
FCID
07.08.09
Fibre Channel and FCoE Technology 71
- 72. Copyright © 2013 EMC Corporation. All Rights Reserved.
FCoE Initialization Protocol (FIP) and FC Logins
CNA
CNA (ENode)
Lossless Ethernet
Switch (FIP Snooping
Bridge)
FCoE Switch
(FCF)
Storage
FCoE Switch
(FCF)
FC SAN
FC SAN
Storage
ENode (FCoE Controller)
Send FIP VLAN Request
Send FIP Solicitation
Send FIP FLOGI
FCF
Reply FIP VLAN Notification
Reply FIP Advertisement
Reply FIP FLOGI ACC
FIP Multicast
Fibre Channel and FCoE Technology 72
- 73. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP, the FCoE Initialization Protocol – How does it
work?
Fibre Channel and FCoE Technology 73
- 74. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP VLAN Request
Fibre Channel and FCoE Technology 74
- 75. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP Discovery Solicitation
Fibre Channel and FCoE Technology 75
- 76. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP Discovery Advertisement
Fibre Channel and FCoE Technology 76
- 77. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP Virtual Link Instantiation Request (FLOGI)
Fibre Channel and FCoE Technology 77
- 78. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP Virtual Link Instantiation Request (FLOGI ACC)
Fibre Channel and FCoE Technology 78
- 79. Copyright © 2013 EMC Corporation. All Rights Reserved.
FIP Snooping Bridge (FSB)
Fibre Channel and FCoE Technology 79
- 80. Copyright © 2013 EMC Corporation. All Rights Reserved.
Question 2: How SP coordinates the host to handle
hardware assist locks (Reservation, ATS)
- 81. Copyright © 2013 EMC Corporation. All Rights Reserved.
Verify VAAI Support Details
• The compare write(F1) operation is exclusively used by VAAI
enabled VMware host
- 82. Copyright © 2013 EMC Corporation. All Rights Reserved.
VAAI – Hardware Assisted Locking
• Scalable Lock Management – Atomic Test and Set (ATS)
• A number of VMFS operations cause the LUN to temporarily
become locked for exclusive write use by one of the ESXi nodes,
including:
– Moving a VM with vMotion
– Creating a new VM or deploying a VM from a template
– Powering a VM on or off
– Creating a template
– Creating or deleting a file, including snapshots
• VAAI feature, atomic_test_and_set (ATS) allows the ESXi host to
offload the management of the required locks to the storage and
avoids locking the entire VMFS file system
- 83. Copyright © 2013 EMC Corporation. All Rights Reserved.
Atomic Test & Set
• Original file locking technique
1. Acquire SCSI reservation
2. Acquire file lock
3. Release SCSI reservation
4. Do work on VMFS file/metadata
5. Release file lock
• New file locking technique
1. Acquire ATS lock
2. Acquire file lock
3. Release ATS lock
4. Do work on VMFS file/metadata
5. Release file lock
Note - The main difference with using the ATS lock is that it
does not affect the other ESXi hosts sharing the datastore
- 84. Copyright © 2013 EMC Corporation. All Rights Reserved.
VMFS Scalability with Atomic Test and
Set
Makes VMFS more scalable overall, by offloading block locking
mechanism
Using Atomic Test and Set (ATS) capability provides an
alternate option to use of SCSI reservations to protect the
VMFS metadata from being written to by two separate ESXi
hosts at one time.
Normal VMware
Locking (No ATS)
Enhanced VMware
Locking (With ATS)
- 85. Copyright © 2013 EMC Corporation. All Rights Reserved.
How to resolve SCSI Reserves on
VMware ESXi servers
• Refer to the following VMware KB articles:
– SCSI Reservation Issue with Fibre Channel HBAs (4365932)
– Troubleshooting SCSI Reservation failures on Virtual Infrastructure 3.x and
vSphere 4.x (1005009)
– VMware KB Article 1002293
• Warning! DO NOT release the reservation with INLINES when
customer is using vSphere.
• It is also NOT recommended to try to release via Solutions
Enabler
• The CORRECT way to release a SCSI Reserve in ESXi is to ask the
customer to use OS tools as described in the VMware
documentation and KB articles that are listed.
- 86. Copyright © 2013 EMC Corporation. All Rights Reserved.
Question 3: The terms or tricks to match WWPN
with SP port, which would help us to understand
mpath failover sequence
- 87. Copyright © 2013 EMC Corporation. All Rights Reserved.
During this lesson the following topics are covered:
• PowerPath configuration and PowerPath’s function in the I/O
path
• How PowerPath supports EMC and non-EMC arrays
• Key functionality included in current releases of PowerPath
• How PowerPath is licensed
• Hardware, software, clusters, and storage arrays that PowerPath
supports
Lesson 1: PowerPath Architecture
87
- 88. Copyright © 2013 EMC Corporation. All Rights Reserved.
EMC PowerPath - A Family of Products
• PowerPath traditionally viewed only as “Path Management Software”
Performed Multipathing, load balancing, failover, etc.
• Functionality Expanded
PowerPath Family now has 4 distinct products
PowerPath can now be used for solutions previously handled by separate
software
Common Interface allows for simplified management
• Optimized for heterogeneous host / storage environments
88
PowerPath Multipathing Path
Management /
OptimizationPowerPath/VE
PowerPath Migration
Enabler Data Protection
PowerPath Encryption
- 89. Copyright © 2013 EMC Corporation. All Rights Reserved.
• PowerPath sits
between the
applications and the
HBA driver
• Applications direct
I/O to PowerPath
• PowerPath directs
I/O to an optimal
path based on
current workload
and path availability
PowerPath Configuration
89
OpenSystemsHost
Logical volume manager
File system
PowerPath Driver
Applications
Management
utilitiesDBMS
Storage Array
Host bus
adapter
Host bus
adapter
Host bus
adapter
Host bus
adapter
Storage Area Network
- 90. Copyright © 2013 EMC Corporation. All Rights Reserved.
• PowerPath keeps a
table of paths
• Presents a single
device to the
application
• Combines
equivalent paths
into a single “Path
Set”
Volume Path Set
90
PowerPath
HBAHBA
SERVERSTORAGE
Front-end
Ports
Storage Area
Network
Volume Path Set
Application
HBAHBA
POWERPATH
“Power”
Device
- 91. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath Array Support Concepts
91
SERVERSTORAGE
SERVERSTORAGE
Active-Active Arrays Active-Passive Arrays
Owned by SP-A
HBA HBA HBAHBA
SAN
SP-A
Active
Active
SP-B
Passive
Passive
PowerPath
0 1 0 1
Host Application(s)
Mapped to all
Front-end ports
Active
Active
Active
Active
SAN
Front-end Ports
HBA HBA HBAHBA
PowerPath
Host Application(s)
- 92. Copyright © 2013 EMC Corporation. All Rights Reserved.
• Dead
indicates the path is
not usable.
PowerPath driver
will not direct user
I/O to this logical
I/O path.
• Alive
indicates the path is
usable
The PowerPath
driver can direct I/O
to this logical I/O
path.
Device (LUN) Path States
92
SERVERSTORAGE
Front-end
Ports
Storage Area
Network
Host Application(s)
HBA HBA HBAHBA Host Bus
Adapter
PowerPath
A
B
- 93. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath Licensing
• PowerPath
Unlocks the full load balancing and path failover capabilities of PowerPath and
supports all EMC and certain non-EMC arrays
Configures optimized default load balancing policies for EMC VMAX, VNX and
supported third-party arrays
• PowerPath/VE
Full feature set specifically for virtual server environments based on VMware
vSphere and Windows Hyper-V
• PowerPath SE
Backend (between switch and array) failover only
Single HBA supported only
No load balancing available
Path Failover across two backend paths
93
PowerPath SE
- 94. Copyright © 2013 EMC Corporation. All Rights Reserved.
During this lesson the following topics are covered:
• Benefits of PowerPath/VE
• Features and functionality of PowerPath/VE
• PowerPath/VE configuration in an ESXi environment
Lesson 2: Multipathing in ESXi
94
- 95. Copyright © 2013 EMC Corporation. All Rights Reserved.
What is PowerPath/VE?
• PowerPath advanced multipathing capabilities for virtual environments hosted on
VMware vSphere
• Features include:
Dynamic load balancing
Device prioritization
Automated optimization of server, storage, and path utilization
Dynamic path failover and path recovery
Monitoring performance and path status, I/O statistics and reports
Automatic path testing
Automatic path restore
95
x86 Architecture
PowerPath
- 96. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath/VE Key Benefits
• Automates the virtual server-storage connection:
Assures tunable, predictable performance
Optimizes server, storage, and data-path utilization
Maximizes data availability
• Uses standard management features across physical and virtual
environments:
PowerPath/VE experience similar to rest of environment (physical servers)
Common user experience across all classes of supported platforms
• Saves Money:
Simple to implement application service levels for performance and
availability
Eliminates need to monitor and rebalance the dynamic environment
Reduces number of multi-pathing software variants to manage
96
- 97. Copyright © 2013 EMC Corporation. All Rights Reserved.
• PowerPath manages complexity:
Constantly adjusts I/O path usage
and changes in I/O loads from VMs
Simplifies provisioning by pooling
all connections
Optimizes overall I/O performance
in VMware ESX environments
PowerPath/VE Load Balancing
97
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
OS
APP
STORAGE
SAN
PowerPath PowerPath PowerPath PowerPath
- 98. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath/VE supports:
• Changes to Permanent
Device Loss (PDL) handling
• VMFS 5 Filesystem
vSphere 5.0 Kernel Support
98
EMC and Non-EMC Storage
Host
Bus
Adapte
r
Host
Bus
Adapte
r
Host
Bus
Adapte
r
Host
Bus
Adapte
r
vSphere Kernel
PowerPath/VE
Guest OS
Applications
DBMS Management
Utilities
File System
Logical Volume Manager
PowerPath/VE
runs in the kernel
- 99. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath/VE – Two Major Components
99
PowerPath Remote
Management Server
ESX Hosts
PowerPath remote tools
(rpowermt) installed
Used to remotely manage
PowerPath/VE on any number of
ESX hosts
PowerPath software is installed
- 100. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath Remote Management Server
• Supported platforms:
Can be a physical or virtual host
Check ESM for version compatibility
• Software:
rpowermt
Sends PowerPath management commands to a remote ESX host
Is the only way to manage PowerPath/VE (no local commands)
VMware vSphere CLI
Installs PowerPath remotely onto an ESX host
License file
Uses rpowermt to register license with an ESX host
• Can manage any number of ESX hosts.
100
PowerPath Remote
Management Server
- 101. Copyright © 2013 EMC Corporation. All Rights Reserved.
PowerPath Remote Tools
• Are installed on the PowerPath Remote Management Server
• Provide remote versions of existing PowerPath commands:
Example:
Local command used by previous versions of PowerPath:
powermt display dev=all
Remote command, supported by PowerPath/VE:
rpowermt host=<hostname> display dev=all
• Note: Remote Tools version and PowerPath version must be
compatible:
Possible to update either Remote Tools or PowerPath, without updating the
other
101
- 102. Copyright © 2013 EMC Corporation. All Rights Reserved.
Installation and Configuration Steps
1. Upload Files to ESXi Server
2. Install PowerPath software on the ESXi server
3. Install PowerPath Remote Tools on PowerPath Remote
Management Server (Optional)
4. Configure and register license file
102
- 103. Copyright © 2013 EMC Corporation. All Rights Reserved.
Upload Files to ESXi Server
103
Create
New Folder
Upload File
or Folder
- 104. Copyright © 2013 EMC Corporation. All Rights Reserved.
Install PowerPath on ESXi
104
# cd /vmfs/volumes/Storage1/PPVE/
# ls
PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle.zip
# unzip PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle.zip
. . .
# esxcli software vib install --depot=/vmfs/volumes/Storage1/PPVE/
PowerPath_VE_5_7_for_VMWARE_vSphere-Install_SW_Bundle/EMCPower.
VMWARE.5.7.b173.zip
Installation Result
Message: The update completed successfully, but the system needs
to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: EMC_bootbank_powerpath.cim.esx_5.7.0.00.00-b173,
EMC_bootbank_powerpath.lib.esx_5.7.0.00.00-b173,
EMC_bootbank_powerpath.plugin.esx_5.7.0.00.00-b173
VIBs Removed:
VIBs Skipped:
- 105. Copyright © 2013 EMC Corporation. All Rights Reserved.
Reboot ESXi Server
105
- 106. Copyright © 2013 EMC Corporation. All Rights Reserved.
Verify PowerPath /VE Installation
106
# cd /opt/emc/powerpath/bin
/opt/emc/powerpath/bin # ls
powermt
/opt/emc/powerpath/bin # ./powermt version
EMC powermt for PowerPath (c) client Version 5.7 (build 173)
EMC PowerPath (c) Version 5.7 (build 173)
Warning: License not installed.
/opt/emc/powerpath/bin #