HP StorageWorks
MPxIO for Sun Solaris
application notes
Part number: AA-RW7CE-TE
Third edition: December 2006
About this document
This section describes the following topics:
•
•
•
Application notes information
These application notes describe the following:
•
•
•
•
•
•
•
Intended audience
This document is intended for customers who are using Sun Solaris MPxIO with HP storage systems.
Other documentation
A complete library of related documentation is available at:
For Solaris 8 & 9, the following documentation is available at http://www.sun.com/products-n-solutions/
•
•
•
Sun StorEdge SAN Foundation Software 4.4 Installation Guide
Sun StorEdge SAN Foundation Software 4.4 Configuration Guide
Sun StorEdge SAN Foundation Software 4.4.x Release Notes, including Sun StorEdge Traffic
Manager for the Solaris Operating System
•
•
Sun StorEdge Traffic Manager Installation and Configuration Guide For the Solaris Operating System
Sun StorEdge SAN Foundation Software 4.4
For Solaris 10, the Solaris Fibre Channel and Storage Multipathing Administration Guide is available at
MPxIO overview
MPxIO provides basic failover and load-balancing capability to HP StorageWorks EVA3000/5000 V4,
EVA4000/6000/8000, and the XP line of storage systems. Different storage devices can connect to the
same system. Failover to other storage devices is allowed.
New features
This release provides support for Sun Solaris MPxIO multipathing on EVA3000/5000 4.004 and
EVA4000/6000/8000 disk arrays.
3
MPxIO for Sun Solaris
NOTE:
MPxIO is not supported on versions of the EVA3000/5000 prior to 4.004.
Supported hardware and software
Table 1 describes supported hardware and software.
Table 1 Supported hardware and software
Component
Version
Solaris 8, 9, 10 (SPARC)1
Operating system
Sun native adapters
SG-XPCI1FC-QF2 (X6767A)
SG-XPCI2FC-QF2 (X6768A)
SG-XPCI2FC-QF2–Z
SG-XPCI1FC-QF4
SG-XPCI2FC-QF4
SG-XPCI1FC-EM4-Z
SG-XPCI2FC-EM4-Z
SG-XPCIE2FC-QB4-Z2
SG-XPCIE1FC-QF42
SG-XPCIE2FC-QF42
SG-XPCI1FC-QL2
SG-XPCI1FC-EM2
SG-XPCI2FC-EM2
SG-XPCIE1FC-EM42
SG-XPCIE2FC-EM42
QLA2310F
QLA2340
QLA2342
QLA2460
QLA2462
QLE24602
QLE24622
QLogic adapters
LP10000
LP10000DC
LP11000
LP11002
LPe110002
LPe110022
Emulex adapters
Storage Systems
HP StorageWorks EVA3000/5000 4.004
HP StorageWorks EVA4000/6000/8000
HP StorageWorks XP128/1024
HP StorageWorks XP10000/12000
1
For Solaris 10 x86 only Sun native adapters or specific FCAs for HP Proliant Blade servers are supported. Please check
the array release notes or with your HP representative for the supported FCAs.
2
PCI-Express FCAs are not supported on Solaris 8 and are supported on Solaris 9 only with some server models.
4
Supported multipathing combinations
The combination of MPxIO and Secure Path on the same server can only be supported if FCA2257P is
used for Secure Path and Sun native or Emulex adapters are used for MPxIO.
To view more multipathing combinations supported by MPxIO, go to http://www.hp.com/go/
(1-5) and check the table Sun Solaris multipathing coexistence support.
Preparing the host system
To set up the host system:
1. For Solaris 8 & 9 and Fibre Channel Adapters (FCAs) from Sun or QLogic:
If the OS has been installed before adding QLogic based FCAs, you may need to install SUNWqlc
2. Install the necessary software components for the different Solaris versions:
a. For Solaris 8 and 9, download the Sun SAN Foundation Software (SFS) 4.4.8 or later,
SAN_4.4.x_install_it.tar.Z which includes all the patches. Install it with the included
selecting SAN 4.4 release Software/Firmware Upgrades & Documentation. To verify which
b. For Solaris 10 (SPARC), the required patches depend on the FCA.
•
•
•
With QLogic based FCAs (from Sun or QLogic) SUNWqlc with patch 119130–26 or
later is required
With Emulex based FCAs (from Sun or Emulex), SUNWemlxs and SUNWemlxu with
patches 120222–11 and 119130-26 or later
update to the current Sun SAN patch level
c. For Solaris 10 (x86), the required patches depend on the FCA. Choose from one of the
following
•
•
For QLogic FCAs (from Sun or QLogic) SUNWqlc with patch 119131-24 or later
For Emulex FCAs (from Sun or Emulex) SUNWemlxs and SUNWemlxu with patches
119131 -24 and 120223-11 or later are requ ired
•
To update to the current Sun Solaris SAN patch level, enter/usr/bin/updatemanager
3. Reboot the Host after installing the (software and) patches.
4. Upgrade each FCA to the latest FCA Fcode/OpenBoot if required (typically requires single user
mode and no I/O activity):
•
•
fcode update and follow the instructions in the patch description in case a patch is available
OpenBoot version for the given FCA. Download the fcode and FCA utilities from that web
site and use /opt/EMLXemlxu/bin/emlxadm to check and update the fcode
•
latest qlc Fcode. Download the fcode and SANSurfer CLI from that web site and use
/opt/QLogic_Corporation/SANsurferCLI/scli to check and update the fcode
5. Reboot the Host after updating the Fcode on the FCAs.
5
MPxIO for Sun Solaris
Configure and verify the host
Verifying configuration settings
Verify that all paths to the storage system are present with cfgadm -al command. To display the LUNs
configured to each path, execute cfgadm -al -o show_FCP_dev as shown in the example below.
The following examples are with 4 paths (ports) visible from the array on the host and with 2 LUNs
configured on the array for the host. You should normally see 4 paths with EVA3000/ EVA5000/
EVA4000/ EVA6000, however you might also see up to 8 paths with EVA8000.
1. Enter the cfgadm command to verify the paths to the LUNs (note that you need to have
configured LUNs on the array for the host in order to see the “disk” lines):
Example: # cfgadm -al -o show_FCP_dev
Ap_Id
c2
Type
Receptacle
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
Occupant
Condition
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
fc-fabric
array-ctrl
disk
disk
array-ctrl
disk
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
unconfigured
c2::50001fe1002709f8,0
c2::50001fe1002709f8,1
c2::50001fe1002709f8,2
c2::50001fe1002709fc,0
c2::50001fe1002709fc,1
c2::50001fe1002709fc,2
c3
c3::50001fe1002709f9,0
c3::50001fe1002709f9,1
c3::50001fe1002709f9,2
c3::50001fe1002709fd,0
c3::50001fe1002709fd,1
c3::50001fe1002709fd,2
disk
c-fabric
array-ctrl
disk
disk
array-ctrl
disk
disk
With Solaris 8 or 9, the Occupant status will be displayed as above in a SAN environment. With
Solaris 10 (SPARC and x86), you will see a configured status.
2. For Solaris 8 and 9, use cfgadm -c configure c# command to configure the host to
access the storage system as shown in the example below. The value of # is gathered from the
output of the cfgadm -al previously executed.
Example:
# cfgadm -c configure c2
# cfgadm -c configure c3
3. Execute the cfgadm command to view configured devices. After the configuration, the following
command will show the array controllers as configured:
# cfgadm -al
Ap_Id
c0
c0::dsk/c0t0d0
c0::dsk/c0t6d0
c1
Type
scsi-bus
disk
Receptacle
connected
connected
connected
connected
connected
connected
connected
connected
connected
connected
Occupant
Condition
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
unknown
configured
configured
configured
unconfigured
configured
configured
configured
configured
configured
configured
CD-ROM
scsi-bus
fc-fabric
array-ctrl
array-ctrl
fc-fabric
array-ctrl
array-ctrl
c2
c2::50001fe1002709f8
c2::50001fe1002709fc
c3
c3::50001fe1002709f9
c3::50001fe1002709fd
6
NOTE:
Using the option –o show_FCP_dev could result in the output of the cfgadm command to
display the Type array-ctrl as unconfigured. This is not a problem. (See example below).
# cfgadm -al -o show_FCP_dev
Ap_Id
c2
Type
Receptacle
connected
connected
connected
connected
Occupant
Condition
unknown
unknown
unknown
unknown
fc-fabric
array-ctrl
disk
configured
unconfigured
configured
configured
c2::50001fe1002709f8,0
c2::50001fe1002709f8,1
c2::50001fe1002709f8,2
...
disk
4. Verify that the LUNs/disks are visible with the format command through all the EVA paths.
a. The following output is for all Solaris releases on SPARC:
# echo | format
Searching for disks...done
c2t50001FE1002709F8d1: configured with capacity of 1008.00MB
c2t50001FE1002709F8d2: configured with capacity of 1008.00MB
c2t50001FE1002709FCd1: configured with capacity of 1008.00MB
c2t50001FE1002709FCd2: configured with capacity of 1008.00MB
c3t50001FE1002709F9d1: configured with capacity of 1008.00MB
c3t50001FE1002709F9d2: configured with capacity of 1008.00MB
c3t50001FE1002709FDd1: configured with capacity of 1008.00MB
c3t50001FE1002709FDd2: configured with capacity of 1008.00MB
AVAILABLE DISK SELECTIONS:
1. c2t50001FE1002709F8d1 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709f8,1
2. c2t50001FE1002709F8d2 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709f8,2
3. c2t50001FE1002709FCd1 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709fc,1
4. c2t50001FE1002709FCd2 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709fc,2
5. c3t50001FE1002709F9d1 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709f9,1
6. c3t50001FE1002709F9d2 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709f9,2
7. c3t50001FE1002709FDd1 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709fd,1
8. c3t50001FE1002709FDd2 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709fd,2
Specify disk (enter its number)
b. Solaris 10 (x86) is shipped with MPxIO enabled by default. When you enter the format
command, a single device is displayed for each LUN regardless of the number of paths
connected. The following output is for all Solaris 10 (x86):
# echo | format
1. c4t600508B40010293400006000041B0000d0 <HP-HSV210-6000 cyl 1916 alt 2 hd 128 sec 128>
/scsi_vhci/disk@g600508b40010293400006000041b0000
2. c4t600508B40010293400006000040C0000d0 <HP-HSV210-6000 cyl 1916 alt 2
/scsi_vhci/disk@g600508b40010293400006000040c0000
3. c4t600508B40010293400006000040F0000d0 <HP-HSV210-6000 cyl 1916 alt 2
/scsi_vhci/disk@g600508b40010293400006000040f0000
4. c4t600508B4001029340000600004250000d0 <HP-HSV210-6000 cyl 1916 alt 2
/scsi_vhci/disk@g600508b4001029340000600004250000
5. c4t600508B4001029340000600004220000d0 <HP-HSV210-6000 cyl 1916 alt 2
/scsi_vhci/disk@g600508b400102934000060000422000
hd 128 sec 128>
hd 128 sec 128>
hd 128 sec 128>
hd 128 sec 128>
7
MPxIO for Sun Solaris
5. For all Solaris releases on SPARC, check the multi-pathing information for all LUNs (first example)
and either for a given disk/LUN or for the array WWN or a port WWN. The multipathing
information is already available although MPxIO has not yet been enabled:
# luxadm probe
No Network Array enclosures found in /dev/es
Found Fibre Channel device(s):
Node WWN:50001fe1002709f0 Device Type: Disk device
Logical Path:/dev/rdsk/c2t50001FE1002709F8d1s2
Logical Path:/dev/rdsk/c2t50001FE1002709FCd1s2
Logical Path:/dev/rdsk/c3t50001FE1002709F9d1s2
Logical Path:/dev/rdsk/c3t50001FE1002709FDd1s2
Node WWN:50001fe1002709f0 Device Type: Disk device
Logical Path:/dev/rdsk/c2t50001FE1002709F8d2s2
Logical Path:/dev/rdsk/c2t50001FE1002709FCd2s2
Logical Path:/dev/rdsk/c3t50001FE1002709F9d2s2
Logical Path:/dev/rdsk/c3t50001FE1002709FDd2s2
# luxadm display /dev/rdsk/c2t50001FE1002709F8d1s2
DEVICE PROPERTIES for disk: /dev/rdsk/c2t50001FE1002709F8d1s2
Vendor: HP
Product ID: HSV210
Revision: 5100
Serial Num: Unavailable
Unformatted capacity: 1024.000 MBytes
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c2t50001FE1002709F8d1s2
/devices/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709f8,1:c,raw
LUN path port WWN:
Host controller port WWN:
Path status:
50001fe1002709f8
210000e08b1759bd
O.K.
/dev/rdsk/c2t50001FE1002709FCd1s2
/devices/pci@1f,4000/QLGC,qla@4/fp@0,0/ssd@w50001fe1002709fc,1:c,raw
LUN path port WWN:
Host controller port WWN:
Path status:
50001fe1002709fc
210000e08b1759bd
O.K.
/dev/rdsk/c3t50001FE1002709F9d1s2
/devices/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709f9,1:c,raw
LUN path port WWN:
Host controller port WWN:
Path status:
50001fe1002709f9
10000000c93d5cee
O.K.
/dev/rdsk/c3t50001FE1002709FDd1s2
/devices/pci@1f,4000/lpfc@5/fp@0,0/ssd@w50001fe1002709fd,1:c,raw
LUN path port WWN:
Host controller port WWN:
Path status:
50001fe1002709fd
10000000c93d5cee
O.K.
Errors for Solaris 8, 9, and 10
You will see error messages similar to the following for each EVA controller for LUN 0 on the console
and in /var/adm/messages. You can ignore these error messages.
May 18 15:10:30 simonb fcp: [ID 305930 kern.warning] WARNING: fp1: no
driver for device @w50001fe1002709e8,0:
May 18 15:10:30 simonb compatible: scsiclass,0c.vHP.pHSV210.r5100
scsiclass,0c.vHP.pHSV210 scsa,0c.bfcp scsiclass,0c scsiclass
May 18 15:10:30 simonb scsi: [ID 243001 kern.warning] WARNING:
/pseudo/fcp@0 (fcp0):
8
May 18 15:10:30 simonb Create devinfo failed.
Enabling MPxIO
To enable MPxIO for HP storage devices, the appropriate information for device-type-scsi-
options-list need to be added in the /kernel/drv/scsi_vhci.conf file.
1. Use a text editor to change the configuration file. For example:
# vi /kernel/drv/scsi_vhci.conf
2. For Solaris 8 and 9 perform the following steps:
a. Change the mpxio-disable parameter to the following:
mpxio-disable="no";
b. Verify that the load-balance and auto-failback parameters are set to the following values
(the default setting may vary depending on the patch level / OS version):
load-balance="none";
auto-failback="disable";
c. Add the following lines to cover all HP arrays:
device-type-scsi-options-list =
"HP
HSV101", "symmetric-option",
"COMPAQ HSV111", "symmetric-option",
"HP
"HP
HSV2", "symmetric-option",
OPEN", "symmetric-option";
symmetric-option = 0x1000000;
NOTE:
Enter six spaces after ”HP” and two spaces after “COMPAQ”.
3. For Solaris 10 (patch 118833-24 for SPARC, patch 118855-19 for x86), the scsi_vhci driver is
capable of detecting Asymmetric Logical Unit Access (ALUA) compliant devices advertised in the
Target Port Group Support (TPGS) field. The EVA 4000/6000/8000 and the EVA 3000/5000
V4 are ALUA compliant arrays. Add the following lines to cover all HP arrays:
device-type-scsi-options-list =
"HP
OPEN", "symmetric-option";
symmetric-option = 0x1000000;
NOTE:
Enter six spaces after ”HP”.
4. Depending on the Solaris version, select one of the following options:
a. For Solaris 8 or 9—run a reconfiguration reboot in order to activate the changes:
# reboot -- -r
b. For Solaris 10 (SPARC)—run the stmsboot command and confirm the reboot:
# stmsboot –e
c. For Solaris 10 (x86)–reboot the system to load the new definition if you are configuring
for XP. Configuring for EVA does not require a reboot since definition has already been
configured.
9
MPxIO for Sun Solaris
Verify the MPxIO configuration
After the system has rebooted, use the format or luxadm command to verify that the MPxIO
enablement has been successful.
NOTE:
In the format or luxadm output, controllers c2 and c3 no longer show up. A single, virtual controller
path such as c4 now displays the MPxIO–configured LUNs. The cfgadm output does NOT change
after the MPxIO enablement.
1. Verify with either format or luxadm probe that only one device file is shown for each LUN:
# echo | format
Searching for disks...done
c4t600508B400102E640000B000081C0000d0: configured with capacity of 1008.00MB
c4t600508B400102E640000B00008100000d0: configured with capacity of 1008.00MB
AVAILABLE DISK SELECTIONS:
0. c0t0d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248>
/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037e46c64,0
1. c4t600508B400102E640000B000081C0000d0 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/scsi_vhci/ssd@g600508b400102e640000b000081c0000
2. c4t600508B400102E640000B00008100000d0 <HP-HSV210-5100 cyl 126 alt 2 hd 128 sec 128>
/scsi_vhci/ssd@g600508b400102e640000b00008100000Specify disk (enter its number):
# luxadm probe
No Network Array enclosures found in /dev/es
Found Fibre Channel device(s):
Node WWN:50001fe1002709f0 Device Type:Disk device
Logical Path:/dev/rdsk/c4t600508B400102E640000B000081C0000d0s2
Node WWN:50001fe1002709f0 Device Type:Disk device
Logical Path:/dev/rdsk/c4t600508B400102E640000B00008100000d0s2
2. Check the different paths with the luxadm display command (either for a given disk or for
the array WWN or a port WWN):
a. For Solaris 8 and 9:
# luxadm display 50001fe1002709f0
DEVICE PROPERTIES for disk: 50001fe1002709f0
Vendor:
HP
HSV210
5100
Product ID:
Revision:
Serial Num:
Unavailable
Unformatted capacity: 1024.000 MBytes
Read Cache:
Enabled
0x0
0x0
Minimum prefetch:
Maximum prefetch:
Device Type:
Path(s):
Disk device
/dev/rdsk/c4t600508B400102E640000B000081C0000d0s2
/devices/scsi_vhci/ssd@g600508b400102e640000b000081c0000:c,raw
Controller
/devices/pci@1f,4000/QLGC,qla@4/fp@0,0
50001fe1002709f8,2
210000e08b1759bd
Device Address
Host controller port WWN
Class
primary
State
Controller
Device Address
Host controller port WWN
ONLINE
/devices/pci@1f,4000/QLGC,qla@4/fp@0,0
50001fe1002709fc,2
210000e08b1759bd
10
Class
State
primary
ONLINE
Controller
Device Address
Host controller port WWN
Class
/devices/pci@1f,4000/lpfc@5/fp@0,0
50001fe1002709f9,2
10000000c93d5cee
primary
State
Controller
Device Address
Host controller port WWN
Class
State
...
ONLINE
/devices/pci@1f,4000/lpfc@5/fp@0,0
50001fe1002709fd,2
10000000c93d5cee
primary
ONLINE
b. For Solaris 10 (SPARC and x86), the following is displayed:
NOTE:
The Class of the device shows primary through one controller and shows secondary through the
other controller. This is because the array is now configured as an Asymmetric Active/Active
(A/A/A) array, and the array controller that indicates the primary path owns the LUN.
# luxadm -v display /dev/rdsk/c4t600508B4001029340000600004180000d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c4t600508B4001029340000600004180000d0s2
Vendor:
HP
HSV210
6000
Product ID:
Revision:
Serial Num:
Unsupported
Unformatted capacity: 15360.000 MBytes
Read Cache:
Enabled
0x0
0x0
Minimum prefetch:
Maximum prefetch:
Device Type:
Path(s):
Disk device
/dev/rdsk/c4t600508B4001029340000600004180000d0s2
/devices/scsi_vhci/disk@g600508b4001029340000600004180000:c,raw
Controller
Device Address
Host controller port WWN
Class
State
/dev/cfg/c1
50001fe100270c29,5
10000000c9551826
secondary
ONLINE
Controller
/dev/cfg/c1
Device Address
Host controller port WWN
Class
50001fe100270c2d,5
10000000c9551826
primary
State
Controller
ONLINE
/dev/cfg/c2
Device Address
Host controller port WWN
Class
50001fe100270c2a,5
10000000c9551827
secondary
State
ONLINE
Controller
/dev/cfg/c2
Device Address
Host controller port WWN
Class
50001fe100270c2e,5
10000000c9551827
primary
State
ONLINE
11
MPxIO for Sun Solaris
|