Testing data consistency during Mutipathing switchover

SysAdmin Topics

Testing Data Consistency During Multipathing Switchover

(a submission from BigAdmin community, March 2009)

Note: This was tested on Solaris 10 OS and it should work on Solaris 9 as well.

Removing EMC PowerPath and enabling MPXIO on Solaris server while the filesystems are in use

Plan:

1. Create Disk group using EMC PowerPath pseudo devices
2. Create volume and filesystem on them
3. Write some data on this filesystem
4. Remove PowerPath
5. Enable MPXIO

Intention:
To check whether the data is consistent across this removal of pseudo devices and getting disks under MPXIO.

I have initialized two pseudo devices under Veritas as shown below:

bash-3.00# vxdisk list
DEVICE TYPE DISK GROUP STATUS
Disk_0 auto:none - - online invalid
Disk_1 auto:none - - online invalid
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
emcpower0s2 auto:none - - online invalid
emcpower1s2 auto:none - - online invalid
emcpower2s2 auto:sliced MP_Test01 MP_Test online
emcpower3s2 auto:sliced MP_Test02 MP_Test online

Created a regular volume:

bash-3.00# vxprint \-t TestVolume
Disk group: MP_Test

V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE

v TestVolume - ENABLED ACTIVE 41932800 SELECT - fsgen

Created filesystem on it and mounted it on /test:

bash-3.00# df \-kh
Filesystem size used avail capacity Mounted on
/dev/md/dsk/d0 3.9G 2.0G 1.9G 52% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 29G 1.3M 29G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
/platform/sun4u-us3/lib/libc_psr/libc_psr_hwcap1.so.1
3.9G 2.0G 1.9G 52% /platform/sun4u-us3/lib/libc_psr.so.1
/platform/sun4u-us3/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
3.9G 2.0G 1.9G 52% /platform/sun4u-us3/lib/sparcv9/libc_psr.so.1
fd 0K 0K 0K 0% /dev/fd
/dev/md/dsk/d30 3.9G 613M 3.3G 16% /var
swap 29G 0K 29G 0% /tmp
swap 29G 32K 29G 1% /var/run
/dev/md/dsk/d40 2.0G 15M 1.9G 1% /export
/dev/vx/dsk/MP_Test/TestVolume
20G 191M 19G 1% /test

Wrote some data into the mountpoint:

bash-3.00# ls \-l /test
total 248446
-rw-r{-}{-}r-\- 1 root root 45000192 Feb 12 10:42 EMCPower.SOLARIS.4.5.0.tar
\-rwxr-xr-x 1 nobody nobody 2702848 Aug 24 2005 VRTSvlic.tar
\-rwxr-xr-x 1 nobody nobody 79499776 Aug 24 2005 VRTSvxfs.tar
drwxr-xr-x 2 root root 96 Feb 12 11:52 lost+found
drwx-----\- 2 root root 96 Feb 12 12:02 sdk

Now, I have removed the package EMCPower. Below o/p shows the list of devices before rescanning devices:

bash-3.00# vxdisk list
DEVICE TYPE DISK GROUP STATUS
Disk_0 auto:none - - online invalid
Disk_1 auto:none - - online invalid
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_0 auto - - error
emcpower0s2 auto:none - - online invalid
emcpower1s2 auto:none - - online invalid
emcpower2s2 auto:sliced MP_Test01 MP_Test online
emcpower3s2 auto:sliced MP_Test02 MP_Test online



bash-3.00# vxprint \-t TestVolume
Disk group: MP_Test

V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE

v TestVolume - ENABLED ACTIVE 41932800 SELECT - fsgen

This change had taken place in format:

bash-3.00# echo \| format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c2t5006016939A039D1d0 <drive type unknown>
/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2/fp@0,0/ssd@w5006016939a039d1,0
1. c2t5006016139A039D1d0 <drive type unknown>
/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2/fp@0,0/ssd@w5006016139a039d1,0
2. c3t5006016B39A039D1d0 <drive type unknown>
/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2,1/fp@0,0/ssd@w5006016b39a039d1,0
3. c3t5006016339A039D1d0 <drive type unknown>
/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2,1/fp@0,0/ssd@w5006016339a039d1,0
4. c4t500000E016721B00d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/scsi_vhci/disk@g500000e016721b00
5. c4t500000E016723320d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/scsi_vhci/disk@g500000e016723320
6. c4t6006016016E71E00A8DE771C9AF7DD11d0 <DGC-RAID5-0326 cyl 32766 alt 2 hd 64 sec 10>
/scsi_vhci/ssd@g6006016016e71e00a8de771c9af7dd11
7. c4t6006016016E71E00ACBA29FE99F7DD11d0 <DGC-RAID5-0326 cyl 32766 alt 2 hd 64 sec 10>
/scsi_vhci/ssd@g6006016016e71e00acba29fe99f7dd11
8. c4t6006016016E71E00B4669D129AF7DD11d0 <DGC-RAID5-0326 cyl 32766 alt 2 hd 64 sec 10>
/scsi_vhci/ssd@g6006016016e71e00b4669d129af7dd11
9. c4t6006016016E71E0066325E069AF7DD11d0 <DGC-RAID5-0326 cyl 32766 alt 2 hd 64 sec 10>
/scsi_vhci/ssd@g6006016016e71e0066325e069af7dd11
Specify disk (enter its number): Specify disk (enter its number):

Rescanning devices and checking:

bash-3.00# vxdisk scandisks

bash-3.00# vxdisk list
DEVICE TYPE DISK GROUP STATUS
Disk_0 auto:none - - online invalid
Disk_1 auto:none - - online invalid
Disk_2 auto:sliced MP_Test02 MP_Test online
Disk_3 auto:sliced MP_Test01 MP_Test online
Disk_4 auto:none - - online invalid
Disk_5 auto:none - - online invalid
EMC_CLARiiON0_0 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_1 auto - - error
EMC_CLARiiON0_2 auto - - error
EMC_CLARiiON0_3 auto - - error

Now, pseudo devices are gone. But the respective disks that were mapped to the pseudo devices are under DG as it was; and the volume on the disks is running fine and the data is intact.

bash-3.00# vxprint \-t TestVolume
Disk group: MP_Test

V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE

v TestVolume - ENABLED ACTIVE 41932800 SELECT - fsgen


bash-3.00# ls \-l /test
total 248446
-rw-r{-}{-}r-\- 1 root root 45000192 Feb 12 10:42 EMCPower.SOLARIS.4.5.0.tar
\-rwxr-xr-x 1 nobody nobody 2702848 Aug 24 2005 VRTSvlic.tar
\-rwxr-xr-x 1 nobody nobody 79499776 Aug 24 2005 VRTSvxfs.tar
drwxr-xr-x 2 root root 96 Feb 12 11:52 lost+found
drwx-----\- 2 root root 96 Feb 12 12:02 sdk

I now enabled MPXIO:

bash-3.00# luxadm display /dev/rdsk/c4t6006016016E71E0066325E069AF7DD11d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c4t6006016016E71E0066325E069AF7DD11d0s2
Vendor: DGC
Product ID: RAID 5
Revision: 0326
Serial Num: APM000749007
Unformatted capacity: 10240.000 MBytes
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):

/dev/rdsk/c4t6006016016E71E0066325E069AF7DD11d0s2
/devices/scsi_vhci/ssd@g6006016016e71e0066325e069af7dd11:c,raw
Controller /devices/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2,1/fp@0,0
Device Address 5006016b39a039d1,2
Host controller port WWN 10000000c95eb0c7
Class primary
State ONLINE
Controller /devices/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2,1/fp@0,0
Device Address 5006016339a039d1,2
Host controller port WWN 10000000c95eb0c7
Class secondary
State STANDBY
Controller /devices/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2/fp@0,0
Device Address 5006016939a039d1,2
Host controller port WWN 10000000c95eb0c6
Class primary
State ONLINE
Controller /devices/pci@1e,600000/pci@0/pci@1/pci@0/pci@8/lpfc@2/fp@0,0
Device Address 5006016139a039d1,2
Host controller port WWN 10000000c95eb0c6
Class secondary
State STANDBY

Errored disk list is flushed away, and the data is available.

bash-3.00# vxdisk list
DEVICE TYPE DISK GROUP STATUS
Disk_0 auto:none - - online invalid
Disk_1 auto:none - - online invalid
Disk_2 auto:sliced MP_Test02 MP_Test online
Disk_3 auto:sliced MP_Test01 MP_Test online
Disk_4 auto:none - - online invalid
Disk_5 auto:none - - online invalid


bash-3.00# ls \-l /test
total 248446
-rw-r{-}{-}r-\- 1 root root 45000192 Feb 12 10:42 EMCPower.SOLARIS.4.5.0.tar
\-rwxr-xr-x 1 nobody nobody 2702848 Aug 24 2005 VRTSvlic.tar
\-rwxr-xr-x 1 nobody nobody 79499776 Aug 24 2005 VRTSvxfs.tar
drwxr-xr-x 2 root root 96 Feb 12 11:52 lost+found
drwx-----\- 2 root root 96 Feb 12 12:02 sdk

Conclusion: Data that was created on pseudo devices will be available on the disks mapped to the pseudo devices, even after the EMC PowerPath is removed and MPXIO is enabled.

Should you have any question or need more information on this please write it on http://unixarena.blogspot.com/
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

Copyright 1994-2009 Sun Microsystems, Inc.
Powered by Atlassian Confluence Sun Guidelines on Public Discourse Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact