RAID-Z (5 Minutes)

Description

This demo is based on the Introduction to ZFS Pools tutorial by Stefan Schneider. We'll be using a USB hub and USB sticks to create a RAIDZ2 pool, simulate a disk failure and then recover. We'll also use the iobar utility to graphically show what's going on.

OpenSolaris Versions Supported

2008.05, 2008.11

Points to Hit

Using ZFS you can easily create a software RAID. In this case we'll be creating a 4GB high throughput USB volume with 2*1GB parity disks.

Demo Prep

You need a USB hub and 6 USB sticks. Have the USB hub plugged and all 6 sticks plugged in. If you don't have a USB hub but 2 free USB ports on your laptop, you can do a modified version of this demo using just a mirror (RAID0).
Install iobar, which you can get from the Solaris Performance Tools CD
Start iobar. From wherever you extracted the cd run ./tools/bin/iobar
Identify some directory with about 1GB of disk space. This will be large enough to allow us to do our demo (pull disks, etc).
You'll need 2 terminal windows open. The 2nd is to monitor the status of the zpool while operations are occurring in the other window.

Gotchas

Don't try removing more than 2 disks. This will hang ZFS.

Demo

Introduce iobar to the audience. Explain the each column represents a disk, with the far left column representing your hard drive. Each disk has a read and write column. In this demo we will create a pool. As we copy data from our hard disk to the new pool representing the USB sticks, will see the read column light up for the hard disk and the write column light up for the USB disks. The column on the far right is the total for all disks.

pfexec su
Run rmformat to check for existing removable media. The 6 disks plus your CD/DVD drive should be listed. Note the device names of the 6 USB sticks.

Create the RAIDZ2 File System

With iobar visible, run zpool create z2pool raidz2 c0t0d0p0 c1t0d0p0 c2t0d0p0 c5t0d0p0 c6t0d0p0 c7t0d0p0 - adjust your device names accordingly. To save typing time, I use a script for this create_raidz2_pool.
Run zpool status z2pool to check out the new pool
Run zpool list to see it's capacity - 0% should be used

Creating a Disk Failure

Here we'll tar up our 1GB of data, saving the file to the new z2pool file system. During the tar process, we'll start pulling disks .

Change to the directory with about 1GB of data cd ~. You can run du -hs Documents/ to show its size (assuming you're using the Documents directory). If you're doing this demo with only a 2 disk mirror, it will be much slower so choose a smaller file set (100M).
Run tar -cf /z2pool/documents.tar ./Documents/. While the tar operation is running, you'll see all 6 disks in action in iobar. Pull out a disk. One iobar column should go dark.

In the other terminal window, run zpool status z2pool. You'll see a disk marked as removed. You'll also see some status information:

status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P

Run zpool list to see that the disk's health is DEGRADED.

Pull out another disk and you should have only 4 disk being written to in iobar.
Run zpool status z2pool to see that you're now 2 disks down.
Run zpool list to see that the disk's health is DEGRADED.

Wait for the tar command to complete. Switch to the /z2pool directory and run tar -tvf documents.tar to verify the integrity of the archive.

Recovering

Now will put the disks back. To make it fun, we'll do this while untaring the file.

Run tar -xvf documents.tar
Reinsert the 2 USB sticks

In the other window, run zpool status z2pool. Notice the disks are back on-line and are in the process of been resilvered. You may see some status messages such as:

status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: resilver in progress for 0h0m, 1.36% done, 0h14m to go

Run zpool clear z2pool to clear the status messages.
There's no need to wait for the resilvering process to complete, but to speed it up, delete the contents of the /z2pool directory.

Demo Cleanup

These are the necessary steps to take in order to successfully run the demo again on the same machine.

Run zpool destroy z2pool

RAIDZDemo