ZFS, Crossbow, Zones and DTrace Demo (20 Minutes)
Description
This demo attempts to showcase 4 unique OpenSolaris features in one comprehensive demo (and could certainly be expanded to show more). It begins with an introduction to ZFS, followed by an introduction to Crossbow. A zone is then configured with a VNIC (Crossbow) and installed by cloning an existing zone (ZFS). Then, DTrace is used to monitor network performance between the global and local zone.
OpenSolaris Versions Supported
2009.06
Points to Hit
The points your demo gets across
Demo Prep
- Install netperf, a network benchmarking tool.
- Install a zone named "basezone" from which the clone will be made. Install netperf in the zone as well.
- Save the following to ~/myzone.config:
create set zonepath=/zones/myzone set ip-type=exclusive add net set physical=myzone0 end
Gotchas
- Crossbow requires a wired Internet connection, which is rare these days on the road. I find it easier to do this demo in VirtualBox, which will set up a private network, regardless of the physical connection.
Demo
ZFS Demo
Crossbow Demo
Crossbow and Zones Demo
Explain that zone creation involves 4 steps: zone configuration, installation (or clone), boot and system configuration.
Configuration
Show the myzone.config file.
bleonard@os200906:~# cat myzone.config create set zonepath=/zones/myzone set ip-type=exclusive add net set physical=myzone0 end
Note the zone's IP type is exclusive and the physical interface is actually a virtual NIC 'myzone0' (which hasn't be created yet).
Create myzone0:
dladm create-vnic -l e1000g0 myzone0
Now we're ready to configure the zone. Note that zone configuration could be done interactively:
zonecfg -z myzone myzone: No such zone configured Use 'create' to begin configuring a new zone. zonecfg:myzone> zonecfg:myzone> exit
But the configuration steps can also be supplied, automating the process:
bleonard@os200906:~# zonecfg -z myzone -f myzone.config bleonard@os200906:~# zonecfg -z myzone info zonename: myzone zonepath: /zones/myzone brand: ipkg autoboot: false bootargs: pool: limitpriv: scheduling-class: ip-type: exclusive hostid: net: address not specified physical: myzone0 defrouter not specified
Installation
Zone installation in OpenSolaris requires a roughly 50MB download from the IPS repository. In the interest of time, OpenSolaris also provides the option to clone an existing zone. Show the existing zone, basezone:
bleonard@os200906:~# zoneadm list -cv ID NAME STATUS PATH BRAND IP 0 global running / native shared - basezone installed /zones/basezone ipkg shared - myzone configured /zones/myzone ipkg excl
You can see that basezone also has its own file system:
bleonard@os200906:~# zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 4.90G 26.4G 80.5K /rpool rpool/ROOT 3.13G 26.4G 19K legacy rpool/ROOT/opensolaris-1 3.13G 26.4G 3.13G / rpool/dump 511M 26.4G 511M - rpool/export 444M 26.4G 21K /export rpool/export/home 444M 26.4G 21K /export/home rpool/export/home/bleonard 444M 26.4G 444M /export/home/bleonard rpool/swap 512M 26.7G 137M - rpool/zones 347M 26.4G 21K /zones rpool/zones/basezone 347M 26.4G 22K /zones/basezone rpool/zones/basezone/ROOT 347M 26.4G 19K legacy rpool/zones/basezone/ROOT/zbe 347M 26.4G 347M legacy
Typically you would install a zone using the following, zoneadm -z myzone install. However, once you have a zone installed, it's much quicker to use the clone command to install the zone:
bleonard@os200906:~# zoneadm -z myzone clone basezone sys-unconfig started Fri Sep 18 14:20:06 2009 rm: /zones/myzone/root/etc/vfstab.sys-u: No such file or directory grep: can't open /zones/myzone/root/etc/dumpadm.conf sys-unconfig completed Fri Sep 18 14:20:06 2009
Now looking at the zone list again, we see myzone's status is installed:
bleonard@os200906:~# zoneadm list -cv ID NAME STATUS PATH BRAND IP 0 global running / native shared - basezone installed /zones/basezone ipkg shared - myzone installed /zones/myzone ipkg excl
You many (or may not) want to point out the difference in IP type. Since I'm only using the basezone as a template, I didn't see a need to create and assign a VNIC. It also keeps any pre-existing VNICs from appearing in my dladm show-vnic output which I don't have to explain.
Boot and System Configure
zoneadm -z myzone boot zlogin -C myzone
Work through the system configuration steps. Note that these could be automated as well. When prompted, assign an IP address of 10.0.2.15. Once complete, log into the zone when prompted.
From myzone0, run ifconfig:
root@myzone:~# ifconfig -a4
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
myzone0: flags=1000863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.0.2.25 netmask ffffff00 broadcast 10.0.2.255
ether 2:8:20:89:e5:83
Not the interface is the virtual one, myzone0. It is fully manageable from w/in the zone.
Crossbow Bandwidth Limits
Besides being able to assign zones exclusive NICs, virtual NICs also provide additional quality of service capabilities such as bandwidth controls. For example, let's run a quick network benchmark between the glocal and local zone.
In the local zone, start netserver:
root@myzone:~# netserver Starting netserver at port 12865 Starting netserver at hostname 0.0.0.0 port 12865 and family AF_UNSPEC
Then in the global zone run a 5 second benchmark:
bleonard@os200906:/zones/myzone/root/etc# netperf -H 10.0.2.25 -l 5 TCP STREAM TEST from ::ffff:0.0.0.0 (0.0.0.0) port 0 AF_INET to ::ffff:10.0.2.25 (10.0.2.25) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 49152 49152 49152 5.31 788.15
Notice the throughput in Megabits per second (788 in my case). Show the speed of the VNIC:
bleonard@os200906:/zones/myzone/root/etc# netperf -H 10.0.2.25 -l 5 WARNING! getaddrinfo returned a protocol other than the requested protocol. Please contact your vendor for a fix to this bug in getaddrinfo() TCP STREAM TEST from ::ffff:0.0.0.0 (0.0.0.0) port 0 AF_INET to ::ffff:10.0.2.25 (10.0.2.25) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 49152 49152 49152 5.31 788.15
Reduce the speed to 400 Megabits per second and run the benchmark again:
bleonard@os200906:~# dladm set-linkprop -p maxbw=400 myzone0 bleonard@os200906:~# dladm show-vnic myzone0 LINK OVER SPEED MACADDRESS MACADDRTYPE VID myzone0 e1000g0 400 2:8:20:89:e5:83 random 0 bleonard@os200906:~# netperf -H 10.0.2.25 -l 5 TCP STREAM TEST from ::ffff:0.0.0.0 (0.0.0.0) port 0 AF_INET to ::ffff:10.0.2.25 (10.0.2.25) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 49152 49152 49152 5.31 288.55
For fun, bring it down to something ridiculously low:
bleonard@os200906:~# netperf -H 10.0.2.25 -l 5 TCP STREAM TEST from ::ffff:0.0.0.0 (0.0.0.0) port 0 AF_INET to ::ffff:10.0.2.25 (10.0.2.25) port 0 AF_INET Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 49152 49152 49152 5.40 3.17
DTrace Chime Basics
[DTrace, Zones and Crossbow]
First, set the bandwidth back to 1000:
dladm set-linkprop -p maxbw=1000 myzone0
Run a 5 minute benchmark. The point here is to really just put continuous load on the system:
netperf -H 10.0.2.25 -l 300
Now, run the DTrace Network script from Chime and plot the throughput for netperf over time. While the graph is running, dynamically adjust the throughput. Looking at the output we see that the good folks using myzone are consuming way more bandwidth then we'd like. They're actually starving the other zones on this machine (remember, there's only a single physical interface). How can I lasso these cowboys in?
dladm set-linkprop -p maxbw=200 myzone0
Demo Cleanup
These are the necessary steps to take in order to successfully run the demo again on the same machine.
- Delete myzone
- Delete myzone0 vnic
Sign up or Log in to add a comment or watch this page.