Crossbow and Zones (in VirtualBox) Demo (10 Minutes)

Description

Giving networking demos is always challenging because you never know what sort of networking will be available at the place you're giving the demo. In the case of Crossbow, having an Ethernet connection is nice, otherwise the link speed appears as 0, which isn't useful for showing how Crossbow allows you to throttle bandwidth:

bleonard@opensolaris:~$ dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
e1000g0      Ethernet             down       0      half      e1000g0
bleonard@opensolaris:~$ dladm show-vnic
LINK         OVER         SPEED  MACADDRESS           MACADDRTYPE         VID
vnic0        e1000g0      0      2:8:20:36:f7:50      random              0

Yes, we could manually configure the network interface for the demo, but I prefer to leave that to NWAM. Fortunately, VirtualBox VMs are always assigned an IP address, whether your connected to the Internet or not. So in this demo, we use VirtualBox do show off Crossbow, which has the added benefit of allowing you to demo Crossbow no matter what your base operating system happens to be.

This demos shows how to create a virtual NIC. It also shows how virtual NICs can be used with zone, however, the zone creation is not part of the demo (unless you're trying to kill extra time) and is set up beforehand. The demo concludes by setting the bandwidth of the local zone's vnic very low and copying a large file from the local zone to the global zone. The copy command will estimate that it will take hours to complete, but by dynamically adjusting the VNICs bandwidth, we can get the copy to complete in under 1 minute.

OpenSolaris Versions Supported

2009.06

Points to Hit

With OpenSolaris 2009.06, virtualization has expanded to the network. This allows you to better take advantage of other virtualization technologies such as containers.

Demo Prep

• Install OpenSolaris 2009.06 in VirtualBox
• Switch to manual network configuration (it works best with Crossbow):

svcadm disable nwam
svcadm enable physical:default
pfexec ifconfig e1000g0 10.0.2.15/24
pfexec route add default 10.0.2.2 -ifp e1000g0

• Create a VNIC for the zone:

pfexec dladm create-vnic -l e1000g0 myzone0

• Create a ZFS file system for the zone:

 pfexec zfs create -o mountpoint=/zones rpool/zones

• Set up the zone, myzone, as documented in this Zones and Crossbow blog.

Gotchas

• Networking in VirtualBox can be a bit flaky. I've notice over time things start to degrade or just stop working altogether.

Demo

Create the VNIC

• Look at the physical network interfaces on the machine

bleonard@opensolaris:~$ dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
e1000g0      Ethernet             up         1000   full      e1000g0

• Show how you can use the physical interfaces to create a virtual interface, replacing e1000g0 with your LINK (adapter) name:

pfexec dladm create-vnic -l e1000g0 vnic0

• Show the new virtual interface:

bleonard@opensolaris:~$ dladm show-vnic
LINK         OVER         SPEED  MACADDRESS           MACADDRTYPE         VID
myzone0      e1000g0      1000   2:8:20:51:dd:c7      random              0
vnic0        e1000g0      1000   2:8:20:73:7e:fb      random              0

Note the machine had an existing virtual NIC, myzone0. We'll discuss that in a moment.

• If you run ifconfig -a4, you'll notice the new VNIC doesn't appear in the output. That's because it needs to be plumbed:

pfexec ifconfig vnic0 plumb

• The interface still needs and IP address and needs to be brought up:

pfexec ifconfig vnic0 10.0.2.16/24
pfexec ifconfig vnic0 up

• Now view the interfaces again:

bleonard@os200906:~$ ifconfig -a4
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
	inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
	inet 10.0.2.15 netmask ffffff00 broadcast 10.0.2.255
vnic0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
	inet 10.0.2.16 netmask ffffff00 broadcast 10.0.2.255

• The virtual NIC is now ready for use, such as assigning it to a zone. Speaking of which, notice myzone0 isn't in the list of interfaces. Try to plumb it:

bleonard@os200906:~$ pfexec ifconfig myzone0 plumb
myzone0 is used by non-globalzone: myzone

Ah, the 'myzone0' interface doesn't appears in the global zone's listing because it's exclusively managed by the local zone. This is an extremely powerful feature of Crossbow, giving the local zones exclusive control over the IP stack.

Crossbow and Zones

In the interest of time, state you've already created a zone exclusively using the 'myzone0' virtual network interface. You can see in the first output below that the IP type is excl. You can see the same in the 2nd output below, as well as the physical interface used, myzone0.

bleonard@os200906:~$ zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP
   0 global           running    /                              native   shared
   1 myzone           running    /zones/myzone                  ipkg     excl

bleonard@os200906:~$ zonecfg -z myzone info
zonename: myzone
zonepath: /zones/myzone
brand: ipkg
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
hostid:
net:
	address not specified
	physical: myzone0
	defrouter not specified

• Log into the zone and look at its interface configuration

bleonard@os200906:~$ pfexec zlogin -C myzone
[Connected to zone 'myzone' console]

• Log into the zone and look at its interface configuration

pfexec zoneadm -z myzone boot

• The back in the original terminal, login into the zone, using root/abc123.
• Look at the interface configuration and note myzone0 is there:

root@myzone:~# ifconfig -a4
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
myzone0: flags=1000863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.0.2.20 netmask ffffff00 broadcast 10.0.2.255
        ether 2:8:20:51:dd:c7

Testing the VNIC

• In the global zone, list the VNICs again, showing their link speed, which should be 1000 MB/Sec:

bleonard@os200906:~$ pfexec dladm show-vnic
LINK         OVER         SPEED  MACADDRESS           MACADDRTYPE         VID
myzone0      e1000g0      1000   2:8:20:51:dd:c7      random              0
vnic0        e1000g0      1000   2:8:20:ee:73:f3      random              0

• Show how the link speed can be adjusted:

pfexec dladm set-linkprop -p maxbw=50 myzone0

• And then view the VNICs again:

bleonard@os200906:~$ pfexec dladm show-vnic
LINK         OVER         SPEED  MACADDRESS           MACADDRTYPE         VID
myzone0      e1000g0      50     2:8:20:51:dd:c7      random              0
vnic0        e1000g0      1000   2:8:20:ee:73:f3      random              0

• In the local zone, create a 250MB file:

 mkfile 250M big-file

• The copy the file over to the global zone.

root@myzone:~# scp big-file bleonard@10.0.2.15:big-file
The authenticity of host '10.0.2.15 (10.0.2.15)' can't be established.
RSA key fingerprint is 6c:c7:63:7f:dc:1f:33:1e:94:ee:eb:24:23:de:8f:90.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.2.15' (RSA) to the list of known hosts.
Password:
Password:
big-file               5% |*                            | 13568 KB    01:28 ETA

At a 50 megabit / second link speed, it should estimate over an hour to complete.
Note, this is the riskiest part of the demo, I've I've seen the link just deteriorate. If this happens, the only solution I've found is to reboot the zone (pfexec reboot), which makes me wonder if its actually a VirtualBox issue.

• Now, back in the global zone, dynamically adjust the link speed:

pfexec dladm set-linkprop -p maxbw=1000 myzone0

• And then watch the copy finish in a matter of seconds!

big-file             100% |*****************************|   250 MB    00:47
root@myzone:~# ping 10.0.2.15

Demo Cleanup

These are the necessary steps to take in order to successfully run the demo again on the same machine.

• Delete the VNIC:

pfexec ifconfig vnic0 down
pfexec ifconfig vnic0 unplumb
pfexec dladm delete-vnic vnic0

• Halt the zone:

pfexec zoneadm -z myzone halt