Deployment Guide

This chapter provides guidelines for the design of the hardware architecture for VDI 3 deployments with VirtualBox. The information provided here is derived from a sizing test with 1000 desktops which were running a script to simulate an office workload (for closer details see the 'Appendix' chapter). The workload is different for every single installation and relatively small changes in the usage patterns can have noticeable effects on the hardware requirements. Therefore it is a good practice to size every deployment individually. This guide provides cornerstones for such efforts.

The hardware environment for a VDI 3 deployment typically looks like this:

Every (production) deployment consists of one primary VDI core server and at least two secondary VDI core servers to provide redundancy. The VDI core servers host a clustered MySQL database for the VDI data (optional remote databases are supported), route information between clients and desktops, and provide the broker functionality which delivers the desktops to the clients. The VirtualBox servers run the virtual machines which provide the desktops. The storage(s) provide the virtual disks which are interpreted as physical disks by the operating systems running within the virtual machines. The iSCSI protocol is used to transfer the disk data between the VirtualBox servers and the storages. That iSCSI data creates a major part of the total network traffic of a VDI system (for a closer discussion see the 'Storage' chapter).

Other consumers of network bandwidth worth mentioning are the clients of VDI 3 (Sun Rays, RDP clients and the Sun Secure Global Desktop). The clients connect to the VirtualBox servers via the VDI core servers. In case of a Sun Ray client, which uses the ALP protocol to transfer the desktop graphics, the VDI core servers convert the RDP protocol received by the VirtualBox servers to the ALP protocol. So there is one data stream for each client connection between the client, the VDI core server and the VirtualBox server. RDP clients, like the windows connector (uttsc), connect to the VDI core server which in turn uses the 'RDP redirect' feature to instruct the clients to connect to the VirtualBox servers directly as there is no need to translate the RDP protocol. In this case there is a data stream between the soft client and the VirtualBox server.

The texts behind the bold terms are rules of thumb for calculating the according resource requirements.

VDI Core Servers

The primary VDI core server requires a dual-core CPU and 2 GB of memory. As long as the VDI services are not configured on that server (which is not recommended) these hardware requirements do not change with the number of running desktops.

The secondary VDI core server requirements for the number of cores and memory size varies with the number of running desktops supported, as well as the required network bandwidth. The bandwidth also varies with the content displayed. The numbers given below are typical for office work. Displaying videos or web pages with flash content can multiply the required bandwidth.

Number of cores = number of running desktops / 20

Example: Two secondary VDI core servers with 8 CPUs  and 4 cores per CPU can serve 2 * 8 * 4 * 20 = 1280 running desktops

Memory size [MB] = number of desktops * 110 MB + 2048 MB

Example: Two secondary VDI core servers with 64 GB of memory can serve (2 * 64 * 1024 MB - 2 * 2048 MB) / 110 MB = 1154 running desktops

Network bandwidth [Mb/s] = number of running desktops * 0.15 [Mb/s]

Example: One secondary VDI core server with one 1 Gb Ethernet interface can serve 1024 / 0.15 Mb/s = 6827 running desktops

Please refer also to the Complete Sun Ray Server Sizing Guide

VirtualBox Servers

VDI 3 supports any server running Solaris 10u6 to host VirtualBox.

Number of cores = number of running desktops / 4

Example: A server with 8 CPUs and 4 cores per CPU can support up to 8 * 4 * 4 = 128 running desktops

Memory size [MB] = number of running desktops * memory size of a desktop * 1.2 + 1024 MB

Example: A server with 64 GB of memory can support (64 * 1024 MB - 1024 MB) / (512 MB * 1.2) = 105 running desktops of 512 MB in size

A rule of thumb for VirtualBox servers is: "A server with 32 cores and 64 GB of memory supports 100 desktops." While the CPU power of the server chosen for the examples above allows to support 128 desktops it is not advisable to increase the memory size to do so. At least 20% of the available CPU power should be available as security margin.

Network bandwidth [Mb/s] = storage network bandwidth / number of VirtualBox servers

For a closer discussion of the network bandwidth see the chapter 'Storage'.

100+ VMs: If you want to run more than 100 VMs on a single VirtualBox server you need to increase the SYSV semaphores on the VirtualBox server. You need to set the number of available semaphores to the number of VMs you intend to run including a security margin for other processes. To set the SYSV semaphores for 1000 VMs type as root:

prctl -r -n project.max-sem-ids -v 1024
projmod -s -K "project.max-sem-ids=(priv,1024,deny)" user.root

The first line changes the available semaphores for the current process, the second line makes this a permanent system setting for the 'root' user. If the VBoxSVC process is run by another user add a user.myuser line to the /etc/project file and change the second line accordingly.

The maximum number of virtual machines on a single VirtualBox server is 1023.

Storage

VDI 3 supports any Sun Storage 7000 Unified Storage System and any server running the OpenSolaris 2008.11 operating system.

The recommended disk layout is RAID 10 (mirrored sets in a striped set; ZFS stripes the data automatically between multiple sets). It is called 'Mirrored' by the 7000 series. While this disk layout uses 50% of the available disk capacity for redundancy it is faster than RAID 5 for intense small random read/writes which is the typical access characteristic for iSCSI.

The storages provide the virtual disks which are accessed by VirtualBox via iSCSI. iSCSI is a CPU-intensive protocol therefore the number of cores of the storage are a decisive factor for its performance which makes the x7410 the best-suited solution for heavy-duty installations as it can be equipped with up to 16 cores. Other important factors are the memory size (cache), the number of disks and the available network bandwidth.

The network bandwidth is very volatile and determined by the relation of desktops starting up (peak network bandwidth) and desktops that have cached the application(s) in use (avarage network bandwidth). Starting a VM creates a network load of 150 MB which needs to be satisfied in ~30 seconds. If many desktops are started at the same point in time the requested network bandwidth may exceed 1 Gb/s (if the CPUs of the storage can handle the load created by the iSCSI traffic). This senario is typical for shift-work companies. In such a case it is recommended to set the "Pool / Cloning / Machine State" option to "Running" which keeps the desktops always running and therefore decouples the OS boot from the login of a user. Another option is to trunk several interfaces as a cheap way to provide more than 1 Gb/s bandwidth via one IP. It is also possible to use Jumbo Frames to speedup iSCSI connections. Jumbo Frames need to be configured for all participants of the network (storages, VirtualBox servers and switches) and it is important to note that Jumbo Frames are not standardized so there is a risk of incompatibilities.

Typically there is no shortage of disk space. VDI 3 in combination with VirtualBox uses the 'sparse' volume feature of ZFS which allows to allocate more space for volumes than physically available as long as the actual data written does not exceed the capacity of the storage. This feature in combination with the fact that cloned desktops reuse unchanged data of their templates results in a very effective usage of the available disk space. In this light the calculation for disk space below is a worst-case scenario assuming that all volumes are completely used by data which differs from the template.

Number of cores = number of virtual disks in use / 200

Example: A x7210 storage with 2 CPUs and 4 cores per CPU can serve up to 2 * 4 * 200 = 1600 virtual disks

Memory size. The more the better as the free memory can be used as disk cache which reduces the access time

Average Network bandwidth [Mb/s] = number of  virtual disks in use * 0.032 Mb/s

Example: A x7210 storage with one Gigabit Ethernet interface can serve up to 1000 / 0.032 = 31250 virtual disks

Peak Network bandwidth [Mb/s] = number of  virtual disks in use * 40 Mb/s

Example: A x7210 storage with one Gigabit Ethernet interface can serve up to 1000 / 40 = 25 virtual disks

Disk space [GB] = number of desktops * size of the virtual disk [GB]

Example: A x7210 storage with a capacity of 46 TB can support 46 * 1024 GB / 2 / 8 GB = 2944 8 GB disks in a RAID 10 configuration

Helpful Hints

• The graphic performance of desktops is better without background images.
• Avoid processes which generate constant or, even worse, burst disk I/O, as for example the indexing service of MS Windows or virus scanners with a scheduled scan every Friday at 9 pm.

Appendix

The script used during the sizing tests starts a sequence of applications generating a workload which is aligned with the 'heavy worker' workload as defined in VMware's 'VDI server sizing and scaling':

1. Start PowerPoint. Load a massive presentation and browse the slides. Close PowerPoint.
2. Start Internet Explorer. Browse three different Web pages. Close Internet Explorer.
3. Start Command Prompt. Do a directory listing.
4. Start PowerPoint. Load a massive presentation and browse the slides. Close PowerPoint.
5. Start Excel. Open an Excel sheet. Close Excel.
6. Start PowerPoint. Load a massive presentation and browse the slides. Close PowerPoint.
7. Start Word. Type a small document. Close Word.