Goal/Vision

Building OpenSolaris-based virtual appliances entails significant Solaris expertise that goes beyond what an appliance developer normally needs in terms of developing, packaging and installing software. Building virtual appliances in a scripted manner (instead of manually clicking here and typing there) also requires sophisticated expertise in the target virtualization platforms.

Therefore, to enable product development engineers (PDEs) and release engineers (REs) to efficiently build virtual appliances, we need to abstract some of these gory details away.

The virtual appliance build service is a step toward this direction. This is a black box service inside Sun's network where PDEs submit a recipe for an appliance and after a while, they get the completed virtual machine (VM) images. The key benefits are:

• Entirely scripted build enables this to become a part of the product build process: for example, a new VM build for every weekly promoted build.
• Blackbox server eliminates the need for setting up one's own build server: because the build environment of VM images itself cannot be virtualized, asking PDEs/REs to set up such an environment on their own is too much work.
• Fixed environment simplifies the overhead for this team: the tool that carries out a build only needs to run on one specific environment (namely the server we run inside SWAN), so no need for nice packaging, configuration file, etc.

Eventually, as the build service is proven, it may even be made available for users on the Internet.

Overview

Virtual Appliance Projects are created and managed by appliance developers who supply an appliance build "recipe" that defines how a set of external content, their own content or "glue" and reusable appliance kit modules are assembled to produce custom virtual appliances. These projects may or may not be run by the same people that own the development projects associated with the main external content.

The Appliance Kit Lite Project helps appliance developers easily define, build and maintain virtual appliances by providing documentation, reusable components, starter building block VM images and a virtual appliance build service.

Implementation Approaches

The main consideration in designing the service to make this easy enough to use for people, and hide enough things so that the AKLite team can change the build process later.

Input to the blackbox (AKA "recipe")

• JeOS image as a binary dependency: for this build service, JeOS should be treated as a opaque binary dependency. If I understand the build process correctly, at the implementation level, "the binary JeOS dependency" is a list of IPS packages that goes into the build process (other than the user-defined IPS packages.) We'll likely have a several different options of the base image (such as just plain JeOS, JeOS+GlassFish, etc.) from which the user can choose from.
• Additional IPS packages to be installed (and their repository URL): software being packaged into VMs should be also separately available for non-v12n use, so having them in IPS packages should be reasonable.
• Pre/post-installation scripts: since IPS lacks the ability to run scripts, the build service should allow user-defined scripts to run in crucial moments of the build process. The trick is to define the points that are relatively unaffected by particular ways of building VMs. We should be able to do at least "post installation script", run from the current BE (which can have a richer set of tools), with the installed BE accessible. Any other interception points?

Output from the blackbox

• Console output from the script: Hudson as a front end can capture this.
• Images: The build service needs to make the resulting images available for some time window (Hudson as a front end can do this), and the user has to pick that up and store them on their storage during this window.

Scripted build process

The main part of this task is to come up with a script that does the build without human intervention, in a headless environment. Note that because this is SaaS and not installable software, the script can be hard-coded against a particular environment.

How the actual script would look will highly depend on which one of several techniques we use for a build, and what v12n technology we use, but for example if we assume multiple BEs on a local system method, it could be something like:

#!/bin/bash -ex

# extract recipe
wget -O recipe.zip $1
unzip recipe.zip

# start VirtualBox (assume the machine is already configured with the JeOS boot environment)
VBoxManage startvm vmfactory

# configure port forwarding from host to guest
VBoxManage setextradata ... 
VBoxManage setextradata ... 
VBoxManage setextradata ... 

# ship the data and script to the guest
scp the-script-to-be-run-from-guest.sh me@guest:~/
scp recip.ezip me@guest:~/

# login to the guest and do the real work
ssh me@guest the-script-to-be-run-from-guest.sh

# reset VM
VBoxManage controlvm vmfactory acpipowerbutton

# do some more work on the guest
ssh me@guest some-more-script.sh

# and I'm sure I missed million other steps
...

# eventually we power off the VM
VBoxManage controlvm vmfactory acpipowerbutton

# and maybe some V2V conversions here?
...

# if we need to do the work in another server, we can do so, too
ssh me@another-server yet-more-script.sh

# done!

At this point, we have a script that, if run with the right input on the right machine, will produce images in some known place in the file system.

Hooking up to Hudson

The final wrap up is to hook this up to Hudson so that people can run the script on their own. In particular, it ...:

• allows users to provide the recipe file, either from GUI via browser or from scripted client.
• checks out the AKLite build script from a subversion repository
• make sure multiple executions are serialized so that two executions don't interfere with each other.
• start the script and monitor its progress
• records the console output from the build script
• capture the resulting image files and make it available from HTTP and/or NFS.

See the mock up configuration in Hudson

Future Proofing

The input as a combination of a list of user packages and a post-installation script should work consistently regardless of both installation methods that Rudolf came up so far (switching BEs in the same guest and AI-based install), plus additional variations in details (such as how to compress disk images, V2V conversion or not, etc.) So it should give us a reasonable assurance that it'll work with new methods that may come along in the future.

Build process can be decomposed to several parts (installation, virtual disk compression, and V2V conversions, for example) for better pipelining and faster turn-around time during debugging.

If the demand of the build service grows, we can have multiple identically configured servers that'll do the builds, and let Hudson do the scheduling.

The build script in the Subversion repository will encourage interested parties to adopt it and modify it.

We might consider adding test steps to this, to ensure that the resulting image qualifies the basic quality criteria, or even running the existing product test suites.

Resources

• Rudolf's Investigations
  • Restoring OpenSolaris 2008.11 JeOS Prototype from ZFS Stream
  • Original Development of JeOS Building Block - Rudolf Kutina
  • Using Automated Installer to Create VMware ESX-based Image - Rudolf Kutina
  • JeOS Downloads - Rudolf Kutina
• OpenSolaris Documentation
  • OpenSolaris Install Documentations: Automated Installer and Distro Constructor