LDoms Community Cookbook

Contents

In this Section ...

Section Introduction

This section provides examples of basic and advanced administrative procedures to assist in your management of the Logical Domains environment.

LDoms installation

(Contributed by Peter A. Wilson)

Installation Overview

The first and obvious recommendation regarding LDoms installation is to follow the Administration Guide documents and adhere to the requirements set out there. There are a couple of pre-requisites that should be noted as an absolute minimum.

Factory Installation Note, that all systems delivered fresh from the factory have at least the required minimum versions of firmware and OS, plus patches, already applied and the pre-installed image of Solaris on your system also includes (amongst other goodies) the current (at the time of the pre-install) version of LDoms packages. You are just required to enable the LDoms management daemon (as shown below) to be able to work with LDoms.

If you are carrying out a fresh installation of the Solaris OS, you will need to add the SUNWldm.v (and optional associated) packages to be able to use LDoms.

• You can only deploy LDoms on Sun SPARC based servers with the sun4v architecture, this encompasses all 'Niagara' CPU's, such as the UltraSPARC T1, the UltraSPARC T2 and the latest CPU, the UltraSPARC T2 Plus. All future version of Sun SPARC chips will also support Logical Domains.
• You will be required to be running at least a minimum level of system firmware. System firmware is distributed as a single file, with a given version number, which comprises various files updating many elements of the system to latest versions, including the Service Processor, OBP, POST, the Hypervisor, and vBSC. You can determine your current version using many methods, I prefer to use the Service Processor, the methods for this are shown below.
• A minimum version of Solaris is required to run in the Control Domain (also called the primary domain), and for various platforms there will be different additional required patches that must be installed on top of that minimum release of OS.

All of these minimum requirements will be discussed in detail in the Product Notes issued with the particular version of LDoms that you are about to install, these can be accessed here LDoms Documentation.

Once you have complied with minimum version requirements, the installation process is very straightforward. The downloadable LDoms tarball included all the required toos to carry out a complete installation as indicated in the Administration Guide.

It is also possible to take some shortcuts if you are looking to test LDoms and are not too concerned about enforcing security at this stage.

In fact in order to install Ldoms only a single package is required from the tarball that you downloaded, this is the SUNWldm.v package. The process of installing must be carried out with 'root' privileges but a simple 'pkgadd -d SUNWldm.v' is sufficient. The binaries will be installed in /opt/SUNWldm/bin, this path is not automatically added to your environment, its definitely going to save you time to do so.

The installation process does not automaticlally activate LDoms, in order to do that it is necessary to enable the LDoms management daemon, this can be done the command 'svcadm enable ldmd':

Once the LDom management daemon is running a simple test can be run to ensure that all is well, run the command '/opt/SUNWldm/bin/ldm list', this will produce an abbreviated listing of all of the currently running domains and their status, in the default case, this will indicate a single 'primary' domain that owns all of the resources of the server.

At this stage you have a working installation of LDoms and can carry on with further testing and configurations.

Firmware Updates

You are always recommended to stay current wherever possible with the latest versions of firmware and patches on any systems deployment.

If during your checks to determine if your system required updated firmware, the following chapter describes the options available to you for the various servers in the CMT family line.

Updating UltraSPARC T1 Systems

Sun Fire T1000/Sun SPARC Enterprise T1000/Sun Fire T2000/Sun SPARC Enterprise T2000/Sun Blade T6300

These servers utilise a Service Processor running ALOM and as a result expect to be able to communicate with an FTP server to download and install their required firmware upgrades.

You can check your current firmware version number by running the 'showhost command:

In this case the T1000 is running firmware release version 6.5.11.

You must complete a few steps to prepare for a system firmware upgrade.

• You must locate the relevant firmware update image, these can be located on the SunSolvewebsite. Firmware updates are distrbuted as Patches, the minimum required version is indicated in the LDoms version specific Release Notes document.
• Copy the firmware file to a suitable FTP server, accessible from the network that the SP is configured onto, and uncompress the patch ZIP file into a suitable directory.
• Ensure that the Service Processor is suitably networked to access the FTP server.
• Firmware upgrades require that the main server be powered off, so bring the main server down in an orderly fashion 'shutdown -i0 -g0 -y'
• Update the firmware using the 'flashupdate' command and then reset the SP using the 'resetsc' command.

Remember that, when running the flashupdate command, the SP will prompt you for a username and password, these are the username and password of a user on the FTP server in whose directories the firmware file can be found for downloading.

Once the SP has reset, you can check again for the version of the firmware by running the SP command 'showhost'.

Updating UltraSPARC T2 & Later Systems

Sun SPARC Enterprise T5120/T5220/T5140/T5240/T5440, Sun Blade T6320/T6340

These servers have a more powerful SP running the ILOM interface, but also implement an ALOM command line interface for backwards compatibility with the older ALOM only systems.

ILOM is a much more powerful remote management interface and as a result provides a wider range of options for upgrading system firmware, these are...

• From the ILOM CLI, (the preferred option for the future.)
• From the ALOM CLI, (the simplest option for experienced ALOM users.)
• From the WEBGUI, (the simplest point and click option, requiring no local TFTP server)
• From the ALOM or ILOM CLI even if the SP is not networked.

Each of these methods requires the main server to be powered down before starting the upgrade, but in the case of the non-networked SP, the main server must first be used to download the firmware internally to the SP, after which the main server must be shutdown in order to carry out the upgrade.

Below are the commands that are run in both the ILOM CLI and the ALOM CLI to check the firmware versions and to then carry out the firmware upgrade:

In this case, the server is running firmware release 7.1.5.b.

If you have old firmware and must upgrade, the CLI methods are simple but require that the firmware file be first copied onto a TFTP server on the local sub-net of of the SP.

In both cases, the IP and PATH shown in the command line represent the IP address of the TFTP server being used, and the PATH of the file being server from the TFTP server.

TFTP Path NOTE, that the PATH must exclude the TFTP servers default serving directory. In most cases this will be /tftpboot, so a file located on the TFTP server at /tftpboot/t5440/firmware/Sun_System_Firmware-7_1_5_b-SPARC_Enterprise_T5440.pkg, would have the path specified on the command line as /t5440/firmware/Sun_System_Firmware-7_1_5_b-SPARC_Enterprise_T5440.pkg .

Also note that for the case of the SP not being networked the procedure requires that the firmware patch file be first downloaded to the host server OS, and from the host server OS, the patch file can be unzipped and the frmware.pkg file should be downloaded using the sysfwdownload command directly to the SP:

Now, at this stage all thats happened is that the Host OS has downloaded the firmware file to the local memory of the Service Processor, the firmware has not been installed yet. You must now switch to the SP CLI and carry out the rest of the process.

First you must power down the main server and then carry out a normal firmware update command with the one change being the IP address of the tftp server, instead of being a remote server the IP address should be specified as 127.0.0.1, in fact the SP is downloading the file from itself and knows to look in its memory for the file you previously downloaded from the Host.

Cloning Guest Domains

(Contributed by Francois Marcoux)

Clone a Guest Domain locally

Cloning a Guest Domain locally can be done as follows:

• clone the source Guest Domain's OS disk.
• create a target Guest Domain with the same configuration as the source Guest Domain.

One considers here that the source Guest Domain does not have any data disks (only a boot disk).

Duplicate the source OS disk
Several duplication techniques can be used. A few examples are:

• dd for disk partitions, LUNs or slices
• ZFS snapshot and cloning features
• SVM mirrors
• ufsdump/ufsrestore

At the end of this step, the target OS disk should be an exact copy of the source OS disk.
Create the virtual volume associated with the target OS disk

• Here, with a SAN LUN:

  primary# ldm add-vdsdev /dev/dsk/c4t600601604FE610007A635A903FA0D811d0s2 ldg1_VOL0@VDS

Create the target domain from the source domain configuration

• Source domain : ldg0
• Target domain : ldg1

Check that enough resources (CPU, memory) are available to create a target domain as large as the source domain. If not, the XML configuration file used to create the target domain would have to take that into account.

• Here source domain ldg0 has 32 vCPUs and 16 GB of RAM. 48 vCPUs and 24 GB of RAM remain available, so enough to create a target domain as large as ldg0:

  primary# ldm list ldg0

  NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
  ldg0             active   -t---   5000    32    16G      7.1%  4s
  

  primary# ldm list-devices vcpu | grep 100 | wc -l
  48
  primary# ldm list-devices mem

  MEMORY
  PA                   SIZE
  0xa0e000000          24352M
  

• Export the source domain configuration to an XML file. Modify this file and adapt it to the characteristics of the target domain:

  primary# ldm list-constraints -x ldg0 > /var/tmp/ldg0.xml
  primary# cp /var/tmp/ldg0.xml /var/tmp/ldg1.xml

The following lines should be changed :

• lines containing the domain's name
• lines containing MAC addresses. Note : if dynamic MAC address allocation is preferred, the address can be replaced by the string 'auto-allocated ' giving a line like <rasd:Address>auto-allocated</rasd:Address>
• if the source domain does not have the same number of resources (vCPUs, RAM, MAUs, ...), the corresponding lines must be changed. Those lines use the <rasd:AllocationUnits> XML tag. This type of entries is not covered in the example below.

6c6
<       <Content xsi:type="ovf:VirtualSystem_Type" ovf:id="ldg0">
---
>       <Content xsi:type="ovf:VirtualSystem_Type" ovf:id="ldg1">
34c34
<             <rasd:Address>00:14:4f:fc:04:6b</rasd:Address>
---
>             <rasd:Address>00:14:4f:fc:04:ab</rasd:Address>
42c42
<             <rasd:Address>00:14:4f:fc:04:6c</rasd:Address>
---
>             <rasd:Address>00:14:4f:fc:04:ac</rasd:Address>
50c50
<             <rasd:Address>00:14:4f:fc:04:63</rasd:Address>
---
>             <rasd:Address>00:14:4f:fc:04:a3</rasd:Address>
60c60
<             <gprop:GenericProperty key="vol_name">ldg0_VOL0</gprop:GenericProperty>
---
>             <gprop:GenericProperty key="vol_name">ldg1_VOL0</gprop:GenericProperty>

• Import the modified XML file to create the target domain:

  primary# ldm add-domain -i /var/tmp/ldg1.xml

Start the Target Domain

• Bind and start the target domain:

  primary# ldm bind ldg1
  primary# ldm start ldg1

Upgrading Logical Domains

Upgrade from LDoms 1.0.1 and LDoms 1.0.2 to LDoms 1.0.3

(Contributed by Jason Bolero from a mail thread by Nick Ruth)

• Ensure Firmware is at the appropriate level (minimum of 6.6.x for T1 platform and 7.1.x) for T2 platform. Note that the Admin guide states that 6.5 and 7.0 are the minimum levels but the release notes state that 6.6 and 7.1 are the required level for 1.0.3 functionality.
• Ensure the Solaris OS is at the appropriate kernel level. The Release notes have Solaris 10 11/06 (update 3) at kernel level 127127-11 as the absolute minimum. For update 4 (08/07) it's still kernel level 127127-11 and update 5 (05/08) starts at kernel level 127127-11.
• Stop the guest domains. I brought the guest domains down to the OK prompt and then issued the ldm stop-domain ldomname from the control domain before proceeding.

  Backup your configurations For safety sake issue the ldm ls-constraints ldomname > /tmp/ldomname.xml command for each domain (including primary), so that you have a listing of the definitions for the ldom.

• Disable the logical domains manager daemon service.

  primary# svcadm disable ldmd

• Remove the logical domains manager packages.

  primary# pkgrm SUNWldm

• Add the new logical domains manager packages (Note: the -G option forces installation only in global zone).

  primary# pkgadd -Gd . SUNWldm.v

• Refresh the SMF database for new logical domains services configuration.

  primary# svcadm refresh ldmd

• Start the logical domains manager daemon service.

  primary# svcadm enable ldmd

• After ensuring that the ldmd service is "online" start the various domains as needed.

  primary# ldm start-domain ldomname

  
  

  Other Notes The following caveat should also be noted: The admin guide states that existing LDOMS 1.0.1 or 1.0.2 configurations are compatible with 1.0.3. This is not entirely the case. The upgrade section needs to be amended to point out the following: If the backend volume is a disk slice or a volume manager volume then the virtual disk backend must be re-exported with options=slice. If this reconfiguration is not done the guest domain will not be able to see the disk service and will get I/O errors.

  Hope this helps others.