xVM short cuts for noobs

SysAdmin Topics

by Henry Pepper

Introduction

Purpose

Scope

References

Acronyms

  • dom0: The privileged domain, that controls the other domains. There can be only one.
  • domU: Unprivileged domain, this is where it is happening.
  • Hypervisor:
  • HVM: Hard Virtualized. Fully virtualized, the Guest OS is completely unaware of the underlying dom0 OS.
  • PV: Para Virtualized. Uses virtual network, disk, and console devices.

Section descriptions

Interesting issues

Getting started

You can get your Solaris Fix from the OpenSolaris Download Center, Solaris Express Community Edition - DVD Download or CD Download

It should be updated bi-weekly, but that doesn't seem to hold true.

See also ONNV Flag Days

You need

  • an x86 machine with a processor that support Virtualization. Intel-VT or AMD Secure Virtual Machine (SVM) processors.
    • And the firmware needs to be able to enable the processors Virtualization support.
  • OpenSolaris x86 sn77 or later. As of sn77 xVM is part of the release.

Installing OS

  1. Install Developer edition
  2. Select the xVM entry on the GRUB boot window.
    • To make the selection permanent set ''default 0'' to ''default 1'' in /boot/grub/menu.lst
  3. Verify that xVM is active
    • ''uname -i' must answer: ''i86xpv''
    • ''svcs -a | grep xen''

Installing virt-install style PV

  • --nographics Must be used with --paravirt according to (Ref1)

Open Solaris Express snv_107

  • virt-install -n snv107 --nographics --paravirt --ram=1024 --file-size=10 --mac=random --file=/common/sun/snv107.xvm --cdrom=/common/packages/sol-nv-b107-x86-dvd.iso 
    Starting install...
Creating storage file...  100% |=========================|   10 B    00:00     
Creating domain...                                                 0 B 00:05 
v3.1.4-xvm chgset 'Tue Jan 20 23:12:49 2009 -0800 15913:d5288ddf57b5'
SunOS Release 5.11 Version snv_107 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
NOTICE: Invalid iBFT table 0x1
Configuring /dev
/

    1.     Solaris Interactive Text (Console session) (default)
    2.     Apply driver updates
    3.     Single user shell

Enter the number of your choice.
Selected: 1                                        

Solaris Interactive Text (Console session)

Using install cd in /dev/dsk/c0t1d0p0
Using RPC Bootparams for network configuration information.
Attempting to configure interface xnf0...
Skipped interface xnf0
Reading ZFS config: done.
Setting up Java. Please wait...
Serial console, reverting to text install
Beginning system identification...
Searching for configuration file(s)...
Search complete.
Discovering additional network configuration...

Solaris 10u6 - Does Not Work

  • virt-install -n iss --nographics --paravirt --ram=1024 --file-size=10 --mac=random --file=/common/sun/iss.xvm --cdrom=/common/packages/sol-10-u6-ga1-x86-dvd.iso

Installing virt-install style HVM

Jumpstart

http://opensolaris.org/os/community/xen/docs/virtinstall/

Solaris 10u6 hvm

  • virt-install -n sol10u6 --hvm --vnc --ram=1024 --file-size=10 --file=/common/sun/sol10u6.xvm --os-type=solaris --os-variant=solaris10 --locatio=/common/packages/sol-10-u6-ga1-x86-dvd.iso

Solaris 10u7 hvm

  • virt-install --name sol10u7 --hvm --vnc --ram=1024 --file-size=10 --file=/common/xvm/sol10u7.xvm --cdrom=/common/packages/iso/sol-10-u7-ga-x86-dvd.iso
VNC connection lost for graphical install

For some reason, when selecting the default graphic installation method. The vnc connection was lost right before selecting the Disk layout.

Win2k3

  • ''virt-install -n Windows2003 --hvm -r 640 --vnc -f /zfs/boot/win2k3/root.raw -s 15 -c /dev/dsk/c0t0d0s2''
    • source: Jason Doyle [xen-discuss]

WinXP

  • virt-install -n winxp --hvm --ram=512 --vnc --file-size=4 --file=/common/sun/winp.xvm --cdrom=/common/packages/xp_slipstream.iso
Starting install...
Creating storage file...  100% |=========================|  4.0 B    00:00     
Creating domain...                                                 0 B 00:03 
Unable to connect to graphical console; DISPLAY is not set.  Please connect to localhost:5900
Domain installation still in progress.  You can reconnect to 
the console to complete the installation process.

Ubuntu 8.10

  1. virt-install -n ubuntu --nographics --paravirt --ram=512 --file-size=6 --mac=random --file=/common/sun/ubuntu.xvm --cdrom=/common/packages/ubuntu-8.10-desktop-i386.iso

Administation

VNC

Get vncport

  • virsh vncdisplay <dom_name>
    • virsh vncdisplay sol10u7

Setting VNC password

svccfg -s xvm/xend setprop config/vncpasswd = astring: \"YOUR PASSWORD\"
svcadm refresh xvm/xend
svcadm restart xvm/xend

Grant vnc/xend access rights to user

usermod -K defaultpriv=basic,virt_manage xenadm

Domain states

Source: http://opensolaris.org/os/community/xvm/gettingstarted

A domain can be in one of six states. States are shown in virt-manager screens and in xm list displays:

Name            ID   Mem VCPUs      State   Time(s)
Domain-0              0  2049     2     r-----   4138.5
sxc18                 3   511     1     -b----    765.5

The states are:

  • b, blocked: – The domain is blocked, and not running or able to be run. This can be caused because the domain is waiting on IO (a traditional wait state) or it has gone to sleep because there was nothing running in it.
  • c, crashed: – The domain has crashed. Usually this state can only occur if the domain has been configured not to restart on crash. See xmdomain.cfg(5) for more information.
  • d, dying: The domain is in the process of shutting down or crashing.
  • p, paused: – The domain has been paused, usually through the administrator running xm pause. When in a paused state, the domain will still consume allocated resources like memory, but will not be eligible for scheduling by the hypervisor. Run xm unpause to place the domain in the running state.
  • s, shutdown: – The domain is shut down.
  • r, running: – The domain is currently running on a CPU.

Cloning

Clone a container.

  1. virt-clone -d --original=sol10u7 --name=solclone7 --file=/common/xvm/solclone7.xvm
    • Please note: This works on snv_121, but not snv_117 YMMV

workaround to disk expansion

From e-mail answer by Mark Johnson: "You can mirror the rpool and swap the disk"
You need at least snv 119, tested on snv 121.

  1. zpool status
  2. zfs list 
    NAME                        USED  AVAIL  REFER  MOUNTPOINT
rpool                      5.23G  14.3G  81.5K  /rpool
rpool/ROOT                 4.23G  14.3G    19K  legacy
rpool/ROOT/opensolaris     4.62M  14.3G  2.87G  /
rpool/ROOT/small-be        1.72M  14.3G   365M  /
rpool/ROOT/small-be-clone  3.89M  14.3G   481M  /
rpool/ROOT/snv111b         3.02G  14.3G  2.87G  /
rpool/ROOT/snv121          1.20G  14.3G   825M  /
rpool/dump                  512M  14.3G   512M  -
rpool/export               63.5K  14.3G    21K  /export
rpool/export/home          42.5K  14.3G    21K  /export/home
rpool/export/home/mrj      21.5K  14.3G  21.5K  /export/home/mrj
rpool/swap                  512M  14.7G   137M  -
  3. vdiskadm create -s 30g /vdisks/opensolaris-bigger
    • create a new vdisk
  4. virsh attach-disk opensolaris /vdisks/opensolaris-bigger xvdb --driver tap --subdriver vdisk
    • and attach it to the guest
  5. echo "" | format 
    Searching for disks...done
AVAILABLE DISK SELECTIONS:
      0. c7t0d0 <DEFAULT cyl 2609 alt 0 hd 255 sec 63>
         /xpvd/xdf@51712
      1. c7t1d0 <DEFAULT cyl 3916 alt 0 hd 255 sec 63>
         /xpvd/xdf@51728
Specify disk (enter its number): Specify disk (enter its number):
    • fdisk and format the disk so all the space is in slice 0
  6. /usr/sbin/fdisk -n -B /dev/rdsk/c7t1d0p0
  7. format /dev/rdsk/c7t1d0p0
    1. selecting /dev/rdsk/c7t1d0p0
    2. format> partition
    3. partition> 0 
      Part      Tag    Flag     Cylinders        Size            Blocks
 0 unassigned    wm       0 - 3914       29.99GB    (3915/0/0) 62894475

Enter partition id tag[unassigned]: root
Enter partition permission flags[wm]:
Enter new starting cyl[1]:
Enter partition size[62878410b, 3914c, 3914e, 30702.35mb, 29.98gb]: 3914e
partition> label
Ready to label disk, continue? y
    4. partition> quit
    5. format> quit
  8. zpool attach -f rpool c7t0d0s0 c7t1d0s0
    • mirror the rpool disk
  9. installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c7t1d0s0
    • install grub on the new disk
  10. zpool status
    • wait for reslivering to be complete
  11. at this point I would shutdown, snapshot your original disk
  12. modify the guest config so the new disk was set as bootable
    • (e.g. xm list -l <guest> > /tmp/myguest.sxp;vim /tmp/myguest;xm new -F /tmp/myguest
  13. reboot to test the new rpool
  14. zpool detach rpool c7t0d0s0
    • then remove the old disk from the mirror
  15. Not sure if you need to reboot to see the larger size or not?
  16. zpool status

legacy: xm configuration file

the configuration file, or the .py file is used for creation and for running a domU.

Conceptual creation of domU

  1. Write the .py file for creation
  2. run xm create -f .py
  3. Write the .py file for the created domU
    • This .py file is different from the Creation .py file. Different boot drive etc.
  4. xm new .py
  5. xm start <DOMAIN>

Content

See also: http://opensolaris.org/os/community/xen/docs/virtinstall/

  • __: —
  • __: —
  • __: —
  • __: —
  • boot: —
  • device_model: —
  • disk: —
  • extra: — ?
  • import: —
  • name: — Name of the domain.
    • name = "solaris"
  • memory: — Memory to allocate for the domain.
    • memory = "512"
    • Is there a minimum? I seem to recall reading about 512 as a minimum
    • Is the memory allocated from the memory available to dom0?
    • Is the block locked in memory?
  • on_crash: —
    • on_crash = "destroy"
  • on_reboot: —
    • on_reboot = "restart"
  • on_shutdown: —
    • on_shutdown = "destroy"
  • on_xend_start: — default; invalid
    • on_xend_start = "start"
  • on_xend_stop: — default; invalid
    • on_xend_stop = "shutdown"
  • root: —
  • serial: — enable the ttya serial port in your domU. When your domain is created, it will pause before booting and wait for a connection to this port. See also: http://opensolaris.org/os/community/xen/docs/solhvmguest/
    • serial="telnet:0.0.0.0:port,server"
    • where port is an available, unprivileged (anything higher than 1024) TCP port to use on dom0.
    • You can configure it to boot without waiting by adding ",nowait" to the end of the serial config line
  • usb: —
  • usbdevice: —
  • vcpus: — Number of virtual CPU's to provide. default; 1
    • vcpus = 2
  • vif: —
  • vnc: —
  • vnclisten: —
  • vncpassword: —

Cloning an image

virt-clone

Please note

This doesn't seem to work on snv_117, at least not on a UFS formatted HDD.

It fails with: ERROR: list index out of range

  1. virt-clone -d --original=sol10u7 --name=solclone71 --file=/common/xvm/solclone71.xvm

Troubleshooting

OS Installation issues

OS report installation failure, though no errors found in install log files.

AnalysisPhase

CustomerDescription

report installation failure, though no errors found in install log files.

While installing the OS on the box.

ProblemStatement
ListingFacts

IdentifyInformationSources

FirstObservedBy
LocationOfFault

On a HP 8510w, firmware ???, Developer installation.
snv_78

FirstObservedDate

OtherResources

CollectRelevantErrorMessages

Diagnosis

See what happens at reboot

reboot the box, it came up working, without any problems.

xVM creation issues

Fault 2: 'vncpasswd is not set up in VMconfig and xend-config.'

AnalysisPhase

CustomerDescription
# svcs -x
svc:/system/xvm/domains:default (Hypervisor domain autoboot and graceful shutdown)
 State: maintenance since Tue Jan 15 14:15:04 2008
Reason: Start method failed repeatedly, last exited with status 1.
   See: http://sun.com/msg/SMF-8000-KS
   See: xVM(5)
   See: /var/svc/log/system-xvm-domains:default.log
Impact: This service is not running.
/var/svc/log/system-xvm-domains:default.log

[ Jan 15 14:15:04 Executing start method ("/lib/svc/method/xvm-domains start"). ]
Traceback (most recent call last):
File "/lib/svc/method/xvm-domains", line 54, in ?
server.xend.domain.autostart()
File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in _call_
return self._send(self._name, args)
File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 152, in __request
File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
verbose=self.__verbose
File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 118, in request
File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
return u.close()
File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 2: 'vncpasswd is not set up in VMconfig and xend-config.'>

ProblemStatement
ListingFacts
FirstObservedDate

OtherResources

CollectRelevantErrorMessages

See: /var/svc/log/system-xvm-domains:default.log

Diagnosis

From: http://bugs.opensolaris.org/view_bug.do;jsessionid=4efdcbd88d1f4103e271abb41ba?bug_id=6567064

See: 'Setting VNC password' above

svcadm clear domains

% wait 20 seconds

svcs -x
should return empty list.

No data returned

Could not find an installable distribution

For more information see: /root/.virtinst/virt-install.log

Starting install...
Could not find an installable distribution at '/common/packages/sol-10-u6-ga1-x86-dvd.iso'
Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start iss'; otherwise, please
 restart your installation.
Fri, 13 Feb 2009 16:49:02 ERROR    
    Could not find an installable distribution at 
       '/common/packages/sol-10-u6-ga1-x86-dvd.iso'
Traceback (most recent call last):
  File "/usr/bin/virt-install", line 511, in ?
    main()
  File "/usr/bin/virt-install", line 471, in main
    dom = guest.start_install(conscb,progresscb)
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/Guest.py", 
    line 882, in start_install
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/ParaVirtGuest.py", 
    line 83, in _prepare_install
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", 
    line 233, in prepare
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", 
    line 194, in _prepare_kernel_and_initrd
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", 
    line 106, in acquireKernel
  File "/export/builds/xvm_107///proto/install/usr/lib/python2.4/site-packages/virtinst/DistroManager.py", 
    line 92, in _storeForDistro
ValueError: Could not find an installable distribution at '/common/packages/sol-10-u6-ga1-x86-dvd.iso'

vncviewer: unable to open display ""

# vncviewer localhost:5900

VNC Viewer Free Edition 4.1.3 for X - built Jan 17 2009 00:04:40
Copyright (C) 2002-2008 RealVNC Ltd.
See http://www.realvnc.com for information on VNC.
vncviewer: unable to open display ""
  1. Try running vncviewer without any parms, and see if that fails.
    • In this case the DISPLAY env var was missing
      • setenv DISPLAY :0
      • export DISPLAY=:0

Labels

xvm xvm Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

Copyright 1994-2009 Sun Microsystems, Inc.
Powered by Atlassian Confluence Sun Guidelines on Public Discourse Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact