Documentation on managing KVM guests

Configuring a machine to manage KVM guests

Include the following in the machine's profile :-

#include <dice/options/kvm-client.h>

Creating KVM guests

There are a number of ways to create KVM guests - virt-manager (GUI based), virt-install (command line) or a combination of virt-clone and virsh. The problem with these tools is that there is no easy way to standardise on a few templated configurations, and the default values provided by KVM install several devices which we don't need for DICE servers (and which increase CPU utilisation). Please do NOT use these tools for creating guests on the MPU managed KVM servers.

KVM guests should be created using the local kvmtool, which was written to standardise on a few VM configurations and to simplify KVM guest creation (particularly from the command line).

kvmtool create --name myvm [--host remoteserver] [--uri uri] [--flavour template]
               [--cpus number] [--memory number] [--bridge interface-name]
               [--mac mac-address] [--pool storagepoolname] [--disksize gigabytes]
               [--diskallocation number] [--bootorder option[,option..]]

This can be used to create a guest VM on the localhost or on a remote KVM hosting server. By default it will generate and display a mac-address. See first FAQ item below if this doesn't work.

Default values are :- --host=localhost, --flavour=sl7, --cpus=1, --memory=4096 (MB), --bridge=br0 (same wire as VM host), --disksize=20 (GB) These are sufficient for DICE servers with the <dice/options/small-server.h> header.

The current "flavours" are sl7, sl6_64 and sl6. The default is sl7.

Once the guest has been created, add it to live/console_server.h to get console logging via conserver and the console servers. There are three "KVM" sections in live/console_server.h, one for each site; just find the right section add add your new guest's name to an unused line there.

Please also add it to the wiki page of the server you chose. They're all linked from SimpleKVMDocs.

Managing KVM guests

You should be able to perform most guest management tasks using the local kvmtool command. There are also two native tools for managing KVM guests - one command line (virsh) and one GUI (virt-manager). There is also a local wrapper (rvirsh) to virsh which simplifies connections to KVM hosts.

You can get our normal console logging and access via conserver by registering your KVM guest with conserver as described above.

Deleting KVM guests

Use kvmtool delete to delete a guest.

kvmtool delete --name myvm [--host remoteserver] [--uri uri] [--purge]
By default, the guest's disk(s) will not be removed. If you want the disk(s) removed, add the --purge flag.

If you mess up the creation/install of a new guest, it can be simplest and quickest to delete the guest then make a fresh one.

If you have added your guest to live/console_server.h as described above then remember to remove it again when you delete the guest.

Please also remove your VM from its KVM server's wiki page. They're all linked from SimpleKVMDocs.


The "kvmtool" command appears to do nothing when trying to create a guest (does not return a MAC for example), you can't connect with kvmtool or virt-manager or rvirsh or virsh
it's possible that libvirtd has died on the KVM server. Here's how to restart it.

How do I specify a different network for my guest
Guests will default to the subnet of the KVM host. Use the --bridge option (eg --bridge=br33) to specify a different network.

How do I add a second network interface to my guest
to be written Probably easiest way is to use virt-manager

How do I create an extra disk for my guest?
If you need an extra disk for a guest then firstly check with MPU to ensure you are using the most suitable volume group. Do something like the following: virsh vol-create-as <pool> <volname> <disksize> --format raw (For example, virsh vol-create-as circlepool1 circlevm6_extra 2g --format raw)

How do I add a second disk to my guest?
This should be done whilst your guest is running: virsh attach-disk --domain <guest> --source /dev/<pool>/<volname>  --target <disk> --persistent --cache none Once that is done it should be immediately accessible as /dev/<disk> in your guest. You can then partition it with parted and format it as normal. You need the cache to be disabled - use virsh edit <guest> or virtmanager to check this. The disk "driver" line should include cache='none' (For example virsh attach-disk --domain circlevm6 --source /dev/cp1/circlevm6_extra --target vdb --persistent --cache none)

How do I increase the memory of my guest?
Use kvmtool setmemory --name myvm --memory size [--host remoteserver] where size is the amount of memory in megabytes. Shut down the guest before doing this.

How do I access console logs
add the guest's name to the live/console_server.h header as described in Creating KVM guests above, then login to the site's console server and find the log in /var/consoles as usual.

How do I delete a virtual disk if I forgot to use kvmtool delete --purge
use virsh vol-delete <vol> --pool <pool>

How do I work out how much free space a KVM host has available
use /sbin/vgs on the KVM host to list free space in the host's storage pools.

Should I use live/circle_guests.h in my guest's profile?
No, there's no need, it was a temporary header.

Guest fails to boot after host reboot
After a host reboot a guest might fail to start with error: Unable to read from monitor: Connection reset by peer. Sometimes when a host has been rebooted to upgrade the kernel the guests which have been suspended will not resume. The solution is to delete the suspended session and do a cold boot. A session can be removed using the managedsave-remove virsh command. (For example rvirsh jubilee managedsave-remove heda)

ntp does not run on our VMs
This is deliberate. The virtual hardware clocks on our VMs are too poor to let us run ntp on them. Instead we regularly run a script /usr/sbin/detecttimeslew to correct the time if it's wandered.

Add an ISO (e.g. CD) image to a guest
kvmtool edit --name <guest>
then add the following snippet to the <devices> section of the guest's XML file, remembering to use your own source file address in place of the one in this example:
<disk type='file' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <source file='/var/lib/libvirt/isos/rhel-server-7.0-x86_64 dvd.iso'/>
     <target dev='hdc' bus='ide'/>
     <address type='drive' controller='0' bus='1' target='0' unit='0'/>

How do I change the number of virtual CPUs for a guest?
There are two settings to change: the number of virtual CPUs (or VCPUs) and the maximum number of virtual CPUs. Consider an existing VM called xyzzy on server blob. xyzzy has one virtual CPU, but needs to have two. Here's how to achieve that. Note that the VM will need to be shutdown to change the number of VCPUs:
  1. Find out how many VCPUs xyzzy has:
    $ rvirsh blob vcpucount xyzzy
    maximum      config         1
    maximum      live           1
    current      config         1
    current      live           1
  2. Shut down the VM:
    $ kvmtool --name xyzzy shutdown
  3. Verify that it has stopped running:
    $ kvmtool --name xyzzy info
    Found guest xyzzy on
    Guest is not currently running
  4. Increase its maximum number of VCPUs:
    $ rvirsh blob setvcpus xyzzy --count 2 --maximum --config
  5. Increase its number of VCPUs:
    $ rvirsh blob setvcpus xyzzy --count 2 --config
  6. Start xyzzy. It may fail to start on the first attempt, but a second attempt should succeed:
    $ kvmtool start --name xyzzy
    Found guest xyzzy on
    error: Failed to start domain xyzzy
    error: internal error: qemu didn't report thread id for vcpu '1'
    Failed to start virsh - at /usr/bin/kvmtool line 577.
    $ kvmtool start --name xyzzy
    Found guest xyzzy on
    Domain xyzzy started
  7. As a check, rvirsh blob vcpucount xyzzy should now show a higher number of VCPUs, and more /proc/cpuinfo in the VM should show details of each of them.
Topic revision: r41 - 27 Apr 2020 - 08:58:53 - ChrisCooke
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies