MPU Meeting 27th July 2012

Simple KVM Service

The server summary and detailed wiki pages for the KVM service have been fully updated.

Chris and Alastair have tested hot migration and it all worked fine. At KB the VMs are now spread across both northern (10) and piccadilly (13).

There were some problems with accessing ISO images on northern. This turned out to be related to the way in which libvirt scans through all the storage pools. On northern there were two LVM pools (np1 and np2) but only one was being used the other was empty (i.e. contained no volumes). As there were no volumes there were no entries in the /dev/ directory. We wondered if this is really a udev bug? libvirt takes quite a while to timeout when it cannot find any volumes in a pool, ideally libvirt would just ignore empty pools entirely. It didn't help that it returned a spurious error message suggesting that the ISO was already being used exclusively by another VM. To solve the problem a dummy volume was created in the empty pool. The documentation for the procedure for creating new storage pools has now been updated to add this step so it doesn't happen again. We probably ought to submit a Redhat bug report, probably needs checking with EL6.3 first to see if it still exists in newer versions of the software.

The AT1 VLAN is available in the Forum. We should add it to jubilee and bakerloo to allow migration of VMs from the KVM server in AT.

Alastair is working on updating the script which mails out a daily usage summary for each KVM server. This will show usage levels for CPU, memory and storage and should help us avoid having any resources heavily overcommitted.

SL6 Upgrades

Nothing happened.

Server Hardware

Chris has added support for finding the various version information for fibre channel cards. This returns 7 different strings, need to think about how to store this data.

Alastair and Chris will meet to discuss storing all the BIOS and firmware information in an SQL database.

Security Enhancements

Stephen now has some scripts which can parse the centrally stored rsyslog files. He is now working on filtering the entries to find events of interest and then storing them into a database.

Miscellaneous Developments

Not much happened.


  • toohot sensor numbers - Stephen noted that it would be nice if the various TOOHOT sensor numbers could be stored in the lcfg-level headers instead of the dice-level so we can share this information more widely.

  • podium machines - Nearly all the podium machines have now been upgrade to SL6. The last two in 4.31/4.33 will be done next week.

  • KVM and release testing - The release testing is now being done using KVM guests. If nothing else, this should make it much easier to cover for Richard when he is away.

  • mptctl kernel module - Not all machines have the mptctl kernel module loaded. Stephen will look into getting it loaded automatically whenever the main mpt modules are loaded.

This Week

  • Alastair
    • Add at-1 wire to jubilee and bakerloo and test
    • Chris and Alastair to look at how can store BIOS revisions in orders database
    • Mods to kvm mail - group by pool, warn < %
    • kvmtool - add log of who created kvm
    • Work through LCFG bugs

  • Chris
    • Server hardware project
    • Chris and Alastair to look at how can store BIOS revisions in orders database
    • Migrate some domains off circle as necessary or ask other units to
    • Look after R720 backplane fix if necessary
    • Install new R210 to replace split
    • Check for excess toohot in hw headers

  • Stephen
    • System security project
    • Sort out subversion component problem for Graham
    • Sort out mptctl module loading
    • Upgrade telford to SL6
    • Reboot hogwood for SL6.2 kernel
    • Process other units' responses about their perl-AFS module usage (which functions etc)
    • Speak to Graham about Theon work
    • Look at removing smartd applet from gnome-disk-utility (palimpsest) so can use gnome-disk-utility's disk encryption functionality

-- AlastairScobie - 27 Jun 2012

Edit | Attach | Print version | History: r10 < r9 < r8 < r7 < r6 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r9 - 10 Jul 2012 - 15:17:36 - StephenQuinney
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies