Informatics Mirror Service - End User Docs

The following describes the little that you need to do, to setup a mirror server, and a mirror client. The technical details are in another document MirrorServiceDetails.

Set up a mirror server

To configure a machine as a mirror server it should be located at one of our 3 sites (JCMB, Forum, Appleton Tower) and have disk space on which to store the mirrored data. You only need to add one of the approriate header files to your machines profile, pick one from:

#include <dice/options/rmirror-server-jcmb.h>
#include <dice/options/rmirror-server-forum.h>
#include <dice/options/rmirror-server-at.h>

Notice the header is "rmirror..." as the component that does the mirroring is the "rmirror" component. These headers set certain values eg safety limits at which point to not mirror, eg if more than 70% of the files would be deleted. If you want to override these, see the man page for lcfg-rmirror.

Also, the time that the mirrors run is defined via a crontab entry in the included files, by default it's is once a day, at a time between 11pm and 12am. How long the mirror's take to run, depends on the amount of data to mirror.

Next you then need to configure it to say which clients you want to mirror. The simplest way is to use one of the following macros:

MIRROR_CLIENT(HOST,FQN,MODULE,DEST)
MIRROR_INF_CLIENT(HOST,MODULE,DEST)

Where:

HOST

is actually just a short lcfg tag, but usually the short name of the host you want to mirror, eg jings, and must be if you want to use the latter - MIRROR_INF_CLIENT - version, see note at the end of this list

FQN

the fully qualified name of the host you want to mirror, eg jings.inf.ed.ac.uk

MODULE

the name of the rsync module on the FQN you want to mirror from, this needs to match what was configured on the host you are mirroring.

DEST

A path in the file system to where you want to store the mirror, eg /disk/rmirror1/jings

The two macros are nearly identical, except in the latter the FQN is assumed to be HOST.inf.ed.ac.uk.

That's it for the setup.

Managing the mirror server

There's only one thing you need to keep an eye one - whether the mirrors are running successfully. There are various ways you can do this:

1. Look at the rmirror component log on your mirror server, either via the lcfg web interface or /var/lcfg/log/rmirror
2. Run the om rmirror report method
3. Look at the web page which reports on all the mirror servers http://groups.inf.ed.ac.uk/cos/mirrors/

The last two items should give a summary of the state of the mirrors. If there are problems, looking in the actual mirror logs should say what the problem was.

Note that while the mirrors are still in a state of migration between the old mirrors system and the new mirrors, you may say lots of warnings along the lines of:

WARN mook::mook on lammasu.inf.ed.ac.uk Up-to-date mirror, but tape status wrong

This is likely to be that (in this case) mook isn't using the new rmirrorclient macros (see below), which means it isn't explicitly saying if the mirror should be going to tape or not. So it's indeterminate as to wether the mook mirror should be on a partition that is backed up to tape or not. But the mirror itself is up-to-date.

"Up-to-date" or "Mirrored recently" means a successful mirror within the last 24 hours.

The other warning you may get is:

WARN curlew::curlewnagioslog on lammasu.inf.ed.ac.uk Once mirrored (10 days ago), no longer configured, old results entry?

This can happen if a mirror changes name, or is removed from the mirrors. If you know this is the case, then you should remove the offending line from the rmirror status file on the rmirror server, ie /var/lcfg/conf/rmirror/results . Arguably there should be a more user friendly way of doing this, but there isn't - yet.

If the mirrors report a failure, then look in the actual rmirror logs. If it is a fail along the lines of the following, eg:

mirrors FAIL stoater::stoater on client not configured to be mirrored 

Then it means that the machine stoater is requesting that it's rsync module called stoater is to be mirrored, but no server is actually configured to mirror it. The solution would be to add something like:

MIRROR_INF_CLIENT(stoater,stoater,/disk/mirror1/stoater)

to a mirror server based at JCMB.

Setup A Mirror Client

To configure a machine to be mirrored by a mirror server, then include one of the appropriate headers according to the site at which the client it based:

#include <dice/options/rmirrorclient-jcmb.h> /* client in JCMB */
#include <dice/options/rmirrorclient-forum.h> /* client in Forum */
#include <dice/options/rmirrorclient-at.h> /* client in Appleton Tower */

If you are adding this to an existing machine and are not planning on rebooting. Then once the profile has published, you'll need to run

om updaterpms run
om rsync start

The rmirrorclient header files really just configure the rsync component. Though there is an actual rmirrorclient component, it is only used as a place to publish spanning map information and host a "report" method. The reason you may care, is if your machine was already using rsync, then you have to be aware that the macros below will also be configuring the existing rsync.

There are "TAPE" and "NO_TAPE" versions of the macros, this is so you can specify if the mirror should also be backed up to tape or not. Note that the backing up to tape part is nothing to do with the mirror service, just that the mirror server checks (via tibs) if the destination path for the mirror is configured to be backed up to tape.

MIRROR_THIS_AND_TAPE(TAG,SRC)
MIRROR_THIS_NO_TAPE(TAG,SRC)

The above macros create an rsync module called TAG pointing at the file system path SRC. TAG needs to be unique across all rsync targets on the client. Only the offsite mirror machines will be given rsync read access to the rsync module.

MIRROR_THIS_TAG_AND_TAPE(TAG)
MIRROR_THIS_TAG_NO_TAPE(TAG)

These versions of the macros are for the case where you've already configured an rsync module called TAG, and you just want it to be mirrored. Note that you'll need to make sure the mirror servers have permission to mirror from your manually created rsync module. In which case you may find either the macro _RMIRROR_SERVERS useful, or the value of the resource rmirrorclient.mirrorserversfqn

Aliases If you'd like your machine rmirrored via a DNS alias, rather than the host name, eg prometheus.inf.ed.ac.uk, rather than vandellas. Then in vandellas' profile you'd need to add:

rmirrorclient.aliases       prometheus

as it's assumed to be a .inf.ed.ac.uk alias. If it isn't, then give the FQN, eg rmirrorclient.aliases www.lcfg.org. The resource is a list, so you can add multiple alaises if you wish.

One final thing you may want to override, is the rmirrorclient.group resource. It only affects the machine readable report output by the rmirror servers, and the output of the mirrors summary web page. And can be used (by you) to filter out the mirrors you are interested in. The default is:

!rmirrorclient.group   mSET(mirrors)

Checking your client mirror

To monitor the state of your client's mirror you could look for it's entry in the mirror report at http://groups.inf.ed.ac.uk/cos/mirrors/. Or on the client you can run om rmirrorclient report and it will output something like the following:

neilb> om rmirrorclient report
SUCCESS stumer::ptn178 on lammasu.inf.ed.ac.uk Mirrored recently ALIAS adminsmb
SUCCESS stumer::adminsamba on lammasu.inf.ed.ac.uk Mirrored recently ALIAS adminsmb
SUCCESS stumer::ptn143 on lammasu.inf.ed.ac.uk Mirrored recently ALIAS adminsmb
[OK] rmirrorclient: report

Note that it can take a moment or two for the output to appear.

You can also have the results of client rmirror mailed to by setting the resource rmirrorclient.email eg:

!rmirrorclient.email    mSET(neilb)

it should be a space separated list of valid email addresses. The mails are sent after the server has run all its mirrors. It usually only emails warnings and errors, but you can change the threshold via the:

!rmirrorclient.emaillevel    mSET(all)

resources. I can take 1 of 3 values:

all

See all messages: successes, warnings and failures

warn

See warnings and failures.

error

Only see error/failure messages.

These emails will not report if your client isn't even configured to be mirrored, or of the mirror service is not running at all. The best way to check for those, is to run the om rmirrorclient report method above.

-- NeilBrown - 01 Nov 2011