Packages - Sites Mirror
The package repository site mirrors are stored on local disk on the master packages server deneb (an MPU-managed server). There is currently one (virtual disk) partition,
/disk/data
. Before adding any new site mirrors you
MUST check that there is sufficient space on this partition.
The site mirrors do not necessarily contain a complete copy. For instance on EPEL we are not copying any ppc64 directories as we do not need them. On SL and epel we exclude those releases we don't use, leaving only the particular ones we need (e.g. sl72, sl73, epel7, epel6). It should also be noted that, in the directories we do mirror,
we do not delete files which disappear from the upstream site. This is in case we ever need a package (or source) which can no longer be found anywhere else.
We currently have these mirrors:
- CentOS virtualisation packages -
/disk/data/centosvirt
- Ceph distributed storage platform -
/disk/data/ceph
- EPEL (6 and 7) -
/disk/data/epel
- IS (packages from other schools) -
/disk/data/is
- PostgreSQL -
/disk/data/postgresql
- Scientific Linux (6 and 7) -
disk/data/sl
- we only mirror those minor releases which we currently need and support.
- Software Collections -
/disk/data/scl
Adding new mirrors
The mirroring is done by the
mirror_sites
script. This lives in the
scripts
subdirectory of the
pkglist-tools
package (in svn.lcfg.org). The configuration for
mirror_sites
- the mirror definitions - is in the
conf
subdirectory of
pkglist-tools
. The main configuration file is called
site_mirrors.ini
. The syntax of the configuration file is described in the help message of the
mirror_sites
script:
[deneb]cc: /usr/share/pkglist-tools/scripts/mirror_sites --help
Some of the current site definitions include other files containing extra snippets of configuration. These other files can also be found in the
conf
directory of
pkglist-tools
.
Building and installing the package
Once the changes have been made and committed, the package rebuilt with a new version number and the new version built and submitted to the package servers, the package version will need to be updated in the
lcfg/options/pkglist-tools.h
header. To get the package installed more quickly you can use
live/package-mirror.h
whilst waiting for it to filter through the weekly release process. However remember that the primary package server gets its packages from the DR server, so the new version will not be available to updaterpms until the DR server has run its own nightly mirror of the packages. (The DR mirror happens before the primary package server runs updaterpms so the new
pkglist-tools
should be installed that night.)
Running a mirror sync by hand
The mirror sync process is done every night via cron so if you are happy to wait then everything will happen automatically. If you want to do an initial mirror immediately then this can be done. First, remind yourself of the name of the site to be mirrored:
$ /usr/share/pkglist-tools/scripts/mirror_sites --list
If you have added a new mirror called
blob then
blob should appear in the output of that command. Then, to sync the
blob mirror - always as the
pkgmirror
user please! - type:
$ nsu pkgmirror
% /usr/share/pkglist-tools/scripts/mirror_sites --site blob