Building Packages using mock

The recommended approach for building RPM packages for DICE SL7 machines is to use PkgForge. This guarantees that the packages are rebuildable in the future and that the dependencies are reasonably well understood. The downside of using PkgForge is that when a package fails to build, potentially several times before success is achieved, the process can become rather slow. PkgForge is basically a service which wraps the mock package build tool and it is possible to speed-up the whole build/test process by using mock directly.

Basic Config

If the profile for your machine includes the dice/options/co-desktop.h header then you should already have everything necessary installed and configured. If not then mock can be added to any DICE machine by including the dice/options/mock.h LCFG header and tweaking a couple of resources:

#ifdef LINUX_REDHAT
#include <dice/options/mock.h>
!auth.gr_mem_mock       mADD(<%sysinfo.allocated%>)
!fstab.mntopts_sda6     mSET(defaults)
#endif /* LINUX_REDHAT */

The way mock works means that it needs various root capabilities so it uses a PAM config (/etc/pam.d/mock) to authenticate user access. On DICE we require that users who wish to use the mock system are members of the local mock unix group, if the machine is not allocated to you then you will need to add youself to the group, for example:

!auth.gr_mem_mock       mADD(squinney)

The mock tool also needs to be able to use device and suid files on the partition where the build roots are stored, this is usually in /disk/scratch/mock (see the mock.basedir resource). That directory is likely to be on the sda6 partition (unless you have a VM or an nvme-based machine) and it will, by default, have the defaults,nosuid,nodev mount options. Changing the fstab.mntopts_sda6 resource to allow suid and device files is not sufficient to fix this issue, you will also need to edit the /var/lcfg/conf/fstab/fstab.sda file and then call the fstab component configure method:

$ chmod u+w /var/lcfg/conf/fstab/fstab.sda
$ emacs -nw /var/lcfg/conf/fstab/fstab.sda
$ om fstab configure
[INFO] fstab: /etc/fstab has changed - requesting reboot
[OK] fstab: configure

You will then need to remount the partition (or just reboot):

$ mount /disk/scratch -o remount

Building Packages

If you have an SRPM then building it using mock is very easy:

$ mock -r rootname example-1-2.src.rpm

Just as with rpmbuild you can --define stuff on the command-line and also use the --with and --without options to enable/disable package build options.

Generating an SRPM

If you need to generate the SRPM and you don't wish to install all the build-dependencies for your package then that can be done using rpmbuild like:

$ rpmbuild -bs --nodeps ~/rpmbuild/SPECS/example.spec

or using the LCFG build tools:

$ lcfg-reltool srpm 

(for dev builds you need devrpm --source)

Selecting the Relevant Build Root

The LCFG mock component manages the configuration for many build roots. These are the same as used by PkgForge which makes it easy to do test builds of your packages with reasonable confidence that the package will then build on the PkgForge service. The configuration files are stored in /etc/mock. The mock package ships with many configurations for Centos and Fedora, we also add the following on DICE SL7:

• el7-x86_64.cfg - SL7 base/security
• el7-lcfg-x86_64.cfg - SL7 base/security + epel + lcfg bucket
• el7-world-x86_64.cfg - SL7 base/security + epel + lcfg and world buckets
• el7-uoe-x86_64.cfg - SL7 base/security + epel + lcfg, world and uoe buckets
• el7-inf-x86_64.cfg - SL7 base/security + epel + lcfg, world, uoe and inf buckets
• el7-devel-x86_64.cfg - SL7 base/security + epel + lcfg, world, uoe, inf and devel buckets

These are all identical in terms of build environment except for which package repositories are available, via yum, for satisfying the build dependencies.

If you use bash-completion then it will auto-complete the chroot name when you press the <tab> key after the -r- command-line option.

Note that you should not specify the .cfg suffix, just the name of the chroot.

Build Products

The results of your build (both packages and logs) will be usually be in a sub-directory of /disk/scratch/mock/results/ which depends on the chroot used (for example /disk/scratch/mock/results/world/el7/ for the el7-world-x86_64 chroot).

$ ls /disk/scratch/mock/results/world/el7/
build.log
hw_info.log
installed_pkgs.log
postgres11-links-0.2-2.inf.noarch.rpm
postgres11-links-0.2-2.inf.src.rpm
postgres11-links-defenv-0.2-2.inf.noarch.rpm
postgres11-links-module-0.2-2.inf.noarch.rpm
repodata
root.log
state.log

If your build did not succeed then you will want to check the various .log files to find out what went wrong. If it succeeded then we still recommend that you submit the package via PkgForge, if that is unlikely to work for other reasons (e.g. the timeout for the build is not long enough) then you can use pkgsubmit:

cd /disk/scratch/mock/results/world/el7
pkgsubmit -B world -x example*.rpm

Using Locally Built Packages in Subsequent Builds

The packages are never automatically removed from the results directories. They are instead made available as a local yum repository for subsequent builds using the same chroot config. This means it is possible to build large projects with complex sets of dependencies and ensure that everything is working before final submission to PkgForge.

Building Python Packages

Some Python packages require extra macros packages to be installed in the chroot, these aren't always specified as build-dependencies (although arguably they should be). You can add extra packages to a chroot configuration via your LCFG profile like this:

MOCK_EXTRA_PACKAGE(el7_world,python-srpm-macros python2-rpm-macros python-rpm-macros python3-rpm-macros epel-rpm-macros openblas-srpm-macros)

That adds several extra macros packages to the chroot associated with the el7_world tag (el7-world-x86_64)

Excluding Packages from Build Root

Sometimes local builds of packages will be selected as build-dependencies when you want to build against the upstream version. Packages can be excluded in the usual yum style like this:

!mock.exclude_el7inf mADD(krb5-*)

In this case, an inf-level rebuild of the krb5 packages is excluded so that other inf-level packages can be built against the upstream version provided in SL7.

Selecting the Minor Platform Version

There is a macro, named MOCK_EL7_VERSION, which can be used to control which version of SL7 is used for the chroots. By default it is set to <%sysinfo.os_release_major%>.<%sysinfo.os_release_minor%> which will make mock use the same minor release as the host machine. Occasionally it is necessary to build against an older or newer release, that can be done like this:

#define MOCK_EL7_VERSION 7.7

Note that due to header ordering issues you will need to do this very early in the LCFG profile, probably best for it to go at the very start of the profile.

-- StephenQuinney - 07 Oct 2019