Solaris Patching

This page explains how our Suns get patched.

Summary

Patching is done on a roughly bimonthly cycle.

1. The MPU downloads the latest release of pca. This is used for all patch downloading and installation.
2. The MPU downloads the latest set of recommended and security patches needed by our Suns.
3. The MPU installs these patches on its test machine (currently cherub).
4. If satisfactory, the MPU then patches the Solaris login server salamander.
5. If satisfactory, the MPU puts a copy of the latest patches and of the version of pca used into /export on phoenix.
6. The nightly rmirror mechanism then copies pca and patches to /export on every Sun fileserver.
7. The Services Unit patches one machine per week until all of its Suns have been patched.
8. The cycle repeats.

These factors impact on the frequency with which we patch the Suns:

• The desire for security suggests frequent patching with the most up-to-date patches.
• The nature of the patching process - a drop to single user mode, followed by up to three hours of patching, rebooting and reconfiguring, before normal multi-user service can be resumed - encourages less frequent patching, both to minimise service downtime and to minimise staff hours being spent on patching.

We agreed that patching one fileserver per week was a compromise which seemed manageable to both of us. There are seven Services Unit Sun fileservers so this gives a roughly bimonthly patching cycle.

The Latest Release of pca

The latest release of pca can always be downloaded from the pca site. Check the news page to see if there are currently any known problems with the latest official release - it's not unknown for Sun to break its patch distribution mechanism, or change it without warning. When it does break, a new release of pca usually follows shortly afterwards, but at such times it can be advisable to try the development version of pca rather than the latest official release. Pca is just a shell script so no special installation is necessary; just download it to your pca working directory and chmod +x it.

The MPU will do this in each patch cycle.

Documentation for pca

There's also a man page available for download at http://www.par.univie.ac.at/solaris/pca/pca.8. The same information is available in web form at http://www.par.univie.ac.at/solaris/pca/usage.html.

Listing patches with pca

To list which patches are needed on this Sun: ../pca  --list  --askauth  --dltries=2  missingrs

To list which patches are needed on another Sun (for instance phoenix):

On phoenix:

• mkdir phoenix
• cd phoenix
• uname -a > uname.out
• showrev -p > showrev.out
• pkginfo -x > pkginfo.out

On your test Sun: ../pca  --list  --askauth  --dltries=2  -f ../phoenix  missingrs

For each Sun to be patched, make a directory with this information in it.

The MPU will do this in each patch cycle.

Downloading patches with pca

On your test Sun, cd to your pca working directory.

Make a directory to hold the patches. Let's call it patches.

../pca  --download  --askauth  --dltries=2  -P patches  missingrs

Then for each Sun, repeat the next command. The string ../phoenix represents the name of the directory holding the Sun's uname, showrev and pkginfo details:

../pca  --download  --askauth  --dltries=2  -P patches  -f ../phoenix  missingrs

Each patch will only be downloaded once.

The MPU will do this for each patch cycle.

Save the patchdiag.xref

Immediately after downloading the patches, copy /var/tmp/patchdiag.xref to your pca working directory.

The MPU will do this for each patch cycle.

Do the new patches require a reboot?

To find this out, do a "pretend" install. This gives you the output you'd see if you were doing a real install, but it doesn't install anything. It's useful for seeing patch messages requesting a reboot or a reconfigure reboot.

../pca  --pretend  -P patches  -X .  -y  missingrs

If a reboot or reconfiguration is necessary you'll be told, as in this extract from the output:

112837 -- < 13 RS-  25 SunOS 5.9: in.dhcpd patch & libresolv patch
                       Download 1/11: skipped - file exists
                       Install  1/11: pretended - reconfig recommended

112921 08 < 09 RS-  24 SunOS 5.9: libkadm5 Patch
                       Download 2/11: skipped - file exists
                       Install  2/11: pretended - reboot required

112923 03 < 04 RS-  24 SunOS 5.9: krb5 usr/lib patch
                       Download 3/11: skipped - file exists
                       Install  3/11: pretended - reboot recommended

The MPU will do this for each patch cycle.

Patch the test Sun

Copy your whole patch working directory to the root partition of your test Sun.

Drop the Sun to single user mode.

cd to your pca working directory.

../pca  --install  -P patches  -X .  -y  missingrs

When finished, boot the machine (or boot -r) if necessary then return to normal multi-user mode.

The MPU will do this for each patch cycle.

Test the patched Sun

Login

kinit

aklog

From the test Sun, can you access and use AFS directories as normal?

From other machines, can you access and use AFS directories hosted on the test Sun? There's a test AFS service on the test Sun cherub. To test it, first set up this bash alias on a DICE machine other than the test machine:

alias asu='pagsh -c "export KRB5CCNAME=$KRB5CCNAME.asu \\
          && kinit $USER/admin \\
          && aklog \\
          && PS1=[\\\\h]\\\\u/admin: PS2=[\\\\h]\\\\u/admin.. /bin/bash --norc \\
          && kdestroy"'

Patch salamander

If the test Sun appears OK after patching, the MPU will then patch salamander in the same way.

The same tests will then be performed on salamander.

The MPU will do this for each patch cycle.

Distribute the patches

Once salamander has been patched and has passed its tests, the MPU will copy the latest patches, patchdiag.xref and pca to /export/pca on phoenix.

Patching a Services Unit Sun

Finally, the bit that the Services Unit does.

Once a week, pick a Sun that hasn't been patched with the current set of patches. Do a pretend patch session on it to find out if rebooting or reconfiguring will be needed afterwards:

cd /export/mirrors/phoenix/export/pca (or on phoenix, cd /export/pca)

./pca  --pretend  -P patches  -X .  -y  missingrs

To do the actual patching: drop it to single user mode; become root; then do this:

./pca  --install  -P patches  -X .  -y  missingrs

Afterwards, reboot or boot -r as necessary.

-- ChrisCooke - 03 Sep 2007