DICE Releases FAQ

What is a "release"?

Our LCFG header and package files are stored and edited in a Subversion repository.

Before releases, our LCFG servers took those files in the Subversion repository and used them to make an LCFG profile for every machine that needs one. If someone made a mistake while editing a header or package file, and checked this into Subversion, then the LCFG servers could quickly copy the mistake to every machine! This is why we came up with the concept of releases. Under this idea, the profiles for most machines are not any longer made from the header and package files in Subversion. Instead, they are made from a separate copy of the files. Nobody edits this copy. This separate copy is updated once a week to reflect the latest changes in the Subversion repository. The separate copy is a release.

There are actually two releases - the testing release and the stable release. The testing release is a copy, updated just once a week, of what is in Subversion. After it is updated it is then tested to make sure that it works. If it passes the tests, a copy of it is made. This second copy is called the stable release and it is what is used to make profiles for the vast majority of our machines.

Which machines use managed releases of DICE?

DICE Linux machines use managed releases by default. Other types of machine do not use managed releases.

How do I opt (a machine) out of Release Management?

Put this in its LCFG profile:

!profile.release   mSET(develop)

But please read the rest of this FAQ before deciding to do this. It should rarely be necessary.

What is the profile.release resource?

It sets the release level for your machine. These are the possible values:

stable
this release is made from a tried and tested bundle of header and package files. It changes about once per week. It is the default release for all DICE Linux machines.
testing
this release tests the next stable release that's in preparation.
develop
this release opts the machine out of release management. The machine gets its header and package files directly from the Subversion repository, as before. This may be useful for computing staff development machines.
default
this is a synonym of develop. The LCFG server's default release is called default. If a machine's profile fails to compile then the machine will revert to the default release.

What are the advantages of release management?

  • Stability - most machines' profiles (and software) change just once per week.
  • Problems stand a good chance of being discovered in the testing release before they hit most machines.
  • The LCFG servers are less busy! Most machines normally get a new profile only once per week. Header or package file changes only provoke a small rebuild of develop machines.

What are the disadvantages?

  • It can take longer to get header/package changes to most machines - up to nine or ten days in the worst case.

What releases are available?

There are three releases:
stable
this release is what DICE Linux machines use. It is normally updated about once a week.
testing
this release is used for testing a new release of DICE for a couple of days before it becomes stable. Computing staff not using develop are expected to use this release on their machines.
develop
the unmanaged release. This "release" just uses the headers and package files in lcfg/core in the subversion repository - all changes in lcfg/core affect a develop machine quickly. This release is suitable for machines used for testing new developments.

Are the defaults files under release control?

No. There is only one set of defaults files which get used for all the profiles in all the releases.

(Consequently, it is essential that care is taken when updating a schema for a particular platform to ensure it is backwards compatible with all the other platforms. If big changes are necessary then you should incremement the schema number, add a new defaults package to the list and then get the specific platform to use a different schema number.)

How do I specify options for a specific release?

Occasionally a CO needs to find out the specific release and target resource settings at that particular release. For example, if you want to avoid having some new experimental options affecting the stable machines before you are ready. There are three cpp macros available that will be defined, or not, based on the associated release:

  • LCFG_RELEASE_DEVELOP
  • LCFG_RELEASE_TESTING
  • LCFG_RELEASE_STABLE

So you can do something like:

#ifdef LCFG_RELEASE_DEVELOP
!profile.packages mADD(mynewpackage-which-is-not-quite-cooked-0.99.1_dev-1)
#endif

or

#ifdef LCFG_RELEASE_DEVELOP
!mypackage.resource mSET(foo)
#else
!mypackage.resource mSET(bar)
#endif

There is also a cpp macro called LCFG_RELEASE_VERSION which contains the version number of the current release. By default this is set to "unmanaged" to note that a machine is not using a testing or stable release. Typically for a stable or testing release it will look something like "2009082401".

Note that this only works in the headers and profiles, it will not work in the package lists.

How do I add the packages now rather than wait for the next release?

If you have updated a package list but you need to get the changes onto a specific machine immediately rather than wait for the next stable release you should edit the specific profile and make use of the LCFG mADD() mutate macro. For example:

!profile.packages mADD(foo-1-1)
!profile.packages mADD(bar-3-4)

The mADD() only adds the package to the list if it is not already present so should avoid the profile failing to compile when the next release (which contains your changes) is made. Note that you MUST remove these mutations after the release becomes stable as at some point there is the chance that someone else will update the packages again and at that point there will be a conflict.

How do I find out which particular release a machine is using?

There are two resources which you can query with qxprof:

profile.release
this will show one of stable, testing or develop
sysinfo.release_version
shows something like "develop/unmanaged" or "stable/2009082401"

When will my header/package changes be deployed?

They'll normally go into the testing release on the next Monday, and if all goes well that testing release will become the new stable release on the Wednesday.

How and why should I take a header file out of Release Management?

If you're always going to need the ability to make fast-deployed changes to a particular header file, consider moving it to lcfg/live. (SubversionRepository tells you how to check out lcfg/live.) This will remove it from release management altogether.

Examples of data where changes will probably always need to be deployed fairly quickly are

  • network configuration
  • vhost configuration
  • lists of authorised users for some facility or other (where the roles & capabilities system can't be used)

However, not all header files belong in lcfg/live! In particular, if you need to have a header change deployed quickly to many machines as a one-off you should keep the header in lcfg/core as normal, make your change there, and ask Managed Platforms to ship your change more quickly than usual. In general, software and profile.packages specifications should probably be in lcfg/core.

What if I need a change to be deployed quickly?

See also the previous question, How and why should I take a header file out of Release Management?

If you have a changed software package or header file which suddenly needs to get on to the stable machines as soon as possible (perhaps as a result of the discovery and fixing of a critical bug in urgently required software) then speak to someone in the ManagedPlatformUnit. They will often be able to safely rush your change into the testing and stable releases within a few hours.

Servers and the Develop Release

Please resist the temptation to put your server in the develop release if you can. This takes it out of release management, and this is a way of bypassing the delays involved in release management, and benefiting from instant change to header and package files. However you also get the big disadvantage which release management protects you from: mistakes committed into subversion can instantly break your machine! It's far better to separate out the things you'll need to change quickly into an lcfg/live header file, and otherwise keep the machine and its configuration in the normal stable release.

What Belongs in a Machine's Profile / in a Header?

Generally, things should only go in a machine's own profile if they're related to the hardware of that machine. Custom disk partitioning, MAC addresses and serial numbers are examples. For servers in particular, pretty much everything else would usually be better off in a header file, which would then be included into the profile with #include.

Where changes will need to be deployed quickly, consider using a header in lcfg/live. (See the previous questions.) For everything else, put your header in lcfg/core as normal.

Creating a special header file for a server that's performing a particular role makes it obvious which resources relate to the role and which resources relate to the individual machine. This should make it a lot easier to replace the machine with another, whether this is at the time of a planned upgrade or as a result of a hardware failure. To confer a particular role on a new machine, just include the relevant header file.

As an additional benefit, you can sometimes separate out the more general aspects of your header file into a header which can be used more widely. Perhaps someone has already created a useful header which you can use! For instance, when Tim set up our Subversion repositories server, he didn't just create dice/options/inf-subversion-server. Instead he created a general lcfg/options/subversion-server.h which any worldwide user of LCFG could use, and used this in dice/options/subversion-server.h which can be (and has been) used by others in Informatics who want to set up their own subversion server. He then used this in the dice/options/inf-subversion-server.h which he needed.

Where can I get more details?

The detailed procedures - perhaps mainly of interest to the release manager - are in ReleaseManagementProcedures.

If you want to talk to a human, see someone in Managed Platforms.

-- ChrisCooke, StephenQuinney

Topic revision: r17 - 08 Jan 2016 - 13:27:39 - ChrisCooke
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies