OpenLDAP 2.4 Upgrade - Final Report

This is the final report for Project 181 - OpenLDAP 2.4 Upgrade.


1. Introduction

The aim of this project was to migrate our LDAP servers from OpenLDAP 2.3 (2.3.43 on the slaves, 2.3.38 on the master) to a recent release of the current 2.4.x series (in fact, 2.4.31). Initially, the plan was to do this before the upgrade from SL5 to SL6_64, but it was then decided to do both upgrades in one fell swoop.

You would expect that all of this should be a standard enough procedure. The fact that this work was organized as a project was partly to do with the fact that OpenLDAP has a history here, and that its LCFG configuration is rather arcane; but mostly to with the fact that the person to whom the project was assigned had no prior experience with LDAP or our implementation of it. In that respect, the project had the parallel aim of knowledge transfer.

2. Deliverables

  1. All LDAP servers running OpenLDAP 2.4.31 on SL6_64, at the current version of SL6 (namely SL6.3.)
  2. The completed devproj project page for Project 181.
  3. This final report.

3. Observations

This project has been running since mid-2011. Apart from general orientation, much of the actual work involved in the testing phase of this project involved setting up various test LDAP servers and confirming that the LDAP replication we use worked correctly across OS versions, and across versions of OpenLDAP. It did: see the progress to date wiki page for results.

Time was also spent in learning how to use both VirtualBox, and the KVM service: both of those were used to implement test machines at various stages of the work.

Separate work was done by the CO responsible for the LDAP service (Toby Blake) in testing the prometheus->write to LDAP master interface; the majority of writes to our LDAP directory now come from the prometheus service

During the entire project, and most crucially during the final upgrades to the live servers, Toby Blake also provided invaluable advice and direct help. Thank you.

As an aid for future OpenLDAP upgrades, the plan for the final upgrade of the master LDAP server is recorded verbatim here:

- email to cos
- disable ldap parts of prometheus loop
- stop all (apart from one or two) machines talking to franklin - very
  important that nothing filters out to syncrepl or standard dice
  clients, so set OPENLDAP_ALLOW in profile
- om openldap save
- check ldif file generated by save can be loaded onto another machine
  and check that the ldap is the same (use ldap-diff)
- om openldap stop
- take paranoid copy of data dir, log dir
- strip profile of most stuff so it's a standardish dice machine
- om client stop
- set:
  !openldap.type mSET(client)
  !openldap.server mSET(dir)
- change openldap version in franklin's profile
- upgrade to SL6 (can preserve most partitions)...
- update profile with openldap-master header and all the other stuff
- om openldap start -- -f -l /path/to/ldif
- check that it looks OK
- enable and check one syncrepl slave - (the slave will first need to run
  'om openldap restart' in order to get updated Kerberos keys)
- enable and check one slaprepl client
- check prometheus can write to it
- re-enable prometheus loop
- re-enable all slaprepl clients
- ldapBuildAmdMaps will no longer work, but can (presumably) be run
  from SL5
- fix headers...

4. Suggestions for improvement and/or future work

  1. The header files which configure our OpenLDAP service are confusing, over-complicated, and opaque. That's a situation which has developed historically, but it's by now a definite hindrance to the understanding and maintenance of this service. The header files should be simplified completely rewritten to improve this situation. That's easy to say, of course: any such work will have to proceed very carefully ...

    Note that the header files do work as they are. Modularly removing and inserting the various header files in machine profiles does achieve the expected result. That might seem an obvious remark, but it's handy to know, particularly at the time of a live master server upgrade, where the machine involved needs to have its role changed at various points in the process.

  2. Unless there is a compelling reason, we should never again allow our OpenLDAP installations to drift so far from the currently-recommended versions of the s/w; running an old version only complicates the upgrade job when that does finallly need to be tackled. The fact that this happened here before was due to the local history of OpenLDAP, and the fact that the 'old' version of the OpenLDAP code we were running was trusted, and known to be stable.

    The recommendation is that we should routinely track a version or two below the current active release, and implement a regular upgrade (and test) cycle.

  3. At some future release, OpenLDAP will drop support for configuration via slapd.conf and move instead to so-called 'on-line configuration (OLC)' (a.k.a. cn=config configuration.) We should anticipate that, and make sure that our header files and component are ready for the change.

    See Project 266: OpenLDAP: investigate slapd-config

  4. Likewise, at some future release, the mdb database format will become the officially-recommended and supported one. We should also anticipate that.

    See: Project 265: OpenLDAP: investigate mdb backend

  5. The local script ldapBuildAmdMaps still needs to be ported to SL6. That is the responsibility of the Services Unit.

5. Effort

This project was initially estimated at 1 to 2 weeks of CO time.

In the end, I have booked 18 days against this work; Toby Blake has booked 46.75 hours (= 6.2 days).

Total effort: 24 days.

-- IanDurkacz - 29 Nov 2012

Topic revision: r2 - 04 Dec 2012 - 16:49:53 - IanDurkacz
DICE.FinalProjectReport-181 moved from DICE.FinalProjectReport181 on 04 Dec 2012 - 16:49 by IanDurkacz - put it back
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies