Final Report for the LCFG Server Refactoring Project (#78)

This was an ambitious project with the target of refactoring all the code in the LCFG Server to improve maintainability and to provide the option of adding feature enhancements in the future. It has taken 2.5 years from the start of the implementation to achieve only a subset of the original goals but the main cause of this is not overambitious targets so much as a lack of human resources. This project was stalled over a number of lengthy periods to allow the completion of higher priority work (e.g. F13 and SL6 ports). The original intention was also that two developers would work on the code but in reality there was only ever one person involved although all code changes were reviewed by a second person. A total of 374 hours effort were spent on this project, which is just under 11 weeks. This may sound like a great deal but it was spread over 31 months and as we are utterly dependent on the LCFG server code to manage our systems it is undoubtedly effort well spent. There is no doubt that the long interruptions resulted in extra time being spent on getting up to speed each time the project was restarted.

Although it has not received as much effort as originally planned the project has still achieved a good result in terms of upgrading the LCFG server code into "Modern Perl". It is particularly reassuring to know that all of the code now runs with the "strict" and "warnings" pragmas enabled and that the vast majority of uses of uninitialised values have been eradicated, this should avoid unpredictable behaviour from the server. Although this was primarily just a refactoring of the code a few features were substantially altered to fix bugs. In particular, a bug fix to the core perl meant that it became essential to rework the way the profiles were processed in a Safe environment. The improved readability of the code made this work much easier than it would have been otherwise. A large chunk of effort went into the development of the test system, based on the generated XML profiles. This will be hugely beneficial when it comes to making any further bug fixes or adding new features.

Throughout the development process all code changes were reviewed. I feel that this greatly improved the quality of the code and avoided a number of obscure bugs being introduced. Typically when a single developer is working on a project the code ends up being written in their own peculiar style. Also sometimes a lack of knowledge of certain features in a language can lead to unusual workarounds. Having a second pair of eyes really did help make the code much more readable. I felt that as I was having to write for an audience I put in a lot more effort to make the code really clear and understandable. There is no doubt that the code review system did lead to a number of rather frustrating delays when no-one was available to do the reviews. I would definitely recommend that if this approach is used again that there are more reviewers to cover holidays and busy periods. This would also be an excellent way for others to gain knowledge of the code for a project whilst not being the lead developer.

At the beginning of the project we agreed on a house style for how the code would be laid out. We also agreed on a number of standard ways to handle certain scenarios. The Perl community makes a lot of the "There Is More Than One Way To Do It" idea but this means that if you want consistent code then it is best if everyone working on a project agrees on which way. This got all the potentially heated arguments about whitespace indentation out of the way very early and left more room for the constructive criticism of the code itself. We used perltidy to restructure the code and used perlcritic to achieve a consistent coding style. These are both excellent tools which we should aim to use more widely in projects written in Perl.

Due to the various resource constraints we had to drop a number of our original goals. These are still worthwhile activities and should be revisited in the future if we want to continue using LCFG in the long-term. In particular, the reworking of the database interfaces and the compiler itself to provide object-orientated APIs would make extension projects much easier.

-- StephenQuinney - 23 Apr 2012