Final Report for Submit project

This is the final report for project number 67, Review/Update Submit Practical Submission System.

The project came with these aims:

  • If cost effective, rewrite the software in Python or Perl (instead of C).
  • Convert the back end to use AFS or version control rather than NFS.
  • Make changes to support EUCLID-driven database changes.
  • Improve the error return status on zero length submission due to filespace full.
  • Freeze submissions at the deadline point, or maintain a version history.
  • Drop years from all aspects of submit.
  • Reconsider how to upload data to the submission system.
  • Do not rewrite the software.
  • Make "staff use" documentation more prominent.
  • Check that set_submit_file_names works.

The existing code dates back to at least 1997 and is itself a rewrite of a venerable predecessor. Over the years it has had many modifications as bugs have been fixed, features have been added or removed and operating systems have come and gone.

Much of the code is devoted to the detailed manipulation of memory and files for data storage; this would be unnecessary in a higher level language such as Perl.

The code is sufficiently intricate as to make relatively large changes (such as a change to the back end) prohibitively complex. If such changes were required, a total rewrite in a more modern language would be far more cost-effective.

The project faced the choice either of a complete reimplementation, with new desired features, or of keeping the existing code and making the minimum necessary changes to it. Since the teaching committee had already ruled out a total rewrite, and since the minimum necessary changes looked feasible to make, I chose to work on the existing code.

The following aims looked too ambitious for the current code and were therefore not delivered. If they are needed, I recommend a complete reimplementation:

  • Convert the back end to use AFS or version control rather than NFS.
  • Freeze submissions at the deadline point, or maintain a version history. (This could possibly be achieved by for instance moving submission directories around with carefully timed cron jobs?)
  • Reconsider how to upload data to the submission system.

These aims were delivered:

  • Make changes to support EUCLID-driven database changes.
  • Improve the error return status on zero length submission due to filespace full.
  • Drop years from all aspects of submit.
  • Make "staff use" documentation more prominent.
  • Check that set_submit_file_names works.

In more detail, the following changes were made:

  • To adapt the software to data and use changes:
    • The "year" field was completely removed from all aspects of the submit system.
    • "Modules" were renamed "courses".
    • The "handin" command was removed.
  • To aid maintainability and robustness:
    • Obsolete sections of code were removed.
    • Function prototypes were added and the C syntax generally updated where necessary.
    • Obsolete command options were removed.
    • Compiler warnings were largely eliminated.
    • Most comments of purely historical interest were removed.
    • The code is now more liberally and clearly commented.
    • The uid- and gid-changing code was modernised and rationalised. This part of the program is now more robust and dependable than before.
    • A test suite was added. This now defines the correct behaviour of "submit".
    • Run-time requirements were added to the RPM specfile.
  • To fix a bug:
    • The program now does a more thorough check of the exit status of its functions, and reflects this in its own exit status.
  • To improve the documentation:
    • The documentation was brought up to date.
    • Errors were corrected.
    • The docs now clearly refer to each other.

The redeveloped submit software, and associated scripts, are now installed on all relevant DICE machines.

The complete source can be found in the dice-submit directory of the LCFG Subversion repository.

The project used four weeks of effort in spring 2011 and four days of effort in autumn 2011.

-- ChrisCooke - 31 Oct 2011

Topic revision: r2 - 31 Oct 2011 - 12:16:13 - ChrisCooke
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies