An increasing number of our courses are using online programming exams (INF1-FP, INF-OOP, LP, FPS, and CP from 2011/12) These are run in exam-mode, which is more restricted than the usual DICE environment. Standard practise is for a TA or instructor to do a dry run of the exam prior to the actual exam. The dry run is important for checking the following things:

  • IDEs, editors, special libraries are working correctly
  • skeleton files for the exam match up correctly with the exam paper
  • exam instructions are correct
  • answer submission works as required

In addition, exam-specific files need to loaded into the exam-mode directory so that they are available for students taking the exam, and then removed in time for succeeding programming exams.

Since there are a fair number of things that could go wrong, there should ideally be at least 5 working days between the dry run and the actual exam. However, scheduling is becoming increasingly hard. For example, in the August 2011 resits, the following timetable has been set by Registry:

INF1-FP: Monday 15/08/11 14:30-16:30   
FPS: Tuesday 16/08/11 14:30-16:30   
INF1-OP: Wednesday 17/08/2011 14:30-16:30   
LP: Thursday 18/08/11 14:30-16:30 

This makes life hard both everyone involved. Since the current exam-mode environment is the main scheduling bottleneck, I'd like to propose that the environment be virtualized so that several instances of it could persist in parallel. This would make it much easier to schedule testing, and it would potentially allow a much longer lead time (as long as no system upgrades intervened).

-- EwanKlein - 12 Aug 2011

I presume that when we mention virtualisation here, it is meant in an abstract sense; in this case involving any of the school's virtual machine resources would likely be very costly (in terms of development and maintenance effort), impractical to administer and likely insecure. Note also that readying any single machine for more that one exam concurrently would be difficult and require a significant change to the lab exam architecture

That all said, I don't suppose either is necessary: amending the lab exam configuration to allow our more than one exam to be held (or tested) simultaneously would be comparatively straightforward. In practice all that we require is for a lab machine to be locked down to a specific exam, for each lab machine to be given its own distinct configuration, and for it to be possible to switch between exams without upsetting configuration for any of them.

As I see it this this would require (only) the following:

  • Changes to the "live header" - this header stores the exam name and basic configuration details, and it is trivial (a series of #ifdef clauses) to modify this to store more than one configuration at once.

  • Changes to the "examlockdown" script - the script on the profile server should be improved to allow an exam title to be passed in; this defines the macro for use by the live header. The end result of this is that the machine profile would hold a tag representing the exam to which it is locked.

  • Removal of the "examreadonly" directory - this stores data for only the current exam, has to be manually administered "in real time" and is the only true limitation to multiple concurrent exam.

  • Modification of the "getpapers" command - this should be pointed to a second source directory so that it can deliver both the exam paper(s) and non-paper resources.

  • Modification of the instructions given to students - They should be required to make a writable copy of any template files delivered by getpapers.

  • (Optionally, but important for support) lockdown screens to show clearly for which exam a lab machine is configured.

These changes would allow us to testing for one exam potentially even while another exam was taking place. All exams held concurrently (or over a short time period) could share a DICE release, or be branched into their own releases if changes needed to be made for that course.

Note that, because the lab exam environment tracks the current DICE lab environment, it is important that the testing schedule is kept in place, but (at cost of lagging behind the current stable release for longer) the testing window could be stretched by a week or two to allow more flexibility during periods of tight exam scheduling.

-- GrahamDutton - 08 Feb 2012

Edit | Attach | Print version | History: r6 | r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r2 - 09 Feb 2012 - 09:51:40 - GrahamDutton
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies