Original Proposal: Lab Exam System

This proposal was implemented and its documentation is integrated into the Lab Examination Procedure and getpapers pages.
Please note that live documentation on this system is always available and linked from the main ResearchAndTeachingUnit page.

Facilitating running practical examinations on standard lab desktops. Proposal below with questions.

Firewall

There should be a means to enable and disable a set of firewall rules that lock the machine down completely. The only proposed network holes in a locked down machine would support standard infrastructure (eg. KDC, DNS, NTP etc.) and AFS/NFS for remote file access. All other protocols would be blocked, eg. web, ssh, scp, ftp etc.

There should not be any AFS/NFS access to areas that students could previously have deposited files (such as home directory and group space). This may need a restricted map set or separate access permissions?

Communication mechanisms will need to be disabled: email, chat, skype, newsgroups, remote status monitoring (eg. using the number of shell's started to indicate which multiple choice answer to select), etc. - Bob Fisher

gdmr's comments

At the moment most machines run with no local firewall rules at all, though the default set does not seem to have caused problems on those machines which do use them. There is a slight side effect of putting a machine into an exam pool firewalled in this way: because of the way the configuration is done, it would then have to run with local firewalling enabled until reinstalled, though with the default ruleset while not in use for exams. This is almost but not quite transparent, and is maybe something which should be tested out in one of the labs in advance.

It should then be straightforward to set up rules specifying exactly which other hosts a machine is able to speak to. Specifying host/port combinations would be more complicated, and would be something to avoid if possible.

Note that some protocols (NFS, for example) are firewall-unfriendly, and it would be good to tie down exactly what would be needed sufficiently far in advance to be able to iron out wrinkles, or at least put workarounds in place. It might be necessary, for example, to set up a dedicated fileserver in order to be able to separate out permitted traffic and block unwanted traffic.

As an aside, it is possible to tunnel IP over a surprising variety of other protocols...

Hardware

Machines will also need to be locked down to ensure that USB keys, CDROMs, etc. cannot be used.

Web Access

Preferably no web access is allowed, the students can bring in books and notes. Optionally if web access is required then this would be limited to access via a proxy server which could be setup to limit which sites are permitted as well as log which sites are accessed to protect against infringement. An additional firewall hole would need to be setup to access the proxy. Browsers should be reconfigured (automatically) to re-direct via the proxy. This could be done via the local home initial setup. Need to ensure that any sites permitted do not have a route to collaboration web technologies - such as forums, bulletin boards, wikis, blogs and web front ends to newsgroups and email.

Home Directory

Local temporary home directories will be used. They will be setup from a template with a default browser configuration. They will be specific to each machine. This should avoid disk quota problems or access to files left in their home directory (although see above). There is a problem if the machine fails and a local disk account is used as they could lose work (applies equally to a network file space server failing of course). Machines do not normally exhibit a fault while running but at switch on time. We could try to allocate fewer students than the number of machines in the lab, so we can have enough spares for any hardware problems. Recovering lost work could be a problem in this case though. Assuming that this will be very rare, could we have someone to "mark on-line", i.e. ask the student how they were planning to solve the problem without actually re-writing everything?

The students could accidentally delete their own work. We should simply use the submit mechanism for checkpointing, since this allows repeated submissions and is the same as the proposed final submission mechanism. However this would require someone with access to the submit system to recover work if necessary. This leaves the initiative to the students but we could remind them, say, every hour. Also alias the remove command to save a copy somewhere as some people do anyway.

How to guard against a student at 5mins before the end of the exam saying they have accidentally deleted everything and forgot to make any checkpoints (it is easy enough to forge by copying/overwriting files)?

Submission

The final work would be submitted using the familiar submit command and marked off-line. This only requires access to an NFS file system.

Automation

We can probably automate switching lab machines to locked down mode in preparation for an examination, but each may need to be individually checked manually before the exam to ensure it was properly locked down (if it is not updating itself properly from central configuration for some reason the lockdown could fail - we may be able to automate an initial test to minimize the work). If someone was logged on during lockdown it may be possible to subvert some of the mechanism, so the lab needs to be cleared sufficiently before this takes place (and all remote login sessions to machines disabled and existing remote sessions would need to be forcibly disconnected). Machines will also need to be dropped out of the Condor pool, to avoid any possible loading and interaction with other jobs.

Left Content

Students could easily enough leave content on the machines up to a week in advance (eg. in /tmp). As part of the lock down procedure we would need to ensure all publically writable areas are wiped. They could also leave background processes running in memory with content so all user processes would also need to be terminated.

Students should be randomly assigned to machines - Bob Fisher

Interruptions

There will need to be procedures for a single users machine failing as well as for infrastructure failing (affecting all users). I assume something similar exists for normal examinations (eg. power failure)? I don't know what the University rules are for examinations that get interrupted. It shouldn't be hard to adopt the same practices.

Other Points

We should keep logs of all programmes run which are then post-processed to detect anomalies (eg. something we've forgotten, etc) - Bob Fisher

-- TimColles - 15 Aug 2007

-- GeorgeRoss - 20 Aug 2007

Topic revision: r15 - 23 Jan 2019 - 13:44:52 - GrahamDutton
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies