who
output has changed again (actually it seems to have reverted to pre-SL7.2 behaviour). examutils-2.8
has been patched to accommodate this but will require full testing.
Bug (updated 2019): There's a bug - possibly resolved - which routinely results in the log host refusing some/all lab connections on lockdown. See Known Issues / Loghost rsyslog failure for instructions.
examsubmit
to submit their work at the end of an exam. This is a wrapper for the normal submit
command which sets the course
and exercise
(to use the submit
terminology) under which the exam work will be submitted. This is however different for each course and exam type (mock, real or resit). Taking the inf1-fp
course exams for example; while the course remains inf1-fp
the exercise could be one of mpe
for the mock exam, pe
for the real exam and rpe
for the resit exam. The examsubmit
command is a one-line script created by the file component from the contents of the examlock.course
and examlock.paper
resources, prior to the exam. This is done via the examlockdown-lock
command (see below).
In the examsubmit
directory, the following files are important to the submit process:
exnamefile
, modgrp
, master
(mastered in Theon, by the ITO)
accnames
(updated by teaching staff with the set_submit_file_names
command)
examsubmit
area (as near to the exam time as is comfortable for added security). Papers and other pieces of reference documentation can be in any reasonable quantity or format, but must be in a <course>/<exercise>/papers
subdirectory, and readable by submit. User-editable files must be, similarly, in a <course>/<exercise>/templates
subdirectory, for example:
getpapers/ ├── <coursename> │ ├── pe │ │ ├── papers │ │ │ ├── documentation.html │ │ │ ├── exampaper.pdf │ │ │ └── more.pdf │ │ └── templates │ │ ├── afile.txt │ │ └── editable.txt │ ├── <exercise> │ │ ├── papers │ │ │ └── ... : : :The
getpapers
script is configured using the examlockdown
command and will only function during lockdown. See LabExamsGetPapers for full configuration details, or below for details of the examlockdown
command.
Remember to create a matching directory in the exambackup/
partition, too:
exambackup/ ├── <coursename> │ ├── mpe │ ├── pe : :without this, backups will not be created. A script to check this this is pending.
examlockdown-lock
as root on the machine hosting LCFG profiles (named lcfg-master
). This takes mandatory course
and exercise
parameters, and then one or more additional arguments, each of which can be the name either of a host or a student lab room number. All matching hosts will be switched into lockdown mode. Be careful: the command performs a regular expression match on the location, so one character out of place could lock down many more machines than you expect. As of v2.1 the command matches against the whole string (e.g. 4.14 will NOT match hosts in 4.14A); you can use wildcards to broaden matches if required.
--test
option (running as yourself, not root). In this mode a snapshot copy of the profiles is placed in /tmp/examlockdown.<user>
and modified instead of the real profiles (to clear this test data, you can use the --clean
option).
If you run into errors the command will sometimes ask you to "Run with bash -x to see why". To do this just run bash -x examlockdown-lock [...]
(with the same arguments afterward) and you'll hopefully see the test that failed (mismatched site or OS for example) in the final few lines of the bash output.
would lock down all the machines in$ examlockdown-lock inf1-op mpe readonly 5.cl-w 5.cl-n wibble
5.cl-w
and 5.cl-n
(according to inventory data) as well as the machine wibble
. They will be locked to the exam inf1-op
, exercise mpe
(i.e. mock exam), and the storage device policy set to "READONLY".
would lock down all the machines in site$ examlockdown-lock --site FH inf1-op pe log 3.*
FH
in rooms starting with 3
(effectively, the whole floor, for machines whose profiles include one of the studentlabs-fh-*
headers). Note the --site
flag which overrides the default of AT
, and the use of the "log" Storage Device Policy (in effect, read-write, in this case).
The end result of running the lockdown script is to add or remove the following lines from a selection of machine profiles:
The command also performs some sanity checks (does the profile contain an appropriate OS header? Does it contain a "studentlab" header in the site specified by#define EXAM_LOCKDOWN 129.215.X.X /* IP of the profile host */ #define EXAM_COURSE crs #define EXAM_PAPER paper #define EXAM_MOUNT_POLICY POLICY
--site
?). This could be done manually, but would be somewhat tedious and risky — in particular, if a machine locks down with the wrong IP address then it will be completely broken and will need to be recovered by disabling its firewall in single user mode.
After a machine receives a changed configuration (if that configuration changes either the course or the paper it will: systemd
is not a 100% deterministic process as it was in SL6. What's more, this landscape seems to change between (and within) minor releases. So an undefined 'settling' period seems to be necessary under some (unknown, but unlikely) circumstances.
Typically this incomplete state always includes the $HOME
not having switched from an /afs
to a /disk/scratch
path (though AFS is always disabled by this point; this simply manifests as an unwriteable home dir. A log search for 'getpapers.*afs'
will turn up all such failures). In practice, doing one of the following will prevent all such failures before they impact on students:
Lab Examination PC
" in bold letters, and the name of the course, exercise, and Storage policy beneath). This should indicate that the machine should be safe to use for the examination process. If certain errors are detected, they will be flagged at the login screen either with a distinctive black background, or as text in (in safety black/yellow). These should be investigated (or not) as documented.
Note that the site
and policy
arguments are case-insensitive but course
and paper
are case-sensitive as they represent specific directories in group space.
examlockdown-lock
command can be used to switch a machine between course, exam paper or policy without needing to unlock. You will see >>> LOCKED (CHANGED EXAM DETAILS). As usual, you will need to wait for the profile to reach the machine. Rhe examlock component "does the right thing" by clearing out home directories if it detects a change in course or exam paper. Accordingly, be careful using this command during exams: it could actively destroy examinees' sessions and data if course or exam are changed. It will not take forcible action if trivial configuration changes, which means it is safe to change storage policy during an exam. Nonetheless please use extreme caution during exams.
exam-desktop.h
header includes the exam-mount-rules.h
live header, which generates a named udev
rules file when the machine is locked down (and deletes it again on unlocked machines).
The following USB device policies are available:
NOEXAM
LOG
BLOCK
READONLY
blockdev
so should behave as if the device is physically ro
).
examlockdown-lock
command. See above sections on lockdown for details.
If none of the above policies is defined, the profile will throw a #warning
and the login screen will display a Mount Policy Undefined warning.
Under certain circumstances, you may wish to alter the policy on one machine only. This can be done using the normal lockdown tool, and will not force a wipe or re-lockdown.
10-*.rules
as specified in the exam policy header) has been created or deleted in the /etc/udev/rules.d/
directory, on any machine. The policy is explicitly named in the variable ENV{EXAM_POLICY}
.
examlockdown-unlock
in the same way as above (though without course, exercise or policy parameters) to return machines to normal; e.g.
or an example at Forrest Hill:$ examlockdown-unlock 5.cl-w 5.cl-n wibble
Like its$ examlockdown-unlock --site FH 3.D02 3.D01
-lock
counterpart, this command removes the #define EXAM[...]
lines from each specified profile or group, and triggers machine reconfiguration. The machine is safe to use again when its login screen returns to the standard one. If the login screen does not change automatically, something has failed, and investigation is advised.
Beware: examlockdown-unlock
will unlock a machine from any configured exam and destroy data: so if concurrent exams are in progress please be careful to unlock only those rooms required.
examlockdown-unlock
script will currently not remove examrestore
files (though the next, unreleased version will do). Manual removal of /tmp/examrestore
is advised on machines (though its contents are secure, and /tmp
will be cleared automatically on reboot).
getpapers
to check that exam paper content placed in the locked-down home directory is as it should be. Test using a dummy file of some sort if papers are not yet available. Test that template files are user-editable.
examsubmit FILE
, where FILE
is the name of each acceptable file name for the upcoming exam (there does not have to be any meaningful content in FILE
).
getpapers
.
exambackup
, examsubmit
and examreadonly
partitions: exambackup/<course>/<exam>/
and this should be writable by 'submit'.
/presence
subdirectory, as stale files might confuse any analysis of the upcoming exam (these should be removed automatically on unlock but it's unlikely to work 100% of the time).
live/studentlabs.h
header the line !profile.release mSET(NAME)
, where NAME
is the name of the branch on which to freeze.
live/exam-ips.h
.
/disk/home/LOGHOST/rsyslog/byInName/tcpPORT.log
where LOGHOST
is the current Informatics loghost machine name and PORT
is defined in the <dice/options/exam-desktop.h>
. Tailing this log provides an overview of all machines' state.
A good overview of the exam in progress can be had by tailing (remember to use tail -F
) this combined long, grepping for the names of each of the applications involved: getpapers
, examrestore
, exambackup
and submit
. Other analyses such as device and network logging can prove useful. In general, firewall "block" logs are indicative of a service that should've been stopped but hasn't - we're attempting to eliminate these - but are worth casting an eye over.
This only affects the first student to log in after a locked-down machine has been rebooted; it can be trivially changed from the top of the menu. Students are informed by the information sheet provided. MPU are investigating the problem.
They can access it via the command line, or via the "Places" menu.
Pressing Alt+Tab and switching between applications can sometimes bring it back to life.
we've seen the loghost failing to log some or all lab exam PCs remotely. It usually doesn't affect unlocked machines, but results in a near-empty combined log file. This seems to be associated with anrsyslog
process stuck at ~100% CPU onloghost
, so is easy to spot.
To recover: (please Let Infrastructure Unit know you are planning to do this): usesystemctl
to restartrsyslogd
on the log host: logs should start to appear automatically but re-locking down / rebooting clients might be necessary in some cases. Inf Unit are investigating this.
examsubmit
citing a missing /home/submit/master
file. This seems to be a component race on lockdown, and can be confirmed by a failure in the LCFG file log along the lines of " failed to create link: /disk/scratch/home/submit
". The fix is straightforward: run om file configure
on the affected machine, twice if it fails first time, and retry.
ctrl-alt-backspace
to restart the display manager: it might appear to work, but will just mask the problem (likely a systemd
service deadlock, rather than anything exam-specific). It's probably not worth trying to diagnose at first: reboot the machine; if it comes back locked, it will be OK.
getpapers
command is not installed on a machine’s first DICE install. It will also not be installed on a reboot, since updaterpms runs before LDAP and this prevents appropriate permissions being set on the package’s files. The solution is to make sure that all machines participating in an exam lockdown have either been in service for 24 hours (therefore completing an overnight updaterpms run), or have had om updaterpms run
performed manually, prior to an exam. TODO: use an examlock.exec_pre
script to schedule an updaterpms run on lockdown
During open-book exams, we have seen students copy the contents from their USB sticks to the local home directory. This can have serious consequences on the backup partitions! You must identify which backups are causing the problem (usingdu
in theexambackup
directory onexamback
). You can then useless
to look at the backup and identify the likely file or directory that needs to be excluded. You can then exclude this file from backups on an individual user's machine by editing their profile (or, in extreme situations only, inlive/exam-desktop.h
) with the resourcefile.tmpl_exback_x
, for example:/* don't back up bigfile.dat in any directory */ !file.tmpl_exback_x mSET(bigfile.dat) /* do not back up foo or BAR in any directory */ !file.tmpl_exback_x mSETQ('\ foo \n\ BAR \n\ ')This resource forms the contents of an "exclude file", so the syntax for this resource is the same as for thetar -X
argument; seeman tar
on SL7 for details. You can check that the excision has been carried out correctly by running the exambackup command as root on the affected machine, comparing the output with a previous backup.
exambackup/presence/<hostname>
which contains the fields HOSTNAME:course/exam:console-user
. If a host has not refreshed its 'presence' file in over approximately five minutes, it should probably be investigated.
If a computer crashes during the exam follow the technical procedure below (invigilators have complementary procedural documentation of which this is a part):
/tmp
) by running examrestore sMATRIC
(where MATRIC
is the student's matriculation number). You will need the submit
role in order to do this; most academic staff will have this role, as do some support staff and any PhD markers.
examrestore
(with no arguments this time); this extracts the recovered backup in /tmp
into their new local home directory. It will also restore[1] getpapers data on their behalf. You can monitor the activity of this command in the usual system log.
getpapers --restore
restores the local exam paper cache on local disk and will fix any broken symlinks in the restored data, without touching the user's recovered home directory. Students should never need to run this manually but it is an important part of exam state restore.)
!auth.users mSET(@sysmans)
!examlock.lock mSET(hold)
(this must be below the studentlabs
header).
examlock.lock
above, in extremis you should:
auth.users
change above to take effect, and:
#error "EXAM QUARANTINE"
in the profile, with an accompanying comment.
exam_process_submissions
. The full documentation of process_submissions
(on which exam_process_submissions
is based) can be found here: Using the Practical Submission System. Basically, most staff will want to do the following things:
sMATRIC
(where MATRIC
is the student's matriculation number) in the current working directory, each of which having the corresponding student's exam submission. In this example, we want the submitted files called 'exam.hs' from course inf1-fp and exam rpe (repeat practical exercise):
$ exam_process_submissions --comline "cat %f > %u" --file exam.hs inf1-fp rpe
if513m0
and mark the top of the page with the student's martric number and some other relevant information:
Unfortunately$ exam_process_submissions --comline "enscript --header='%u' -G -P if513m0 %f" --file exam.hs inf1-fp rpe
exam_process_submissions
can only be found at present on exam machines themselves, and it is no longer usable thanks to changes in submit
. Both of these issues will be addressed ahead of future exams.
#define
must be made ahead of the studentlabs include, and resource changes should be made after, for example:
#ifndef LIVE_STUDENTLABS_FH_3_D08 #define LIVE_STUDENTLABS_FH_3_D08 /* move examsubmit & examreadonly to new server */ #define NFS1 129.215.nnn.nnn #include <dice/options/studentlabs-at.h> /* override exam mount policy for this lab */ !examlock.mountpol mSET(log) [...]
exambackup
, examreadonly
or examsubmit
to the DICE_OPTIONS_EXAM_NFS_EXPORTS
macro.
vda4
will be sufficiently large to cope.
exam-nfs-server.h
partitions
rfemap, along the lines of hostname1 host.ipv4.address /disk/data/
NFS1
(for examsubmit
, examreadonly
) and/or NFS2
(for exambackup
) on each affected lab (see example lab header change above).
autofs
to pick up the new mounts. But, if individual machines fail to pick up the changes, you might need to forcibly unmount the broken partition and restart autofs. You can do this remotely through the loghost.
amdmap/group
and partition
(SL6 hosts), automap/group
and and automap/partition
(SL7 hosts) and dns/inf
RFE maps, and ensuring the backup stanzas match the file contents, e.g. MIRROR_THIS_AND_TAPE(exambackup,<%nfs.fs_exambackup%>)
.
#define LOGHOST
in the appropriate lab header (see example lab header change above) and await reconfiguration.
exambackup
& examsubmit
.
examsubmit
and support will have to manually submit, or assume submissions from automated backups, when the network becomes available again.
getpapers
and students instructed to run it on any code before submission. Theoretically submit
could be modified to call a check script before submission, but the contents of that check would be exam-specific and responsibility of the setter, and students must not be prevented from submitting failing code should they choose to do so.
/home/submit/submissions/lp/default/jcheney/pe/exam.pl
, but it the path should be /group/examsubmit/submissions/lp
submit
codebase (which I'm reluctant to do) but it's a valid concern. Update: As of Nov 2015 feedback should normally be delivered as soon as the exam PC is unlocked, rather than many hours later as was previously the case. Update: As of Aug 2016 Director of Computing (Perdita) has requested that the confirmation emails be turned off. pending December diet, RT:78588.
exam-ips.h
submit
and exam-submit
UIDs so we don't need a local override: RT:75292
inf1-op
/ pe1
it would check for /disk/data/examsubmit/getpapers/inf1-op/pe1