Submit Requirements

A list of reasons given by teaching staff to preserve submit for their courses during session 2019/20. These should be used to help guide what we can provide/suggest for teaching staff to use instead of submit in session 2020/21, although it should be noted that the vast majority of courses (about 90%) are not going to use submit in 2019/20.

  • Learn is unresponsive and has a terrible user experience. I'd probably write my own submission cgi script if Learn was the only option on offer. However, it would clearly be better to have one supported submission script that everyone could use.
  • The main requirements are to keep file-system access to the submissions, and the ability to restrict the filenames students can use. Then I can incrementally download with rsync, and process the hundreds of submissions with whatever scripts I want.
  • Downloading stuff isn't a one-off task because of late submissions. Command-line interfaces please!
  • I suspect github classroom is overkill and too complicated for my purposes. I'd also probably end up downloading large amounts of junk that students have committed to their repositories at some point, rather than just the final report that I need.
  • Basically I want submit. Maybe with a (simple, responsive) web form front-end for the students, which could be accessible via EASE (or DICE). It could also let students re-download their submission to check it. Would replacing the setuid submit binary with a simple web form for submission remove the security concern?
  • In Learn, lack of command-line access to submissions is not satisfactory. Staying on top of submissions is much more laborious when you need to download the complete set of submissions via a web-interface, then unpack / merge zip files.
  • Regarding the listed alternatives (CodeGrade , gradescope), those seem to be web-based as well.
  • Students sometimes submit large ZIP files for their coursework (> 1Gb).  We know that the DICE submit command can handle large files like this but other mechanisms such as web-based protocols might not be able to do this.  For this reason we wanted to stay with submit until there was an alternative to submit which we knew would work efficiently for large submissions.
  • We don’t know what the alternatives to submit are, or how to use them.  We wanted to write an instruction in the coursework handout telling the students how to submit their work.  We can do this for the DICE submit command but we can’t do this for the alternatives to submit, whatever they are.  We don’t know what the relative strengths or weaknesses of the alternatives are.
  • The course has over 200 students next year.  We know what the submit directory structure is and we have scripts to process it to look for students who failed to submit, or to organise submissions by student and by date (to identify the last submission of a student, and to identify whether it was on time or late).  We don’t know how the alternatives to submit will organise student submissions, or how to access them, and the scripts which we have will need to be updated or rewritten to work with the new folder structure, and log file structure, whatever those may be.
  • Turnitin does not work for us since students submit a bunch of code files which we need to run and execute.
  • Learn submission is tricky because it does not allow us to enforce a submission format. Traditionally, we have near 200 students on the course which means we make use of automarking in an initial marking run to handle the numbers. The automarker requires a very specify file format and directory structure of submitted files to run smoothly. According to Alex submit results under "Submissions should be accepted in a format determined by the individual assignments", enforcing the format we need is not possible in Learn.
  • Right now, we have a bash script which the students execute on DICE. Initially it checks their submission format and gives them warnings should they not meet the requirements. Only if they are met, the submit command is called under the hood.
  • In the future CodeGrade might be an option but that depends on the results of the pilot.
  • I requested submit because it fills current needs, especially including restricting file names/formats and providing directories on which scripts can be run on code submissions. I've read the documents you have sent but these are not completely explicit about what features are provided by all the alternative mechanisms, but it sounds as if Learn might be made to work ("Tim Colles to look into this").
  • Another consideration I didn't voice earlier is security and privacy of personal (student) information. A mechanism which works on our own filesystem may is probably preferable in this case, rather than one where tutors/markers must download submissions from a browser to their own directories/machines leaving work scattered everywhere.
  • I used Learn in 2018/19. Maybe it was just how Learn/Turnitin was set up (or my unfamiliarity), but:
    • I could not identify which students had submitted the assignment (it was anonymous) so I could not identify which students to check up on.
    • I wanted to avoid giving a submission by a Chinese student team to another Chinese student team (ditto for other nationalities), to reduce chances of collusion. So I needed the identities.
    • The peer-review was only done by students who had submitted their own draft assignment, so again I needed to know who they were.
    • I needed to transfer the draft-submission to each marking team, and then the reviews back to the original teams. So I needed to know the identities of the teams. The anonymity would be a problem.
    • The marker needed the batch of submissions, whereas it appeared that Learn/Turnitin could only download one at a time, and taking c. a minute each, which would mean c. 40 minutes work. One smart copy from the submit subfolder worked in 5 minutes.
    • I had to keep an eye on the submit folder because of late submission concessions. These are easy to spot because of the log file. It seemed like Learn/turnitin did not have dates and kept re-ordering the submissions.
  • Another problem was the management of the students in Learn. Students can be added and never removed. For 2018/19 there are 97 entries, of which 78 are listed as students. At the peak, there were over 100 students in Learn. However, there were only 76 being assessed in the course. And, at the time of the assignments, the Learn listing was not up to date. It's only when you get the coursework submissions that you find out who are actually in the course. There are also inconsistencies in the Learn class list, which contained 4 students not being assessed. But did not contain 2 students who were being assessed: Hence, I want to see some experience and debugging before I switch.

-- TimColles - 26 Aug 2019

Topic revision: r1 - 26 Aug 2019 - 16:15:33 - TimColles
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies