Dynamic AFS Quotas New Session Issue

At the beginning of the 2016/17 session we had issues with the dynamic afs quotas not being responsive enough. We don't recall this being a cohort wide problem before, individual cases have cropped up occasionally.

Some points to recap:

• new 1st year max AFS home dir quota is 4GB
• actual quota for new (empty) account will be 1GB
• the hourly dynamic quota system will increase a user's quota by 1GB when their current usage is within 600MB of their current actual AFS quota

In this case the problem was for a 1st year module, the first practical session the students had, was to download some large amount of data. Being new users/accounts their maximum AFS home dir quota would be 4GB, but their actual quota will be set to 1GB.

In class of new students (with only a minimal number of files in their home directories, ie <400MB) their actual current quota will be 1GB. If during a module they are asked to download >1GB of files, it will fail as they are likely to hit their current 1GB limit. An hour later (or sooner depending on timing) their quota would increase to 2GB.

This obviously isn't very acceptable during a lab session for example. What options do we have to avoid this?

1. Get advance warning of future, similar type classes/labs/tutorials and increase the appropriate student quotas ahead of time.

2. Give new students/accounts a bigger initial quota, 2GB instead of 1GB? Given them all their quota, 4GB right at the start. Though that negates the point of the dynamic quotas in the first place (which was to use our disk space more efficiently).

3. Give new students a bigger initial quota (as above), but after some period of time (one month?) shrink their actual quota (not their max quota) back down to what it would be had the dynamic quota system just been allowed to do things at its own speed.

4. Provide some way for students to increase their dynamic quota "now", rather than having to wait till the next hourly run.

Option 1, is probably the most correct and possibly simple for us, but not so simple for the course/module organiser/lecturer to actually inform us in time, and would rely on prometheus roles/entitlements being correct in time.

Option 2, is simplest if we don't care about spending money on disk space that may never actually be used.

Option 3, we'd need to change our account creation scripts and tools, but might be the least noticeable option for users and staff (least work for them).

Option 4, what would stop everyone going "I'll have all my space now", and then them never using it, we're back to something like option 2.

I think option 3 is the most likely. We could create a script to increase a whole year cohort quota to some amount (maybe their max quota), and then (some time later) run a similar script to shrink it back down to something more appropriate to their actual usage.

Update 9/11/2016

Some quick numbers based on students with the following year- roles.

#Role, number of students @maxhomequota = Total space required, Actual dynamic quota applied, actual usage.
UG1 545 @4GB  = 2.2TB, Actual Q 718GB, Usage 120GB
UG2 384 @4GB  = 1.5TB, Actual Q 542GB, Usage 149GB
UG3 273 @10GB = 2.7TB, Actual Q 875GB, Usage 426GB
UG4 151 @10GB = 1.5TB, Actual Q 985GB, Usage 703GB
MSC 460 @10GB = 4.6TB, Actual Q 1.9TB, Usage 935GB

Totals
   1813        12.5TB             5TB        2.3TB

Update 25/1/2017

Quick real world numbers for the AT file servers (mainly student home dirs)

• 3168 users
• 6.7TB disk usage
• 21.7TB total max quota

-- NeilBrown - 25 Oct 2016