DICE OpenAFS service

At the Services meeting where the OpenAFS Pilot proposal was accepted, a request was made to provide a Wiki page for discussion of its development. These are those Wiki pages.

Other notes

  • OpenAFSUserIssues details problems (and some solutions) that may be encountered by users new to AFS

  • AFSonDebian demonstrates self-managed connection to DICE AFS.

Admin Info

  • AFSTopFive - The AFS Top 5 Questions for Informatics COs

  • AFSProblemsSolutions Details of the solutions for the various problems identified with moving users to AFS
  • AFSLegacyWeb Some things that can crop up with a users legacy web page when a user is moved to AFS.
  • OpenAFSUidFix the steps to take when changing the AFS UID of a user (which you should never need to do in theory).
  • AFS Server Details CGI to show you some details about the main AFS file servers, if you chop off the arguments you get all the afs file servers.
  • MoveAFSDB Steps I took to move an AFS DB to another host.
  • UpgradeAFSDBPlan Our plan to migrate from AFS DB servers from 1.4 to 1.6
  • RecoverFilesFromRawAFS

New openafs Component

  • Information on using the new openafs component to configure an AFS Cell is available on the LCFG wiki
  • OpenAfsHeaders - The DICE openafs header structure, where to look and how to change stuff
  • AfsDbSwitch - Plans to switch an AFS DB server to the new openafs component

Ancient Stuff

  • AFSDiceFC3 describes how to get AFS going on a DICE managed FC3 machine
  • AFSDiceRH9 describes how to get AFS going on a DICE managed RH9 machine
  • AFSAddNewDBServer describes how to add a new database server to our existing system
  • OpenAFSStudentIssues A list of issues to be resolved and actions taken before students receive AFS based home directories by default
  • OpenAFSLinuxFileserver What I did to turn hawthorn into a linux fileserver


List of all AFS bugs, list of open AFS bugs.

Things to do

At the fileservices meeting on 18 May 2004, Craig outlined a list of the major issues that we had to look at. I (Simon) remember the following from the list:

  • Client RPM-ification
  • Client LCFG-ification
  • Backups
  • Documentation
  • Sync unix groups into PTS. Plan: getent group > 500; create pts entry inf:groupname and assign membership



Please use the space below to outline issues we've got to resolve.

  • AFS tree layout How should we order homedirectory mount points, given the performace implications of having every home volume mounted under one location. Unforunately, the standard solution of having /user/<initial-letter-of-username>/ doesn't work due to the huge number of 's' users. It was suggested that we could perhaps use the former scheme for staff, but do students as either /user/s/<first-two-digits>, or even /user/s<first-two-digits>/

  • AFS volume naming We should decide, and document, a standard form of naming for volumes.

  • Server key distribution We need a mechanism better than scp for distributing the afs@INFREMOVE_THIS.ED.AC.UK service key to all of our fileservers.

  • Root Volume Should we use a dynamic root volume (populated from ThisCellDB), a dynamic root from the DNS (with AFSDB records), or a static root on our Linux clients. Note that Windows automatically uses a dynamic root. The current LCFG component only supports the user of dynamic roots

  • Cell visibility Should we make our AFS cell available beyond the firewall?

  • AFSDB records Depending on the answers to the above two questions, we should probably put AFSDB records into the DNS. Whilst we can do this with #verbatim entries at present, should we ask gdmr about extending the current SRV record provision so it can all come from LCFG? We've hacked some verbatim records in for now. We will need a better solution

  • AFS kernel module Build at boot, using the LCFG kernel component, or distribute as RPM? Currently distributing as an RPM, but Alastair is very keen to have build-at-boot support (and possibly use this as the only distribution mechanism)

  • AFS cache configuration What settings should we use on the client for best performance?

  • Long running processes How should we handle the issue of processes which run beyond credential expiry. This is probably a problem for the auth team to solve, but it needs to be flagged.

  • Volume database Stanford run a set of tools on top of AFS to, amongst other things, maintain a database of canonical volume mount points and collect performance information. Do we need to look at something like this?

-- SimonWilkinson - 19 May 2005

  • system:admin privileges It seems a bit easy for me (with system:administrators privileges) just to go to someones home dir and modify/delete files. At least with the current FS I have to remember to nsu user on a client, or nsu to root on the fileserver to be able to do this sort of thing. Craig's initial thoughts are perhaps we need an "afsadmin" type user with a separate password, which people need to aklog to before getting the system:administrator rights.

-- NeilBrown - 07 Jun 2005

On the above, I think that the correct way to do this is to use the existing user.admin principals. The simple user principal should be able to do no more than a normal user, and system:administrators privileges should only be given to user.admin priviledges. You can do this to your own account by doing the following:

pts createuser USER.admin -id NEWID
pts adduser USER.admin system:administrators 
pts removeuser USER system:administrators

The main problem is picking a NEWID. We need to come up with some way of allocating unique IDs to instances. At the moment, I've just started from '2' and am working up!

-- SimonWilkinson - 07 Jun 2005

  • How do we populate the UserList list of users with administrative rights. One of these has to be maintained on each fileserver. Should we just use LCFG, or should the list be extracted from a LDAP-borne capability definition? AFS has a mechanism for distributing this file, once its been populated on a master server - should we use it?

-- CraigStrachan - 23 Jun 2006

  • The default permissions for a new mountpoint (drwxrwxrwx), especially when the mountpoint is a home directory, can make programs such as procmail complain. Should part of the proceedure for setting up AFS home directories be to change the permissions to drwx------?

  • When a home volume is moved to another partition the backup volume is deleted which means that Yesterday in the user's home directory will not be accessible until the backup volume is recreated. This should at least be documented in a FAQ somewhere.

  • FC5 has a new, security enhanced, cron that tries to do a 'cd $HOME' of the user running the cron. This fails for people with AFS homedirs, that don't have 'anyuser' permissions on their home directory.

Odd Sticky bit behaviour

# When trying to move Rosemary's homedir from NFS to AFS I did the following, and had permission problems. On interlaken (so it could read her home dir from wyvern). I did:

  kinit neilb/admin
  cd /afs/inf.ed.ac.uk/user/r/rs
  rsync -a wvyern:ptn043/rs/ .

This seemed to start OK, but then gave errors along the lines of:

rsync: cannot move `.foo.1234' to `foo': Operation not permitted

.foo.1234 is the temporary file that rsync first copies across to the destination directory, and then tries to do a 'mv .foo.1234 foo'. I tried doing this 'mv' by hand as my "super me" on interlaken. I too got the permission denied error. I showed Craig, and he could "touch newfoo", but not "mv newfoo bar", as his cms/admin either.

The couple of things that looked odd about this particular directory, was the ownership and "stickbit" were:

drwxrwxr-t 2 28006 submit 2048 Aug 10 15:30 .

28006 being by neilb/admin AFS uid. For some reason the sticky "t" bit look a likely candidate for the problem. We unset this, and could now do the mv's! We reset the sticky bit and tried the mv's on a Solaris AFS client, and this worked, ie even with the sticky bit, our neilb/admin AFS id was able to rename the files.

So one question is, is it the AFS kernel module that is behaving differently, or the OS itself. Looking on the web I did come across this definition of the "others" sticky bit.

A directory with the sticky bit set means that only the file owner and the superuser may remove files from that directory. Other users are denied the right to remove files regardless of the directory permissions

So given this, it is correct that as "not root" I was unable to (re)move the files, but as I (admin me) owned the file I just touched, I would have thought that meant I could (re)move it!?

I didn't like the fact that the files I was rsyncing were owned by "admin me", rather than Rosemary, so as an experiment I nsu'd to root, (who inherited me neilb/admin token) and tried the rsync again. This time everything worked fine. All the file ownerships were correct, and no errors from rsync about not being able to rename files.

So is this the expected behaviour of the sticky bit and AFS, even when you are in system:administrator mode? For now it seems the advise to the like of support when moving people's files to AFS, is to do so as root with their /admin token. -- NeilBrown - 01 Sep 2006

Topic revision: r68 - 14 Jan 2021 - 16:15:25 - NeilBrown
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies