Moving AFS Volumes

The notes below give an example walk through of moving a user volume. The same principle will apply for a group volume, but the number of readonly copies to reinstate, and the partitions on where to store group volumes may differ.

For a list of volumes on servers, and what they are intended for, see the AFSPartitions topic.

The Plan

Assuming you are doing this because an AFS partition is running out of space, then you'd roughly do the following:

  • vos listvol to see who is using the most space, and so moving them would leave more space for the remaining people
  • vos remove ...readonly to remove the readonly copies of the volume you are moving.
  • identify a suitable destination server and partition
  • vos move to actually move the RW volume (do not interrupt this, as it will cause problems)
  • vos backup to recreate the backup volume
  • vos addsite to recreate the readonly volumes on the appropriate partitions/servers.

More detail

You will need to become your admin AFS self (asu is good for this), but then you can do it from any AFS client. You don't need to log into any particular server/machine.

vos listvol

So in this example, phoenix /vicepa is nearly full, we'll move someone off it to make space. So first use vos listvol to identify a suitable person for moving eg:

[doves]neilb: vos listvol phoenix /vicepa
Total number of volumes on server phoenix partition /vicepa: 148 
...
user.johnb                        536871080 RW     600032 K On-line
user.johnb.backup                 536871082 BK     600032 K On-line
user.johnb.readonly               536871081 RO     600032 K On-line
user.ktd                          536871027 RW    1834595 K On-line
user.ktd.backup                   536871029 BK    1834572 K On-line
user.ktd.readonly                 536871028 RO    1834572 K On-line
...
As the delta backup and readonly copies are also reported, it might be clearer to grep out the RW versions and then list them sorted by usage, eg vos listvol phoenix /vicepa | grep RW | sort -k4 -n, this now gives me:
...
user.userx                        536871177 RW    4565170 K On-line
user.usery                        536871152 RW    6219608 K On-line
user.sxw                          536870924 RW    7816342 K On-line
user.dcspaul                      536870983 RW    9167631 K On-line

So dcspaul looks like a good candidate!

vos remove

Find the readonly versions of the volume to remove with vos exam <volume> and then vos remove them. eg

[doves]neilb: vos exam user.dcspaul
user.dcspaul                      536870983 RW    9167639 K  On-line
...
    RWrite: 536870983     ROnly: 536870984     Backup: 536870985 
    number of sites -> 3
       server phoenix.inf.ed.ac.uk partition /vicepa RW Site 
       server phoenix.inf.ed.ac.uk partition /vicepa RO Site 
       server har.inf.ed.ac.uk partition /vicepa RO Site 

So we need to remove the .readonly volumes on phoenix/vicpa and har/vicepa.

[doves]neilb: vos remove har /vicepa user.dcspaul.readonly
WARNING: Volume 536870984 does not exist in VLDB on server and partition
Volume 536870984 on partition /vicepa server har.inf.ed.ac.uk deleted
This can take a while, don't interrupt it, the above example took 5 mins. I like to use the -verbose flag to see what is going on.

Note I actually made a mistake in this example, I should have used the server name har-afs, as it is a dual hosted machine, and this confuses AFS, hence the warning here, and below. Always use har-afs rather than just har when specifying the server har in AFS commands.

Next the phoenix partition:

[doves]neilb: vos remove -verbose phoenix /vicepa user.dcspaul.readonly

user.dcspaul 
    RWrite: 536870983     ROnly: 536870984     Backup: 536870985 
    number of sites -> 3
       server phoenix.inf.ed.ac.uk partition /vicepa RW Site 
       server phoenix.inf.ed.ac.uk partition /vicepa RO Site 
       server har.inf.ed.ac.uk partition /vicepa RO Site 
Trying to delete the volume 536870984 ... done
Marking the readonly volume 536870984 deleted in the VLDB
Volume 536870984 on partition /vicepa server phoenix.inf.ed.ac.uk deleted

vos move

Before you can move the RW volume, you need to know where you are moving it to, in this case we're going to move user.dcspaul to a new partition on the same server vos partinfo phoenix lists the space available on its partitions, and in this case (baring in mind what partitions are to be used for in AFSPartitions) vicepj is what we'll use.

[doves]neilb: vos move -verbose user.dcspaul phoenix /vicepa phoenix /vicepj
Starting transaction on source volume 536870983 ... done
Allocating new volume id for clone of volume 536870983 ... done
Cloning source volume 536870983 ... done
Ending the transaction on the source volume 536870983 ... done
Starting transaction on the cloned volume 536871981 ... done
Setting flags on cloned volume 536871981 ... done
Getting status of cloned volume 536871981 ... done
Creating the destination volume 536870983 ... done
Setting volume flags on destination volume 536870983 ... done
Dumping from clone 536871981 on source to volume 536870983 on destination ... done
Ending transaction on cloned volume 536871981 ... done
Starting transaction on source volume 536870983 ... done
Doing the incremental dump from source to destination for volume 536870983 ...  done
Setting volume flags on old source volume 536870983 ... done
Setting volume flags on new source volume 536870983 ... done
Ending transaction on destination volume 536870983 ... done
Releasing lock on VLDB entry for volume 536870983 ... done
Deleting old volume 536870983 on source ... done
Ending transaction on old volume 536870983 on the source ... done
Creating transaction for backup volume 536870985 on source ... done
Setting flags on backup volume 536870985 on source ... done
Deleting the backup volume 536870985 on the source ... done
Ending transaction on backup volume 536870985 on source ... done
Starting transaction on the cloned volume 536871981 ... done
Deleting the cloned volume 536871981 ... done
Ending transaction on cloned volume 536871981 ... done
WARNING : readOnly copies still exist 
Volume 536870983 moved from phoenix /vicepa to phoenix /vicepj 
Again this will take a while, depending on the amount of data, in this case 11 minutes, but the user should notice no more than a temporary (a few seconds) glitch.

The "WARNING" is because of my previous failure to completely remove the har readonly copy. This was rectified by reissuing the remove with har's AFS alter ego vos remove har-afs /vicepa user.dcspaul.readonly. vos exam user.dcspaul now shows what we expect:

user.dcspaul                      536870983 RW    9167926 K  On-line
    phoenix.inf.ed.ac.uk /vicepj 
    RWrite  536870983 ROnly          0 Backup          0 
    MaxQuota   10000000 K 
    Creation    Thu Dec 14 03:41:10 2006
    Copy        Wed May 30 11:40:43 2007
    Backup      Wed May 30 02:23:59 2007
    Last Update Wed May 30 12:16:10 2007
    2799 accesses in the past day (i.e., vnode references)

    RWrite: 536870983 
    number of sites -> 1
       server phoenix.inf.ed.ac.uk partition /vicepj RW Site 

We now need to recreate the .backup and .readonly copies of this volume.

vos backup

This is simple, just vos backup <volume>

[doves]neilb: vos backup -verbose user.dcspaul
Creating a new backup clone 536870985 ... done
Created backup volume for user.dcspaul 

vos addsite

For user volumes we create a readonly copy on the same partition as the RW volume, and an off site RO copy on the "mirror" volume of the AFS server on the other site, see the table in AFSPartitions. In this case we need an RO copy on phoenix /vicepj and sphinx /vicepk. The basic use is vos addsite <server> <partition> <volume>.

[doves]neilb: vos addsite -verbose phoenix /vicepj user.dcspaul
Adding a new site ... done
Added replication site phoenix /vicepj for volume user.dcspaul

[doves]neilb: vos addsite -verbose sphinx /vicepk user.dcspaul
Adding a new site ... done
Added replication site sphinx /vicepk for volume user.dcspaul

Then run vos release user.dcspaul to update the RO copies.

[doves]neilb: vos release -verbose user.dcspaul

user.dcspaul 
    RWrite: 536870983     Backup: 536870985 
    number of sites -> 3
       server phoenix.inf.ed.ac.uk partition /vicepj RW Site 
       server phoenix.inf.ed.ac.uk partition /vicepj RO Site  -- Not released
       server sphinx.inf.ed.ac.uk partition /vicepk RO Site  -- Not released
This is a complete release of volume 536870983
Cloning RW volume 536870983 to permanent RO... done
Getting status of RW volume 536870983... done
Ending cloning transaction on RW volume 536870983... done
Starting transaction on RO clone volume 536870984... done
Setting volume flags for volume 536870984... done
Ending transaction on volume 536870984... done
Replacing VLDB entry for user.dcspaul... done
Starting transaction on cloned volume 536870984... done
Creating new volume 536870984 on replication site sphinx.inf.ed.ac.uk:  done
Starting ForwardMulti from 536870984 to 536870984 on sphinx.inf.ed.ac.uk (full release).
updating VLDB ... done
Released volume user.dcspaul successfully
Again, this can take minutes to complete depending on the amount of data to copy.

Finally we can check things are as they should be with a final vos exam <volume>

user.dcspaul                      536870983 RW    9167928 K  On-line
    phoenix.inf.ed.ac.uk /vicepj 
    RWrite  536870983 ROnly  536870984 Backup  536870985 
    MaxQuota   10000000 K 
    Creation    Thu Dec 14 03:41:10 2006
    Copy        Wed May 30 11:40:43 2007
    Backup      Wed May 30 12:21:15 2007
    Last Update Wed May 30 12:38:47 2007
    3109 accesses in the past day (i.e., vnode references)

    RWrite: 536870983     ROnly: 536870984     Backup: 536870985 
    number of sites -> 3
       server phoenix.inf.ed.ac.uk partition /vicepj RW Site 
       server phoenix.inf.ed.ac.uk partition /vicepj RO Site 
       server sphinx.inf.ed.ac.uk partition /vicepk RO Site 

-- NeilBrown - 30 May 2007

Topic revision: r3 - 24 Jun 2013 - 15:56:17 - RogerBurroughes
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies