Some notes on Moving mail from mail.inf to staffmail.ed

Here are some notes on what we've noticed when moving to using staffmail rather than mail.inf. We plan on having a general section to do with the basics of the move, and then mail client sections for specific client issues.

There is a web page that describes the migration process (intended for Computing Staff for now), it's MoveToStaffMail, let me know if there is anything unclear/wrong! There is now a user version on www.inf: http://www.inf.ed.ac.uk/systems/email/staffmail/moveto.html

Staffmail for new Staff and Visitors

All new Informatics users (non-students) are getting staffmail accounts. More correctly, they are not getting mail.inf accounts. All staff should be getting staffmail.ed accounts, but not all visitors do. In which case we hope to forward visitor@inf to visitor@someREMOVE_THIS.existing.address.

There is documentation for new staffmail users at http://www.inf.ed.ac.uk/systems/email/staffmail/ .

For more information about how we manage these new accounts, see StaffMailManagement.

General Points

It can be tricky to do. Not so much technically, but from a management point of view. Especially if you are a problem user like Neil, that already has a bunch of mail, filters on staffmail.ed and has non-simple filters on mail.inf. EUCS have offered to take our .procmailrc and files and just put them on their machine, but if we were to do this for everyone en-masse, then that's a lot of data, 110+GB and currently 449 .procmailrcs.

So as to not get bogged down in that stage right at the beginning, I (Neil) am probably going to move my filters and mail to staffmail myself.

See Filtering below: Gordon

Once you've changed your email address, you may not be able to post to lists that restrict posting to the address you're registered with. This means unsubscribing one name and re-subscribing with another.

Specific Mail Client Issues

Though I'm mainly a pine user, I do use other clients, and hopefully other COs that try the move, can jot down any problems they may have with their clients. Please do add your comments to this page.

Pine/Alpine

For now, in no particular order, just as I come across them.

  • Now that kerberos is looking up the kerberos realm from the DNS, the only thing you should need to do to get cross-realm auth working is setup the ACLs at the staffmail.ed end, see MoveToStaffMail#Kerberos_Authentication. Though the EUCS are currently working on removing this requirement too, by making UUN@inf equivalent to UUN@ease (or something like that!).

  • A lot of "KDC did not match expectations" in the message line - This has gone away with cross realm auth.

  • Even with the cross realm stuff working, I've still set my ssh-connection-timeout option to "0" to disable it attempting to connect via ssh. Update: This has been set as the default from alpine-1.10-3.inf.

  • It's not easy to see at first how you'd be able to use pine transitioning between the two servers and be able to see INBOXes on both - Pine does have a notion of a single primary INBOX, but there are settings to allow multiple INBOXes. The keyword is "Incoming-Folders", and the internal documentation will guide you through setting this up.

  • Update on the above: Alpine now provides INBOX for each folder collection. This will help with the transition. If you are planning on using both INBOXes on a more prolonged basis, "Incoming Folders" are still the best way to do this.

  • Having set userdomain to staffmail.ed.ac.uk, you will be unable to use standard user-completion, e.g. typing services-unit [rtn], to email an inf list. However, this change is not required. If you wish to change your email address to '@staffmail' whilst retaining an inf userdomain for completion, you may do so either by use of Roles, or by adding From: Full Name <user@staffmail.ed.ac.uk> to your Customized Headers configuration (see the help on this item before proceeding). Alternatively, use the Informatics LDAP directory to complete addresses in the composer.

Pine exhibits some irritating delays when attempting to list the root folder collection for Staffmail. You can work around this by moving all of your folders into a container folder, and changing your Staffmail folder collection to:

    Server    : imap.staffmail.ed.ac.uk/ssl
    Path      : container/
    View      : *
Alternatively you can leave all your folders in the root, but rename them with a common prefix ("pre-" in this example), then use a 'view' of the root folder:
    Server    : imap.staffmail.ed.ac.uk/ssl
    Path      : 
    View      : pre-*

If you use IMP occasionally to configure filters, don't forget to update your IMP folder locations and to resubscribe to the newly-created container.

Thunderbird

Kerberos Auth does work for Thunderbird, again see the MoveToStaffMail#Kerberos_Authentication notes on setting up ACLs via the web interface, then tick the Use secure authentication option on the Server Settings page from the Accounts Option page. Note I had to completely remove my previous mail.inf.ed.ac.uk account settings and recreate them with the staffmail IMAP options. Just changing those IMAP settings in my previous settings was not enough.

Outlook Express

Outlook Express Select Columns screen shot

The default for Outlook Express is to show the date of arrival of the message in your INBOX, rather than the date it was sent, ie the =Date: = field in the mail message. If you are copying messages from mail.inf to staffmail.ed, then the arrival date for those messages will be the date you moved them, not the date they were sent. This means that when you look at the folder in Outlook Express, you'll won't see the actual date the messages were sent.

You can change this by going to the View menu and selecting Columns... option. In the window that appears untick the Sent option and tick the Received option, and then click on OK.


Kerberos issues

Neil's comment about cross-realm Kerberos auth got me thinking. I did a load of interop testing with the "still-beta" staffmail service with Graeme, and got all of the cross realm stuff going. So here's a guide to what you need to do ...

  • Configure your Kerberos software Staffmail is actually a machine in the .ucs.ed.ac.uk domain, and you need to tell Kerberos to use the EASE.ED.AC.UK realm for machines in that domain. I'm typing this from my (DICE)ish Mac - the Kerberos configuration file is in /Library/Preferences/edu.mit.Kerberos, and I need to add the following to the domain_realm section of the file
        ucs.ed.ac.uk = EASE.ED.AC.UK
        .ucs.ed.ac.uk = EASE.ED.AC.UK

There's some resources that will let you do the same with the lcfg kerberos component - these should probably get set centrally.
UPDATED This has now been done, and DICE machines are now setup to use DNS to discover the kerberos realm for services. Look back to version 1.18 of this document to see Simon's previous instructions if you are curious.

  • Configure your staffmail account Using the web interface, login to staffmail. Select 'Options' and then 'Share Folders'. Grant access to <uun>@INF.ED.AC.UK

  • Test it out With Mail.app this all 'just works' ...

-- SimonWilkinson - 22 Jan 2007

Filtering

I set up my filters by hand, modelling them on my informatics webmail filters, so all point and click nothing clever. There are a few gotchas so far.

Whitelists: It's counter intuitive to me, and the documentation isn't perfectly clear that passing the whitelist doesn't mean "Accept this mail and then pass it to the filters below," it just means "deliver to Inbox"...I think Neil pointed that out to someone by email, but it still got me. Yes if an address matches in the white list, it goes straight to INBOX and by passes all other rules - this really sucks. It means you have to do all your filtering before the white list, and probably before your spam filter! It also means that any subsequent vacation folder will be skipped by mail matching your Whitelist.

[ Delivering into unsubscribed folders has been fixed by Graeme CMS ref was 469049 - Neil.]

From: also counterintuitive, and a different behaviour from procmailrc which would have broken any attempt to do this automagically. Trying to match a From: "Gordon Reid" <gordonr@staffmail.ed.ac.uk> with 'From: contains Reid' fails because it apparently only looks at the address inside the < > angle brackets. The 'help' button "Filter Rule :matches" examples kind of imply this but the full EUCS docs read as I'd think it.

To or Cc: It's quite easy to set up a double filter you think is for To: OR Cc: and forget to push the radio button switching "All of the following" to "Any of the following". So you get To: AND Cc: which won't work. There is no 'To or Cc:' together option as in the inf webmail interface.

Plussed Addresses: These now work, see StaffMailDifferences#Plussed_addresses ie neilb+foo@staffmail is OK.

-- GordonReid - 02 Feb 2007

Apparently, a deleted folder is not recreated by Seive if mail comes in and is supposed to be delivered to that folder.

The web interface/seive does immediately acknowledge folders created by other applications. I created a new folder by saving several messages (in pine) to a folder on staffmail. I then went in to filters on staffmail, to create a rule saying, "save any other mail with that subject to that folder", but when I clicked the drop-down folder list menu it wasn't there. So I had to type in the name, risking creating a folder, and wiping out what I'd just saved. Then the interface informed me that the folder already existed; so why didn't it put it on the menu for me?

-- GordonReid - 15 Feb 2007

You cant filter messages based on the matches in the body of the message. This may be addressed when/if staffmail upgrade to version 2.3 of Cyrus.

Performance

So far using webmail interface can be painfully slow, but Pine is reasonable.

-- GordonReid - 02 Feb 2007

imapsync

I've the required perl modules installed on doves.inf so that the imapsync script works. So far the basic usage is:

  imapsync --host1 mail.inf.ed.ac.uk --user1 neilb  --ssl1 --host2 imap.staffmail.ed.ac.uk --user2 neilb -ssl2 --noauthmd5

There are some problems when syncing messages with old style Unix mailbox message separator headers ie "From" followed by a space. eg: From mc@infREMOVE_THIS.ed.ac.uk Tue Sep  4 12:44:11 2007 +0100. The following incantation to imapsync seems to solve this by rewriting the =From = header to =X-From: =. The skip flags stop subsequent flags copying the message again, as essentially the message has changed during the move, a new header and a different size:

  imapsync --host1 mail.inf.ed.ac.uk --user1 neilb  --ssl1 --host2 imap.staffmail.ed.ac.uk --user2 neilb -ssl2 --noauthmd5
     --syncinternaldates --regexmess  's/^From ([^@]+@\S+\s\w+\s+\w+\s+[-0-9: \+]+.*)$/X-From: $1/m'
     --skipheader X-From --skipsize

It's been split over lines for clarity, you could use the --folder option to only sync affected folders.

Mass Move

How do we force the mass move of everyone that hasn't moved themselves to staffmail? I see two options:

  1. We just turn off our IMAP service and forward all the remaining UUNs to UUN@staffmailREMOVE_THIS. We'd need to get IS to disable any circular forwards this might introduce, and we'd have to decide what to with the mailfolders left on mail.inf. We could move them to users home directories, and possibly setup a separate IMAP service to serve the files from there, though AFS authentication may be an issue. This would probably be easiest and quickest, but most disruptive.
  2. We disable delivery to those users and IMAP, while we some how move all their mailfolders and filters to staffmail for them. This will require a lot of help from IS, it is likely to take a couple of days, during which the user would have no access to mail. Once the data was copied we could then reenable delivery, but that would just then forward to staffmail. Again this would be disruptive to users, but the end result would be more like before the move. All their mail in the one location, just on a different server.

26/9/2008 I've asked Keith Nicol about the logistics of option 2.

Other

If we do end up moving all (most) of our mail service to staffmail.ed, what provision can be made for people who want email addresses like something@someserviceREMOVE_THIS.inf.ed.ac.uk. This is something that we don't really cater for at the moment, should be technically possible, but could be complicated to do.

Topic attachments
I Attachment Action Size Date Who Comment
pngPNG OEColumns.PNG manage 15.7 K 12 Jun 2007 - 11:29 NeilBrown Outlook Express Select Columns screen shot
Topic revision: r29 - 16 Oct 2008 - 10:27:06 - NeilBrown
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies