Plone Web CMS Service Management

2019 - We are working to retire the use of Plone within Informatics, so no new sites should be created in Plone.

Details on how to manage the Informatics (Plone) Web Content Management Service, http://wcms.inf.ed.ac.uk/. The first thing to know is that Plone is something that sits on top of the web frame work Zope, so you'll see it mentioned throughout the following.

Note: the path /opt/wcms/ is actually a symlink to /disk/data/wcms/

Basics

  • The WCMS is hosted on cigarvm.inf in the Forum server room
  • It is mirrored (currently) to stanier where it gets copied to tape. A special script is run by cron and dumps the live Zope DB in a format suitable for backing up and restoring.
  • We are running Plone 3.2.3 on port 8080, but that only answers to localhost traffic.
  • The apache on wcms.inf is configured to Proxy the requested sites to localhost:8080
  • No optimisation has been done - yet.
  • Though we have Cosign access to most sites, some of the first sites are using Plone's own authentication.
  • There used to be separate zope and plone RPMs supplied by Scientific Linux, they nolonger do that so we've built our own plone RPM that provide zope and self contained python-2.4 in /usr/lib64/plone.

Setting up a new Plone Site

There are two parts to setting up a new Plone managed web site:

  • create the actual Plone site within Zope
  • configure apache to access the site

Creating the Plone site within Zope

The Zope management interface is http://localhost:8080/ on cigar, to connect to it you'll want to run firefox on cigar with =firefox --no-remote= so that it doesn't just open a new window on your existing running firefox on your desktop. Once you've got the local firefox running, browse to the above URL. Or you can use tunneling/port forwarding techniques to get your desktop browser to talk to cigar's 8080.

We'll also assume you want to create a new site based on the "Institute In A Box" (IIAB) template, which will (once you've setup the apache bit) give you a Cosign authenticated site.

  • Click on the "Zope Management Interface" (ZMI) link and give the username and password that you should know! Note this is an HTTP connection, not HTTPS, but as it is localhost traffic it won't be go across the network, and your X session should be tunneled via ssh.
  • Once logged in to the ZMI, you'll probably find you need to scroll to the bottom of right hand pane to see various buttons, including the Import/Export one that you're looking for.
  • Click on the Import/Export button and from the bottom half of the page that appears, select institute-template.zexp and ticking the "Retain existing ownership" option, then clicking "Import".
  • You'll now be back to the list of Plone sites, look for the "template-site" tick the checkbox beside it and then click the Rename button (remember you may have to scroll down) and rename the site to something suitable for the new site, eg dice
  • Click on the new site in the list on the left hand side and then the Properties tab on the right, and enter a suitable Title and Description as above.

To make it simpler to manage the site in future, you'll now want to make sure you have admin access to the site. Similarly for the person requesting the new site.

To do this click on the Contents tab then

  • acl_users -> source_users and click on (Add a user). Fill in your UUN for both the id and login name. Make up a random password (though you'll need to enter it twice).
  • Then go to <site> -> acl_users -> portal_role_manager and click on the '?' (question mark) in the Assignments column of =Manager. Search for your UUN and then add it with the right pointing arrow to the list of managers.

![you could remove dsisu's entries if I've not tied them up yet!]

To test this has all worked, go to the URL http://wcms.inf.ed.ac.uk/zope/<site-name> (dice in this example).

You should see the default template site, click the "login" link on the upper right and you should be automatically logged in via Cosign. And if you've been correctly identified as having admin rights, you should have a Site Setup link on the top right of the page. This will bring you to the Plone Control Panel for the new site, where you can do the last bits of the Plone setup.

![you can logout and close the firefox you had running on cigar]

Configure the site with the Plone Control Panel

First you'll see a warning about needing to configure the Mail:

  • Then click on the Mail link and fill out suitable values for the Site From Name and the Site From Address, again the requester of the site is probably the correct thing to put in here. Then click Save.
  • Click on the Security link (on the left) and make sure the check boxes are appropriate. At the moment I'd say none should be ticked.

The Manager Role allows the users to use the Plone Control Panel, which also lets you access the ZMI for that site.

Assuming you don't want the site to remain with the URL http://wcms.inf.ed.ac.uk/zope/<site id> but rather as wcms.inf.ed.ac.uk/dice or www.project.com. Then you need to delve into configuring apache. See next section.

Configure Apache

There are basically two types of URL you'll want to point at your new Plone site, ones of the form wcms.inf.ed.ac.uk/dice/ which are straight forward, and ones of the form www.emime.org, which are not so straight forward.

A wcms.inf URL

Edit /opt/wcms/apache/wcms.conf and /opt/wcms/apache/wcms80.conf to add (as a minimum) the appropriate RewriteRule. Note that the RewriteRule contains string references to the site name (as it appears in the URL) and corresponding location (as it appears in the Zope DB)... these can be different. eg

RewriteRule ^/pgrguide(|/.*)$ http://127.0.0.1:8080/VirtualHostBase/http/wcms.inf.ed.ac.uk:80/pgr-student-guide/VirtualHostRoot/_vh_pgrguide$1 [L,P]

Note this example (taken from "wcms80.conf" the HTTP site config), the zope id for the site is pgr-student-guide, but the URL we are creating is wcms.inf.ed.ac.uk/pgrguide/. The "wcms.conf" (HTTPS) is similar but for the port changes to 443 and http->https.

If the site manager is happy to administer all other access controls, then no other configuration is required.

A new hostname

  • Edit the live/plone-wcms-server.h header file and use the PLONE_SITE macro to create a new site.

You also need to create /opt/wcms/apache/<tag>.conf and /opt/wcms/apache/<tag>80.conf and files as the main apache config will include those. See the others as guides.

* There are some ordering issues with when each component configures (x509, cosign and apacheconf), so you may have to run configure apacheconf again. Check the logs.

Note that's the apacheconf component, not the apache one, so to restart apache you need to do om apacheconf restart, when and if you make changes to the conf files in /opt/wcms/apache/

Backup and Restore (Disaster Recovery)

Standard Backup Procedure

There is a locally-written backup script, /opt/wcms/zopebackup/dobackup, called nightly by cron. This script rsyncs the top level files in /opt/wcms/zope/ and the subdirs [bin|products|var/instance|var/log] (ie things that are unique to that instance) into the /opt/wcms/zopebackup/=*INSTANCE*=/instance/ directory, and calls the repozo.py script to backup the var/filestorage/Data.fs database into /opt/wcms/zopebackup/=*INSTANCE*=/repozo-backup/.

This makes sure the Data.fs file is in a consistent state that can be restored. This zopebackup directory is then rsynced nightly, and hence copied to tape.

This script has been updated to backup all the instances specified in /etc/sysconfig/plone.

Restore Procedure

To restore the data (or copy it to another machine for testing), first make sure the plone RPM is installed on the recovery machine, and that it is running as an Apache web server.

The simplest way to setup Zope/Plone would be to include the following in the machine's profile:

    #define WCMS_DATA_DIR /opt/wcms

    !file.files            mADD(optwcms)
    file.file_optwcms      WCMS_DATA_DIR
    file.tmpl_optwcms      /path/to/wcms
    file.type_optwcms      link

    #include <dice/options/plone-wcms-server.h>

specifying the location of the root of your data with #define WCMS_DATA_DIR, and the wcms directory via the file.tmpl_optwcms resource.

Once the profile has compiled and all RPMs are installed, initialise a new zope instance with (as root):

    /usr/sbin/mkploneinstance -d WCMS_DATA_DIR/zope -u zope -p changeme

It is probably a good idea to run the above (with WCMS_DATA_DIR instantiated) and make sure it works before restoring the data, although any initial admin user and passwd ("zope" and "changeme" above) will be replaced when restored from the backup (so it doesn't really matter what username and password you set).

Check permissions on WCMS_DATA_DIR/zope - all files should be zope-owned.

Start the zope by hand with systemctl start plone and browse to http://localhost:8080/ - this should give you the "Zope Quick Start" page.

Once the recovery machine is successfully running a vanilla zope/plone, then stop zope and restore the data.

The data comes from the mirror location /opt/wcms/zopebackup/_opt_wcms_zope/ (currently in stanier:/disk/rmirror33/cigarvm). The restore is a two-stage process:

  • restore file structure (copy zopebackup/_opt_wcms_zope/ - at least the "instance" and "repozo-backup" directories)
  • restore DB (run repozo, which re-creates the Data.fs file)

Copy the zopebackup/_opt_wcms_zope/instance/ directory into the new /opt/wcms/zope/ (overwriting any that may have been created when you were testing). If you restore the data to a different location (say /disk/restore/wcms/), then you'll need to update the /opt/wcms/ symlink (resources) to that new location.

[for now there's also psycopg2-2.0.14.tar.gz in the backed up data that needs to be copied to /usr/lib64/plone/buildout-cache/downloads/dist until we find a proper fix for the buildout failing]

Once those files are in place, run the buildout script (as root) to make sure things are consistent.

  cd /opt/wcms/zope
  ./bin/buildout

Now restore the Data.fs file with the ./bin/repozo script and copy that into /opt/wcms/zope/var/filestorage/ eg:

    cd /opt/wcms/zope
    ./bin/repozo -R -r <path_to_repozo-backup-dir> -o /opt/wcms/zope/var/filestorage/Data.fs

(Note that the <path_to_repozo-backup-dir> is wherever the "repozo-backup" directory was restored to.)

Check ownership - it should be user & group "zope". It may be simplest to run chown -R zope:zope /opt/wcms/zope

Now start zope again. For the first run after a restore, it's probably a good idea to run zope in the foreground:

    /opt/wcms/zope/bin/plonectl fg

which sends error/warnings to STDOUT so you can see if there are any problems.

If, after restoring things, zope is complaining about a corrupt zope/var/filestorage/Data.fs file (the main zope database), which it shouldn't be, then there's the possibility that the Data.fs file that was mirrored may work. So you could try copying it back into place. But that's like using a mirror of a mysql database, rather than restoring from a mysqldump.

Ugrading Plone

Try this on a copy of the live service before doing it for real! See above on how to restore a copy of the live service.

  • Stop zope (systemctl stop plone)
  • Upgrade Plone RPM to new version
  • Start zope (systemctl start plone), perhaps keep an eye on the /opt/wcms/zope/var/log/instance.log to make sure it starts OK.
  • Go to the Zope Management Interface(ZMI) and for each Plone site, you should see a red exclamation mark against the portal_migration module. Click on this and follow the instructions. I use the "Dry run" option first, just to make sure.
  • That should be it. It is a pain that we have to do this for each site, perhaps I'm missing something.
  • If there are problems, it is likely to be with any extra products we've installed, hence testing the upgrade on a copy of the live service before doing it for real.

Cosigning

Here are some initial notes on taking a Plone site setup as above and turning it on Cosign authed. Note that there are still some rough edges, particularly the login and logout mechanic and error pages when access is denied.

  • First you need setup the Cosign part of the web service. See the other vhosts sections in wcms-server.h for setting up the vhost. If the site is hosted under the URL http://wcms.inf.ed.ac.uk/ then this part is(will be) unnecessary as it is already done.
  • If you settings something up for a site like www.emime.org which is hosts on cigar, then you'll need to setup an HTTPS site for www.emime.org, which will require an additional IP address to that its HTTPS certificate can be assigned to it. Again look at existing apache config files, but you need something like:
DocumentRoot /opt/wcms/html
ErrorDocument 401 /errors/emime401.html
ErrorDocument 403 /errors/emime403.html
<Location / >
  CosignProtected On
  CosignAllowPublicAccess Off
  AuthType Cosign
  AuthGroupFile   /opt/wcms/apache/htgroups
  Require group emime
</Location>
<Location /errors>
  Require all granted
  RequestHeader unset X_REMOTE_USER
# This doesn't quite work as we want as the page includes other HTTPS URLs
# which will still be denied. We really want to redirect back to a non-HTTP
# url, but than can cause other problems
</Location>
RequestHeader set X_REMOTE_USER %{REMOTE_USER}s
# Virtual Monster Via Proxy
ProxyRequests off
ProxyVia      on
ProxyPass /errors !
ProxyPass / http://127.0.0.1:8080/VirtualHostBase/https/www.emime.org:443/emime/VirtualHostRoot/

  • Then you need to do some tweaking to the site - not necessary if you used the template method above to create the site
    • Go to Site Setup -> Add-on Products and add "apacepas" and "AutoMemberMakerPasPlugin"
    • Once they are installed, go to the ZMI for this site and acl_users -> apachepas and untick the "Strip domain names from usernames" and save the changes.
    • Again in the ZMI goto portal_actions -> user -> login and change the URL (Expression) to python: request.ACTUAL_URL.replace('http://', 'https://', 1) and save changes.
    • then portal_actions -> user -> logout and change the URL (Expression) to python: request.ACTUAL_URL.replace('https://', 'http://', 1) and save changes.

  • You'll also want to create the minimal error pages in /opt/wcms/html/errors.
  • How you restrict access to just DICE users or DICE and iFriend depends. You can use capabilities (entitlements) if only DICE users need access, see the .conf of other sites. For sites that need iFriend access, then use the htgroup fudge for now.

Products

If you decide to install an additional Product for someone/reason, then always test it out on a copy of the service first. I'd also say (ideally) only install released stable versions, not betas or RCs.

Most Products can be tested by running ./bin/plonectl test -s ProductToTest from the instance directory.

If a Product has dependencies, make sure they are not already part of the default Plone install before adding another version in the instance Product directory.

--Main.NeilBrown - 1 Oct 2013

Reviewed - 11/3/2019

Topic revision: r22 - 11 Mar 2019 - 16:38:06 - NeilBrown
DICE.ServicesUnitPloneWCMSManagement moved from DICE.ServicesUnitWCMSManagement on 19 Jul 2010 - 11:47 by NeilBrown - put it back
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies