TWiki> DICE Web>PandemicPlanning>PloneTop5 (revision 6)EditAttach

Plone "Top Five"

Plone (or wcms.inf.ed.ac.uk) is the new Web Content Management System that we use to host various project web sites and, increasingly, more Informatics-based content. Currently hosted on cigar.inf, it is built using the Zope web application server written in Python. The hardware is physically located in the AT server room.

Although Zope provides its own HTTP server, Zserver (so Apache isn't actually needed), we do put Apache in front of the Zserver to give us Cosign authentication and other functionality. The Zserver running on cigar.inf only listens to port 8080 on the loopback device (127.0.0.1), so browsing to http://cigar.inf.ed.ac.uk:8080/ shouldn't (and better not!) work.

Apache handles the various requests for the virtual hosts, eg www.emime.org, www.not-a-service.inf.ed.ac.uk, etc, and proxies the request to localhost:8080/.... It also passes authentication information (which is taken as gospel by the Zserver), which is why you don't want the outside world to be able to talk directly to the Zserver on port 8080.

If you do need to talk directly to the Zserver, then you can log into cigar.inf and run firefox locally to talk to localhost:8080. I do

firefox --no-remote localhost:8080

Things of Note

The zope component configures /etc/sysconfig/zope to set the INSTANCE to /opt/wcms/zope/ which is actually a symlink to /disk/cigar1/wcms/zope/ (this is all the data and config for the zope service, and is what needs to be restored following a disaster).

There have been problems reinstalling/upgrading the zope rpm, as it knobbles the default config /var/lib/zope/etc/zope.conf . This should not matter now that we have the component set to run zope out of a different area, but in case it does, the INSTANCE/etc/zope.conf file is where we configure it to only listen on 127.0.0.1), otherwise it listens on all interfaces (which is bad). There should be a zope.conf.wcms file, which is a copy of the correct zope.conf file, should you need to repair it.

You can't just upgrade the Plone RPM and expect things to keep working - you need to go through additional steps, mainly the "portal_migration" tool for each site in the ZMI. If you must do this, try it on a clone of the wcms server first.

Cigar has multiple virtual network interfaces defined for all the HTTPS certificates we require for the cosign authenticated sites. These are on the etherbond interface bond0.

Apache is configured using the apacheconf component, which expects the included apache config files to live in /etc/httpd/conf.d/. We maintain our apache config files in /opt/wcms/apache/ (a symlink to /disk/cigar1/...) so it gets mirrored - which means that we have to symlink (via the file component) from files in /opt/wcms/apache/ to /etc/httpd/conf.d/.

How it works

See ServicesUnitPloneWCMSManagement for a bit more detail, but simply:

  • a new Plone site is created in the Zope via the Zope Management Interface (ZMI) and given an id eg "dice". You need to know the zope admin username and password.

  • At this point (due to the way Apache is set up to proxy zope), the site is now live at the URL wcms.inf.ed.ac.uk/zope/dice/

  • If you want a friendlier URL, eg
    • wcms.inf.ed.ac.uk/dice/, then you need to update the two wcms.conf and wcms80.conf apache config files with the required Proxy and potential access restrictions
    • or something like www.dice.org or www.dice.inf.ed.ac.uk needs more work:
      • Create another new IP for cigar (needed for X509 HTTPS cert)
      • create the new X509 certificate.
      • create a new virtualhost (via LCFG, using the macros, as we are using apacheconf) and the corresponding config file(s) with the necessary Proxy and access restrictions.
  • If you do want to restrict access to certain groups of people (even before zope/plone does its own authorisation) then you can do this using various apache techniques, the most common being the require group ... where the group is defined in the apache/htgroups file.

Apache is partly configure via the apacheconf component and partly by the config files in /opt/wcms/apache/ (really /disk/cigar/wcms/apache).

Common Problems

HTTPS funnies

The only regular problem we have at the moment is that for some reason some of the HTTPS sites are mis-caching content, and so when your browser asks for a JPG, it gets the wrong JPG, or a style sheet instead. Or it asks to fetch an HTML doc and gets a GIF! There doesn't seem to be any rhyme or reason about when/why this happens, though we think Apache is to blame, rather than Zope.

You could try restarting apache (om apacheconf restart) and then getting the user to "shift reload" the page.

Or have them try accessing the site via an alternative wcms.inf.ed.ac.uk/zope/<site-id> URL. Note this will only work for DICE users, not iFriends, and may have further restrictions as defined in the apache config.

We have yet to have any reports of this mis-caching on HTTP sites.

Sites just not there

This has only happened if the network is down (someone else's problem) or if apacheconf hasn't restarted following the Sunday morning logrotate restart (very rare). However if networking is fine, then check if:

  • apache is running - grep for httpd processes on cigar
  • Zserver is running - grep for zope owned proceses on cigar, you
should see something like:
zope      7057     1  0 Aug11 ?        00:00:00 /usr/bin/python /usr/lib/zope/lib/python/zdaemon/zdrun.py -S /usr/lib/zope/lib/python/Zope2/Startup/zopeschema.xml -b 10 -d -s /var/lib/zope/var/zopectlsock -u zope -x 0,2 -z /var/lib/zope /var/lib/zope/bin/runzope
zope     19194  7057  2 Sep17 ?        02:46:11 /usr/bin/python /usr/lib/zope/lib/python/Zope2/Startup/run.py -C /var/lib/zope/etc/zope.conf

You can just try restarting apache if it isn't running, using the component. If it doesn't start, try looking in the /var/lcfg/log/apacheconf and apacheconf.error files for clues.

If you want to try restarting zope, do:

  om zope stop
  om zope start

om zope restart should work as well, but I like to give things time to settle down before starting it up again.

Zope takes a while to start up (this can be 2-3 mins), so keep an eye on the log file /opt/wcms/zope/log/event.log. When it is ready it will say "INFO Zope Ready to handle requests", or give error messages saying why it didn't start. Good luck if that's the case! Though the only time I've seen that happen is when there's been a change to the ../zope/Products/ directory, eg something got deleted or there's been a major version change of something. Try reverting back to the last known good state from the backups/mirrors.

I should also say the the Zserver logs its HTTP requests like apache does, but to the file /opt/wcms/zope/log/Z2.log. Remember that /opt/wcms/zope/ is a symlink to /disk/cigar1/zope/, which does sometimes lead to confusing warnings about "duplicate products detected" from the alternative path.

Disaster recovery

Assuming it has all gone bang, and you need to rebuild the service, then life will be a lot simpler if the new hardware re-uses all the same DNS names and IP addresses as the current cigar.inf, otherwise you'll have to track down all the sites that have non-wcms.inf.ed.ac.uk URLs, eg www.emime.org, www.classic-project.org, and update the DNS to point at the new name/IP addresses.

The service isn't completely wrapped-up in dice/options/plone-wcms-server.h, so you will also have to look at the current cigar profile and make some appropriate changes to fstab, various virtual network interfaces and #defines.

Assuming that you have now re-installed cigar, or have an equivalent machine, you should have a vanilla zope instance and a broken apache - as the data/config that is the service needs to be restored. You need to restore what was /disk/cigar1/wcms/ from the mirror (currently on lammasu rmirror16/wcms). If you restore it back to the same path, ie /disk/cigar1/wcms/ then that should be it. If you restore it back to a different path, ie /disk/restore/wcms/, then you'll need to update the /opt/wcms/ symlink to /disk/restore/wcms/ .

There is currently around 50GB of data in /disk/cigar1/wcms/. There's other stuff in /disk/cigar1/ (nutty stuff and archives) - these are not essential, and were only "temporary".

Note there is also a comment from the Infrastructure unit about the manually-installed SSL certificates: http://www.dice.inf.ed.ac.uk/units/infrastructure/Documentation/cybertrust-certificates.html

If, after restoring things, zope is complaining about a corrupt zope/var/Data.fs file (the main zope database), then it could be because it wasn't in a consistent state at the time of the mirror. There is an extra dump done of the data (which is also mirrored) which does ensure a consistent backup state (a bit like mysqldump) which could be used to restore the data. See /usr/lib/zope/bin/repozo.py and the files in /opt/wcms/zopebackup/

That's it.

-- NeilBrown - 22 Sep 2009

Edit | Attach | Print version | History: r10 | r8 < r7 < r6 < r5 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r6 - 07 Mar 2011 - 09:03:56 - RogerBurroughes
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies