Pandemic docs for "Traditional Web"

This will cover the basics for any managed apache based web server, what to look for and where to start. Some familiarity with Apache and DICE is assumed.

Docs and useful locations

First off, all our servers should be running Apache 2.4 now. The official docs are online at http://httpd.apache.org/docs/2.4/

Some useful logs to check:

• /var/lcfg/log/apacheconf (component log)
• default apache logs /var/log/httpd/{access_log,error_log}
• many web servers will have reconfigured their logging location, you'll have to look at the lcfg, or parse the apache config to find out where. Many services unit servers log to /var/lcfg/log/apacheconf.SERVICE.{access,error}

The default apache config is /etc/httpd/conf/httpd.conf but that will include other files, which may or may not be under LCFG control. Some web servers override that default location, but not so many now. Check:

neilb> qxprof apacheconf.serverroot
serverroot=/etc/httpd

Site not reachable

If a web server is not responding, then the basic things to check are:

• Is the server up?
  • use nslookup host to find the hostname does it ping
  • are there network issues, rather than web server issues?
• Is the apache httpd daemon running?
  • login and look for the httpd process ps -ef | grep httpd
  • if not, then are there messages in /var/lcfg/log/apacheconf saying why?
  • try restarting apache/httpd om apacheconf stop then om apacheconf start
  • if it doesn't start, check the logs to see if a reason is given.

If there are reports of syntax errors, then (as root) try

• apachectl configtest

If the configuration is broken (and previously wasn't) then check what profile changes have been made recently. Does the profile have an error?

If restarting apache and fixing any profile errors hasn't solved things, then you are into the realm of debugging apache config.

Debugging

If the site is running, but you're not getting the result you expect, then some basics to try.

I find that using "curl -i http://problem.site.inf.ed.ac.uk/" is a useful command line way to check basic web function. Check logs, particularily apache error logs after you do your "curl".

If all the config looks good, then perhaps it's the content that has changed? You'll need to track down the appropriate documentroot, and check for any recent changes, particularily .htaccess files.

If the server has multiple vhosts, and assuming they are logging to different files, then check the expected vhost is responding to HTTP requests. eg try to fetch a unique URL (doesn't have to be a valid one) and make sure that request appears expected access log.

If the problem is in actually accessing the document root, is it even mounted? If it is in AFS space, then have ACLs or user ids changed?

HTTPS problems

If the problem is with the HTTPS certificate, check it's expiry date. The x509 component should regenerate automatic ones as they approach their expiry date (or you can force it by removing (or moving to one side) the .crt file and then running om x509 run). If it isn't an autogenerated cert, then you'll have to get a new one from wherever the last one came from, which is beyond the scope of this document.

-- NeilBrown - 27 Feb 2019