Disaster Recovery considerations for COVID-19 related incidents

Specific

  • Cluster - cluster wise, we lose the forum we lose ilcc and all the individual machines, we lose KB we lose 9 landonia nodes, we lose AT and we lose the lot effectively. If we lose AT and KB we can cobble together a mini cluster from the bits that are left at KB but there's no cluster DR infrastructure as such.
  • Exam preparation - As it stands, the existing master, slaves and DR standins are spread between IF and AT. Maybe we should get one up and running at KB. Procedure for promoting examprep slave is documented.
  • PGResearch - sponge has some fairly critical stuff on it. Currently in IF and no repl slave. Would need some hardware and ~300Gb if it's to hold backups, too.
  • PGTeach - pgteach disk usage has dropped dramatically this session; it looks as if this hardware is underused, and the service could be restored on a modest VM (at some loss of performance/data security). If teacake were moved to an alternative site, it could be useful for database DR.
  • Teaching software - bear in mind some heavyweight packages (e.g. MATLAB) are probably only viable on DICE if there are enough machines for students to run in parallel.
  • Webmark - cold / warm spare is possible, but as it is simple to restore from backup perhaps of minimal concern. Benefits from lots of CPU/RAM/disk but doesn't require much. Could be hosted on a Theon VM but preferably on its own. Some forms have a dependency on pgresearch but could host its own smaller database if needed.
  • Theon - hosted in IF, DR replication server in KB. No DR UI / portal in KB. Capacity on the DR machine to keep all three running, but it should be set up as KVM for safer separation of services.
  • RT4, ISSRT ... ?

General

Server room access; IF, AT, KB

-- RichardBell - 25 Mar 2020

  • Coronavirus!! Together we'll beat this:
    corona.jpg
Topic attachments
I Attachment Action Size Date Who Comment
jpgjpg corona.jpg manage 160.4 K 25 Mar 2020 - 15:18 RichardBell corona
Topic revision: r3 - 06 Apr 2020 - 14:20:44 - TimColles
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies