Computational Resources for ANC

Support for desktop DICE machines can be found at the DICE Computing page and through the DICE Computing Support Form.

Many staff and students in ANC do computationally intensive work, requiring hours or days of computer time, and large amounts of disk and memory. Besides your own workstation, the main Edinburgh-based resources available to all ANC members are:

• Informatics compute servers (staff.compute.inf.ed.ac.uk, student.compute.inf.ed.ac.uk; 32 cores total)
  • Strengths: 48GB of main memory each, no need to mess with job control, uses DICE, free for all to use
  • Limitations: no control over load balance, student.compute is free for all to use
  • Info: Support FAQ question on compute servers
• MachineLearningComputeServers
  • Strengths: Only available for ML students/staff
  • Limitations: Only available for ML students/staff
• ANC servers
  • Circa 2014: Salmon - 64 slower cores (AMD Opteron) 256GB RAM, 4TB disk (in /dataN directories).
  • Circa 2014: Trout - 16 faster hyperthreaded cores (Intel Xeon), 128GB RAM, 4TB disk in /data1 directory.
  • (venus, mercury; 4 CPUs total)
  • Strengths: no need to mess with job control, uses DICE, dedicated to ANC
  • Limitations: no control over load balance, few CPUs total
  • Each machine has 4 GB RAM, and 2 CPUs (hyperthreaded to appear as 4).
  • Usage of the compute servers machines is restricted to ANC academic staff, research staff, and PhD students.
  • To preserve a reasonable response time on the command line, please nice all background processes you start on the compute servers. For instance, rather than "simulate -time long > out &", do "nice 15 simulate -time long > out &". See man nice for more details. Similar guidelines apply when running processes on machines owned by other ANC members — please use nice to avoid interfering with their work.
  • Further Info: Amos Storkey

• Large-memory machine (heffalump.inf.ed.ac.uk; 16 cores total)
  • Strengths: 64GB of main memory, 64-bit processor, uses DICE
  • Limitations: no control over load balance
  • Access is available on request (to computing support)
• Condor: submits jobs to run on idle Informatics machines (374 cores total as of 7/2010)
  • Strengths: no cost, many machines available, uses DICE
  • Limitations: not all jobs complete, timing is haphazard, AFS home directory access difficult or impossible
  • Info: Condor Wiki
  • Condor now shut down?
• Informatics GridEngine cluster (hermes; 80 cores total)
  • Strengths: good interconnect, solid MPI support
  • Limitations: heavily used
  • Info: DistributedComputing Wiki
• ECDF (eddie.ecdf.ed.ac.uk; 1456 (as of 5/2010) CPUs total)
  • Strengths: many cpus, very good interconnect, big disk, 25% of capacity reserved for free user, GPU option
  • Limitations: requires registration, needs payment if you want backups or priority service, can be crowded
  • Info: ECDF site

Accessing your DICE home directory from non-DICE machines

If your DICE home directory is on AFS (which is probably true for everyone now), you can access it at /afs/inf.ed.ac.uk/user/s01/s0123456 after obtaining the appropriate credentials. Here is an example of obtaining credentials on eddie.ecdf.ed.ac.uk:

[eddie]$ kinit s0123456@INF.ED.AC.UK
[eddie]$ aklog

To make it simple to access, you could create a shortcut, e.g.:

ln -s /afs/inf.ed.ac.uk/user/s01/s0123456 ~/DICE

Every time you log in (and every 18 hours), you will have to renew your credentials. (18 hours is the limit set by DICE.)

Disk Space

There are two places where we have group disk space. On AFS there is some space for machine learning people

/afs/inf.ed.ac.uk/group/ANC

Also on Eddie there is

/exports/informatics/inf_ianc

Each of them currently have 1TB of space.

Matlab

For people using Matlab, there are some special considerations, outlined on our MatlabComputing page.

Other resources

There may also be systems available from the Edinburgh Parallel Computing Centre (EPCC) or from the National e-Science Centre (NeSC). Neuroinformatics DTC students also have DTC-specific resources.

If any of this information is out of date, or if you have specific tips for the various systems, please edit this page to fix it!

ANC Computing Workshop, 30 October 2007

Please see the slides attached below. Thanks to all of our presenters!