NX Service Admin Notes

List active sessions

There are 2 NX servers - nx.inf.ed.ac.uk and staff.nx.inf.ed.ac.uk.

You can list all active sessions like this (as root):

nsu
/usr/libexec/nx/nxserver --list

Terminate a session

This is a two stage process. Note the session ID of the session to be killed (the last field in the list above) then search for processes with that session ID:

ps axuww | grep session_ID

Ignore your grep process but kill the other two with kill -9 and the process IDs (the second item of each line of the 'ps' output)

kill pid pid

Find out who's using the most space in /tmp :

nsu
cd /tmp
du -sx *.* | sort -n

After terminating sessions, kill any remaining processes belonging to a particular user:

nsu
killall --user username

When /var/nx is 100% full

NX session information is stored in /var/nx. Most of the time there's a lot of free space there, but occasionally a long-running session can go crazy and fill the partition. When the partition is full, NX can't function properly, so that for example it may be difficult to start a new session. However, often just one or two processes are responsible for filling the partition. Once you find and kill them, the problem sorts itself out.

1. First find out who's using the most space in /var/nx :

   ssh nx
   cd /var/nx
   nsu
   du -sk .??* | sort -n | tail
   

   Look for the username at the bottom of the list. That account often owns a startlingly high proportion of the total.
2. Look for nxagent processes owned by that account (biguser in this example):

   ps axuww | grep biguser | grep nxagent
   

   If the process was started a month or more ago, kill it. If there are multiple processes, kill any that were started a month or more ago.
3. df /var/nx to see if you've solved the problem. It can take several seconds for the files owned by the killed session to be deleted from /var/nx.
4. If the partition is still quite full, another session may be running rampant, so repeat this process.
5. If you're still looking for more sessions to kill, find out which accounts are running multiple sessions:

   ps axuww|grep nxagent|awk '{print $1}'|sort | uniq -c|sort -n
   

   Look at the bottom of the output for the most prolific users. Any user with several simultaneous sessions running has probably abandoned some of them, so these can be killed. Check the dates on that account's sessions with:

   ps axuww | grep nxagent | grep account

   and kill the oldest of them.
6. As before, keep doing this until /var/nx has enough free space.

If new sessions can't be started

Sometimes so many old sessions and user processes are lying about on the machine that no new sessions can be started. The solution is to clear out the old sessions and processes. One quick way to do this is to reboot the machine, but failing that here's what to do: Looking for the oldest user processes using:

ps axuww --sort=start_time|less

For each user with an old process I've been checking that all its processes are old:

ps axuww | grep $user

If they are, and if the user has a registered NX session, I terminate it:

/usr/libexec/nx/nxserver --list $user
/usr/libexec/nx/nxserver --terminate $user

If it doesn't have an NX session I just kill its processes:

killall --user $user

A bit of this sort of thing and it's soon possible to start new NX sessions once more.