Scanning for compromised machines

This is a report for Project 103 - Scanning for compromised machines.

The idea of this project is to investigate/prototype ways of detecting self-managed machines on our network which might have been compromised in some way. The self-managed machines involved will be running a mixture of Linux, Mac and Windows.

Note that 'compromised' != 'compromisable' (i.e. 'at risk of being compromised'). Ideally, however, one would like to scan for and detect machines in either state.


What do other people do?

To find out what other people do in this regard, a few colleagues were approached. Edited replies are in the Appendix. The summary is that detection of compromised machines is currently being done using analysis of NetFlow/sFlow network traffic data; but that nobody approached currently has an active system in place to scan for compromisable machines.


After a survey, three tools were investigated: Nessus, OpenVAS, and nmap.


Nessus ( is a commercial security scanner. It works by scanning target machines over the network, and comparing their responses with a regularly-updated database of signatures.

A brief evaluation was done using a time-limited demo version of the tool.


  1. The tool works. It did immediately find some potential vulnerabilities with machines here, including DICE machines.
  2. The vulnerability signature database is regularly updated.


  1. This software is intended to be operated and used via a human operator using a web browser. It is not designed to be automated and run in batch mode.
  2. The tool can be made to produce reports, but these are in fixed formats and would not necessarily suit our purposes. It might be possible to post-process some such reports in order to improve their usefulness.
  3. Like any commercial offering, this is a closed 'magic box.'
  4. The demo version is obviously limited in various ways, e.g. number of hosts scanned per subnet. How well the full tool would handle our actual collection of machines isn't clear. The likelihood is that this wouldn't be a problem in practice.
  5. Cost. The licence cost for each 'seat' is US$1500 pa.


OpenVAS ( is an open source vulnerability scanner which was forked from Nessus at the time (v3) when that project became a commercial offering.

Evaluation wasn't possible: this project tried but failed to install this software for evaluation. It appears that the software in its current state is developed and maintained on non-RedHat platforms - see Appendix B.


  1. Open source.


  1. Obviously developed on platforms other than RHEL and currently (at version 5.*) not supported for Scientific Linux.
  2. Development appears spotty.
  3. Not known whether or not the vulnerability signature is well-maintained.


nmap ( is an open source network scanner. It's not a vulnerability scanner, but it does have hooks for vulnerability detection scripts.


  1. Open source.
  2. Actively developed.
  3. Scanning is extensible through use of NSE scripts - either developed by the community, or by us. Available scripts include vulnerability and malware scanners..


  1. Not a vulnerabilty scanner per se. Needs work to get it to produce output that's useful to us.
  2. Not known whether vulnerability/malware signature scripts are well-maintained; not clear in practice yet how useful these scans are.
  3. Default version on SL6 (5.51) lags current version (6.*). Will have to look again at v6.


  1. This evaluation has proved difficult since:
    1. We haven't got any concrete aims against which to evaluate.
    2. We haven't got machines in 'known-bad' states against which we can evaluate the performance of any tools.
  2. If we want a scanning tool which runs on SL6 and which has a regularly-updated vulnerability database, then it might be worth buying Nessus for a year (cost: US$1500), using it in earnest, and evaluating its usefulness after that time.
  3. The current state of OpenVAS is not conducive to its use in our environment. If we want to even consider its use, then would first have to consider running it on a machine running something other than Scientific Linux. This is worth debating, but would represent a move from normal practice. I don't recommend pursuing this.
  4. nmap is worth pursuing but whatever we do with it will need more work. Producing information from nmap is no problem; the difficulty is producing accurate, targeted and relevant information. Results using nmap on our self-managed subnets aren't encouraging: OS detection accuracy seems poor; the only positive malware reports are false positives.
    1. Regular nmap reports which report changes in the configuration of machines on our self-managed subnets (ideally including the VPN subnets) might be a lightweight way to start doing something, and could be implemented.
    2. nmap vulnerabilty/malware reports could be implemented, provided experience here shows them to be accurate and useful. In testing that, hasn't proven to be the case.
  5. What we can or should do regarding machines on the University-managed wireless network is an unknown question.
  6. To detect compromised machines on our network, we should investigate the use of NetFlow and/or sFlow to monitor network activity in order to look for suspicious traffic patterns. (The data obtainable from either of these would be useful to us in any case: we currently have no real tools for network traffic visualization.) Project 98 is immediately relevant regarding sFlow on our core switches. The Netflow iptables module might be relevant for NetFlow on our external Linux routers. An unknown difficulty is obtaining up-to-date information on behavioural signatures which could be used to make a decision regarding good/bad behaviour.
  7. Other tools/resources to keep an eye on:
  8. As a 'late-breaking' item: After work on the current project had finished, we received the following communication from IS. It should obviously be followed up:
        Date: Wed, 10 Apr 2013 13:14:14 +0100
        From: Alan Boyd
        Subject: Security scanning

        The University/IS have bought a license from ESISS
        to allow us to run penetration testing against University systems. This
        comprises both a network scanner and web application scanner.

        We're looking to delegate this service out to those who want it, and
        your network is a well defined unit.

        If you are interested in trying out this service, I can have an account
        set up for you ... which allows scans against your
        IP address ranges and whatever web URLs you think it would be useful to
        look at.

        FWIW the scanning service essentially appears to be a fancy 
        front-end onto Nessus, but tests we have run have given useful results.

Summary and recommendations

  1. In response to the 'late-breaking news' from IS: before we spend money on our own copy of Nessus, we should now first investigate what the IS-provided penetration-testing system can offer us.
  2. We should investigate the use of NetFlow and/or sFlow - see Project 98.


A. What do other people do?


"We don't do much. We secure the configurations of our own managed boxes and services, but that's prevention rather than detection.

"On detection, we have some firewall rules designed to prevent some of the common Windows worms spreading; we monitor the log files and occasionally spot outbreaks."

"The main tool we tried was Nessus but I didn't think it was actually worthwhile. Once most of the more vulnerable systems were running Windows Managed Desktop, the number of infected systems dropped dramatically, and the amount of false positives was counterproductive.

"I did consider trying to cross correlate with analysing netflows across the boundary. E.g. looking for local systems that access more remote systems than usual, or using unusual ports."

A N Other site

"Our virus detection technology is based on NetFlow, the Cisco flow accounting feature.

"We have a netflow 'collector' which receives the netflow data from our routers. Analysis is run against the data. Virus-infected machines show up as flows which have the same source IP, same destination port, and differing/sequential/random destination IPs."

B. State of OpenVAS platform support

For example, a quote from the maintainer of OpenVAS, on

Date: Mon, 4 Jun 2012 11:09:08 +0200
List-id: OpenVAS discussions 

>  fact is the openvas is the only application making
>  troubles on some recent systems and the developers
>  seems not to be interested in setup a small F16
>  VM to debug this :-(
>  with F16 openvas4 did run fine, on F16 OpenVAS4 AND OpenVAS5 are dead

it is not a lack of interest, it is lack of resources.

Developers with Fedora background welcome (we don't have any yet)!
Submit your patches to fix the issues!

This being said, I was told the Atomic and OBS packages
do work for FC16. Either they actually don't work or
they do a magic trick (via .spec files).
Or maybe it is something about the hardware architecture or dependent

However: all sources are there to check/debug/patch ;-)

-- IanDurkacz - 05 Feb 2013

Topic revision: r11 - 30 Apr 2013 - 15:36:46 - IanDurkacz
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies