Parsing Infrastructure Vulnerability Scans

by

I had this idea for demonstrating how to represent vulnerability data, but knew I couldn’t use the actual data I work with on a day-to-day basis for obvious reason. So I set out to find a sample Nessus scan file, and load it in a human readable way. I hope to reference this in many future updates, but for know here is how you can take a .nessus file (which incidentally XML), and load it into a database to be useful for querying later. I did this in a Python Jupyter notebook, which you could fine here if you were so inclined. I also added a line to my “blog ideas” notebook to figure out a way to pull the markdown from git directly and embed that..

Import necessary libraries

Maria DB will let us save our processed file into a database to use later
xml.etree.ElementTree will help us parse the nessus XML file
requests will allow us to grab the sample xml from github
sys helps with error handeling

import mariadb as db
import xml.etree.ElementTree as et
import requests
import sys

Finding a sample file

I found this sample nessus file on github to use. This is what the raw output looks like. We’ll need to “parse” this to something more human-readable before its useful.

<ReportItem> port=”445″ svc_name=”cifs” protocol=”tcp” severity=”0″ pluginID=”10785″ pluginName=”Microsoft Windows SMB
NativeLanManager Remote System Information Disclosure” pluginFamily=”Windows”>
<description>It is possible to get the remote operating system name and version (Windows and/or Samba) by sending an authentication request to port 139 or 445.</description>
<fname>smb_nativelanman.nasl</fname>
<plugin_modification_date>2013/06/25</plugin_modification_date>
<plugin_name>Microsoft Windows SMB NativeLanManager Remote System Information Disclosure</plugin_name>
<plugin_publication_date>2001/10/17</plugin_publication_date>
<plugin_type>remote</plugin_type>
<risk_factor>None</risk_factor>
<script_version>Revision: 1.40 </script_version>
<solution>n/a</solution>
<synopsis>It is possible to obtain information about the remote operating system.</synopsis>
<plugin_output>The remote Operating System is : Windows Server 2008 R2 Standard 7601 Service Pack 1
The remote native lan manager is : Windows Server 2008 R2 Standard 6.1
The remote SMB Domain Name is : RB
</plugin_output>
</ReportItem>

url = "https://raw.githubusercontent.com/DefectDojo/sample-scan-files/master/nessus/nessus_v_unknown.nessus"

download = requests.get(url).text

tree = et.fromstring(download)

Create a data array

For each “report item” in the xml, we want to grab certain characteristics. In the XML tree, the “report Items” are a subset of a host/ip address. We’re going to create a tuple for the ip address, and vulnerability information, and add that to an array of the accumulated data.

data = []
for host in tree.findall('Report/ReportHost'):
    ipaddr = host.find("HostProperties/tag/[@name='host-ip']").text

    for item in host.findall('ReportItem'):
        risk_factor = item.find('risk_factor').text
        pluginID = item.get('pluginID')
        pluginName = item.get('pluginName')
        port = item.get('port')
        protocol = item.get('protocol')  
        data.append(
          (ipaddr,
          risk_factor,
          protocol+" "+port,
          pluginID,
          pluginName)
        )
print(data[0:5])

[('10.31.112.29', 'None', 'tcp 0', '19506', 'Nessus Scan Information'), ('10.31.112.29', 'None', 'tcp 0', '10919', 'Open Port Re-check'), ('10.31.112.29', 'Medium', 'tcp 3389', '58453', "Terminal Services Doesn't Use Network Level Authentication (NLA)"), ('10.31.112.29', 'None', 'tcp 0', '54615', 'Device Type'), ('10.31.112.29', 'None', 'tcp 0', '45590', 'Common Platform Enumeration (CPE)')]

Connect to a database

I created a database on my local computer for this project called “riskrptdb”. We need to establish a connection to that database which we can write to. I created a user which only has access to this database (least privilage) and am hiding their credentials by using a python get() request. This database won’t be around long, but you should still never store plain text passwords in scripts.

from dbConnection import key

try:
    conn = db.connect(
        user=key.get('DATABASE_USER'),
        password=key.get('DATABASE_PASSWORD'),
        host="localhost",
        port=3306,
        database ="riskrptdb"
    )
except db.Error as e:
    print(f"Error connecting to MariaDB Platform: {e}")
    sys.exit(1)
    
cur = conn.cursor()

Write to the database

We’re going to create a table on the database called “nessus_sample”. That’s what I wanted to accomplish in this update. Later we might do some analysis on this sample file, or look up more information about each “plugin” (aka vulnerability) that Nessus gave us to find more details than we have here. But I’ll leave it there for now.

table_name = "nessus_sample"
params = (table_name,)
columns = """
    ip%s,
    risk%s,
    port%s,
    pluginID%s,
    pluginName%s
    """
drop = ("DROP TABLE IF EXISTS %s" %params)
create = ("CREATE TABLE %s" %params)
insert = ("INSERT INTO %s "%params) + "(ip, risk, port, pluginID, pluginName) VALUES (%s, %s, %s, %s, %s)"

with conn.cursor() as cursor:
    cursor.execute(drop)
    cursor.execute(create + "(" + columns % (' VARCHAR(50)',' VARCHAR(24)',' VARCHAR(50)',' VARCHAR(15)',' VARCHAR(256)',)+",rowID int PRIMARY KEY AUTO_INCREMENT)")
    cursor.executemany(insert, data)
conn.commit()