During a large penetration test or red team exercise, you might start to get overwhelmed by scan results. This could easily lead to missing something critical. Since being organized is so important, I wanted to look into scan storage and visualization solutions. This series of posts will explore some options I've come across for managing all that scan data. Our goal here is to store, organize, and visualize networks, hosts, and ports.
The first solution I'd like to dive into is IVRE. It is an open-source framework designed for network recon, written in Python, and with a MongoDB backend. On an engagement, time is a resource you cannot waste. Therefore, we want to be able to deploy and set up a tool quickly. To do that, we'll use Docker. We've written about it in previous posts, and you can read about the setup here.
First we need to grab our container images:
docker pull ivre/agent
docker pull ivre/base
docker pull ivre/client
docker pull ivre/db
docker pull ivre/webStarting with the db, we'll want to create a volume to store the MongoDSB data:
mkdir -m 1777 var_{lib,log}_{mongodb,neo4j}
Then we create the container with the name ivredb by running:
$ docker run -d --name ivredb --hostname ivredb --volume "`pwd`/var_lib_mongodb":/var/lib/mongodb --volume "`pwd`/var_log_mongodb":/var/log/mongodb -p 27017:27017 ivre/dbNext up is the web server. We'll create a container named ivreweb by running:
$ docker run -d --name ivreweb --hostname ivreweb volume "`pwd`/ivre.conf:/etc/ivre.conf" --volume "`pwd`/nginx-default-site:/etc/nginx/sites-available/default" --link ivredb:ivredb -p 80:80 ivre/webNow we can browse to localhost and see the GUI for IVRE. Of course, if you don't want to (or can't) dedicate port 80 to this tool, you can specify another by changing the first number of -p in the above command.
Finally, we need a client to run the scans from. This can be done with:
$ docker run -i -t --name ivreclient --hostname ivreclient link ivredb:ivredb --volume "`pwd`/ivre-share":/ivre-share ivre/clientYou should be dropped into a shell on the ivreclient container. One last step for setup is to initialize the database by running:
root@ivreclient:/# ivre ipinfo --init
This will remove any passive information in your database. Process ? [y/N] y
root@ivreclient:/# ivre scancli --init
This will remove any scan result in your database. Process ? [y/N] y
root@ivreclient:/# ivre runscansagentdb --init
This will remove any agent and/or scan in your database and files. Process ? [y/N] y
root@ivreclient:/# ivre ipdata --download --import-allNote that the last command will take a considerable amount of time.When we're finally up and running, we can run scans from our client CLI. This tool is designed for massive nmap scans, but we'll start with just some sample data. Within the client CLI, we can run ivre runscans -h to get the help output. That will show us that we can run ivre runscans -n <IP range> --output=XMLFork --processes 5, for example. Once the scan completes, we can run ivre scan2db -s <source> -c <category> /path/to/scan/results/*.
Back in our IVRE web window, we should see the results of our scan:
Now we can drill down to host info or filter by ports/source/category. We can even add notes by selecting the pencil. There are also some quick graphs that can be made by selecting an option from the left sidebar:
By selecting the Share -> Report, we can configure and view our data based on parameters of our choosing. For example. here is a report-ready image of the top ports:
There is also the option to export all of your data into JSON format. IVRE is clearly a powerful tool for managing scan data and here I've just given a quick overview with a small data set.
IVRE comes from:Pierre Lalet, Florent Monjalet, Camille Mougey and Vivien Venuti. IVRE, a network recon framework.https://github.com/cea-sec/ivre, 2011-2018.
