In my continued efforts to make sense of large amounts of scan data, I came across Dataiku. From their website: "Dataiku DSS is the collaborative data science software platform for teams of data scientists, data analysts, and engineers to explore, prototype, build, and deliver their own data products more efficiently." While this product may not be designed for what we're trying to accomplish, I have found it to be an easy way to turn the scan data into charts. For example, you could get insight into how many hosts are running a certain protocol. Most importantly, it's extremely easy to set up and use.
Once again, we'll be using Docker for setup. We essentially just need to run one command docker run -p 10000:10000 -d dataiku/dss . Now just browse to http://localhost:10000/ .We'll be greeted with a blank instance of Dataiku DSS:
Now we're already up and running. This speed and ease of setup can be great for spinning up an instance on different engagements such as on different (or re-imaged) machines. Now we'll start up a blank project, enter a name, and click create.
We'll be greeted with an entirely empty project. Now we just need to import our data. For this example, I ran multiple nmap scans within metasploit (db_nmap) and then exported the database with db_export xml [filename]. I've found that I have better luck with csv, so I converted the export and got rid of some of the columns that I won't need for this purpose:Click + Import your first dataset and you can see all the options available to you for data upload. Some options are locked behind the paid version. For our purposes, we'll just select Files -> Upload your files.
Next, we simply drag our exported scans and drop it into the file upload. You'll want to go over to the Format/Preview tab and make sure everything looks the way you want it to. For example, I renamed the columns to be more accurate (open/closed and service specifically). When you're done and everything looks correct, click create.
On the next page, we can see multiple tabs. First, let's look at the Charts tab. Here we can select which data we want to show as well as what type of chart we want to generate.
For example, we can create a count of certain services. Let's say we want to know how many hosts are running some sort of web server. Take note of the filter column. You'll want to drag the service column into the filter box, uncheck everything, and then narrow down your filter to what you want to see. This can be saved and included in a pen test report to show the size of an attack surface for certain ports (plaintext logins, anyone?).
Over in the explore tab, we can show and filter the data we want to see. This can be a nice, organized way to view things such as which hosts have port 445 open. Of course, I can't find any way to export that list and it's much easier to just dump it into metasploit's database and then search by port. That would even let you set the list as RHOSTS...
So in summary, this tool is an easy to deploy graphing tool. This definitely isn't perfect for my purposes, but I thought I'd share my experience with it. The main draw here really just comes down to reporting and counting and while it might be a powerful tool for data analytics, it doesn't really give us anything we can work with as we progress through the testing phases.
I hope that it helps someone prepare a report for "the higher ups". They sure love their graphs and charts!
