Howto: Collecting Snort logs with Splunk

Ok, now this time we will try to make snort logs into nice little colorful graphs. 

First off you need to think a little about your network design. Snort is very easily overloaded due to high CPU usage when using lots of signatures, thats why its always good to use several Snort installations(or Snort 3.0 which is multi-threaded) or use Vlans to delimit your network.

For example this is how i set it up:

By setting it up this way i get two advantages:

1. This only scans firewalled traffic which takes off a big amount of load.

2. By using two Snorts i can use different signatures on each, for example on the DMZ snort its more likely that i will use WEB-ATTACKS rules etc.

Ok now lets start setting everything up:

1. Installing Snort

Install and setup snort on as many machines as you want.

I am as always using Xubuntu 7.10 but Ubuntu should work aswell as many other dists.

2. Traffic to Snort

 Make Trunk ports from your switches to your Snort machines. Remember that this is a great way to delimit the amount of traffic that will get to your Snort.

On a Cisco machine with IOS you would use something like this:

monitor session 1 source vlan 3 , 4 , 7 - 10

monitor session 1 destination interface Gi3/18

On an Extreme with Extremeware something like this: 

enable mirroring to port 4:26 tagged

configure mirror add vlan "Default"

configure mirror add vlan "Dmz1"

configure mirror add vlan "Dmz2"

configure mirror add vlan "Dmz3"

configure mirror add vlan "Dmz4"

configure mirror add vlan "Dmz5"

configure mirror add vlan "Dmz6"

configure mirror add vlan "Dmz7"

For other switches please check this site out:

3. Install and setup Splunk

 Now you should have traffic flowing to your Snorts, so head over to www.splunk.com and download the latest version, im using the windows 3.4.1 for this test.

If possible you should use a server with dualcore since Splunk can draw lots of CPU and its nice to be able to administrate it while you are running some graphing.

Now that you have setup Splunk and confirmed that its working go to www.splunkbase.com and install this addon.

Now go to your admin area of Splunk and go to application and enable splunk for snort.

We also need to setup which port Splunk will listen to log files on so go to Data inputs -> Network ports and add TCP and UDP 514. Configure them to listen to Snot application like this:

Now i had lots of trouble to get Splunk to eat the snort logs correctly so i had a discussion on the Splunk forum and got some great help. 

You need to go to: %splunk system folder%\etc\system\local and edit props.conf

under the line [source::tcp:514] add:

SHOULD_LINEMERGE=true

BREAK_ONLY_BEFORE = ^.{3}\w{3}\s\d+\s\d+\:\d+\:\d+\s+\w+\s\[[^\]]+\]\s\[\d+\:\d+\:\d+

MAX_EVENTS = 500

TIME_PREFIX = (^|^\<\d\>)

TIME_FORMAT = %b %d %T

MAX_TIMESTAMP_LOOKAHEAD = 20

This enables all syslogs on TCP port 514 to be shown and proccesed in the right way.

Now restart Splunk.

4. Configure Snort to send logs

Ok now we actually need to have our Snorts to send the logs to Splunk. 

First off download syslog-ng using your terminal in Xubuntu:

sudo apt-get install syslog-ng

Edit the config file with your favorite editor:

sudo gedit /etc/syslog-ng/syslog-ng.conf

Add this:

source s_tail { file("/var/log/snort/alert"

                    follow_freq(1) flags(no-parse) ) ; };

destination stail2 {  tcp("10.10.10.10") ;

};

log {

        source(s_tail);

        destination(stail2);

flags(flow-control);

};

Everything in bold you might need to edit. The first bold is where your alert logfile is located. By default it is at /var/log/snort/alert. The second is the adress of your Splunk server.

So what this does is that it tails your alert logfile for updates, when its updated its sent to Splunk via TCP so it wont be discarded on the way. You can change it to UDP by switching out

"{  tcp("10.10.10.10") ;"  to  "{  udp("10.10.10.10") ;"

Note that TCP only makes sure it gets where its supposed to go, not that its encrypted and safe from interception. If you want that you should look in to Stunnel.

5. Create Graphs in Splunk

Ok now you should have Snort logs arriving in your Splunk(if not activate the ICMP signature and ping a bit with your computer) so now we should make some nice graphs.

First we need to extract the fields to make Splunk identify the Snort signatures, this is a very important lesson since you can use this to extract everything and have Splunk identify/graph it.

Click on the little arrow next to the logfile to open the extract menu:

Now the extract menu will open, so select to name of the signature, in this example it is NETBIOS SMB-DS C$ unicode share access. Now paste it into the example box and select preview. Now Splunk is trying to identify the name of the signatures. This is how it should look like if its working:

Now save it with a name, signature for ex.

Now we have names for the signatures and Splunk can identify them. So lets get to graphing.

So open up the logs again in Splunk and click "Report on results:

Now to the left click on the extracted field you made. To the top left you can choose which timeline you want eg 15min 60min 3months etc. And then choose some graphing options, heres an example of how mine looks like:

Ok now save your graph and put it on your dashboard and your done :) Happy Splunking!

6. Enjoy the examples

Here are some pictures of what you can Graph with Splunk:

Snort:

Juniper SA concurrent users and failed logins graphed with Splunk:

DHCP - Subnets without any ipadress left:

Snort also has a default preprocessor namne snort-stats that lets you export loads of stuff, eg. cpu load, blocks/min etc:

Ok, hopefully this will help you alot in securing your network. I will post some guides about nice rules and hints and tips later using Snort. Remember that Splunk can be used for lots of other stuff than graphing, just enter anything in the searchbar.

Enjoy!