There is a trick for using Suricata IDS/IPS that has come in handy - in my experience that I thought would share and might be useful to others.
My HW was CPU 1x E5-2680 with 64 GB RAM , NIC a 82599EB 10-Gigabit SFI/SFP+ and mirroring about 9,5Gbps:
So I wanted to get a better understanding of what is the max log output from Suricata in that particular environment without putting detection into consideration. Just pure event logging and profiling - DNS,HTTP,TLS,SSH, File transactions, SMTP all of these.
Suricata offers just that - with a really low need for HW (having in mind we are looking at 9,5Gbps). What i did was compile suricata with the "--disable-detection" switch. What it does it simply disables detection(alerts) in Suricata - however every other logging/parsing capability is preserved ( DNS,HTTP,TLS,SSH, File transactions, SMTP). So i downloaded a fresh copy:
git clone git://phalanx.openinfosecfoundation.org/oisf.git && cd oisf/ && git clone https://github.com/ironbee/libhtp.git -b 0.5.x
then
./autogen.sh && ./configure --disable-detection && make clean && make && make install && ldconfig
enabled all JSON outputs in the eve-log section in suricata.yaml. I confirmed that detection was disabled:
and started Suricata.
Careful what you asked for :) - I was getting 10-15K logs per second :
root@suricata:/var/log/suricata# tail -f eve.json |perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'
1
6531
13860
10704
10877
10389
10664
10205
9996
14798
15427
14223
And the HW(CPU/RAM/HDD) usage was not much at all:
As you can see - 30-40% CPU with a third of RAM usage.
I used this command to count logs per second -
tail -f eve.json |perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'
more about similar commands and counting Suricata logs you could find here:
- http://pevma.blogspot.se/2014/06/24-hr-full-log-run-with-suricata-idps.html
- http://pevma.blogspot.se/2014/05/logs-per-second-on-evejson-good-and-bad.html
So this can came in handy in a few scenarious:
- you can actually do a "profiling" run on that particular set up in order to size up a SIEM specification
- and/or you can size up your prod IDS/IPS deployment needs
- you can also just feed all those logs info to an existing log analysis system
Thanks