Suricata - more data for your alerts

As of Suricata 2.1beta1  - Suricata IDS/IPS provides the availability of packet data and information in a standard JSON output logging capability supplementing further the alert logging output.

This guide makes use of Suricata and ELK - Elasticsearch, Logstash, Kibana.
You can install all of them following the guide HERE
 ...or you can download and try out SELKS  and use directly.


After everything is in place, we need to open the suricata.yaml and make the following editions in the eve.json section:

 # "United" event log in JSON format
  - eve-log:
      enabled: yes
      type: file #file|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert:
            payload: yes           # enable dumping payload in Base64
            payload-printable: yes # enable dumping payload in printable (lossy) format
            packet: yes            # enable dumping of packet (without stream segments)
            http: yes              # enable dumping of http fields

       
You can start Suricata and let it inspect traffic for some time in order to generate alert log data.
Then navigate to your Kibana web interface, find an alert record/log and you could see the usefulness of the extra data yourself.

Some examples though :) :

Lets kick it up  notch.....

We want to search through -

1. all the generated alerts that have 
2. a printable payload data 
3. that have the following string: uid=0(root)

 Easy, here is the query:
 

payload_printable:"uid=0\(root\)"

You should enter it like this in Kibana:

Well what do you know - we got what we were looking for:

Some more useful reading on the Lucene Query Syntax (you should at least have a look :) ):
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

http://www.solrtutorial.com/solr-query-syntax.html

http://lucene.apache.org/core/2_9_4/queryparsersyntax.html

Suricata - filtering tricks for the fileinfo output with eve.json

As of Suricata 2.0  - Suricata IDS/IPS provides the availability of a standard JSON output logging capability. This guide makes use of Suricata and ELK - Elasticsearch, Logstash, Kibana.

You can install all of them following the guide HERE
 ...or you can download and try out SELKS  and use directly.

Once you have the installation in place and have the Kibana web interface up and running you can make use of the following fileinfo filters (tricks :).
You can enter the queries like so:

 fileinfo.magic:"PE32" -fileinfo.filename:*exe

will show you all "PE32 executable" executables that were seen transferred that have no exe extension in their file name:

 Alternatively

fileinfo.magic:"pdf" -fileinfo.filename:*pdf

will show you all "PDF document version......" files that were transferred that have no PDF extension in their file name.

You can explore further :)

Elasticsearch - err failed to connect to master - when changing/using a different IP address

It is a general rule of thumbs to check first your

/var/log/elasticsearch/elasticsearch.log

and

/var/log/logstash/logstash.log

when you experience any form of issues when using Kibana.

I stumbled upon this when I changed the IP/Network of the interface of my test virtual machine holding an ELK (Elasticsearch/Logstash/Kibana) installation to do log analysis for Suricata IDPS.

I managed to solve the issue based on those two sources:
https://github.com/elasticsearch/elasticsearch/issues/4194
http://www.concept47.com/austin_web_developer_blog/errors/elasticsearch-error-failed-to-connect-to-master/

The new IP that I changed is - 192.168.1.166 and the old one was 10.0.2.15
(notice the errs in the logs. It was still trying to connect to the old one below):

root@debian64:~/Work/# more /var/log/elasticsearch/elasticsearch.log
[2014-05-04 07:17:24,960][INFO ][node                     ] [Jamal Afari] version[1.1.0], pid[7178], build[2181e11/2014-03-25T15:59:51Z]
[2014-05-04 07:17:24,960][INFO ][node                     ] [Jamal Afari] initializing ...
[2014-05-04 07:17:24,964][INFO ][plugins                  ] [Jamal Afari] loaded [], sites []
[2014-05-04 07:17:27,828][INFO ][node                     ] [Jamal Afari] initialized
[2014-05-04 07:17:27,828][INFO ][node                     ] [Jamal Afari] starting ...
[2014-05-04 07:17:27,959][INFO ][transport                ] [Jamal Afari] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.166:9300]}
[2014-05-04 07:17:57,977][WARN ][discovery                ] [Jamal Afari] waited for 30s and no initial state was set by the discovery
[2014-05-04 07:17:57,978][INFO ][discovery                ] [Jamal Afari] elasticsearch/F9HgSmYJQcS6bxdgdeurAA
[2014-05-04 07:17:57,986][INFO ][http                     ] [Jamal Afari] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.166:9200]}
[2014-05-04 07:17:58,017][INFO ][node                     ] [Jamal Afari] started
[2014-05-04 07:18:01,026][WARN ][discovery.zen            ] [Jamal Afari] failed to connect to master [[Hellion][zcx2fIF2SrmwSYQ08la6PQ][LTS-64-1][inet[/10.0.2.15:9300]]], retrying...
org.elasticsearch.transport.ConnectTransportException: [Hellion][inet[/10.0.2.15:9300]] connect_timeout[30s]
    at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:718)
    at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:647)
    at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:615)
    at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
    at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:338)
    at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:79)
    at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:286)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:701)
Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection timed out: /10.0.2.15:9300
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
.......
.......
.......
[2014-05-04 07:37:05,783][WARN ][discovery.zen            ] [Vivisector] failed to connect to master [[Hellion][zcx2fIF2SrmwSYQ08la6PQ][LTS-64-1][inet[/10.0.2.15:9300]]], retrying...
org.elasticsearch.transport.ConnectTransportException: [Hellion][inet[/10.0.2.15:9300]] connect_timeout[30s]
    at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:718)
    at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:647)
    at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:615)
    at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129)
    at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:338)
    at org.elasticsearch.discovery.zen.ZenDiscovery.access$500(ZenDiscovery.java:79)
    at org.elasticsearch.discovery.zen.ZenDiscovery$1.run(ZenDiscovery.java:286)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:701)
Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection timed out: /10.0.2.15:9300
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
    at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
    at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
    at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
    at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    ... 3 more
   
   

That was giving me all sorts of wired errs and failed queries in Kibana. The base of the problem was that I did change IP addresses on the ELK server.

The solution is simple.
Find the Discovery section in  /etc/elasticsearch/elasticsearch.yml
and edit this line from :

# 1. Disable multicast discovery (enabled by default):
#
# discovery.zen.ping.multicast.enabled: false

to

# 1. Disable multicast discovery (enabled by default):
#
 discovery.zen.ping.multicast.enabled: false

Only remove the " # " in front of "discovery.zen.ping.multicast.enabled: false".
Save and restart the service.

service elasticsearch restart

Then everything went back to normal.
In /var/log/elasticsearch/elasticsearch.log:
   

[2014-05-04 07:37:07,936][INFO ][node                     ] [Vivisector] stopping ...
[2014-05-04 07:37:07,970][INFO ][node                     ] [Vivisector] stopped
[2014-05-04 07:37:07,971][INFO ][node                     ] [Vivisector] closing ...
[2014-05-04 07:37:07,979][INFO ][node                     ] [Vivisector] closed
[2014-05-04 07:37:09,685][INFO ][node                     ] [Vibraxas] version[1.1.0], pid[5291], build[2181e11/2014-03-25T15:59:51Z]
[2014-05-04 07:37:09,686][INFO ][node                     ] [Vibraxas] initializing ...
[2014-05-04 07:37:09,689][INFO ][plugins                  ] [Vibraxas] loaded [], sites []
[2014-05-04 07:37:12,597][INFO ][node                     ] [Vibraxas] initialized
[2014-05-04 07:37:12,597][INFO ][node                     ] [Vibraxas] starting ...
[2014-05-04 07:37:12,751][INFO ][transport                ] [Vibraxas] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.1.166:9300]}
[2014-05-04 07:37:15,777][INFO ][cluster.service          ] [Vibraxas] new_master [Vibraxas][esQHE1EtTuWVK9MVNiQ5jA][debian64][inet[/192.168.1.166:9300]], reason: zen-disco-join (elected_as_master)
[2014-05-04 07:37:15,806][INFO ][discovery                ] [Vibraxas] elasticsearch/esQHE1EtTuWVK9MVNiQ5jA
[2014-05-04 07:37:15,877][INFO ][http                     ] [Vibraxas] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.1.166:9200]}
[2014-05-04 07:37:16,893][INFO ][gateway                  ] [Vibraxas] recovered [16] indices into cluster_state
[2014-05-04 07:37:16,898][INFO ][node                     ] [Vibraxas] started
[2014-05-04 07:37:17,547][INFO ][cluster.service          ] [Vibraxas] added {[logstash-debian64-3408-4020][dTsgT1H9Srq6mUr_w5rpXQ][debian64][inet[/192.168.1.166:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[logstash-debian64-3408-4020][dTsgT1H9Srq6mUr_w5rpXQ][debian64][inet[/192.168.1.166:9301]]{client=true, data=false}])

 It is also higly recommended that you read the whole Discovery section in your elasticsearch.yml:

############################# Discovery #############################

# Discovery infrastructure ensures nodes can be found within a cluster
# and master node is elected. Multicast discovery is the default.

.....