Introduction
NOTE: An updated article is available here.
This is Chapter III - AF_PACKET of a series of articles about high performance and advance tuning of Suricata IDPS
This article will consist of series of instructions on setting up and configuring Suricata IDPS with AF_PACKET for a 10Gbps traffic interface monitoring.
Chapter III - AF_PACKET
AF_PACKET works "out of the box " with Suricata. Please make sure your kernel level is at least 3.2 in order to get the best results.Once you have followed all the steps in Chapter I - Preparation The only thing left to do is adjust the suricata.yaml settings.
AF_PACKET - suricata.yaml tune up and configuration
NOTE:
AF_PACKET - Which kernel version not to use with Suricata in AF_PACKET mode
(thanks to Regit)
We make sure we use runmode workers (feel free to try other modes and experiment what is best for your specific set up):
#runmode: autofp
runmode: workers
Adjust the packet size:
# Preallocated size for packet. Default is 1514 which is the classical
# size for pcap on ethernet. You should adjust this value to the highest
# packet size (MTU + hardware header) on your system.
default-packet-size: 1520
Use custom profile in detect-engine with a lot more groups (high gives you about 15 groups per variable, but you can customize as needed depending on the network ranges you monitor ):
detect-engine:
- profile: high
- custom-values:
toclient-src-groups: 200
toclient-dst-groups: 200
toclient-sp-groups: 200
toclient-dp-groups: 300
toserver-src-groups: 200
toserver-dst-groups: 400
toserver-sp-groups: 200
toserver-dp-groups: 250
- sgh-mpm-context: full
- inspection-recursion-limit: 3000
Adjust your defrag settings:
# Defrag settings:
defrag:
memcap: 512mb
hash-size: 65536
trackers: 65535 # number of defragmented flows to follow
max-frags: 65535 # number of fragments to keep
prealloc: yes
timeout: 30
Adjust your flow settings:
flow:
memcap: 1gb
hash-size: 1048576
prealloc: 1048576
emergency-recovery: 30
Adjust your per protocol timeout values:
flow-timeouts:
default:
new: 3
established: 30
closed: 0
emergency-new: 10
emergency-established: 10
emergency-closed: 0
tcp:
new: 6
established: 100
closed: 12
emergency-new: 1
emergency-established: 5
emergency-closed: 2
udp:
new: 3
established: 30
emergency-new: 3
emergency-established: 10
icmp:
new: 3
established: 30
emergency-new: 1
emergency-established: 10
Adjust your stream engine settings:
stream:
memcap: 16gb
checksum-validation: no # reject wrong csums
prealloc-sesions: 500000 #per thread
midstream: true
asyn-oneside: true
inline: no # auto will use inline mode in IPS mode, yes or no set it statically
reassembly:
memcap: 20gb
depth: 12mb # reassemble 12mb into a stream
toserver-chunk-size: 2560
toclient-chunk-size: 2560
randomize-chunk-size: yes
#randomize-chunk-range: 10
Make sure you enable suricata.log for troubleshooting if something goes wrong:
outputs:
- console:
enabled: yes
- file:
enabled: yes
filename: /var/log/suricata/suricata.log
The AF_PACKET section:
af-packet:
- interface: eth3
# Number of receive threads (>1 will enable experimental flow pinned
# runmode)
threads: 16
# Default clusterid. AF_PACKET will load balance packets based on flow.
# All threads/processes that will participate need to have the same
# clusterid.
cluster-id: 98
# Default AF_PACKET cluster type. AF_PACKET can load balance per flow or per hash.
# This is only supported for Linux kernel > 3.1
# possible value are:
# * cluster_round_robin: round robin load balancing
# * cluster_flow: all packets of a given flow are send to the same socket
# * cluster_cpu: all packets treated in kernel by a CPU are send to the same socket
cluster-type: cluster_cpu
# In some fragmentation case, the hash can not be computed. If "defrag" is set
# to yes, the kernel will do the needed defragmentation before sending the packets.
defrag: no
# To use the ring feature of AF_PACKET, set 'use-mmap' to yes
use-mmap: yes
# Ring size will be computed with respect to max_pending_packets and number
# of threads. You can set manually the ring size in number of packets by setting
# the following value. If you are using flow cluster-type and have really network
# intensive single-flow you could want to set the ring-size independantly of the number
# of threads:
ring-size: 200000
# On busy system, this could help to set it to yes to recover from a packet drop
# phase. This will result in some packets (at max a ring flush) being non treated.
#use-emergency-flush: yes
# recv buffer size, increase value could improve performance
# buffer-size: 100000
# Set to yes to disable promiscuous mode
# disable-promisc: no
# Choose checksum verification mode for the interface. At the moment
# of the capture, some packets may be with an invalid checksum due to
# offloading to the network card of the checksum computation.
# Possible values are:
# - kernel: use indication sent by kernel for each packet (default)
# - yes: checksum validation is forced
# - no: checksum validation is disabled
# - auto: suricata uses a statistical approach to detect when
# checksum off-loading is used.
# Warning: 'checksum-validation' must be set to yes to have any validation
checksum-checks: kernel
# BPF filter to apply to this interface. The pcap filter syntax apply here.
#bpf-filter: port 80 or udp
We had these rules enabled:
rule-files:
- trojan.rules
- md5.rules # 134 000 specially selected file md5s
- dns.rules
- malware.rules
- local.rules
- current_events.rules
- mobile_malware.rules
- user_agents.rules
Make sure you adjust your Network and Port variables:
# Holds the address group vars that would be passed in a Signature.
# These would be retrieved during the Signature address parsing stage.
address-groups:
HOME_NET: "[ HOME NET HERE ]"
EXTERNAL_NET: "!$HOME_NET"
HTTP_SERVERS: "$HOME_NET"
SMTP_SERVERS: "$HOME_NET"
SQL_SERVERS: "$HOME_NET"
DNS_SERVERS: "$HOME_NET"
TELNET_SERVERS: "$HOME_NET"
AIM_SERVERS: "$EXTERNAL_NET"
DNP3_SERVER: "$HOME_NET"
DNP3_CLIENT: "$HOME_NET"
MODBUS_CLIENT: "$HOME_NET"
MODBUS_SERVER: "$HOME_NET"
ENIP_CLIENT: "$HOME_NET"
ENIP_SERVER: "$HOME_NET"
# Holds the port group vars that would be passed in a Signature.
# These would be retrieved during the Signature port parsing stage.
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
ORACLE_PORTS: 1521
SSH_PORTS: 22
DNP3_PORTS: 20000
Your app parsers:
# Holds details on the app-layer. The protocols section details each protocol.
# Under each protocol, the default value for detection-enabled and "
# parsed-enabled is yes, unless specified otherwise.
# Each protocol covers enabling/disabling parsers for all ipprotos
# the app-layer protocol runs on. For example "dcerpc" refers to the tcp
# version of the protocol as well as the udp version of the protocol.
# The option "enabled" takes 3 values - "yes", "no", "detection-only".
# "yes" enables both detection and the parser, "no" disables both, and
# "detection-only" enables detection only(parser disabled).
app-layer:
protocols:
tls:
enabled: yes
detection-ports:
tcp:
toserver: 443
#no-reassemble: yes
dcerpc:
enabled: yes
ftp:
enabled: yes
ssh:
enabled: yes
smtp:
enabled: yes
imap:
enabled: detection-only
msn:
enabled: detection-only
smb:
enabled: yes
detection-ports:
tcp:
toserver: 139
# smb2 detection is disabled internally inside the engine.
#smb2:
# enabled: yes
dnstcp:
enabled: yes
detection-ports:
tcp:
toserver: 53
dnsudp:
enabled: yes
detection-ports:
udp:
toserver: 53
http:
enabled: yes
Libhtp body limits:
libhtp:
default-config:
personality: IDS
# Can be specified in kb, mb, gb. Just a number indicates
# it's in bytes.
request-body-limit: 12mb
response-body-limit: 12mb
# inspection limits
request-body-minimal-inspect-size: 32kb
request-body-inspect-window: 4kb
response-body-minimal-inspect-size: 32kb
response-body-inspect-window: 4kb
Run it
/usr/local/bin/suricata -c /etc/suricata/suricata.yaml --af-packet=eth3 -D -v
Results
We take a look at the suricata.log file:
[13915] 4/12/2013 -- 15:38:15 - (suricata.c:962) <Notice> (SCPrintVersion) -- This is Suricata version 2.0dev (rev e7f6107)
[13915] 4/12/2013 -- 15:38:15 - (util-cpu.c:170) <Info> (UtilCpuPrintSummary) -- CPUs/cores online: 16
[13915] 4/12/2013 -- 15:38:15 - (app-layer-dns-udp.c:315) <Info> (DNSUDPConfigure) -- DNS request flood protection level: 500
[13915] 4/12/2013 -- 15:38:15 - (util-ioctl.c:99) <Info> (GetIfaceMTU) -- Found an MTU of 1500 for 'eth3'
[13915] 4/12/2013 -- 15:38:15 - (defrag-hash.c:212) <Info> (DefragInitConfig) -- allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56
[13915] 4/12/2013 -- 15:38:15 - (defrag-hash.c:237) <Info> (DefragInitConfig) -- preallocated 65535 defrag trackers of size 152
[13915] 4/12/2013 -- 15:38:15 - (defrag-hash.c:244) <Info> (DefragInitConfig) -- defrag memory usage: 13631336 bytes, maximum: 536870912
[13915] 4/12/2013 -- 15:38:15 - (tmqh-flow.c:76) <Info> (TmqhFlowRegister) -- AutoFP mode using default "Active Packets" flow load balancer
[13916] 4/12/2013 -- 15:38:15 - (tmqh-packetpool.c:142) <Info> (PacketPoolInit) -- preallocated 2048 packets. Total memory 7151616
[13916] 4/12/2013 -- 15:38:15 - (host.c:205) <Info> (HostInitConfig) -- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
[13916] 4/12/2013 -- 15:38:15 - (host.c:228) <Info> (HostInitConfig) -- preallocated 1000 hosts of size 112
[13916] 4/12/2013 -- 15:38:15 - (host.c:230) <Info> (HostInitConfig) -- host memory usage: 390144 bytes, maximum: 16777216
[13916] 4/12/2013 -- 15:38:15 - (flow.c:386) <Info> (FlowInitConfig) -- allocated 67108864 bytes of memory for the flow hash... 1048576 buckets of size 64
[13916] 4/12/2013 -- 15:38:15 - (flow.c:410) <Info> (FlowInitConfig) -- preallocated 1048576 flows of size 280
[13916] 4/12/2013 -- 15:38:15 - (flow.c:412) <Info> (FlowInitConfig) -- flow memory usage: 369098752 bytes, maximum: 1073741824
[13916] 4/12/2013 -- 15:38:15 - (reputation.c:459) <Info> (SRepInit) -- IP reputation disabled
[13916] 4/12/2013 -- 15:38:15 - (util-magic.c:62) <Info> (MagicInit) -- using magic-file /usr/share/file/magic
[13916] 4/12/2013 -- 15:38:15 - (suricata.c:1769) <Info> (SetupDelayedDetect) -- Delayed detect disabled
[13916] 4/12/2013 -- 15:38:17 - (detect-filemd5.c:275) <Info> (DetectFileMd5Parse) -- MD5 hash size 2143616 bytes
...8 rule files, 7947 rules loaded
[13916] 4/12/2013 -- 15:38:17 - (detect.c:453) <Info> (SigLoadSignatures) -- 8 rule files processed. 7947 rules successfully loaded, 0 rules failed
[13916] 4/12/2013 -- 15:38:17 - (detect.c:2568) <Info> (SigAddressPrepareStage1) -- 7947 signatures processed. 1 are IP-only rules, 1976 are inspecting packet payload, 6714 inspect application laye
r, 0 are decoder event only
[13916] 4/12/2013 -- 15:38:17 - (detect.c:2571) <Info> (SigAddressPrepareStage1) -- building signature grouping structure, stage 1: preprocessing rules... complete
[13916] 4/12/2013 -- 15:38:17 - (detect.c:3194) <Info> (SigAddressPrepareStage2) -- building signature grouping structure, stage 2: building source address list... complete
[13916] 4/12/2013 -- 15:39:51 - (detect.c:3836) <Info> (SigAddressPrepareStage3) -- building signature grouping structure, stage 3: building destination address lists... complete
[13916] 4/12/2013 -- 15:39:51 - (util-threshold-config.c:1186) <Info> (SCThresholdConfParseFile) -- Threshold config parsed: 0 rule(s) found
[13916] 4/12/2013 -- 15:39:51 - (util-coredump-config.c:122) <Info> (CoredumpLoadConfig) -- Core dump size set to unlimited.
[13916] 4/12/2013 -- 15:39:51 - (util-logopenfile.c:168) <Info> (SCConfLogOpenGeneric) -- fast output device (regular) initialized: fast.log
[13916] 4/12/2013 -- 15:39:51 - (util-logopenfile.c:168) <Info> (SCConfLogOpenGeneric) -- http-log output device (regular) initialized: http.log
[13916] 4/12/2013 -- 15:39:51 - (util-logopenfile.c:168) <Info> (SCConfLogOpenGeneric) -- tls-log output device (regular) initialized: tls.log
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "management-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:265) <Info> (AffinitySetupLoadFromConfig) -- Using default prio 'low'
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "receive-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "decode-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "stream-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "detect-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:265) <Info> (AffinitySetupLoadFromConfig) -- Using default prio 'high'
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "verdict-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:265) <Info> (AffinitySetupLoadFromConfig) -- Using default prio 'high'
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "reject-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:265) <Info> (AffinitySetupLoadFromConfig) -- Using default prio 'low'
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:217) <Info> (AffinitySetupLoadFromConfig) -- Found affinity definition for "output-cpu-set"
[13916] 4/12/2013 -- 15:39:51 - (util-affinity.c:265) <Info> (AffinitySetupLoadFromConfig) -- Using default prio 'medium'
[13916] 4/12/2013 -- 15:39:51 - (runmode-af-packet.c:200) <Info> (ParseAFPConfig) -- Enabling mmaped capture on iface eth3
[13916] 4/12/2013 -- 15:39:51 - (runmode-af-packet.c:268) <Info> (ParseAFPConfig) -- Using cpu cluster mode for AF_PACKET (iface eth3)
[13916] 4/12/2013 -- 15:39:51 - (util-runmodes.c:545) <Info>
...going to use 16 threads:
(RunModeSetLiveCaptureWorkersForDevice) -- Going to use 16 thread(s)
[13918] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 0
[13918] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth31" Module to cpu/core 0, thread id 13918
[13918] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13918] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13919] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 1
[13919] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth32" Module to cpu/core 1, thread id 13919
[13919] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13919] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13920] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 2
[13920] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth33" Module to cpu/core 2, thread id 13920
[13920] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13920] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13921] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 3
[13921] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth34" Module to cpu/core 3, thread id 13921
[13921] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13921] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13922] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 4
[13922] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth35" Module to cpu/core 4, thread id 13922
[13922] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13922] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13923] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 5
[13923] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth36" Module to cpu/core 5, thread id 13923
[13923] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13923] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13924] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 6
[13924] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth37" Module to cpu/core 6, thread id 13924
[13924] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13924] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13925] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 7
[13925] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth38" Module to cpu/core 7, thread id 13925
[13925] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13925] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13926] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 8
[13926] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth39" Module to cpu/core 8, thread id 13926
[13926] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13926] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13927] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 9
[13927] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth310" Module to cpu/core 9, thread id 13927
[13927] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13927] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13928] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 10
[13928] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth311" Module to cpu/core 10, thread id 13928
[13928] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13928] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13929] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 11
[13929] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth312" Module to cpu/core 11, thread id 13929
[13929] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13929] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13930] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 12
[13930] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth313" Module to cpu/core 12, thread id 13930
[13930] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13930] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13931] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 13
[13931] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth314" Module to cpu/core 13, thread id 13931
[13931] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13931] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13932] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 14
[13932] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth315" Module to cpu/core 14, thread id 13932
[13932] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13932] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
[13933] 4/12/2013 -- 15:39:51 - (util-affinity.c:319) <Info> (AffinityGetNextCPU) -- Setting affinity on CPU 15
[13933] 4/12/2013 -- 15:39:51 - (tm-threads.c:1332) <Info> (TmThreadSetupOptions) -- Setting prio -2 for "AFPacketeth316" Module to cpu/core 15, thread id 13933
[13933] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1554) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode
[13933] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1564) <Info> (ReceiveAFPThreadInit) -- Enabling zero copy mode by using data release call
...reading in some memory settings from yaml:
[13916] 4/12/2013 -- 15:39:51 - (runmode-af-packet.c:529) <Info> (RunModeIdsAFPWorkers) -- RunModeIdsAFPWorkers initialised
[13934] 4/12/2013 -- 15:39:51 - (tm-threads.c:1338) <Info> (TmThreadSetupOptions) -- Setting prio 2 for "FlowManagerThread" thread , thread id 13934
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:376) <Info> (StreamTcpInitConfig) -- stream "prealloc-sessions": 375000 (per thread)
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:392) <Info> (StreamTcpInitConfig) -- stream "memcap": 17179869184
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:398) <Info> (StreamTcpInitConfig) -- stream "midstream" session pickups: enabled
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:404) <Info> (StreamTcpInitConfig) -- stream "async-oneside": disabled
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:421) <Info> (StreamTcpInitConfig) -- stream "checksum-validation": disabled
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:443) <Info> (StreamTcpInitConfig) -- stream."inline": disabled
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:456) <Info> (StreamTcpInitConfig) -- stream "max-synack-queued": 5
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:474) <Info> (StreamTcpInitConfig) -- stream.reassembly "memcap": 21474836480
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:492) <Info> (StreamTcpInitConfig) -- stream.reassembly "depth": 12582912
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:575) <Info> (StreamTcpInitConfig) -- stream.reassembly "toserver-chunk-size": 2671
[13916] 4/12/2013 -- 15:39:51 - (stream-tcp.c:577) <Info> (StreamTcpInitConfig) -- stream.reassembly "toclient-chunk-size": 2582
[13935] 4/12/2013 -- 15:39:51 - (tm-threads.c:1338) <Info> (TmThreadSetupOptions) -- Setting prio 2 for "SCPerfWakeupThread" thread , thread id 13935
[13936] 4/12/2013 -- 15:39:51 - (tm-threads.c:1338) <Info> (TmThreadSetupOptions) -- Setting prio 2 for "SCPerfMgmtThread" thread , thread id 13936
[13916] 4/12/2013 -- 15:39:51 - (tm-threads.c:2191) <Notice> (TmThreadWaitOnThreadInit) -- all 16 packet processing threads, 3 management threads initialized, engine started.
....have a look - Suricata detects if OFFloading (discussed in Chapter I - Preparation) is used on the network interface:
[13918] 4/12/2013 -- 15:39:51 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13918] 4/12/2013 -- 15:39:51 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13918] 4/12/2013 -- 15:39:51 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13918] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 8
[13918] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth31 using socket 8
[13919] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13919] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13919] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13919] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 9
[13919] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth32 using socket 9
[13920] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13920] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13920] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13920] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 10
[13920] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth33 using socket 10
[13921] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13921] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13921] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13921] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 11
[13921] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth34 using socket 11
[13922] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13922] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13922] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13922] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 12
[13922] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth35 using socket 12
[13923] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13923] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13923] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13923] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 13
[13923] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth36 using socket 13
[13924] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13924] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13924] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13924] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 14
[13924] 4/12/2013 -- 15:39:52 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth37 using socket 14
[13925] 4/12/2013 -- 15:39:52 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13925] 4/12/2013 -- 15:39:52 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13925] 4/12/2013 -- 15:39:52 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13925] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 15
[13925] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth38 using socket 15
[13926] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13926] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13926] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13926] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 16
[13926] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth39 using socket 16
[13927] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13927] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13927] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13927] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 17
[13927] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth310 using socket 17
[13928] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13928] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13928] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13928] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 18
[13928] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth311 using socket 18
[13929] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13929] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13929] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13929] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 19
[13929] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth312 using socket 19
[13930] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13930] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13930] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13930] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 20
[13930] 4/12/2013 -- 15:39:53 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth313 using socket 20
[13931] 4/12/2013 -- 15:39:53 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13931] 4/12/2013 -- 15:39:53 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13931] 4/12/2013 -- 15:39:53 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13931] 4/12/2013 -- 15:39:54 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 21
[13931] 4/12/2013 -- 15:39:54 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth314 using socket 21
[13932] 4/12/2013 -- 15:39:54 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13932] 4/12/2013 -- 15:39:54 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13932] 4/12/2013 -- 15:39:54 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13932] 4/12/2013 -- 15:39:54 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 22
[13932] 4/12/2013 -- 15:39:54 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth315 using socket 22
[13933] 4/12/2013 -- 15:39:54 - (util-ioctl.c:175) <Info> (GetIfaceOffloading) -- Generic Receive Offload is unset on eth3
[13933] 4/12/2013 -- 15:39:54 - (util-ioctl.c:194) <Info> (GetIfaceOffloading) -- Large Receive Offload is unset on eth3
[13933] 4/12/2013 -- 15:39:54 - (source-af-packet.c:1189) <Info> (AFPComputeRingParams) -- AF_PACKET RX Ring params: block_size=32768 block_nr=10001 frame_size=1584 frame_nr=200020
[13933] 4/12/2013 -- 15:39:54 - (source-af-packet.c:1380) <Info> (AFPCreateSocket) -- Using interface 'eth3' via socket 23
[13933] 4/12/2013 -- 15:39:54 - (source-af-packet.c:439) <Info> (AFPPeersListReachedInc) -- All AFP capture threads are running.
[13933] 4/12/2013 -- 15:39:54 - (source-af-packet.c:988) <Info> (ReceiveAFPLoop) -- Thread AFPacketeth316 using socket 23
htop - Now that we have been up and running for a while (6-7 hrs) on a 10Gbps link ( 9.3 Gbps traffic - to be precise - at the moment of these statistics):
we have about 1-2% drops in total (on 7947 rules):
and then after 13 hrs:
we still have 1-2% drops
(1.897% to be precise - total kernel drops 1 337 487 757 out of total packets 70 491 114 835 is 1.897%) :
And that is just half the job done on Suricata's high performance tuning. Before you arrive at this point there is much more work to be done - pre-study, HW choice, rule selection and tuning, traffic analysis , office/organization needs analysis, network location design and deployment, testing/PoCs and more...
Next - Chapter IV - Logstash / Kibana / Elasticsearch
No comments:
Post a Comment