Failed to open ethX: pfring_open error

This is a blogpost about getting around the following error when using Suricata with pfring:

(source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
(tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1

However in my case eth2 existed, was up and running and the pfring module was loaded. So what happened in a bit more detail below :

I experienced this after a git pull update/upgrade of Suricata (latest at the moment of this writing) and after I re compiled pfring (using the latest pfring from git (https://github.com/ntop/PF_RING.git).

My set up (linux Debian/Ubuntu like systems):

root@suricata:/var/data/log/suricata# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:e0:ed:19:e3:e0
          inet6 addr: fe80::2e0:edff:fe19:e3e0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2962266192 errors:0 dropped:5527381 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2867936692537 (2.8 TB)  TX bytes:3345 (3.3 KB)

The pfring set up I had was configured like this below:

root@suricata:/var/data/log/suricata# modprobe pf_ring transparent_mode=0 min_num_slots=65534

A regular check reveals nothing abnormal:

root@suricata:/var/data/log/suricata# modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/3.14.0-031400-generic/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     E344EB01757B55E97A93D0C
depends:     
vermagic:       3.14.0-031400-generic SMP mod_unload modversions
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           transparent_mode:(deprecated) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into the syslog (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets (uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one socket per interface (uint)
PF_RING Version          : 6.1.1 (dev:250a67fe1082121ac511a19ebc3fe1fc5f494bfe)
Total rings              : 0

Standard (non DNA/ZC) Options
Ring slots               : 65534
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0

Suricata and pfring have been installed as explained here - on the Suricata redmine wiki.

root@suricata:~# ldd /usr/local/bin/suricata
    linux-vdso.so.1 =>  (0x00007fff419fe000)
    libhtp-0.5.17.so.1 => /usr/local/lib/libhtp-0.5.17.so.1 (0x00007f32af5a1000)
    libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1 (0x00007f32af372000)
    libluajit-5.1.so.2 => /usr/local/lib/libluajit-5.1.so.2 (0x00007f32af103000)
    libmagic.so.1 => /usr/lib/x86_64-linux-gnu/libmagic.so.1 (0x00007f32aeee7000)
    libcap-ng.so.0 => /usr/local/lib/libcap-ng.so.0 (0x00007f32aece2000)
    libpfring.so => /usr/local/lib/libpfring.so (0x00007f32aeaa3000)
    libpcap.so.1 => /usr/local/pfring/lib/libpcap.so.1 (0x00007f32ae80e000)
    libnet.so.1 => /usr/lib/x86_64-linux-gnu/libnet.so.1 (0x00007f32ae5f5000)
    libjansson.so.4 => /usr/lib/x86_64-linux-gnu/libjansson.so.4 (0x00007f32ae3e8000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32ae1ca000)
    libyaml-0.so.2 => /usr/lib/x86_64-linux-gnu/libyaml-0.so.2 (0x00007f32adfaa000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f32add6b000)
    libnss3.so => /usr/lib/x86_64-linux-gnu/libnss3.so (0x00007f32ada31000)
    libnspr4.so => /usr/lib/x86_64-linux-gnu/libnspr4.so (0x00007f32ad7f4000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32ad42e000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f32ad215000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f32acf0f000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f32acd0a000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f32acaf4000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f32af7d4000)
    libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f32ac8e9000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f32ac6e0000)
    libnssutil3.so => /usr/lib/x86_64-linux-gnu/libnssutil3.so (0x00007f32ac4b5000)
    libplc4.so => /usr/lib/x86_64-linux-gnu/libplc4.so (0x00007f32ac2b0000)
    libplds4.so => /usr/lib/x86_64-linux-gnu/libplds4.so (0x00007f32ac0ab000)

Further more my Suricata start line was like this:

suricata --pfring-int=eth2 --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Even though everything seems fine - I could  not start Suricata with pfring:

[31591] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31591] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31592] 5/8/2015 -- 17:10:31 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31592] 5/8/2015 -- 17:10:31 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31593] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31593] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
[31594] 5/8/2015 -- 17:10:32 - (tmqh-packetpool.c:394) <Info> (PacketPoolInit) -- preallocated 65534 packets. Total memory 230679680
pfring_set_channel_id() failed: -1
[31594] 5/8/2015 -- 17:10:32 - (source-pfring.c:444) <Error> (ReceivePfringThreadInit) -- [ERRCODE: SC_ERR_PF_RING_OPEN(34)] - Failed to open eth2: pfring_open error. Check if eth2 exists and pf_ring module is loaded.
....

I was getting that error even though I reloaded the pfring module:

rmmod pr_ring
modprobe pf_ring transparent_mode=0 min_num_slots=65534

the way I usually do...

In short - this is the fix:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata --pfring-int=eth2  --pfring-cluster-id=99 --pfring-cluster-type=cluster_flow  -c /etc/suricata/peter-yaml/suricata-pfring.yaml --pidfile /var/run/suricata.pid -v

Notice the use of:

LD_LIBRARY_PATH=/usr/local/pfring/lib suricata 

More information about what is LD_LIBRARY_PATH     

To get rid of LD_LIBRARY_PATH you can create a pfring.conf file in /etc/ld.so.conf.d/ containing:

/usr/local/pfring/lib

and run

sudo ldconfig

Suricatasc unix socket interaction for Suricata IDS/IPS

Suricatasc is a unix socket interaction  script that is automatically installed when one compiles/installs Suricata IDS/IPS. An in depth description, prerequisites and how to documentation is located here - https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Interacting_via_Unix_Socket

However  lets look at a quick usage example - that can come very handy in certain situations.

Once you have unix socket command enabled in suricata.yaml :

unix-command:
    enabled: yes
    #filename: custom.socket # use this to specify an alternate file

the traditional way to use the script would be type suricatasc and hit Enter (on the machine running Suricata):

However you can also use it directly as a command line parameter for example :

root@suricata:~# suricatasc -c version

like so:

NOTE:
You need to quote commands involving interfaces:

root@debian64:~# suricatasc -c "iface-stat eth0"

Very handy when you want quick interaction and info from the currently running Suricata IDS/IPS.

Upgrading Ubuntu's kernel

Upgrading Ubuntu Quantal/Precise 64 bit kernel 3.2.0 to 3.14.0 kernel level/version

Method 1

We upgrade just the kernel version/release.

We have:

root@suricata:~/kernel-upgrade# uname -a
Linux suricata 3.2.0-60-generic #91-Ubuntu SMP Wed Feb 19 03:54:44 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@suricata:~/kernel-upgrade#

Make sure you have the latest updates:

 apt-get update && apt-get upgrade
 apt-get dist-upgrade

 Download the new kernel image and headers:

 wget  http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-trusty/linux-headers-3.14.0-031400-generic_3.14.0-031400.201403310035_amd64.deb

 wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-trusty/linux-headers-3.14.0-031400_3.14.0-031400.201403310035_all.deb

 wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-trusty/linux-image-3.14.0-031400-generic_3.14.0-031400.201403310035_amd64.deb

 Install the packages and reboot:

 dpkg -i linux-headers-3.14.0-*.deb linux-image-3.14.0-*.deb
 reboot

 log in and update/upgrade:

 apt-get update && apt-get upgrade

 Let's see:

root@suricata:~# uname -a
Linux suricata 3.14.0-031400-generic #201403310035 SMP Mon Mar 31 04:36:23 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@suricata:~#

Done.

Method 2

We upgrade the whole release (including the kernel) from Precise to Trusty. This will also update the /etc/apt/ repo lists and more.

 apt-get update && apt-get upgrade
 apt-get dist-upgrade
 do-release-upgrade -d
 reboot

Done.

Suricata - preparing 10Gbps network cards for IDPS and file extraction

OS used/tested for this tutorial - Debian Wheezy and/or Ubuntu LTS 12.0.4
With 3.2.0 and 3.5.0 kernel level respectively with Suricata 2.0dev at the moment of this writing.



This article consists of the following major 3 sections:

• Network card drivers and tuning
• Kernel specific tunning
• Suricata.yaml configuration  (file extraction specific)

Network and system  tools:

apt-get install ethtool bwm-ng iptraf htop

Network card drivers and tuning

Our card is Intel 82599EB 10-Gigabit SFI/SFP+

rmmod ixgbe
sudo modprobe ixgbe FdirPballoc=3
ifconfig eth3 up

then (we disable irqbalance and make sure it does not enable itself during reboot)

 killall irqbalance
 service irqbalance stop

 apt-get install chkconfig
 chkconfig irqbalance off

Get the Intel network driver form here (we will use them in a second) - https://downloadcenter.intel.com/default.aspx

Download to your directory of choice then unzip,compile and install:

wget http://sourceforge.net/projects/e1000/files/ixgbe%20stable/3.18.7/ixgbe-3.18.7.tar.gz
tar -zxf ixgbe-3.18.7.tar.gz      
cd /home/pevman/ixgbe-3.18.7/src      
make clean && make && make install

Set irq affinity - do not forget to change eth3  below with the name of the network interface you are using:

 cd ../scripts/
 ./set_irq_affinity  eth3

 You should see something like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ./set_irq_affinity  eth3
no rx vectors found on eth3
no tx vectors found on eth3
eth3 mask=1 for /proc/irq/101/smp_affinity
eth3 mask=2 for /proc/irq/102/smp_affinity
eth3 mask=4 for /proc/irq/103/smp_affinity
eth3 mask=8 for /proc/irq/104/smp_affinity
eth3 mask=10 for /proc/irq/105/smp_affinity
eth3 mask=20 for /proc/irq/106/smp_affinity
eth3 mask=40 for /proc/irq/107/smp_affinity
eth3 mask=80 for /proc/irq/108/smp_affinity
eth3 mask=100 for /proc/irq/109/smp_affinity
eth3 mask=200 for /proc/irq/110/smp_affinity
eth3 mask=400 for /proc/irq/111/smp_affinity
eth3 mask=800 for /proc/irq/112/smp_affinity
eth3 mask=1000 for /proc/irq/113/smp_affinity
eth3 mask=2000 for /proc/irq/114/smp_affinity
eth3 mask=4000 for /proc/irq/115/smp_affinity
eth3 mask=8000 for /proc/irq/116/smp_affinity
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#

Now we have the latest drivers installed (at the time of this writing) and we have run the affinity script:

   *-network:1
       description: Ethernet interface
       product: 82599EB 10-Gigabit SFI/SFP+ Network Connection
       vendor: Intel Corporation
       physical id: 0.1
       bus info: pci@0000:04:00.1
       logical name: eth3
       version: 01
       serial: 00:e0:ed:19:e3:e1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress vpd bus_master cap_list ethernet physical fibre
       configuration: autonegotiation=off broadcast=yes driver=ixgbe driverversion=3.18.7 duplex=full firmware=0x800000cb latency=0 link=yes multicast=yes port=fibre promiscuous=yes
       resources: irq:37 memory:fbc00000-fbc1ffff ioport:e000(size=32) memory:fbc40000-fbc43fff memory:fa700000-fa7fffff memory:fa600000-fa6fffff

We need to disable all offloading on the network card in order for the IDS to be able to see the traffic as it is supposed to be (without checksums,tcp-segmentation-offloading and such..) Otherwise your IDPS would not be able to see all "natural" network traffic the way it is supposed to and will not inspect it properly.

This would influence the correctness of ALL outputs including file extraction. So make sure all offloading features are OFF !!!

When you first install the drivers and card your offloading settings might look like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
root@suricata:/home/pevman/ixgbe-3.18.7/scripts#

So we disable all of them, like so (and we load balance the UDP flows for that particular network card):

ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off

Your output should look something like this:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 lro off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 gso off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 tx off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 sg off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 rxvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -K eth3 txvlan off
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp4 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -N eth3 rx-flow-hash udp6 sdfn
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp6
UDP over IPV6 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -n eth3 rx-flow-hash udp4
UDP over IPV4 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 rx-usecs 0 rx-frames 0
root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -C eth3 adaptive-rx off

Now we doublecheck and run ethtool again to verify that the offloading is OFF:

root@suricata:/home/pevman/ixgbe-3.18.7/scripts# ethtool -k eth3
Offload parameters for eth3:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off

Ring parameters on the network card:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:            4096

Current hardware settings:
RX:             512
RX Mini:        0
RX Jumbo:       0
TX:             512

We can increase that to the max Pre-set RX:

root@suricata:~# ethtool -G eth3 rx 4096

Then we  have a look again:

root@suricata:~# ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             4096
Current hardware settings:
RX:             4096
RX Mini:        0
RX Jumbo:       0
TX:             512

Making network changes permanent across reboots

On Ubuntu for example you can do:

root@suricata:~# crontab -e

Add the following:

# add cronjob at reboot - disbale network offload
@reboot /opt/tmp/disable-network-offload.sh

and your disable-network-offload.sh script (in this case under /opt/tmp/ ) will contain the following:

#!/bin/bash
ethtool -K eth3 tso off
ethtool -K eth3 gro off
ethtool -K eth3 ufo off
ethtool -K eth3 lro off
ethtool -K eth3 gso off
ethtool -K eth3 rx off
ethtool -K eth3 tx off
ethtool -K eth3 sg off
ethtool -K eth3 rxvlan off
ethtool -K eth3 txvlan off
ethtool -N eth3 rx-flow-hash udp4 sdfn
ethtool -N eth3 rx-flow-hash udp6 sdfn
ethtool -C eth3 rx-usecs 1 rx-frames 0
ethtool -C eth3 adaptive-rx off

with:

chmod 755 disable-network-offload.sh

To make sure you have the ixgbe module always loaded at boot time you can add "ixgbe" to the  /etc/modules file.

Kernel specific tunning

Certain adjustments in parameters in the kernel can help as well :

sysctl -w net.core.netdev_max_backlog=250000
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.rmem_default=16777216
sysctl -w net.core.optmem_max=16777216

Making kernel changes permanent across reboots

example:

echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf

reload the changes:

sysctl -p

OR for all the above adjustments:

echo 'net.core.netdev_max_backlog=250000' >> /etc/sysctl.conf
echo 'net.core.rmem_max=16777216' >> /etc/sysctl.conf
echo 'net.core.rmem_default=16777216' >> /etc/sysctl.conf
echo 'net.core.optmem_max=16777216' >> /etc/sysctl.conf
sysctl -p

Suricata.yaml configuration  (file extraction specific)

As of Suricata 1.2  - it is possible to detect and extract/store over 5000 types of files from HTTP sessions.

Specific file extraction instructions can also be found in the official page documentation.

The following libraries are needed on the system running Suricata :

apt-get install libnss3-dev libnspr4-dev

Suricata also needs to be compiled with file extraction enabled (not covered here).

In short in the suriacta.yaml, those are the sections below that can be tuned/configured and affect the file extraction and logging:
(the bigger the mem values the better on a busy link )

  - eve-log:
      enabled: yes
      type: file #file|syslog|unix_dgram|unix_stream
      filename: eve.json
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      types:
        - alert
        - http:
            extended: yes     # enable this for extended logging information
        - dns
        - tls:
            extended: yes     # enable this for extended logging information
        - files:
            force-magic: yes   # force logging magic on all logged files
            force-md5: yes     # force logging of md5 checksums
        #- drop
        - ssh

For file store to disk/extraction:

   - file-store:
      enabled: yes       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: yes   # force logging magic on all stored files
      force-md5: yes     # force logging of md5 checksums
      #waldo: file.waldo # waldo file to store the file_id across runs

 stream:
  memcap: 32mb
  checksum-validation: no      # reject wrong csums
  inline: auto                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 128mb
    depth: 1mb                  # reassemble 1mb into a stream

  
depth: 1mb , would mean that in one tcp reassembled flow, the max size of the file that can be extracted is just about 1mb.

Both stream.memcap and reassembly.memcap (if reassembly is needed) must be big enough to accommodate the whole file on the fly in traffic that needs to be extracted PLUS any other stream and reassembly tasks that the engine needs to do while inspecting the traffic on a particular link.

 app-layer:
  protocols:
....
....
     http:
      enabled: yes
      # memcap: 64mb

The default limit for mem usage for http is 64mb   , that could be increased , ex - memcap: 4GB -  since HTTP is present everywhere and a low memcap on a busy HTTP link would limit the inspection and extraction size ability.

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 3072
           response-body-limit: 3072

The default values above control how far the HTTP request and response body is tracked and also limit file inspection. This should be set to a much higher value:

        libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 1gb
           response-body-limit: 1gb

 or 0 (which would mean unlimited):

       libhtp:

         default-config:
           personality: IDS

           # Can be specified in kb, mb, gb.  Just a number indicates
           # it's in bytes.
           request-body-limit: 0
           response-body-limit: 0

and then of course you would need a rule loaded(example):

alert http any any -> any any (msg:"PDF file Extracted"; filemagic:"PDF document"; filestore; sid:11; rev:11;)

That's it.