Monday, July 28, 2014

Compiling your own perf utility tools

The perf utility is part of the linux-tools package and can be installed via :
apt-get install linux-tools

However..... if you have just installed a newer (and or custom) kernel version (as explained here for example)....let's say 3.14.0 on Ubuntu Precise ... the apt-get install would not find the appropriate linux-tools package for that kernel version.

So this is what yo can do:

apt-get install libdw-dev libnewt-dev binutils-dev git
git clone https://github.com/torvalds/linux.git
cd linux
git tag -l

Make sure you checkout the same version as on the currently installed on the system kernel.





git checkout tags/v3.14
cd tools/perf
make -j `getconf _NPROCESSORS_ONLN` perf
make install


EXAMPLE:
root@suricata:~/oisf# perf list

List of pre-defined events (to be used in -e):
  cpu-cycles OR cycles                               [Hardware event]
  instructions                                       [Hardware event]
  cache-references                                   [Hardware event]
  cache-misses                                       [Hardware event]
  branch-instructions OR branches                    [Hardware event]
  branch-misses                                      [Hardware event]
  bus-cycles                                         [Hardware event]
  stalled-cycles-frontend OR idle-cycles-frontend    [Hardware event]
  stalled-cycles-backend OR idle-cycles-backend      [Hardware event]
  ref-cycles                                         [Hardware event]

  cpu-clock                                          [Software event]
  task-clock                                         [Software event]
  page-faults OR faults                              [Software event]
  context-switches OR cs                             [Software event]
  cpu-migrations OR migrations                       [Software event]
  minor-faults                                       [Software event]
  major-faults                                       [Software event]
  alignment-faults                                   [Software event]
  emulation-faults                                   [Software event]
  dummy                                              [Software event]

  L1-dcache-loads                                    [Hardware cache event]
  L1-dcache-load-misses                              [Hardware cache event]
  L1-dcache-stores                                   [Hardware cache event]
  L1-dcache-store-misses                             [Hardware cache event]
  L1-dcache-prefetch-misses                          [Hardware cache event]
  L1-icache-load-misses                              [Hardware cache event]
  LLC-loads                                          [Hardware cache event]
  LLC-load-misses                                    [Hardware cache event]
  LLC-stores                                         [Hardware cache event]
  LLC-store-misses                                   [Hardware cache event]
  LLC-prefetches                                     [Hardware cache event]
  LLC-prefetch-misses                                [Hardware cache event]
  dTLB-loads                                         [Hardware cache event]
  dTLB-load-misses                                   [Hardware cache event]
  dTLB-stores                                        [Hardware cache event]
  dTLB-store-misses                                  [Hardware cache event]
  iTLB-loads                                         [Hardware cache event]
  iTLB-load-misses                                   [Hardware cache event]
  branch-loads                                       [Hardware cache event]
  branch-load-misses                                 [Hardware cache event]
  node-loads                                         [Hardware cache event]
  node-load-misses                                   [Hardware cache event]
  node-stores                                        [Hardware cache event]
  node-store-misses                                  [Hardware cache event]
  node-prefetches                                    [Hardware cache event]
  node-prefetch-misses                               [Hardware cache event]

  branch-instructions OR cpu/branch-instructions/    [Kernel PMU event]
  branch-misses OR cpu/branch-misses/                [Kernel PMU event]
  bus-cycles OR cpu/bus-cycles/                      [Kernel PMU event]
  cache-misses OR cpu/cache-misses/                  [Kernel PMU event]
  cache-references OR cpu/cache-references/          [Kernel PMU event]
  cpu-cycles OR cpu/cpu-cycles/                      [Kernel PMU event]
  instructions OR cpu/instructions/                  [Kernel PMU event]
  mem-loads OR cpu/mem-loads/                        [Kernel PMU event]
  mem-stores OR cpu/mem-stores/                      [Kernel PMU event]
root@suricata:~/oisf#


EXAMPLE:
root@suricata:~/oisf#perf top



..so you are back in business.

Monday, July 7, 2014

Kernel upgrade for Debian

Kernel upgrade for Debian - a quick and useful tutorial. This guide will also show how to make kernel*.deb - debian packages ready for easy "dpkg -i ..* " installation of your desired kernel version.

Some packages you might want to have on the system prior to building the new kernel:

apt-get install wget fakeroot kernel-package gcc libncurses5-dev bc ca-certificates pkg-config make flex bison build-essential autoconf automake

Choose the kernel you want to build Debian packages for from here:
https://www.kernel.org

Please make sure you read here :
https://www.kernel.org/category/faq.html
and here:
https://www.kernel.org/category/releases.html

and know the difference between "Mainline,Stable and Longterm" with regards to kernel versions.

Then you could look up the kernel version changes in here:
http://kernelnewbies.org/LinuxChanges
http://kernelnewbies.org/LinuxVersions
http://kernelnewbies.org/Linux_3.15

For the purpose of this tutorial I have chosen kernel version 3.15.1

wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.15.1.tar.xz
tar xfJ linux-3.15.1.tar.xz
cd linux-3.15.1

If you will be updating/installing the new kernel on the same machine that you will be building it, you could do:
yes "" | make oldconfig

If you will be updating/installing the new kernel on a different machine than the one that you are building it - it is better to do:
make defconfig

More about these differences and the implications of it you can read here:
https://www.kernel.org/doc/makehelp.txt
and here
https://www.kernel.org/doc/index-old.html#Using_an_existing_configuration

    make defconfig - Set all options to default values
    make allnoconfig - Set all yes/no options to "n"
    make allyesconfig - Set all yes/no options to "y"
    make allmodconfig - Set all yes/no options to "y" and all "yes/module/no" options to "m"
    make randconfig - Set each option randomly (for debugging purposes).
    make oldconfig - Update a .config file from a previous version of the kernel to work with the current version.


Then we do:

make clean && \
make -j `getconf _NPROCESSORS_ONLN` deb-pkg LOCALVERSION=-custom KDEB_PKGVERSION=3.15.1
above
make -j `getconf _NPROCESSORS_ONLN`
means that if you have 4 CPUs it will start 4 kernel make jobs in parallel - provides for much faster building.
Now in the directory above
cd ..
you should have the new kernel.deb packages for the same architecture as the machine that you just build it on - example 64 bit

drwxrwxr-x 25 root root 4.0K Jul  7 14:40 linux-3.15.1
-rw-r--r--  1 root root  76M Jun 16 16:54 linux-3.15.1.tar.xz
-rw-r--r--  1 root root 9.9M Jul  7 14:40 linux-headers-3.15.1-custom_3.15.1_amd64.deb
-rw-r--r--  1 root root 6.1M Jul  7 14:40 linux-image-3.15.1-custom_3.15.1_amd64.deb
-rw-r--r--  1 root root 979K Jul  7 14:40 linux-libc-dev_3.15.1_amd64.deb


install as follows and then reboot:
dpkg -i linux-headers-3.15.1-custom_3.15.1_amd64.deb linux-image-3.15.1-custom_3.15.1_amd64.deb linux-libc-dev_3.15.1_amd64.deb


So only the commands themselves - as easy as one two three...:

apt-get install wget fakeroot kernel-package gcc libncurses5-dev bc ca-certificates pkg-config make flex bison build-essential autoconf automake

wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.15.1.tar.xz

tar xfJ linux-3.15.1.tar.xz

cd linux-3.15.1

yes "" | make oldconfig

make clean && \
make -j `getconf _NPROCESSORS_ONLN` deb-pkg LOCALVERSION=-NewKernel KDEB_PKGVERSION=3.15.1

cd ..

dpkg -i linux-headers-3.15.1-custom_3.15.1_amd64.deb linux-image-3.15.1-custom_3.15.1_amd64.deb linux-libc-dev_3.15.1_amd64.deb
   
    as easy as that...