Hello ​XDP_DROP

For the last few years extended Berkeley Packet Filter (eBPF) has been the topic of interest for kernel developers and researchers, however this is no longer the case. Very recently, eBPF has been deployed into real-life applications such as Facebook’s Katran Load Balancer, proving the benefits over existing solutions.

eBPF functionality is included by default on most recent Linux distributions. Open source tools for eBPF are also readily available, however the one important step currently missing is communicating how simple it is to use. This blog aims to rectify this and demonstrate how easy it is to load your first XDP program on a fresh Linux host to filter network packets. A step-by-step video demonstration of the tutorial is also included at the end of this blog.

We will first demonstrate how simple it is to deploy an eBPF program onto the XDP hook. We will later demonstrate how the same program can be offloaded on to an Agilio SmartNIC, which allows for the processing to be carried out on the network card hardware to greatly improve performance. The below video helps demonstrate the actions being performed in this blog.

Driver Mode

Ingress packets are transmitted up through the kernel layers from the network card as shown in the diagram below. Packets may be destined for a userspace application, or may be intercepted by a kernel module such as iptables located within Netfilter. eBPF programs can hook into various locations within the Linux kernel and filter packets. Optimum performance can be obtained by filtering packets at the earlier kernel layers, resulting in the XDP hook becoming the optimum choice for eBPF.

The XDP hook occurs within the driver of the network device, allowing for the packets to be processed by the CPU at the earliest point. XDP driver compatibility can be seen on this page. If you wish to use a virtual machine or a non compatible XDP device for experimenting, a socket version of XDP can be used by using just “xdp” in the ip link commands below.

The following example will require an up-to-date kernel, clang and iproute2. All of which are available from the Ubuntu 18.04 or Fedora 28 repositories. Refer to our eBPF Offload Getting Started Guide for more in depth instructions.

To begin, we write a simple C program to process incoming packets and return XDP_DROP which will block all incoming packets from reaching the host.

#include <linux/bpf.h>

int main()
{
    return XDP_DROP;
}

After saving the file as xdp.c, the code can be compiled to an eBPF object using a simple clang command. Clang contains an eBPF backend allowing for eBPF assembly code to be compiled.

$ clang -target bpf -O2 -c xdp.c -o xdp.o

Once the program is compiled, it can be loaded using ip link (change [DEV] to the relevant interface name). Our code does not contain a section label, therefore the default ELF section name .text is used for loading.

# ip -force link set dev [DEV] xdpdrv obj xdp.o sec .text

Now that the program has been loaded, ip link will now show the eBPF program is attached to the interface on the XDP hook.

$ ip link show dev [DEV]
6: DEV: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc mq state 
UP mode DEFAULT group default qlen 1000
    link/ether 00:15:4d:13:08:80 brd ff:ff:ff:ff:ff:ff
    prog/xdp id 27 tag f95672269956c10d jited

hping3 can be used on another host as a simple traffic generator..

$ hping3 [IP Address of Host]

Traffic can be monitored using tcpdump, however it will show that no packets are received. This is due to XDP dropping packets at the start of the kernel path, before the packets can reach tcpdump.

To unload the XDP program, a similar ip link command can be used, but with the off parameter.

# ip link set dev [DEV] xdpdrv off

The assembly code generated by the clang compiler, can be viewed using llvm-objdump.

$ llvm-objdump -d xdp.o

xdp.o:      file format ELF64-BPF 
Disassembly of section .text:
main:
      0:   b7 00 00 00 01 00 00 00    r0 = 1
      1:   95 00 00 00 00 00 00 00    exit

The return value of the program is stored in register 0. Each of the XDP Actions have a corresponding value.

0     XDP_ABORTED  -  Error, Block the packet
1     XDP_DROP     -  Block the packet
2     XDP_PASS     -  Allow the packet to continue up the kernel
3     XDP_TX       -  Bounce the packet back in the direction it came from

These XDP action can be tested using the same method as above.

Offload

We have just run eBPF on XDP, this hook is located within the driver, and utilizes the host x86 CPU for packet handling. If the network interface supports eBPF offload, this will allow for the program to be executed directly on the network card processor. Currently, the only hardware with this support is a Netronome Agilio CX SmartNIC.

All versions of the kernel and iproute newer than 4.16 contain Agilio CX offload support. The only installation required is our eBPF firmware located on our support site. Check our eBPF Offload Getting Started Guide for installation information.

To offload the program, the same method can be utilized, the only exception is that xdpoffload is used within the ip link command.

# ip link set dev [DEV] xdpoffload obj xdp.o sec .text

$ ip link show dev [DEV]
6: DEV: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpoffload qdisc 
mq state UP mode DEFAULT group default qlen 1000
   link/ether 00:15:4d:13:08:80 brd ff:ff:ff:ff:ff:ff
   prog/xdp id 26 tag f95672269956c10d jited

Monitoring tcpdump again will show traffic being dropped. If high rate traffic is sent to the interface, the host CPU will remain idle as the processing will be carried out on the energy efficient SmartNIC.

To unload the program, set xdpoffload to off.

# ip link set dev [DEV] xdpoffload off

Monitoring XDP statistics

Enhanced monitoring of XDP statistics is available using either eBPF maps to record counters, or by monitoring ethtool statistics. eBPF map functionality will be demonstrated in a future tutorial, for now we recommend using ethtool statistics to indicate the offload actions performed by the eBPF program.

Ethtool statistics can be shown using the following command:

# ethtool -S [DEV]

To visually display these counters in a readable format, we recommend the use of the “stat watch“ script, available from the Netronome driver repository.

$ nfp-drv-kmods/tools/stat_watch.py -c [DEV]

Conclusion

In this article we have demonstrated how simple it is to load an eBPF program on XDP. More complex eBPF functionality is available and will be explained in a later post. If you cannot wait, check out our eBPF Demo repository and the video shown below.

Also, Netronome is hosting a three-part Fall eBPF Webinar Series. Register here!