Kernel 4.18 and eBPF Sample Apps
This month marks the release of both kernel 4.18 and our eBPF sample repository. The kernel capabilities have been improved with the release of AF_XDP, allowing for user space applications to bypass the networking stack and communicate directly with the XDP driver. The core bpfilter functionality has also been introduced, laying the foundations for future eBPF firewalling solutions. eBPF offload capabilities have also improved including libbpf support, perf event map support and programmable RSS. Interesting times for offloading eBPF programs!
We have also introduced an extended Berkeley Packet Filter (eBPF) sample repository, which currently contains two XDP demo apps. We currently have a Layer 4 load balancer and a programmable RSS demo with more to come.
Our load balancer application demonstrates how an eBPF program can be utilized to distribute incoming traffic to up to 512 servers. Netronome is certainly not the first to showcase a load balancer app, for Facebook has recently published their fully operational Katran eBPF load balancer. Katran is highly sophisticated, so it can be a steep learning curve to install and configure, especially for those starting out with eBPF. With our demo app, we demonstrate a simpler load balancer, which can be controlled using Python scripts and bpftool, a tool that we have open sourced and is now part of the Linux kernel.
Our load balancer receives incoming packets, extracts the IP address and port numbers and calculates a hash based on these values. The hash value is subsequently used with an eBPF map to determine the network destination for the packet. The IP hash values will stay constant for the duration of each flow, resulting in a constant destination for packets from the same flow.
An outer destination IP header is inserted into the start of the packet, and the packet is transmitted back out of the interface to the target destination. Load balancer statistics can also be monitored with the included Python stats script.
Programmable RSS
A world first for us, is eBPF programmable RSS. What is RSS? It stands for “Receive Side Scaling,” a method utilized by modern network cards to determine which host CPU the packet should be sent to, enabling packet distribution. A hash function is applied to the packet headers (commonly the four-tuple IP and port, source and destination) to generate a hash for the flow, which designates which CPU RSS queue to send the packet to.
The RSS algorithms available on network cards are commonly closed source or fixed in hardware, therefore cannot be adapted to non-standard use cases. With kernel 4.18, my colleague, Jakub Kicinski, has addressed this issue by implementing an exciting new feature, programmable RSS using eBPF. This eBPF feature is only available on offload, i.e. when the eBPF program is run directly on the network card.
Our demo app illustrates how packets can be distributed to:
● a single queue, chosen by the user through a userspace utility,
● distributed queues using a hash algorithm,
● distributed queues using a Symmetric RSS hash algorithm,
● distributed queues using a hash algorithm against the IPinIP inner headers.
RSS distribution is a major concern for certain applications, such as Suricata or Snort intrusion detection systems (IDS). These applications require CPU state access locality across all packets of the flow, to help minimize cache bouncing and achieve higher performance. Symmetric RSS is a mode which ensures that packets sent in both directions of a flow map to the same queue. For example, a connection between hosts A and B, both packets sent by A and by B will be placed on the same CPU queue.
Custom header RSS (or Encapsulated RSS) allows users to benefit from RSS even if they are using custom headers or uncommon encapsulations. Default RSS implementations are only able to parse common protocols. With programmable RSS users can include any field of the headers in RSS calculation. Users can also parse any encapsulation protocol they have in their networks. This is beneficial to overlay and trunk networks, where the outer IP header is relatively static resulting in a badly distributed RSS. The inner IP header addresses can have more variance hence a better RSS distribution.
Our demo showcases several RSS modes. However, as this functionality is fully programmable, a RSS queue could be based on any header data such UDP encapsulations (VXLAN, Geneve, FOU, GUE, etc.), NSH, QUIC or any other protocol.
We hope that these eBPF apps will be of use to the eBPF community. Check them out on our GitHub repository and also refer to our eBPF Getting Started Guide for eBPF installation instructions!
Also, Netronome is hosting a three-part Fall eBPF Webinar Series. Register here!
Netronome
Subscribe
Categories
Recent Blogs
- What We Do
- Optimizing BPF: Smaller Programs for Greater Performance
- Open-Sourcing the CoreNIC Firmware
- Libkefir: All Your Rules in One Bottle
- Overcoming Tail Latency with Netronome SmartNICs
Archives
- February 2020
- January 2020
- September 2019
- July 2019
- June 2019
- February 2019
- January 2019
- November 2018
- October 2018
- September 2018
- August 2018
- May 2018
- April 2018
- March 2018
- February 2018
- November 2017
- September 2017
- June 2017
- May 2017
- April 2017
- March 2017
- February 2017
- January 2017
- December 2016
- November 2016
- October 2016
- September 2016
- August 2016
- June 2016
- May 2016
- April 2016
- March 2016
- February 2016
Featured Author
Quentin Monnet
Quentin Monnet is a senior software engineer at Netronome, where he works on advanced technologies at the driver level, such as the software pieces of eBPF offload, and modestly contributes to several open source projects. Before joining Netronome, he was part of the Research & Development team at 6WIND, and was involved in European research projects that focused on fast, flexible and stateful packet processing for SDNs. Quentin holds a Ph.D. in Computer Science from Université Paris-Est, France.