SmartNICs – Requirements for Successful and Mainstream Deployments

I had the pleasure of attending the Future:Net event in Las Vegas last week. It was invigorating to see the experts in networking technologies come together, including many former colleagues from companies I have worked at in the past. The theme that evolved on the second day was network programmability, focusing on the technologies that are relevant in the server. SmartNICs – a core part of network programmability on the server – featured prominently at a panel that I participated in at the event. This session was ably moderated by Constantine Polychronopoulos, Vice President and Chief Technology Officer of Telco/NFV at VMware. Other panelists were from Broadcom, Cavium and Mellanox – significant players in the world of networking. This blog focuses on Netronome’s position that I discussed on the panel. It was great to see other panelists agree on critical requirements for success.

We all know NICs or network interface cards. SmartNICs are a new breed of NIC, and the market for them is growing rapidly, thanks to the adoption of software-defined networking (SDN) and network functions virtualization (NFV). A SmartNIC offloads SDN and NFV-centric functions like virtual switching, virtual routing and, in general, match-action processing. Also, SmartNICs are programmable, enabling the feature set to evolve quickly and synergistic to the mantra of being software-defined.

NICs have been deployed for years in mainstream applications. NICs are built using fixed-function ASICs and their requirements for success are well established and understood. Success of the SmartNIC market however depends on a new set of factors – namely, the robust sum of the following 6 parts:

Part I: Hardware and Software Supplier Ecosystem

Multiple, healthy SmartNIC offload initiatives are gaining momentum in the industry. Cavium and Netronome have been playing in the SmartNIC market for a while now. Recently, NIC incumbents Mellanox and Broadcom have entered the SmartNIC market with new products. Multiple Linux-based SmartNIC offload software initiatives are very active in the open source community, specifically OVS-TC (Open vSwitch data plane offload using the Linux Traffic Control interface for configuring the Linux kernel packet scheduler) and eBPF (extended Berkeley Packet Filter). In the Linux user space, DPDK and FD.IO/VPP (vector packet processing) are worth mentioning. Finally, many commercial SDN solutions have been announced that enable offload to SmartNICs (Ericsson, Nuage from Nokia, Mirantis and Juniper, for example) and more are expected in the coming quarters.

Part II: Mainstream Use Cases

SmartNICs are serving a growing and large set of data center applications that broadly fall under the category of SDN and NFV. Specific use cases are network virtualization, security, load balancing and telemetry/visibility at scale. Micro-services for containers is an emerging and promising use case.

Part III: Balanced Silicon Architecture

This aspect of SmartNICs requires careful attention. The silicon in a SmartNIC includes multiple blocks that are tightly linked together. Delivering the highest networking performance at the lowest cost and power requires all the blocks and links to perform in tandem and at scale. The blocks include the SerDes, MAC, processing cores for a data plane that can be changed, fixed data planes, hardware accelerators, memory for tables and buffer and host interfaces (PCIe). For example, enabling 25 or 50Gb/s of throughput through the SerDes, MAC and host interface is futile if the processing cores and access to tables in memory perform only at 8Gb/s.

Part IV: Open Data Plane Software and Offload Interfaces

Availability of open and standard data plane software that can be offloaded to SmartNICs is important. There are many such options today, including OVS, OpenContrail, Linux Bridge, IP Tables, Connection Tracking, Linux TC, VPP, and DPDK amongst others. Some options support industry and community-accepted methods for offloading such data planes to SmartNICs. They are either kernel or user space-based offload interfaces. History tells us to “never bet against the kernel,” so accepted and standard kernel-based offload interfaces are critical. In this context, OVS-TC with Linux kernel packet scheduler and eBPF are noteworthy – areas Netronome is squarely focused on.

Part V: Balanced Software Architecture

With any data plane software, there is usually associated control plane software. Within the data plane software, sometimes there is the notion of the slow path and the fast path. There are many interactions across the fast path, slow path and the control plane software as packets traverse through a SmartNIC. Some components such as the fast path perform best in multi-threaded environments encompassing the processing cores, memory and hardware accelerators. In other cases such as control plane software, single-threaded processing environments are appropriate. SmartNICs operate as a coprocessor to the host general-purpose CPU, which is typically x86 today and could be arm as well in the future. General-purpose CPUs are single-threaded while some SmartNICs support optimal multi-threaded processing environments for the data plane, especially the fast path. To deliver the best price-performance, optimum placement of software across those processing environments is crucial.

Part VI: Vendor-agnostic Programmability

Finally, SmartNICs by definition need to be programmable to enable flexibility and fast feature innovation. Open source Linux-based programming methods that enable creation of data planes in the Linux kernel and their transparent offload to SmartNICs are ideal. Currently, SmartNIC vendors offer the following options which are not completely vendor agnostic:

• Using Linux-based compilers that enable porting of x86-based data plane software to the SmartNIC. Use of hardware accelerators and any available optimal memory access methods usually require porting of the code to the specific SmartNIC architecture.
• Proprietary programming tools that support standard and open programming languages such as P4, C or multi-threaded versions of C (similar to CUDA in the world of graphics and AI).

Today, eBPF is used heavily by some of the largest data center operators. Standard user mode Linux LLVM compilers can be used to develop data plane functions that can be injected dynamically into the Linux kernel. When a SmartNIC JIT (just in time) compiler is available in the kernel, such data plane functions can be transparently offloaded to the SmartNIC, providing automatic acceleration for data plane code that was written for the kernel. This is the Holy Grail for vendor-agnostic programmability.

So with these six factors in place, the future of the SmartNIC market looks bright indeed.