The Arrival of SDN 2.0: SmartNIC Performance, COTS Server Efficiency and Open Networking

The era of SDN 2.0 has arrived, with strong emphasis on efficient scaling of server-based networking and moving the brains of networking closer to virtual machines (VMs) and virtual network functions (VNFs) now, and containers in the near future. The importance of open networking – aka hardware and software disaggregation and vendor independence – persists. The era of SDN 1.0 was a period of learning and bringing disaggregation and vendor independence to data center network switches. The largest and most advanced data center operators also innovated and deployed server-based networking. With SDN 2.0, those benefits will come to all data centers that want to reap the benefits of SDN and cloud practices. SmartNICs are needed to free up the x86 CPU cores from being consumed for server-based networking functions and deliver needed performance to VMs. This means that portions of the server-based networking functions – especially the data plane functions – need to be offloaded from the x86 (that is open) and implemented in the SmartNIC. Does this result in vendor lock? If done right, it should not.

On the surface, the easiest path would simply be to compile the x86 C-based data plane functions into Arm or MIPS cores included in a SmartNIC. Vendor independence available in the x86 is maintained in the SmartNIC. The question is how many Arm or MIPS cores can be packed into a sub 25W low profile SmartNIC form factor needed in commercial-of-the-shelf (COTS) servers (namely within COTS server economies), and what performance can be achieved in Mpps (million packets per second) with such a SmartNIC. The answer, based on feedback from multiple data center operators I have worked with, is ~5Mpps, which is barely enough for a 10GbE network. Add in more sophisticated network processing (such as much needed scale in flow rules processing for security, or tunnel encap/decap processing for network virtualization) and the performance is even lower. Ultimately, while this approach is vendor-independent, performance scaling and future proofing (to higher speed networks) are significant challenges.

Many in the industry believe using commodity hardware is important. FPGAs are considered commodity because they are available from large vendors, and there are many FPGA programmers available. If the data center operator takes on the task of converting the x86 C code to FPGA code, vendor independence can be maintained. This of course entails converting the server-based networking functions (such as the virtual switch (vSwitch) datapath) C code on the x86 server to HDL/Verilog code, fitting the functions within the resources available in the FPGA, closing timing and delivering the needed 20-30Mpps performance in modern 25/40/50GbE networks, and accomplishing all of that within COTS server economies. To date, only Microsoft is known to have ventured into building SmartNICs with FPGAs, and has spoken publicly about offloading its proprietary Virtual Filtering Platform (VFP) to FPGAs, which has demonstrated encrypted virtual networking using such a SmartNIC. It is not clear how many chips are used in such a SmartNIC, the size and cost of the FPGA used and if it performs open virtual switching offloads with flow processing scale within COTS server economies. Aside from the potential high cost of such a SmartNIC, this entails significant R&D and unpredictable ROI that only companies with deep pockets can sustain. There is no publicly available information that proves that an FPGA-based SmartNIC solution that implements server-based networking functions such as vSwitch offload is able to meet the performance, power and cost profiles acceptable in COTS servers. In summary, this approach can provide vendor independence, however, it requires significant investment and there is no proof that the solution can scale to needed performance with COTS server economies.

The Netronome Agilio server-based networking platform optimally solves performance, COTS server economies and vendor independence requirements. There are many previous blogs and collateral on the Netronome website that show how high performance can be achieved for server-based networking within COTS server economies, lowering TCO by up to 6X. As such, in this blog, I will focus on the open networking and vendor-independence aspects of the Netronome Agilio solution. There are many facets of vendor-independence, let me address them one by one.

Is the SmartNIC hardware disaggregated from the SmartNIC data plane software?

Yes.

The Netronome Agilio solution includes SmartNIC and multiple open-source based off-the-shelf data plane software options such as Agilio OVS, Agilio vRouter and Agilio Firewall (based on Connection tracking or Conntrack). The off-the-shelf data plane software options help reduce development costs and shorten time to market significantly. The software is supplied separately from the hardware and different software options highlighted above can be installed on any of the available hardware SKUs (2x10GbE, 1x40GbE, 2x40GbE now and 2x25GbE available soon). The data plane software is loaded from the host or the server adapter flash. The data plane software can also be upgraded or changed in the field.Software drivers associated with the data plane software are installed using standard mechanisms e.g., Linux packages.

Can I control the evolution of the data plane software on the SmartNIC?

Yes.

The off-the-shelf Agilio OVS, Agilio vRouter and Agilio Firewall data plane software packages are provided by Netronome under a Netronome software license. This week, Netronome announced the industry’s first P4 and C Integrated Development Environment (IDE) for the Agilio Platform. This IDE will enable you to add additional customer or third party-defined functions to the Netronome-provided data path functions, at your own pace and when you want to do so.

Is the SDN Controller and Cloud Orchestration Software disaggregated from the SmartNIC hardware and data plane software?

Yes.

The Netronome data plane software offload implementations for OVS, vRouter and Linux Firewall are compatible with existing, standard and open source datapaths on the host/x86. As such, SDN Controllers and Cloud Orchestration Software developed based on such datapaths on the host/x86 are disaggregated from the Netronome data plane software offload implementations the same way they are disaggregated from the corresponding datapaths implemented on the host/x86. SDN Controllers such as Open Daylight and ONOS are applicable and interchangeable. Cloud Orchestration Software such as OpenStack is also applicable, including independence from specific OpenStack distributions.

Do I need to install Netronome drivers in the VMs?

Netronome supports both SR-IOV and Virtio. With Netronome’s Virtio implementation, no Netronome-specific drivers are needed in the VM. Both netdev and DPDK are supported in the VMs and the solution delivers high performance (Mpps) to the applications in VMs. This means that the VMs and VNFs in the server can be independent of underlying Netronome hardware, and still enjoy the benefits of performance and CPU savings enabled by the data plane software offloads.

Can I define and develop the SmartNIC data plane software from scratch?

The P4 and C IDE announced this week can be utilized to define and develop your own SmartNIC data plane software from scratch. This means you do not have to use the off-the-shelf Agilio data plane software that Netronome provides. A beta release of the IDE is available now and will be generally available (GA) in July 2016. Netronome experts will present and conduct multiple hands-on labs and tutorials at the P4 Language Consortium (www.P4.org) workshop being held this week at Stanford. If you are attending this workshop, you can get a flavor of what’s it like to write your own cool data plane software and actually run it on Netronome Agilio SmartNICs. Netronome hosts open development work using the IDE on the portal www.Open-NFP.org. Please visit this site to get a preview of additional tutorials, projects and research being pursued using the IDE.

Can I protect my investment in SmartNIC data plane software development from being vendor-locked?

Yes.

That is the essence of the open source P4 language and compilers being defined by the P4 Language Consortium. When you use the Netronome P4 and C IDE, you can write P4-based data plane software for the Netronome Agilio hardware. P4 is vendor hardware-independent, hence your investment to develop P4-based data path software remains vendor-independent. The Netronome IDE includes the open source P4 complier from the P4 Language Consortium. The code compiled by the P4 compiler is fed into a Netronome back-end compiler that automatically creates the loadable firmware for the Netronome Agilio hardware. In scenarios where the P4 language is not sufficient or suitable for certain complex networking functions, Netronome’s IDE provides the ability to develop such functions using the C language and integrate those with the P4-based data path.

In summary, the Netronome Agilio solution is ready for SDN 2.0. Are you?