Category Archives: NPAP Architecture

System-Level Architecture Aspects

MLE NPAP System-Level Architecture Aspects for TCP/UDP/IPv4 Team MLE has gained long and deep experiences when using TCP/UDP/IPv4 for networked and distributed systems and will support you in identifying, implementing and testing the right architecture choice. Key aspects are outlined below: Picking the “Right” TCP Peer Chosing the Right TCP Rx/Tx Buffer Sizes Scaling the Number of TCP Cores Optimize for Linerate Performance Hybrid Acceleration Time-Sensitive Networking (TSN) Linux Kernel Bypass Transport Layer Security (TLS)

Read More

Picking the “Right” TCP Peer

Picking the “Right” TCP Peer MLE NPAP has a clear focus on highest performance in a very resource efficient FPGA implementation which is reliable (the key reason to use TCP) and interoperable with (almost) any other TCP/UDP/IPv4 stack.  When you optimize data transports towards low latency and/or high bandwidth, keep in mind how both, TCP flow control and TCP congestion control, function and do parameterize both peers accordingly.  MLE NPAP runs the entire protocol stack as a digital circuit. So, when MLE NPAP is on the receiving side, TCP packets will be checked, and acknowledged (ACK’ed), in a very short time and at a very high rate (close to line rate). That may challenge a “slow sender”. Similarly, when MLE NPAP is on the sender side, TCP packets will be generated and sent in a very short time and at a very high rate (close to line rate). That may challenge a “slow receiver”. Experimenting, and tweaking parameters on either side, is key to delivering good performance. Predictable high bandwidth and low latency is typically delivered by a “balanced” TCP connection, for example by putting MLE NPAP on both sides. When implementing so-called High-Speed Data Acquisition systems we do recommend investigating NPAP not only for sensor-side TCP/IPv4 acceleration but also for PC-side:

Read More

TCP Rx/Tx Buffer Sizes

Chosing the Right TCP Rx/Tx Buffer Sizes Chosing the right TCP Rx/Tx buffer sizes is fundamental to any TCP connection: If the TCP Rx/Tx buffers are undersized, then the TCP throughput may not reach maximum line rate. Hence, in software systems we typically set those TCP Rx/Tx buffers to some large value. However, in a Full Accelerator TCP Tx buffers and TCP Rx buffers require expensive on-chip memory resources. So, we suggest to chose the “right” TCP Rx/Tx buffer sizes. To help you pick a good tradeoff, please use the so-called Bandwidth-Delay Product (BDP) metric to determine TCP buffer sizes for TCP (keep in mind, TCP buffers are placed on both ends: Tx side and Rx side): Buffer size (in bits) = Bandwidth (in bits-per-second) * RTT (in seconds) RTT is the Round-Trip Time which is the time for the sender to transmit the data plus the time-of-flight for the data, plus the time it takes the recipient to check for packet correctness (CRC), plus the time for the recipient to send out the ACK, plus the time-of-flight for the ACK, plus the time it takes the sender to process the ACK and release the buffer. Here examples: Assuming a direct connection with the recipient being MLE NPAP, and a 25 GBit/s

Read More

Scaling the Number of TCP Cores

Scaling the Number of TCP Cores Table of Contents Scaling the Number of TCP CoresEffects of On-Chip Routing:Effects of TCP Bandwidth Sharing:Multiple MLE NPAP Subsystems with a Separate IPv4 Addresses:Hybrid Acceleration: In typical software systems, the cost of opening a TCP connection is quite CPU expensive, and may take a long time because of RTT and processing times in the operating system. Therefore, most software driven systems keep a TCP connection alive “forever” rather than closing it. In software, the low costs of system RAM for storing each TCP connection’s state are not worth the CPU processing costs. For MLE NPAP, each TCP connection which is open at the same time requires a dedicated TCP Core – which costs FPGA / ASIC resources. However, if RTT is low such as in a LAN, and with the very low costs of opening and closing a TCP connection in MLE NPAP (a few hundred FPGA clock cycles), “time sharing” TCP Cores can save a lot of FPGA resources without any negatives. Please refer to the MLE Technical Brief TB20201203 “Deterministic Networking with TSN-10/25/50/100G” for more information on “time sharing” TCP Cores. Deterministic Networking with TSN-10/25/50/100G Effects of On-Chip Routing: Typically, FPGA routing is a key limiting factor: The more TCP Cores you instantiate, the

Read More

Optimize for Performance

Optimize MLE NPAP for Performance Scaling performance for MLE NPAP is like any on-chip full accelerator: The wider the datapaths and the higher clock speed the better the throughput. ASIC implementations can clock at 1 GHz, or faster (depends on with ASIC process you will be using) and power is not so much of an issue because MLE NPAP can be quite small (typ. less then 200k gates). For FPGAs, however, things are different, clock frequencies above 500 MHz may be hard to achieve. MLE is constantly working with FPGA vendors to improve MLE NPAP clock frequency. While MLE NPAP originally was designed for ASIC implementation, MLE has adopted MLE NPAP for efficient implementation using modern FPGA fabric. Unlike other TCP stacks for FPGA, MLE NPAP features a 128 bit wide bi-directional datapath which puts MLE NPAP into a unique position for realizing high-bandwidth FPGA-based SmartNICs without “FPGA bloat”.  Larger bit widths, 512 bits or more, cause “FPGA bloat” which is wasting FPGA resources. Smaller bit widths, 64 bits or less, do require unrealistic high clock frequencies to deliver high line rates as the following table shows:   10 Gbps 25 Gbps 50 Gbps 100 Gbps 32 bit 312.5 MHz 781.25 MHz 1,562.5 MHz 3,125.0 MHz 64 bits 156.25 MHz 390.625 MHz

Read More

Hybrid Acceleration

Hybrid Acceleration with MLE NPAP Hybrid acceleration refers to the concept of combining full acceleration with an on-chip Network Interface Card (NIC). This NIC processes ingress network traffic in FPGA logic and then copies the network traffic via DMA to a host CPU where it is processed via the Linux TCP/UDP/IPv4 protocol stack (and vice versa for egress network traffic). Hybrid acceleration can give you the best of both worlds: MLE NPAP as the Full Accelerator takes care of certain TCP connections where high throughput and/or low, deterministic latency is important while many, many other TCP connections are processed via the standard software protocol stack without consuming chip resources. Because TCP does so-called fair bandwidth sharing (please refer to RFC2914, RFC5166 and RFC8312) Hybrid Acceleration allow for clever trade-offs between performance (as in throughput and latency) over FPGA resource usage for many TCP/IPv4 connections. For example, if you run many TCP connections over one single 100 GigE link, you may want full acceleration for, say, up to 20 TCP connections: When 3 TCP connections are open, then each one gets 33 Gbps bandwidth which clearly needs MLE NPAP as a Full Accelerator. But for moments, when 2000 TCP connections are open, then each TCP connection gets 50 Mbps bandwidth, so you can

Read More

Linux Kernel Bypass

Linux Kernel Bypass With MLE NPAP Increased Ethernet speeds push a need to offload CPUs from the burden of TCP/UDP/IPv4 processing. Various kernel bypass options exist, some include so-called RDMA (Remote DMA). NPAP can provide architecture choices for implementing such kernel bypasses. Normally, from a CPU’s perspective, TCP socket I/O means sending and/or receiving (raw) network data between the host CPU and the NIC. The kernel runs network protocol processing for IPv4, TCP, and socket APIs. User space applications that generate and/or digest the network data add to the CPU processing burden. RDMA skips most steps and a so-called rNIC (RDMA NIC) device driver directly interfaces with user space memory, effectively bypassing the kernel: MLE NPAP can operate in a similar way, where NPAP does all TCP/IPv4 processing in dedicated ASIC / FPGA logic and then a (Linux) device driver copies the payload data between user space memory and MLE NPAP. Next: Implementing Time-Sensitive Networking (TSN)

Read More

Time-Sensitive Networking (TSN)

Implementing Time-Sensitive Networking (TSN) TSN has become a set of emerging, open IEEE standards with momentum in industrial markets (for 10/100/1000 Mbps speed) and in next-generation Automotive Zone architectures (for 10/25/50 Gbps speeds). Aspects such as Traffic Shaping also find application in telecommunication, Provider Back-Bone (PBB) Switching or Software-Define Wide Area Networks (SD-WAN), for example. TSN and TCP can be combined according to the OSI Layers.  The outcome is a deterministic and reliable network protocol, which makes TCP/IPv4 over TSN a very good candidate for all networking where IT (Information Technology) and OT (Operations Technology) converge, or in Systems-of-Systems backbones. TSN itself is quickly evolving so please contact us for more information! Contact Us Next: MLE NPAP with Transport Layer Security (TLS)

Read More

Transport Layer Security (TLS)

MLE NPAP with Transport Layer Security (TLS) MLE has been working with partner Xiphera to integrate MLE NPAP with Xiphera’s TLS IP Cores for FPGA. Successful integration has been delivered to first customers. TLS is a cryptographic protocol that provides end-to-end data security, on top of the Transmission Control Protocol (TCP) layer. Implementing TLS has become a standard practice for building secure web apps. With growing needs for security compliance, for example under IEC 62443, TLS is also an option for protecting sensitive data transported using MLE NPAP. Collaboration within the FPGA ecosystem created joint solutions combining MLE’s TCP/IPv4 Network Protocol Accelerator Platform (NPAP) with Xiphera’s TLS 1.3 to ensure secure and reliable connection between devices over LAN and WAN.  Since the TCP/IPv4 stack and the TLS 1.3 security protocol – including importantly both key exchange and key management – are both executed entirely in hardware, the joint solution has both scalable high-speed performance and minimizes attack surface, especially when compared to a software-based approach. The FPGA hereby utilizes the hardware Root of Trust, best suited for applications like critical communication in defense, space technology, and energy production and distribution.

Read More