Presentation at SNIA Storage Developers Conference, Fremont, CA, Sept. 18-21, 2023
Tail latencies in networking tend to worry us all, whether we implement distributed storage and compute or whether we connect systems-of-systems in automotive or factory automation, for example. Same goes for the computational burden of processing networking protocols.
One of the foundations of reliable networking is TCP, the Transmission Control Protocol which was introduced half a century ago. Today, TCP is ubiquitous: In the data center, in mobile communication, the Internet and in (embedded) systems-of-systems. However, TCP has some significant drawbacks including unpredictable tail latency and a significant computational burden.
This presentation aims to guide engineers in their systems design by sharing quantitative analysis results and describing alternatives. For latency analysis we present benchmarks results from network simulation which show the significant tail latencies for TCP. We also share benchmark results from running TCP implementations of different Linux kernels – with sometimes surprising outcomes.
Benchmarks for the computational burden extend our analysis of implementations of TCP processing in various Linux kernels. We discuss a metric for Efficiency as in Throughput per CPU load and compare results.
Finally, we present Homa from John Ousterhout’s team at Stanford University. Homa is an implementation of a so-called Quad-RP, i.e. Rapid, Reliable Request-Response Protocol: Instead of streams it is message based and connectionless, which caters better to the needs of modern distributed, microserviced, virtualized architectures. It puts the recipients in control which enables proactive approaches to congestion control and thereby achieves better network infrastructure utilization. While not TCP API compatible, overall, Homa provides significant advantages over TCP when it comes to tail latency and infrastructure efficiency in real-life networks.
We close with an outlook of the potential realizations in ASIC and FPGA to enable energy – efficient, deterministic networking as well as potential use in networked storage.