Heterogeneous Architectures for
Implementation of High-Capacity
Hyper-Converged Storage Devices
Endric Schubert (MLE),
Michaela Blott (Xilinx Research)
SDC, 2016
© Copyright 2016 Xilinx
.
Heterogeneous Architectures for Implementation of
High-capacity Hyper-converged Storage Devices
Who – Xilinx Research and Missing Link Electronics
Why – High-capacity hyper-converged storage needs predictable scalability
in performance, and programmability for flexibility
What – A single-chip heterogeneous active storage solution for Terabit per
second processing
How – By combining modern FPGA design methodologies, including High-
Level Synthesis, with IP cores for full acceleration of rich software
Content
Page 2
© Copyright 2016 Xilinx
.
Page 3
Xilinx Research and Missing Link Electronics
© Copyright 2016 Xilinx
.
Page 4
Xilinx – The All Programmable Company
$2.38B FY15 revenue
>55% market segment share
3,500+ employees worldwide
20,000 customers worldwide
3,500+ patents
60 industry firsts
XILINX - Founded 1984
Headquarters
Research and Development
Sales and Support
Manufacturing
© Copyright 2016 Xilinx
.
Xilinx is Diversified Across Multiple Markets
Mars Rover
3D televisions
MRI scanners
Page 5
ADAS
AR
Machine Learning
© Copyright 2016 Xilinx
.
A field-programmable gate array (FPGA) is an integrated
circuit designed to be configured by the customer or
designer after manufacturing—hence “ field-programmable“
(Source: Wikipedia)
In their simplest form FPGAs contain:
– Configurable Logic Blocks (AND, OR..)
– Configurable interconnect
– I/O Interfaces
Today:
– 3.4M
– 6.3Tbps IO
– + DSPs, ARM, 2.5D…
Page 6
What are FPGAs
Custom-tailored hardware accelerator for your
application, while providing programmability
© Copyright 2016 Xilinx
.
Flexibility to support many types of interfaces
Single-chip solution
– BOM cost reduction
– PCB footprint reduction
– Power/energy reduction
– Performance
Customized memory systems
– Hybrid memory
– Application specific cache hierarchies
Customizable Interfaces & Memory Architectures
Page 7
© Copyright 2016 Xilinx
.
Heterogeneous Multicore with Programmable Logic
Page 8
© Copyright 2016 Xilinx
.
Xilinx Research - Ireland
Page 9
Applications & Architectures
Through application-driven
technology development with
customers, partners, and
engineering & marketing
© Copyright 2016 Xilinx
.
Vision: The convergence of software and off-the-shelf programmable logic
opens-up more economic system realizations with predictable scalability!
Mission: To de-risk the adoption of heterogeneous compute technology by
providing pre-validated IP and expert design services.
Certified Xilinx Alliance Partner since 2011, Preferred Xilinx PetaLinux Design
Service Partner since 2013.
Missing Link Electronics
Xilinx Ecosystem Partner
Page 10
© Copyright 2016 Xilinx
.
Page 11
Missing Link Electronics Products & Services
TCP/IP & UDP/IP Network
Protocol Accelerators for FPGA
(patent pending).
Patented Mixed Signal
systems solutions with
integrated Delta-Sigma
converters in FPGA logic.
SATA Storage Extension for Xilinx
Zynq All-Programmable
Systems-on-Chip.
MLE markets and supports the
Xilinx XPS USB 2.0 EHCI Host
Controller IP core.
A team of FPGA and Linux
engineers to support our
customer’s technology projects
in the USA and Europe.
Tools for architecture analysis
and optimization and RTL and
C/C++ based FPGA design.
© Copyright 2016 Xilinx
.
Page 12
Motivation
© Copyright 2016 Xilinx
.
Software significantly impacts
latency and energy efficiency
in systems with nonvolatile
memory
However, software-defined
flexibility is necessary to fully
utilize novel storage
technologies
Hyper-capacity hyper-
converged storage systems
need more performance, but
within cost and energy
envelopes
Technology Forces in Storage
Page 13
Source: Steven Swanson and Adrian M. Caulfield, UCSD
IEEE Computer, August 2013
© Copyright 2016 Xilinx
.
CPU system performance scalability is limited
The Von Neumann Bottleneck [J. Backus, 1977]
Page 14
New Compute Architectures are needed
© Copyright 2016 Xilinx
.
CPU system performance scalability is limited
Spatial computing offers further scaling opportunity
Spatial vs Temporal Computing
Page 15
New Compute Architectures are needed to
take advantage of this
Sequential Processing
with CPU
Parallel Processing
with Logic Gates
Source: Dr. Andre DeHon, Upenn: “Spatial vs. Temporal Computing”
© Copyright 2016 Xilinx
.
Architectural Choices for Storage Devices
Page 16
Log P E R F O R M A N C E
Log F L E X I B I L I T Y
Log P O W E R D I S S I P A T I O N
10
3
. . . 10
4
10
5
. . . 10
6
Application
Specific Signal
Processors
Digital
Signal
Processors
General
Purpose
Processors
Application
Specific
ICs
Physically
Optimized
ICs
StrongARM110
0.4 MIPS/mW
TMS320C54x
3MIPS/mW
ICORE
20-35 MOPS/mW
Source: T.Noll, RWTH Aachen
Field
Programmable
Devices
© Copyright 2016 Xilinx
.
Page 17
Terabit Processing with Single-Chip Solutions
Source: http://www.xilinx.com/products/silicon-devices/fpga.html
800Gbps – 8.4Tbps
© Copyright 2016 Xilinx
.
FPGA
Hyper-Converge into one single device
Tight coupling of compute + storage + networking
Page 18
Current architecture limits
maximum performance to total
DMA bandwidth through many
hops
Storage and compute directly
integrated into the network
Dataflow processing for higher
bandwidth via FPGA-based inline
processing
RAM
NIC HBA
CPU
RAM
NIC HBA
CPU
Source: Lim et al: Thin servers with smart pipes: designing {SoC} accelerators for memcached; ISCA 2013
© Copyright 2016 Xilinx
.
Semi-automated block-based RTL synthesis design flow
combined with modern C/C++ High-Level-Synthesis for
application specific functionality
Fully integrated software environment are emerging to abstract
data movement (SDSoC and SDAccel)
Design Flow Options
Page 19
© Copyright 2016 Xilinx
.
Page 20
Architectural Concepts
© Copyright 2016 Xilinx
.
Heterogeneous compute device as a single-chip solution
Direct network interface with full accelerator for protocols
Performance scaling with dataflow architectures
Scaling capacity and cost with a Hybrid Storage subsystem
Software-defined services
Key Concepts for an Extensible Architecture for
Storage Devices
Page 21
© Copyright 2016 Xilinx
.
Concept 1: Single-Cip Solution for Storage
Page 22
DDRx
channels
DDRx
channels
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
DDRx
channels
DDRx
channels
DDRx
channels
Data Node
FPGA fabric (PL)
Processing System with quad core 64b processors (A53)
TCP/IP stack
Petalinux
Memory
management
Key Value Store
Abstraction
(memcached)
NVMe
interface
Memory
controller
Hybrid
Memory
System
Network
management
Router
Xilinx IP
SD Services
MLE IP
© Copyright 2016 Xilinx
.
Concept 2: Hardware Accelerated Network Stack
Page 23
DDRx
channels
DDRx
channels
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
DDRx
channels
DDRx
channels
DDRx
channels
Data Node
FPGA fabric (PL)
Processing System with quad core 64b processors (A53)
TCP/IP stack
Petalinux
Memory
management
Key Value Store
Abstraction
(memcached)
NVMe
interface
Memory
controller
Hybrid
Memory
System
Network
management
Router
Fully hardware-
accelerated
TCP/IP stack
© Copyright 2016 Xilinx
.
128bit datapaths for Rx and TX
Scales to 40 GigE (@250 MHz)
No CPU needed – although
embedded CPUs can be
utilized for administrative or
Layer 7 processing
Extensible via HDL or via
C/C++ using High-Level
Synthesis
Technology from:
Concept 2: Hardware Accelerated Network Stack
Page 24
© Copyright 2016 Xilinx
.
Now: 10 Gbps demonstrated with a 64b data path @ 156MHz using 20% of FPGA
Next: 100 Gbps can be achieved by using a 512b @ 200MHz pipeline for example
Concept 3: Dataflow architectures for
performance scaling
Page 25
Streaming architecture:
Flow-controlled series of processing
stages which manipulate and pass
through packets and their associated
state
Source: Blott et al: Achieving 10Gbps line-rate key-value stores with FPGAs; HotCloud 2013
© Copyright 2016 Xilinx
.
SSDs combined with DDRx channels can be used to build high
capacity & high performance object stores
Concepts and early prototype to scale to 40TB & 80Gbps key
value stores
Concept 4: Scaling Capacity
Page 26
Parser
Hash
Lookup
Value
Store
Access
Formatter
DDRx
channels
DDRx
channels
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
DDRx
Hash Table
DDRx
Channels
Hash Table
Value
Store
Value
Store
Hybrid MemorySystem
Source: HotStorage 2015, Scaling out to a Single-Node 80Gbps Memcached Server with 40Terabytes of Memory
© Copyright 2016 Xilinx
.
Advantages:
– Larger objects require larger storage
– Larger granular access to flash suits page-size access granularity of flash
Concerns:
– Large access latency on flash
– Variations in access bandwidth and latency between DRAM and flash
Concept 4:Object distribution on the basis of size
Source: [3] Atikoglu et al: Workload analysis of a large-scale key-value store; SIGMETRICS 2012
Stored in DRAM Stored in Flash
128 256 512 768 1K 4K 8K 32K 1M
0.55 0.075 0.275 0 0 0 0 0 0.1
0 0 0 0.1 0.85 0.05 0 0 0
0 0 0.2 0.1 0.4 0.29 0.008 0.001 0.001
0 0 0 0 0 0.9 0.05 0.03 0.02
Value Size (B)
Facebook
Twitter
Wiki
Flickr
Page 27
© Copyright 2016 Xilinx
.
Concept 4: Handling High Latency Accesses
without Sacrificing Throughput
Read SSDRead SSD
Read SSD
100usec
• Dataflow architectures: no limit to number of outstanding requests
• Flash can be serviced at maximum speed
Read SSDRead SSD
Read SSD
time
Read SSD
Read SSD
Read SSDRead SSD
Read SSD
Request
Buffer
Read SSD
Read SSD
Read SSDRead SSD
Read SSD
ResponseResponseResponse
ResponseResponseResponse
ResponseResponse
Response
ResponseResponseResponse
ResponseResponse
Response
Cmd:
Rsp:
Read SSDRead SSD
Read SSD
Read SSD
Read SSD
Read SSDRead SSD
Read SSD
Read SSD
Read SSD
Read SSDRead SSD
Read SSD
Page 28
© Copyright 2016 Xilinx
.
Concept 4: Custom memory controllers
with out of order processing
© Copyright 2015 Xilinx
Page 29
© Copyright 2016 Xilinx
.
Concept 5: SD Services
Page 30
SD services:
Object recognition
SD services:
Compression
Encryption
Spatial computing of additional services at no performance cost until
resource limitations are reached
© Copyright 2016 Xilinx
.
Page 31
Results
© Copyright 2016 Xilinx
.
Results: Networked Object Storage Board
with Xilinx Zynq Ultrascale+ MPSoC
50Gbps key value store with 2TB, on a 35W board
Page 32
Dual DDR4 SODIMM
16GB x72 ECC DR
273 Gb/s @ 2133 Mb/s
Dual M.2
2x SSD 512 GB
Dual SFP+
2x 10/25 Gbps
16nm MPSoC
Quad A53 CPU
Embedded FPGA
© Copyright 2016 Xilinx
.
Results: Current Prototype Architecture
Page 33
DDRx
channels
DDRx
channels
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
M.2 NVMe
drives
DDRx
channels
DDRx
channels
DDRx
channels
Data Node
FPGA fabric (PL)
Processing System with quad core 64b processors (A53)
TCP/IP stack
Petalinux
Memory
management
Key Value Store
Abstraction
(memcached)
NVMe
interface
Memory
controller
Network
management
Router
Xilinx IP
SD Services
MLE IP
Netperf
NVMe over IP
© Copyright 2016 Xilinx
.
Experiments
Page 34
PC connected to ZU9SN:
Netperf, NVMe over IP
Spirent network tester connected to ZU9SN:
Memcached @ 10Gbps
35 Watt BLP
© Copyright 2016 Xilinx
.
Software (CPU + NIC)
Results: Latency Analysis of Full Accelerator
Page 35
FPGA-based Full Accelerator
• Lower and predictable latency with very little jitter
© Copyright 2016 Xilinx
.
Results: Throughput Analysis of Full Accelerator
Page 36
© Copyright 2016 Xilinx
.
Transport NVMe (ie. PCIe) Transaction-Layer Packets (TLP) via
standard TCP/IP
Results: Feasibility of “NVMe-over-IP”
Page 37
© Copyright 2016 Xilinx
.
Results: Feasibility of non-legacy NVMe-over-IP
Page 38
© Copyright 2016 Xilinx
.
Results: Feasibility of non-legacy NVMe-over-IP
Page 39
© Copyright 2016 Xilinx
.
Results: Dataflow Architecture for
Acceleration of Key-Value-Stores (KVS)
Page 40
Line-rate maximum
response rate
Achieved by FPGA
FPGA is network bound, supports currently 52MRPS
© Copyright 2016 Xilinx
.
Results: Comparison with best published results
X86 Platforms GB KRPS Watt
Dual x86 (Mica) [12] – research 64 76,900 478
Dual x86 (FlashStore) [6] - research 80 57 84
Source:
HotStorage 2015, Scaling out to a Single-Node 80Gbps Memcached Server with 40Terabytes of Memory
[6] Debnath et al: Flashstore: High throughput persistent key-value store; PVLDB 2010
[12] Lim et al: Mica: A holistic approach to fast in memory key-value storage; NSDI 2014
Page 41
FPGA Platforms GB KRPS Watt
Prototype (10Gbps) 512 13’000 35
Estimation (50Gbps, 2 M.2) 8’192 65’000 TBD
Estimation (100Gbps) 40’000 130’000 TBD
X86 are limited extracting performance out of flash
• Support either high performance or high capacity
Dataflow architecture on MPSoC can support high
performance and high capacity at lower power
© Copyright 2016 Xilinx
.
Page 42
Conclusion & Outlook
© Copyright 2016 Xilinx
.
Trend towards unconventional architectures
– A diversification of increasingly heterogeneous devices and systems
– Convergence of networking, compute and storage within single
nodes
Key concepts for implementation of hyper-converged storage
nodes
– Heterogeneous compute device as a single-chip solution
– Direct network interface with a full hardware TCP/IP stack
– Data flow architecture to accelerate all data processing
– NVMe for multi-terabyte storage capacity
– Hybrid memory system (DRAM & flash) for high capacity and high
performance
Results:
– First prototype board build for 50Gbps with 2TB key value store
– Proof of concept demonstrates:
10Gbps TCP/IP stack, 13MRPS, 10Gbps key value store, 35Watt
Conclusion
Page 43
© Copyright 2016 Xilinx
.
Exploration of first software defined services
Joint evaluation with potential customers & universities, MLE and Xilinx
to measure system-level benefits
Outlook
Page 44
© Copyright 2016 Xilinx
.
Page 45
Backup
© Copyright 2016 Xilinx
.
Applications require
– Increasing compute (machine learning, data
analytics)
– Increasing storage capacity (photos, videos)
– Lower power (OPEX = 2x CAPEX)
Architectural innovation is required
– to provide further performance and storage
scaling while reducing power
Integration of compute, memory and
network within individual nodes
– Hyper-converged storage
Computing: Increasingly Heterogeneous and Integrated
Page 46
© Copyright 2016 Xilinx
.
Maximize Performance via Application Specific Processing
– Function mapping to optimal processing element
Reduce Power via System Integration
– Function optimization to processing element
– Reduced interconnect power
Reduce BOM Cost via System Integration
– Fewer components
– Reduced board space
Increased Safety via Functional Safety Components
– Critical component redundancy
– Increased integration
IP, Peripheral & Processing Customization via Programmable
Logic
Value of Heterogeneous Multicore Processing
© Copyright 2016 Xilinx
.
Concept: SD Services
Spatial computing of additional services at no performance cost
until resource limitations are reached
Page 48
Network
Stack
Compression
Encryption
Key Value
Stores
Network
Stack
FPGA
Hybrid Storage Subsystem
Object
Recognition
© Copyright 2016 Xilinx
.
Data Node
FPGA fabric (PL)
PS
DRAM channel
M.2 channel
10G if TCP/IP stack
(limited session
support)
KVS
NVMe if
M.2 channel
Memory management
Hybrid
memory
system
MIG
DRAM channel
NIC/Bypass
(routes traffic on basis
of port)
Petalinux
SD Services
Xilinx IP
MLE IP
Processing System with 64bit Quad-Core ARM A53
© Copyright 2016 Xilinx
.
Design automation runs scheduling and resource binding to generate
RTL code comprising data paths plus state machines for control flow
Working Principles of High-Level Synthesis
Page 50
© Copyright 2016 Xilinx
.
Automated performance
optimizations via parallelization
at dataflow level
Benefits of HLS-Based C/C++ FPGA Design
Page 51
Automatic interface synthesis
and driver code generation for
HW/SW connectivity