Using RDMA for Media Transport - Design

date
Jun 17, 2024
slug
rdma-design
status
Published
tags
RDMA
Network
summary
type
Post
Background: What Is It Used For
To create a new library based on RDMA for efficient data exchange, especially for media communication mesh (MCM), we must first understand its intended use. MCM facilitates swift data sharing between containers, both within a single node and across multiple nodes. It leverages the Media Transport Library (MTL) for its data path, utilizing the ST2110 stack for large video frame buffers. This requires a DPDK environment where the Network Interface Card (NIC) is bound to a Poll Mode Driver (PMD), preventing kernel use for data paths. A dedicated CPU is also necessary for packet polling, which can be a significant overhead in cloud or edge environments where every computing resource is valuable.
RDMA is an excellent alternative for cross-node data transport as it minimizes CPU intervention and allows the NIC to use the kernel stack for control path operations. Additionally, RDMA's support for GPU memory offers performance benefits for certain media workloads.
How to Align with MTL API
To align with MTL’s API, the new library will use get/put methods for transmission and reception. In MTL TX, users will:
  1. Create a session and then call tx_get_frame to retrieve a frame buffer from the library.
  1. Fill the buffer with content, either by copying or writing directly to the buffer.
  1. After processing, call tx_put_frame to signal that the frame is ready for transmission by the library.
The availability of frames for processing depends on the Frames Per Second (FPS) and the total number of frame buffers configured. Users may need to wait if a frame is not immediately available.
In MTL RX, users will:
  1. Call rx_get_frame to receive a frame buffer when one is available.
  1. Use the frame buffer and then call rx_put_frame to return it to the library for reuse in subsequent receptions.
The New API
The new RDMA-based API extends these capabilities, allowing not just video and audio frames but any data type to be transported. Here are the buffer manipulation APIs:
The buffer is defined as a contiguous block of memory with the following structure:
This structure includes a reserved area for user_meta, allowing users to include custom metadata that can be transported alongside the buffer.
For a visual representation of the API design, please refer to the provided diagram.
 

© Ric Li 2021 - 2024