Remote Direct Memory Access (RDMA) is a concept whereby two or more computers communicate via Direct Memory Access directly from the main memory of one system to the main memory of another. As there is no CPU, cache, or context switching overhead needed to perform the transfer, and transfers can continue in parallel with other system operations, this is particularly useful in applications where high throughput, low latency networking is needed such as in massively parallel Linux clusters. The most common RDMA implementation is over InfiniBand. Although RDMA over InfiniBand is technologically superior to most alternatives, it faces an uncertain commercial future.
High latencies can negatively impact the bandwidth that can be achieved. When latencies are high, the protocol overhead can overwhelm the work needed to deliver the data. High latencies can create a bottleneck which prevents full utilization of the network, thus decreasing network bandwidth. Latency reduces overall performance by limiting how fast an application can get the data it needs, and limits the overall size and scalability of a cluster by limiting the number of messages that can be effectively put on the wire.
Once a connection has been established, RDMA enables the movement of data from the memory of one server directly into the memory of another server without involving the operating system of either node. RDMA supports “zero-copy” networking by enabling the network adapter to transfer data directly to or from application memory, eliminating the need to copy data between application memory and the data buffers in the operating system. When an application performs an RDMA Read or Write request, the application data is delivered directly to the network, hence latency is reduced and applications can transfer messages faster.
RDMA’s widespread acceptance has been limited by the historical need to install a new and different networking infrastructure. However, new standards are enabling RDMA to be implemented using Ethernet as the physical layer and TCP/IP as the transport, thus combining all the performance and latency advantages of RDMA with the low-cost and standards benefit of Ethernet and TCP/IP. The RDMA Consortium and the DAT Collaborative have played key roles in the development of RDMA protocols and APIs for consideration by standards groups such as the Internet Engineering Task Force and the Interconnect Software Consortium. Major software vendors such as Oracle ® Corporation are now supporting these new APIs in their latest products, and new network adapters that implement RDMA over Ethernet are expected this year.