Infiniband network cards

What is an InfiniBand adapter?

Network cards or network interface cards used in servers communicate with switches, storage systems, other servers or clients. Since some connections require different performance levels, speeds or protocols, there are also different types of network cards – such as RJ-45 adapters, SFP+ NICs or InfiniBand cards.

InfiniBand network technology offers a special solution when maximum bandwidth, low latency and high reliability are required. This is because the InfiniBand protocol is multi-layered, highly efficient and enables data transfer directly via RDMA (Remote Direct Memory Access). This significantly reduces the load on the CPU. The data is transferred directly between the main memories of two systems, resulting in significantly lower network transmission delays than with Ethernet, for example.

InfiniBand-Adapter connect physically to the host system via the PCIe interface, just like standard NICs. This architecture enables low latency, high bandwidth and good scalability. PCIe InfiniBand adapters are therefore often used in high-performance computers (HPC), AI and big data applications (e.g. storage clustering, AI training), scientific calculations and simulations. Thanks to high transfer rates and low-loss communication, even large amounts of data, computationally intensive workloads and parallel processes can be processed efficiently.

Furthermore, InfiniBand adapters are frequently used in the storage sector, enabling fast and reliable access to SAN systems and databases. The physical connection is usually established via fibre optic cables or twinax cables and is supplemented by special InfiniBand switches and suitable transceivers.

Similar to RJ-45 NICs or HBAs, InfiniBand adapters are also available in different designs. InfiniBand PCIe plug-in cards are available in both low-profile and full-profile versions. InfiniBand mezzanine adapters are used for blades or nodes. InfiniBand plug-in cards are also often referred to as host channel adapters (HCA) – occasionally incorrectly as ‘QSFP NICs’. Although QSFP adapters and InfiniBand NICs are visually similar and both work with high-speed networks, they are different plug-in cards with different protocols and purposes.

Characteristics of InfiniBand NICs include the use of the InfiniBand protocol (technology: RDMA; optional IPoIB for low-latency IP-based communication). Latency is extremely low and throughput is very high – depending on the generation, 40, 100 or even up to 400 Gbit/s. QSFP NICs, on the other hand, primarily describe the connector type (Quad Small Form-factor Pluggable) and are mostly Ethernet-based. Some models also support InfiniBand, but this is not the norm. QSFP NICs also have low latency, with throughput typically around 40 Gbit/s.

InfiniBand vs. Ethernet

InfiniBand is typically used in high-performance applications, while Ethernet is more commonly used for connecting end devices. A key difference lies in bandwidth: While traditional Ethernet connections do not have particularly high bandwidth requirements, an InfiniBand network offers high-speed transmission with significantly higher throughput. RDMA technology significantly reduces CPU load and enables fast data traffic—even with large data volumes or parallel data transfers.

The latency times of Ethernet and InfiniBand also differ significantly. Low latency is crucial, especially in high-performance data centres or real-time applications (e.g. financial transactions). InfiniBand network cards were developed specifically for extremely low latency – classic Ethernet NICs, especially TCP/IP-based connections, have significantly higher latency times. InfiniBand uses a lightweight protocol with Local Identifiers (LIDs), which minimises overhead. In addition, latency-critical data can be prioritised, and a flow control system precisely controls data flows to prevent congestion.

In terms of reliability, both technologies are considered stable – each with different strengths. InfiniBand impresses with its high stability even in complex data centre environments, under high loads and with parallel data streams. Ethernet, on the other hand, offers a proven and cost-effective solution for broad deployment. Its enormous compatibility and modern standards with 10, 25, 40 or even 100 gigabits make Ethernet an established practice in everyday use.

Advantages of InfiniBand

When it comes to performance, latency and scalable connection solutions, no other plug-in card can match InfiniBand cards. They score particularly highly in HPC, cloud computing, AI, big data and finance, as well as in high-performance storage applications, thanks to their high bandwidth, reliability and low latency. They actively contribute to achieving high server efficiency and greater productivity in the application area.

Because data is moved via RDMA (Remote Direct Memory Access), the CPU is greatly relieved and can process other workloads efficiently and ultra-fast. This is particularly relevant for simulations, parallel algorithms and large amounts of data, as found in modern data centres and AI training environments.

Buy InfiniBand NICs for servers at low prices

Just as with RJ-45 network cards, you can also benefit from inexpensive, refurbished hardware when purchasing InfiniBand network cards for servers. The adapters are technically flawless and almost as good as new – but many times cheaper than comparable new products.

When selecting the card you want, you should pay attention to the following points:

• Compatibility with the device
  Check how many and which adapters can be installed in the existing server (e.g. low profile or full profile slot bracket) and which models and interfaces (e.g. PCIe 3.0 x8 or higher) are compatible with the host system.
• Data transfer speed
  There are various InfiniBand network cards for servers – with 100, 200 or even 400 gigabits of data transfer per second, depending on your needs. The more data-intensive the applications are, the higher the transfer rate should be.
• Number of ports
  The number of ports should be based on how many connections are needed at the same time – for example, for parallel data streams or redundant paths.
• Special functions
  Depending on the application, it may be useful to look for special features such as Virtual Protocol Interconnect (VPI), multi-host, socket direct, HDR, EDR or FDR speeds, and support for PCIe 4.0 or PCIe 5.0.

InfiniBand speeds compared

For AI training with multiple GPUs, connections below 100 Gbit/s are hardly useful anymore, as they can quickly lead to bottlenecks. Actual performance also depends on factors such as PCIe version, RDMA support, switch architecture and the software stack used.

In any case, it is wise to rely on high-quality and cost-effective used hardware – this is both easy on the wallet and the environment. If you cannot find the InfiniBand adapter, suitable transceivers or switches you are looking for in our shop, please do not hesitate to contact us.

We are networked with IT dealers worldwide and can procure the desired model safely and inexpensively.