InfiniBand™ networking for Compute virtual machines with GPUs

You can group your virtual machines with GPUs into a GPU cluster. The cluster accelerates high-performance computing (HPC) tasks such as training and inference. These tasks require a lot of processing power that a single VM cannot provide. The GPU clusters are built with InfiniBand secure high-speed networking. Each GPU in a VM is connected through a network interface card (NIC) that provides 400 Gbps. As a compute VM for GPU clusters consists of 8 GPUs, the total bandwidth for a node is 3.2 Tbps. Nebius AI Cloud uses GPUDirect RDMA, an NVIDIA technology of remote direct memory access (RDMA) that allows data to flow directly between each GPU and its NIC, avoiding CPU, thus boosting the data exchange speed.

InfiniBand fabrics

Each GPU cluster is created in one of physical InfiniBand fabrics. This is where GPUs interconnected over InfiniBand are located. Each fabric has limited GPU capacity. When creating a GPU cluster, select an InfiniBand fabric for it. Take into account the type of GPUs you are going to use. For example, if you select fabric-7, you can only add NVIDIA® H200 NVLink with Intel Sapphire Rapids GPUs to this cluster. Available fabrics and corresponding regions (private regions are marked with *):

Fabric	GPU platform	Region
`fabric-2`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-3`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-4`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-5`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-west1`
`fabric-6`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-7`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-north1`
`eu-north2-a`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-north2`*
`eu-west2-a`	NVIDIA® B300 NVLink with Intel Granite Rapids (`gpu-b300-sxm`)	`eu-west2`*
`me-west1-a`	NVIDIA® B200 NVLink with Intel Emerald Rapids (`gpu-b200-sxm-a`)	`me-west1`
`uk-south1-a`	NVIDIA® B300 NVLink with Intel Granite Rapids (`gpu-b300-sxm`)	`uk-south1`
`us-central1-a`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`us-central1`
`us-central1-b`	NVIDIA® B200 NVLink with Intel Emerald Rapids (`gpu-b200-sxm`)	`us-central1`

In most cases, you do not need to change the preselected fabric. We recommend that you create a GPU cluster in another fabric only if it is better suited for a different platform or if you experience capacity issues with an existing GPU cluster.

Isolation and security of InfiniBand traffic

Nebius AI Cloud isolates InfiniBand traffic between GPU clusters by using InfiniBand partition keys (P-Keys). Each GPU cluster is assigned a unique P-Key to create isolation inside shared physical InfiniBand fabrics. This way, nodes in different GPU clusters cannot communicate over InfiniBand even if they use the same fabric infrastructure. This creates isolation between tenants without requiring a dedicated physical fabric for each cluster.

How to enable InfiniBand for VMs with GPUs

Web console
CLI

Create a GPU cluster:
1. In the sidebar, go to Compute → GPU clusters.
2. Click Create GPU cluster.
3. On the page that opens, specify the cluster name. It should contain from 3 to 63 characters: lowercase letters, numbers and hyphens.
4. Select the InfiniBand fabric.
5. Click Create GPU cluster.
Add VMs to the cluster. You can assign a GPU cluster only when creating a VM:
All virtual machines added to the GPU cluster, including Managed Service for Kubernetes® nodes, must be in the same project.
1. In the sidebar, go to Compute → Virtual machines.
2. Click Create resource → Virtual machine.
3. On the Compute step of the VM creation wizard, select a platform and preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.
4. In the Settings section, select an existing GPU cluster in the GPU cluster field, or create a new one.
5. On the Storage step, select the boot disk for NVIDIA GPUs. For details, see Boot disk images for Compute virtual machines.

You can also create a GPU cluster while creating the first VM in it:

On the Compute step of the VM creation wizard:
1. Select a platform and a preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.
2. In the Settings section, create or select a GPU cluster in the GPU cluster field.
On the Storage step, select the boot disk for NVIDIA GPUs. For details, see Boot disk images for Compute virtual machines.

Check that your project ID is saved in the Nebius AI Cloud CLI profile configuration:
cat ~/.nebius/config.yaml
If you have not set your project ID as parent-id, or you want to create resources in a different project, get the project ID and update your CLI profile:
nebius profile update --parent-id <project_ID>
Depending on your project’s region, select an InfiniBand fabric for VM interconnection and save it to an environment variable:
export INFINIBAND_FABRIC=<fabric>

Create a GPU cluster and save its ID:

export GPU_CLUSTER_ID=$(nebius compute gpu-cluster create \
  --name <gpu_cluster_name> \
  --infiniband-fabric $INFINIBAND_FABRIC \
  --format json \
  | jq -r ".metadata.id")

Where:

name: A cluster name that you can use to quickly find the cluster.

Create a boot disk optimized for VMs with NVIDIA GPUs:

export BOOT_DISK_ID=$(nebius compute disk create \
  --name <boot_disk_name> \
  --size-gibibytes 200 \
  --type network_ssd \
  --source-image-family-image-family ubuntu24.04-cuda13.0 \
  --block-size-bytes 4096 \
  --format json \
  | jq -r ".metadata.id")

For compatible boot disk images (--source-image-family-image-family), see Boot disk images.

Create a virtual machine with GPUs and specify the GPU cluster ID in its parameters.
All virtual machines added to the GPU cluster, including Managed Service for Kubernetes nodes, must be in the same project.
For example:
nebius compute instance create \ --resources-platform gpu-h100-sxm \ --resources-preset 8gpu-128vcpu-1600gb \ --gpu-cluster-id $GPU_CLUSTER_ID \ --boot-disk-existing-disk-id $BOOT_DISK_ID \ ...
Specify a VM platform with GPUs in --resources-platform, and a preset compatible with GPU clusters in --resources-preset. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.

How to test the connection with the NCCL tests

To test InfiniBand performance in a Compute cluster, you can run the NVIDIA NCCL test in it. For instructions, see our tutorial on running distributed jobs with MPIrun: it uses the NCCL test as an example.

How to delete a GPU cluster

Before deleting a GPU cluster, make sure all virtual machines in the cluster are deleted or moved to another cluster.

Web console
CLI

In the sidebar, go to Compute → GPU clusters.
In the row of the GPU cluster you want to delete, click → Delete.
In the window that opens, confirm the deletion.

Get the ID of the GPU cluster you want to delete:
nebius compute gpu-cluster list

Delete the GPU cluster:

nebius compute gpu-cluster delete <GPU_cluster_ID>

Virtual machines

Slurm and Soperator in Nebius AI Cloud

Managed Service for Kubernetes®

InfiniBand™ networking for Compute virtual machines with GPUs

InfiniBand fabrics

Isolation and security of InfiniBand traffic

How to enable InfiniBand for VMs with GPUs

How to test the connection with the NCCL tests

How to delete a GPU cluster

See also

​InfiniBand fabrics

​Isolation and security of InfiniBand traffic

​How to enable InfiniBand for VMs with GPUs

​How to test the connection with the NCCL tests

​How to delete a GPU cluster

​See also

InfiniBand fabrics

Isolation and security of InfiniBand traffic

How to enable InfiniBand for VMs with GPUs

How to test the connection with the NCCL tests

How to delete a GPU cluster

See also