InfiniBand™ networking for Compute virtual machines with GPUs

You can group your virtual machines with GPUs into a GPU cluster. The cluster accelerates high-performance computing (HPC) tasks such as training and inference. These tasks require a lot of processing power that a single VM cannot provide. The GPU clusters are built with InfiniBand™ secure high-speed networking. Each GPU in a VM is connected through a network interface card (NIC) that provides 400 Gbps. As a compute VM for GPU clusters consists of 8 GPUs, the total bandwidth for a node is 3.2 Tbps. Nebius AI Cloud uses GPUDirect RDMA, an NVIDIA® technology of remote direct memory access (RDMA) that allows data to flow directly between each GPU and its NIC, avoiding CPU, thus boosting the data exchange speed.

InfiniBand™ fabrics

Each GPU cluster is created in one of the physical InfiniBand™ fabrics. This is where GPUs interconnected over InfiniBand™ are located. Each fabric has limited GPU capacity. When creating a GPU cluster, select an InfiniBand™ fabric for it. Take into account the type of GPUs you are going to use. For example, if you select fabric-7, you can only add NVIDIA® H200 NVLink with Intel Sapphire Rapids to this cluster. Available fabrics and corresponding regions (private regions are marked with *):

Fabric	GPU platform	Region
`fabric-2`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-3`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-4`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-5`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-west1`
`fabric-6`	NVIDIA® H100 NVLink with Intel Sapphire Rapids (`gpu-h100-sxm`)	`eu-north1`
`fabric-7`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-north1`
`eu-north2-a`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`eu-north2`*
`eu-west2-a`	NVIDIA® B300 NVLink with Intel Granite Rapids (`gpu-b300-sxm`)	`eu-west2`
`me-west1-a`	NVIDIA® B200 NVLink with Intel Emerald Rapids (`gpu-b200-sxm-a`)	`me-west1`
`uk-south1-a`	NVIDIA® B300 NVLink with Intel Granite Rapids (`gpu-b300-sxm`)	`uk-south1`
`us-central1-a`	NVIDIA® H200 NVLink with Intel Sapphire Rapids (`gpu-h200-sxm`)	`us-central1`
`us-central1-b`	NVIDIA® B200 NVLink with Intel Emerald Rapids (`gpu-b200-sxm`)	`us-central1`
`us-north1-a`	NVIDIA® B300 NVLink with Intel Granite Rapids (`gpu-b300-sxm`)	`us-north1`

In most cases, you do not need to change the preselected fabric. We recommend that you create a GPU cluster in another fabric only if it is better suited for a different platform or if you experience capacity issues with an existing GPU cluster.

Isolation and security of InfiniBand™ traffic

Nebius AI Cloud isolates InfiniBand™ traffic between GPU clusters by using InfiniBand™ partition keys (P-Keys). Each GPU cluster is assigned a unique P-Key to create isolation inside shared physical InfiniBand™ fabrics. This way, nodes in different GPU clusters cannot communicate over InfiniBand™ even if they use the same fabric infrastructure. This creates isolation between tenants without requiring a dedicated physical fabric for each cluster.

Prerequisites

If you use the web console, you don’t need to complete any prerequisites.

CLI
Go SDK
Python SDK
JavaScript SDK

Install and configure the Nebius AI Cloud CLI.

How to enable InfiniBand™ for VMs with GPUs

Web console
CLI
Go SDK
Python SDK
JavaScript SDK

Create a GPU cluster:
1. In the sidebar, go to Compute → GPU clusters.
2. Click Create GPU cluster.
3. On the page that opens, specify the cluster name. It should contain from 3 to 63 characters: lowercase letters, numbers and hyphens.
4. Select the InfiniBand™ fabric.
5. Click Create GPU cluster.
Add VMs to the cluster. You can assign a GPU cluster only when creating a VM:
All virtual machines added to a GPU cluster must be in the same project as the GPU cluster.
1. In the sidebar, go to Compute → Virtual machines.
2. Click Create resource → Virtual machine.
3. On the Compute step of the VM creation wizard, select a platform and preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.
4. In the Settings section, select an existing GPU cluster in the GPU cluster field, or create a new one.
5. On the Storage step, select the boot disk for NVIDIA® GPUs. For details, see Boot disk images for Compute virtual machines.

You can also create a GPU cluster while creating the first VM in it:

On the Compute step of the VM creation wizard:
1. Select a platform and a preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.
2. In the Settings section, create or select a GPU cluster in the GPU cluster field.
On the Storage step, select the boot disk for NVIDIA® GPUs. For details, see Boot disk images for Compute virtual machines.

Check that your project ID is saved in the Nebius AI Cloud CLI profile configuration:
cat ~/.nebius/config.yaml
If you have not set your project ID as parent-id, or you want to create resources in a different project, get the project ID and update your CLI profile:
nebius profile update --parent-id <project_ID>
Depending on your project’s region, select an InfiniBand™ fabric for VM interconnection and save it to an environment variable:
export INFINIBAND_FABRIC=<fabric>

Create a GPU cluster and save its ID:

export GPU_CLUSTER_ID=$(nebius compute gpu-cluster create \
  --name <gpu_cluster_name> \
  --infiniband-fabric $INFINIBAND_FABRIC \
  --format jsonpath='{.metadata.id}')

Where:

Name: A cluster name that you can use to quickly find the cluster.

Create a boot disk optimized for VMs with NVIDIA® GPUs:

export BOOT_DISK_ID=$(nebius compute disk create \
  --name <boot_disk_name> \
  --size-gibibytes 200 \
  --type network_ssd \
  --source-image-family-image-family ubuntu24.04-cuda13.0 \
  --block-size-bytes 4096 \
  --format jsonpath='{.metadata.id}')

For compatible boot disk images (--source-image-family-image-family), see Boot disk images.

Create a virtual machine with GPUs and specify the GPU cluster ID in its parameters.
All virtual machines added to a GPU cluster must be in the same project as the GPU cluster.
For example:
nebius compute instance create \ --resources-platform gpu-h100-sxm \ --resources-preset 8gpu-128vcpu-1600gb \ --gpu-cluster-id $GPU_CLUSTER_ID \ --boot-disk-existing-disk-id $BOOT_DISK_ID \ ...
Specify a VM platform with GPUs in --resources-platform, and a preset compatible with GPU clusters in --resources-preset. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.

Depending on your project’s region, select an InfiniBand™ fabric for VM interconnection and set it in the code:
infinibandFabric := "<fabric>"

Create a GPU cluster and save its ID:

gpuClusterOperation, err := sdk.Services().Compute().V1().
    GpuCluster().Create(
        ctx,
        &compute.CreateGpuClusterRequest{
            Metadata: &common.ResourceMetadata{
                Name: "<gpu_cluster_name>",
            },
            Spec: &compute.GpuClusterSpec{
                InfinibandFabric: infinibandFabric,
            },
        },
    )
if err != nil {
    return err
}
if _, err = gpuClusterOperation.Wait(ctx); err != nil {
    return err
}
gpuClusterID := gpuClusterOperation.ResourceID()

Where:

Metadata.Name: A cluster name that you can use to quickly find the cluster.

Create a boot disk optimized for VMs with NVIDIA® GPUs:

diskOperation, err := sdk.Services().Compute().V1().
    Disk().Create(
        ctx,
        &compute.CreateDiskRequest{
            Metadata: &common.ResourceMetadata{
                Name: "<boot_disk_name>",
            },
            Spec: &compute.DiskSpec{
                Size: &compute.DiskSpec_SizeGibibytes{
                    SizeGibibytes: 200,
                },
                BlockSizeBytes: 4096,
                Type:           compute.DiskSpec_NETWORK_SSD,
                Source: &compute.DiskSpec_SourceImageFamily{
                    SourceImageFamily: &compute.SourceImageFamily{
                        ImageFamily: "ubuntu24.04-cuda13.0",
                    },
                },
            },
        },
    )
if err != nil {
    return err
}
if _, err = diskOperation.Wait(ctx); err != nil {
    return err
}
bootDiskID := diskOperation.ResourceID()

For compatible boot disk images, see Boot disk images.

Create a virtual machine with GPUs and specify the GPU cluster ID in its parameters.

All virtual machines added to a GPU cluster must be in the same project as the GPU cluster.

attachMode := compute.AttachedDiskSpec_READ_WRITE
instanceOperation, err := sdk.Services().Compute().V1().
    Instance().Create(
        ctx,
        &compute.CreateInstanceRequest{
            Metadata: &common.ResourceMetadata{
                Name: "vm",
            },
            Spec: &compute.InstanceSpec{
                Stopped: true,
                Resources: &compute.ResourcesSpec{
                    Platform: "gpu-h100-sxm",
                    Size: &compute.ResourcesSpec_Preset{
                        Preset: "8gpu-128vcpu-1600gb",
                    },
                },
                GpuCluster: &compute.InstanceGpuClusterSpec{
                    Id: gpuClusterID,
                },
                BootDisk: &compute.AttachedDiskSpec{
                    AttachMode: attachMode,
                    Type: &compute.AttachedDiskSpec_ExistingDisk{
                        ExistingDisk: &compute.ExistingDisk{
                            Id: bootDiskID,
                        },
                    },
                },
                NetworkInterfaces: []*compute.NetworkInterfaceSpec{
                    {
                        Name:      "ni",
                        SubnetId:  subnetID,
                        IpAddress: &compute.IPAddress{},
                        PublicIpAddress: &compute.PublicIPAddress{},
                    },
                },
            },
        },
    )
if err != nil {
    return err
}
if _, err = instanceOperation.Wait(ctx); err != nil {
    return err
}

Specify a VM platform with GPUs and a preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.

Depending on your project’s region, select an InfiniBand™ fabric for VM interconnection and set it in the code:
infiniband_fabric = "<fabric>"

Create a GPU cluster and save its ID:

gpu_cluster_service = GpuClusterServiceClient(sdk)
create_cluster_operation = await gpu_cluster_service.create(
    CreateGpuClusterRequest(
        metadata=ResourceMetadata(
            name="<gpu_cluster_name>",
        ),
        spec=GpuClusterSpec(
            infiniband_fabric=infiniband_fabric,
        ),
    ),
)
await create_cluster_operation.wait()
gpu_cluster_id = create_cluster_operation.resource_id

Where:

metadata.name: A cluster name that you can use to quickly find the cluster.

Create a boot disk optimized for VMs with NVIDIA® GPUs:

disk_service = DiskServiceClient(sdk)
create_disk_operation = await disk_service.create(
    CreateDiskRequest(
        metadata=ResourceMetadata(
            name="<boot_disk_name>",
        ),
        spec=DiskSpec(
            block_size_bytes=4096,
            type=DiskSpec.DiskType.NETWORK_SSD,
            source_image_family=SourceImageFamily(
                image_family="ubuntu24.04-cuda13.0",
            ),
            size_gibibytes=200,
        ),
    ),
)
await create_disk_operation.wait()
boot_disk_id = create_disk_operation.resource_id

For compatible boot disk images, see Boot disk images.

Create a virtual machine with GPUs and specify the GPU cluster ID in its parameters.

All virtual machines added to a GPU cluster must be in the same project as the GPU cluster.

instance_service = InstanceServiceClient(sdk)
create_instance_operation = await instance_service.create(
    CreateInstanceRequest(
        metadata=ResourceMetadata(
            name="vm",
        ),
        spec=InstanceSpec(
            stopped=True,
            resources=ResourcesSpec(
                platform="gpu-h100-sxm",
                preset="8gpu-128vcpu-1600gb",
            ),
            gpu_cluster=InstanceGpuClusterSpec(
                id=gpu_cluster_id,
            ),
            boot_disk=AttachedDiskSpec(
                attach_mode=(
                    AttachedDiskSpec.AttachMode.READ_WRITE
                ),
                existing_disk=ExistingDisk(id=boot_disk_id),
            ),
            network_interfaces=[
                NetworkInterfaceSpec(
                    name="ni",
                    subnet_id=subnet_id,
                    ip_address=IPAddress(),
                    public_ip_address=PublicIPAddress(),
                ),
            ],
        ),
    ),
)
await create_instance_operation.wait()

Specify a VM platform with GPUs and a preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.

Depending on your project’s region, select an InfiniBand™ fabric for VM interconnection and set it in the code:
const infinibandFabric = "<fabric>";

Create a GPU cluster and save its ID:

const gpuClusterService = new GpuClusterService(sdk);
const createGpuClusterOperation = await gpuClusterService.create(
  CreateGpuClusterRequest.create({
    metadata: ResourceMetadata.create({
      name: "<gpu_cluster_name>",
    }),
    spec: GpuClusterSpec.create({
      infinibandFabric,
    }),
  }),
).result;
await createGpuClusterOperation.wait();
const gpuClusterId = createGpuClusterOperation.resourceId();

Where:

metadata.name: A cluster name that you can use to quickly find the cluster.

Create a boot disk optimized for VMs with NVIDIA® GPUs:

const diskService = new DiskService(sdk);
const createDiskOperation = await diskService.create(
  CreateDiskRequest.create({
    metadata: ResourceMetadata.create({
      name: "<boot_disk_name>",
    }),
    spec: DiskSpec.create({
      blockSizeBytes: 4096,
      type: DiskSpec_DiskType.NETWORK_SSD,
      source: {
        $case: "sourceImageFamily",
        sourceImageFamily: SourceImageFamily.create({
          imageFamily: "ubuntu24.04-cuda13.0",
        }),
      },
      size: {
        $case: "sizeGibibytes",
        sizeGibibytes: 200,
      },
    }),
  }),
).result;
await createDiskOperation.wait();
const bootDiskId = createDiskOperation.resourceId();

For compatible boot disk images, see Boot disk images.

Create a virtual machine with GPUs and specify the GPU cluster ID in its parameters.

All virtual machines added to a GPU cluster must be in the same project as the GPU cluster.

const instanceService = new InstanceService(sdk);
const createInstanceOperation = await instanceService.create(
  CreateInstanceRequest.create({
    metadata: ResourceMetadata.create({
      name: "vm",
    }),
    spec: InstanceSpec.create({
      stopped: true,
      resources: ResourcesSpec.create({
        platform: "gpu-h100-sxm",
        size: {
          $case: "preset",
          preset: "8gpu-128vcpu-1600gb",
        },
      }),
      gpuCluster: InstanceGpuClusterSpec.create({
        id: gpuClusterId,
      }),
      bootDisk: AttachedDiskSpec.create({
        attachMode: AttachedDiskSpec_AttachMode.READ_WRITE,
        type: {
          $case: "existingDisk",
          existingDisk: ExistingDisk.create({
            id: bootDiskId,
          }),
        },
      }),
      networkInterfaces: [
        NetworkInterfaceSpec.create({
          name: "ni",
          subnetId,
          ipAddress: IPAddress.create({}),
          publicIpAddress: PublicIPAddress.create({}),
        }),
      ],
    }),
  }),
).result;
await createInstanceOperation.wait();

Specify a VM platform with GPUs and a preset compatible with GPU clusters. For more information, see Types of virtual machines and GPUs in Nebius AI Cloud.

How to test the connection with the NCCL tests

To test InfiniBand™ performance in a Compute cluster, you can run the NVIDIA® Collective Communications Library (NCCL) test in it. For instructions, see our tutorial on running distributed jobs with MPIrun: it uses the NCCL test as an example.

How to delete a GPU cluster

Before deleting a GPU cluster, make sure all virtual machines in the cluster are deleted or moved to another cluster.

Web console
CLI
Go SDK
Python SDK
JavaScript SDK

In the sidebar, go to Compute → GPU clusters.
In the row of the GPU cluster you want to delete, click → Delete.
In the window that opens, confirm the deletion.

Get the ID of the GPU cluster you want to delete:
nebius compute gpu-cluster list

Delete the GPU cluster:

nebius compute gpu-cluster delete <GPU_cluster_ID>

Get the ID of the GPU cluster you want to delete:

gpuClusters, err := sdk.Services().Compute().V1().
    GpuCluster().List(
        ctx,
        &compute.ListGpuClustersRequest{
            ParentId: nbProject,
        },
    )
if err != nil {
    return err
}
fmt.Println(gpuClusters)

Delete the GPU cluster:

delClusterOp, err := sdk.Services().Compute().V1().
    GpuCluster().Delete(
        ctx,
        &compute.DeleteGpuClusterRequest{
            Id: "<GPU_cluster_ID>",
        },
    )
if err != nil {
    return err
}
if _, err = delClusterOp.Wait(ctx); err != nil {
    return err
}

Get the ID of the GPU cluster you want to delete:

gpu_cluster_service = GpuClusterServiceClient(sdk)
gpu_clusters = await gpu_cluster_service.list(
    ListGpuClustersRequest(parent_id=nb_project),
)
print(gpu_clusters)

Delete the GPU cluster:

gpu_cluster_service = GpuClusterServiceClient(sdk)
delete_cluster_operation = await gpu_cluster_service.delete(
    DeleteGpuClusterRequest(id="<GPU_cluster_ID>"),
)
await delete_cluster_operation.wait()

Get the ID of the GPU cluster you want to delete:

const listGpuClusterService = new GpuClusterService(sdk);
const gpuClusters = await listGpuClusterService.list(
  ListGpuClustersRequest.create({
    parentId: nbProject,
  }),
);
console.log(gpuClusters);

Delete the GPU cluster:

const deleteGpuClusterService = new GpuClusterService(sdk);
const deleteClusterOperation = await deleteGpuClusterService.delete(
  DeleteGpuClusterRequest.create({
    id: "<GPU_cluster_ID>",
  }),
).result;
await deleteClusterOperation.wait();

Virtual machines

Slurm and Soperator in Nebius AI Cloud

Managed Service for Kubernetes®

InfiniBand™ networking for Compute virtual machines with GPUs

InfiniBand™ fabrics

Isolation and security of InfiniBand™ traffic

Prerequisites

How to enable InfiniBand™ for VMs with GPUs

How to test the connection with the NCCL tests

How to delete a GPU cluster

See also

​InfiniBand™ fabrics

​Isolation and security of InfiniBand™ traffic

​Prerequisites

​How to enable InfiniBand™ for VMs with GPUs

​How to test the connection with the NCCL tests

​How to delete a GPU cluster

​See also

InfiniBand™ fabrics

Isolation and security of InfiniBand™ traffic

Prerequisites

How to enable InfiniBand™ for VMs with GPUs

How to test the connection with the NCCL tests

How to delete a GPU cluster

See also