eu-north1 region:
- Virtual machine (VM) with eight GPUs and a shared filesystem for training
- Virtual machine with one GPU for inference
Prerequisites
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
Generate an SSH key pair.
- Install and configure the Nebius AI Cloud CLI.
-
Install jq to extract IDs from JSON data returned by the Nebius AI Cloud CLI:
sudo apt-get install jqbrew install jq - Generate an SSH key pair.
- Install and configure the Nebius AI Cloud provider for Terraform.
- Generate an SSH key pair.
- Install and initialize the Nebius SDK for Go.
- Generate an SSH key pair.
- Install and initialize the Nebius SDK for Python.
- Generate an SSH key pair.
Create a VM with eight GPUs with InfiniBand™ and a shared filesystem for training
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
-
In the sidebar, go to
Compute → Virtual machines.
- Click Create resource → Virtual machine. The creation flow is a step-by-step wizard. The sidebar shows your progress through the configuration sections. To move between sections, click Back and Next.
-
On the Compute step, configure computing resources:
- In the Platform section, select With GPUs and Regular. Then, select a platform with NVIDIA® H100, H200, or B200 GPUs. Only these platforms support GPU clusters.
- In the Settings section, select a Preset with 8 GPUs.
-
Create a GPU cluster for the VM. InfiniBand in the GPU cluster interconnects the VM GPUs for high-speed networking and efficient training.
To use a GPU cluster, select an existing one or create a new cluster:
- Click
Create in the GPU cluster field.
- In the window that opens, specify the cluster name and InfiniBand fabric. To select the fabric, see InfiniBand fabrics.
- Click Create.
- Click
- In the same section, select the Project for the VM location and specify the VM name.
-
On the Storage step, configure the boot disk and attach a shared filesystem:
-
In the Boot disk section, click
next to the boot disk. In the window that opens, keep New VM-managed disk selected, choose an Ubuntu operating system with pre-installed NVIDIA® GPU drivers, set the size to 50 GiB, and click Save.
-
In the Shared filesystems section, click
Attach shared filesystem.
- In the window that opens, create a new filesystem: specify its name, set the size to 50 GiB and the block size to 4 KiB. Click Attach filesystem.
-
After the window is closed, in the Mount tag field, specify a tag for mounting the filesystem to the VM.
Create your own tag, such as
my-filesystem. Make sure that it’s unique within the VM. - To mount the filesystem to the VM automatically, keep the Auto mount option enabled.
-
In the Boot disk section, click
- On the Network step, select the Network and Subnet. In the Public IP address field, select Auto (dynamic), so you can later connect to the VM by SSH.
-
On the Configuration step, in the Access section, add credentials so you can connect to the VM:
-
In the Username and SSH key field, click
Create.
-
In the window that opens, specify the username of the VM user, a public key of your SSH key pair and the credentials name to recognize the key in the list.
Do not use the
rootoradminusernames. They are reserved for internal needs and are not allowed to connect to a VM by SSH. - Click Add credentials.
-
In the Username and SSH key field, click
-
On the Review step, check the full VM configuration. To change a section quickly, click
next to the corresponding block. The wizard opens the relevant step with your current settings. Then, click Create VM.
-
Create a shared filesystem and save its ID to an environment variable:
The command creates a 50 GiB SSD shared filesystem with 4 KiB blocks.
export TR_VM_FILESYSTEM_ID=$(nebius compute filesystem create \ --name training-vm-filesystem-1 \ --size-gibibytes 50 \ --type network_ssd \ --block-size-bytes 4096 \ --format jsonpath='{.metadata.id}') -
Get the subnet ID and save it to an environment variable:
Possible subnet ID:
export SUBNET_ID=$(nebius vpc subnet list \ --format jsonpath='{.items[0].metadata.id}')vpcsubnet-***. -
For high-speed networking and efficient training, consider interconnecting the GPUs of multiple VMs in a GPU cluster by using InfiniBand. To do this, before creating the VM, create a GPU cluster to connect the VM and get its ID:
export GPU_CLUSTER_ID=$(nebius compute gpu-cluster create \ --name gpu-cluster-name \ --infiniband-fabric fabric-3 \ --format jsonpath='{.metadata.id}') -
Create a VM with 8 GPUs for training:
For more information about creating VMs and managing their network parameters, see How to create a virtual machine in Nebius AI Cloud.
export USER_DATA=$(jq -Rrs '.' <<EOF #cloud-config users: - name: user sudo: ALL=(ALL) NOPASSWD:ALL shell: /bin/bash ssh_authorized_keys: - $(cat ~/.ssh/id_ed25519.pub) EOF ) export TR_VM_ID=$(nebius compute instance create \ --name training-vm \ --resources-platform gpu-h100-sxm \ --resources-preset 8gpu-128vcpu-1600gb \ --boot-disk-managed-disk-name training-vm-disk-1 \ --boot-disk-managed-disk-type network_ssd \ --boot-disk-managed-disk-size-gibibytes 50 \ --boot-disk-managed-disk-block-size-bytes 4096 \ --boot-disk-managed-disk-source-image-family-image-family ubuntu24.04-cuda13.0 \ --boot-disk-attach-mode READ_WRITE \ --cloud-init-user-data "$USER_DATA" \ --filesystems "[{\"existing_filesystem\": {\"id\": \"$TR_VM_FILESYSTEM_ID\"}, \"attach_mode\": \"READ_WRITE\", \"mount_tag\": \"training-vm-filesystem-1\"}]" \ --network-interfaces "[{\"name\": \"eth0\", \"subnet_id\": \"$SUBNET_ID\", \"ip_address\": {}, \"public_ip_address\": {}}]" \ --gpu-cluster-id "$GPU_CLUSTER_ID" \ --format jsonpath='{.metadata.id}')
-
Create a boot disk:
The file contains the following parameters:
resource "nebius_compute_v1_disk" "<boot_disk_name>" { name = "training-vm-disk-1" parent_id = "<project_ID>" type = "NETWORK_SSD" size_gibibytes = 50 block_size_bytes = 4096 source_image_family = { image_family = "ubuntu24.04-cuda13.0" } }name: Disk name.parent_id: ID of the project where the disk is created.type: Disk type.size_gibibytes: Disk size in gibibytes.block_size_bytes: Block size in bytes.source_image_family.image_family: Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
-
Create a shared filesystem:
The configuration creates a 50 GiB SSD shared filesystem with 4 KiB blocks. The
resource "nebius_compute_v1_filesystem" "<filesystem_name>" { name = "training-vm-filesystem-1" parent_id = "<project_ID>" type = "NETWORK_SSD" size_gibibytes = 50 block_size_bytes = 4096 }name,parent_id,type,size_gibibytes, andblock_size_bytesparameters have the same meaning as for the disk. For the full resource reference, see nebius_compute_v1_filesystem. -
For high-speed networking and efficient training, consider interconnecting the GPUs of multiple VMs in a GPU cluster by using InfiniBand. To do this, create a GPU cluster to connect the VM:
The file contains the following parameters:
resource "nebius_compute_v1_gpu_cluster" "<gpu_cluster_name>" { name = "gpu-cluster-name" parent_id = "<project_ID>" infiniband_fabric = "fabric-3" }name: Cluster name.parent_id: ID of the project where the cluster is created.infiniband_fabric: InfiniBand™ fabric for the cluster. Select it depending on the region and GPU platform.
-
Create a user data configuration that adds a user with your SSH public key, and assign it to the
user_dataTerraform variable. For instructions, see Create a user data configuration. -
Create a VM with 8 GPUs for training:
The file contains the following parameters:
resource "nebius_compute_v1_instance" "<VM_name>" { name = "training-vm" parent_id = "<project_ID>" resources = { platform = "gpu-h100-sxm" preset = "8gpu-128vcpu-1600gb" } gpu_cluster = { id = nebius_compute_v1_gpu_cluster.<gpu_cluster_name>.id } boot_disk = { attach_mode = "READ_WRITE" existing_disk = { id = nebius_compute_v1_disk.<boot_disk_name>.id } } filesystems = [ { attach_mode = "READ_WRITE" mount_tag = "training-vm-filesystem-1" existing_filesystem = { id = nebius_compute_v1_filesystem.<filesystem_name>.id } } ] cloud_init_user_data = var.user_data network_interfaces = [ { name = "eth0" subnet_id = "<subnet_ID>" ip_address = {} public_ip_address = {} } ] }-
name: VM name. -
parent_id: ID of the project where the VM is created. Use the same project as for the GPU cluster. -
resources.platform: VM platform. -
resources.preset: VM preset with 8 GPUs. Depends on the chosen platform. -
gpu_cluster.id: ID of the GPU cluster created earlier. -
boot_disk.existing_disk.id: ID of the boot disk created earlier. -
boot_disk.attach_mode: Write permission of the boot disk. The only supported value isREAD_WRITE. -
filesystems: Settings of the shared filesystem attached to the VM:existing_filesystem.id: ID of the filesystem created earlier.attach_mode: Write permission of the filesystem. The only supported value isREAD_WRITE.mount_tag: Tag for mounting the filesystem to the VM. Make sure that it’s unique within the VM.
-
cloud_init_user_data: Configuration of VM users in the cloud-init format. It’s taken from theuser_datavariable. -
network_interfaces: Network settings:name: Name of the network interface.subnet_id: ID of the subnet that the VM is connected to.ip_address: Set to{}to assign a private IP address automatically.public_ip_address: Set to{}to assign a dynamic public IP address, so you can later connect to the VM by SSH.
public_ip_addressfromnetwork_interfaces. To access the VM, you can set up a WireGuard jump server later. This approach enhances security and still provides access to the VM within the same subnet. For the full resource reference, see nebius_compute_v1_instance. For more information about creating VMs and managing their network parameters, see How to create a virtual machine in Nebius AI Cloud. -
-
Check that the configuration is correct:
terraform validate -
Apply the changes:
terraform apply
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
trainingDiskOperation, err := sdk.Services().Compute().V1(). Disk().Create( ctx, &compute.CreateDiskRequest{ Metadata: &common.ResourceMetadata{ Name: "training-vm-disk-1", }, Spec: &compute.DiskSpec{ Size: &compute.DiskSpec_SizeGibibytes{ SizeGibibytes: 50, }, BlockSizeBytes: 4096, Type: compute.DiskSpec_NETWORK_SSD, Source: &compute.DiskSpec_SourceImageFamily{ SourceImageFamily: &compute.SourceImageFamily{ ImageFamily: "ubuntu24.04-cuda13.0", }, }, }, }, ) if err != nil { return err } if _, err = trainingDiskOperation.Wait(ctx); err != nil { return err } trainingBootDiskID := trainingDiskOperation.ResourceID() -
Create a shared filesystem:
The code creates a 50 GiB SSD shared filesystem with 4 KiB blocks.
trainingFSOperation, err := sdk.Services().Compute().V1(). Filesystem().Create( ctx, &compute.CreateFilesystemRequest{ Metadata: &common.ResourceMetadata{ Name: "training-vm-filesystem-1", }, Spec: &compute.FilesystemSpec{ Size: &compute.FilesystemSpec_SizeGibibytes{ SizeGibibytes: 50, }, BlockSizeBytes: 4096, Type: compute.FilesystemSpec_NETWORK_SSD, }, }, ) if err != nil { return err } if _, err = trainingFSOperation.Wait(ctx); err != nil { return err } trainingFilesystemID := trainingFSOperation.ResourceID() -
Get the subnet ID:
Possible subnet ID:
subnets, err := sdk.Services().VPC().V1(). Subnet().List( ctx, &vpc.ListSubnetsRequest{}, ) if err != nil { return err } if len(subnets.GetItems()) == 0 { return errors.New("no subnets found") } subnetID := subnets.GetItems()[0].GetMetadata().GetId()vpcsubnet-e0dcbaa76x2024xyz8. -
For high-speed networking and efficient training, consider interconnecting the GPUs of multiple VMs in a GPU cluster by using InfiniBand™. To do this, before creating the VM, create a GPU cluster to connect the VM and get its ID:
gpuClusterOperation, err := sdk.Services().Compute().V1(). GpuCluster().Create( ctx, &compute.CreateGpuClusterRequest{ Metadata: &common.ResourceMetadata{ Name: "gpu-cluster-name", }, Spec: &compute.GpuClusterSpec{ InfinibandFabric: "fabric-3", }, }, ) if err != nil { return err } if _, err = gpuClusterOperation.Wait(ctx); err != nil { return err } gpuClusterID := gpuClusterOperation.ResourceID() -
Create a VM with 8 GPUs for training:
The given example assumes that you work with VMs that have public addresses, so you can later connect to these VMs by SSH. However, if you need isolated VMs without public addresses, do not set
var gpuCluster *compute.InstanceGpuClusterSpec if gpuClusterID != "" { gpuCluster = &compute.InstanceGpuClusterSpec{ Id: gpuClusterID, } } trainingNetworkName := "multi-gpu-node-compute-api-" + "network-interface" trainingResources := &compute.ResourcesSpec{ Platform: "gpu-h100-sxm", Size: &compute.ResourcesSpec_Preset{ Preset: "8gpu-128vcpu-1600gb", }, } trainingBootDisk := &compute.AttachedDiskSpec{ AttachMode: compute.AttachedDiskSpec_READ_WRITE, Type: &compute.AttachedDiskSpec_ExistingDisk{ ExistingDisk: &compute.ExistingDisk{ Id: trainingBootDiskID, }, }, } trainingExistingFS := &compute.ExistingFilesystem{ Id: trainingFilesystemID, } trainingFilesystem := &compute.AttachedFilesystemSpec{ AttachMode: compute.AttachedFilesystemSpec_READ_WRITE, MountTag: "training-vm-filesystem-1", Type: &compute.AttachedFilesystemSpec_ExistingFilesystem{ ExistingFilesystem: trainingExistingFS, }, } trainingNetwork := &compute.NetworkInterfaceSpec{ Name: trainingNetworkName, SubnetId: subnetID, IpAddress: &compute.IPAddress{}, PublicIpAddress: &compute.PublicIPAddress{}, } trainingSpec := &compute.InstanceSpec{ Resources: trainingResources, GpuCluster: gpuCluster, BootDisk: trainingBootDisk, Filesystems: []*compute.AttachedFilesystemSpec{ trainingFilesystem, }, CloudInitUserData: userData, NetworkInterfaces: []*compute.NetworkInterfaceSpec{ trainingNetwork, }, } trainingVMOperation, err := sdk.Services().Compute().V1(). Instance().Create( ctx, &compute.CreateInstanceRequest{ Metadata: &common.ResourceMetadata{ Name: "training-vm", }, Spec: trainingSpec, }, ) if err != nil { return err } if _, err = trainingVMOperation.Wait(ctx); err != nil { return err } trainingVMID := trainingVMOperation.ResourceID()PublicIpAddressin the network interface specification. To access the VM, you can set up a WireGuard jump server later. This approach enhances security and still provides access to the VM within the same subnet. For more information about creating VMs and managing their network parameters, see How to create a virtual machine in Nebius AI Cloud.
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
disk_service = DiskServiceClient(sdk) training_disk_operation = await disk_service.create( CreateDiskRequest( metadata=ResourceMetadata( name="training-vm-disk-1", ), spec=DiskSpec( block_size_bytes=4096, type=DiskSpec.DiskType.NETWORK_SSD, source_image_family=SourceImageFamily( image_family="ubuntu24.04-cuda13.0", ), size_gibibytes=50, ), ), ) await training_disk_operation.wait() training_boot_disk_id = training_disk_operation.resource_id -
Create a shared filesystem:
The code creates a 50 GiB SSD shared filesystem with 4 KiB blocks.
filesystem_service = FilesystemServiceClient(sdk) training_fs_operation = await filesystem_service.create( CreateFilesystemRequest( metadata=ResourceMetadata( name="training-vm-filesystem-1", ), spec=FilesystemSpec( block_size_bytes=4096, type=FilesystemSpec.FilesystemType.NETWORK_SSD, size_gibibytes=50, ), ), ) await training_fs_operation.wait() training_filesystem_id = training_fs_operation.resource_id -
Get the subnet ID:
Possible subnet ID:
subnet_service = SubnetServiceClient(sdk) subnets = await subnet_service.list(ListSubnetsRequest()) if not subnets.items: raise ValueError("no subnets found") subnet_id = subnets.items[0].metadata.idvpcsubnet-e0dcbaa76x2024xyz8. -
For high-speed networking and efficient training, consider interconnecting the GPUs of multiple VMs in a GPU cluster by using InfiniBand™. To do this, before creating the VM, create a GPU cluster to connect the VM and get its ID:
gpu_cluster_service = GpuClusterServiceClient(sdk) gpu_cluster_operation = await gpu_cluster_service.create( CreateGpuClusterRequest( metadata=ResourceMetadata( name="gpu-cluster-name", ), spec=GpuClusterSpec( infiniband_fabric="fabric-3", ), ), ) await gpu_cluster_operation.wait() gpu_cluster_id = gpu_cluster_operation.resource_id -
Create a VM with 8 GPUs for training:
The given example assumes that you work with VMs that have public addresses, so you can later connect to these VMs by SSH. However, if you need isolated VMs without public addresses, do not set
gpu_cluster = None if gpu_cluster_id: gpu_cluster = InstanceGpuClusterSpec(id=gpu_cluster_id) training_network_name = ( "multi-gpu-node-compute-api-" "network-interface" ) instance_service = InstanceServiceClient(sdk) training_vm_operation = await instance_service.create( CreateInstanceRequest( metadata=ResourceMetadata(name="training-vm"), spec=InstanceSpec( resources=ResourcesSpec( platform="gpu-h100-sxm", preset="8gpu-128vcpu-1600gb", ), gpu_cluster=gpu_cluster, boot_disk=AttachedDiskSpec( attach_mode=( AttachedDiskSpec.AttachMode.READ_WRITE ), existing_disk=ExistingDisk( id=training_boot_disk_id, ), ), filesystems=[ AttachedFilesystemSpec( attach_mode=( AttachedFilesystemSpec.AttachMode.READ_WRITE ), existing_filesystem=ExistingFilesystem( id=training_filesystem_id, ), mount_tag="training-vm-filesystem-1", ), ], cloud_init_user_data=user_data, network_interfaces=[ NetworkInterfaceSpec( name=training_network_name, subnet_id=subnet_id, ip_address=IPAddress(), public_ip_address=PublicIPAddress(), ), ], ), ), ) await training_vm_operation.wait() training_vm_id = training_vm_operation.resource_idpublic_ip_addressin the network interface specification. To access the VM, you can set up a WireGuard jump server later. This approach enhances security and still provides access to the VM within the same subnet. For more information about creating VMs and managing their network parameters, see How to create a virtual machine in Nebius AI Cloud.
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
const diskService = new DiskService(sdk); const trainingDiskOperation = await diskService.create( CreateDiskRequest.create({ metadata: ResourceMetadata.create({ name: "training-vm-disk-1", }), spec: DiskSpec.create({ blockSizeBytes: 4096, type: DiskSpec_DiskType.NETWORK_SSD, source: { $case: "sourceImageFamily", sourceImageFamily: SourceImageFamily.create({ imageFamily: "ubuntu24.04-cuda13.0", }), }, size: { $case: "sizeGibibytes", sizeGibibytes: 50, }, }), }), ).result; await trainingDiskOperation.wait(); const trainingBootDiskId = trainingDiskOperation.resourceId(); -
Create a shared filesystem:
The code creates a 50 GiB SSD shared filesystem with 4 KiB blocks.
const filesystemService = new FilesystemService(sdk); const trainingFsOperation = await filesystemService.create( CreateFilesystemRequest.create({ metadata: ResourceMetadata.create({ name: "training-vm-filesystem-1", }), spec: FilesystemSpec.create({ blockSizeBytes: 4096, type: FilesystemSpec_FilesystemType.NETWORK_SSD, size: { $case: "sizeGibibytes", sizeGibibytes: 1024, }, }), }), ).result; await trainingFsOperation.wait(); const trainingFilesystemId = trainingFsOperation.resourceId(); -
Get the subnet ID:
Possible subnet ID:
const subnetService = new SubnetService(sdk); const subnets = await subnetService.list( ListSubnetsRequest.create({}), ); const subnetId = subnets.items[0]?.metadata?.id; if (!subnetId) { throw new Error("no subnets found"); }vpcsubnet-e0dcbaa76x2024xyz8. -
For high-speed networking and efficient training, consider interconnecting the GPUs of multiple VMs in a GPU cluster by using InfiniBand™. To do this, before creating the VM, create a GPU cluster to connect the VM and get its ID:
const gpuClusterService = new GpuClusterService(sdk); const gpuClusterOperation = await gpuClusterService.create( CreateGpuClusterRequest.create({ metadata: ResourceMetadata.create({ name: "gpu-cluster-name", }), spec: GpuClusterSpec.create({ infinibandFabric: "fabric-3", }), }), ).result; await gpuClusterOperation.wait(); let gpuClusterId = gpuClusterOperation.resourceId(); -
Create a VM with 8 GPUs for training:
The given example assumes that you work with VMs that have public addresses, so you can later connect to these VMs by SSH. However, if you need isolated VMs without public addresses, do not set
const gpuCluster = gpuClusterId ? InstanceGpuClusterSpec.create({ id: gpuClusterId }) : undefined; const trainingNetworkName = "multi-gpu-node-compute-api-" + "network-interface"; const instanceService = new InstanceService(sdk); const trainingVmOperation = await instanceService.create( CreateInstanceRequest.create({ metadata: ResourceMetadata.create({ name: "training-vm", }), spec: InstanceSpec.create({ resources: ResourcesSpec.create({ platform: "gpu-h100-sxm", size: { $case: "preset", preset: "8gpu-128vcpu-1600gb", }, }), gpuCluster, bootDisk: AttachedDiskSpec.create({ attachMode: AttachedDiskSpec_AttachMode.READ_WRITE, type: { $case: "existingDisk", existingDisk: ExistingDisk.create({ id: trainingBootDiskId, }), }, }), filesystems: [ AttachedFilesystemSpec.create({ attachMode: AttachedFilesystemSpec_AttachMode.READ_WRITE, mountTag: "training-vm-filesystem-1", type: { $case: "existingFilesystem", existingFilesystem: ExistingFilesystem.create({ id: trainingFilesystemId, }), }, }), ], cloudInitUserData: userData, networkInterfaces: [ NetworkInterfaceSpec.create({ name: trainingNetworkName, subnetId, ipAddress: IPAddress.create({}), publicIpAddress: PublicIPAddress.create({}), }), ], }), }), ).result; await trainingVmOperation.wait(); const trainingVmId = trainingVmOperation.resourceId();publicIpAddressin the network interface specification. To access the VM, you can set up a WireGuard jump server later. This approach enhances security and still provides access to the VM within the same subnet. For more information about creating VMs and managing their network parameters, see How to create a virtual machine in Nebius AI Cloud.
Create a VM with one GPU for inference
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
-
In the sidebar, go to
Compute → Virtual machines.
- Click Create resource → Virtual machine.
-
On the Compute step, configure computing resources:
- In the Platform section, select With GPUs and Regular. Then, select any GPU platform.
- In the Settings section, select a Preset with one GPU and the Project for the VM location, and specify the VM name.
-
On the Storage step, in the Boot disk section, click
next to the boot disk. In the window that opens, keep New VM-managed disk selected, choose an Ubuntu operating system with pre-installed NVIDIA® GPU drivers, set the size to 50 GiB, and click Save.
- On the Network step, select the Network and Subnet. In the Public IP address field, select Auto (dynamic), so you can later connect to the VM by SSH.
- On the Configuration step, in the Access section, add the same username and SSH public key that you used for the VM with eight GPUs.
- On the Review step, check the full VM configuration and click Create VM.
Create a VM with one GPU for inference:The command creates the VM with a boot disk that has the same settings as the boot disk of the training VM. It uses the same
export INF_VM_ID=$(nebius compute instance create \
--name inference-vm \
--resources-platform gpu-h100-sxm \
--resources-preset 1gpu-16vcpu-200gb \
--boot-disk-managed-disk-name inference-vm-disk-1 \
--boot-disk-managed-disk-type network_ssd \
--boot-disk-managed-disk-size-gibibytes 50 \
--boot-disk-managed-disk-block-size-bytes 4096 \
--boot-disk-managed-disk-source-image-family-image-family ubuntu24.04-cuda13.0 \
--boot-disk-attach-mode READ_WRITE \
--cloud-init-user-data "$USER_DATA" \
--network-interfaces "[{\"name\": \"eth0\", \"subnet_id\": \"$SUBNET_ID\", \"ip_address\": {}, \"public_ip_address\": {}}]" \
--format jsonpath='{.metadata.id}')
USER_DATA variable and cloud-init configuration as the training VM.-
Create a boot disk:
The disk has the same settings as the boot disk of the training VM. Give the resource a label that differs from the label of the training disk.
resource "nebius_compute_v1_disk" "<boot_disk_name>" { name = "inference-vm-disk-1" parent_id = "<project_ID>" type = "NETWORK_SSD" size_gibibytes = 50 block_size_bytes = 4096 source_image_family = { image_family = "ubuntu24.04-cuda13.0" } } -
Create a VM with one GPU for inference:
Give the resource a label that differs from the label of the training VM, and in
resource "nebius_compute_v1_instance" "<VM_name>" { name = "inference-vm" parent_id = "<project_ID>" resources = { platform = "gpu-h100-sxm" preset = "1gpu-16vcpu-200gb" } boot_disk = { attach_mode = "READ_WRITE" existing_disk = { id = nebius_compute_v1_disk.<boot_disk_name>.id } } cloud_init_user_data = var.user_data network_interfaces = [ { name = "eth0" subnet_id = "<subnet_ID>" ip_address = {} public_ip_address = {} } ] }boot_disk.existing_disk.id, reference the disk that you created in the previous step. The VM uses the sameuser_datavariable and cloud-init configuration as the training VM. For the meaning of other parameters, see the training VM configuration. -
Check that the configuration is correct:
terraform validate -
Apply the changes:
terraform apply
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
inferenceDiskOperation, err := sdk.Services().Compute().V1().Disk().Create( ctx, &compute.CreateDiskRequest{ Metadata: &common.ResourceMetadata{ Name: "inference-vm-disk-1", }, Spec: &compute.DiskSpec{ Size: &compute.DiskSpec_SizeGibibytes{ SizeGibibytes: 50, }, BlockSizeBytes: 4096, Type: compute.DiskSpec_NETWORK_SSD, Source: &compute.DiskSpec_SourceImageFamily{ SourceImageFamily: &compute.SourceImageFamily{ ImageFamily: "ubuntu24.04-cuda13.0", }, }, }, }, ) if err != nil { return err } if _, err = inferenceDiskOperation.Wait(ctx); err != nil { return err } inferenceBootDiskID := inferenceDiskOperation.ResourceID() -
Create a VM with one GPU for inference:
inferenceNetworkName := "single-gpu-node-compute-api-" + "network-interface" inferenceResources := &compute.ResourcesSpec{ Platform: "gpu-h100-sxm", Size: &compute.ResourcesSpec_Preset{ Preset: "1gpu-16vcpu-200gb", }, } inferenceBootDisk := &compute.AttachedDiskSpec{ AttachMode: compute.AttachedDiskSpec_READ_WRITE, Type: &compute.AttachedDiskSpec_ExistingDisk{ ExistingDisk: &compute.ExistingDisk{ Id: inferenceBootDiskID, }, }, } inferenceNetwork := &compute.NetworkInterfaceSpec{ Name: inferenceNetworkName, SubnetId: subnetID, IpAddress: &compute.IPAddress{}, PublicIpAddress: &compute.PublicIPAddress{}, } inferenceSpec := &compute.InstanceSpec{ Resources: inferenceResources, BootDisk: inferenceBootDisk, CloudInitUserData: userData, NetworkInterfaces: []*compute.NetworkInterfaceSpec{ inferenceNetwork, }, } inferenceVMOperation, err := sdk.Services().Compute().V1(). Instance().Create( ctx, &compute.CreateInstanceRequest{ Metadata: &common.ResourceMetadata{ Name: "inference-vm", }, Spec: inferenceSpec, }, ) if err != nil { return err } if _, err = inferenceVMOperation.Wait(ctx); err != nil { return err } inferenceVMID := inferenceVMOperation.ResourceID()
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
inference_disk_service = DiskServiceClient(sdk) inference_disk_operation = await inference_disk_service.create( CreateDiskRequest( metadata=ResourceMetadata( name="inference-vm-disk-1", ), spec=DiskSpec( block_size_bytes=4096, type=DiskSpec.DiskType.NETWORK_SSD, source_image_family=SourceImageFamily( image_family="ubuntu24.04-cuda13.0", ), size_gibibytes=50, ), ), ) await inference_disk_operation.wait() inference_boot_disk_id = inference_disk_operation.resource_id -
Create a VM with one GPU for inference:
inference_network_name = ( "single-gpu-node-compute-api-" "network-interface" ) inference_service = InstanceServiceClient(sdk) inference_vm_operation = await inference_service.create( CreateInstanceRequest( metadata=ResourceMetadata(name="inference-vm"), spec=InstanceSpec( resources=ResourcesSpec( platform="gpu-h100-sxm", preset="1gpu-16vcpu-200gb", ), boot_disk=AttachedDiskSpec( attach_mode=( AttachedDiskSpec.AttachMode.READ_WRITE ), existing_disk=ExistingDisk( id=inference_boot_disk_id, ), ), cloud_init_user_data=user_data, network_interfaces=[ NetworkInterfaceSpec( name=inference_network_name, subnet_id=subnet_id, ip_address=IPAddress(), public_ip_address=PublicIPAddress(), ), ], ), ), ) await inference_vm_operation.wait() inference_vm_id = inference_vm_operation.resource_id
-
Create a boot disk:
The code creates a 50 GiB SSD disk with a 4 KiB block size and an Ubuntu boot image with pre-installed NVIDIA® GPU drivers. For details about boot disk images, see Boot disk images for Compute virtual machines.
const inferenceDiskService = new DiskService(sdk); const inferenceDiskOperation = await inferenceDiskService.create( CreateDiskRequest.create({ metadata: ResourceMetadata.create({ name: "inference-vm-disk-1", }), spec: DiskSpec.create({ blockSizeBytes: 4096, type: DiskSpec_DiskType.NETWORK_SSD, source: { $case: "sourceImageFamily", sourceImageFamily: SourceImageFamily.create({ imageFamily: "ubuntu24.04-cuda13.0", }), }, size: { $case: "sizeGibibytes", sizeGibibytes: 50, }, }), }), ).result; await inferenceDiskOperation.wait(); const inferenceBootDiskId = inferenceDiskOperation.resourceId(); -
Create a VM with one GPU for inference:
const inferenceNetworkName = "single-gpu-node-compute-api-" + "network-interface"; const inferenceService = new InstanceService(sdk); const inferenceVmOperation = await inferenceService.create( CreateInstanceRequest.create({ metadata: ResourceMetadata.create({ name: "inference-vm", }), spec: InstanceSpec.create({ resources: ResourcesSpec.create({ platform: "gpu-h100-sxm", size: { $case: "preset", preset: "1gpu-16vcpu-200gb", }, }), bootDisk: AttachedDiskSpec.create({ attachMode: AttachedDiskSpec_AttachMode.READ_WRITE, type: { $case: "existingDisk", existingDisk: ExistingDisk.create({ id: inferenceBootDiskId, }), }, }), cloudInitUserData: userData, networkInterfaces: [ NetworkInterfaceSpec.create({ name: inferenceNetworkName, subnetId, ipAddress: IPAddress.create({}), publicIpAddress: PublicIPAddress.create({}), }), ], }), }), ).result; await inferenceVmOperation.wait(); const inferenceVmId = inferenceVmOperation.resourceId();
Connect to the VMs
Connect to the VM for training via SSH:- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
-
In the sidebar, go to
Compute → Virtual machines.
- Open the page of the VM and copy its public IP address.
-
Use the public IP address to connect to the VM:
Use the username that you specified when creating the VM.
ssh <username>@<public_IP_address>
-
Get the public IP address of your VM and save it to an environment variable:
export TR_PUBLIC_IP_ADDRESS=$(nebius compute instance get \ --id $TR_VM_ID \ --format json \ | jq -r '.status.network_interfaces[0].public_ip_address.address | split("/")[0]') -
Use the public IP address to connect to the VM:
ssh user@$TR_PUBLIC_IP_ADDRESS
-
Add an output that returns the public IP address of your VM:
The output takes
output "training_vm_public_ip" { value = split("/", nebius_compute_v1_instance.<VM_name>.status.network_interfaces[0].public_ip_address.address)[0] }addressfromstatus.network_interfaces[0].public_ip_addressof the VM and removes the prefix length after/. -
Apply the changes:
Terraform doesn’t change the VM. It only calculates the new output.
terraform apply -
Get the public IP address:
terraform output -raw training_vm_public_ip -
Use the public IP address to connect to the VM:
ssh user@<training_public_IP_address>
-
Get the public IP address of your VM:
trainingInstance, err := sdk.Services().Compute().V1(). Instance().Get( ctx, &compute.GetInstanceRequest{ Id: trainingVMID, }, ) if err != nil { return err } trainingAddress := trainingInstance.GetStatus(). GetNetworkInterfaces()[0]. GetPublicIpAddress().GetAddress() trainingPublicIP := strings.Split(trainingAddress, "/")[0] if trainingPublicIP == "" { return fmt.Errorf("training VM public IP is missing") } -
Use the public IP address to connect to the VM:
ssh user@<training_public_IP_address>
-
Get the public IP address of your VM:
training_ip_service = InstanceServiceClient(sdk) training_instance = await training_ip_service.get( GetInstanceRequest(id=training_vm_id), ) training_address = ( training_instance.status.network_interfaces[0] .public_ip_address.address ) training_public_ip = training_address.split("/")[0] -
Use the public IP address to connect to the VM:
ssh user@<training_public_IP_address>
-
Get the public IP address of your VM:
const trainingIpService = new InstanceService(sdk); const trainingInstance = await trainingIpService.get( GetInstanceRequest.create({ id: trainingVmId, }), ); const trainingAddress = trainingInstance.status ?.networkInterfaces[0]?.publicIpAddress?.address; const trainingPublicIp = trainingAddress?.split("/")[0]; if (!trainingPublicIp) { throw new Error("training VM public IP is missing"); } -
Use the public IP address to connect to the VM:
ssh user@<training_public_IP_address>
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
Do the same steps as for the training VM. Open the page of the inference VM, copy its public IP address, and then connect to the inference VM:Use the username that you specified when creating the inference VM.
ssh <username>@<public_IP_address>
-
Get the public IP address of your VM and save it to an environment variable:
export INF_PUBLIC_IP_ADDRESS=$(nebius compute instance get \ --id $INF_VM_ID \ --format json \ | jq -r '.status.network_interfaces[0].public_ip_address.address | split("/")[0]') -
Use the public IP address to connect to the VM:
ssh user@$INF_PUBLIC_IP_ADDRESS
-
Add an output that returns the public IP address of your VM:
output "inference_vm_public_ip" { value = split("/", nebius_compute_v1_instance.<VM_name>.status.network_interfaces[0].public_ip_address.address)[0] } -
Apply the changes:
terraform apply -
Get the public IP address:
terraform output -raw inference_vm_public_ip -
Use the public IP address to connect to the VM:
ssh user@<inference_public_IP_address>
-
Get the public IP address of your VM:
inferenceInstance, err := sdk.Services().Compute().V1(). Instance().Get( ctx, &compute.GetInstanceRequest{ Id: inferenceVMID, }, ) if err != nil { return err } inferenceAddress := inferenceInstance.GetStatus(). GetNetworkInterfaces()[0]. GetPublicIpAddress().GetAddress() inferencePublicIP := strings.Split(inferenceAddress, "/")[0] if inferencePublicIP == "" { return fmt.Errorf("inference VM public IP is missing") } -
Use the public IP address to connect to the VM:
ssh user@<inference_public_IP_address>
-
Get the public IP address of your VM:
inference_ip_service = InstanceServiceClient(sdk) inference_instance = await inference_ip_service.get( GetInstanceRequest(id=inference_vm_id), ) inference_address = ( inference_instance.status.network_interfaces[0] .public_ip_address.address ) inference_public_ip = inference_address.split("/")[0] -
Use the public IP address to connect to the VM:
ssh user@<inference_public_IP_address>
-
Get the public IP address of your VM:
const inferenceIpService = new InstanceService(sdk); const inferenceInstance = await inferenceIpService.get( GetInstanceRequest.create({ id: inferenceVmId, }), ); const inferenceAddress = inferenceInstance.status ?.networkInterfaces[0]?.publicIpAddress?.address; const inferencePublicIp = inferenceAddress?.split("/")[0]; if (!inferencePublicIp) { throw new Error("inference VM public IP is missing"); } -
Use the public IP address to connect to the VM:
ssh user@<inference_public_IP_address>
What’s next
- Learn about VM and GPU types
- Learn how to create different types of VMs
- Learn more about VM networking
- Learn how to work with GPU clusters