> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nebius.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Deploying an LLM on a virtual machine with Nebius Tunnels
You can deploy and run a large language model (LLM) on a GPU virtual machine (VM) and then chat with this model via a secure [tunnel](/tunnels/overview#how-tunnels-work) by Nebius Tunnels. The VM goes without a public IP address, but by using the tunnel, anyone from the internet can access the LLM and connect to the VM. Meanwhile, the VM stays isolated.
To establish the tunnel, the VM configuration includes a tunnel agent. The agent runs on the VM automatically after the VM starts. When you send a message to the model, the tunnel routes the traffic to the agent. Thus, only the IP address of the tunnel is exposed to the internet.
Use one VM per tunnel. Do not add several VMs to a single tunnel.
## Costs
Nebius AI Cloud charges you for the following billing items:
* [Compute virtual machine](/compute/resources/pricing#virtual-machines-gpus-vcpus-ram)
* [Boot disk](/compute/resources/pricing#disks) attached to the VM
Nebius Tunnels is [in preview](/overview/services#service-and-application-stages) and provided free of charge.
## Steps
### Create a tunnel
1. In the [web console](https://console.nebius.com), select the project where you want to create the infrastructure. All resources must be located in the same project.
2. Go to **Networking** → **Tunnels**.
3. Click **Create tunnel**.
4. On the page that opens, set the tunnel name to `my-tunnel`.
5. Click **Create tunnel**.
1. [Install and configure](/cli/install) the Nebius AI Cloud CLI.
When configuring the CLI profile, set the project where you want to create the infrastructure. All resources must be located in the same project.
All `nebius` commands by default use the [project ID](/iam/manage-projects#how-to-get-a-project-id) from your CLI profile configuration.
2. Create a tunnel:
```bash theme={null}
nebius tunnel create --title my-tunnel
```
### Prepare a service account
To authorize the VM in the tunnel, prepare a service account with the `applicationtunnel.agent` role. Later, you attach this service account to the VM.
1. Create the service account:
1. In the sidebar, go to **Administration** → **IAM**.
2. Click **Create resource** → **Service account**.
3. In the window that opens, set the name to `tunnel-agent-sa` and select the project.
4. Click **Create and continue**.
```bash theme={null}
nebius iam service-account create --name tunnel-agent-sa
```
2. Create a [custom group](/iam/authorization/groups/index#custom-groups) in Identity and Access Management to assign the role to the service account:
1. In the sidebar, go to **Administration** → **IAM**.
2. Click **Create resource** → **Group**.
3. In the window that opens, set the group name to `tunnel-agents`. In the **Scope** field, select the project where you create all the resources.
4. Click **Create**.
```bash theme={null}
nebius iam group create --name tunnel-agents
```
3. Attach the tunnel and grant the `applicationtunnel.agent` role to this group. You can only grant this role by using the CLI.
1. Get the group ID:
```bash theme={null}
nebius iam group list
```
The ID is specified in the `items.metadata.id` parameter in the output.
2. Get the tunnel ID:
```bash theme={null}
nebius tunnel list
```
3. Create an access permit in the custom group:
```bash theme={null}
nebius iam access-permit create \
--parent-id \
--resource-id \
--role applicationtunnel.agent
```
4. Add the service account to the group:
1. Go to **Administration** → **IAM**.
2. On the **Groups** tab, find and open the `tunnel-agents` group.
3. Click **Add members**.
4. In the window that opens, add the `tunnel-agent-sa` service account.
5. Once the account is added, close the window.
Check the group page in general. Make sure that:
* The `tunnel-agent-sa` service account is specified on the **Members** tab.
* The tunnel and the `applicationtunnel.agent` role are specified on the **Access permits** tab.
```bash theme={null}
nebius iam group-membership create \
--parent-id \
--member-id
```
To get the service account ID, run:
```bash theme={null}
nebius iam service-account list
```
### Create a virtual machine
Prepare the VM configuration that includes the Qwen/Qwen3-8B model and the tunnel agent. Then, create the VM — the model and agent are deployed on the VM automatically.
If you need to change the VM configuration later, delete the VM and create a new one. A tunnel should be associated with only one VM, and you can't change the cloud-init configuration of an existing VM.
1. [Generate an SSH key pair](/compute/virtual-machines/ssh-keys#generating-a-key-pair).
2. Prepare the cloud-init configuration file. You'll attach it to the VM.
In this file, specify two parameters:
* `users.ssh_authorized_keys`: Public SSH key.
* `write_files.content.tunnel_id`: ID of the tunnel.
```yaml theme={null}
users:
- name: ubuntu
sudo: "ALL=(ALL) NOPASSWD:ALL"
shell: /bin/bash
ssh_authorized_keys:
-
write_files:
- path: /etc/nebius-tunnel-agent/config.yaml
permissions: "0640"
content: |
tunnel_id:
iam:
metadata:
enabled: true
logging:
level: info
format: json
services:
- name: llm
type: http
target: 127.0.0.1:8000
- name: ssh
type: tcp
target: 127.0.0.1:22
- path: /etc/systemd/system/nebius-tunnel-agent.service
permissions: "0644"
content: |
[Unit]
Description=Nebius tunnel agent
Documentation=https://docs.nebius.com/tunnels
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=tunna
Group=tunna
ExecStart=/usr/local/bin/nebius-tunnel-agent --config /etc/nebius-tunnel-agent/config.yaml
Restart=always
RestartSec=5
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadOnlyPaths=/etc/nebius-tunnel-agent
[Install]
WantedBy=multi-user.target
- path: /usr/local/sbin/install-nebius-tunnel-agent
permissions: "0755"
content: |
#!/bin/bash
set -euo pipefail
VERSION="${AGENT_VERSION:-latest}"
BASE="https://storage.eu-north1.nebius.cloud/products/releases/nebius-tunnel-agent"
CONF_DIR=/etc/nebius-tunnel-agent
case "$(uname -m)" in
x86_64) PLATFORM=linux-x86_64 ;;
aarch64|arm64) PLATFORM=linux-aarch64 ;;
*) echo "unsupported architecture: $(uname -m)" >&2; exit 1 ;;
esac
ASSET="nebius-tunnel-agent-${PLATFORM}.tar.gz"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
fetch() {
curl -fsSL --retry 5 --retry-delay 3 --retry-connrefused -o "$2" "$1"
}
fetch "$BASE/$VERSION/$ASSET" "$TMP/$ASSET"
fetch "$BASE/$VERSION/checksums-sha256.txt" "$TMP/checksums-sha256.txt"
(cd "$TMP" && sha256sum --ignore-missing -c checksums-sha256.txt)
tar -xzf "$TMP/$ASSET" -C "$TMP"
install -m0755 "$TMP/nebius-tunnel-agent" /usr/local/bin/nebius-tunnel-agent
getent group tunna >/dev/null || groupadd --system tunna
id -u tunna >/dev/null 2>&1 || useradd --system --gid tunna \
--no-create-home --shell /usr/sbin/nologin tunna
chown -R tunna:tunna "$CONF_DIR"
chmod 0750 "$CONF_DIR"
chmod 0640 "$CONF_DIR/config.yaml"
systemctl daemon-reload
systemctl enable nebius-tunnel-agent.service
systemctl restart nebius-tunnel-agent.service
- path: /etc/systemd/system/vllm.service
permissions: "0644"
content: |
[Unit]
Description=vLLM (OpenAI-compatible API behind a Nebius tunnel)
After=network-online.target docker.service
Wants=network-online.target
Requires=docker.service
[Service]
ExecStartPre=-/usr/bin/docker rm -f vllm
ExecStartPre=/usr/bin/docker pull vllm/vllm-openai:latest
ExecStart=/usr/bin/docker run --rm --name vllm \
--gpus all \
-p 127.0.0.1:8000:8000 \
-v /opt/vllm/hf-cache:/root/.cache/huggingface \
vllm/vllm-openai:latest \
--model Qwen/Qwen3-8B \
--served-model-name qwen3-8b \
--max-model-len 8192 \
--host 0.0.0.0 --port 8000
ExecStop=/usr/bin/docker stop vllm
Restart=on-failure
RestartSec=10
TimeoutStartSec=0
[Install]
WantedBy=multi-user.target
runcmd:
# Agent first: the tunnel is up while the backend is still pulling images.
- [/usr/local/sbin/install-nebius-tunnel-agent]
- mkdir -p /opt/vllm/hf-cache
- systemctl enable --now vllm.service
```
This configuration enables the following actions:
* Add a username and a public SSH key to the VM.
* Authenticate by using the service account attached to the VM.
* Create two [Nebius Tunnels services](/tunnels/overview#supported-traffic):
* `llm`: To address the model and chat with it.
* `ssh`: To connect to the VM by using SSH.
* Deploy the tunnel agent on the VM.
* Deploy a Docker image with the Qwen/Qwen3-8B model on the VM.
3. Create the VM:
1. In the sidebar, go to **Compute** → **Virtual machines**.
2. Click **Create resource** → **Virtual machine**.
The creation flow is a step-by-step wizard. The sidebar shows your progress through the configuration sections. To move between sections, click **Back** and **Next**.
3. On the **Compute** step, configure computing resources:
1. In the **Platform** section, select:
* With GPUs
* Regular
* NVIDIA® H100 NVLink
2. In the **Settings** section, set:
* **Preset**: 1 GPU - 16 CPUs - 200 GiB RAM.
* **Project**: The project where you create all resources for the tunnel.
* **VM name**: `tunnel-llm`.
4. On the **Storage** step:
1. Preserve the **Ubuntu 24.04 LTS for NVIDIA® GPUs (CUDA® 13)** disk image.
2. Change the disk type to Network SSD IO M3.
3. Change the disk size to 186 GiB.
5. On the **Network** step, check the network and subnet. Keep **None** in the **Public IP address** field.
6. On the **Configuration** step, configure access, identity and VM startup settings:
1. In the **Service account** field, select the `tunnel-agent-sa` service account.
2. Enable a custom cloud-init configuration file.
3. In the field that appears, paste the contents of the `llm.yaml` file prepared earlier.
7. On the **Review** step, check the full VM configuration. To change a section quickly, click next to the corresponding block. The wizard opens the relevant step with your current settings.
8. Click **Create VM**.
```bash theme={null}
nebius compute instance create \
--name tunnel-llm \
--resources-platform gpu-h100-sxm \
--resources-preset 1gpu-16vcpu-200gb \
--boot-disk-attach-mode READ_WRITE \
--boot-disk-managed-disk-name tunnel-llm-boot \
--boot-disk-managed-disk-type network_ssd_io_m3 \
--boot-disk-managed-disk-size-gibibytes 186 \
--boot-disk-managed-disk-source-image-family-image-family ubuntu24.04-cuda13.0 \
--network-interfaces "[{\"name\": \"eth0\", \"ip_address\": {}, \"subnet_id\": \"\"}]" \
--service-account-id \
--cloud-init-user-data "$(cat llm.yaml)"
```
In the command, specify the [subnet ID](/vpc/networking/resources#how-to-get-a-subnet-id) and the ID of the `tunnel-agent-sa` service account.
Wait for the VM to start running.
### Check the result and chat with the LLM
The first VM boot pulls the LLM Docker image and model weights. The model deployment takes 10-20 minutes. The tunnel agent becomes available before the LLM starts.
1. To check the status of the model deployment, connect to the VM through the `ssh` tunnel service:
```bash theme={null}
ssh -o ProxyCommand="openssl s_client -quiet -verify_quiet -servername %h -connect %h:443" \
ubuntu@ssh-.tunnel.applications..nebius.cloud
```
In the VM address, specify:
* `tunnel_masked_ID`: Mask of the tunnel ID without the `applicationtunnel-` prefix and regional routing code. Contains the last 15 characters of the tunnel ID. For example, for `applicationtunnel-e00abcdef123456789`, the masked ID of the tunnel is `abcdef123456789`.
* `region`: [Region](/overview/regions) of the project where you created the resources.
For more information about the tunnel URL, see [Supported traffic](/tunnels/overview#supported-traffic).
2. Check logs:
```bash theme={null}
journalctl -u vllm -f
```
When the logs contain `INFO: Application startup complete`, the model is ready to serve requests.
3. Disconnect from the VM.
4. Send a chat completion request from your local machine:
```bash theme={null}
curl https://llm-.tunnel.applications..nebius.cloud/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3-8b",
"stream": false,
"messages": [{"role": "user", "content": "Say hi"}]
}'
```
The result is a JSON response with the model's answer in the `choices[0].message.content` parameter.
If the response is `failed to connect to local service`, this means that the model hasn't started yet. Wait a few minutes and try again.
## How to delete the created resources
The created VM and its boot disk are chargeable. If you don't need them, [delete the VM](/compute/virtual-machines/delete), so Nebius AI Cloud doesn't charge for the resources. The boot disk is deleted automatically when you delete the VM.
You can also delete free resources:
* [Tunnel](/tunnels/delete)
* [Service account](/iam/service-accounts/manage#deleting-a-service-account)
* [Custom group](/iam/authorization/groups/delete-groups)