Skip to main content
You can use nebius storage v1alpha1 transfer commands of the Nebius AI Cloud CLI to move data to your Object Storage buckets. The commands support moving data in the following directions:
  • From other storage providers to Nebius AI Cloud. The commands support S3-compatible storage services and Azure Blob Storage.
  • From one bucket in Nebius AI Cloud to another. For example, between buckets in different regions.

​
Background

Each data transfer consists of consecutive iterations. At each iteration, Object Storage performs the following steps:
  1. Makes a ListObjects request for every 1000 objects in the source bucket.
  2. Makes HeadObject requests to both the source and destination buckets for every listed object. After the first successful iteration, objects with last modified timestamps before the last successful iteration are skipped to reduce synchronization costs.
  3. For every object that needs to be transferred (depending on your overwrite strategy):
    • If the object is less than 100 MB, makes a single GetObject request to the source bucket and a single PutObject request to the destination bucket.
    • For larger objects, makes the following requests:
      1. CreateMultipartUpload request to the destination bucket.
      2. GetObject requests to the source bucket and UploadPart requests to the destination bucket for every approximately 50 MB of the object.
      3. CompleteMultipartUpload request to destination bucket after uploading all parts.
    The same logic applies to blobs in Azure Blob Storage containers, regardless of the type of blob.
  4. Checks whether the stop condition is satisfied:
    • If it is, the transfer is complete.
    • If it is not, the next iteration starts after the inter-iteration interval.
For versioned buckets, only the current object version is transferred. When objects are restored—for example, by removing delete markers or the current version—they retain their original last modified timestamp. If this timestamp is earlier than the start of the last successful iteration, the object is not transferred during synchronization. If data is transferred from another S3-compatible storage provider, verify the consistency of object metadata before starting the transfer. The Object Storage relies on the source provider to assign accurate last modified timestamps. If the provider sets timestamps incorrectly—for example, to dates in the past during object creation—such objects may be skipped during synchronization, even if they are new or updated.

​
Costs

The cost of a data transfer is made up of the costs for the source and destination buckets. The nebius storage v1alpha1 transfer commands themselves do not incur additional costs.

​
Source bucket costs

If your source bucket is at another storage provider, it may charge you for egress traffic and requests made during the transfer. For details, see your provider’s documentation. If your source bucket is in Nebius AI Cloud, Object Storage charges you for the following billing items:

​
Destination bucket costs

The destination bucket of a data transfer is always in Nebius AI Cloud. Object Storage charges you for the following billing items:
  • HTTP requests on data in the destination bucket, as described in Background: class A (PUT, POST) and class B (HEAD) operations.
  • Storing data in the destination bucket. Charges for each individual object or its part (for multipart uploads) starts once it is uploaded to the destination bucket.

​
Prerequisites

  1. Make sure you are in a group that has at least the editor role within your tenant; for example, the default editors group. You can check this in the Administration → IAM section of the web console.
  2. Create a destination bucket in Nebius AI Cloud.
  3. Install and configure the Nebius AI Cloud CLI.
  4. Get the ID of the project that contains the destination bucket:
    In the sidebar, go to https://mintcdn.com/nebius-ai-cloud/1Ha0sWR6e1mnIaHS/_assets/sidebar/storage.svg?fit=max&auto=format&n=1Ha0sWR6e1mnIaHS&q=85&s=0a2dad6b48aea10e85f6f3e2343aee26 Storage → Object Storage and then find the destination bucket in the list:
    • If the bucket is in the list, open the project menu and then click https://mintcdn.com/nebius-ai-cloud/1Ha0sWR6e1mnIaHS/_assets/button-vellipsis.svg?fit=max&auto=format&n=1Ha0sWR6e1mnIaHS&q=85&s=e80b8e57c43bfd117679262e6a1334ad → Copy project ID next to the selected project.
    • If the bucket is not in the list, open the project menu and then select another project to find the bucket in.
  5. Set up access to the buckets:
    • For the destination bucket, create access keys for a user or service account from a group that has at least the editor role within your tenant; for example, the default editors group. This grants the account permissions to make HEAD and PUT requests to the bucket.
    • For the source bucket, create access keys that grant permissions to make GET, HEAD and LIST requests:
      • If your source bucket is in Nebius AI Cloud, you can create access keys for a user or service account from a group that has at least the viewer role within your tenant; for example, the default viewers group.
      • If your source bucket is in Azure Blob Storage, you can create access keys for an Azure storage account that has at least the Storage Blob Data Reader role.

​
Steps

​
Create the transfer configuration

Create a transfer.json file with the following contents and change values in it: 
{
  "metadata": {
    "name": "<transfer_name>"
  },
  "spec": {
    "source": {
      "endpoint": "<source-s3-endpoint, e.g. https://storage.us-central1.nebius.cloud>",
      "region": "<source-s3-region, e.g. us-central1>",
      "bucket_name": "<source-bucket-name>",
      "credentials": {
        "access_key": {
          "access_key_id": "<source-s3-access-key-id>",
          "secret_access_key": "<source-s3-secret-access-key>",
        },
        // Or, for Azure Blob Storage:
        "azure_access_key": {
          "account_name": "<source-azure-storage-account-name>",
          "access_key": "<source-azure-access-key>",
        },
        // Or, for anonymous access:
        "anonymous": {}
      }
    },
    "destination": {
      "bucket_name": "<destination-bucket-name, must be in the same region as parent_id project>",
      "credentials": {
        "access_key": {
          "access_key_id": "<destination-s3-access-key-id>",
          "secret_access_key": "<destination-s3-secret-access-key>"
        },
        // Or, for anonymous access:
        "anonymous": {}
      }
    },
    "inter_iteration_interval": "<value, e.g. 2h30m10s, default is 15m>",
    "infinite": {}, // Or some other stop condition.
    "overwrite_strategy": "<NEVER | IF_NEWER | ALWAYS>"
  }
}
For details about some of the fields, see the following subsections.

​
Access to source containers in Azure Blob Storage

If your source bucket (container) is in Azure Blob Storage, .spec.source should look like this: 
{
  "spec": {
    "source": {
      "endpoint": "<azure-storage-account-endpoint, e.g. https://<storage-account-name>.blob.core.windows.net>",
      "bucket_name": "<azure-container-name>",
      "credentials": {
        "azure_access_key": {
          "account_name": "<azure-storage-account-name>",
          "access_key": "<azure-access-key>"
        }
      }
    }
  }
}
  • .spec.source.endpoint: the endpoint of the Azure storage account in the format https://<storage-account-name>.blob.core.windows.net>.
  • .spec.source.bucket_name: the name of the source container in Azure Blob Storage.
  • .spec.source.credentials.azure_access_key.account_name: the name of the Azure storage account. Not to be confused with the name of your Azure account.
  • .spec.source.credentials.azure_access_key.access_key: the access key that the storage account uses for authentication.

​
Anonymous access

If a bucket has anonymous access enabled, you can pass {"anonymous": {}} as the value for .spec.source.credentials or .spec.destination.credentials. For example: 
{
  "spec": {
    "source": {
      "credentials": {
        "anonymous": {}
      }
    }
  }
}
Here, “anonymous access” means that unauthenticated users have permissions to make the following requests, as described in Background:
  • Source bucket: ListObjects, HeadObject and GetObject.
  • Destination bucket: HeadObject, PutObject, CreateMultipartUpload, UploadPart and CompleteMultipartUpload.

​
Stop condition

After each iteration, the transfer will evaluate its stopping condition to decide if it should stop. The example above uses the .spec.infinite condition. The configuration must include exactly one of the following stop conditions:
  • .spec.after_one_iteration: the transfer stops after completing its first iteration. The value should be an empty object. For example: 
    {
  "spec": {
    "after_one_iteration": {}
  }
}
  • .spec.after_n_empty_iterations: the transfer stops after a number of consecutive empty iterations (.spec.after_n_empty_iterations.empty_iterations_threshold). For example: 
    {
  "spec": {
    "after_n_empty_iterations": {
      "empty_iterations_threshold": 5
    }
  }
}
  • .spec.infinite: the transfer continues indefinitely until manually stopped. For example: 
    {
  "spec": { 
    "infinite": {}
  }
}

​
Overwrite strategy

The overwrite strategy determines how the transfer handles objects already present in the destination bucket. The configuration must include .spec.overwrite_strategy with one of the following values:
  • NEVER: the transfer does not overwrite objects that exist in the destination bucket. If an object with the same name already exists, it is skipped. This is the safest option to prevent accidental data loss.
  • IF_NEWER: the transfer only overwrites an object if the source object has a newer timestamp (based on the Last-Modified header) compared to the existing object in the destination bucket. This strategy is recommended for incremental synchronization scenarios.
  • ALWAYS: the transfer always overwrites objects in the destination bucket. Use this option with caution since it may lead to data loss in the destination bucket. It is suitable when you want to fully rewrite data in destination bucket. After the first successful iteration, the transfer only checks objects modified since the last iteration. This means that objects manually added to the destination bucket will not be overwritten by older source versions, because those source objects are not processed unless they were modified after the last iteration.

​
Configuration examples

​
Transfer between regions (eu-north1 → us-central1)

Sample setup for transferring data from the source_bucket in the eu-north1 region to the destination_bucket in the us-central1 region. Requires a project in the us-central1 (for example, project-u00xxxx). 
{
  "metadata": {
    "parent_id": "project-u00xxxx",
    "name": "eu-north1-to-us-central1-transfer"
  },
  "spec": {
    "source": {
      "endpoint": "https://storage.eu-north1.nebius.cloud",
      "region": "eu-north1",
      "bucket_name": "source_bucket",
      "credentials": {
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "destination": {
      "bucket_name": "destination_bucket",
      "credentials": {
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "after_one_iteration": {},
    "overwrite_strategy": "IF_NEWER"
  }
}

​
Cross-region backup (us-central1 → eu-north1)

Sample setup for continuous backup from the bucket in the us-central1 region to the backup_bucket in the eu-north1 region. Requires a project in the eu-north1 (for example, project-e00xxxx). 
{
  "metadata": {
    "parent_id": "project-e00xxxx",
    "name": "us-central1-to-eu-north1-backup"
  },
  "spec": {
    "source": {
      "endpoint": "https://storage.us-central1.nebius.cloud",
      "region": "us-central1",
      "bucket_name": "bucket",
      "credentials": {
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "destination": {
      "bucket_name": "backup_bucket",
      "credentials": {
        "access_key": {
          "access_key_id": "****",
          "secret_access_key": "****"
        }
      }
    },
    "infinite": {},
    "overwrite_strategy": "IF_NEWER"
  }
}

​
Launch the transfer

Run the following command: 
nebius --parent-id <project_id> \
  storage v1alpha1 transfer create --file transfer.json
In --parent-id, specify the ID of the project that contains the destination bucket. For details about getting the ID, see Prerequisites. You can also create the transfer in another project in the same Nebius AI Cloud region as the destination bucket, but not in a project in another region.
If you get an error about the source bucket endpoint not being in the allowlist, contact support.

​
Manage the transfer

You can use the nebius storage v1alpha1 transfer update command to update certain parameters of a transfer, such as the name, inter-iteration interval, stop condition, overwrite condition, and credentials for both the source and destination buckets. Changes to configurations may require several minutes to take effect, during which the old settings could still be in use temporarily.
You cannot change the names or endpoints of the source and destination buckets of a transfer.
To stop the transfer, run the following command: 
nebius storage v1alpha1 transfer stop --id <transfer_ID>
When transfer is in stopped or failed state, you can resume or delete it:
  • To resume the transfer, run the following command: 
    nebius storage v1alpha1 transfer resume --id <transfer_ID>
  • To delete the transfer, run the following command: 
    nebius storage v1alpha1 transfer delete --id <transfer_ID>

​
Limits

You can control transfer limits by setting request rate and bandwidth limits in the spec.source.limiters section of the configuration. These values are not guaranteed throughput targets but maximum thresholds that the transfer will not exceed. This is useful in the following cases:
  • Your source provider imposes rate or bandwidth restrictions.
  • You need to reserve request capacity and bandwidth for other processes.
Example configuration that limits the transfer rate to 1000 requests per second and 1 GiB/s: 
{
  ...
  "spec": {
    "source": {
      "limiters": {
        "requests_per_second": 1000,
        "bandwidth_bytes_per_second": 1073741824
      },
      ...
    },
    ...
  }
}

​
Incompleted multipart uploads

In rare cases, Object Storage might leave incomplete multipart uploads in the destination bucket. To prevent this, set a lifecycle rule on the destination bucket to automatically abort incomplete multipart uploads. For example, you can configure this rule by using the CLI: 
nebius storage bucket update --id <destination_bucket_ID> \
  --lifecycle-configuration-rules '[{"abort_incomplete_multipart_upload": {"days_after_initiation": 7}, "id": "abort_incomplete_multipart_upload", "status": "ENABLED"}]'

​
Check transfer progress

To monitor your current transfer iteration status, check the status.last_iteration field of the transfer:
  • status.last_iteration.objects_discovered_count: Number of objects identified for processing in current iteration. First iteration: All objects in the bucket. Subsequent iterations: Only objects created/modified since the last successful iteration start. Increases as new objects are discovered.
  • status.last_iteration.objects_migrated_count: Objects successfully transferred.
  • status.last_iteration.objects_skipped_count: Objects skipped due to overwrite strategy or because they were already transferred.
If the transfer fails, check the status.error for an endpoint where the error occurred (source or destination), the error code and message.