Prerequisites
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
-
Make sure you are in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. - In the web console, select the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service).
-
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
- Install and configure the Nebius AI Cloud CLI.
-
Make sure that you or the service account that you use is in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. -
Get the ID of the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service):
nebius storage bucket get <bucket_ID> --format jsonpath='{.metadata.parent_id}' -
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
- Install and configure the Nebius AI Cloud provider for Terraform.
-
Make sure that you or the service account that you use is in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. -
Save to a local variable the ID of the project that contains the destination bucket (if it’s in Nebius AI Cloud) or the source bucket (if the destination is in an external storage service):
data "nebius_storage_v1_bucket" "nebius_bucket" { id = "<source_or_destination_bucket_ID>" } locals { transfer_project_id = data.nebius_storage_v1_bucket.nebius_bucket.parent_id } -
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
- Install and initialize the Nebius SDK for Go.
-
Make sure that you or the service account that you use is in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. -
Get the project ID:
In the code, set
bucketSvc := sdk.Services().Storage().V1().Bucket() bucket, err := bucketSvc.Get( ctx, &storage.GetBucketRequest{ Id: "<bucket_ID>", }, ) if err != nil { return err } fmt.Println(bucket.GetMetadata().GetParentId())Idto the bucket ID. The code prints the project ID. -
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
- Install and initialize the Nebius SDK for Python.
-
Make sure that you or the service account that you use is in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. -
Get the project ID:
In the code, set
bucket_service = BucketServiceClient(sdk) bucket = await bucket_service.get( GetBucketRequest(id="<bucket_ID>"), ) print(bucket.metadata.parent_id)idto the bucket ID. The code prints the project ID. -
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
- Install and initialize the Nebius SDK for JavaScript.
-
Make sure that you or the service account that you use is in a group that has at least the
editorrole within your tenant or project; for example, the defaulteditorsgroup. You can check this in the Administration → IAM section of the web console. -
Get the project ID:
In the code, set
const bucketService = new BucketService(sdk); const bucket = await bucketService.get( GetBucketRequest.create({ id: "<bucket_ID>" }), ).result; console.log(bucket.metadata.parentId);idto the bucket ID. The code prints the project ID. -
Make sure that your account has access to the buckets. Prepare credentials for the buckets, for example, access and secret keys. For more information, see Access to buckets for data transfers.
To transfer data from S3-compatible storage to a shared filesystem, create a filesystem bucket attached to the filesystem first. Then, use the filesystem bucket as the destination bucket in the data transfer.
How to launch a data transfer
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
-
In the sidebar, go to
Storage → Data Transfer and then click
Create data transfer.
-
Configure the data transfer:
- Specify its name.
- Select the data transfer type: one-time transfer or replication.
- Configure the source and destination buckets:
- Specify the path to the buckets. In the Bucket field, you can optionally add object key prefixes to the bucket name. In the source, you filter objects. In the destination, you overwrite prefixes of the source objects. Filesystem buckets are listed in the Bucket field together with other buckets.
- Set credentials.
- Configure how the data transfer handles destination objects. See the settings under Overwriting destination objects, Deleting destination objects, and Unmanaged objects.
-
Configure limits on request rate and bandwidth. If you don’t set them or set them to zero, the default values apply:
- Request rate: 4000 requests per second.
- Bandwidth: 50 /s.
- Click Create transfer.
-
Create a
transfer.jsonfile with the data transfer configuration and replace values in the file. Use one of the files below, depending on the source and destination storage services:For details, see:Nebius AI Cloud to Nebius AI Cloud
transfer.json{ "metadata": { "name": "<transfer_name>", "parent_id": "<project_ID>" }, "spec": { "source": { "nebius": { "region": "<Nebius_AI_Cloud_region>", "bucket_name": "<bucket_name>", // Either: "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" }, // Or, for anonymous access: // "anonymous": {} }, "prefix": "<source_object_key_prefix>" // Optional }, "destination": { "nebius": { "region": "<Nebius_AI_Cloud_region>", "bucket_name": "<bucket_name>", "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" } }, "prefix": "<destination_object_key_prefix>" // Optional }, // Either: "after_one_iteration": {}, // Or: // "after_n_empty_iterations": { // "empty_iterations_threshold": <number_of_empty_iterations> // }, // Or: // "infinite": {}, // // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s" "overwrite_strategy": "<NEVER|IF_NEWER>", "enable_deletes_in_destination": true|false, "touch_unmanaged": true|false, // Optional "limiters": { "requests_per_second": <request_rate_limit>, "bandwidth_bytes_per_second": <bandwidth_limit> } } }Nebius AI Cloud to another storage service
transfer.json{ "metadata": { "name": "<transfer_name>", "parent_id": "<project_ID>" }, "spec": { "source": { "nebius": { "region": "<Nebius_AI_Cloud_region>", "bucket_name": "<bucket_name>", // Either: "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" }, // Or, for anonymous access: // "anonymous": {} }, "prefix": "<source_object_key_prefix>" // Optional }, "destination": { "s3_compatible": { "endpoint": "<storage_service_endpoint>", "region": "<bucket_region>", "bucket_name": "<bucket_name>", // Either: "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" }, // Or, for anonymous access: // "anonymous": {} }, "prefix": "<destination_object_key_prefix>" // Optional }, // Either: "after_one_iteration": {}, // Or: // "after_n_empty_iterations": { // "empty_iterations_threshold": <number_of_empty_iterations> // }, // Or: // "infinite": {}, // // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s" "overwrite_strategy": "<NEVER|IF_NEWER>", "enable_deletes_in_destination": true|false, "touch_unmanaged": true|false, // Optional "limiters": { "requests_per_second": <request_rate_limit>, "bandwidth_bytes_per_second": <bandwidth_limit> } } }Azure Blob Storage to Nebius AI Cloud
transfer.json{ "metadata": { "name": "<transfer_name>", "parent_id": "<project_ID>" }, "spec": { "source": { "azure_blob_storage": { "endpoint": "<Azure_Blob_Storage_endpoint>", "container_name": "<container_name>", // Either: "azure_storage_account": { "account_name": "<Azure_storage_account_name>", "access_key": "<access_key>" }, // Or, for anonymous access: // "anonymous": {} }, "prefix": "<source_object_key_prefix>" // Optional }, "destination": { "nebius": { "region": "<Nebius_AI_Cloud_region>", "bucket_name": "<bucket_name>", "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" } }, "prefix": "<destination_object_key_prefix>" // Optional }, // Either: "after_one_iteration": {}, // Or: // "after_n_empty_iterations": { // "empty_iterations_threshold": <number_of_empty_iterations> // }, // Or: // "infinite": {}, // // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s" "overwrite_strategy": "<NEVER|IF_NEWER>", "enable_deletes_in_destination": true|false, "touch_unmanaged": true|false, // Optional "limiters": { "requests_per_second": <request_rate_limit>, "bandwidth_bytes_per_second": <bandwidth_limit> } } }Another storage service to Nebius AI Cloud
transfer.json{ "metadata": { "name": "<transfer_name>", "parent_id": "<project_ID>" }, "spec": { "source": { "s3_compatible": { "endpoint": "<storage_service_endpoint>", "region": "<bucket_region>", "bucket_name": "<bucket_name>", // Either: "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" }, // Or, for anonymous access: // "anonymous": {} }, "prefix": "<source_object_key_prefix>" // Optional }, "destination": { "nebius": { "region": "<Nebius_AI_Cloud_region>", "bucket_name": "<bucket_name>", "access_key": { "access_key_id": "<access_key>", "secret_access_key": "<secret_key>" } }, "prefix": "<destination_object_key_prefix>" // Optional }, // Either: "after_one_iteration": {}, // Or: // "after_n_empty_iterations": { // "empty_iterations_threshold": <number_of_empty_iterations> // }, // Or: // "infinite": {}, // // "inter_iteration_interval": "<HhMmSs>", // For example, "1h20m40s" "overwrite_strategy": "<NEVER|IF_NEWER>", "enable_deletes_in_destination": true|false, "touch_unmanaged": true|false, // Optional "limiters": { "requests_per_second": <request_rate_limit>, "bandwidth_bytes_per_second": <bandwidth_limit> } } }- Examples
- Reference for the
nebius storage transfer createCLI command - Transfer configuration
-
Launch the data transfer:
nebius storage transfer create \ --parent-id <project_ID> \ --file transfer.json
-
Add a data transfer block to your Terraform configuration.
We recommend using Terraform ephemeral resource blocks and write-only
sensitivefields when passing access credentials for buckets. You can also use regular fields, but the credential values are then stored in Terraform plan and state files, which is less secure. The following examples show how to launch a data transfer for different sources and destinations:Nebius AI Cloud to Nebius AI Cloud
ephemeral "nebius_iam_v2_access_key_secret" "ak_secret" { aws_access_key_id = "<access_key>" } resource "nebius_storage_v1_transfer" "my_transfer" { name = "<transfer_name>" parent_id = local.transfer_project_id source { nebius { region = "<Nebius_AI_Cloud_region>" bucket_name = "<bucket_name>" # Either: access_key { access_key_id = "<access_key>" } # Or, for anonymous access: # anonymous {} } # Optional: prefix = "<source_object_key_prefix>" } destination { nebius { region = "<Nebius_AI_Cloud_region>" bucket_name = "<bucket_name>" access_key { access_key_id = "<access_key>" } } # Optional: prefix = "<destination_object_key_prefix>" } sensitive = { source = { nebius = { access_key = { secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret } } } destination = { nebius = { access_key = { secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret } } } } # Either: after_one_iteration {} # Or: # after_n_empty_iterations { # empty_iterations_threshold = <number_of_empty_iterations> # } # Or: # infinite {} # # inter_iteration_interval = "<HhMmSs>" # Optional. For example, "1h20m40s" overwrite_strategy = "NEVER" # Or "IF_NEWER" enable_deletes_in_destination = false touch_unmanaged = false # Optional limiters { requests_per_second = <request_rate_limit> bandwidth_bytes_per_second = <bandwidth_limit> } }Nebius AI Cloud to another storage service
ephemeral "nebius_mysterybox_v1_secret_payload_entry" "transfer_secret" { secret_id = "<mysterybox_secret_ID>" key = "<data_key_of_secret>" } resource "nebius_storage_v1_transfer" "my_transfer" { name = "<transfer_name>" parent_id = local.transfer_project_id source { nebius { region = "<Nebius_AI_Cloud_region>" bucket_name = "<bucket_name>" # Either: access_key { access_key_id = "<source_access_key_ID>" } # Or, for anonymous access: # anonymous {} } # Optional: prefix = "<source_object_key_prefix>" } destination { s3_compatible { endpoint = "<storage_service_endpoint>" region = "<bucket_region>" bucket_name = "<bucket_name>" # Either: access_key { access_key_id = "<destination_access_key_ID>" } # Or, for anonymous access: # anonymous {} } # Optional: prefix = "<destination_object_key_prefix>" } sensitive = { source = { nebius = { access_key = { secret_access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } destination = { s3_compatible = { access_key = { secret_access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } } # Either: after_one_iteration {} # Or: # after_n_empty_iterations { # empty_iterations_threshold = <number_of_empty_iterations> # } # Or: # infinite {} # # inter_iteration_interval = "<HhMmSs>" # Optional. For example, "1h20m40s" overwrite_strategy = "NEVER" # Or "IF_NEWER" enable_deletes_in_destination = false touch_unmanaged = false # Optional limiters { requests_per_second = <request_rate_limit> bandwidth_bytes_per_second = <bandwidth_limit> } }Azure Blob Storage to Nebius AI Cloud
ephemeral "nebius_mysterybox_v1_secret_payload_entry" "transfer_secret" { secret_id = "<mysterybox_secret_ID>" key = "<data_key_of_secret>" } resource "nebius_storage_v1_transfer" "my_transfer" { name = "<transfer_name>" parent_id = local.transfer_project_id source { azure_blob_storage { endpoint = "<Azure_Blob_Storage_endpoint>" container_name = "<container_name>" # Either: azure_storage_account { account_name = "<Azure_storage_account_name>" } # Or, for anonymous access: # anonymous {} } # Optional: prefix = "<source_object_key_prefix>" } destination { nebius { region = "<Nebius_AI_Cloud_region>" bucket_name = "<bucket_name>" access_key { access_key_id = "<destination_access_key_ID>" } } # Optional: prefix = "<destination_object_key_prefix>" } sensitive = { source = { azure_blob_storage = { azure_storage_account = { access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } destination = { nebius = { access_key = { secret_access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } } # Either: after_one_iteration {} # Or: # after_n_empty_iterations { # empty_iterations_threshold = <number_of_empty_iterations> # } # Or: # infinite {} # # inter_iteration_interval = "<HhMmSs>" # Optional. For example, "1h20m40s" overwrite_strategy = "NEVER" # Or "IF_NEWER" enable_deletes_in_destination = false touch_unmanaged = false # Optional limiters { requests_per_second = <request_rate_limit> bandwidth_bytes_per_second = <bandwidth_limit> } }Another storage service to Nebius AI Cloud
ephemeral "nebius_mysterybox_v1_secret_payload_entry" "transfer_secret" { secret_id = "<mysterybox_secret_ID>" key = "<data_key_of_secret>" } resource "nebius_storage_v1_transfer" "my_transfer" { name = "<transfer_name>" parent_id = local.transfer_project_id source { s3_compatible { endpoint = "<storage_service_endpoint>" region = "<bucket_region>" bucket_name = "<bucket_name>" # Either: access_key { access_key_id = "<source_access_key_ID>" } # Or, for anonymous access: # anonymous {} } # Optional: prefix = "<source_object_key_prefix>" } destination { nebius { region = "<Nebius_AI_Cloud_region>" bucket_name = "<bucket_name>" access_key { access_key_id = "<destination_access_key_ID>" } } # Optional: prefix = "<destination_object_key_prefix>" } sensitive = { source = { s3_compatible = { access_key = { secret_access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } destination = { nebius = { access_key = { secret_access_key = ephemeral.nebius_mysterybox_v1_secret_payload_entry.transfer_secret.data.string_value } } } } # Either: after_one_iteration {} # Or: # after_n_empty_iterations { # empty_iterations_threshold = <number_of_empty_iterations> # } # Or: # infinite {} # # inter_iteration_interval = "<HhMmSs>" # Optional. For example, "1h20m40s" overwrite_strategy = "NEVER" # Or "IF_NEWER" enable_deletes_in_destination = false touch_unmanaged = false # Optional limiters { requests_per_second = <request_rate_limit> bandwidth_bytes_per_second = <bandwidth_limit> } }For details, see:The examples use the same secret for the source and destination. If needed, provide different secrets for each side.- Examples
- Reference for the
nebius_storage_v1_transferresource - Transfer configuration
-
Check that the configuration is correct:
terraform validate -
Apply the changes:
terraform apply
Create a one-time transfer between two Nebius AI Cloud buckets:In the code, set the following parameters:
srcAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "<access_key>",
SecretAccessKey: "<secret_key>",
}
srcCredentials :=
&storage.TransferSource_NebiusProvider_AccessKey{
AccessKey: srcAccessKey,
}
dstAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "<access_key>",
SecretAccessKey: "<secret_key>",
}
dstCredentials :=
&storage.TransferDestination_NebiusProvider_AccessKey{
AccessKey: dstAccessKey,
}
stopCondition := &storage.TransferSpec_AfterOneIteration{
AfterOneIteration:
&storage.TransferSpec_StopConditionAfterOneIteration{},
}
transferSvc := sdk.Services().Storage().V1().Transfer()
transferOp, err := transferSvc.Create(
ctx,
&storage.CreateTransferRequest{
Metadata: &common.ResourceMetadata{
Name: "launch-transfer-test",
ParentId: "<project_ID>",
},
Spec: &storage.TransferSpec{
Source: &storage.TransferSource{
Provider: &storage.TransferSource_Nebius{
Nebius: &storage.TransferSource_NebiusProvider{
Region: "eu-north1",
BucketName: "<source_bucket_name>",
Credentials: srcCredentials,
},
},
},
Destination: &storage.TransferDestination{
Provider:
&storage.TransferDestination_Nebius{
Nebius:
&storage.TransferDestination_NebiusProvider{
Region: "eu-north1",
BucketName: "<destination_bucket_name>",
Credentials: dstCredentials,
},
},
},
StopCondition: stopCondition,
OverwriteStrategy: storage.TransferSpec_NEVER,
},
},
)
if err != nil {
return err
}
if _, err = transferOp.Wait(ctx); err != nil {
return err
}
AccessKeyId: Access key ID for the source bucket and for the destination bucket.SecretAccessKey: Secret access key for the source bucket and for the destination bucket.Metadata.ParentId: Project ID.BucketNamein the source: Source bucket name.BucketNamein the destination: Destination bucket name.
launch-transfer-test, uses the eu-north1 region for both buckets, stops after one iteration, and sets OverwriteStrategy to TransferSpec_NEVER.Create a one-time transfer between two Nebius AI Cloud buckets:In the code, set the following parameters:
transfer_service = TransferServiceClient(sdk)
stop_condition = TransferSpec__StopConditionAfterOneIteration()
transfer_operation = await transfer_service.create(
CreateTransferRequest(
metadata=ResourceMetadata(
name="launch-transfer-test",
parent_id="<project_ID>",
),
spec=TransferSpec(
source=TransferSource(
nebius=TransferSource__NebiusProvider(
region="eu-north1",
bucket_name="<source_bucket_name>",
access_key=TransferCredentialsAccessKey(
access_key_id="<access_key>",
secret_access_key="<secret_key>",
),
),
),
destination=TransferDestination(
nebius=TransferDestination__NebiusProvider(
region="eu-north1",
bucket_name="<destination_bucket_name>",
access_key=TransferCredentialsAccessKey(
access_key_id="<access_key>",
secret_access_key="<secret_key>",
),
),
),
after_one_iteration=stop_condition,
overwrite_strategy=TransferSpec.OverwriteStrategy.NEVER,
),
),
)
await transfer_operation.wait()
access_key_id: Access key ID for the source bucket and for the destination bucket.secret_access_key: Secret access key for the source bucket and for the destination bucket.metadata.parent_id: Project ID.bucket_namein the source: Source bucket name.bucket_namein the destination: Destination bucket name.
launch-transfer-test, uses the eu-north1 region for both buckets, sets after_one_iteration, and sets overwrite_strategy to TransferSpec.OverwriteStrategy.NEVER.Create a one-time transfer between two Nebius AI Cloud buckets:In the code, set the following parameters:
const transferService = new TransferService(sdk);
const transferOperation = await transferService.create(
CreateTransferRequest.create({
metadata: ResourceMetadata.create({
name: "launch-transfer-test",
parentId: "<project_ID>",
}),
spec: TransferSpec.create({
source: TransferSource.create({
provider: {
$case: "nebius",
nebius: TransferSource_NebiusProvider.create({
region: "eu-north1",
bucketName: "<source_bucket_name>",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "<access_key>",
secretAccessKey: "<secret_key>",
}),
},
}),
},
}),
destination: TransferDestination.create({
provider: {
$case: "nebius",
nebius: TransferDestination_NebiusProvider.create({
region: "eu-north1",
bucketName: "<destination_bucket_name>",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "<access_key>",
secretAccessKey: "<secret_key>",
}),
},
}),
},
}),
stopCondition: {
$case: "afterOneIteration",
afterOneIteration:
TransferSpec_StopConditionAfterOneIteration.create(),
},
overwriteStrategy: TransferSpec_OverwriteStrategy.NEVER,
}),
}),
).result;
await transferOperation.wait();
accessKeyId: Access key ID for the source bucket and for the destination bucket.secretAccessKey: Secret access key for the source bucket and for the destination bucket.metadata.parentId: Project ID.bucketNamein the source: Source bucket name.bucketNamein the destination: Destination bucket name.
launch-transfer-test, uses the eu-north1 region for both buckets, sets stopCondition to afterOneIteration, and sets overwriteStrategy to TransferSpec_OverwriteStrategy.NEVER.Examples
One-time transfer between Nebius AI Cloud regions
The following configuration examples show how to create a one-time transfer (a transfer that stops after one iteration) from a bucket in theeu-north1 region to a bucket in the us-central1 region:
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
Create the data transfer in the project in
us-central1 where the destination bucket is located:- Type: One-time transfer
-
Source: Nebius
- Region:
eu-north1 - Bucket name (example):
source_bucket - Credentials: Access and secret keys
- Region:
-
Destination: Nebius
- Region:
us-central1 - Bucket name (example):
destination_bucket - Credentials: Access and secret keys
- Overwriting destination objects: Overwrite if source is newer
- Deleting destination objects: Do not delete
- Region:
The
.metadata.parent_id parameter should be the ID of the project in us-central1 that contains the destination bucket.{
"metadata": {
"name": "eu-north1-to-us-central1-transfer",
"parent_id": "project-***"
},
"spec": {
"source": {
"nebius": {
"region": "eu-north1",
"bucket_name": "source_bucket",
"access_key": {
"access_key_id": "****",
"secret_access_key": "****"
}
}
},
"destination": {
"nebius": {
"region": "us-central1",
"bucket_name": "destination_bucket",
"access_key": {
"access_key_id": "****",
"secret_access_key": "****"
}
}
},
"after_one_iteration": {},
"overwrite_strategy": "IF_NEWER",
"enable_deletes_in_destination": false
}
}
The
parent_id parameter should be the ID of the project in us-central1 that contains the destination bucket.ephemeral "nebius_iam_v2_access_key_secret" "ak_secret" {
aws_access_key_id = "****"
}
resource "nebius_storage_v1_transfer" "my_transfer" {
name = "eu-north1-to-us-central1-transfer"
parent_id = "project-***"
source {
nebius {
region = "eu-north1"
bucket_name = "source_bucket"
access_key {
access_key_id = "****"
}
}
}
destination {
nebius {
region = "us-central1"
bucket_name = "destination_bucket"
access_key {
access_key_id = "****"
}
}
}
sensitive = {
source = {
nebius = {
access_key = {
secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret
}
}
}
destination = {
nebius = {
access_key = {
secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret
}
}
}
}
after_one_iteration {}
overwrite_strategy = "IF_NEWER"
enable_deletes_in_destination = false
}
The
Metadata.ParentId parameter should be the ID of the project in us-central1 that contains the destination bucket.srcAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "****",
SecretAccessKey: "****",
}
srcCredentials :=
&storage.TransferSource_NebiusProvider_AccessKey{
AccessKey: srcAccessKey,
}
dstAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "****",
SecretAccessKey: "****",
}
dstCredentials :=
&storage.TransferDestination_NebiusProvider_AccessKey{
AccessKey: dstAccessKey,
}
stopCondition := &storage.TransferSpec_AfterOneIteration{
AfterOneIteration:
&storage.TransferSpec_StopConditionAfterOneIteration{},
}
transferSvc := sdk.Services().Storage().V1().Transfer()
transferOp, err := transferSvc.Create(
ctx,
&storage.CreateTransferRequest{
Metadata: &common.ResourceMetadata{
Name: "eu-north1-to-us-central1-transfer",
ParentId: "project-***",
},
Spec: &storage.TransferSpec{
Source: &storage.TransferSource{
Provider: &storage.TransferSource_Nebius{
Nebius: &storage.TransferSource_NebiusProvider{
Region: "eu-north1",
BucketName: "source_bucket",
Credentials: srcCredentials,
},
},
},
Destination: &storage.TransferDestination{
Provider:
&storage.TransferDestination_Nebius{
Nebius:
&storage.TransferDestination_NebiusProvider{
Region: "us-central1",
BucketName: "destination_bucket",
Credentials: dstCredentials,
},
},
},
StopCondition: stopCondition,
OverwriteStrategy: storage.TransferSpec_IF_NEWER,
EnableDeletesInDestination: false,
},
},
)
if err != nil {
return err
}
if _, err = transferOp.Wait(ctx); err != nil {
return err
}
The
metadata.parent_id parameter should be the ID of the project in us-central1 that contains the destination bucket.transfer_service = TransferServiceClient(sdk)
stop_condition = (
TransferSpec__StopConditionAfterOneIteration()
)
transfer_operation = await transfer_service.create(
CreateTransferRequest(
metadata=ResourceMetadata(
name="eu-north1-to-us-central1-transfer",
parent_id="project-***",
),
spec=TransferSpec(
source=TransferSource(
nebius=TransferSource__NebiusProvider(
region="eu-north1",
bucket_name="source_bucket",
access_key=TransferCredentialsAccessKey(
access_key_id="****",
secret_access_key="****",
),
),
),
destination=TransferDestination(
nebius=TransferDestination__NebiusProvider(
region="us-central1",
bucket_name="destination_bucket",
access_key=TransferCredentialsAccessKey(
access_key_id="****",
secret_access_key="****",
),
),
),
after_one_iteration=stop_condition,
overwrite_strategy=(
TransferSpec.OverwriteStrategy.IF_NEWER
),
enable_deletes_in_destination=False,
),
),
)
await transfer_operation.wait()
The
metadata.parentId parameter should be the ID of the project in us-central1 that contains the destination bucket.const transferService = new TransferService(sdk);
const transferOperation = await transferService.create(
CreateTransferRequest.create({
metadata: ResourceMetadata.create({
name: "eu-north1-to-us-central1-transfer",
parentId: "project-***",
}),
spec: TransferSpec.create({
source: TransferSource.create({
provider: {
$case: "nebius",
nebius: TransferSource_NebiusProvider.create({
region: "eu-north1",
bucketName: "source_bucket",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "****",
secretAccessKey: "****",
}),
},
}),
},
}),
destination: TransferDestination.create({
provider: {
$case: "nebius",
nebius: TransferDestination_NebiusProvider.create({
region: "us-central1",
bucketName: "destination_bucket",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "****",
secretAccessKey: "****",
}),
},
}),
},
}),
stopCondition: {
$case: "afterOneIteration",
afterOneIteration:
TransferSpec_StopConditionAfterOneIteration.create(),
},
overwriteStrategy:
TransferSpec_OverwriteStrategy.IF_NEWER,
enableDeletesInDestination: false,
}),
}),
).result;
await transferOperation.wait();
Replication between Nebius AI Cloud regions
The following configuration examples show how to create a replication (a continuous transfer that you stop manually) from a bucket in theus-central1 region to a bucket in the eu-north1 region:
- Web console
- CLI
- Terraform
- Go SDK
- Python SDK
- JavaScript SDK
Create the data transfer in the project in
eu-north1 where the destination bucket is located:- Type: Replication
-
Source: Nebius
- Region:
us-central1 - Bucket name (example):
primary_bucket - Credentials: Access and secret keys
- Region:
-
Destination: Nebius
- Region:
eu-north1 - Bucket name (example):
secondary_bucket - Credentials: Access and secret keys
- Overwriting destination objects: Overwrite if source is newer
- Deleting destination objects: Delete if not in source
- Region:
The
.metadata.parent_id parameter should be the ID of the project in eu-north1 that contains the destination bucket.{
"metadata": {
"name": "us-central1-to-eu-north1-backup",
"parent_id": "project-***"
},
"spec": {
"source": {
"nebius": {
"region": "us-central1",
"bucket_name": "primary_bucket",
"access_key": {
"access_key_id": "****",
"secret_access_key": "****"
}
}
},
"destination": {
"nebius": {
"region": "eu-north1",
"bucket_name": "secondary_bucket",
"access_key": {
"access_key_id": "****",
"secret_access_key": "****"
}
}
},
"infinite": {},
"overwrite_strategy": "IF_NEWER",
"enable_deletes_in_destination": true
}
}
The
parent_id parameter should be the ID of the project in eu-north1 that contains the destination bucket.ephemeral "nebius_iam_v2_access_key_secret" "ak_secret" {
aws_access_key_id = "****"
}
resource "nebius_storage_v1_transfer" "my_transfer" {
name = "us-central1-to-eu-north1-backup"
parent_id = "project-***"
source {
nebius {
region = "us-central1"
bucket_name = "primary_bucket"
access_key {
access_key_id = "****"
}
}
}
destination {
nebius {
region = "eu-north1"
bucket_name = "secondary_bucket"
access_key {
access_key_id = "****"
}
}
}
sensitive = {
source = {
nebius = {
access_key = {
secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret
}
}
}
destination = {
nebius = {
access_key = {
secret_access_key = ephemeral.nebius_iam_v2_access_key_secret.ak_secret.secret
}
}
}
}
infinite {}
overwrite_strategy = "IF_NEWER"
enable_deletes_in_destination = true
}
The
Metadata.ParentId parameter should be the ID of the project in eu-north1 that contains the destination bucket.srcAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "****",
SecretAccessKey: "****",
}
srcCredentials :=
&storage.TransferSource_NebiusProvider_AccessKey{
AccessKey: srcAccessKey,
}
dstAccessKey := &storage.TransferCredentialsAccessKey{
AccessKeyId: "****",
SecretAccessKey: "****",
}
dstCredentials :=
&storage.TransferDestination_NebiusProvider_AccessKey{
AccessKey: dstAccessKey,
}
stopCondition := &storage.TransferSpec_Infinite{
Infinite: &storage.TransferSpec_StopConditionInfinite{},
}
transferSvc := sdk.Services().Storage().V1().Transfer()
transferOp, err := transferSvc.Create(
ctx,
&storage.CreateTransferRequest{
Metadata: &common.ResourceMetadata{
Name: "us-central1-to-eu-north1-backup",
ParentId: "project-***",
},
Spec: &storage.TransferSpec{
Source: &storage.TransferSource{
Provider: &storage.TransferSource_Nebius{
Nebius: &storage.TransferSource_NebiusProvider{
Region: "us-central1",
BucketName: "primary_bucket",
Credentials: srcCredentials,
},
},
},
Destination: &storage.TransferDestination{
Provider:
&storage.TransferDestination_Nebius{
Nebius:
&storage.TransferDestination_NebiusProvider{
Region: "eu-north1",
BucketName: "secondary_bucket",
Credentials: dstCredentials,
},
},
},
StopCondition: stopCondition,
OverwriteStrategy: storage.TransferSpec_IF_NEWER,
EnableDeletesInDestination: true,
},
},
)
if err != nil {
return err
}
if _, err = transferOp.Wait(ctx); err != nil {
return err
}
The
metadata.parent_id parameter should be the ID of the project in eu-north1 that contains the destination bucket.transfer_service = TransferServiceClient(sdk)
stop_condition = TransferSpec__StopConditionInfinite()
transfer_operation = await transfer_service.create(
CreateTransferRequest(
metadata=ResourceMetadata(
name="us-central1-to-eu-north1-backup",
parent_id="project-***",
),
spec=TransferSpec(
source=TransferSource(
nebius=TransferSource__NebiusProvider(
region="us-central1",
bucket_name="primary_bucket",
access_key=TransferCredentialsAccessKey(
access_key_id="****",
secret_access_key="****",
),
),
),
destination=TransferDestination(
nebius=TransferDestination__NebiusProvider(
region="eu-north1",
bucket_name="secondary_bucket",
access_key=TransferCredentialsAccessKey(
access_key_id="****",
secret_access_key="****",
),
),
),
infinite=stop_condition,
overwrite_strategy=(
TransferSpec.OverwriteStrategy.IF_NEWER
),
enable_deletes_in_destination=True,
),
),
)
await transfer_operation.wait()
The
metadata.parentId parameter should be the ID of the project in eu-north1 that contains the destination bucket.const transferService = new TransferService(sdk);
const transferOperation = await transferService.create(
CreateTransferRequest.create({
metadata: ResourceMetadata.create({
name: "us-central1-to-eu-north1-backup",
parentId: "project-***",
}),
spec: TransferSpec.create({
source: TransferSource.create({
provider: {
$case: "nebius",
nebius: TransferSource_NebiusProvider.create({
region: "us-central1",
bucketName: "primary_bucket",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "****",
secretAccessKey: "****",
}),
},
}),
},
}),
destination: TransferDestination.create({
provider: {
$case: "nebius",
nebius: TransferDestination_NebiusProvider.create({
region: "eu-north1",
bucketName: "secondary_bucket",
credentials: {
$case: "accessKey",
accessKey: TransferCredentialsAccessKey.create({
accessKeyId: "****",
secretAccessKey: "****",
}),
},
}),
},
}),
stopCondition: {
$case: "infinite",
infinite: TransferSpec_StopConditionInfinite.create(),
},
overwriteStrategy:
TransferSpec_OverwriteStrategy.IF_NEWER,
enableDeletesInDestination: true,
}),
}),
).result;
await transferOperation.wait();