Skip to main content

Retrieving the Data Source

This data source can be retrieved by one of ID or name.
Retrieve by ID
To retrieve by ID, fill in only the id field:
data ... {
    id = "your-ID"
}
Retrieve by Name
To retrieve by name, fill in only the name and parent_id fields:
data ... {
    name      = "your name"
    parent_id = "data-source-parent-id"
}
Transfer that migrates data from other providers or across different regions of Nebius Object Storage. Transfer consists of consecutive iterations where the service lists objects in the source bucket and moves those that need to be transferred according to the specified overwrite strategy and touch unmanaged flag value. After an iteration completes, the transfer will stop if its stop condition is met. Otherwise, it will wait for the defined inter-iteration interval before starting the next iteration.

Schema

Optional

  • id (String) Identifier for the resource, unique for its resource type.
  • name (String) Human readable name for the resource.
  • parent_id (String) Identifier of the parent resource to which the resource belongs.

Read-Only

  • after_n_empty_iterations (Attributes) Cannot be set alongside after_one_iteration or infinite. (see below for nested schema)
  • after_one_iteration (Attributes) Cannot be set alongside after_n_empty_iterations or infinite. (see below for nested schema)
  • created_at (String) : Timestamp indicating when the resource was created. A string representing a timestamp in ISO 8601 format: YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SS.SSS±HH:MM
  • destination (Attributes) Destination to which the transfer writes data. (see below for nested schema)
  • infinite (Attributes) : Infinite transfers do not stop automatically and can be stopped manually by the user. Cannot be set alongside after_one_iteration or after_n_empty_iterations. (see below for nested schema)
  • inter_iteration_interval (String) : The time to wait since the previous iteration before starting the next one. Default is 15 minutes if not specified. Duration as a string: possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as 300ms, -1.5h or 2h45m. Valid time units are ns, us (or µs), ms, s, m, h, d.
  • labels (Map of String) : Labels associated with the resource.
  • limiters (Attributes) : Limiters applied to source bucket operations. These limits include all operations required for transfer (HeadObject, GetObject, ListObjects). For details about which operations are used, see public documentation: http://docs.nebius.com/object-storage/transfer/quickstart#background. Note: Since the service performs comparable request volumes and data transfers to both source and destination, these limits will indirectly constrain destination operations as well. (see below for nested schema)
  • metadata (Attributes) :

    Inner value description

    Common resource metadata. (see below for nested schema)
  • overwrite_strategy (String) : Overwrite strategy set logic of overwrite already existed objects in destination bucket.

    Supported values

    Possible values:
    • OVERWRITE_STRATEGY_UNSPECIFIED
    • NEVER: Never overwrite objects that exist in the destination. If object exists in destination bucket, skip it. Safest option to prevent any data loss.
    • IF_NEWER: Overwrite only if source object is newer than destination. Comparison based on Last-Modified timestamp. Recommended for incremental sync scenarios. If touch_unmanaged flag isn’t set, we do not overwrite objects that haven’t been created by Data Transfer service.
  • resource_version (Number) : Version of the resource for safe concurrent modifications and consistent reads. Positive and monotonically increases on each resource spec change (but not on each change of the resource’s container(s) or status). Service allows zero value or current.
  • source (Attributes) Source from which the transfer reads data. (see below for nested schema)
  • status (Attributes) (see below for nested schema)
  • touch_unmanaged (Boolean) : If touch_unmanaged flag is set, service will be allowed to overwrite and delete from destination objects that were not created by Data Transfer Service. If this flag is false, Data Transfer Service will never overwrite or delete objects that haven’t been created by Data Transfer service.
  • updated_at (String) : Timestamp indicating when the resource was last updated. A string representing a timestamp in ISO 8601 format: YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SS.SSS±HH:MM

Nested Schema for after_n_empty_iterations

Read-Only:
  • empty_iterations_threshold (Number) Number of consecutive iterations with zero transferred objects required to stop transfer.

Nested Schema for after_one_iteration

Nested Schema for destination

Read-Only:
  • nebius (Attributes) Cannot be set alongside s3_compatible. (see below for nested schema)
  • prefix (String) : Prefix to add to the beginning of each transferred object key in the destination. During transfer, the resulting object key in the destination is computed by removing source.prefix (if provided) from the original key and then prepending destination.prefix. Important: This transformation may result in an empty object key or one that exceeds allowed length limits. Use prefixes that guarantee valid resulting object keys for your objects after transformation.
  • s3_compatible (Attributes) Cannot be set alongside nebius. (see below for nested schema)

Nested Schema for destination.nebius

Read-Only:
  • access_key (Attributes) (see below for nested schema)
  • bucket_name (String) Name of the destination bucket.
  • region (String) Nebius region where the destination bucket is located.

Nested Schema for destination.nebius.access_key

Read-Only:
  • access_key_id (String) Access key ID.

Nested Schema for destination.s3_compatible

Read-Only:
  • access_key (Attributes) Cannot be set alongside anonymous. (see below for nested schema)
  • anonymous (Attributes) Cannot be set alongside access_key. (see below for nested schema)
  • bucket_name (String) Name of the destination bucket.
  • endpoint (String) : The endpoint must be in the form of a URL, starting with the protocol (https), followed by the endpoint address without a trailing slash. Example: “https://storage.some-cloud”.
  • region (String) S3-compatible provider region where destination bucket is located.

Nested Schema for destination.s3_compatible.access_key

Read-Only:
  • access_key_id (String) Access key ID.

Nested Schema for destination.s3_compatible.anonymous

Nested Schema for infinite

Nested Schema for limiters

Read-Only:
  • bandwidth_bytes_per_second (Number) Maximum bandwidth in bytes per second. If set to zero, default limit will be applied.
  • requests_per_second (Number) Maximum number of requests per second. If set to zero, default limit will be applied.

Nested Schema for metadata

Nested Schema for source

Read-Only:
  • azure_blob_storage (Attributes) Cannot be set alongside nebius or s3_compatible. (see below for nested schema)
  • nebius (Attributes) Cannot be set alongside s3_compatible or azure_blob_storage. (see below for nested schema)
  • prefix (String) : Prefix to filter objects in the source. Only objects whose keys start with this prefix will be transferred. During transfer, the resulting object key in the destination is computed by removing source.prefix from the original key and then prepending destination.prefix (if provided). Important: This transformation may result in an empty object key or one that exceeds allowed length limits. Use prefixes that guarantee valid resulting object keys for your objects after transformation.
  • s3_compatible (Attributes) Cannot be set alongside nebius or azure_blob_storage. (see below for nested schema)

Nested Schema for source.azure_blob_storage

Read-Only:
  • anonymous (Attributes) Cannot be set alongside azure_storage_account. (see below for nested schema)
  • azure_storage_account (Attributes) Cannot be set alongside anonymous. (see below for nested schema)
  • container_name (String) Name of the source Azure Blob Storage container.
  • endpoint (String) : The endpoint must be in the form of a URL, starting with the protocol (https), followed by the endpoint address without a trailing slash. Example: “https://storageaccountname.blob.core.windows.net”.

Nested Schema for source.azure_blob_storage.anonymous

Nested Schema for source.azure_blob_storage.azure_storage_account

Read-Only:
  • account_name (String) Storage account name.

Nested Schema for source.nebius

Read-Only:
  • access_key (Attributes) Cannot be set alongside anonymous. (see below for nested schema)
  • anonymous (Attributes) Cannot be set alongside access_key. (see below for nested schema)
  • bucket_name (String) Name of the source bucket.
  • region (String) Nebius region where the source bucket is located.

Nested Schema for source.nebius.access_key

Read-Only:
  • access_key_id (String) Access key ID.

Nested Schema for source.nebius.anonymous

Nested Schema for source.s3_compatible

Read-Only:
  • access_key (Attributes) Cannot be set alongside anonymous. (see below for nested schema)
  • anonymous (Attributes) Cannot be set alongside access_key. (see below for nested schema)
  • bucket_name (String) Name of the source bucket.
  • endpoint (String) : The endpoint must be in the form of a URL, starting with the protocol (https), followed by the endpoint address without a trailing slash. Example: “https://storage.some-cloud”.
  • region (String) S3-compatible provider region where source bucket is located.

Nested Schema for source.s3_compatible.access_key

Read-Only:
  • access_key_id (String) Access key ID.

Nested Schema for source.s3_compatible.anonymous

Nested Schema for status

Read-Only:
  • error (Attributes) Error information if the transfer has failed. (see below for nested schema)
  • last_iteration (Attributes) : If there is active iteration, its status will be given. If it is a new transfer that doesn’t have iteration yet, nothing will be returned. Otherwise, last finished transaction will be returned. (see below for nested schema)
  • state (String) : Current transfer state.

    Supported values

    Transfer state. Possible values:
    • STATE_UNSPECIFIED
    • ACTIVE
    • STOPPING
    • STOPPED
    • FAILING
    • FAILED
    • DELETING
  • suspension_state (String) : If the transfer is suspended, transfer’s suspension state becomes SUSPENDED.

    Supported values

    Transfer suspension state. Possible values:
    • SUSPENSION_STATE_UNSPECIFIED
    • NOT_SUSPENDED
    • SUSPENDED

Nested Schema for status.error

Read-Only:
  • code (String) Error code, usually taken from the endpoint response.
  • message (String) Error message, usually taken from the endpoint response.
  • origin (String) : Endpoint where the error occurred.

    Supported values

    Possible values:
    • ORIGIN_UNSPECIFIED
    • SOURCE - Error originated from the source.
    • DESTINATION - Error originated from the destination.

Nested Schema for status.last_iteration

Read-Only:
  • average_throughput_bytes (Number) Average throughput in bytes per second during the iteration.
  • end_time (String) : Iteration end time. A string representing a timestamp in ISO 8601 format: YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SS.SSS±HH:MM
  • error (Attributes) Human-readable error description. Populated only if state is FAILED. (see below for nested schema)
  • objects_transferred_count (Number) Number of objects transferred during this iteration.
  • objects_transferred_size (Number) Total size of objects transferred during this iteration.
  • sequence_number (Number) Sequence number of the iteration in the transfer, starting from 1.
  • start_time (String) : Iteration start time. A string representing a timestamp in ISO 8601 format: YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SS.SSS±HH:MM
  • state (String) : Current iteration state.

    Supported values

    Iteration state. Possible values:
    • STATE_UNSPECIFIED
    • IN_PROGRESS
    • COMPLETED
    • INTERRUPTED
    • FAILED

Nested Schema for status.last_iteration.error

Read-Only:
  • code (String) Error code, usually taken from the endpoint response.
  • message (String) Error message, usually taken from the endpoint response.
  • origin (String) : Endpoint where the error occurred.

    Supported values

    Possible values:
    • ORIGIN_UNSPECIFIED
    • SOURCE - Error originated from the source.
    • DESTINATION - Error originated from the destination.