On-demand Migration: Pull Objects into RustFS

On-demand migration is an inbound pull into a local RustFS bucket. When a client reads a key that is not already stored there, RustFS reads that object from a configured external source, returns it, and writes it locally. You can also start a backfill to pull keys that no client has requested yet.

Use it when you want RustFS to fill a bucket from a source you can read, and to serve later reads of those stored keys from RustFS. The enable form states that RustFS never writes or deletes objects on the source. After a key is stored locally, on-demand migration does not keep following the source. A later change to that key on the source is not pulled over the local object. A local PUT or DELETE is not written back to the source.

Bucket replication is a separate feature. It pushes object versions from a RustFS bucket to another bucket, and it requires versioning on both buckets. Site replication synchronizes site configuration and identity data across deployments. On-demand migration does neither of those jobs.

The walkthrough below uses Cloudflare R2 as the worked example. The same Console sequence applies to every provider in the list. Only the endpoint, credential, and bucket fields change. The lab used Console v1.0.3 on port 9001. Figures later in this post are what that run reported. They are not a throughput or capacity claim.

Field notes and the rest of the screenshots are in the documentation: On-demand migration (Chinese).

Which path to use

Goal	Use
Pull objects into RustFS from a provider in the list below, on a GET miss or by backfill	On-demand migration
Push object versions from a RustFS bucket to another bucket	Bucket replication
Synchronize site configuration and identity data across deployments	Site replication
Copy current objects with an S3 client, verify them, then point clients at RustFS	A client mirror, as in Migrating from MinIO to RustFS

Do not select a bucket that is already a replication target of this local bucket as the on-demand migration source.

Migration active means the bucket is attached to the source. Runtime counters stay at zero until a client reads a missing key or you start a backfill.

Supported source providers

Source connection → Provider lists these sources. The names below are the Console labels. Google Cloud Storage is two entries, HMAC and Native.

Amazon S3
S3-compatible storage
MinIO
RustFS
Cloudflare R2
Google Cloud Storage (HMAC)
Azure Blob Storage (Native)
Google Cloud Storage (Native)

If you need a provider that is not in this list, open a pull request in the RustFS repository for official review.

Requirements

An installed RustFS instance. The Linux quick start covers install, Console sign-in, and creating a bucket.
The RustFS CLI, rc, on the host that will read the local bucket.
A source bucket you can read. This example uses a Cloudflare R2 bucket and an R2 API token that can read that bucket and cannot write or delete there. Public access on the R2 bucket is not part of this flow.
A Console account that can open the local bucket's Settings.

Point rc at the local RustFS S3 API, not at the source:

rc alias set rustfs http://localhost:9000 <your-access-key> <your-secret-key> --region us-east-1 --bucket-lookup path

The local bucket in this walkthrough is migration. The R2 bucket is dl-rustfs. Versioning was enabled on migration, and Object Lock was disabled. The walkthrough did not need Object Lock to enable on-demand migration, and it did not treat versioning as a requirement for enabling it. With versioning enabled, a local delete marker can hide the source key.

The R2 S3 endpoint has the form https://<account-id>.r2.cloudflarestorage.com. If the bucket uses a jurisdiction, copy the endpoint from the R2 dashboard. Screenshots in this post use placeholders for the account id and access key. The form states that RustFS stores the secret and does not return it to the Console.

1. Create an empty local bucket

In the Console, open Browser → Buckets, create a bucket named migration, and confirm that it has no objects. Open Settings from that bucket when you are ready to attach the source.

2. Open on-demand migration

Open bucket migration and select Settings.
Select Automation.
Find On-demand migration.
When it is not configured, select Open migration settings or + Enable on-demand migration. When it is already active, select Manage migration.

The unconfigured page in this walkthrough says that no migration source is configured and offers + Enable on-demand migration. After the source is enabled, Automation shows Migration active and Manage migration.

3. Choose the provider

In Enable on-demand migration, set the source. This lab selected Cloudflare R2. Replace the account id and keys with your own read token. Leave the session token empty unless the token requires one. Leave Local prefix empty to expose the whole source namespace. This walkthrough saved no extra prefix (All keys, No added prefix).

Field	Value in this walkthrough
Source	Cloudflare R2
Endpoint	`https://<account-id>.r2.cloudflarestorage.com`
Region	`us-east-1` (the form also accepts `auto`)
Source Bucket	`dl-rustfs`
Addressing style	Automatic
Access Key	R2 access key id
Secret Key	R2 secret
Session token	Empty
Local prefix	Empty

For another provider in the list, enter that system's endpoint, read credential, and bucket name in the same fields.

4. Test the connection and enable migration

Select Test connection. A successful check reports Connection test passed and shows a sample object key from the source bucket. Then select Enable migration.

The configuration status becomes Migration active. Until something is read or backfilled, Runtime status stays at zero migrated objects and zero migrated data. Latest job stays No backfill yet.

5. Pull one object on a GET miss

Read the local bucket with rc. Do not send this request to the R2 endpoint. In this walkthrough the local bucket was still empty, and one missing object was pulled. The first miss returns the object to the client and stores it locally. A later read of the same key is served from RustFS.

rc cp rustfs/migration/artifacts/rustfs/packages/nightly/rustfs-nightly-2026-11-02.deb /tmp/

rustfs/migration/artifacts/rustfs/packages/nightly/rustfs-nightly-2026-11-02.deb -> /tmp/rustfs-nightly-2026-11-02.deb (71.34 MiB)

Use another key if the source bucket does not contain this object. After this pull, the Console Browser shows that object in the local bucket. The 71.34 MiB line is this single rc cp. It is not the backfill total.

6. Backfill a prefix or the source namespace

Start backfill scans the source and pulls keys that are not already local. A prefix backfill in this walkthrough used these settings:

Option	Meaning	Walkthrough value
Source listing prefix	List only this prefix	`artifacts/rustfs/packages/nightly/`
Existing local objects	What to do when the key is already local	Always skip
Dry run	List matches without queueing or writing objects	On for the rehearsal, then off for the pull

Dry run lists matching objects without queueing or writing any object. Confirm the prefix, then turn Dry run off before you pull.

Leave Source listing prefix empty to scan the configured source namespace. Run Dry run first and confirm the listed set. A dry run in this walkthrough finished with objects listed and nothing pulled:

Listed	Pulled	Skipped	Failed objects	Data
128	0	0	0	0 B

The full backfill that followed, with an empty prefix, finished as follows. These figures are that lab job. They are not a rate, and they are not a sizing guide for another bucket.

Listed	Pulled	Skipped	Failed objects	Data
128	127	1	0	8.0 GiB

While a backfill is writing objects, the Console shows listed and pulled counts. It does not show a completion percentage while the source scan is still running.

7. Verify in Browser and Runtime

On the on-demand migration page, check:

Runtime status — migrated objects and migrated data are no longer zero, and no pull failures are recorded.
Background backfill — the latest job shows the listed, pulled, skipped, and failed counts.
Browser and rc — list the prefix with rc ls, and read a local key again with rc cp.

In this walkthrough the runtime panel showed Migrated data 8.1 GiB and Migrated objects 128. The latest backfill showed Listed 128, Pulled 127, Skipped 1, Failed objects 0, and Data 8.0 GiB. The runtime panel and the backfill job are separate readouts in the lab record. The bucket list showed migration with 128 objects and 8.1 GiB.

A later change to the same key on the source is not pulled over the local object. A local PUT of that key is not written back to the source, and it stays in front of the source key. On-demand migration is not continuous replication from the source.