Replace <ACCESS_KEY>
and <SECRET_KEY>
with the credentials of your API key.
Migrating Scaleway Object Storage data with MinIO client
- MinIO
- object-storage
- s3
The MinIO Client provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, etc. It can communicate with any Amazon S3-compatible cloud storage provider and can be used to migrate data from one region to another.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- An SSH key
- An Instance
- Object Storage buckets in two different regions
Installing the MinIO client
- Connect to your Instance as
root
via SSH. - Update the APT package cache and the software already installed on the Instance:
apt update && apt upgrade -y
- Download the MinIO client:
wget https://dl.minio.io/client/mc/release/linux-amd64/mc -P /usr/bin/
- Make the file executable:
chmod +x /usr/bin/mc
Configuring the MinIO client and migrating data
-
Configure the endpoints you wish to use:
mc config host add scw-fr-par https://s3.fr-par.scw.cloud <ACCESS_KEY> <SECRET_KEY> --api S3v4mc config host add scw-nl-ams https://s3.nl-ams.scw.cloud <ACCESS_KEY> <SECRET_KEY> --api S3v4Note -
Optionally, add other providers:
For Amazon S3-compatible storage:
mc config host add s3 <ENDPOINT> <OTHER_ACCESS_KEY> <OTHER_SECRET_KEY> --api S3v4For Google Cloud Storage:
mc config host add gcs https://storage.googleapis.com <ACCESS_KEY> <SECRET_KEY> --api S3v2 -
Create the buckets you need on the Scaleway Object Storage:
mc mb --region fr-par scw-fr-par/<BUCKET_NAME>mc mb --region nl-ams scw-nl-ams/<OTHER_BUCKET_NAME> -
Start the migration of your data:
mc cp --recursive s3/<BUCKET_NAME> scw-fr-par/<BUCKET_NAME>mc cp --recursive gcs/<OTHER_BUCKET_NAME> scw-nl-ams/<OTHER_BUCKET_NAME>The commands above:
- Migrates data from an Amazon S3-compatible Object Storage to Scaleway’s fr-par Object Storage
- Migrates data from GCS Object Storage to Scaleway’s nl-ams Object Storage
ImportantNote that this migration can incur some cost from the Object Storage you are migrating from since they may bill egress bandwidth.
The MinIO client displays a status bar during the transfer, allowing to observe the status of the migration:
For more information about MinIO Client refer to the official documentation.