In this tutorial, we'll employ rclone, a Go-based program bundled as a standalone binary, to transfer data from AWS S3 to Utho Object Stores. Rclone, an open-source tool, facilitates file management across various cloud storage services. If you've previously worked with an S3-compatible storage system, you may be acquainted with s3cmd. While offering similar functionalities, rclone extends its support beyond S3-compatible storage, encompassing services like Google Drive and Dropbox.
Configurations on the AWS side:
Step 1: Generate a user with programmatic access and retain both the Access ID and Secret Key securely.
Step 2: Grant the new user at least read access to the S3 resource.
Configurations on the Utho Cloud side:
Step 3: Establish a new Bucket on Utho Cloud.
Step 4: Generate a new access key to associate it with the newly created bucket.
Step 5: Link the new access key with the bucket.
Step 6: Log in to your Linux server, from which you'll configure the data transfer from AWS S3 object storage to Utho object storage.
Getting started with rclone:
Installing rclone
Before commencing the data migration process, you must first install rclone.
Step 7: Install rclone using the following command:
| apt-get install rclone |
Configuring rclone:
Now that rclone is installed on your system, the subsequent step involves configuring it with your AWS security credentials and your Utho object store credentials.
Step 8: Create a configuration file to input the details of the object storages:
| mkdir -p ~/.config/rclone vi ~/.config/rclone/rclone.conf [s3] type = s3 env_auth = false access_key_id = AKbffaww secret_access_key = sjFWwbfadaw region = ap-south-1 acl = private [utho] type = s3 env_auth = false access_key_id = xICXfdhdrrrsesa secret_access_key = gIMQA57tHrFbfddf endpoint = innoida.utho.io acl = private |
Step 9: Modify the file permissions:
| chmod 600 ~/.config/rclone/rclone.conf |
Step 10: Verify if the correct details are set:
| rclone listremotes rclone lsd utho: rclone s3: rclone tree s3: |
Step 11: Transfer the data from AWS S3 to Utho object storage
| rclone copy s3: aws_bucket_name utho:utho_bucketname |
Task completed successfully. Thank you.