How to do S3 migration to another AWS Account

Hello everyone! Sometimes when you want to achieve something on the cloud the official documentation is not gonna be enough for you. So when I tried the official documentation on AWS for migrating S3 Buckets, I received the error Access Denied for copying the objects to another AWS Accounts Bucket. And this is my solution to overcome that!

First things first, in order to create a migration plan we should already know the following variables:

  1. source_bucket_address:
  2. destination_bucket_address:
  3. source_iam_user_arm:
  4. source_account_user_canonical_id:

Step 1 – Creating the Configurations for the Source Bucket

After filling in these variables time to create a policy to allow us to upload objects to the destination bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::source_bucket_address",
                "arn:aws:s3:::source_bucket_address/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::destination_bucket_address",
                "arn:aws:s3:::destination_bucket_address/*"
            ]
        }
    ]
}

After creating that policy add that to your user. If your user has to copy objects that have an object tag you should also add this permission s3:GetObjectTagging to be able to copy the objects to the destination bucket.

Note: After doing all of these steps if you still receive an Access Denied error try to open your bucket for public access during migration time. Even though we gave the right access to our user, closing the bucket for public access sometimes might create errors.


Step 2 – Creating the Configurations for the Destination Bucket

On the destination bucket go to the Permissions > Object Ownership and change that to the ACLs enabled with Bucket Owner preferred. And save changes.

After doing that we should create a bucket policy, in order to let our user upload objects to that bucket.

{
    "Version": "2012-10-17",
    "Id": "Policy1611277539797",
    "Statement": [
        {
            "Sid": "Stmt1611277535086",
            "Effect": "Allow",
            "Principal": {
                "AWS": "source_iam_user_arm"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::destination_bucket_address/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        },
        {
            "Sid": "Stmt1611277877767",
            "Effect": "Allow",
            "Principal": {
                "AWS": "source_iam_user_arm"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::destination-DOC-EXAMPLE-BUCKET"
        }
    ]
}

Add that to Permissions > Bucket Policy section.

After doing that time to add our user to the destination buckets Access control list (ACL).

Go to the Permissions > Access Control List and then click on Add grantee, after that enter your user canonical ID and give the permissions as the image:


Step 3 – Copy the files

Time to start the copying process. In order to do that fire up your AWS CLI and type the following command:

aws s3 cp s3://source_bucket_address s3://destination_bucket_address --recursive

Referances: