how to configure s3 bucket with aws-cli

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services that provides object storage through a web service interface.

 

Step 1) Click on Services >> Write IAM and enter

 

Step 2) Click on Users >> Click on add user

 

Step 3) Write user name >> tick on "Programmatic access" >> click on next

 

Step 4) Click on "Attach existing policies directly" >> write S3 >> tick on "AmazonS3FullAccess" >> click on next

Or

Provide Limited Access to the bucket so that you will be prevent from the accidental deletion and being hacked.

Below is S3-Custom-Inline-Policy allow only Download/Upload the object in the bucket.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:Get*", "s3:put*" ], "Resource": [ "arn:aws:s3:::Bucket-Name-1/*", "arn:aws:s3:::Bucket-Name-2/*" ] } ] }

Note : This user only can Put/Get the object from the source code and see the object via console.

 

Step 5) Write the tag >> click on next

 

Step 6) cilck on create user 

 

Step 7) Download .csv file and click on close

.csv file have "Access key ID" and "Secret access key"

 

Login into Ubuntu terminal and install below following service 

$ sudo apt-get install awscli
$ sudo apt-get install python-pip

 

Step 8) fill the details by below fllowing commands. 

Note:- Access key and Secret access key will be copy from the .csv file. 

$ aws configure

 

Step 9) Create New Bucket by below command.

$ aws s3 mb s3://mynews3awsclibucket/

 

Step 10) Create a test directory and create test file 

$ mkdir test
$ touch test/test.txt

 

Step 11) Sync the local system folder from aws S3 bucket.

$ aws s3 sync "/home/ubuntu/test/" s3://mynews3awsclibucket/

 

Step 12) Now check your S3 bucket in aws

Click on services >> write s3 >> click on s3

 

 

Integration with cron 

*/1 * * * * aws s3 sync "/home/ubuntu/test/" s3://mynews3awsclibucket/

Remove file older than 10 days

0 9 * * * /usr/bin/find /home/ubuntu/test/ -name "*.gz" -type f -mtime +10 -exec rm -f {} \;

Another one more Way to get securely access of S3 bucket, In this process you don't need to configure aws configuration file.

• Create IAM Role and Attach S3 access policy.

Create IAM Role >>  EC2 >> attach S3 Policy 

• Attach IAM Role to EC2 

Select EC2 >> Action >> Security >> Attach IAM Role

 

 

AWS S3 CLI commands

 Install awscli on respected machine.

 

$  sudo apt  install awscli

 

1) Listing S3 buckket

$ aws s3 ls

2) Download content from S3

$ aws s3 cp s3://ec22access "/home/ubuntu/S3_local" --recursive

3) Copying an S3 object from one bucket to another

For Single content

$ aws s3 cp s3://ec22access/abc.txt s3://mynews3awsclibucket123/abc.txt

For all content

$ aws s3 cp s3://mybucket/ s3://mybucket2/ --recursive

4) Sync the local system folder from aws S3 bucket.

$ aws s3 sync "/home/ubuntu/test/" s3://mynews3awsclibucket/

 
4.1) upload local system folder to S3 bucket.

 

$ aws s3 cp --recursive "/home/ubuntu/pdf/" "s3://bucketname/pdf"

 

5) Delete/Remove Only Single Object

 

$ aws s3 rm  s3://bucketname/test.html

 

6) Delete/Remove all Object Inside the bucket

$ aws s3 rm --recursive s3://bucketname

7) Create New Bucket with region

$ aws s3 mb s3://bucketname --region us-east-2

8) Delete/Remove Only Empty Bucket

 

$ aws s3api delete-bucket --bucket bucketname

9) Delete/Remove bucket that contains objects

$ aws s3 rb s3://bucketname --force 

10) Will add more cmd soon...