aws s3 upload fail only at production envrionment, but success at local environment - amazon-web-services

I tried to upload image using aws-sdk, multer-s3.
In my local environment, uploading image was succeed, but in production environment(aws lambda), it fail with error status 403 forbidden.
But my aws credential key and secret-key is same as local environment. also i checked aws key in production environment successfully.
I think difference between two other environment is nothing.What am I missing?
I have even tried setting aws key in my router code like below, but it also failed.
AWS.config.accessKeyId = 'blabla';
AWS.config.secretAccessKey = 'blalbla';
AWS.config.region = 'ap-northeast-2';
and here is my policy
{
"Id": "Policy1536755128154",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1536755126539",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::elebooks-image/*",
"Principal": "*"
}
]
}

Update your attached s3 bucket policy to a user according to below policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::YOUR-BUCKET",
"arn:aws:s3:::YOUR-BUCKET/*"
]
}
]
}
it's working on my server.

I haven't worked with AWS Lambda but I am familiar with S3. When you're using the AWS SDK in your local environment, you're probably using the root user with default full access, so it will just work.
With Lambda however, according to the following extract from the documentation, you need to make sure that the IAM role you specified when you created the Lambda function has the appropriate permissions to do an s3:putObject to that bucket.
Permissions for your Lambda function – Regardless of what invokes a Lambda function, AWS Lambda executes the function by assuming the IAM role (execution role) that you specify at the time you create the Lambda function. Using the permissions policy associated with this role, you grant your Lambda function the permissions that it needs. For example, if your Lambda function needs to read an object, you grant permissions for the relevant Amazon S3 actions in the permissions policy. For more information, see Manage Permissions: Using an IAM Role (Execution Role).
See Writing IAM policies: How to grant access to an S3 bucket

Related

AWS S3 ACL Permissions

So my bucket was and is still functioning correctly, I'm able to upload images through the API with no issues. However, I was messing around with the user policy and I made a change to the Resource for my User Policy and this caused some settings to change.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1420751757000",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": CHANGE MADE HERE
}
]
}
When I try to upload an image through my AWS account (not using the API), then the ACL public access is private by default. I tried changing my Policy version back to what I had, but no change. I am pretty inexperienced with S3, so if I'm missing crucial info regarding this issue I can provide it.
If you want all objects to be public, then you should use a Bucket Policy.
This should typically be limited to only allowing people to download (Get) an object if they know the name of the object. You can use this Bucket Policy (which goes on the bucket itself):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::YOUR-BUCKET-NAME/*"
]
}
]
}
This policy is saying: "Allow anyone to get an object from this bucket, without knowing who they are"
It does not allow listing of the bucket, upload to the bucket or deleting from the bucket. If you wish to do any of these operations, you would need to use your own credentials via an API call or using the AWS CLI.
For examples of bucket policies, see: Bucket policy examples - Amazon Simple Storage Service
Your IAM User should probably have a policy like this:
{
"Version":"2012-10-17",
"Statement":[
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
"Resource": [
"arn:aws:s3:::YOUR-BUCKET-NAME",
"arn:aws:s3:::YOUR-BUCKET-NAME/*"
]
}
]
}
This is saying: "Allow this IAM User to do anything in Amazon S3 to this bucket and the contents of this bucket"
That will grant you permission to do anything with the bucket (including uploading, downloading and deleting objects, and deleting the bucket).
For examples of IAM Policies, see: User policy examples - Amazon Simple Storage Service

allowing a third party application to write to your aws bucket

I have a bit of a problem, to which Iam not sure I know the answer to. I have a bucket named staging and I would like to give access to a third party dev (which is building the webapp) to allow file uploads into this bucket.
What is the correct way to go about doing this? Surely, not giving away my aws secrets?
Would be great if someone can point me in the right direction for this.
You can achieve it using Resource Based Policies in Staging S3 bucket.
Add a Resource Based Policy to Staging bucket that allow access to Dev account's IAM User/Role.
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Principal": {
"AWS": "<ARN of IAM User/Role from Dev Account>"
},
"Action": [
"s3:GetObject",
"s3:PutObject"
]
"Resource": "arn:aws:s3:::staging-bucket/*"
}]
}
Next, add an IAM Policy in Dev account, that allow access to S3 bucket in Staging account.
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
]
"Resource": "arn:aws:s3:::staging-bucket/*"
}]
}
References
How can I provide cross-account access to objects that are in Amazon S3 buckets?
How to access S3 bucket from another AWS account
if you are talking about a third party app is uploading content; one option is You can expose an API via apigateway to upload content to the bucket. Remember to remove public access from the bucket permission.

AWS Backup: Missing permission iam:PassRole

I'm currently spinning in circles trying to restore from an AWS Backup and am running into permissions errors. I have administrator access to my AWS account. I've tried creating a new policy and attach it to my user account in IAM as follows:
The issue I can't seem to get around is that I need to add the permission iam:PassRole but I can't seem to find it anywhere within the AWS portal. How can I add this permission to my policy?!
EDIT: I've created a policy with all backup permissions allowed and including iam:PassRole however I am still receiving the error message You are not authorized to perform this operation. when trying to perform the backup. The policy I've created and attached to my user looks as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"backup:*",
"iam:PassRole",
"iam:GetRole"
],
"Resource": "*"
}
]
}
“To successfully do a restore with the original instance profile, you will need to make changes to the restore policy. If you apply instance profile during the restore, you must update the operator role and add PassRole permissions of the underlying instance profile role to EC2. Otherwise, Amazon EC2 won’t be able to authorize the instance launch and it will fail.”
Here is the policy you can attach to the AWS default Backup role “AWSBackupDefaultServiceRole” to work around this issue:
{
"Version": "2012–10–17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<Account-ID>:role/*"
}]}
Source: https://medium.com/contino-engineering/new-aws-backup-features-for-quick-and-easy-ec2-instance-recovery-c8887365ca6a

How to create an IAM role of specific type using boto3?

I'm trying to lock down a user to a specific VPC in AWS and following How to Help Lock Down a User’s Amazon EC2 Capabilities to a Single VPC | AWS Security Blog.
It is mentioned that we need to create an IAM role with name VPCLockDown of type AWS Service
and add the services for which the role needs access to. like ec2, lambda etc.
I was trying to create this role programatically using boto3.
I checked the create_role documentation for creating a role using boto3.
However, they haven't mentioned anything to specify the type of role and the services that I can specify that the role should have access to.
Is there any way to specify these items while creation of the IAM role using boto3
Edit1:
I tried creating a service_linked_role as per Sudarshan Rampuria's answer like
response = iam.create_service_linked_role(
AWSServiceName='ec2.amazonaws.com',
)
But getting the following error:
An error occurred (AccessDenied) when calling the
CreateServiceLinkedRole operation: Cannot find Service Linked Role
template for ec2.amazonaws.com
You can use create_service_linked_role() function boto3 to link a role to a service.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.create_service_linked_role
Here is a policy that allows a specific IAM User to launch an instance (RunInstances), but only in a given VPC:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2RunInstancesVPC",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:ap-southeast-2:111111111111:subnet/*",
"Condition": {
"StringEquals": {
"ec2:vpc": "arn:aws:ec2:ap-southeast-2:111111111111:vpc/vpc-abcd1234" <--- Change this
}
}
},
{
"Sid": "RemainingRunInstancePermissions",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:ap-southeast-2:111111111111:instance/*",
"arn:aws:ec2:ap-southeast-2:111111111111:volume/*",
"arn:aws:ec2:ap-southeast-2::image/*",
"arn:aws:ec2:ap-southeast-2::snapshot/*",
"arn:aws:ec2:ap-southeast-2:111111111111:network-interface/*",
"arn:aws:ec2:ap-southeast-2:111111111111:key-pair/*",
"arn:aws:ec2:ap-southeast-2:111111111111:security-group/*"
]
}
]
}
You might need to change the Region. (I tested it in the Sydney region.)
For anyone trying to do this for Lambda, we get the similar error mentioned by the question author under "Edit". Lambda doesn't have a service linked role. You can see from the AWS Lambda documentation that "create-role" is used for creating lambda execution role.
You can also see here that only Lambda#Edge has service linked role.
One just needs to use use boto3 create-role with a policy document
response = iam_client.create_role(
RoleName="some-role-name",
AssumeRolePolicyDocument='{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}',
Description='Lambda role'
)

Amazon S3 Bucket Policy: How to lock down access to only your EC2 Instances

I am looking to lock down an S3 bucket for security purposes - i'm storing deployment images in the bucket.
What I want to do is create a bucket policy that supports anonymous downloads over http only from EC2 instances in my account.
Is there a way to do this?
An example of a policy that I'm trying to use (it won't allow itself to be applied):
{
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[my bucket name]",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:ec2:us-east-1:[my account id]:instance/*"
}
}
}
]
}
Just to clarify how this is normally done. You create a IAM policy, attach it to a new or existing role, and decorate the ec2 instance with the role. You can also provide access through bucket policies, but that is less precise.
Details below:
S3 buckets are default deny except for my the owner. So you create your bucket and upload the data. You can verify with a browser that the files are not accessible by trying https://s3.amazonaws.com/MyBucketName/file.ext. Should come back with error code "Access Denied" in the xml. If you get an error code of "NoSuchBucket", you have the url wrong.
Create an IAM policy based on arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess. Starts out looking like the snip below. Take a look at the "Resource" key, and note that it is set to a wild card. You just modify this to be the arn of your bucket. You have to do one for the bucket and its contents so it becomes: "Resource": ["arn:aws:s3:::MyBucketName", "arn:aws:s3:::MyBucketName/*"]
Now that you have a policy, what you want to do is to decorate your instances with a IAM Role that automatically grants it this policy. All without any authentication keys having to be in the instance. So go to Role, create new role, make an Amazon EC2 role, find the policy you just created, and your Role is ready.
Finally you create your instance, and add the IAM role you just created. If the machine already has its own role, you just have to merge the two roles into a new one for the machine. If the machine is already running, it wont get the new role until you restart.
Now you should be good to go. The machine has the rights to access the s3 share. Now you can use the following command to copy files to your instance. Note you have to specify the region
aws s3 cp --region us-east-1 s3://MyBucketName/MyFileName.tgz /home/ubuntu
Please Note, the term "Security through obscurity" is only a thing in the movies. Either something is provably secure, or it is insecure.
I used something like
{
"Version": "2012-10-17",
"Id": "Allow only My VPC",
"Statement": [
{
"Sid": "Allow only My VPC",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject", "s3:ListBucket",
"Resource": [
"arn::s3:::{BUCKET_NAME}",
"arn::s3:::{BUCKET_NAME}/*"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "{VPC_ID}" OR "aws:sourceVpce": "{VPCe_ENDPOINT}"
}
}
}
]
}