I've created a s3 server which contain a large number of images. I'm now trying to create a bucket policy, which fits my needs. First of all i want everybody to have read permission, so they can see the images. However i also want to give a specific website the permission to upload and delete images. this website is not stored on a amazon server? how can i achieve this? so far i've created an bucket policy which enables everybody to see the images
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::examplebucket/*"
}
]
}
You can delegate access to your bucket. To do this, the other server will need AWS credentials.
If the other server were an EC2 instance that you owned then you could do this easily by launching it with an IAM role. If the other server were an EC2 instance that someone else owned, then you could delegate access to them by allowing them to assume an appropriate IAM role in your account. But for a non-EC2 server, as seems to be the case here, you will have to provide AWS credentials in some other fashion.
One way to do this is by adding an IAM user with a policy allowing s3:PutObject and s3:DeleteObject on resource "arn:aws:s3:::examplebucket/*", and then give the other server those credentials.
A better way would be to create an IAM role that has the same policy and then have the other server assume that role. The upside is that the credentials must be rotated periodically so if they are leaked then the window of exposure is smaller. To assume a role, however, the other server will still need to authenticate so will need some base IAM user credentials (unless you have some way to get credentials via identity federation). You could add a base IAM user who has permissions to assume the aforementioned role (but has no other permissions) and supply the base IAM user credentials to the other server. When using AssumeRole in this fashion you should require an external ID. You may also be able to restrict the entity assuming this role to the specific IP address(es) of the other server using a policy condition (not 100% sure if this is possible).
The Bucket Policy will work nicely to give everybody read-only access.
To give specific permissions to an application:
Create an IAM User for the application (this also creates access credentials)
Assign a policy to the IAM User that gives the desired permissions (very similar to a Bucket Policy)
The application then makes API calls to Amazon S3 using the supplied access credentials
See also: Amazon S3 Developer Guide
Related
I am working on aws SAM project and i have a requirement of giving access to my S3 bucket to multiple iam users from unknown aws accounts but i can't make bucket publicly accessible. I want to secure my bucket as well as i want any iam user from any aws account to access the contents of my S3 bucket. Is this possible?
Below is the policy i tried and worked perfectly.
{
"Version": "2012-10-17",
"Id": "Policy1616828964582",
"Statement": [
{
"Sid": "Stmt1616828940658",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/STS_Role_demo"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::new-demo-bkt/*"
}
]
}
Above policy is for one user but i want any user from other AWS account to access my contents without making the bucket and objects public so how can i achieve this?
This might be possible using a set of Conditions on the incoming requests.
I can think of two options:
You create an IAM role that your SAM application uses even when running in other accounts
You create S3 bucket policies that allow unknown users access
If you decide to look into S3 bucket policies, I suggest using an S3 Access Point to better manage access policies.
Access points are named network endpoints that are attached to buckets
that you can use to perform S3 object operations, such as GetObject
and PutObject. Each access point has distinct permissions and network
controls that S3 applies for any request that is made through that
access point. Each access point enforces a customized access point
policy that works in conjunction with the bucket policy that is
attached to the underlying bucket.
You can use a combination of S3 Conditions to restrict access. For example, your SAM application could include specific condition keys when making S3 requests, and the bucket policy then allows access based on those conditions.
You can also apply global IAM conditions to S3 policies.
This isn't great security though, malicious actors might be able to figure out the headers and spoof requests to your bucket. As noted on some conditions such as aws:UserAgent:
This key should be used carefully. Since the aws:UserAgent value is
provided by the caller in an HTTP header, unauthorized parties can use
modified or custom browsers to provide any aws:UserAgent value that
they choose. As a result, aws:UserAgent should not be used to
prevent unauthorized parties from making direct AWS requests. You can
use it to allow only specific client applications, and only after
testing your policy.
Is it possible to disable AWS S3 management console for the security reasons?
We don't want anyone including root/admin users to access customer files directly from the AWS S3. We should just have programmatic access to the files stored in S3.
If this is not possible, is it possible to stop listing the directories inside the bucket for all users ?
This is a tricky one to implement, however the following should be able to fulfill the requirements.
Programmatic Access Only
You need to define exactly which actions should be denied you would not want to block access completely otherwise you might lose the ability to do anything.
If you're in AWS you should use IAM roles, and a VPC endpoint to connect to the S3 service. Both of these support the ability to control access within your S3 buckets Bucket Policy.
You would use this to deny List* actions where the source is not the VPC endpoint. You could also deny where its not a specific subset of roles.
This works for all programmatic use cases and for people who login as an IAM user from the console, however this does not deny access to the root user.
Also bear in mind for any IAM user/IAM role that they do not have access unless you explicitly give it to them via an IAM policy.
Denying Access To The Root User
There is currently only one way to deny access to the root user of an AWS account (although you should share these credentials with anyone, even within your company) as that is using a Service Control Policy.
To do this the account would need to be part of an AWS organisation (as an organisational unit). If/once it is you would create a SCP that denies access to the root principal for the specific actions that you want.
An example of this policy for you would be
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RestrictS3ForRoot",
"Effect": "Deny",
"Action": [
"s3:List*"
],
"Resource": [
"*"
],
"Condition": {
"StringLike": {
"aws:PrincipalArn": [
"arn:aws:iam::*:root"
]
}
}
}
]
}
Yes, it is possible to disable the Management Console: Don't give users a password.
When creating IAM Users, there are two ways to provide credentials:
Sign-in Credentials (for the Console)
Access Key (for API calls)
Only give users an Access Key and they won't be able to login to the console.
However, please note that when when using the Management Console, users have exactly the same permissions as using an Access Key. Thus, if they can do it in the console, then they can do it via an API call (if they have an Access Key).
If your goal is to prevent anyone from accessing customer files, then you can add a Bucket Policy with a Deny on s3:* for the bucket, where the Principal is not the customer.
Please note, however, that the Root login can remove such a policy.
If the customers really want to keep their own data private, then they would need to create their own AWS account and keep their files within it, without granting you access.
I'm developing a mobile application & i want to upload/get/delete a file in AWS S3 bucket.
But I'm very concern about the security problem.
S3 Bucket: It should not be public and only authorize IAM user can access who have the permission to access my bucket.
So, need help to configure permission of my S3 bucket & create an IAM user.
That is not how you authorize access for mobile applications. Yes, you can create IAM user, generate access key and secret access key, store those keys in the application code and configure right permissions for the IAM user. Then you don't even need to configure bucket policy. By default, bucket is private and only IAM users in your account with appropriate permissions are able to access it. If you allow IAM user to access specific S3 bucket then you would need to configure explicit deny on bucket policy to override it.
But the above approach is against every security good practice. What you really want to do is to create IAM role that allows access to the bucket and assume that role from within the application. You can set up Cognito + web federation (or some other web federation provider) for your users and ask STS service to generate short lived credentials using sts:assumeRoleWithWebIdentity command.
As for the IAM permissions, you will need to allow s3:PutObject, s3:GetObject and s3:DeleteObject so the policy can look something like this.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "<arn-of-your-bucket>"
}
]
}
You can be even more granular and allow Cognito users to access only "their" folder inside of a bucket if you need to.
As for the role, you just need to attach the above policy to it and configure trust relationship between the role and web identity provider (as mentioned above, this can be Cognito or any OpenID provider).
Is there any way to allow all instances created by a specific AWS account access to an S3 bucket?
I would like to provide data that should be very simple for clients to download to their instances. Ideally, automatically via the post_install script option of AWS ParallelCluster.
However, it seems like this requires a lot of setup, as is described in this tutorial by AWS:
https://aws.amazon.com/premiumsupport/knowledge-center/s3-instance-access-bucket/
This is not feasible for me. Clients should not have to create IAM roles.
The best I came up with at the moment is allowing S3 bucket access to a specific AWS account and then working with access keys:
export AWS_ACCESS_KEY_ID=<key-id>
export AWS_SECRETE_ACCESS_KEY=<secret-key>
aws s3 cp s3://<bucket> . --recursive
Unfortunately, this is also not ideal as I would like to provide ready-to-use AWS Parallelcluster post_install scripts. These scripts should automatically download the required data on cluster startup.
Is there any way to allow all instances created by a specific AWS account access to an S3 bucket?
Yes. It's a 2 step process. In summary:
1) On your side, the bucket must trust the account id of the other accounts that will access it, and you must decide which calls you will allow.
2) On the other accounts that will access the bucket, the instances must be authorised to run AWS API calls on your bucket using IAM policies.
In more detail:
Step 1: let's work through this and break it down.
On your bucket, you'll need to configure a bucket policy like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "111",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNT_ID_TO_TRUST:root"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME_HERE/*"
}
]
}
You can find more examples of bucket policies in the AWS documentation here.
WARNING 1: "arn:aws:iam::ACCOUNT_ID:root" will trust everything that has permissions to connect to your bucket on the other AWS account. This shouldn't be a problem for what you're trying to do, but it's best you completely understand how this policy works to prevent any accidents.
WARNING 2: Do not grant s3:* - you will need to scope down the permissions to actions such as s3:GetObject etc. There is a website to help you generate these policies here. s3:* will contain delete permissions which if used incorrectly could result in nasty surprises.
Now, once that's done, great work - that's things on your end covered.
Step 2: The other accounts that want to read the data will have to assign an instance role to the ec2 instances they launch and that role will need a policy attached to it granting access to your bucket. Those instances can then run AWS CLI commands on your bucket, provided your bucket policy authorises the call on your side and the instance policy authorises the call on their side.
The policy that needs to be attached to the instance role should look something like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:*",
"Effect": "Allow",
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME_HERE/*"
}
]
}
Keep in mind, just because this policy grants s3:* it doesn't mean they can do anything on your bucket, not unless you have s3:* in your bucket policy. Actions of this policy will be limited to whatever you've scoped the permissions to in your bucket policy.
This is not feasible for me. Clients should not have to create IAM roles.
If they have an AWS account it's up to them on how they choose to access the bucket as long as you define a bucket policy that trusts their account the rest is on them. They can create an ec2 instance role and grant it permissions to your bucket, or an IAM User and grant it access to your bucket. It doesn't matter.
The best I came up with at the moment is allowing S3 bucket access to a specific AWS account and then working with access keys:
If the code will run on an ec2 instance, it's bad practice to use access keys and instead should use an ec2 instance role.
Ideally, automatically via CloudFormation on instance startup.
I think you mean via instance userdata, which you can define through CloudFormation.
You say "Clients should not have to create IAM roles". This is perfectly correct.
I presume that you are creating the instances for use by the clients. If so, then you should create an IAM Role that has access to the desired bucket.
Then, when you create an Amazon EC2 instance for your clients, associate the IAM Role to the instance. Your clients will then be able to use the AWS Command-Line Interface (CLI) to access the S3 bucket (list, upload, download, or whatever permissions you put into the IAM Role).
If you want the data to be automatically downloaded when you first create their instance, then you can add User Data script that will execute when the instance starts. This can download the files from S3 to the instance.
I have been struggling to figure out how to communicate with the Amazon ES service from my EC2 instances.
The documentation clearly states that the Amazon ES service supports IAM User & Role based access policies. http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
However, when I have this access policy for my ES domain:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789:role/my-ec2-role"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-west-2:123456789:domain/myDomain/*"
}
]
}
I can't log into an ec2 instance and run a curl to hit my elasticsearch cluster.
Trying to do a simple curl of the _search API:
curl "http://search-myDomain.es.amazonaws.com/_search"
Produces an authentication error response:
{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-west-2:123456789:domain/myDomain/_search"}
Just to be extra safe I put the AmazonESFullAccess Policy on my IAM Role, still doesn't work.
I must be missing something, because being able to programmatically interact with Elasticsearch from ec2 instances that use an IAM Role is essential to getting anything accomplished with the Amazon ES Service.
I also see this contradictory statement in the docs.
IAM-based Policy Example You create IAM-based access policies by
using the AWS IAM console rather than the Amazon ES console. For
information about creating IAM-based access policies, see the IAM
documentation.
That link to IAM documentation, is to the home page of IAM and contains exactly zero information about how to do it. Anyone got a solution for me?
When using IAM service with AWS, you must sign your requests. curl doesn't support signed requests (which consists of hashing the request and adding a parameter to the header of the request). You can use one of their SDK's that has the signing algorithm built in, and then submit that request.
See:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html#signing-requests
You can find the SDKs for popular languages here:
http://aws.amazon.com/tools/
First, you said you can't login to an EC2 instance to curl the ES instance? You can't login? Or you can't curl it from EC2?
I have my Elasticsearch (Service) instance open to the world (with nothing on it) and am able to curl it just fine, without signing. I changed the access policy to test, but unfortunately it takes forever to come back up after changing it...
My policy looks like this:
{ "Version": "2012-10-17", "Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain/*"
},
{
"Sid": "",
"Effect": "Allow",
"Principal": "*",
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:843348267853:domain/myDomain"
}
]
}
I realize this isn't exactly what you want, but start off with this (open to the world), curl from outside AWS and test it. Then restrict it, that way you're able to isolate the issues.
Also, I think you have an issue with the "Principal" in your access policy. You have your EC2 Role. I understand why you're doing that, but I think the Principal requires a USER, not a role.
See below:
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
Principal
Specifies the AWS account or IAM user that is allowed or denied access
to a resource. Specifying a wildcard (*) enables anonymous access to
the domain, which is not recommended. If you do enable anonymous
access, we strongly recommend that you add an IP-based condition to
restrict which IP addresses can submit requests to the Amazon ES
domain.
EDIT 1
To be clear, you added the AmazonESFullAccess policy to the my-ec2-role? If you're going to use IAM access policies, I don't think you can have a resource based policy attached to it (which is what you're doing).
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_compare-resource-policies.html
For some AWS services, you can grant cross-account access to your
resources. To do this, you attach a policy directly to the resource
that you want to share, instead of using a role as a proxy. The
resource that you want to share must support resource-based policies.
Unlike a user-based policy, a resource-based policy specifies who (in
the form of a list of AWS account ID numbers) can access that
resource.
Possibly try removing the access policy altogether?
Why you don't create a proxy with elastic ip and allow your proxy to access your ES?
Basically exists three forms that you can limit access in your ES:
Allow everyone
White IP list
Signing the access key and secret key provided by AWS.
I'm using two forms, in my php apps I prefer to use proxy behind the connection to ES and in my nodejs app I prefer to sign my requests using the http-aws-es node module.
It's useful to create a proxy environment because my users needs to access the kibana interface to see some reports and it's possible because they have configured the proxy in their browsers =)
I must recommend to you close the access to your ES indexes, because it's pretty easy to delete them, curl -XDELETE https://your_es_address/index anyone can do it but you can say: "how the others users will get my ES address?" and I will answer you: "Security based in dimness isn't a real security"
My security access policy is basically something like it:
http://pastebin.com/EUKT1ekX
I encountered this issue recently and the root problem is that none of the Amazon SDKs yet support calling Elasticsearch operations like search, put, etc.
The only workaround at the moment is to execute requests directly against the endpoint using signed requests:
http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
The example here is for calling EC2, but it can be modified to instead call against Elasticsearch. Just modify the "service" value to "es". From there, you have to fill in values for
the endpoint (which is the full URL of your cluster including operation without request parameters)
the host (the part between https:// and your canonical URI like /_status
the canonical uri which is the URI after the first / inclusive (like /_status) but without the query string
the request parameters (everything after ? inclusive)
Note that I've only managed to get this working so far using AWS credentials as the assumption is that you pass in an access key and secret key to the various signing calls (access_key and secret_key in the example). It should be doable using IAM roles but you'll have to call into the security token service first to get temporary credentials that can be used to sign the request. Until you do that, be sure to edit your access policy on the Elasticsearch cluster to allow user creds (user/
you need to sign your request and unfortunately, it is no longer supported by the official elasticsearch library. Check this Github issue (https://github.com/elastic/elasticsearch-js/issues/1182#issuecomment-630641702)
They want to enforce their own cloud solution