I'm trying to lock down a user to a specific VPC in AWS and following How to Help Lock Down a User’s Amazon EC2 Capabilities to a Single VPC | AWS Security Blog.
It is mentioned that we need to create an IAM role with name VPCLockDown of type AWS Service
and add the services for which the role needs access to. like ec2, lambda etc.
I was trying to create this role programatically using boto3.
I checked the create_role documentation for creating a role using boto3.
However, they haven't mentioned anything to specify the type of role and the services that I can specify that the role should have access to.
Is there any way to specify these items while creation of the IAM role using boto3
Edit1:
I tried creating a service_linked_role as per Sudarshan Rampuria's answer like
response = iam.create_service_linked_role(
AWSServiceName='ec2.amazonaws.com',
)
But getting the following error:
An error occurred (AccessDenied) when calling the
CreateServiceLinkedRole operation: Cannot find Service Linked Role
template for ec2.amazonaws.com
You can use create_service_linked_role() function boto3 to link a role to a service.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.create_service_linked_role
Here is a policy that allows a specific IAM User to launch an instance (RunInstances), but only in a given VPC:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2RunInstancesVPC",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:ap-southeast-2:111111111111:subnet/*",
"Condition": {
"StringEquals": {
"ec2:vpc": "arn:aws:ec2:ap-southeast-2:111111111111:vpc/vpc-abcd1234" <--- Change this
}
}
},
{
"Sid": "RemainingRunInstancePermissions",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:ap-southeast-2:111111111111:instance/*",
"arn:aws:ec2:ap-southeast-2:111111111111:volume/*",
"arn:aws:ec2:ap-southeast-2::image/*",
"arn:aws:ec2:ap-southeast-2::snapshot/*",
"arn:aws:ec2:ap-southeast-2:111111111111:network-interface/*",
"arn:aws:ec2:ap-southeast-2:111111111111:key-pair/*",
"arn:aws:ec2:ap-southeast-2:111111111111:security-group/*"
]
}
]
}
You might need to change the Region. (I tested it in the Sydney region.)
For anyone trying to do this for Lambda, we get the similar error mentioned by the question author under "Edit". Lambda doesn't have a service linked role. You can see from the AWS Lambda documentation that "create-role" is used for creating lambda execution role.
You can also see here that only Lambda#Edge has service linked role.
One just needs to use use boto3 create-role with a policy document
response = iam_client.create_role(
RoleName="some-role-name",
AssumeRolePolicyDocument='{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}',
Description='Lambda role'
)
Related
I tried to upload image using aws-sdk, multer-s3.
In my local environment, uploading image was succeed, but in production environment(aws lambda), it fail with error status 403 forbidden.
But my aws credential key and secret-key is same as local environment. also i checked aws key in production environment successfully.
I think difference between two other environment is nothing.What am I missing?
I have even tried setting aws key in my router code like below, but it also failed.
AWS.config.accessKeyId = 'blabla';
AWS.config.secretAccessKey = 'blalbla';
AWS.config.region = 'ap-northeast-2';
and here is my policy
{
"Id": "Policy1536755128154",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1536755126539",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::elebooks-image/*",
"Principal": "*"
}
]
}
Update your attached s3 bucket policy to a user according to below policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::YOUR-BUCKET",
"arn:aws:s3:::YOUR-BUCKET/*"
]
}
]
}
it's working on my server.
I haven't worked with AWS Lambda but I am familiar with S3. When you're using the AWS SDK in your local environment, you're probably using the root user with default full access, so it will just work.
With Lambda however, according to the following extract from the documentation, you need to make sure that the IAM role you specified when you created the Lambda function has the appropriate permissions to do an s3:putObject to that bucket.
Permissions for your Lambda function – Regardless of what invokes a Lambda function, AWS Lambda executes the function by assuming the IAM role (execution role) that you specify at the time you create the Lambda function. Using the permissions policy associated with this role, you grant your Lambda function the permissions that it needs. For example, if your Lambda function needs to read an object, you grant permissions for the relevant Amazon S3 actions in the permissions policy. For more information, see Manage Permissions: Using an IAM Role (Execution Role).
See Writing IAM policies: How to grant access to an S3 bucket
There's some CSV data files I need to get in S3 buckets belonging to a series of AWS accounts belonging to a third-party; the owner of the other accounts has created a role in each of the accounts which grants me access to those files; I can use the AWS web console (logged in to my own account) to switch to each role and get the files. One at a time, I switch to the role for each of the accounts and then get the files for that account, then move on to the next account and get those files, and so on.
I'd like to automate this process.
It looks like AWS Glue can do this, but I'm having trouble with the permissions.
What I need it to do is create permissions so that an AWS Glue crawler can switch to the right role (belonging to each of the other AWS accounts) and get the data files from the S3 bucket of those accounts.
Is this possible and if so how can I set it up? (e.g. what IAM roles/permissions are needed?) I'd prefer to limit changes to my own account if possible rather than having to ask the other account owner to make changes on their side.
If it's not possible with Glue, is there some other easy way to do it with a different AWS service?
Thanks!
(I've had a series of tries but I keep getting it wrong - my attempts are so far from being right that there's no point in me posting the details here).
Yes, you can automate your scenario with Glue by following these steps:
Create an IAM role in your AWS account. This role's name must start with AWSGlueServiceRole but you can append whatever you want. Add a trust relationship for Glue, such as:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "glue.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Attach two IAM policies to your IAM role. The AWS managed policy named AWSGlueServiceRole and a custom policy that provides the access needed to all the target cross account S3 buckets, such as:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketAccess",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::examplebucket1",
"arn:aws:s3:::examplebucket2",
"arn:aws:s3:::examplebucket3"
]
},
{
"Sid": "ObjectAccess",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::examplebucket1/*",
"arn:aws:s3:::examplebucket2/*",
"arn:aws:s3:::examplebucket3/*"
]
}
]
}
Add S3 bucket policies to each target bucket that allows your IAM role the same S3 access that you granted it in your account, such as:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::your_account_number:role/AWSGlueServiceRoleDefault"
},
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::examplebucket1"
},
{
"Sid": "ObjectAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::your_account_number:role/AWSGlueServiceRoleDefault"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::examplebucket1/*"
}
]
}
Finally, create Glue crawlers and jobs in your account (in the same regions as the target cross account S3 buckets) that will ETL the data from the cross account S3 buckets to your account.
Using the AWS CLI, you can create named profiles for each of the roles you want to switch to, then refer to them from the CLI. You can then chain these calls, referencing the named profile for each role, and include them in a script to automate the process.
From Switching to an IAM Role (AWS Command Line Interface)
A role specifies a set of permissions that you can use to access AWS
resources that you need. In that sense, it is similar to a user in AWS
Identity and Access Management (IAM). When you sign in as a user, you
get a specific set of permissions. However, you don't sign in to a
role, but once signed in as a user you can switch to a role. This
temporarily sets aside your original user permissions and instead
gives you the permissions assigned to the role. The role can be in
your own account or any other AWS account. For more information about
roles, their benefits, and how to create and configure them, see IAM
Roles, and Creating IAM Roles.
You can achieve this with AWS lambda and Cloudwatch Rules.
You can create a lambda function that has a role attached to it, lets call this role - Role A, depending on the number of accounts you can either create 1 function per account and create one rule in cloudwatch to trigger all functions or you can create 1 function for all the accounts (be cautious to the limitations of AWS Lambda).
Creating Role A
Create an IAM Role (Role A) with the following policy allowing it to assume the role given to you by the other accounts containing the data.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1509358389000",
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": [
"",
"",
....
"
]// all the IAM Role ARN's from the accounts containing the data or if you have 1 function for each account you can opt to have separate roles
}
]
}
Also you will need to make sure that a trust relationship with all the accounts are present in Role A's Trust Relationship policy document.
Attach Role A to the lambda functions you will be running. you can use serverless for development.
Now your lambda function has Role A attached to it and Role A has sts:AssumeRole permissions over the role's created in the other accounts.
Assuming that you have created 1 function for 1 account in you lambda's code you will have to first use STS to switch to the role of the other account and obtain temporary credentials and pass these to S3 options before fetching the required data.
if you have created 1 function for all the accounts you can have the role ARN's in an array and iterate over it, again when doing this be aware of the limits of AWS lambda.
This is the policy I have:
{
"Version": "2012-10-17",
"Id": "Policy1477084949492",
"Statement": [
{
"Sid": "Stmt1477084932198",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::__redacted__"
},
{
"Sid": "Stmt1477084947291",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::__redacted__/*"
}
]
}
I am able to view the files in the bucket via aws s3 ls but am not able to download.
My understanding is that these permissions should give full access to any AWS identity.
Question- Is there some reason that is not the case here?
Your policy works for me when I test it in my account.
In IAM, a deny overwrites an allow, and I suspect that you have a conflicting policy somewhere. Check all user policies, and groups that the user is a member of for conflicting policies.
You don't explicitly say you are doing this, but just to cover all bases. If you are running the s3 get on an instance with an IAM Role associated with it, check to make sure that the IAM Roles permissions are appropriate.
Depending on what you are actually doing this could explain your situation. If you are using an EC2 instance with an IAM Role, it will be using that IAM Role for permissions by default not your IAM User permissions. If you run aws configure and explicitly configure it with IAM User issued key and secret then it will use the IAM User policies.
Best practices say that if you are performing work on an EC2 instance, where possible and where your use case allows for it; you should not be using keys and secrets on the host but using an EC2 IAM Role.
Additional Reading:
IAM Policy Evaluation Logic
http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html
I am trying to launch an AWS EMR Cluster from the AWS Console, and am getting the following error:
Failed to provision ec2 instances because 'IAM Instance Profile "arn:aws:iam::553706642095:instance-profile/EMR_EC2_DefaultRole" has no associated IAM Roles
Any one know what this means and how to resolve it?
The following is the role policy:
{
"Statement": [
{
"Action": [
"cloudwatch:*",
"dynamodb:*",
"ec2:Describe*",
"elasticmapreduce:Describe*",
"rds:Describe*",
"s3:*",
"sdb:*",
"sns:*",
"sqs:*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Its trust policy document is:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
I finally resolved this issue. This was confusing because the instance-profile and the role use the same name by default. Full steps outline below, but you may be able to skip various steps.
Create default roles (if error, downgrade to awscli version 1.10.30)
aws emr create-default-roles
Create instance profile if it doesn't already exist:
aws iam create-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Verify that instance profile exists but doesn't have any roles:
aws iam get-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Add the role using:
aws iam add-role-to-instance-profile --instance-profile-name EMR_EC2_DefaultRole --role-name EMR_EC2_DefaultRole
You have only readonly permission for EMR
"elasticmapreduce:Describe*",
You need to give full access to elastic map reduce so that you can launch cluster/terminate
once you give this access role policy will look like
"elasticmapreduce:*",
I tried around and could get it to work without the tool using my own Cloudformation stack.
The key you have to have a InstanceProfile for the flow role and both flow and service role have to be provided as ARN.
That's how I got it to work for me!
Hope that helps someone else as well.
I got the same issue. Instead of giving new cluster name, i just kept the same default cluster name 'My Cluster' and clicked on 'Create cluster' again. It created without this error.
I am looking to lock down an S3 bucket for security purposes - i'm storing deployment images in the bucket.
What I want to do is create a bucket policy that supports anonymous downloads over http only from EC2 instances in my account.
Is there a way to do this?
An example of a policy that I'm trying to use (it won't allow itself to be applied):
{
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[my bucket name]",
"Condition": {
"ArnEquals": {
"aws:SourceArn": "arn:aws:ec2:us-east-1:[my account id]:instance/*"
}
}
}
]
}
Just to clarify how this is normally done. You create a IAM policy, attach it to a new or existing role, and decorate the ec2 instance with the role. You can also provide access through bucket policies, but that is less precise.
Details below:
S3 buckets are default deny except for my the owner. So you create your bucket and upload the data. You can verify with a browser that the files are not accessible by trying https://s3.amazonaws.com/MyBucketName/file.ext. Should come back with error code "Access Denied" in the xml. If you get an error code of "NoSuchBucket", you have the url wrong.
Create an IAM policy based on arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess. Starts out looking like the snip below. Take a look at the "Resource" key, and note that it is set to a wild card. You just modify this to be the arn of your bucket. You have to do one for the bucket and its contents so it becomes: "Resource": ["arn:aws:s3:::MyBucketName", "arn:aws:s3:::MyBucketName/*"]
Now that you have a policy, what you want to do is to decorate your instances with a IAM Role that automatically grants it this policy. All without any authentication keys having to be in the instance. So go to Role, create new role, make an Amazon EC2 role, find the policy you just created, and your Role is ready.
Finally you create your instance, and add the IAM role you just created. If the machine already has its own role, you just have to merge the two roles into a new one for the machine. If the machine is already running, it wont get the new role until you restart.
Now you should be good to go. The machine has the rights to access the s3 share. Now you can use the following command to copy files to your instance. Note you have to specify the region
aws s3 cp --region us-east-1 s3://MyBucketName/MyFileName.tgz /home/ubuntu
Please Note, the term "Security through obscurity" is only a thing in the movies. Either something is provably secure, or it is insecure.
I used something like
{
"Version": "2012-10-17",
"Id": "Allow only My VPC",
"Statement": [
{
"Sid": "Allow only My VPC",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject", "s3:ListBucket",
"Resource": [
"arn::s3:::{BUCKET_NAME}",
"arn::s3:::{BUCKET_NAME}/*"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "{VPC_ID}" OR "aws:sourceVpce": "{VPCe_ENDPOINT}"
}
}
}
]
}