I need a queue in my elastic beanstalk application and I therefore create the queue and the queue policy with this snippet in my .ebextensions/app.conf:
Resources:
BackgroundTaskQueue:
Type: "AWS::SQS::Queue"
AllowWorkerSQSPolicy:
Type: "AWS::SQS::QueuePolicy"
Properties:
Queues:
-
Ref: "BackgroundTaskQueue"
PolicyDocument:
Version: "2008-10-17"
Id: "PublicationPolicy"
Statement:
-
Sid: "Allow-Create-Task"
Effect: "Allow"
Principal:
AWS: "*"
Action:
- "sqs:SendMessage"
Resource:
Fn::GetAtt:
- "BackgroundTaskQueue"
- "Arn"
Unfortunately I cannot find a way to reference the Instance profile of my EC2 instances in the autoscaling group. (At the moment the queue is open to the world) I tried two approaches:
Reading the configuration:
Principal:
AWS:
Fn::GetOptionSetting:
OptionName: "IamInstanceProfile"
The OptionName is always retrieved from the aws:elasticbeanstalk:customoption namespace but the IamInstanceProfile is defined in the aws:autoscaling:launchconfiguration namespace as far as I know. -> No luck
Reading from the actual AWSEBAutoScalingLaunchConfiguration resource:
Principal:
AWS:
Fn::GetAtt:
- "AWSEBAutoScalingLaunchConfiguration"
- "IamInstanceProfile"
This approach fails cause the property IamInstanceProfile is not exposed.
Has anyone found a way to make such a policy work?
Does anyone know how to instruct GetOptionSetting to look in a different namespace?
Anyone found a way to GetAtt the instance profile?
You need setup the instances profile outside of eb environment. You can use 'aws iam' command to create policies, roles and instance profiles (http://docs.aws.amazon.com/cli/latest/reference/iam/index.html#cli-aws-iam), then specify the profile in option settings:
namespace: aws:autoscaling:launchconfiguration
option_name: IamInstanceProfile
value: your-instance-profile-name
If you are using eb_deployer, there is a self-contained way doing it:
Create a CloudFormation template to define your resources stack, e.g. config/my-resources.json:
{
"Outputs": {
"InstanceProfile": {
"Description": "defines what ec2 instance can do with aws resources",
"Value": { "Ref": "InstanceProfile" }
}
},
"Resources": {
"Role": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": ["ec2.amazonaws.com"]
},
"Action": ["sts:AssumeRole"]
}]
},
"Path": "/",
"Policies": [ {
"PolicyName": "S3Access",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*",
"s3:PutObject"
],
"Resource": "*"
}
]
}
}, {
"PolicyName": "SQSAccess",
"PolicyDocument": {
"Statement": [ {
"Effect": "Allow",
"Action": [
"sqs:ChangeMessageVisibility",
"sqs:DeleteMessage",
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "*"
}]
}
}]
}
},
"InstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ { "Ref": "Role" } ]
}
}
}
}
Add a "resources" section into your eb_deployer.yml
resources:
template: config/my-resources.json
capabilities:
- CAPABILITY_IAM
outputs:
InstanceProfile:
namespace: aws:autoscaling:launchconfiguration
option_name: IamInstanceProfile
In the above example we defined an instance profile with policies enable specific accesses to S3 and SQS. Then map the instance profile name (output of the template) to Elastic Beanstalk option settings.
Take a look at this: https://github.com/ThoughtWorksStudios/eb_deployer/wiki/Elastic-Beanstalk-Tips-and-Tricks#setup-instance-profile-for-your-ec2-instances
Related
I'm trying to create a lambda fuction with cloudformation but it requires a lambda execution role - is there a way I can generate one using cloudformation?
Yes, CloudFormation can be used to create an IAM role. The lambda execution role is an IAM role like any other IAM role. The documentation for doing so shows this example:
MyRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument: Json
Description: String
ManagedPolicyArns:
- String
MaxSessionDuration: Integer
Path: String
PermissionsBoundary: String
Policies:
- Policy
RoleName: String
Tags:
- Tag
Then in the lambda, you reference it using a ref to the name of the role resource. Ex:
MyLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Role: !Ref MyRole
You can create an IAM role with a role policy where it will take region and account id from predefined AWS FloudFormation variables and assign it to lambda elements in cloud formation. please refer following example
"Resources": {
"AheadLambdaRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": {
"Fn::Sub": "AHEADLambdaRole-${EnvName}"
},
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": [
"sts:AssumeRole"
],
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
}
}
],
"Version": "2012-10-17"
},
"Policies": [{
"PolicyDocument" : {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": {
"Fn::Sub": "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*"
}
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
{ "Fn::Sub": "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/LambdaName:*"}
]
}
]
},
"PolicyName" : "NameOfInlinepolicy"
}]
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess",
"arn:aws:iam::aws:policy/AmazonSSMFullAccess"
],
"Path": "/"
}
}}
I created an SQS queue and added policy under permission tab allowing only my account users to configure the configure the notification
Policy Document
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:us-east-1:111111111111:sqsqueue/SQSDefaultPolicy",
"Statement": [
{
"Sid": "Sid111111111111",
"Effect": "Allow",
"Principal": {
"AWS": "111111111111"
},
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage"
],
"Resource": "arn:aws:sqs:us-east-1:111111111111:queue"
}
]
Navigate to S3 and try to configure event notification for the above queue, it is throwing an error
Unable to validate the following destination configurations. Permissions on the destination queue do not allow S3 to publish
notifications from this bucket.
(arn:aws:sqs:us-east-1:111111111111:queue)*
am I doing something wrong? Can someone help me please
I was able to resolve this issue by adding "Service": "s3.amazonaws.com"
in the Principal tag.
Here the policy document
{
"Version": "2012-10-17",
"Id": "arn:aws:sqs:us-east-1:111111111111:sqsqueue/SQSDefaultPolicy",
"Statement": [
{
"Sid": "Sid111111111111",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage"
],
"Resource": "arn:aws:sqs:us-east-1:111111111111:queue"
}
]
This is explained in https://forums.aws.amazon.com/thread.jspa?threadID=173251
This template file creates a bucket, SQS Queue and a policy to connect the two:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
IncomingBucketName:
Type: 'String'
Description: 'Incoming Bucket Name'
Default: 'some-bucket-name-here'
Resources:
IncomingFileQueue:
Type: 'AWS::SQS::Queue'
Properties: {}
SQSQueuePolicy:
Type: 'AWS::SQS::QueuePolicy'
Properties:
PolicyDocument:
Id: 'MyQueuePolicy'
Version: '2012-10-17'
Statement:
- Sid: 'Statement-id'
Effect: 'Allow'
Principal:
AWS: "*"
Action: 'sqs:SendMessage'
Resource:
Fn::GetAtt: [ IncomingFileQueue, Arn ]
Queues:
- Ref: IncomingFileQueue
IncomingFileBucket:
Type: 'AWS::S3::Bucket'
DependsOn:
- SQSQueuePolicy
- IncomingFileQueue
Properties:
AccessControl: BucketOwnerFullControl
BucketName:
Ref: IncomingBucketName
NotificationConfiguration:
QueueConfigurations:
- Event:
s3:ObjectCreated:Put
Queue:
Fn::GetAtt: [ IncomingFileQueue, Arn ]
I was getting the same issue but used this page to work out how to connect the three resources in order to successfully deploy the stack:
https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/
I'm still working on the Policy Condition as the form recommended in the above link doesn't work for SQS. That being the case, the above template is not secure and shouldn't be used in production as it allows anyone to add messages to the queue.
I'll update this answer once I've figured that bit out...
i have a month developing alexa skills and want to create then via Cloudformation. And for that i am using this:
Lambda function
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Lambda Function from Cloud Formation by Felix Vazquez",
"Resources": {
"Lambda1": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "felix-lambda-code",
"S3Key": "hello_lambda.zip"
},
"Description": "Test with Cloud Formation",
"FunctionName": "Felix-hello-world1234",
"Handler": "lambda_function.lambda_handler",
"Role": "arn:aws:iam::776831754616:role/testRol",
"Runtime": "python2.7"
}
}
}
}
Alexa Skill
"Resources": {
"23LT3": {
"Type": "Alexa::ASK::Skill",
"Properties": {
"AuthenticationConfiguration": {
"ClientId": "+my client ID+",
"ClientSecret": "+my client Secret+",
"RefreshToken": "+The token i generate via lwa+"
},
"VendorId": "+my vendor ID+",
"SkillPackage": {
"S3Bucket": "myskillpackagebucket",
"S3Key": "my_function10.zip",
"S3BucketRole": {
"Fn::GetAtt": [
"IAMRU6TJ",
"Arn"
]
},
"Overrides": {
"Manifest": {
"apis": {
"custom": {
"endpoint": {
"uri": {
"Fn::GetAtt": [
"Lambda1",
"Arn"
]
}}}}}}}}
IAM Role
{
"Resources": {
"IAMRU6TJ": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"s3.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "root",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}]}}]}}}}
The skill depends on the lambda and the IAM Role. When i "Create the Stack" after some seconds it gives me this error:
Could not assume the provided role. Reason: Access denied (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: b2e8762c-2593-11e9-b3ec-872599411915)
For the Token i use
ask util generate-lwa-tokens --scope "alexa::ask:skills:readwrite alexa::ask:models:readwrite profileā
image of the events:
Event after execution
your Alexa::ASK::Skill Resource: 23LT3['Properties']['SkillPackage']['S3BucketRole']
The docs say
ARN of the role that grants the Alexa service permission to access the bucket and retrieve the skill package. This role is optional, and if not provided the bucket must be configured with a policy allowing this access, or be publicly accessible, in order for AWS CloudFormation to create the skill.
currently your role is allowing s3.amazonaws.com and lambda.amazonaws.com to Assume a role that can do anything in your AWS account, however you need to allow "The Alexa Service the permission..."
Best Practice would be to use least privilege necessary, but I get it if you are just testing it out.
I struggled to find the details necessary documented anywhere. Here is the role I used to get this working.
AlexaReadRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- alexa-appkit.amazon.com
Sid: AllowServiceToAssumeRole
Version: 2012-10-17
Policies:
- PolicyName: "AlexaS3Read"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action: "s3:GetObject"
Resource: "arn:aws:s3:::<bucket-name>/<path-to-alexa-files>/*"
Type: AWS::IAM::Role
I am trying to create a CloudWatch rule that triggers on a schedule and executes a state machine (Step Functions). I'm using CloudFormation to create this, and everything creates fine except for the association of the IAM role used by the rule, to the rule itself. Here is what I mean:
Notice under 'Use Existing Role' it's blank.
Here is the CF template portion that deals with the rule and its role.
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Sub": "states.${AWS::Region}.amazonaws.com"
}
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
},
"CloudWatchStateMachineSDCEventRule": {
"Type":"AWS::Events::Rule",
"Properties": {
"Description":"CloudWatch trigger for the InSite Static Data Consumer",
"ScheduleExpression": "rate(5 minutes)",
"State":"ENABLED",
"Targets":[{
"Arn":{ "Ref" : "StateMachine"},
"Id":"StateMachineTargetId",
"RoleArn":{
"Fn::GetAtt": [
"SFInvoke",
"Arn"
]
}
}]
}
},
You want the SFInvoke role to show up on the Use existing role selector?
If that is the case, you need to set the Principal to events instead of states.
You're editing the event target in the screenshot above, not the step function. Principal defines the service that can assume the role, in your case that is the events service.
Try this for role creation:
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
}
Probably the Yaml would be as:
Based on the Principal: as an Events Based Services and Actions: to start the execution of a StepFunctions State Machine.
AWSEventsInvokeStepFunctions:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: AWSEventsInvokeStepFunctions
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- states:StartExecution
Resource: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*"
The Role which is now generic in nature can be applied to a CloudWatch Event Rule, gives Rule with the permissions to be able to start the Execution of a StepFunctions State Machine based on an Amazon S3 Event.
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
requestParameters:
bucketName:
- !Ref EventBucket
Targets:
-
RoleArn: !GetAtt AWSEventsInvokeStepFunctions.Arn
Arn: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:MyStateMachine"
Id: !Sub "StepExecution"
You can probably check more on Start the Execution of State Machine based on Amazon S3 Event
I am trying to create a new project in AWS CodeBuild. Every time I attempt to I receive the following error:
Not authorized to perform DescribeSecurityGroups
Any help would be greatly appreciated.
You are likely missing the VPC related permissions in your service role. You need to update the role to have the following policy:
https://docs.aws.amazon.com/codebuild/latest/userguide/auth-and-access-control-iam-identity-based-access-control.html#customer-managed-policies-example-create-vpc-network-interface
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:DescribeDhcpOptions",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeVpcs"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterfacePermission"
],
"Resource": "arn:aws:ec2:{{region}}:{{account-id}}:network-interface/*",
"Condition": {
"StringEquals": {
"ec2:Subnet": [
"arn:aws:ec2:{{region}}:{{account-id}}:subnet/[[subnets]]"
],
"ec2:AuthorizedService": "codebuild.amazonaws.com"
}
}
}
]
}
It means that associated IAM Role doesn't have attached policy allowing CodeBuild to describe Security Groups.
If you trying to create a new Build project and have selected "New Service Role" (Create a service role in your account), and in the same time added VPC, Subnets and Security Groups in Additional Configuration section - you will get "Not authorized to perform DescribeSecurityGroups" error.
For some reason AWS auto-created policy looks like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"ssm:GetParameters",
"logs:PutLogEvents",
"logs:CreateLogStream",
"logs:CreateLogGroup",
"ecr:UploadLayerPart",
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:GetAuthorizationToken",
"ecr:CompleteLayerUpload",
"ecr:BatchCheckLayerAvailability"
],
"Resource": "*"
}
]
}
It's not allowing anything VPC/EC2 related, so you can either pre-create correct policy and use it, or let AWS create project without VPC, and modify new policy by adding required services in "Action" block:
"Action": [
"ssm:GetParameters",
"logs:PutLogEvents",
"logs:CreateLogStream",
"logs:CreateLogGroup",
"ecr:UploadLayerPart",
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:GetAuthorizationToken",
"ecr:CompleteLayerUpload",
"ecr:BatchCheckLayerAvailability",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets"
],
I had this same issue when using cloudformation. The issue was the IAM role was being created before CodeBuild started creation, but the Policy attached the IAM role was being created after CodeBuild was created.
The remedy for this was to add a DependsOn to CodeBuild saying it needs the Policy to be created first.
Ex:
CodeBuildIamRole:
Type: 'AWS::IAM::Role'
Properties:
RoleName: 'CodeBuildAutomatedTestingRole'
AssumeRolePolicyDocument:
Statement:
- Action: 'sts:AssumeRole'
Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Path: /
CodeBuildIamPolicy:
Type: 'AWS::IAM::Policy'
Properties:
PolicyName: !Sub 'CodeBuildServiceRolePolicy-${AWS::StackName}'
PolicyDocument:
Statement:
- Action:
- 's3:PutObject'
- 's3:GetObject'
- 's3:GetObjectVersion'
- 's3:ListBucket'
Effect: Allow
Resource: '*'
- Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'ec2:CreateNetworkInterface'
- 'ec2:DescribeDhcpOptions'
- 'ec2:DescribeNetworkInterfaces'
- 'ec2:DeleteNetworkInterface'
- 'ec2:DescribeSubnets'
- 'ec2:DescribeSecurityGroups'
- 'ec2:DescribeVpcs'
- 'ec2:CreateNetworkInterfacePermission'
- 'ecr:*'
- ...
Effect: Allow
Resource:
- '*'
Roles:
- !Ref CodeBuildIamRole
CodeBuild:
DependsOn:
- CodeBuildIamPolicy
Type: "AWS::CodeBuild::Project"
Properties:
...
Hopefully this is helpful