ECS unable to assume role - amazon-web-services

From the console, I am invoking a lambda which submits a batch job. The batch job fails, indicating that ECS is unable to assume the role that is provided to execute the job definition.
For the role, I've added the lambda and ECS services.
The error message:
"ECS was unable to assume the role
'arn:aws:iam::749340585813:role/golfnow-invoke-write-progress' that
was provided for this task. Please verify that the role being passed
has the proper trust relationship and permissions and that your IAM
user has permissions to pass this role."
"TrainingJobRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "golfnow-invoke-write-progress",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com",
"ecs.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/"
}
}
The batch job:
"TrainingJob": {
"Type": "AWS::Batch::JobDefinition",
"Properties": {
"Type": "container",
"JobDefinitionName": {
"Fn::Sub": "c12e-golfnow-${Environment}-job"
},
"ContainerProperties": {
"Image": {
"Fn::Join": [
"",
[
"{{ image omitted }}",
{
"Ref": "AWS::Region"
},
".amazonaws.com/amazonlinux:latest"
]
]
},
"Vcpus": 2,
"Memory": 2000,
"Command": [
"while", "True", ";", "do", "echo", "'hello';", "done"
],
"JobRoleArn": {
"Fn::GetAtt": [
"TrainingJobRole",
"Arn"
]
}
},
"RetryStrategy": {
"Attempts": 1
}
}
},
"JobQueue": {
"Type": "AWS::Batch::JobQueue",
"Properties": {
"Priority": 1,
"ComputeEnvironmentOrder": [
{
"Order": 1,
"ComputeEnvironment": {
"Ref": "ComputeEnvironment"
}
}
]
}
}
Is the issue with the way it's being invoked? My user has admin privileges, so I don't think this is an issue with my user having insufficient permissions.

You have to add the principal "ecs-tasks.amazonaws.com" to the trust policy for the role that's submitting a Batch job (not "ecs.amazonaws.com").
Revised role:
"TrainingJobRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "golfnow-invoke-write-progress",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com",
"ecs-tasks.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/"
}
},

And for those who are writing CDK script in Java, while defining the TaskDefinition you don't have to explicitly provide any taskRole and executionRole. CDK will create appropriate Role for you.

You would need to add a trust policy to ECS to call the Batch service.
"Principal": {
"Service": [
"batch.amazonaws.com"
]
},

My issue was resolved by adding role name in the CDK script.
const ecsFargateServiceRole = new iam.Role(this, 'execution-role', {
assumedBy: new iam.ServicePrincipal('ecs-tasks.amazonaws.com'),
roleName: "execution-role"
});
ecsFargateServiceRole.addToPolicy(executionRolePolicy);

Related

SQS API: sqs:CreateQueue Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied on `amplify push` using Cloudformation

I'm implementing SQS fifo queue. I have to implement i using cloudformation template.
When I do amplify push, I get
Error
API: sqs:CreateQueue Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied
I've added SQS policy followed from aws docs
. Except for accountID, I'm using service in the "Principal" as "sqs.amazonaws.com".
My cloudformation looks like:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "SQS fifo queue",
"Parameters": {
"env": {
"Type": "String"
}
},
"Resources": {
"QueueExecutionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": {
"Fn::Join": [
"",
[
"queue-exec-role-",
{
"Ref": "env"
}
]
]
},
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sqs.amazonaws.com"
},
"Action": ["sts:AssumeRole"]
}
]
}
}
},
"SQSPolicy": {
"Type": "AWS::SQS::QueuePolicy",
"Properties": {
"Queues": [{ "Ref": "groupingQueue" }],
"PolicyDocument": {
"Statement": [
{
"Action": ["SQS:SendMessage", "SQS:ReceiveMessage"],
"Effect": "Allow",
"Resource": {
"Fn::GetAtt": ["groupingQueue", "Arn"]
},
"Principal": {
"Service": "sqs.amazonaws.com"
}
}
]
}
}
},
"groupingQueue": {
"Type": "AWS::SQS::Queue",
"Properties": {
"FifoQueue": "true",
"QueueName": {
"Fn::Join": [
"",
[
"grouping-queue-",
{
"Ref": "env"
},
".fifo"
]
]
}
}
}
},
"Outputs": {
"QueueURL": {
"Description": "URL of new Amazon SQS Queue",
"Value": { "Ref": "groupingQueue" }
},
"QueueARN": {
"Description": "ARN of new Amazon SQS Queue",
"Value": { "Fn::GetAtt": ["groupingQueue", "Arn"] }
},
"QueueName": {
"Description": "Name new Amazon SQS Queue",
"Value": { "Fn::GetAtt": ["groupingQueue", "QueueName"] }
}
}
}
I do not want to give AccountID in "Principal", That why used sqs service.
With this exact template, I get access denied on amplify push -y.
I was doing amplify push from server. When I pushed it from my local computer it worked.
Turns out the aws profile I set in server did not have sqs:CreateQueue permissions while my local had the administrator access.
So, I added administrator full access to my server user from console, did amplify push again and it worked smoothly.
PS: you don't need to give administrator permission, you can just give sqs:CreateQueue permission. I did it because I was testing.

CloudFormation removing AWS Cognito Lambda Triggers on update stack operations

I️ have noticed whenever a new CloudFormation stack change is deployed, my User Pool triggers are removed and have to be manually re-added within the AWS dashboard or programmatically. This is a bit of a concern as these triggers conduct some crucial operations with communication between Cognito and the backend system.
At first I️ thought it was the deployment framework we are using, but here is a barebones example of a CF template I️ was able to replicate it with:
Updated to reflect Lambda attachment to User Pool
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"UserPool": {
"Type": "AWS::Cognito::UserPool",
"Properties": {
"UserPoolName": "test",
"UsernameAttributes": [
"email"
],
"EmailVerificationMessage": "Your verification code is {####}.",
"EmailVerificationSubject": "Your verification code",
"Policies": {
"PasswordPolicy": {
"MinimumLength": 8,
"RequireLowercase": true,
"RequireNumbers": true
}
}
}
},
"UserPoolClient": {
"Type": "AWS::Cognito::UserPoolClient",
"Properties": {
"ClientName": "Test Client",
"UserPoolId": {
"Ref": "UserPool"
},
"ExplicitAuthFlows": [
"ALLOW_REFRESH_TOKEN_AUTH",
"ALLOW_USER_PASSWORD_AUTH",
"ALLOW_USER_SRP_AUTH"
],
"GenerateSecret": false
}
},
"PreSignUpHandlerLambdaFunction": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Role": "arn:aws:iam::...",
"Code": {
"S3Bucket": "code-bucket",
"S3Key": "code-bucket/functions.zip"
},
"Handler": "handlers/pre-sign-up.default",
"Runtime": "nodejs12.x",
"FunctionName": "test-preSignUpHandler",
"MemorySize": 1024,
"Timeout": 6
}
},
"PreSignUpHandlerCustomCognitoUserPool1": {
"Type": "Custom::CognitoUserPool",
"Version": 1,
"DependsOn": [
"PreSignUpHandlerLambdaFunction"
],
"Properties": {
"ServiceToken": "arn:aws:lambda:...",
"FunctionName": "test-preSignUpHandler",
"UserPoolName": "test",
"UserPoolConfigs": [
{
"Trigger": "PreSignUp"
}
]
}
}
}
}
I️ have dug into CloudWatch logs generated by the update, but nothing is transparent regarding the User Pool update and the removal of the triggers.
Has anyone else experienced this and are there any work-arounds?
This is the expected behavior of CloudFormation. When config drift is detected on stack update it will bring it back in line with your stack template. If you want to retain the changes you should specify the triggers in your CFN template. Be sure to grant cognito access in the resource policy:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "lambda-allow-cognito-my-function",
"Effect": "Allow",
"Principal": {
"Service": "cognito-idp.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:123456789012:function:my-function",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "123456789012"
},
"ArnLike": {
"AWS:SourceArn": "arn:aws:cognito-idp:us-east-1:123456789012:userpool/us-east-1_myUserPoolId"
}
}
}
]
}

Is this the correct way to write a CFT for lambda to be triggered by SQS?

Hello I recently made a CFT for my lambda and sqs (both on the same account/region) but I noticed that when I go to the aws console, it shows the SQS resource both on the "input side" and the "output side" of my lambda. Is this intentional? or did I just do my CFTs incorrectly. Below is the relevant JSON code.
"MyLambdaFunctionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "root",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
}
}
]
}
}
]
}
},
"OutputRouterEventSource": {
"Type": "AWS::Lambda::EventSourceMapping",
"Properties": {
"BatchSize": 10,
"Enabled" : true,
"EventSourceArn": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
},
"FunctionName": {
"Fn::GetAtt": [
"MyLambdaFunction",
"Arn"
]
}
}
},
"OutputRouterLambdaInvokePermission": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"Action": "lambda:InvokeFunction",
"Principal": "sns.amazonaws.com",
"SourceArn": {
"Fn::Sub": "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:test-${Environment}-my-queue"
},
"FunctionName": {
"Fn::Sub": "test-${Environment}-my-lambda"
}
}
}
It is normal to have SQS on both side in this case: SQS triggers your Lambda function (left side) and your Lambda function's role must have the permission to receive and delete messages from the queue (right side).
(I think your are mixing services in your OutputRouterLambdaInvokePermission resource where the principal is SNS and the source references the arn of a SQS queue, you can drop this resource as it is not needed for SQS triggers)

cloudwatchevent_rule default to latest version of lambda function

I am trying to automate the creation of a lambda function and cloudwatch rule for it. However it seems that the cloudwatchevent_rule ansible task requires a version id to attach itself to my lambda function. This is causing an error:
No target to arn:aws:lambda:us-east-
1:MYACCOUNTID:function:MYFUNCTIONNAME could be found on the rule
MYFUNCTIONNAME.
How can I change this so that the cloudwatch rule will always attach itself to the latest version of my lambda function:
- name: create cloudwatch rule
cloudwatchevent_rule:
name: 'name_for_rule'
region: "{{region}}"
description: 'trigger on new instance creation'
state: present
event_pattern: |-
{
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com"
],
"eventName": [
"RunInstances"
]
}
}
targets:
- id: "{{ lambda.configuration.version }}"
arn: "{{ lambda.configuration.function_arn }}"
I've configured a Lambda Function with CloudWatch rule triggering it. The following SAM Template also contains permission, policy and roles I require. Please ignore those if not required.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "AWS SAM template configuring lambda functions written in test package.",
"Resources": {
"OrchestratorTestLambdaFunction": {
"DependsOn": [
"LambdaPolicy"
],
"Type": "AWS::Lambda::Function",
"Properties": {
"Handler": "com.test.TestClass::orchestrateTestLambda",
"FunctionName": "OrchestratorTestLambda",
"Runtime": "java8",
"MemorySize": 256,
"Timeout": 60,
"Code": {
"S3Bucket": "BATS::SAM::CodeS3Bucket",
"S3Key": "BATS::SAM::CodeS3Key"
},
"Role": {
"Fn::GetAtt": [
"LambdaRole",
"Arn"
]
},
"Description": "Lambda reads from SQS provided in the cloud watch."
}
},
"LambdaRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": "LambdaRole",
"AssumeRolePolicyDocument": {
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
}
},
"LambdaPolicy": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "lambda_policy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:ReceiveMessage"
],
"Resource": [
{
"Fn::Sub": "arn:aws:sqs:eu-west-1:${AWS::AccountId}:TestUpdates"
}
]
},
{
"Sid": "",
"Action": [
"lambda:InvokeAsync"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*"
}
]
},
"Roles": [
{
"Ref": "LambdaRole"
}
]
}
},
"PermissionForEventsToInvokeLambda": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"FunctionName": {
"Ref": "OrchestratorTestLambdaFunction"
},
"Action": "lambda:InvokeFunction",
"Principal": "events.amazonaws.com",
"SourceArn": {
"Fn::GetAtt": [
"TestRule",
"Arn"
]
}
}
},
"TestRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"Name": "TestRule",
"Description": "Rule to Trigger OrchestratorTestLambdaFunction",
"ScheduleExpression": "rate(1 minute)",
"State": "ENABLED",
"Targets": [
{
"Arn": {
"Fn::GetAtt": [
"OrchestratorTestLambdaFunction",
"Arn"
]
},
"Id": "TestRuleV1",
"Input": {
"Fn::Sub": "{\"queueUrl\":\"https://sqs.eu-west-1.amazonaws.com/${AWS::AccountId}/TestUpdates\"}"
}
}
]
}
}
},
"Outputs": {
"StackArn": {
"Value": {
"Ref": "AWS::StackId"
},
"Description": "Use this as the stack_arn in your cloud_formation_deployment_stack override."
}
}
}
I've noticied that function_arn registered from lambda ansible module output is not consistent.
Some times is
"function_arn": "arn:aws:lambda:zone:account:function:name"
other time is:
"function_arn": "arn:aws:lambda:zone:account:function:name:version"
So I've construct the arn appending always the $LATEST version:
- cloudwatchevent_rule:
profile: "{{ aws_profile }}"
name: StartStop
schedule_expression: cron(* * * * ? *)
description: trigger my lambda
targets:
- id: StartStop
arn: "arn:aws:lambda:{{aws_zone}}:{{aws_account_id}}:function:{{lambdadeploy.configuration.function_name}}:$LATEST"

Attach policy to a IAM Role

Following cloudformation template gives error on line 9 :
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "Policy to allow send receive message from SQS Queue",
"Resources" : {
"MyPolicy" : {
"Type" : "AWS::IAM::Policy",
"Properties" : {
"PolicyName" : "CFUsers",
"Roles": [ { "arn:aws:iam::710161973367:role/Cognito_CFIAuth_Role" } ],
"PolicyDocument" : {
"Version" : "2012-10-17",
"Statement": [
{
"Sid": "Sid1482400105445",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::710161973367:role/Cognito_CFIAuth_Role"
},
"Action": [
"SQS:SendMessage",
"SQS:ReceiveMessage",
"SQS:DeleteMessage",
"SQS:GetQueueUrl"
],
"Resource": "arn:aws:sqs:ap-south-1:710161973367:CFI-Trace"
}
]
}
}
}
}
I want role Cognito_CFIAuth_Role to have message send/read/delete previleges on SQS queue CFI-Trace. How do I attach SQS operation privileges to IAM Role ?
First, Line 9 contains a JSON syntax error, the brackets {} around your Role string should be removed:
"Roles": [ "arn:aws:iam::710161973367:role/Cognito_CFIAuth_Role" ],
Second, AWS::IAM::Policy's Roles property accepts "The names of AWS::IAM::Roles to attach to this policy", not full ARNs, so your line should be:
"Roles": [ "Cognito_CFIAuth_Role" ],
You also need a missing closing bracket } at the end of your example.
With the AWS::IAM::Policy resource, you're creating an inline policy. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-policy.html explains that this takes a list of "The names of AWS::IAM::Roles, which I take to be the logical name of role resources defined within the same stack.
If you want to attach the policy to a preexisting role, you should use the ManagedPolicy type instead. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-managedpolicy.html#cfn-iam-managedpolicy-roles takes the name of the preexisting role(s).
Cloudformation type IAM::Policy is for Users and Groups. Roles and instance profiles are for ec2. You have conflated both ideas. If you have the role predefined in a different CFN then you use just an Instance Profile for your EC2 instance, if not you can create it too and then ref it
"RootInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ {
"arn:aws:iam::710161973367:role/Cognito_CFIAuth_Role"
} ]
}
}
or
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"SQSRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "root",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"SQS:SendMessage",
"SQS:ReceiveMessage",
"SQS:DeleteMessage",
"SQS:GetQueueUrl"
],
"Resource": "arn:aws:sqs:ap-south-1:710161973367:CFI-Trace"
}
]
}
}
]
}
},
"RootInstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [
{
"Ref": "SQSRole"
}
]
}
}
}
}
IAM Policy
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iam-policy.html
IAM role
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
Now there is also SQS Policy
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sqs-policy.html
You can also attach ManagedPolicyArns to CloudFormation resource type AWS::IAM::Role https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html#cfn-iam-role-managepolicyarns