Related
I'm trying to create a lambda fuction with cloudformation but it requires a lambda execution role - is there a way I can generate one using cloudformation?
Yes, CloudFormation can be used to create an IAM role. The lambda execution role is an IAM role like any other IAM role. The documentation for doing so shows this example:
MyRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument: Json
Description: String
ManagedPolicyArns:
- String
MaxSessionDuration: Integer
Path: String
PermissionsBoundary: String
Policies:
- Policy
RoleName: String
Tags:
- Tag
Then in the lambda, you reference it using a ref to the name of the role resource. Ex:
MyLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Role: !Ref MyRole
You can create an IAM role with a role policy where it will take region and account id from predefined AWS FloudFormation variables and assign it to lambda elements in cloud formation. please refer following example
"Resources": {
"AheadLambdaRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"RoleName": {
"Fn::Sub": "AHEADLambdaRole-${EnvName}"
},
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": [
"sts:AssumeRole"
],
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
}
}
],
"Version": "2012-10-17"
},
"Policies": [{
"PolicyDocument" : {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": {
"Fn::Sub": "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:*"
}
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
{ "Fn::Sub": "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/LambdaName:*"}
]
}
]
},
"PolicyName" : "NameOfInlinepolicy"
}]
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess",
"arn:aws:iam::aws:policy/AmazonSSMFullAccess"
],
"Path": "/"
}
}}
I have a lambda function created using the serverless framework that is running into an error when deploying it to a dev environment. Issue seems to be IAM related and isn't a perissions issue so much as AWS says it's a malformed policy. I've included the error message seen in the cloudformation console as well as the entire serverless.yml file. Was hoping some could help me fix it, I had it working on another account though have made changes like memory limit and specifying the resource for iam policy. Should add to that, I have confirmed the sqs queue referenced in the iam part does in fact exist and I've tried the exact arn without the opt:accID part.
Error message
The policy failed legacy parsing (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument;
Serverless.yaml file
provider:
name: aws
runtime: nodejs10.x
region: us-east-1
vpc:
securityGroupIds:
- ${ssm:/${opt:stage}/securityGroupIds}
subnetIds:
- ${ssm:/${opt:stage}/subnetIds}
iamRoleStatements:
- Effect: 'Allow'
Resource: arn:aws:sqs:us-east-1:{opt:accID}:influxdb_perf_mon
Action:
- 'sqs:*'
environment:
tsQueue: https://sqs.us-east-1.amazonaws.com/${opt:accID}/influxdb_perf_mon
functions:
perf:
handler: handler.perf
memorySize: 128 # in MB
events:
- sqs:
arn: arn:aws:sqs:us-east-1:{opt:accID}:influxdb_perf_mon
batchSize: 1
Cloudformation in S3
"IamRoleLambdaExecution": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Policies": [
{
"PolicyName": {
"Fn::Join": [
"-",
[
"timeseries-lambda",
"dev",
"lambda"
]
]
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup"
],
"Resource": [
{
"Fn::Sub": "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/timeseries-lambda-dev*:*"
}
]
},
{
"Effect": "Allow",
"Action": [
"logs:PutLogEvents"
],
"Resource": [
{
"Fn::Sub": "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/timeseries-lambda-dev*:*:*"
}
]
},
{
"Effect": "Allow",
"Resource": "arn:aws:sqs:us-east-1:100525853236:influxdb_perf_mon",
"Action": [
"sqs:*"
]
},
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": [
"arn:aws:sqs:us-east-1:{opt:accID}:influxdb_perf_mon"
]
}
]
}
}
],
"Path": "/",
"RoleName": {
"Fn::Join": [
"-",
[
"timeseries-lambda",
"dev",
{
"Ref": "AWS::Region"
},
"lambdaRole"
]
]
},
"ManagedPolicyArns": [
{
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
]
]
}
]
}
},
The line that defines the resource for the IAM premissions you have listed has a small typo and is missing the $:
Resource: arn:aws:sqs:us-east-1:{opt:accID}:influxdb_perf_mon
should be
Resource: arn:aws:sqs:us-east-1:${opt:accID}:influxdb_perf_mon
That may fix your issue
I redeployed your role in my sandabox account. I found that the issue comes from:
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": [
"arn:aws:sqs:us-east-1:{opt:accID}:influxdb_perf_mon"
]
}
Specifically, from {opt:accID}. Changing the Resource to (real account number) fixed the issue in my verification:
"arn:aws:sqs:us-east-1:324124214:influxdb_perf_mon"
Thus the question is, where does this policy come from? Its not listed in your Serverless.yaml provided in the question? Are you sure this is the most recent Serverless.yaml you use?
The AWS Lambda function pipeline I did just set up following the official AWS documentation (Building a Continuous Delivery Pipeline for a Lambda Application with AWS CodePipeline) fails for me on deploy stage (both source and build stages do succeed) with this message:
Action execution failed
Role arn:aws:iam::************:role/service-role/AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline
is invalid or cannot be assumed
(Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: ********-****-****-************)
I did try to edit the deploy stage and change the specified service role (WSCodePipelineServiceRole-eu-west-1-lambda-pipeline), an run again the pipeline, with no success... More, editing again the deploy stage after re-running the pipeline, I see the previous role is there again... ( solved myself: I was not saving changes... :-( )
This is my template.yml:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Outputs the time
Resources:
TimeFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
Runtime: nodejs10.x
CodeUri: ./
Events:
MyTimeApi:
Type: Api
Properties:
Path: /TimeResource
Method: GET
This is my buildspec.yml:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
build:
commands:
- npm install
- export BUCKET=lambda-pipeline-repo-bucket
- aws cloudformation package --template-file template.yml --s3-bucket $BUCKET --output-template-file outputtemplate.yml
artifacts:
type: zip
files:
- template.yml
- outputtemplate.yml
This is the WSCodePipelineServiceRole-eu-west-1-lambda-pipeline role policy JSON:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEqualsIfExists": {
"iam:PassedToService": [
"cloudformation.amazonaws.com",
"elasticbeanstalk.amazonaws.com",
"ec2.amazonaws.com",
"ecs-tasks.amazonaws.com"
]
}
}
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"opsworks:DescribeStacks",
"rds:*",
"devicefarm:GetRun",
"cloudformation:CreateChangeSet",
"autoscaling:*",
"codebuild:BatchGetBuilds",
"servicecatalog:ListProvisioningArtifacts",
"devicefarm:ScheduleRun",
"devicefarm:ListDevicePools",
"cloudformation:UpdateStack",
"servicecatalog:DescribeProvisioningArtifact",
"cloudformation:DescribeChangeSet",
"devicefarm:ListProjects",
"cloudformation:ExecuteChangeSet",
"sns:*",
"codedeploy:RegisterApplicationRevision",
"cloudformation:*",
"opsworks:DescribeDeployments",
"devicefarm:CreateUpload",
"cloudformation:DescribeStacks",
"codecommit:GetUploadArchiveStatus",
"cloudwatch:*",
"opsworks:DescribeInstances",
"cloudformation:DeleteStack",
"ecr:DescribeImages",
"ecs:*",
"ec2:*",
"codebuild:StartBuild",
"opsworks:DescribeApps",
"opsworks:UpdateStack",
"cloudformation:ValidateTemplate",
"codedeploy:CreateDeployment",
"codedeploy:GetApplicationRevision",
"codedeploy:GetDeploymentConfig",
"servicecatalog:CreateProvisioningArtifact",
"sqs:*",
"cloudformation:DeleteChangeSet",
"codecommit:GetCommit",
"servicecatalog:DeleteProvisioningArtifact",
"codedeploy:GetApplication",
"cloudformation:SetStackPolicy",
"codecommit:UploadArchive",
"s3:*",
"elasticloadbalancing:*",
"codecommit:CancelUploadArchive",
"devicefarm:GetUpload",
"elasticbeanstalk:*",
"opsworks:UpdateApp",
"opsworks:CreateDeployment",
"cloudformation:CreateStack",
"servicecatalog:UpdateProduct",
"codecommit:GetBranch",
"lambda:*",
"codedeploy:GetDeployment",
"opsworks:DescribeCommands"
],
"Resource": "*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "codestar-connections:UseConnection",
"Resource": "*"
}
]
}
Please feel free to ask for any additional information...
Please give me some direction, I'm out of my depth here...
UPDATE 1
After #omuthu answer, I did check my pipeline policy (AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline):
{
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "*",
"Condition": {
"StringEqualsIfExists": {
"iam:PassedToService": [
"cloudformation.amazonaws.com",
"elasticbeanstalk.amazonaws.com",
"ec2.amazonaws.com",
"ecs-tasks.amazonaws.com"
]
}
}
},
...
],
"Version": "2012-10-17"
}
and I see I already have iam:PassRole...
It is conditioned to services "cloudformation.amazonaws.com", "elasticbeanstalk.amazonaws.com", "ec2.amazonaws.com", "ecs-tasks.amazonaws.com"... Should I add some other services? Or better remove the condition?
UPDATE 2
I did try to remove the condition, with no different result.
UPDATE 3
#shariqmaws: I did export my pipeline (named "lambda-pipeline"); here it is:
{
"pipeline": {
"version": 1,
"artifactStore": {
"location": "codepipeline-eu-west-1-##########",
"type": "S3"
},
"roleArn": "arn:aws:iam::##########:role/service-role/AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline",
"name": "lambda-pipeline",
"stages": [
{
"name": "Source",
"actions": [
{
"inputArtifacts": [],
"region": "eu-west-1",
"name": "Source",
"configuration": {
"PollForSourceChanges": "true",
"BranchName": "master",
"RepositoryName": "lambda-pipeline-repo"
},
"outputArtifacts": [
{
"name": "SourceArtifact"
}
],
"runOrder": 1,
"actionTypeId": {
"provider": "CodeCommit",
"owner": "AWS",
"version": "1",
"category": "Source"
}
}
]
},
{
"name": "Build",
"actions": [
{
"inputArtifacts": [
{
"name": "SourceArtifact"
}
],
"region": "eu-west-1",
"name": "Build",
"configuration": {
"ProjectName": "lambda-pipeline-build"
},
"outputArtifacts": [
{
"name": "BuildArtifact"
}
],
"runOrder": 1,
"actionTypeId": {
"provider": "CodeBuild",
"owner": "AWS",
"version": "1",
"category": "Build"
}
}
]
},
{
"name": "Deploy",
"actions": [
{
"inputArtifacts": [
{
"name": "BuildArtifact"
}
],
"region": "eu-west-1",
"name": "Deploy",
"configuration": {
"TemplatePath": "BuildArtifact::outputtemplate.yml",
"ActionMode": "CHANGE_SET_REPLACE",
"ChangeSetName": "lambda-pipeline-changeset",
"Capabilities": "CAPABILITY_IAM",
"RoleArn": "arn:aws:iam::##########:role/service-role/AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline",
"StackName": "lambda-pipeline-stack"
},
"outputArtifacts": [],
"runOrder": 1,
"actionTypeId": {
"provider": "CloudFormation",
"owner": "AWS",
"version": "1",
"category": "Deploy"
}
}
]
}
]
},
"metadata": {
"updated": 1584382274.615,
"created": 1584382274.615,
"pipelineArn": "arn:aws:codepipeline:eu-west-1:##########:lambda-pipeline"
}
}
Can you find any issue with this pipeline? (and thanks for your help!)
UPDATE 4
#Shawn: I did read the document you pointed me to, and checked my configuration... I am in region eu-west-1, and it is enabled, as I see in my account settings:
Thanks for any comment...
Add iam:PassRole to your codepipeline policy as it tries to pass the role (attach the role) to lambda
Read more about it here
The problem is with the CloudFormation 'Deploy' action configuration:
"configuration": {
"TemplatePath": "BuildArtifact::outputtemplate.yml",
"ActionMode": "CHANGE_SET_REPLACE",
"ChangeSetName": "lambda-pipeline-changeset",
"Capabilities": "CAPABILITY_IAM",
"RoleArn": "arn:aws:iam::##########:role/service-role/AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline",
"StackName": "lambda-pipeline-stack"
},
The 'RoleArn' above is a role in target account that CloudFormation will assume to execute the stack. Think of this role as the identity that will actually be creating the resources you have defined in the CloudFormation template. Now since CloudFormation service need to assume this role, the trust policy of this role should be for Cloudformation and NOT CodePipeline as it currently is.
Correct Trust Policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "cloudformation.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Fix:
Create a new role for the CloudFormation service
Provide the permissions to this role that are required to create the resources defined in the template
Update the pipeline Action 'Deploy' in Stage 'Deploy' and set the RoleARN in configuration properties of this Action to the new role's ARN instead of "arn:aws:iam::##########:role/service-role/AWSCodePipelineServiceRole-eu-west-1-lambda-pipeline".
I am trying to create a CloudWatch rule that triggers on a schedule and executes a state machine (Step Functions). I'm using CloudFormation to create this, and everything creates fine except for the association of the IAM role used by the rule, to the rule itself. Here is what I mean:
Notice under 'Use Existing Role' it's blank.
Here is the CF template portion that deals with the rule and its role.
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Sub": "states.${AWS::Region}.amazonaws.com"
}
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
},
"CloudWatchStateMachineSDCEventRule": {
"Type":"AWS::Events::Rule",
"Properties": {
"Description":"CloudWatch trigger for the InSite Static Data Consumer",
"ScheduleExpression": "rate(5 minutes)",
"State":"ENABLED",
"Targets":[{
"Arn":{ "Ref" : "StateMachine"},
"Id":"StateMachineTargetId",
"RoleArn":{
"Fn::GetAtt": [
"SFInvoke",
"Arn"
]
}
}]
}
},
You want the SFInvoke role to show up on the Use existing role selector?
If that is the case, you need to set the Principal to events instead of states.
You're editing the event target in the screenshot above, not the step function. Principal defines the service that can assume the role, in your case that is the events service.
Try this for role creation:
"SFInvoke":{
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
},
"Policies": [
{
"PolicyName": "StepFunctionsInvoke",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"states:StartExecution"
],
"Resource": { "Ref" : "StateMachine"}
}
]
}
}
]
}
}
Probably the Yaml would be as:
Based on the Principal: as an Events Based Services and Actions: to start the execution of a StepFunctions State Machine.
AWSEventsInvokeStepFunctions:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: AWSEventsInvokeStepFunctions
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- states:StartExecution
Resource: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:*"
The Role which is now generic in nature can be applied to a CloudWatch Event Rule, gives Rule with the permissions to be able to start the Execution of a StepFunctions State Machine based on an Amazon S3 Event.
AmazonCloudWatchEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
requestParameters:
bucketName:
- !Ref EventBucket
Targets:
-
RoleArn: !GetAtt AWSEventsInvokeStepFunctions.Arn
Arn: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:MyStateMachine"
Id: !Sub "StepExecution"
You can probably check more on Start the Execution of State Machine based on Amazon S3 Event
I need a queue in my elastic beanstalk application and I therefore create the queue and the queue policy with this snippet in my .ebextensions/app.conf:
Resources:
BackgroundTaskQueue:
Type: "AWS::SQS::Queue"
AllowWorkerSQSPolicy:
Type: "AWS::SQS::QueuePolicy"
Properties:
Queues:
-
Ref: "BackgroundTaskQueue"
PolicyDocument:
Version: "2008-10-17"
Id: "PublicationPolicy"
Statement:
-
Sid: "Allow-Create-Task"
Effect: "Allow"
Principal:
AWS: "*"
Action:
- "sqs:SendMessage"
Resource:
Fn::GetAtt:
- "BackgroundTaskQueue"
- "Arn"
Unfortunately I cannot find a way to reference the Instance profile of my EC2 instances in the autoscaling group. (At the moment the queue is open to the world) I tried two approaches:
Reading the configuration:
Principal:
AWS:
Fn::GetOptionSetting:
OptionName: "IamInstanceProfile"
The OptionName is always retrieved from the aws:elasticbeanstalk:customoption namespace but the IamInstanceProfile is defined in the aws:autoscaling:launchconfiguration namespace as far as I know. -> No luck
Reading from the actual AWSEBAutoScalingLaunchConfiguration resource:
Principal:
AWS:
Fn::GetAtt:
- "AWSEBAutoScalingLaunchConfiguration"
- "IamInstanceProfile"
This approach fails cause the property IamInstanceProfile is not exposed.
Has anyone found a way to make such a policy work?
Does anyone know how to instruct GetOptionSetting to look in a different namespace?
Anyone found a way to GetAtt the instance profile?
You need setup the instances profile outside of eb environment. You can use 'aws iam' command to create policies, roles and instance profiles (http://docs.aws.amazon.com/cli/latest/reference/iam/index.html#cli-aws-iam), then specify the profile in option settings:
namespace: aws:autoscaling:launchconfiguration
option_name: IamInstanceProfile
value: your-instance-profile-name
If you are using eb_deployer, there is a self-contained way doing it:
Create a CloudFormation template to define your resources stack, e.g. config/my-resources.json:
{
"Outputs": {
"InstanceProfile": {
"Description": "defines what ec2 instance can do with aws resources",
"Value": { "Ref": "InstanceProfile" }
}
},
"Resources": {
"Role": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": ["ec2.amazonaws.com"]
},
"Action": ["sts:AssumeRole"]
}]
},
"Path": "/",
"Policies": [ {
"PolicyName": "S3Access",
"PolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*",
"s3:PutObject"
],
"Resource": "*"
}
]
}
}, {
"PolicyName": "SQSAccess",
"PolicyDocument": {
"Statement": [ {
"Effect": "Allow",
"Action": [
"sqs:ChangeMessageVisibility",
"sqs:DeleteMessage",
"sqs:ReceiveMessage",
"sqs:SendMessage"
],
"Resource": "*"
}]
}
}]
}
},
"InstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [ { "Ref": "Role" } ]
}
}
}
}
Add a "resources" section into your eb_deployer.yml
resources:
template: config/my-resources.json
capabilities:
- CAPABILITY_IAM
outputs:
InstanceProfile:
namespace: aws:autoscaling:launchconfiguration
option_name: IamInstanceProfile
In the above example we defined an instance profile with policies enable specific accesses to S3 and SQS. Then map the instance profile name (output of the template) to Elastic Beanstalk option settings.
Take a look at this: https://github.com/ThoughtWorksStudios/eb_deployer/wiki/Elastic-Beanstalk-Tips-and-Tricks#setup-instance-profile-for-your-ec2-instances