I am new to Serverless but take a look at this config file
provider:
name: aws
runtime: nodejs10.x
stage: dev
region: us-east-1
# you can add statements to the Lambda function's IAM Role here
# iamRoleStatements:
# - Effect: "Allow"
# Action:
# - "s3:ListBucket"
# Resource: { "Fn::Join" : ["", ["arn:aws:s3:::", { "Ref" : "ServerlessDeploymentBucket" } ] ] }
# - Effect: "Allow"
# Action:
# - "s3:PutObject"
# Resource:
# Fn::Join:
# - ""
# - - "arn:aws:s3:::"
# - "Ref" : "ServerlessDeploymentBucket"
# - "/*"
Does this resources refers to fnproject?
I looked at aws-resource-type but could not find reference information.
Fn::Join is an intrinsic function, and it appends a set of values into a single value, separated by the specified delimiter. If a delimiter is the empty string, the set of values are concatenated with no delimiter.
You can use different intrinsic functions inside join and pseudo parameters as well.
For more information check the official AWS documentation
Fn::Join is a function in CloudFormation to concatenate strings. Ref is another function to reference elements created in CloudFormation.
In you example, Fn::Join is used to concatenate strings to create the ARN of the S3 bucket, whose format is arn:aws:s3:::bucketname and is needed to give permissions to the bucket in Lambda.
Related
In my serverless.yml file, I want to be able to add iamRoleStatements from two differents files (this cannot change). So I tried doing it like this:
provider:
iamRoleStatements:
- ${file(__environments.yml):dev.iamRoleStatements, ''}
- ${file(custom.yml):provider.iamRoleStatements, ''}
Each of these files have an iamRoleStatements section.
__environments.yml:
dev:
iamRoleStatements:
- Effect: 'Allow'
Action: 'execute-api:Invoke'
Resource: '*'
custom.yml:
provider:
iamRoleStatements:
- Effect: "Allow"
Action:
- lambda:InvokeFunction
Resource:
- "*"
Individually, each of them works great. But when I try to run sls deploy with both of them, I encounter the following error:
iamRoleStatements should be an array of objects, where each object has Effect, Action / NotAction, Resource / NotResource fields. Specifically, statement 0 is missing the following properties: Effect, Action / NotAction, Resource / NotResource; statement 1 is missing the following properties: Effect, Action / NotAction, Resource / NotResource
I searched online and this appears to work for other sections of the serverless file such as resources:
# This works perfectly well.
resources:
- ${file(custom.yml):resources, ''}
- ${file(__environments.yml):resources, ''}
So I wonder if there is any solution to this or if it is something that is not currently supported by the Serverless Framework.
Thanks for your help.
You're going to have to jump through a few hoops to get there.
File Merge Limitations
The serverless framework allows file imports anywhere in the configuration but only merges resources and functions sections.
Your example:
provider:
iamRoleStatements:
- ${file(__environments.yml):dev.iamRoleStatements, ''}
- ${file(custom.yml):provider.iamRoleStatements, ''}
Results in an array of arrays like this:
{
"provider": {
"iamRoleStatements": [
[
{
"Effect": "Allow",
"Action": "execute-api:Invoke",
"Resource": "*"
}
],
[
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"*"
]
}
]
]
}
}
You might be able to submit a very small pull request to rectify this.
IAM Managed Policies using References
It might be possible to define each of your IAM roles as custom resources, and use the iamManagedPolicies provider config to point to each of those resources. Something like:
provider:
name: aws
iamManagedPolicies:
- Ref: DevIamRole
- Ref: CustomIamRole
resources:
- ${file(__environments.yml):resources, ''}
- ${file(custom.yml):resources, ''}
Of course you'd need to change the structure of those two files to be AWS::IAM::Role resources.
Custom IAM Role
The framework also gives you the option to take complete control, which is fully documented.
I hope this helps.
When creating ECS infrastructure we describe our Task Definitions with CloudFormation. We want to be able to dynamically pass environment variables as a parameter to the template. According to the docs, Environment has a KeyValuePair type, but CloudFormation parameters do not have this type.
We can not hardcode Environment variables to the template, because this template is used as a nested stack so environment variables will be dynamically passed inside it.
The only possible way I see so far is to pass all arguments as a CommaDelimitedList, and then somehow parse and map it using CloudFormation functions. I can Fn::Split every entity in key and value, but how to dynamically build an array of KeyValuePair in CloudFormation?
Or maybe there is an easier way, and I'm missing something? Thanks in advance for any ideas.
I know it's late and you have already found a workaround. However, the following is the closest I came to solve this. Still not completely dynamic as expected parameters have to be defined as placeholders. Therefore the maximum number of environment variables expected should be known.
The answer is based on this blog. All credits to the author.
Parameters:
EnvVar1:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
EnvVar2:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
EnvVar3:
Type: String
Description: A possible environment variable to be passed on to the container definition.
Should be a key-value pair combined with a ':'. E.g. 'envkey:envval'
Default: ''
Conditions:
Env1Exist: !Not [ !Equals [!Ref EnvVar1, '']]
Env2Exist: !Not [ !Equals [!Ref EnvVar2, '']]
Env3Exist: !Not [ !Equals [!Ref EnvVar3, '']]
Resources:
TaskDefinition:
ContainerDefinitions:
-
Environment:
- !If
- Env1Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar1]]
Value: !Select [1, !Split [":", !Ref EnvVar1]]
- !Ref "AWS::NoValue"
- !If
- Env2Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar2]]
Value: !Select [1, !Split [":", !Ref EnvVar2]]
- !Ref "AWS::NoValue"
- !If
- Env3Exist
-
Name: !Select [0, !Split [":", !Ref EnvVar3]]
Value: !Select [1, !Split [":", !Ref EnvVar3]]
- !Ref "AWS::NoValue"
You may want to consider using the EC2 Parameter Store to create secured key/value pairs, which is supported in CloudFormation, and can be integrated with ECS environments.
AWS Systems Manager Parameter Store
AWS Systems Manager Parameter Store provides secure, hierarchical
storage for configuration data management and secrets management. You
can store data such as passwords, database strings, and license codes
as parameter values. You can store values as plain text or encrypted
data. You can then reference values by using the unique name that you
specified when you created the parameter. Highly scalable, available,
and durable, Parameter Store is backed by the AWS Cloud. Parameter
Store is offered at no additional charge.
While Parameter Store has great security features for storing application secrets, it can also be used to store nonsensitive application strings such as public keys, environment settings, license codes, etc.
And it is supported directly by CloudFormation, allowing you to easily capture, store and manage application configuration strings which can be accessed by ECS.
This template allows you provide the Parameter store key values at stack creation time via the console or CLI:
Description: Simple SSM parameter example
Parameters:
pSMTPServer:
Description: SMTP Server URL eg [email-smtp.us-east-1.amazonaws.com]:587
Type: String
NoEcho: false
SMTPServer:
Type: AWS::SSM::Parameter
Properties:
Name: my-smtp-server
Type: String
Value: !Ref pSMTPServer
Any AWS runtime environment (EC2, ECS, Lambda) can easily securely retrieve the values. From the console side, there is great Parameter manager interface that maintains parameter version history. Its intergated with IAM, so permissions are controlled with standard IAM policy syntax:
{
"Action": [
"ssm:GetParameterHistory",
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:GetParametersByPath"
],
"Resource": [
"arn:aws:ssm:us-west-2:555513456471:parameter/smtp-server"
],
"Effect": "Allow"
},
{
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:us-west-2:555513456471:key/36235f94-19b5-4649-84e0-978f52242aa0a"
],
"Effect": "Allow"
}
Finally, this blog article shows a technique to read the permissions into a Dockerfile at runtime. They suggest a secure way to handle environment variables in Docker with AWS Parameter Store. For reference, I am including their Dockerfile here:
FROM grafana/grafana:master
RUN curl -L -o /bin/aws-env https://github.com/Droplr/aws-env/raw/master/bin/aws-env-linux-amd64 && \
chmod +x /bin/aws-env
ENTRYPOINT ["/bin/bash", "-c", "eval $(/bin/aws-env) && /run.sh"]
With that invocation, each of the parameters are available as an environment variable in the container. You app may or may not need a wrapper to read the parameters from the environment variables.
I was facing the same problem ,I needed to create a lambda resource with environment variables.
We decided to fix initial set of environment variable and keys name are also decided in advance.
So I added four parameters , and used Ref for values while keeping fixed keys name.
There is another way too - which may sound overkill but it allows to put whatever env. to the function you wish, no need to "predefine" how many env. variables, only restriction in sample below - can not use :::: or |||| inside value of the key. Key can't have such symbols by AWS docs already.
Gameplan:
Make an inline CF Lambda Function with code which accepts all env in any format you wish as a string and uses any code you want to use inside that function (i use JS with NodeJS env) and while it's your code, parse how you wish that string passing in and use aws-sdk to update the function. Call function once inside the CF template.
In this sample you pass in env as such string:
key1::::value1||||key2::::value2 If you need to use :::: or |||| in your value, of course update to some other divider.
Not big fan of running lambda for such task, yet i want to have option of passing in virtually any env. to the CF template and this works.
LambdaToSetEnvRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/CloudWatchLambdaInsightsExecutionRolePolicy
Policies:
- PolicyName: cloudwatch-logs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource:
- !Sub "arn:aws:logs:*:${AWS::AccountId}:log-group:*:*"
- !Sub "arn:aws:logs:*:${AWS::AccountId}:log-group:/aws/lambda-insights:*"
- PolicyName: trigger-lambda-by-cloud-events
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'lambda:UpdateFunctionConfiguration'
Resource:
- !GetAtt OriginalLambda.Arn
Tags:
- { Key: managed-by, Value: !Ref AWS::StackId }
LambdaToSetEnv:
Type: AWS::Lambda::Function
DeletionPolicy: Delete
Properties:
Code:
ZipFile: |
const response = require('cfn-response');
const aws = require('aws-sdk');
exports.handler = (event, context) => {
console.log(JSON.stringify({event, context}));
try {
if (event.RequestType === "Delete") {
response.send(event, context, response.SUCCESS, {RequestType: event.RequestType});
} else {
const client = new aws.Lambda({apiVersion: '2015-03-31'});
const Variables = {
"All": process.env.FunctionEnvVariables,
};
console.log('process.env.FunctionEnvVariables: ', process.env.FunctionEnvVariables);
if(process.env.FunctionEnvVariables){
process.env.FunctionEnvVariables.split('||||').forEach((pair) => {
if(pair && pair.trim() !== ''){
Variables[pair.split('::::')[0]] = pair.split('::::')[1];
}
})
}
const result = client.updateFunctionConfiguration({ FunctionName: process.env.LambdaToUpdateArn, Environment: { Variables } }, function (error, data){
console.log('data: ', data);
console.log('error: ', error);
if(error){
console.error(error);
response.send(event, context, response.ERROR, {});
} else {
response.send(event, context, response.SUCCESS, {});
}
});
}
} catch (e) {
response.send(event, context, response.ERROR, e.stack);
}
}
Role: !GetAtt LambdaToSetEnvRole.Arn
Handler: index.handler
Runtime: nodejs14.x
Timeout: '300'
Environment:
Variables:
LambdaToUpdateArn: !GetAtt OriginalLambda.Arn
FunctionEnvVariables: !Ref FunctionEnvVariables
LambdaCall:
DependsOn:
- OriginalLambda
- LambdaToSetEnv
Type: Custom::LambdaCallout
Properties:
ServiceToken: !GetAtt LambdaToSetEnv.Arn
I want to grant access to a group of users to perform certain operations on certain Lambda functions. My Lambdas are already tagged properly to allow this, for instance: "department:hr". Can I tie this together with IAM?
I have seen documentation on conditionals that allow comparison of ResourceTag\* to a value, but these do not seem to be available in the visual editor (which unfortunately I depend on) for Lambda functions.
I want something like this:
"Effect": "Allow",
"Action": [
"lambda:ListFunctions",
"lambda:ListVersionsByFunction",
"lambda:GetLayerVersion",
"lambda:GetEventSourceMapping",
"lambda:GetFunction",
"lambda:ListAliases",
"lambda:GetAccountSettings",
"lambda:GetFunctionConfiguration",
"lambda:GetLayerVersionPolicy",
"lambda:ListTags",
"lambda:ListEventSourceMappings",
"lambda:ListLayerVersions",
"lambda:ListLayers",
"lambda:GetAlias",
"lambda:GetPolicy"
],
"Resource": "*"
"Condition": {
"StringEquals": {
"lambda:ResourceTag/department": "hr"
}
I can't build this in the visual editor and I get syntax errors when I attempt it myself.
I don't believe that lambda:ResourceTag/${TagKey} is a context condition available for any lambda actions (REF: https://docs.aws.amazon.com/IAM/latest/UserGuide/list_awslambda.html).
With that said, incorrect use of context keys typically fails silently. Could you include the full statement? For example, in the above snippet, there is a missing } for the condition.
If your IAM users are tagged with department:hr and if they assume the below IAM role via console, they should be able to access the lambda functions that have been tagged with department:hr.
HRDepartmentLambdaFunctionsAccessRole:
Type: AWS::IAM::Role
Properties:
RoleName: "HRDepartmentLambdaFunctionsAccessRole"
AssumeRolePolicyDocument:
# Allow users in account X to perform operations on lambda functions
Statement:
- Effect: Allow
Principal:
AWS:
- "AWS_ACCOUNT_NUMBER"
Action:
- sts:AssumeRole
Condition:
StringEquals:
aws:PrincipalTag/department:
- hr
Path: /
Policies:
- PolicyName: AllowAccessToLambdaFunctionsInHRDepartment
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- lambda:ListFunctions
- lambda:ListVersionsByFunction
- lambda:GetLayerVersion
- lambda:GetEventSourceMapping
- lambda:GetFunction
- lambda:ListAliases
- lambda:GetAccountSettings
- lambda:GetFunctionConfiguration
- lambda:GetLayerVersionPolicy
- lambda:ListTags
- lambda:ListEventSourceMappings
- lambda:ListLayerVersions
- lambda:ListLayers
- lambda:GetAlias
- lambda:GetPolicy
Resource: '*'
Condition:
StringEquals:
lambda:ResourceTag/department: 'hr'
Ref: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html
Can I impose a restriction on an IAM policy to run on a specific account only? I have searched for documentation or examples online but could not find anything on it.
Edited:
There are multiple accounts and similar policies to implement, each with a different restriction. In such cases, to prevent any mixing up while implementation of policies; I want to ensure that there is a restriction that imposed on the policy that tells which AWS account this policy can live in.
CFT:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Policy for XYZ'
Resources:
XYZPolicy:
Type: "AWS::IAM::ManagedPolicy"
Properties:
Description: "Restrictions apply only to Account XYZ"
Path: "/"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- "cloudformation:*"
- "cloudtrail:*"
- "cloudwatch:*"
- "ec2:*"
- "sso:*"
- "s3:*"
Resource: "*"
Roles:
-
Ref: "XYZRole"
ManagedPolicyName: "XYZPolicy
By default, a policy only applies to the account it belongs to, although you can enable cross-account policies, so what you're trying to do is basically the default.
If you really want to do this anyways, you may be able try something along the lines of:
"Condition": {
"ArnEquals": {
"iam:PolicyArn": [
"arn:aws:iam::AWS-ACCOUNT-ID:policy/XYZPolicy"
]
}
}
I am tagging my resources using Tags in my cfn script:
"Tags" : [ { "Key" : "Owner", "Value" : "my name" },
{ "Key" : "Name", "Value" : "instance name" }
{ "Key" : "DateCreated", "Value" : <something goes here> }
],
I would like to create a tag with the current date as per the example above. Is it possible?
You can use a "custom resource" to generate a timestamp (or any other value).
Custom resources are a newish feature in CloudFormation (introduced around 2014) and allow you to basically call a lambda function to "create", "update" or "delete" a resource for which CloudFormation does not provide language support (can even be resources outside AWS).
I use custom resource a lot just to compute some values for use in other parts of the stack, for example to create "variables" that hold computed values (e.g. using !Join and similar functions) that I need to use often and would like to compute once.
You can easily use a custom resource to just generate a time stamp. Here is some example code that is very close to what I actually use in production:
Create the "resource" implementation
Resources:
ValueFunc:
Type: AWS::Lambda::Function
Properties:
Code:
ZipFile: >
var r = require('cfn-response');
exports.handler = function(ev, ctx) {
ev.ResourceProperties.Time = new Date().toISOString();
r.send(ev, ctx, r.SUCCESS, ev.ResourceProperties);
};
Handler: index.handler
Runtime: nodejs6.10
Timeout: 30
Role: !GetAtt ValueFunctionExecutionRole.Arn
ValueFunctionExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal: { Service: [ lambda.amazonaws.com ] }
Action: sts:AssumeRole
Policies:
- PolicyName:
Fn::Sub: "value-custom-res-${AWS::StackName}-${AWS::Region}"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: "arn:aws:logs:*:*:*"
- Effect: Allow
Action: cloudformation:DescribeStacks
Resource: "arn:aws:cloudformation:*:*:*"
Then wherever you want to generate a time stamp, you do something like this (Scheduled action example taken from here):
Create a custom resource that calculates its creation time
GetTimeThisTime:
Type: Custom::Value
Properties:
ServiceToken: !GetAtt ValueFunc.Arn
Read the created timestamp using the Time attribute
ScheduledActionUp:
Type: AWS::AutoScaling::ScheduledAction
Properties:
AutoScalingGroupName: !Ref WebServerGroup
DesiredCapacity: 2
StartTime: !GetAtt GetTimeThisTime.Time
Recurrence: "0 7 * * *"
You can generate multiple time stamps at different times of the stack creation by simply creating a new "custom value" that depends on the logical entity whose creation you want to time.
The advice by #Guy is correct, you can access the creation timestamp of the stack from the stack properties.
If you still need to specify tags as parameters then you can do it the following way. Currently the JSON syntax supports an extremely limited set of functions. Because of this the possibilities for dynamically modifying your templates are very tiny. The only way I see to introduce this the tag you want is by adding another parameter to the template itself. Depending on the way you initialize the stack, you can script the parameter to be specified dynamically or provide it in the web console.
For example, if you have this in your template:
"Parameters" : {
"CreationDate" : {
"Description" : "Date",
"Type" : "String",
"Default" : "2013-03-20 21:15:00",
"AllowedPattern" : "^\\d{4}(-\\d{2}){2} (\\d{2}:){2}\\d{2}$",
"ConstraintDescription" : "Date and time of creation"
}
},
You can later reference it using the Ref keyword in the tags like this:
"Tags" : [ { "Key" : "Owner", "Value" : "my name" },
{ "Key" : "Name", "Value" : "instance name" },
{ "Key" : "DateCreated", "Value" : { "Ref" : "CreationDate" } }
],
It is not trivial to automatically assign the current time if you create the stack from the AWS console, but if you use the CLI tools you can call cfn-create-stack like this:
cfn-create-stack MyStack --template-file My.template --parameters "CreationDate=$(date +'%F %T')"
Hope this helps!