I have serverless.yaml with many SNS topics that I need to build ARNs from names across accounts and regions, so I come with a custom prefix
custom:
snsTopicPrefix:
Fn::Join:
- ":"
- - "arn:aws:sns"
- !Ref "AWS::Region"
- !Ref "AWS::AccountId"
topic01: Topic-01-test
functions:
from-SNS:
handler: src/fromSNS/handler.fromSNS
events:
- sns:
arn: ${self:custom.snsTopicPrefix}:${self:custom.topic01}
topicName: ${self:custom.topic01}
however, when I deploy it I can not pass an error Trying to populate non string value into a string for variable ${self:custom.snsTopicPrefix}. Please make sure the value of the property is a string.
Related
I am trying to deploy my lambdas to AWS, but getting this error.
An error occurred: AssignOrderLambdaFunction - Unable to retrieve TopicName attribute for AWS::SNS::Topic, with error message Rate exceeded (Service: Sns, Status Code: 400, Request ID: 79f1648a-90e0-5ebf-bb34-3f0993f6ca08).
I did not change anything in the serverless.yml. I have used this serverless.yml for over two years and have had no issues. Please help if you have any info about this error.
BTW I am not an AWS expert, just using it :)
Here are some details:
I am using Lambda with NodeJs (Typescript)
Using SNS and SQS to pass events across my services.
My serverless.yml structure:
service:
name: myService
provider:
name: aws
runtime: nodejs14.x
memorySize: 1024
region: us-west-1
tracing:
apiGateway: true
lambda: true
excludeDevDependencies: true
iamRoleStatements:
- Effect: Allow
Action:
- sns:Publish
Resource:
- Fn::Join:
- ''
- - 'arn:aws:sns:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':'
- Fn::GetAtt:
- orderPaidTopic
- TopicName
environment:
ORDER_PAID_TOPIC: ${self:custom.topicsArn.orderPaid}
custom:
topics:
orderPaid: ${self:service.name}-orderPaid-${self:custom.stage}
topicsArn:
orderPaid:
Fn::Join:
- ''
- - 'arn:aws:sns:'
- Ref: AWS::Region
- ':'
- Ref: AWS::AccountId
- ':'
- Fn::GetAtt:
- orderPaidTopic
- TopicName
awsAccountId: ${ssm:${self:custom.stage}.aws.accountId~true}
acmCertificateName: ${ssm:${self:custom.stage}.domains.api.acm~true}
awsLambdaAuthorizer: ${ssm:${self:custom.stage}.aws.lambda.authorizer~true}
resources:
Resources:
orderPaidTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: ${self:custom.topics.orderPaid}
I figured out that I had various SNS topics in one serverless file or, better to say, had many SNS topics in one cloud formation stack. So what I did was, create three cloud formations (three serverless files), and I distributed them between three files
I am trying to create a Cloudwatch event rule which will use multiple Codepipelines as source and triggers target.
Parameters:
SourcePipeline:
Description: Name of Source codepipeline
Type: CommaDelimitedList
Default: 'test3, test4'
Resources:
PipelineTrigger:
Type: 'AWS::Events::Rule'
Properties:
EventPattern:
source:
- aws.codepipeline
resources: !Split
- ','
- !Sub
- 'arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${pipeline}'
- pipeline: !Join
- ',arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:'
- !Ref SourcePipeline
Expecting resources as below:
"resources": ["arn:aws:codepipeline:us-east-1:123:test3","arn:aws:codepipeline:us-east-1:123:test4"],
Any idea how to pass the list of names as a parameter?
#FYI Reference i am following Using Lists of ARNs
First, it should be !Ref SourcePipelines. Second you forgot about comma. you can't do this the way you want. This is because, the first parameter to join must be literal string, not any CloudFormation expression or function. So you have to hardcode your account id and region:
resources: !Split
- ','
- !Sub
- 'arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${pipeline}'
- pipeline: !Join
- ',arn:aws:codepipeline:us-east-1:12312321:'
- !Ref SourcePipelines
I am trying to define value in one of the attributes for Type: AWS::Lambda::EventSourceMapping
Below is my snippet (latest attempt):
FunctionName: #trigger the echo function previously defined
"Fn::Join": [":", [{"Fn::GetAtt" : ["LambdaName", "Arn"]}, "live" ]]
My cloud formation stack, however fails to deploy with following error:
"StatusReason": "Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined."
I have tried couple of variations with the brackets, however keep getting same error. What am I missing in the syntax?
P.S. I am defining this in a yaml file
My first question is whether LambdaName references something that can be used with GetAtt to provide an ARN. Otherwise, it could just be a formatting issue. I'm not sure AWS CFN can read the embedded curly brackets you're using to wrap the Fn::GetAtt.
Maybe one of these would work better?
FunctionName:
Fn::Join:
- ':'
- - Fn::GetAtt:
- LambdaName
- Arn
- "live"
Or
FunctionName: !Join [':', [!GetAtt LambdaName.Arn, 'live]]
LambdaSourceMapping:
Type: AWS::Lambda::EventSourceMapping
Properties:
Enabled: 'true'
EventSourceArn: <SQS ARN> or <Kinesis ARN> or <DynamoDb ARN>
FunctionName:
Fn::Join:
- ':'
- - Fn::GetAtt:
- LambdaName
- Arn
- 'live'
I’m trying to make a DynamoDB table, without having a name property in the .yml file so that it’s name by cloud formation, and export it’s name to python for access can I do that if so how?
My current idea is to to export the name as a ssm parameter but I’m not sure how.
You can tag it in the cloudformation template and get resources by tag in boto.
import boto3
client = boto3.client('rds')
custom_filter = [{
'Name':'tag:Owner',
'Values': ['user#example.com']}]
response = client.describe_instances(Filters=custom_filter)
(This code is mostly copied from https://stackoverflow.com/a/48073016/10553976)
And this would correspond to tagging instances with the following:
Tags:
-
Key: "Owner"
Value: "user#example.com"
Important: if you want to apply the tags to something other than the instances you would need to use a different method from the describe methods in the rds client.
I assume your question is related to AWS CloudFormation, because you mention .yml file.
You can report the name in the output section of your cloudformation template, if your python function is declared in another template. You can then use the describe-stack API or CLI to fetch the value from the output.
See :
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html and https://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stacks.html
If your python function is declared in the same template, you can just refer to your logical resource to get the name (as per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html)
For example,
The section that creates the table
MyTable:
Type: AWS::DynamoDB::Table
Description: your decsriptions
Properties:
... your properties ...
The section that refers to it (here is an example with AWS::AppSync::DataSource but it applies to any type of resources
MyTableDataSource:
Type: AWS::AppSync::DataSource
Properties:
...
DynamoDBConfig:
TableName:
Ref: MyTable
AwsRegion:
Fn::Sub: ${AWS::Region}
or to get the table ARN in a IAM Policy
Policies:
- PolicyName: mypolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:DeleteItem
- dynamodb:UpdateItem
- dynamodb:Query
- dynamodb:Scan
Resource:
- Fn::Join:
- ''
- - Fn::GetAtt:
- MyTable
- Arn
- '*'
According to your reply to my comment, you wish to add the DynamoDB table into the yml file. If the table and the lambda are in the same .yml, then you can simply do !Ref YourTable, inside the Lambda Environment variables.
Something like this:
YourLambda:
Type: AWS::Serverless::Function
Properties:
Environment:
Variables:
YourTableName: !Ref YourTable
You will also need to add a policy attached to the Lambda, under Properties -> Policies, and you can reference the table name there the same way.
However, if you wish to reference the name without moving the DynamoDB instance inside the .yml file, then you have to make it a static reference by making an entry in Parameter Store, and then referencing it like so (making sure your CFN has access to ssm:getParameter):
YourLambda:
Type: AWS::Serverless::Function
Properties:
Environment:
Variables:
YourTableName: '{{resolve:ssm:/PATH/TO/TABLENAME:1}}'
Here’s how I ended up doing it:
Python:
ssm = boto3.client('ssm')
resp = dict(ssm.get_parameter(Name='TableName', WithDecryption=False))
tableName = str(json.loads(json.dumps(resp['Parameter'],default=str))['Value'])
This is my cloudformation yaml file:
Resources:
DynamoTable:
Type: "AWS::DynamoDB::Table"
Properties:
AttributeDefinitions:
- AttributeName: A_Key
AttributeType: "S"
- AttributeName: Serial
AttributeType: "S"
KeySchema:
- AttributeName: A_Key
KeyType: HASH
- AttributeName: Serial
KeyType: RANGE
DynamoTableParameter:
Type: "AWS::SSM::Parameter”
Properties:
Name: "TableName”
Type: String
Value: !Ref DynamoTable
I want to build an ARN in my file dynamically, but I need to get my current AccountId. How can I access it as a variable?
For example:
example: arn:aws:states:${region}:${accountId}:stateMachine:${self:service}-${self:custom.stage}-example
What is the proper way to reference the current region and accountId?
This is now supported natively in Serverless Framework.
functions example
functions:
hello:
handler: my-function.handler
environment:
var: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/*:*:*'
iam role example
iam:
role:
statements:
- Effect: Allow
Action:
- dynamodb:*
Resource: !Sub arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${AWS::StackName}-*
See Pseudo Parameters Reference for the official docs.
There's a handy serverless plugin https://www.npmjs.com/package/serverless-pseudo-parameters that adds the ability to reference aws parameters such as region and account id that i've just started using to much success.
This is now supported natively since version 2.3.0.
Just reference it via ${aws:accountId}. You can also reference region via ${aws:region}. Documentation here: https://www.serverless.com/framework/docs/providers/aws/guide/variables#referencing-aws-specific-variables
service: new-service
provider: aws
functions:
func1:
name: function-1
handler: handler.func1
environment:
ACCOUNT_ID: ${aws:accountId}
REGION: ${aws:region}
Serverless itself cannot reference those variables since those are defined within CloudFormation, but not exposed in serverless.
If you need those in the resources section you can directly access them via "Ref"-call.
AWS CloudFormation Pseudo-variables
If you need those variable as function environment variables, you can overwrite the serverless generated function code with CloudFormation code.
So to achieve this, you must modify you serverless.yml by the following pattern.
functions:
hello:
handler: handler.hello
resources:
Resources:
HelloLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Environment:
Variables:
accountId:
Ref: AWS::AccountId
region:
Ref: AWS::Region
arn:
Fn::Join:
- ""
- - "arn:aws:states:"
- Ref: AWS::Region
- ":"
- Ref: AWS::AccountId
- ":stateMachine:"
- ${self:service}
- "-"
- ${self:custom.stage}
- "-example"
EDIT: this question is probably outdated. Consider this comment and this answer.
AWS CloudFormation offers some variables like AWS::AccountIdand AWS::Region, but you can't use them in the serverless.yml file like ${AWS::AccountId}. Those are not supported.
#jens answer is right. You must use the CloudFormation syntax. In the example below, I provide another way to use CloudFormation.
service: testing-aws-account-id
provider:
name: aws
runtime: nodejs4.3
region: us-east-1
iamRoleStatements:
- Effect: "Allow"
Action:
- "iot:Publish"
Resource: 'Fn::Join: ["", [ "aws:iot:", { "Ref": "AWS::Region" }, ":", { Ref: "AWS::AccountId" }, ":topic/foo" ]]'
functions:
publishIot:
handler: handler.publishIot
The line:
Resource: 'Fn::Join: ["", [ "aws:iot:", { "Ref": "AWS::Region" }, ":", { Ref: "AWS::AccountId" }, ":topic/foo" ]]'
is the same of hard-coding the region and account id:
Resource: "arn:aws:iot:us-east-1:1234567890:topic/foo"