Fairly new to Serverless and am having problems creating a dynamic path to an SSM parameter..... I have tried a fair few ideas but am sure that this is really close but its not quite there....
I'm trying to generate an ssm path as a custom variable that will then be used to populate a value for a Lambda function.
Here's the custom variable code
custom
securityGroupSsmPath:
dev: "${self:service}/${self:custom.stage}/rds/lambdasecuritygroup"
other: "${self:service}/${env:SHARED_INFRASTRUCTURE_ENV}/rds/lambdasecuritygroup"
securityGroupId: ${ssm:, "${self:custom.securityGroupSsmPath.${env:SHARED_INFRASTRUCTURE_ENV}, self:custom.securityGroupSsmPath.other}"}
And here is where it is referenced in the function
functions:
someLambda:
handler: build/handlers/someLambda/handler.handler
timeout: 60
memorySize: 256
vpc:
securityGroupIds:
- ${self:custom.securityGroupId}
And here is the error output. It seems like it is not resolving the ssm parameter
Serverless Error ----------------------------------------
Cannot resolve serverless.yml: Variables resolution errored with:
- Cannot resolve variable at "custom.securityGroupId": Parameter name: can't be prefixed with "ssm" (case-insensitive). If formed as a path, it can consist of sub-paths divided by slash symbol; each sub-path can be formed as a mix of letters, numbers and the following 3 symbols .-_
All help much appreciated,
Thanks!
Sam
In the end we tried numerous implementations and the issue seemed to boil down to trying to both retrieve the ssm value for securityGroupId and also parse and default the second variable within it.
The solution ended up being as follows where we removed the parsing/default variable from within ssm step. Additionally we had to remove some of the double quotes on the custom vars:-
custom
securityGroupSsmPath:
dev: ${self:service}/${self:custom.stage}/rds/lambdasecuritygroup
other: ${self:service}/${env:SHARED_INFRASTRUCTURE_ENV}/rds/lambdasecuritygroup
securityGroupId: ${self:custom.securityGroupSsmPath.${env:SHARED_INFRASTRUCTURE_ENV}, self:custom.securityGroupSsmPath.other}
functions:
someLambda:
handler: build/handlers/someLambda/handler.handler
timeout: 60
memorySize: 256
vpc:
securityGroupIds:
- ${ssm:/${self:custom.securityGroupId}}
Related
I'm struggling with the YAML syntax to import a value that was exported by another CFN stack into the default value of a parameter in a new stack.
What I have at the moment is:
Parameters:
DBEndpoint:
Description: Hostname endpoint for RDS Database
Type: String
Default: Fn::ImportValue: 'db-endpoint'
Where db-endpoint is the value exported by the following YAML template snippet:
Outputs:
dbhost:
Description: "RDS Endpoint Address"
Value: !GetAtt DB.Endpoint.Address
Export:
Name: db-endpoint
The export works fine, but I get a parse error (Template format error: YAML not well-formed. ) when trying to load the template with the ImportValue line.
Update:
I have the YAML parsing correctly now, I think, but now get a new error.
With
Parameters:
DBEndpoint:
Description: Hostname endpoint for RDS Database
Type: String
Default: !ImportValue 'db-endpoint'
I get an error Template format error: Every Default member must be a string..
So, it seems closer, but still not working.
This answer implies this might not even be possible... is that the case?
!ImportValue 'db-endpoint' can't be used in Parameters. It can only be used in Resources and Outputs of your template. You have to "manually" (aka, outside of CloudFormation, e.g. by a wrapper script) set the default value of DBEndpoint to the actual value of your db-endpoint.
I am trying to pass a parameter through the aws cloudformation deploy command like this:
aws cloudformation deploy --template-file sam-template.packaged.yaml --parameter-overrides ExcTime=12345
But it fails deploying because it is complaining that the ExcTime parameter is a string and it needs to be a number. Is it the command casting all the parameters as string? And if so, how can I pass a parameter of type number to the YML file through that command?
The yml file:
ApiKey:
Type: AWS::AppSync::ApiKey
Properties:
ApiId: !GetAtt AppSyncApi.ApiId
Expires: ${ExcTime}
Thank You all!
To refer to your parameter, you should use:
Expires: !Ref ExcTime
So i figured out why. The --parameter-overrides Param=value only accepts strings
--parameter-overrides (list) A list of parameter structures that
specify input parameters for your stack template. If you're updating
a stack and you don't specify a parameter, the command uses the
stack's existing value. For new stacks, you must specify parameters
that don't have a default value. Syntax:
ParameterKey1=ParameterValue1 ParameterKey2=ParameterValue2 ...(string).
https://docs.aws.amazon.com/cli/latest/reference/cloudformation/deploy/index.html
To pass a number, You need to specify that the value is a number in the sam template like this:
Parameters:
YourParam:
Type: Number
Description: Bla bla bla
And then You can use it as a number in the template. For example
Expires: !Ref YourParam
I need to configure a lambda via serverless.yml to use different provision concurrency for different environments. Below is my lambda configuration:
myLambda:
handler: src/lambdas
name: myLambda
provisionedConcurrency: ${self:custom.pc}
...
custom:
pc: ${env:PC}
The value PC is loaded from environment variable. It works for values greater than 0 but I can't set a value 0 in one environment. What I want to do is to disable provision concurrency in dev environment.
I have read through this doc https://forum.serverless.com/t/conditional-serverless-yml-based-on-stage/1763/3 but it doesn't seem to help in my case.
How can I set provisionedConcurrency conditional based on environment?
Method 1: Stage-based variables via default values
This is a fairly simple trick by using a cascading value variable. The first value is the one you want, the second one being a default, or fallback value. Also called cascading variables.
// serverless.yml
provider:
stage: "dev"
custom:
provisionedConcurrency:
live: 100
staging: 50
other: 10
myLambda:
handler: src/lambdas
name: myLambda
provisionedConcurrency: ${self:custom.provisionedConcurrency.${self:provider.stage}, self:custom.provisionedConcurrency.other}
This above with stage set to dev will default to "other" value of 10, but if you set stage via serverless deploy --stage live then it will use the live value of 100.
See here for more details: https://www.serverless.com/framework/docs/providers/aws/guide/variables#syntax
Method 2: Asynchonous Value via Javascript
You can use an js include and put your conditional logic there. It's called "asynchronous value support". Basically, this allows you to put logic in a javascript file which you include and it can return different values depending on various things (like, what AWS account you're on, or if certain variables are set, or whatever). Basically, it allows you to do this...
provisionedConcurrency: ${file(./detect_env.js):get_provisioned_concurrency}
Which works if you create a javascript file in this folder called detect_env.js, and it has the contents similar to...
module.exports.get_provisioned_concurrency = () => {
if ("put logic to detect which env you are deploying to, eg for live") {
return Promise.resolve('100');
} else {
// Otherwise fallback to 10
return Promise.resolve('10');
}
}
For more info see: https://www.serverless.com/framework/docs/providers/aws/guide/variables#with-a-new-variables-resolver
I felt I had to reply here even though this was asked months ago because none of the answers were even remotely close to the right answer and I really felt sorry for the author or anyone who lands here.
For really sticky problems, I find it's useful to go to the Cloudformation script instead and use the Cloudformation Intrinsic Functions.
For this case, if you know all the environments you could use Fn::FindInMap
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-findinmap.html
Or if it's JUST production which needs 0 then you could use the conditional Fn::If and a boolean Condition test in the Cloudformation template to test if environment equals production, use 0, else use the templated value from SLS.
Potential SLS:
resources:
Conditions:
UseZero: !Equals ["production", ${provider.stage}]
Resources:
myLambda:
ProvisionedConcurrency: !If [UseZero, 0, ${self:custom.pc}]
You can explicitly remove the ProvisionedConcurrency property as well if you want:
resources:
Conditions:
UseZero: !Equals ["production", ${provider.stage}]
Resources:
myLambda:
ProvisionedConcurrency: !If [UseZero, AWS::NoValue, ${self:custom.pc}]
Edit: You can still use SLS to deploy; it simply compiles into a Cloudformation JSON template which you can explicitly modify with the SLS resources field.
The Serverless Framework provides a really useful dashboard tool with a feature called Parameters. Essentially what it lets you do is connect your service to it then you can set different values for different stages and then use those values in your serverless.yml with syntax like ${param:VARAIBLE_NANE_HERE} and it gets replaced at deploy time with the right value for whatever stage you are currently deploying. Super handy. There are also a bunch of other features in the dashboard such as monitoring and troubleshooting.
You can find out more about Parameters at the official documentation here: https://www.serverless.com/framework/docs/guides/parameters/
And how to get started with the dashboard here: https://www.serverless.com/framework/docs/guides/dashboard#enabling-the-dashboard-on-existing-serverless-framework-services
Just using a variable with a null value for dev environments during on deploy/package and SLS will skip this property:
provisionedConcurrency: ${self:custom.variables.provisionedConcurrency}
I am using Serverless framework.
I have created Cognito user pool in one stack and importing it into second stack.
When I assign the value of the Cognito User pool ID created in the first stack, to the environment variable it works.
But when I try to use it while creating a ARN for Cognito Authorizer, it doesn't work.
I get an error - Trying to populate non string value into a string for variable
Here is the snippet of my serverless.yml file.
service: myservice
plugins:
- serverless-pseudo-parameters
provider:
name: aws
runtime: go1.x
stage: dev
region: us-east-1
cognitoUserPoolId :
Fn::ImportValue: cloudformation-resources-${self:provider.stage}-CognitoUserPool
cognitoAppClientId :
Fn::ImportValue: cloudformation-resources-${self:provider.stage}-CognitoUserPoolClient
custom:
environment:
COGNITO_USER_POOL_ID : ${self:provider.cognitoUserPoolId}
COGNITO_APP_CLIENT_ID: ${self:provider.cognitoAppClientId}
functions:
myfunction:
handler: bin/handlers/myfunction
package:
exclude:
- "**/**"
include:
- ./bin/handlers/myfunction
events:
- http:
path: mypath
method: put
authorizer:
name: cognitoapiauthorizer
arn: arn:aws:cognito-idp:#{AWS::Region}:#{AWS::AccountId}:userpool/${self:provider.cognitoUserPoolId}
cors: true
Any issues related to indentation or the way it is being used in another variable like ARN?
I see you're trying to resolve the error Trying to populate non string value into a string for variable. In my experience, this means that the variable is empty. What happens if you hardcode the cognitoUserPoolId at the end of the ARN? Is the error resolved? I suspect it would be. Moving forward from there, you should take a closer look at how you declare that variable. Your usage of Fn::ImportValue may not working as intended.
Also, I would definitely run your YAML through a validator. There are too many extra blank lines, and extra spaces before the colons e.g. COGNITO_USER_POOL_ID : ${self:provider.cognitoUserPoolId}. These may be causing problems. Keep your YAML formatting tidy.
I'm trying to access secrets created in secrets manager(https://aws.amazon.com/secrets-manager/) via SSM (Systems Manager- https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html ) i.e. AWS Parameter store, and store it in a custom YAML variable in serverless.yml file?
I am trying to implement cloud formation through serverless framework(https://serverless.com/), and I am trying to implement a nested if statement in cloud formation for implementing the above using the code below.
stage: &stage 'dev' #Hardcoded for now
rdsMasterPassword:
!If
- !Equals [*stage,"prod"]
- ${ssm:/aws/reference/secretsmanager/cred-prod~true:rdsMasterPassword}
- !If
- !Equals [*stage,"staging"]
- ${ssm:/aws/reference/secretsmanager/cred-staging~true:rdsMasterPassword}
- ${ssm:/aws/reference/secretsmanager/cred-dev~true:rdsMasterPassword}
I have tried Cloud formation instrinsic functions Fn::If for this but facing this errror :
Fn::If requires a list argument with the first element being a condition
Just want to point out that if you're looking to load different SSM paths based on environment, you can achieve this many ways, outlined here
I've had a pleasant time loading through json files, for example
-- serverless-staging.json --
{
"ssm_path": "/path/to/staging/ssm/parameter"
}
-- serverless-prod.json --
{
"ssm_path": "/path/to/prod/ssm/parameter"
}
-- serverless.yml --
...
stage: ${opt:stage, 'dev'}
environment:
SSM_PATH: ${file(serverless-${self:provider.stage}.json):ssm_path}
... etc etc
Hope this helps whoever else lands here from a search
Due to a restriction in YAML, it is not possible to use the shortcut syntax for a sequence of intrinsic functions.
See the "Important" section in the docs for reference.
Try this:
stage: &stage 'dev' #Hardcoded for now
rdsMasterPassword:
Fn::If:
- Fn::Equals: [*stage, "prod"]
- ${ssm:/aws/reference/secretsmanager/cred-prod~true:rdsMasterPassword}
- Fn::If:
- Fn::Equals: [*stage, "staging"]
- ${ssm:/aws/reference/secretsmanager/cred-staging~true:rdsMasterPassword}
- ${ssm:/aws/reference/secretsmanager/cred-dev~true:rdsMasterPassword}