Is there a way to have global environment variables in a AWS CloudFormation yaml file for Lambdas?
Currently we are using the SSM Parameter Store for global variables, but we don't want to use that anymore.
I looking forward to have something like this:
Environment:
Variables:
variable1: xxx // local variables
variable2: xxx
...
${file(./globalvariables.yml)} // global variables
Or even better: every lambda is including the global environment variables as default without explicitly calling it.
Is this possible? Or what approach would you suggest? Thanks in advance!
Sadly I'm unaware of having predefined defaults for environment variables to be set through CloudFormation for Lambdas however - One possible option is instead of using env variables in CloudFormation add a lambda layer with all the config and pull the values from there.
Benefits of this are that if a value changes you only have to update your layer once then update lambdas to use new layer which could be a single parameter instead of manually updating every single one.
Docs here: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
Another option would be to use AWS Secrets Manager Or SSM Parameter Store as ServerMonkey suggested.
Related
Is there a way to have a ValueFrom feature in Lambda's environment variable similar to what we have in Task Definition?
How it works.
We have a kv pair in parameter store /dev/db/host=localhost.
In the container definition inside the ECS task definition, we add a new environment variable DB_HOST which has a ValueFrom /dev/db/host. When a new instance of the container is run it will have the value localhost from the parameter store.
I tried on Lambda but it seems like this feature is not available. Is there another way to do this? I wonder if there is a request for this as well.
PS: I'm aware that it can be done via TerraForm or CloudFormation but that will only evaluate and copy the values from parameter store to Lambda environment variables when the infrastructure is built. The problem is some of the values are secured like DB password, thus it cannot be simply copied as it will get exposed.
I have a nested Cloud Formation Template (multiple templates within a root template )to create a complete web application.
Lambda is created in the first template and few environment variables are added to it.
The later part of the templates also produces some values that has to be added as environment variables.
Is there a way to attach these environment variables to the existing lambda function?
I don't think so, but there are a few options. If you could change the stack dependency order, you could build the stack creating the values depended upon first. If you cannot, you can store your environment variables in SSM Parameter Store as mentioned in this knowledge center article.
So you set the environment variable to a path where the value can be expected, then when creating the stack that knows the value, you store it at that path. When the lambda runs, you just do get parameter.
In my stack definition I pull in a number of parameters from SSM Parameter Store...
const p1 = ssm.StringParameter.fromStringParameterAttributes( ... )
const p2 = ssm.StringParameter.fromStringParameterAttributes( ... )
I then pass them along to the relevant lambdas as environment vars...
environment: {
PARAM_ONE: p1.stringValue
PARAM_TWO: p2.stringValue
}
However I don't want all of those parameters to be mandatory. I would like the ones that exist to be passed in as env vars, and the ones that don't to just remain undefined as my app has defaults for them anyway. However, trying to inspect the value of p1.stringValue just gives me a Token, not a value, so I can't do any logic based on it's presence or absence: https://docs.aws.amazon.com/cdk/latest/guide/tokens.html
If I ask for the parameter and it is not defined in SSM Parameter Store I then get an error that I can't catch or ignore when it tries to build the changeset and the deployment fails...
MyApp: creating CloudFormation changeset...
❌ MyAppStack failed: Error [ValidationError]: Unable to fetch parameters [/myapp/param1,/myapp/param2] from parameter store for this account.
So how can I deal with SSM parameters which may or may not exist at deploy time?
I assume you are only grabbing the manager in your import, not the actual values inside your secrets. If this is the case, then your best bet is to leverage the SDK to do this for you - a simple call using the SDK (which will be run during the synth stage of a cdk deploy or cdk synth) to see if said SMM fields/groups exist. If they do, go ahead and import them.
I do something very similar with Layers - the from methods for layers require the version number - that may change at any time. So i have a small function that gets the latest version number of a given layer using the SDK and i can then use that to import the layer definition into my stack.
If you are trying to get the actual secret inside the secret manager parameter ... that is better suited to outside the CDK for most scenarios - done in the exact location you need the secrets so you dont end up with secret value in plain text somewhere.
I'm building a complex application in AWS using Cloudformation.
My setup is the following: I'm going to use yaml files to define the stacks and corresponding json files which contain the stack parameters. Anyway there are parameters which are the same in multiple json files and I'd like to define them globally in one file/stack instead of having to update them in multiple files everytime they change.
What is the recommended way to set such global parameters using cloudformation?
Help would be highly appreciated.
You could possibly create one stack with command parameters, end export their values from this stack. Then, in other stack, the parameter values would be accessed using Fn::ImportValue.
An alternative could be to store common parameters in SSM Parameter Store, and then use dynamic references in your template to access them.
The documentation on the AWS cil lambda states that
...You provide only the parameters you want to change...
Which I assume means that the rest of the settings would still remain the same.
However, say my lambda function has environment variables :
var1=old_val1
var2=old_val2
var3=old_val3
and then when I try doing something like this :
aws lambda update-function-configuration --function-name dummy_fun --environment '{"Variables":{"var1":"new_val1","var2":"new_val2"}}'
with the intention of updating the variables :
var1 and var2 with the new values new_val1 and new_val2 respectively, although these 2 variables DO get updated, but the third one, var3, gets deleted !
Am I doing something wrong ? Or is there a way to make sure this doesn't happen?
I can definitely handle it using a workaround wherein I can fetch the current config and then update the env variables locally and then push the entire updated config, all of this through a python code etc.
But, is that the only way to do it ? Or can there be a simpler way of doing it?
You are misinterpreting the intention of the documentation.
You provide only the parameters you want to change.
--environment is the (singular) "parameter" that you are specifying should be changed -- not the individual variables.
The environment variables are configured en bloc so there is no concept of specifying only certain environment variables that you want to be different.
aws lambda update-function-configuration --function-name my-function-name --environment Variables="{VAR1=variable_value, VAR2=variable_value}"
Description:Above Command will update the environment variables for the lambda function in aws.
It seems not easy to partially update the environment variables for a lambda with awscli.
But with the usage of the built-in JSON-based client-side filtering that uses the JMESPath syntax, I found a way to achieve what I needed to do.
NEW_ENVVARS=$(aws lambda get-function-configuration --function-name your-func-name --query "Environment.Variables | merge(#, \`{\"ENV_VAR_TO_UPDATE\":\"value_here\"}\`)")
aws lambda update-function-configuration --function-name your-func-name --environment "{ \"Variables\": $NEW_ENVVARS }"
Of course, you can update more than one environment variable with that trick.
I had the same problem where I wanted to update only one env variable of a function and not touch the rest.
I ended up writing a script in node and publishing it:
https://www.npmjs.com/package/aws-lambda-update-env
It is pretty simple to use:
update-lambda-env KEY "My New Test Value" --stack-name myApplicationStack
This will only change the variable KEY in the functions located in the stack
myApplicationStack
A better solution might be to use AWS Parameter Store if your variable is going to change often.
https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html