aws cli lambda `update-function-configuration` deletes existing environment variables - amazon-web-services

The documentation on the AWS cil lambda states that
...You provide only the parameters you want to change...
Which I assume means that the rest of the settings would still remain the same.
However, say my lambda function has environment variables :
var1=old_val1
var2=old_val2
var3=old_val3
and then when I try doing something like this :
aws lambda update-function-configuration --function-name dummy_fun --environment '{"Variables":{"var1":"new_val1","var2":"new_val2"}}'
with the intention of updating the variables :
var1 and var2 with the new values new_val1 and new_val2 respectively, although these 2 variables DO get updated, but the third one, var3, gets deleted !
Am I doing something wrong ? Or is there a way to make sure this doesn't happen?
I can definitely handle it using a workaround wherein I can fetch the current config and then update the env variables locally and then push the entire updated config, all of this through a python code etc.
But, is that the only way to do it ? Or can there be a simpler way of doing it?

You are misinterpreting the intention of the documentation.
You provide only the parameters you want to change.
--environment is the (singular) "parameter" that you are specifying should be changed -- not the individual variables.
The environment variables are configured en bloc so there is no concept of specifying only certain environment variables that you want to be different.

aws lambda update-function-configuration --function-name my-function-name --environment Variables="{VAR1=variable_value, VAR2=variable_value}"
Description:Above Command will update the environment variables for the lambda function in aws.

It seems not easy to partially update the environment variables for a lambda with awscli.
But with the usage of the built-in JSON-based client-side filtering that uses the JMESPath syntax, I found a way to achieve what I needed to do.
NEW_ENVVARS=$(aws lambda get-function-configuration --function-name your-func-name --query "Environment.Variables | merge(#, \`{\"ENV_VAR_TO_UPDATE\":\"value_here\"}\`)")
aws lambda update-function-configuration --function-name your-func-name --environment "{ \"Variables\": $NEW_ENVVARS }"
Of course, you can update more than one environment variable with that trick.

I had the same problem where I wanted to update only one env variable of a function and not touch the rest.
I ended up writing a script in node and publishing it:
https://www.npmjs.com/package/aws-lambda-update-env
It is pretty simple to use:
update-lambda-env KEY "My New Test Value" --stack-name myApplicationStack
This will only change the variable KEY in the functions located in the stack
myApplicationStack
A better solution might be to use AWS Parameter Store if your variable is going to change often.
https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html

Related

Can I Install local/chromium on a S3 and use it with puppeteer?

I need to install local/chromium on lambda but it's too big to be stored so, it's possible to stored it in a bucket and use it on my lambda where the code/node_module is?
Instead you can use aws lambda layers and use it within a lambda
Looks like there already a use case for you https://github.com/shelfio/chrome-aws-lambda-layer , similar question

In an AWS lambda, how do I access the image_id or tag of the launched container from within it?

I have an AWS lambda built using SAM. I want to propagate the id (or, if it's easier, the tag) of a lambda's supporting docker image through to the lambda runtime function.
How do I do this?
Note: I do mean image id and NOT container id - what you'd see if you called docker image ls locally. Getting the container id / hostname is the easy bit :D
I have tried to declare a parameter in the template.yaml and have it picked up as an environment variable that way. I would prefer to define the value at most once within the template.yaml, and preferably have it auto-populated, though I am not aware of best practice there. The aim is to avoid human error. I don't want to pass the value on the command line unless I have to.
If it's too hard to get the image id then as a fallback the DockerTag would be fine. Again, I don't want this in multiple places in the template.yaml. Thanks!
Unanswered similar question: Finding the image ID of a container from within the container
The launched image URI is available in the packaged template file after running sam package, so it's possible to extract the tag from there.
For example, if using YAML:
grep -w ImageUri packaged.yaml | cut -d: -f3
This will find the URI in the packaged template (which looks like ImageUri: 12345.dkr.ecr.us-east-1.amazonaws.com/myrepo:mylambda-123abc-latest) and grabs the tag, which is after the 2nd :.
That said, I don't think it's a great solution. I wish there was a way using the SAM CLI.

Lambda ValueFrom Environment Variable Like in Task Definition

Is there a way to have a ValueFrom feature in Lambda's environment variable similar to what we have in Task Definition?
How it works.
We have a kv pair in parameter store /dev/db/host=localhost.
In the container definition inside the ECS task definition, we add a new environment variable DB_HOST which has a ValueFrom /dev/db/host. When a new instance of the container is run it will have the value localhost from the parameter store.
I tried on Lambda but it seems like this feature is not available. Is there another way to do this? I wonder if there is a request for this as well.
PS: I'm aware that it can be done via TerraForm or CloudFormation but that will only evaluate and copy the values from parameter store to Lambda environment variables when the infrastructure is built. The problem is some of the values are secured like DB password, thus it cannot be simply copied as it will get exposed.

Global environment variables for AWS CloudFormation

Is there a way to have global environment variables in a AWS CloudFormation yaml file for Lambdas?
Currently we are using the SSM Parameter Store for global variables, but we don't want to use that anymore.
I looking forward to have something like this:
Environment:
Variables:
variable1: xxx // local variables
variable2: xxx
...
${file(./globalvariables.yml)} // global variables
Or even better: every lambda is including the global environment variables as default without explicitly calling it.
Is this possible? Or what approach would you suggest? Thanks in advance!
Sadly I'm unaware of having predefined defaults for environment variables to be set through CloudFormation for Lambdas however - One possible option is instead of using env variables in CloudFormation add a lambda layer with all the config and pull the values from there.
Benefits of this are that if a value changes you only have to update your layer once then update lambdas to use new layer which could be a single parameter instead of manually updating every single one.
Docs here: https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
Another option would be to use AWS Secrets Manager Or SSM Parameter Store as ServerMonkey suggested.

How do I force a CloudFormation stack to update when the parameter is updated?

I am running a AWS CloudFormation stack that takes in some parameters and launches EC2 instances along with other AWS resources. The parameters are fed into the user data of the EC2 instance and based on that changes are made dynamically to the web application residing on the EC2 instance.
UserData:
Fn::Base64:
Fn::Join:
- ""
-
- "#!/bin/bash \n"
- "sh website-conf/website_mysql_config.sh "
- " -c \""
-
Ref: "CompanyName"
As shown in the example above, CompanyName is one of the many parameters passed to the userdata script. The problem is, when any one or multiple of parameters are updated, CloudFormation does not detect that and instead throws this error.
So, in order to update the stack, I have to edit the stack and make changes to the ASG so that CloudFormation 'sees' the changes and executes the stack update.
Is there a way to force CFN to update the stack when the parameters are updated?
CloudFormation will not update the stack unless there is a change in properties of the resources already created in the stack.
For example:
Consider I have a simple template to create a database where I need to pass 2 parameters:
db-name
region
Assume that I am using db-name passing it as value to DBInstanceIdentifier.
Also assume that I am not using the input parameter region for any purpose in creation of resources (or its properties) of the stack in any way.It is more of a dummy parameter I keep for readability purpose.
I passed (TEST-DB1, us-east-1) as input parameters to the CloudFormation template and successfully created the resources.
Scenario-1:
Now if I update the stack(still using the existing template) and just change the input parameters to (TEST-DB2, us-east-1). ie: changing just the db-name and not the region. Then CloudFormation will detect that, this parameter update, results in change in properties of running resource(s) of the stack and will compute and display the modifications as a change set.
Scenario-2:
Suppose I make another update(still using the existing template) property and just change the input parameters to (TEST-DB1, us-east-2). ie: changing just the region and not the db-name. Then CloudFormation will detect that, this parameter update, result in NO change in properties of running resource(s) of the stack will show the Error creating change set.
Bottomline:
Your change in input parameter must result in an update/replacement of any resources(or its attributes like security-groups,port etc..) of the stack. Then AWS CloudFormation will display them as Change Sets for your review. Also, the method (update or replacement) AWS CloudFormation uses depends on which property you update for a given resource type.
Your parameter "CompanyName" is not making any changes to the running
resources of the stack. Hence it is reporting as Error creating
change set. You need to use it to create any resource/resource properties of the stack. Then CloudFormation will detect the change-sets when you modify it. The same applies for any other input-parameters which you use.
Use the AWS CLI Update-Stack command. If you use the AWS CLI you can inject parameters into your stack so any change to any of the parameters result in a new stack. I do this myself to inject the Git/version commit ID into UserData so simply committing changes to the stack's JSON/Yaml to Git will allow stack updates. Any change to the parameters file will allow stack updates, even just a comment. I reference my Git commit ID in UserData the same way you are referencing Ref:CompanyName so when I change the Git commit ID the userData section is updated on stack updates.
Update Stack Command
aws cloudformation update-stack --stack-name MyStack --template-body file:///Users/Documents/Git/project/cloudformation/stack.json --parameters file:///Users/Documents/Git/project/cloudformation/parameters/stack-parameters.dev.json --capabilities CAPABILITY_IAM
Process
With this approach you make your parameters changes to the parameters json or yaml file then check it into version control. Now if you use a build server you can update your stack by checking out master and just running that one line above. Using AWS CodeBuild makes this easy so you don't need jenkins.
The answer of your problem is already answered with this state, CloudFormation will not update the stack unless there is a change in properties of the resources already created in the stack.
And for the answer for your question, please check the explanation below.
There is a way to force Cloudformation to update the stack using the AWS::CloudFormation::Init.
By using cfn-init, each instance can update itself when it detect the change that made by AWS::CloudFormation::Init in metadata.
There is a concept that we must understand first, that is the difference between UserData and metadata, at least under the AWS::CloudFormation::Init case.
Userdata: Will be only called once when the instance is being launch for the first time (this including update that need the instance to be replaced). So, if you update the stack (not creating a new one), even if you change the parameter value, it won't change anything if you call the parameter under UserData.
Metadata: Can be updated anytime. To make it works, you have to make sure that the daemon that detect the metadata changed is running (the daemon is called the cfn-hup)
If you already use the Metadata and AWS::CloudFormation::Init, the data is not immediately being updated. As far I know, here is the condition the data to be change after change the Metadata value.
Reboot the instance
Run cfn-init command again with it's parameter
Waiting about 15 minutes, because the daemon to check the change in Metadata is checking the change once in 15 minutes.