Problem creating Lambda function that has a Layer using boto3 - amazon-web-services

If I try to use boto3 Lambda create_function() to create a Lambda function, and I try to include Layers via Layers=['string'] parameter, I get the following error message:
Unknown parameter in input: "Layers", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig, DeadLetterConfig, Environment, KMSKeyArn, TracingConfig, Tags
... any ideas? The documentation suggests that this should work, but something is clearly off here. NOTE: I also have a similar problem with "Layers" in update_function_configuration() as well.
My guess is that the version of boto3 that the AWS Lambda console uses has not been updated/refreshed yet to support Layers. Because when I run the same code locally on a machine with a fairly recent version of boto3, it runs without any problems. I have already tried using both listed Python runtimes of 3.6 and 3.7 that in the AWS console, but neither worked. These runtimes have respective versions of boto3 of 1.7.74 and 1.9.42. But my local machine has 1.9.59. So perhaps the addition of Lambda Layers occurred between 1.9.42 and 1.9.59.

My guess is that the version of boto3 that the AWS Lambda console uses has not been updated/refreshed yet to support Layers.
That's completely right. AWS usually updates the available libraries on AWS Lambda regularly, but hasn't updated them for several months now for unknown reasons.
The supported API endpoints are actually not defined in boto3, but in botocore.
Currently botocore 1.10.74 is available on AWS Lambda, while support for AWS Lambda Layers got added in botocore 1.12.56.
To avoid such incompatibilities between your code and the versions of available libraries, you should create a deployment package containing boto3 and botocore in addition to your AWS Lambda function code, so your code uses your bundled versions instead the ones AWS provides. That's what AWS suggests as part of their best practices as well:
Control the dependencies in your function's deployment package.
The AWS Lambda execution environment contains a number of libraries such as the AWS SDK for the Node.js and Python runtimes (a full list can be found here: Lambda Execution Environment and Available Libraries). To enable the latest set of features and security updates, Lambda will periodically update these libraries. These updates may introduce subtle changes to the behavior of your Lambda function. To have full control of the dependencies your function uses, we recommend packaging all your dependencies with your deployment package.

Related

AWS - Added Layers are not listed and available in lambdas

I have a Cloud9 instance with a virtual python environment, what is going to published as a layer. Until today it was possible to zip the env libs and publish it to make it available in lambda functions. (= Standard workflow).
Command (as usual, execution was possible without errors):
aws lambda publish-layer-version --layer-name MyLayerName --zip-file fileb://python.zip --compatible-runtimes python3.9
But...
aws lambda list-layers
delivers an empty array. It also not selectable as a custom layer in my lambdas.
Does anybody has an explanation for this issue?
Best,
Felix
I found the problem. The layer had a size >250 MB.
I found this out after I tried to import it as a ZIP via the web interface. An error message came up. There is no error message on the console...

Where can I find the source code for AWS Lambda's dotnetcore3.1 runtime?

AWS publishes a lot of their source code on GitHub, but so far I haven't been able to find the source for the dotnetcore3.1 Lambda Runtime.
I expect this source will be a console application responsible for startup and IoC (using the Lambda configuration to determine the handler class and function, using reflection to instantiate the handler, as well as the serializer specified on the lambda's assembly attribute). So far I have found a number of repositories containing things like NuGet libraries and dotnet CLI tooling--but haven't located the runtime itself.
Is the AWS Lambda dotnetcore3.1 runtime source publically available? Can you point me to the repo and .csproj? Thanks!
I am not 100% sure, but there is probably nothing like a dedicated ".NET Core Runtime" that is publicly available.
Let's explore how Lambda works to explain why I think that:
First there is an EC2 instance with a host operating system. That's the bare metal your resources are coming from (vCPU, RAM, disk, etc.).
Second, there is a thin management layer for microVMs. For AWS Lambda this is a project called Firecracker.
The microVMs that are started use a micro kernel. In case of AWS Lambda it is the OSv micro kernel. The micro kernel already has some support for runtimes:
OSv supports many managed language runtimes including unmodified JVM, Python 2 and 3, Node.JS, Ruby, Erlang as well as languages compiling directly to native machine code like Golang and Rust.
.NET Core is compiled so I think this falls into the same category. It is a "native" binary, that is self-contained and just started.
You mentioned reflection etc. to run the binary. I think it is not that complicated. This is were the last puzzle piece comes into place: environment variables.
There are a bunch of environment variables thar are set by the "runtime". You can find the full list in the AWS documentation: Defined runtime environment variables.
Every Lambda has a environment variable called _HANDLER which points to the handler. In your case this would be the binary. This would allow the runtime to just "run" your handler.
When the handler runs the underlying AWS SDK is starting some kind of webserver/socket (I think it depends on the framework/language) which exposes everything the "standard" runtime needs to communicate with your code.
So as far as I can tell there is not really a dedicated .NET Core runtime. It is pretty generic all the way down, which simplifies development for the folks at AWS I guess.
The source code for Lambda's dotnet runtime(s) is within the aws/aws-lambda-dotnet repository--it doesn't appear to have a dedicated netcoreapp3.1 library, or a preserved branch, but presumably the 3.1 runtime is contained within this repository's history.
The Amazon.Lambda.RuntimeSupport console application appears to be the entrypoint, which in turn instantiates and invokes the published Handler.

AWS Checks for Lambda functions that are configured to use a runtime that is approaching deprecation or is deprecated

I receive the following warning from AWS, but I don't know how to fix this problem.
My understanding, I'll have to upgrade my local computer Node version to version 14, then re-build my application and push up again to AWS.
Description: Checks for Lambda functions that are configured to use a runtime that is approaching deprecation or is deprecated. Deprecated runtimes are not eligible for security updates or technical support.
My question, do I have to rebuild my application using Node version 14 or something simple like change the runtime version in AWS Lambda?
It is not about local build. Each lambda has a language environment. Change your lambda environment/setting/config.

can we import specific version of botocore when implement Lambda in AWS?

when using cost explorer client in lambda function , 'ce' is not supported. but it may available in new version of botocore. so is it possible to import the new version of botocore in lambda python function ? if yes, how we can implement it. please provide some example
AWS Lambda provides a way to to use a version of Boto3 other than the one included by default, you can include it in your deployment package.
From Creating a Deployment Package (Python)
AWS Lambda includes the AWS SDK for Python (Boto 3), so you don't need
to include it in your deployment package. However, if you want to use
a version of Boto3 other than the one included by default, you can
include it in your deployment package.

AWS lambda versioning

I have a question about the lambda functions versioning capabilities.
I know how the standard way of versioning works out of the box in AWS but I thought there is a way for the publisher to specify the version number which would tag a specific snapshot of the function. More exactly what I was thinking of was including in the uploaded zip file a config.json where the version would be specified. And this would be used afterwards by AWS for tagging.
The reason I am asking is because I would like, for example, to keep in sync the version of the lambda function with the CI job build number that built (zipped) the lambda.
Any ideas?
Many thanks
A good option would be store your CI job build number as an environment variable on the Lambda function.
Its not exactly a recommended way to version AWS Lambda functions, but definitely helps in sticking to typical 1.x.x. versioning strategies and keeping them consistent across the pipeline.
Flipping the topic the other way around. Can we go with AWS Lambda versions 1.2.3., and then find a way to have our CI builds also use a single digit version no? Im not yet comfortable with this approach, and like the flexibility of 1.x.x as a versioning scheme to indicate major.minor.patch.
Standard Lambda versioning.
This is the most detailed blog I came across on this topic.
https://www.concurrencylabs.com/blog/configure-your-lambda-function-like-a-champ-sail-smoothly/
When you are deploying the Lambda function through CLI command or API, its not possible to give a custom version number. Its currently an automatically generated value by aws.
This makes it not possible to map the version number in a configuration file to the Lambda version supporting your use case.