I am using cloudformation to create my lambda function. I have opted to pull the code from S3.
However, it appears to create a nested structure when the lambda function gets created, and I am unable to import my packages unless I move the lambda and associated library packages up to the root level of the lambda function.
Cloudformation value for code section:
Code:
S3Bucket: youll_never_guess-bucket-12345
S3Key: python_data_collector.zip
How it appears in lambda, aws console:
Full path for handler in console:
I've tried: python_data_collector/lambda.lambda_handler and python_data_collector.lambda.lambda_handler
Error message:
Unable to import module 'python_data_collector/lambda': No module named 'requests'"
The Python dependencies should reside at the root level of you lambda deployment package. You can indeed point to a nested file as the entry point of your function, but this does not change the dependency behaviour of your function.
However, the structure of your lambda code has nothing to do with where your zip file is located in the S3 bucket. Presumably, when you are creating the zip file, you are adding a folder at the root level which contains the code & dependencies. You should not have that extra folder in the zip file and simply put the code (nested or not) and the dependencies (not nested) at the root of you zip package. Lambda will simply unzip the file and place the contents as is, in your lambda function.
Related
I have a lambda function written in .NET 6 to which I have attached a lambda layer which is a zip file with many JSON files and folders in it.
I want to read each JSON file in my lambda function.
When I do this I get an error saying Could not find a part of the path '/var/task/geojson-files/UK.json':
var filePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "geojson-files", "UK.json");
var geojson = await File.ReadAllTextAsync(filePath);
I also tried /opt/geojson-files/UK.json. But still unable to read the file.
Is there any way to read the file in a lambda function from a lambda layer?
Thanks to #jarmod for his suggestion in the comments to enumerate the contents of /opt
I iterated over the /opt folder and could get the root cause of my problem i.e. unable to read files from the lambda layer.
I zipped the contents of geojson-files from inside that folder. So when I enumerated /opt, I could directly see /opt/UK.json file. So then I zipped the geojson-files folder and now I can see /opt/geojson-files/UK.json file.
I've working on a project in which the s3 bucket in aws is already created via console and the lambda code is already there as an object. I'm creating a terraform script in which I will reference that zip and then create a lambda function and publish it. Incase there's any change detected in the code (the code zip can be changed from console), then it should publish latest. How can I do that? Currently I'm getting errror-
module "student_lambda"{
source = "https://....." // I'm using a template which creates lambda function
handler..
s3_bucket = "SaintJoseph"
s3_key = "grade5/studentlist.zip"
source_code_hash = filebase64sha256("/grade5/studentlist.zip").etag
.....
}
My bucket structure
SaintJoseph -- bucketname
grade5
studentlist.zip
subjectlist.zip
grade6
Errors I'm getting in plan -
Error in function call - Call to function filebase64sha256 failed: open grade5/studentlist.zip: no such file or directory
The bucket key or source is invalid.
Can someone please also help to let me know what to use like etag/source_code_hash, etc so that it only takes changes when zip file is changed and how to remove existing error?
filebase64sha256 works only on local filesystem. To reference etag of an s3 object you have to use aws_s3_object data source. The source returns etag.
I have a bunch of settings and configs that I need AWS Lambda to be able to access. This is not a .py file, so I cannot just import it. In order to load the YAML, I need to specify a path to the file, so how do I do that in Lambda?
My understanding is that Lambda can only read/write to the /tmp folder. But I would like to include this config file with either the lambda layer or as a file in the lambda code package. Either of those two options are not in the /tmp folder and therefore does not provide direct read/write to files there.
Where can I put this YAML file and how can I reach it to read it during runtime?
Lambda has read access to everything. It only has write access to the /tmp folder. If you are thinking about including the YAML file in a Lambda layer, or in the code package, your code will be able to read the file just fine.
I'm using CDK to set up a CI/CD Pipeline. I have currently a code build from a git into the pipeline. There are then two builds - one that pulls out code for a lambda and builds an artifact for it, and a second that issues the cdk synth to construct the lambda framework (including a nested bucket and dynamo).
Then it heads to a deploy stage, but fails because it can't find the parameters for the location of the lambda code
ive been using this example: https://docs.aws.amazon.com/cdk/latest/guide/codepipeline_example.html
the only differences from this example are that I'm using python for all of it and due to known future needs, the lamdba's are are in a parallel directory from the stack code
|-Lambdas
|--Lambda1
|---Lambda1Code
|--Lambda2
|---Lambda2Code
|-CDKStacks
|--LambdaCreationStack
|--PipelineCreationStack
|--app.py
Everything runs up until deploy where it fails with the error "The following CloudFormation Parameters are missing a value:" and then lists the BucketName and ObjectKey
I assigned those as overrides as per the above link:
admin_permissions=True,
parameter_overrides=dict(
lambda_code.assign(
bucket_name=lambda_location.bucket_name,
object_key=lambda_location.object_key,
object_version=lambda_location.object_version
)
),
as part of the pipeline actions CloudFormationCreateUpdateStackAction, and passed the code just like in the example from lambda stack to the pipeline stack. But every time the lambda stack is attempted to deploy the parameters for the location of the code 'do not exist'
I've tried overriding the parameters, but being in the pipeline and dynamically created I am hesitant to follow further (and my attempts didnt work anyways). I've tried a bunch of different stack/nested stack/single stack configurations but haven't had a Successs yet.
thoughts?
This basically boils down to CodeUri in the Cloudformation template will automatically append the s3 bucket if your CodeUri starts with ./
So you have 2 options.
In your pipeline output your artifact as normal, just do the whole repo from the codebuild into the code deploy. Your code deoploy can pick up the artifact naturally and will automatically append the S3 url to that
if you're using Python however, you MUST be aware that starting from a lambda directory deeper in the tree will mean that the python Imports expect that directory to be a root directory - meaning if you were in Lambdas/Lambda1 and wanted to import a file that existed in the Lambda1 directory, in order for it to work on AWS Lambda you would need to have the import be just the file name, ignoring the rest of the path.
This means that coding can be difficult, and running unit tests can be difficult as well. You'll want to add all the individual lambda folders (and their paths) from root to the PYTHONPATH env variable of your codebuild instance so the unit tests know where to do so (and add a .env file to your IDE as well to handle this in your local)
You use CDK and you cdk synth the stack you want to deploy. This creates a cdk.out folder with a bunch of asset zips in it plus the stack template (a json). you adjust your artifact output in the codebuild to output the cdk.out folder, and the asset zips are automatically (thanks to cdk) subbed into the codeUri locations in the also automatically synthed template. Once you know what the templates name is its easy to set the CodeDeploy to look for that template name and it will find the asset zips individually for each lambda.
Firstly, I am a newbie to AWS. I was able to edit my Lambda code in line, but I recently uploaded a zip file to it(30MB) to S3 bucket and added this zip to my Lambda from S3, and now my Lambda inline editor doesn't open anymore saying the following error
"The deployment package of your Lambda function "LF2" is too large to
enable inline code editing. However, you can still invoke your function."
I tried deleting my zip file from S3 bucket hoping that the URL of zip would not be reachable and the lambda would lose the zip file and let me edit the function again. But, my lambda size still consists of the 30MB zip file size. I am unable to delete this zip and can't figure out a way to get rid of this it and edit my lambda code again.
Note: My Lambda code was written in-line and different from the zip file(which only contains elastic search setup files which I uploaded for using in my code since import elastic search wasn't working). I know there would have been a better way to do this without uploading it's zip.
Yes, you can download the Lambda function. Go to the AWS console for the Lambda function, make sure you are in the Configuration view, then click Actions | Export function. This will allow you to download a ZIP file containing the Lambda function.
Note that once you upload a Lambda function via S3, it's copied by the Lambda service. There's no connection at that point back to the S3 object that you uploaded. One reason for this is that your Lambda function would break if you, accidentally or otherwise, deleted the file from S3.
I had this problem yesterday then I somehow managed to find my code but not that full code that was vanished from AWS lambda. I wrote that code again, tested it, then tried to upload it with the same name of the lambda function and at the same lambda function by compressing it in my own system.
While uploading it, lambda gave me the option to choose between the remote file I uploaded and the local file it had saved previously. I opted for the local file and boom! I got my code back as it was last saved.
So, I suggest you to just try to upload a random blank compressed zip file containing one file name same as the lambda function. It would give you the option to choose from both files, then choose for "local" file. It would take you to the in-line editor where your code was.
I just ran into same soup .. seem like the upload replaced the previous index.js with export handler.