How and what’s to be configured in API gateway to accept file as an input in AJAX call. Also How to read the contents of the file in Python Lambda?
There ain't much help with examples available in the official aws docs
We need to zip the dependencies along with python file and upload it.
For example if you wish to read a file called input.conf in python file reader.py,
you need to create a zip file which includes the above files within it.
/uploader.zip
|- input.conf
| - reader.py
The name of the python file should be the first part of lambda handler, like:
reader.lambda_handler (in case of python)
Related
I have a lambda function written in .NET 6 to which I have attached a lambda layer which is a zip file with many JSON files and folders in it.
I want to read each JSON file in my lambda function.
When I do this I get an error saying Could not find a part of the path '/var/task/geojson-files/UK.json':
var filePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "geojson-files", "UK.json");
var geojson = await File.ReadAllTextAsync(filePath);
I also tried /opt/geojson-files/UK.json. But still unable to read the file.
Is there any way to read the file in a lambda function from a lambda layer?
Thanks to #jarmod for his suggestion in the comments to enumerate the contents of /opt
I iterated over the /opt folder and could get the root cause of my problem i.e. unable to read files from the lambda layer.
I zipped the contents of geojson-files from inside that folder. So when I enumerated /opt, I could directly see /opt/UK.json file. So then I zipped the geojson-files folder and now I can see /opt/geojson-files/UK.json file.
I have a bunch of settings and configs that I need AWS Lambda to be able to access. This is not a .py file, so I cannot just import it. In order to load the YAML, I need to specify a path to the file, so how do I do that in Lambda?
My understanding is that Lambda can only read/write to the /tmp folder. But I would like to include this config file with either the lambda layer or as a file in the lambda code package. Either of those two options are not in the /tmp folder and therefore does not provide direct read/write to files there.
Where can I put this YAML file and how can I reach it to read it during runtime?
Lambda has read access to everything. It only has write access to the /tmp folder. If you are thinking about including the YAML file in a Lambda layer, or in the code package, your code will be able to read the file just fine.
I'm implementing a serverless API, using:
API Gateway
Lambda
S3 Bucket "If needed"
My flow is to :
Call POST or PUT method with a binary file "zip", upload it to Lambda.
In Lambda: unzip the file.
In Lambda: Run a determined script over the extracted files.
In Lambda: Generate a new zip.
Return it to my desktop.
This flow is already implemented and it's working well with small files, 10MB for uploading and 6MB for downloading.
But I'm getting issues when dealing with large files as it'll be the case on many occasions. To solve such issue I'm thinking about the following flow:
Target file gets uploaded S3 Bucket.
A new event is generated and Lambda gets triggered.
Lambda Internal Tasks:
3.1 Lambda Download the file from S3 bucket.
3.2 Lambda Generate the corresponding WPK Package.
3.3 Lambda Upload the generated WPK package into S3.
3.4 Lambda returns a signed URL related to the uploaded file as a response.
But my problem with such design is that it requires more than a request to get completed. I want to do all this process in only 1 request, passing the target zip file in it and get the new zip as the response.
Any Ideas, please?
My Components and Flow Diagram would be:
Component and Flow Diagram
There are a couple of things you could do if you'd like to unzip large files while keeping a serverless approach :
Use Node.js for streaming of the zip file, unzipping the file in a pipe, putting the content in a write stream pipe back to S3.
Deploy your code to an AWS Glue Job.
Upload the file to S3,AWS Lambda gets triggered pass the file name as the key to the glue job and the rest will be done.
This way you have a serverless approach and a code that does not cause memory issues while unzipping large files
I had made a lambda function in the past which I want to change. At the time of making it there were no layers, so I was forced to use a deployment package. I have published this version under an alias, and would like to modify the $latest version .
However, in the $latest version, which is editable, the only code entry types available are zip and S3, the code editor is not there. How can I remove the deployed package so I can use a layer with some libraries, and write the code into the editor.
Easiest way would be to create a new lambda, enter your code in the editor and then export the function as a zip file. Then just go over to your original lambda and upload this zip file. You can delete the new lambda you created after you do this.
Just create a zip file of your code only and upload it using 'upload a zip file'. This will replace your existing package with a smaller code file that can me edited via online editor. NOTE: only zip your lambda_function.py file and nothing else. Rest you can upload via layers as you said.
Firstly, I am a newbie to AWS. I was able to edit my Lambda code in line, but I recently uploaded a zip file to it(30MB) to S3 bucket and added this zip to my Lambda from S3, and now my Lambda inline editor doesn't open anymore saying the following error
"The deployment package of your Lambda function "LF2" is too large to
enable inline code editing. However, you can still invoke your function."
I tried deleting my zip file from S3 bucket hoping that the URL of zip would not be reachable and the lambda would lose the zip file and let me edit the function again. But, my lambda size still consists of the 30MB zip file size. I am unable to delete this zip and can't figure out a way to get rid of this it and edit my lambda code again.
Note: My Lambda code was written in-line and different from the zip file(which only contains elastic search setup files which I uploaded for using in my code since import elastic search wasn't working). I know there would have been a better way to do this without uploading it's zip.
Yes, you can download the Lambda function. Go to the AWS console for the Lambda function, make sure you are in the Configuration view, then click Actions | Export function. This will allow you to download a ZIP file containing the Lambda function.
Note that once you upload a Lambda function via S3, it's copied by the Lambda service. There's no connection at that point back to the S3 object that you uploaded. One reason for this is that your Lambda function would break if you, accidentally or otherwise, deleted the file from S3.
I had this problem yesterday then I somehow managed to find my code but not that full code that was vanished from AWS lambda. I wrote that code again, tested it, then tried to upload it with the same name of the lambda function and at the same lambda function by compressing it in my own system.
While uploading it, lambda gave me the option to choose between the remote file I uploaded and the local file it had saved previously. I opted for the local file and boom! I got my code back as it was last saved.
So, I suggest you to just try to upload a random blank compressed zip file containing one file name same as the lambda function. It would give you the option to choose from both files, then choose for "local" file. It would take you to the in-line editor where your code was.
I just ran into same soup .. seem like the upload replaced the previous index.js with export handler.