I am getting the following error:
Unable to import module '': No module
named 'regex._regex'
The AWS Lambda deployment package runs just fine without import htmldate statement (the module I want to use) which in turn requires regex.
Also the code runs fine locally.
So this seems to be a problem running regex on AWS Lambda.
A new version of htmldate makes some of the dependencies optional, regex is such a case. That should solve the problem. (FYI: I'm the main developer of the package.)
If it runs locally and not in the lambda it may be an issue with the package installation. You may want to install your requirements.txt via a docker replicating the lambdas environment. If it works locally this can be used to ensure you are replicating the environment your lambda is running in during installation.
This docker image can be used to help:
https://hub.docker.com/r/lambci/lambda/
There are some examples specified here: https://github.com/lambci/docker-lambda#build-examples
Related
I have a python code on my desktop where I call Google APIs to get some data. I am trying to deploy it on AWS Lambda to run it periodically but I and running into some issues. Below are the steps I followed:
Downloaded google package using pip3 install google-api-python-client -t . Zipped this folder and uploaded it to a layer in AWS Lambda
Linked the layer with my function but when I am trying to execute the lambda function, I get the following error:
"errorMessage": "Unable to import module 'lambda_function': No module named 'googleapiclient'",
In my code I have the following import statement:
from googleapiclient.discovery import build
Please let me know if I am missing something and how to debug this.
Regards,
Dbeings
You typically receive this error when your Lambda environment can't find the specified library in the Python code. This is because Lambda isn't prepackaged with all third party python libraries.
in your local environment "googleapiclient" are compiled and available during runtime but it's not available in lambda runtime.
To resolve this error, create a deployment package(pre-compilied/zip) or Lambda layer that includes the libraries that you want to use in your Python code for Lambda.
best of luck
I'm working with this article Asynchronous Amazon Transcribe Streaming SDK for Python.
I'm trying to create a lambda layer for the required libraries.
I used the following command:
pip3 install amazon-transcribe aiofile -t .
But I get the following error when I use the layer in my lambda function:
Unable to import module 'lambda_function': No module named '_awscrt'
The same works fine with virtual environment locally. I'm not sure what's the exact issue.
I even tried installing awscrt separately but it didn't work.
Any kind of help will be greatly appreciated. Thanks!
Lambda layers .zip files need to follow a specific directory file structure. Look at this section of the documentation to see how it should be structured for Python. This might be your problem.
I built the layer on Amazon Linux and it worked fine!
The troubleshooting guide in the repo helped:
The caio linux implementation works normal for modern linux kernel versions and file systems. So you may have problems specific for your environment. It's not a bug and might be resolved some ways:
1. Upgrade the kernel
2. Use compatible file system
3. Use threads based or pure python implementation.
I have a SAM app with multiple Lambdas and some utility code I'd like to share between them. When packaging Lambdas using zip files, code sharing can be done with Lambda Layers. However, according to the AWS documentation, Lambda Layers are not supported when using containers.
Functions defined as container images do not support layers. When you build a container image, you can package your preferred runtimes and dependencies as a part of the image
https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
I've tried copying my dependencies into a separate docker image, then pulling from that image in my lambda dockerfile, which is suggested in the AWS blog post Working with Lambda layers and extensions in container images. However, I just get the Unable to import module 'app': No module named '<my_dependency>' error when trying to debug or run sam local invoke. I verified in my final image that the dependencies are in the /opt/python/ directory for my python lambda, and tried other folder structure as well, but no luck.
Has anyone been able to get this to work?
I've figured this out. Dependencies (eg, utils.py) do need to be in the /opt/python/ folder, but when I was trying to test this out, I was manually building my docker image with the docker cli instead of using sam build and I had the wrong image tag. Therefore the debugger used a previous version of my docker image that wasn't working. To fix the issue I just needed to run sam build before trying to debug my lambda locally.
I am trying to write to my PostgreSQL database with AWS Lambda using the python2.7 runtime. I care very little about how I do this, so if anyone has a different way that I can understand that works, I'd love to hear it.
The method I'm currently trying is to use psycopg2, as this is the only way I know. In order to do this, I need to upload the psycopg2 module to my environment on AWS Lambda. As per instructions, I've created a directory with my source and psycopg2 using pip install psycopg2 -t ..\my-project, zipped my-project, and uploaded it.
My error message is this from within the AWS Lambda console: Unable to import module 'lambda_function': No module named _psycopg
The code runs on my windows machine. I think the issue is that when I import psycopg2 from my local windows machine, the _psycopg module is being imported from _psycopg.pyd, and .pyd files are windows specific. I may be wrong about this.
I'm really just looking for any way to achieve the desired result described in my first paragraph, but here's a more specific question: How do I tell windows to pip install and compile psycopg2 without using .pyd files? Is this possible? Do I have something completely wrong?
I know the formatting of this question is a little unorthodox, I think I've given all the necessary information, let me know if there's anything else I can provide.
I solved the problem by opening an ubuntu instance on VirtualBox, pip installing the package there, pulling the relevant folders out, and placing them in my-project before zipping and uploading to AWS Lambda.
See these instructions.
I'm trying to create an AWS lambda function in order to create thumbnail of my uploaded images.
My script is running well locally, I followed this tutorial to deploy my function but I have a problem with the Pillow library, indeed when I'm testing my function I can see this following log :
I found this post with the same issue but in my case I can't execute command line on the machine.
You must include the libjpeg.so in your lambda package, but it will also require some tweaking with the patchelf utility. Assuming that you prepare the lambda package via "pip install module-name -t" (rather than via virtualenv), do the following:
cd into/your/local/lambda/package/dir
cp -L $(ldd PIL/_imaging.so|grep libjpeg|awk '{print $3}') PIL/
patchelf --set-rpath PIL PIL/_imaging.so
# zip, deploy and test the package
This script works for Pillow version 3.2.0.
Regarding patchelf: under Ubuntu it can be 'apt install'ed, but under other Linuxes it may need to be built from source.
The problem here is that Pillow uses native libraries that must be built for the exact correct environment.
I solved this by installing my requirements in a Docker container that replicates very closely the AWS Lambda environment, lambci/lambda. I used the build-python3.8 version.
I installed my requirements there and zipped up the whole contents of /var/lang/lib/python3.8/site-packages/ directory along with my lambda function file.
I tried this with a standard Amazon Linux Docker image and it didn't work. Only the lambci/lambda image worked for me.