I have a python code on my desktop where I call Google APIs to get some data. I am trying to deploy it on AWS Lambda to run it periodically but I and running into some issues. Below are the steps I followed:
Downloaded google package using pip3 install google-api-python-client -t . Zipped this folder and uploaded it to a layer in AWS Lambda
Linked the layer with my function but when I am trying to execute the lambda function, I get the following error:
"errorMessage": "Unable to import module 'lambda_function': No module named 'googleapiclient'",
In my code I have the following import statement:
from googleapiclient.discovery import build
Please let me know if I am missing something and how to debug this.
Regards,
Dbeings
You typically receive this error when your Lambda environment can't find the specified library in the Python code. This is because Lambda isn't prepackaged with all third party python libraries.
in your local environment "googleapiclient" are compiled and available during runtime but it's not available in lambda runtime.
To resolve this error, create a deployment package(pre-compilied/zip) or Lambda layer that includes the libraries that you want to use in your Python code for Lambda.
best of luck
Related
I've created a Python script that use google-api-python-client library to pull data from my DFA account.
I would like to deploy this script to Google Cloud Scheduler so I can run this script on a daily basis. When I deployed the Cloud Function, I received an error that said:
"ModuleNotFoundError: No module named "googleapiclient".
After I've added "googleapiclient" to the requirements.txt and deployed it again, I got a new error that said it couldn't find the googleapiclient library to add.
May I ask if it's possible to install the googleapiclient library under the Google Cloud Platform?
Yes it's possible to install and use the googleapiclient library in GCP. Use googleapiclient module when importing the Google API Client in your python code.
import googleapiclient
Then add the google-api-python-client in your requirements.txt
# Function dependencies, for example:
# package>=version
google-api-python-client
You may check this sample code from github that uses the googleapiclient module and this requirements.txt.
I am relatively new to AWS Lambda and Python. Trying to create a deployment package for AWS Lambda using python-docx library on Windows for Python 3.8. It is failing with the error - cannot import name 'etree' from 'lxml'.
If the precompiling python-docx library is the solution, then can somebody please provide more information on this. Because the information I could find on this was 3 years old and for Python 2.7.
Note: To verify my package creation steps, I tried to create the deployment package just for requests package and it worked fine. Later on, updated the same python function to include python-docx and it is still failing with the above error.
I am getting the following error:
Unable to import module '': No module
named 'regex._regex'
The AWS Lambda deployment package runs just fine without import htmldate statement (the module I want to use) which in turn requires regex.
Also the code runs fine locally.
So this seems to be a problem running regex on AWS Lambda.
A new version of htmldate makes some of the dependencies optional, regex is such a case. That should solve the problem. (FYI: I'm the main developer of the package.)
If it runs locally and not in the lambda it may be an issue with the package installation. You may want to install your requirements.txt via a docker replicating the lambdas environment. If it works locally this can be used to ensure you are replicating the environment your lambda is running in during installation.
This docker image can be used to help:
https://hub.docker.com/r/lambci/lambda/
There are some examples specified here: https://github.com/lambci/docker-lambda#build-examples
Trying to successfully run numpy on AWS Lambda. The information I have read indicates that you need to use numpy libraries specifically compiled/compatible with AWS EC2.
Do do this I first followed the instruction to compile numpy on an ec2 instance, here:
Installing numpy on Amazon EC2
I then copied the newly built numpy into my Lambda application folder on my desktop, zipped up a Lambda deployment package containing the entire directory.
Upon running my Lambda function I still get this error:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all files not under version control). Otherwise reinstall numpy.
Suggestion?
Adding an answer for anyone who finds this old question.
Luckily, this problem has now been solved with Lambda Layer. AWS even provides a NumPy and SciPy layer. You can attach it directly to your Lambda in the web console or use this ARN arn:aws:lambda:us-east-1:668099181075:layer:AWSLambda-Python36-SciPy1x:2
An easy way to make your lambda function support the numpy library for python 3.7:
Go to your lambda function page
Find the Layers section at the bottom of the page.
Click on Add a layer.
Choose AWS layers as layer source.
Select AWSLambda-Python37-Scipy1x as AWS layers.
Select 37 for version.
And finally click on Add.
Now your lambda function is ready to support numpy.
I am writing after a lot of searching and trial and error with no luck.
I am trying to deploy a service in app engine.
You might be aware that deploying on app engine is usually practiced a two step process
1. Deploy on local dev app server
2. If step 1 succeeds deploy on cloud
My problems are with step 1 when I include third party python libraries such as numpy, sklearn, gcloud etc.
I am trying to deploy a service in local devapp server. When I import numpy or any other third party libraries in my main.py script it throws an error saying unable to find the module.
I am using cloud sdk and have two python distributions, the default python 2.7 and anaconda with python 2.7. When I change the path to look for the modules in anaconda distribution, it fails to find module ‘setup’ required by the cloud sdk.
Is there a way to install the cloud sdk for anaconda distribution ?
Any help/pointers will be much appreciated!
When using app engine python standard environment, you can install pure python 3rd party libs using pip by vendoring them as explained here.
There are also a number of libraries included in the python27 runtime which can be requested using the libraries directive in your app.yaml as explained here.
If there's a lib which is not pure python (i.e it uses C extensions) that you want to use in your project, and it's not part of this list, then your only option is to use a flexible VM. If you want to use anaconda, you should consider customizing the runtime for your flexible VM.