I am trying to use Firebase Native mode on Google App Engine - Standard. My python language is Python 2.7. When I try to from google.cloud import firestore, I get an error ImportError: cannot import name cygrpc
I have deployed virtualenv described in the documentation here.
pip install virtualenv
virtualenv env
source env/bin/activate
My appengine_config.py is
from google.appengine.ext import vendor
import os.path
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
my_app.py includes
from google.appengine.ext.webapp import template
from google.appengine.ext import ndb
from google.appengine.api import mail
import os
from google.cloud import firestore
(/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:269)
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 311, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/main.py", line 4, in <module>
from controllers import server, common, header
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/controllers/server.py", line 10, in <module>
from google.cloud import firestore
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore.py", line 18, in <module>
from google.cloud.firestore_v1 import __version__
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/__init__.py", line 22, in <module>
from google.cloud.firestore_v1._helpers import GeoPoint
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/_helpers.py", line 21, in <module>
import grpc
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/grpc/__init__.py", line 23, in <module>
from grpc._cython import cygrpc as _cygrpc
ImportError: cannot import name cygrpc
The question I have - can you use Firestore Native mode on Google App Engine Standard using Python 2.7? I need GAE- Standard because we are using Google Endpoint that is not support on GAE-Flex.
The doc here says App Engine Client library integration is not supported on Python2.7 GAE Standard environment. But I am not trying App Engine Client library, I am trying App Engine Server library in GAE Standard Environment.
How do I solve for import error for cygrpc? The solution here, says -
python -m pip install grpcio --ignore-installed
Is this recommended?
It's 2020, and you can now use grpcio (without explicitly adding it yourself as it's a built-in library) w/Cloud Firestore natively on App Engine with Python 2.7. Three things to make this work:
Add google-cloud-firestore to your requirements.txt file and install with pip install -U -t lib -r requirements.txt.
Add these lines to the libraries section of your app.yaml file:
libraries:
- name: grpcio
version: 1.0.0
- name: setuptools
version: 36.6.0
Ensure these lines are in your appengine_config.py file:
import pkg_resources
from google.appengine.ext import vendor
# Set path to your libraries folder.
path = 'lib'
# Add libraries installed in the path folder.
vendor.add(path)
# Add libraries to pkg_resources working set to find the distribution.
pkg_resources.working_set.add_entry(path)
Of course, we do recommend you eventually migrate to Python 3 to take advantage of the greater flexibility the next generations (of Python and App Engine) provide, esp. the ability to tap into other GCP services. However, caveat such a port isn't without effort if your app is complex & deeply-dependent on App Engine's 1st gen built-in services. Most of the suggestions above are derived from this section of the migration docs.
Once you're on Python 3, everything related to 3P libraries on App Engine becomes much easier, as I demonstrate in this other SO answer.
A while ago GRPC wasn't supported on GAE standard, see GRPC and types import error in App Engine Datastore. I didn't try since, but I don't see newer activity on issue 149.
Also the cython reference in the traceback suggests that it may include compiled code, which would violate the pure python standard environment sandbox restrictions that applies to the code you deploy.
Make sure that your installed python version bit(32/64) and your system bit(32/64) are matched.
If they are not matched please follow the solution mentioned at https://azurelessons.com/cannot-import-name-cygrpc/
Related
I am trying to run a BigQuery query from Google AppEngine (deployed) using Python 2.7, but I am seeing this error in StackDriver's Error Reporting:
ImportError: No module named cloud
This is my code (main.py):
from __future__ import absolute_import
import webapp2
from google.cloud import bigquery
class MainPage(webapp2.RequestHandler):
def get(self):
# Instantiates a client
bigquery_client = bigquery.Client()
# The name for the new dataset
dataset_name = 'my_new_set'
# Prepares the new dataset
dataset = bigquery_client.dataset(dataset_name)
# Creates the new dataset
dataset.create()
# Remove unwanted chars
#self.response.write(str(container))
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
This is my (app.yaml):
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
The error message would make me assume that the BigQuery's library is not being imported. However, if this code is being deployed in AppEngine, shouldn't the library already be installed in AppEngine by default?
Trying to solve the problem
Attempt # 1
I found this post that refers to a similar issue. The suggestion was to add this line to the top of the file. I added the line to my file, but the problem still exists:
from __future__ import absolute_import
Source:
No module named cloud while using google.cloud import bigquery
Attempt # 2
I installed BigQuery's client locally in my laptop:
pip install google-cloud-bigquery==0.22.1
I also installed the same client in the "lib" folder to have it uploaded to AppEngine once it is deployed:
pip install --target='lib' google-cloud-bigquery==0.22.1
This last, also requires a file named "appengine_config.py" to be created with this content:
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
Source: https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27
However, this attempt did not work either. The error message changed to the following:
*File "/base/data/home/apps/p~experimenting-1130/2.400173726395247238/lib/httplib2/__init__.py", line 352: print('%s:' % h, end=' ', file=self._fp) ^ SyntaxError: invalid syntax
at <module> (/base/data/home/apps/p~experimenting-1130/2.400173726395247238/lib/google_auth_httplib2.py:23)
at <module> (/base/data/home/apps/p~experimenting-1130/2.400173726395247238/lib/google/cloud/_helpers.py:31)
at <module> (/base/data/home/apps/p~experimenting-1130/2.400173726395247238/lib/google/cloud/bigquery/_helpers.py:21)
at <module> (/base/data/home/apps/p~experimenting-1130/2.400173726395247238/lib/google/cloud/bigquery/__init__.py:26)
at get (/base/data/home/apps/p~experimenting-1130/2.400173726395247238/main.py:75)
at dispatch (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:545)
at dispatch (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:547)
at __call__ (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:1077)
at default_dispatcher (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:1253)
at __call__ (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:1505)
at __call__ (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py:1511)*
How can I import the BigQuery library correctly in AppEngine (deployed)?
Thanks for your help.
The following solution worked for me without having to use from __future__ import absolute_import. There are three main steps to follow.
1. Copy google-api-python-client and google-cloud into project folder
Even though it sounds counterintuitive, according to the documentation
[...] Python client libraries are not installed in the App Engine Python runtime environment, [so] they must be vendored into your application just like third-party libraries.
So in order to use google.cloud one must copy the library code into the project's source directory. The library code, along with the application code, is uploaded to App Engine.
To copy a library code into your project:
Create a directory (e.g. lib/) to store third-party libraries in your project's root folder
mkdir lib
Copy the google-api-python-client and google-cloud libraries into the folder you just created. I use pip in the following example.
pip install -t lib/ --upgrade google-api-python-client
pip install -t lib/ --upgrade google-cloud
2. Link installed libraries to app
Create a file named appengine_config.pyin the same folder as your app.yaml file
Edit appengine_config.py and include the following code
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
3. Include added libraries to requirements.txt
Edit your requirements.txt file and include the names of your added libraries
# other requirements
google-api-python-client
google-cloud
You should now be able to use from google.cloud import bigquery with no problem after deploying your app.
For more information see using third-party libraries
I was trying to use pyDNS on GAE.
In Base.py there is import asyncore
Thats what I get:
File "/base/data/home/apps/myapp/lib/DNS/Base.py", line 14, in <module>
import asyncore
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/asyncore.py", line 608, in <module>
import fcntl
ImportError: No module named fcntl
What should I do in this case?
If the library implementations meets the GAE restrictions you can include it and upload it to GAE as part of your application.
Here are the official instructions for Adding Third-party Packages to the Application (sometimes called vendoring it in).
Actually there is nothing to do about this, besides avoiding modules that depend on unsupported modules.
GAE just does not support fcntl spawning threads module.
I am in the process of learning the RobotFramework from an absolute beginners point.
I have had experience creating automation scripts using ruby-watir and gherkin. As well as automation in c#.
However I am simply stumped with even the setup of RobotFramework in Pycharm.
I have installed all the packages
I deemed necessary
-including selenium/RobotFramework /RobotFramework-selenium2library.
My Python.exe and scripts are in the correct directory as well as the environment variables being added.
However when I run :
from lettuce import *
from behave import *
from Selenium import webdriver
#step("I am on the web")
def step_impl(context):
context.browser = webdriver.Firefox
pass
I get:
Traceback (most recent call last):
File "C:/Users/Jordan/PycharmProjects/RbtFrameWork/features/steps/Test_steps.py", line 3, in <module>
from Selenium import webdriver
ImportError: No module named Selenium
I imagine it's something to do with the installation location of my selenium - which I have installed through pip
Any help?
Python is case sensitive so instead of from Selenium import webdriver you should use from selenium import webdriver
I am trying to install SQLAlchemy on a Windows 8 64 bit machine for Python 2.7
When I run the setup.py file that comes with the download from the SQLAlchemy site, a .egg file called 'SQLAlchemy-1.0.0-py2.7.egg' is placed in C:\Python27\Lib\site-packages.
But when I try to do a basic import of the library using an import statement on given in the SQLAlchemy documentation, I get an error message.
The import code is:
from flask.ext.sqlalchemy import SQLAlchemy
And the error message I get is:
c:\Users\Me\MyCode>ws_dbwrite.py
Traceback (most recent call last):
File "C:\Users\Me\MyCode\ws_dbwrite.py", line 2, in <module>
from flask.ext.sqlalchemy import SQLAlchemy
File "C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg\flask\exthook.py",
line 87, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.sqlalchemy
Should I be going about installation in a different manner? Or should the .egg file for SQLAlchemy go at a different file level? Or am I missing something obvious here?
Edit: If I change the import line to just
import SQLAlchemy
The import works, but Python does not recognize things like
db = SQLAlchemy(app)
as valid commands.
You have installed SQLAlchemy, but you are trying to use the Flask extension, Flask-SQLAlchemy. While the two are related, they are separate libraries. In order to use
from flask.ext.sqlalchemy import SQLAlchemy
you need to install it first.
pip install Flask-SQLAlchemy
(You installed SQLAlchemy directly from source. You can also do that for Flask-SQLAlchemy.)
I understand that google.appengine.dist was removed from python2.7. What should I use instead?
application works fine on localhost, deployment seems to be successful, but I am getting this message online:
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Here is what my log says:
Traceback (most recent call last):
File
"/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 196, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 255, in _LoadHandler
handler = __import__(path[0])
File "/base/data/home/apps/s~quick-test/1.368856085074570769/django_bootstrap.py", line 54, in <module>
from google.appengine.dist import use_library
ImportError: No module named dist
Here's the line 54 from django_bootstrap.py:
from google.appengine.dist import use_library
use_library('django', '1.2')
GAE LAuncher version 1.8.2, had this problem with 1.8 too.
Python 2.7 configuration requires third-party libraries specified in app.yaml, so it might be sufficient to have this in your app.yaml file:
libraries:
- name: django
version: "1.2"
You can also use webapp2 that includes Django’s templating engine. Version 1.2 included with the SDK is part of App Engine, and you do not need to bundle Django yourself to use it.
import os
from google.appengine.ext.webapp import template
Also, with this code you don't need to call use_library() to explicitly select a Django version:
webapp_django_version = "1.2"