I was trying to use pyDNS on GAE.
In Base.py there is import asyncore
Thats what I get:
File "/base/data/home/apps/myapp/lib/DNS/Base.py", line 14, in <module>
import asyncore
File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/asyncore.py", line 608, in <module>
import fcntl
ImportError: No module named fcntl
What should I do in this case?
If the library implementations meets the GAE restrictions you can include it and upload it to GAE as part of your application.
Here are the official instructions for Adding Third-party Packages to the Application (sometimes called vendoring it in).
Actually there is nothing to do about this, besides avoiding modules that depend on unsupported modules.
GAE just does not support fcntl spawning threads module.
Related
I am trying to use Firebase Native mode on Google App Engine - Standard. My python language is Python 2.7. When I try to from google.cloud import firestore, I get an error ImportError: cannot import name cygrpc
I have deployed virtualenv described in the documentation here.
pip install virtualenv
virtualenv env
source env/bin/activate
My appengine_config.py is
from google.appengine.ext import vendor
import os.path
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
my_app.py includes
from google.appengine.ext.webapp import template
from google.appengine.ext import ndb
from google.appengine.api import mail
import os
from google.cloud import firestore
(/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:269)
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 311, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/main.py", line 4, in <module>
from controllers import server, common, header
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/controllers/server.py", line 10, in <module>
from google.cloud import firestore
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore.py", line 18, in <module>
from google.cloud.firestore_v1 import __version__
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/__init__.py", line 22, in <module>
from google.cloud.firestore_v1._helpers import GeoPoint
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/_helpers.py", line 21, in <module>
import grpc
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/grpc/__init__.py", line 23, in <module>
from grpc._cython import cygrpc as _cygrpc
ImportError: cannot import name cygrpc
The question I have - can you use Firestore Native mode on Google App Engine Standard using Python 2.7? I need GAE- Standard because we are using Google Endpoint that is not support on GAE-Flex.
The doc here says App Engine Client library integration is not supported on Python2.7 GAE Standard environment. But I am not trying App Engine Client library, I am trying App Engine Server library in GAE Standard Environment.
How do I solve for import error for cygrpc? The solution here, says -
python -m pip install grpcio --ignore-installed
Is this recommended?
It's 2020, and you can now use grpcio (without explicitly adding it yourself as it's a built-in library) w/Cloud Firestore natively on App Engine with Python 2.7. Three things to make this work:
Add google-cloud-firestore to your requirements.txt file and install with pip install -U -t lib -r requirements.txt.
Add these lines to the libraries section of your app.yaml file:
libraries:
- name: grpcio
version: 1.0.0
- name: setuptools
version: 36.6.0
Ensure these lines are in your appengine_config.py file:
import pkg_resources
from google.appengine.ext import vendor
# Set path to your libraries folder.
path = 'lib'
# Add libraries installed in the path folder.
vendor.add(path)
# Add libraries to pkg_resources working set to find the distribution.
pkg_resources.working_set.add_entry(path)
Of course, we do recommend you eventually migrate to Python 3 to take advantage of the greater flexibility the next generations (of Python and App Engine) provide, esp. the ability to tap into other GCP services. However, caveat such a port isn't without effort if your app is complex & deeply-dependent on App Engine's 1st gen built-in services. Most of the suggestions above are derived from this section of the migration docs.
Once you're on Python 3, everything related to 3P libraries on App Engine becomes much easier, as I demonstrate in this other SO answer.
A while ago GRPC wasn't supported on GAE standard, see GRPC and types import error in App Engine Datastore. I didn't try since, but I don't see newer activity on issue 149.
Also the cython reference in the traceback suggests that it may include compiled code, which would violate the pure python standard environment sandbox restrictions that applies to the code you deploy.
Make sure that your installed python version bit(32/64) and your system bit(32/64) are matched.
If they are not matched please follow the solution mentioned at https://azurelessons.com/cannot-import-name-cygrpc/
I am working in pycharm(in windows) environment.
I am building a website to send query to datastore and display values. I also have data under kind="SpecialTest" that is automatically saved through Node.js. Now I wanted to Query the data from Python.
I installed google-cloud-datastore from command line.
But I can't establish connection to datastore.
My code:
from google.cloud import datastore
cli=datastore.Client(project="My_project_I")
query4= cli.query(kind="SpecialTest").order('Published_at')
print query4
My result:
Internal Server Error
The server has either erred or is incapable of performing the requested operation.
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1272, in default_dispatcher
self.handlers[handler] = handler = import_string(handler)
File "C:\Program Files\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1850, in import_string
return getattr(__import__(module, None, None, [obj]), obj)
File "C:\Users\Harry\workspace\www\app\account.py", line 11, in <module>
from google.cloud import datastore
File "C:\Python27\lib\site-packages\google\cloud\datastore\__init__.py", line 60, in <module>
from google.cloud.datastore.batch import Batch
File "C:\Python27\lib\site-packages\google\cloud\datastore\batch.py", line 24, in <module>
from google.cloud.datastore import helpers
File "C:\Python27\lib\site-packages\google\cloud\datastore\helpers.py", line 23, in <module>
from google.protobuf import struct_pb2
ImportStringError: import_string() failed for 'app.account.UserAccount'. Possible reasons are:
- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;
Original exception:
ImportError: No module named protobuf
Debugged import:
- 'app' found in 'C:\\Users\\Harry\\workspace\\www\\app\\__init__.pyc'.
- 'app.account' not found.
The datastore client library is not supplied by GAE, you need to install/vendor it into your application (different method than for a standalone script).
Alternatively you can use the GAE-specific datastore library - it's actually recommended as it offers some advantages over the general purpose one. From Google App Engine Standard Environment Client Libraries:
Integrate Cloud Datastore with your App Engine Standard Environment
applications by using the App Engine client libraries.
...
Python Google Datastore NDB Client Library
Note: For App Engine applications that are written in Python, the Google Datastore DB Client Library is no longer recommended; use the
Google Datastore NDB Client Library instead.
my tests work fine when my target is a single function (see 'Target' field in the image):
questionator.test_mturk_views.TestReport.submit
However, when I specify my target to include all tests within my questionator app:
questionator
I get this error:
Error ImportError: Failed to import test module:
src.questionator.test_mturk_views Traceback (most recent call last):
File "C:\Python27\Lib\unittest\loader.py", line 254, in _find_tests
module = self._get_module_from_name(name) File "C:\Python27\Lib\unittest\loader.py", line 232, in
_get_module_from_name
import(name) File "C:\Users\Andy\questionator_app\src__init__.py", line 5, in
from .celery import app as celery_app # noqa ImportError: No module named celery
Note that my tests include my settings via 'Environment variables' (see this in the pic too):
DJANGO_SETTINGS_MODULE=questionator_app.settings.development;PYTHONUNBUFFERED=1
The celery documentation mentions a "Using a custom test runner to test with celery" but this is in the now defunct djcelery package. I did though copy/paste/tweak this mentioned test runner and used it as described, but I get the same error.
Unfortunately using CELERY_ALWAYS_EAGER also does not work http://docs.celeryproject.org/en/latest/configuration.html#celery-always-eager
I would appreciate some guidance. With best wishes,
Andy.
with-the-same-problem (most likely me),
I had followed the official tutorial for getting celery working in my project. They advised the below:
Just making the last import explicit solved my problem:
from taskapp.celery import app as celery_app # noqa
I'll see if I can nudge Celery's creators to update their tutorial (pull request).
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
I am trying to install SQLAlchemy on a Windows 8 64 bit machine for Python 2.7
When I run the setup.py file that comes with the download from the SQLAlchemy site, a .egg file called 'SQLAlchemy-1.0.0-py2.7.egg' is placed in C:\Python27\Lib\site-packages.
But when I try to do a basic import of the library using an import statement on given in the SQLAlchemy documentation, I get an error message.
The import code is:
from flask.ext.sqlalchemy import SQLAlchemy
And the error message I get is:
c:\Users\Me\MyCode>ws_dbwrite.py
Traceback (most recent call last):
File "C:\Users\Me\MyCode\ws_dbwrite.py", line 2, in <module>
from flask.ext.sqlalchemy import SQLAlchemy
File "C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg\flask\exthook.py",
line 87, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.sqlalchemy
Should I be going about installation in a different manner? Or should the .egg file for SQLAlchemy go at a different file level? Or am I missing something obvious here?
Edit: If I change the import line to just
import SQLAlchemy
The import works, but Python does not recognize things like
db = SQLAlchemy(app)
as valid commands.
You have installed SQLAlchemy, but you are trying to use the Flask extension, Flask-SQLAlchemy. While the two are related, they are separate libraries. In order to use
from flask.ext.sqlalchemy import SQLAlchemy
you need to install it first.
pip install Flask-SQLAlchemy
(You installed SQLAlchemy directly from source. You can also do that for Flask-SQLAlchemy.)