my tests work fine when my target is a single function (see 'Target' field in the image):
questionator.test_mturk_views.TestReport.submit
However, when I specify my target to include all tests within my questionator app:
questionator
I get this error:
Error ImportError: Failed to import test module:
src.questionator.test_mturk_views Traceback (most recent call last):
File "C:\Python27\Lib\unittest\loader.py", line 254, in _find_tests
module = self._get_module_from_name(name) File "C:\Python27\Lib\unittest\loader.py", line 232, in
_get_module_from_name
import(name) File "C:\Users\Andy\questionator_app\src__init__.py", line 5, in
from .celery import app as celery_app # noqa ImportError: No module named celery
Note that my tests include my settings via 'Environment variables' (see this in the pic too):
DJANGO_SETTINGS_MODULE=questionator_app.settings.development;PYTHONUNBUFFERED=1
The celery documentation mentions a "Using a custom test runner to test with celery" but this is in the now defunct djcelery package. I did though copy/paste/tweak this mentioned test runner and used it as described, but I get the same error.
Unfortunately using CELERY_ALWAYS_EAGER also does not work http://docs.celeryproject.org/en/latest/configuration.html#celery-always-eager
I would appreciate some guidance. With best wishes,
Andy.
with-the-same-problem (most likely me),
I had followed the official tutorial for getting celery working in my project. They advised the below:
Just making the last import explicit solved my problem:
from taskapp.celery import app as celery_app # noqa
I'll see if I can nudge Celery's creators to update their tutorial (pull request).
Related
I created a small app in Django and runserver and admin works fine.
I wrote some tests which can call with python manage.py test and the tests pass.
Now I would like to call one particular test via PyCharm.
This fails like this:
/home/guettli/x/venv/bin/python
/snap/pycharm-community/179/plugins/python-ce/helpers/pycharm/_jb_pytest_runner.py
--path /home/guettli/x/xyz/tests.py
Launching pytest with arguments /home/guettli/x/xyz/tests.py in /home/guettli/x
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 --
cachedir: .pytest_cache
rootdir: /home/guettli/x
collecting ...
xyz/tests.py:None (xyz/tests.py)
xyz/tests.py:6: in <module>
from . import views
xyz/views.py:5: in <module>
from xyz.models import Term, SearchLog, GlobalConfig
xyz/models.py:1: in <module>
from django.contrib.auth.models import User
venv/lib/python3.6/site-packages/django/contrib/auth/models.py:2: in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
venv/lib/python3.6/site-packages/django/contrib/auth/base_user.py:47: in <module>
class AbstractBaseUser(models.Model):
venv/lib/python3.6/site-packages/django/db/models/base.py:107: in __new__
app_config = apps.get_containing_app_config(module)
venv/lib/python3.6/site-packages/django/apps/registry.py:252: in get_containing_app_config
self.check_apps_ready()
venv/lib/python3.6/site-packages/django/apps/registry.py:134: in check_apps_ready
settings.INSTALLED_APPS
venv/lib/python3.6/site-packages/django/conf/__init__.py:76: in __getattr__
self._setup(name)
venv/lib/python3.6/site-packages/django/conf/__init__.py:61: in _setup
% (desc, ENVIRONMENT_VARIABLE))
E django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS,
but settings are not configured. You must either define the environment variable
DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Assertion failed
collected 0 items / 1 error
I understand the background: My app xyz is reusable. It does not contain any settings.
The app does not know (and should not know) my project. But the settings are in my project.
How to solve this?
I read the great django docs, but could not find a solution.
How to set DJANGO_SETTINGS_MODULE if you execute one particular test directly from PyCharm with "Run" (ctrl-shift-F10)?
You can add DJANGO_SETTINGS_MODULE as an environmental variable:
In the menu: Run -> Edit Configurations -> Templates -> Python Tests -> Unittests
And delete old "Unittests for tests...." entries.
You can specify the settings in your test command
Assuming your in the xyz directory, and the structure is
/xyz
- manage.py
- xyz/
- settings.py
The following command should work
python manage.py test --settings=xyz.settings
Edited: For this method to work django support should me enabled in pycharm. I guess it should be possible to setup the equivalent template in the community edition version of pycharm.
Method with django support enabled:
I find that the most convenient way that also allow you to directly click on a particular test case and run it directly within pycharm without having to set the settings every time is to do the following:
->Edit configuration (Run/Debug configurations)
->Templates and select "Django Tests"
->Tick "Custom settings" and then browse to the settings you want use.
Then when you launch tests directly within pycharm it will use it as a template.
If you test with any other supported method by pycharm, you can pick testing framework in pycharm: Choose testing framework
and then create a template for it.
If you use django and pytest, then I recommend the plugin pytest-django
It provides a simple way to set DJANGO_SETTINGS_MODULE via configuration.
See configuring django
[pytest]
DJANGO_SETTINGS_MODULE = test_settings
I am trying to use Firebase Native mode on Google App Engine - Standard. My python language is Python 2.7. When I try to from google.cloud import firestore, I get an error ImportError: cannot import name cygrpc
I have deployed virtualenv described in the documentation here.
pip install virtualenv
virtualenv env
source env/bin/activate
My appengine_config.py is
from google.appengine.ext import vendor
import os.path
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
my_app.py includes
from google.appengine.ext.webapp import template
from google.appengine.ext import ndb
from google.appengine.api import mail
import os
from google.cloud import firestore
(/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py:269)
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 311, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/43d5822312de17fd/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/main.py", line 4, in <module>
from controllers import server, common, header
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/controllers/server.py", line 10, in <module>
from google.cloud import firestore
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore.py", line 18, in <module>
from google.cloud.firestore_v1 import __version__
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/__init__.py", line 22, in <module>
from google.cloud.firestore_v1._helpers import GeoPoint
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/google/cloud/firestore_v1/_helpers.py", line 21, in <module>
import grpc
File "/base/data/home/apps/s~openbarn-prod/20190602t102855.418624175446659791/lib/grpc/__init__.py", line 23, in <module>
from grpc._cython import cygrpc as _cygrpc
ImportError: cannot import name cygrpc
The question I have - can you use Firestore Native mode on Google App Engine Standard using Python 2.7? I need GAE- Standard because we are using Google Endpoint that is not support on GAE-Flex.
The doc here says App Engine Client library integration is not supported on Python2.7 GAE Standard environment. But I am not trying App Engine Client library, I am trying App Engine Server library in GAE Standard Environment.
How do I solve for import error for cygrpc? The solution here, says -
python -m pip install grpcio --ignore-installed
Is this recommended?
It's 2020, and you can now use grpcio (without explicitly adding it yourself as it's a built-in library) w/Cloud Firestore natively on App Engine with Python 2.7. Three things to make this work:
Add google-cloud-firestore to your requirements.txt file and install with pip install -U -t lib -r requirements.txt.
Add these lines to the libraries section of your app.yaml file:
libraries:
- name: grpcio
version: 1.0.0
- name: setuptools
version: 36.6.0
Ensure these lines are in your appengine_config.py file:
import pkg_resources
from google.appengine.ext import vendor
# Set path to your libraries folder.
path = 'lib'
# Add libraries installed in the path folder.
vendor.add(path)
# Add libraries to pkg_resources working set to find the distribution.
pkg_resources.working_set.add_entry(path)
Of course, we do recommend you eventually migrate to Python 3 to take advantage of the greater flexibility the next generations (of Python and App Engine) provide, esp. the ability to tap into other GCP services. However, caveat such a port isn't without effort if your app is complex & deeply-dependent on App Engine's 1st gen built-in services. Most of the suggestions above are derived from this section of the migration docs.
Once you're on Python 3, everything related to 3P libraries on App Engine becomes much easier, as I demonstrate in this other SO answer.
A while ago GRPC wasn't supported on GAE standard, see GRPC and types import error in App Engine Datastore. I didn't try since, but I don't see newer activity on issue 149.
Also the cython reference in the traceback suggests that it may include compiled code, which would violate the pure python standard environment sandbox restrictions that applies to the code you deploy.
Make sure that your installed python version bit(32/64) and your system bit(32/64) are matched.
If they are not matched please follow the solution mentioned at https://azurelessons.com/cannot-import-name-cygrpc/
I am working through the Python Quickstart for Cloud Endpoints Frameworks on App Engine. I try to generate the OpenAPI configuration file by invoking the Endpoints Tool and get this error:
python lib/endpoints/endpointscfg.py get_swagger_spec main.EchoApi --hostname echo.endpoints.[YOUR-PROJECT-ID].cloud.goog
Traceback (most recent call last):
File "lib/endpoints/endpointscfg.py", line 59, in <module>
import _endpointscfg_setup # pylint: disable=unused-import
File "/Users/myName/lab/python-docs-samples/appengine/standard/endpoints-frameworks-v2/echo/lib/endpoints/_endpointscfg_setup.py", line 98, in <module>
_SetupPaths()
File "/Users/myName/lab/python-docs-samples/appengine/standard/endpoints-frameworks-v2/echo/lib/endpoints/_endpointscfg_setup.py", line 94, in _SetupPaths
from google.appengine.ext import vendor
ImportError: No module named appengine.ext
My understanding is that this is some kind of packaging issue? This is issue is discussed here but i am still stuck: Error running endpointscfg.py get_swagger_spec. Any help would be much appreciated.
It turned out to be an issue with my system variable path. Specifically I added:
ENDPOINTS_GAE_SDK="path to google_appengine"
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
I understand that google.appengine.dist was removed from python2.7. What should I use instead?
application works fine on localhost, deployment seems to be successful, but I am getting this message online:
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Here is what my log says:
Traceback (most recent call last):
File
"/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 196, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 255, in _LoadHandler
handler = __import__(path[0])
File "/base/data/home/apps/s~quick-test/1.368856085074570769/django_bootstrap.py", line 54, in <module>
from google.appengine.dist import use_library
ImportError: No module named dist
Here's the line 54 from django_bootstrap.py:
from google.appengine.dist import use_library
use_library('django', '1.2')
GAE LAuncher version 1.8.2, had this problem with 1.8 too.
Python 2.7 configuration requires third-party libraries specified in app.yaml, so it might be sufficient to have this in your app.yaml file:
libraries:
- name: django
version: "1.2"
You can also use webapp2 that includes Django’s templating engine. Version 1.2 included with the SDK is part of App Engine, and you do not need to bundle Django yourself to use it.
import os
from google.appengine.ext.webapp import template
Also, with this code you don't need to call use_library() to explicitly select a Django version:
webapp_django_version = "1.2"