ImportError: no module named [directory] Google App Engine - python-2.7

I keep getting an error that says no module named backend, this is the directory where my webapp2 application is.
My folder structure:
/project
/backend
/env #python virtual env libraries
main.py #my main entry point where webapp2 app instance is
requirements.txt
app.yaml
My app.yaml:
service: default
handlers:
- url: /dist
static_dir: dist
- url: /.*
script: backend.main.app
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
Before my app.yaml was in backend, but I decided to move to root. Now when I run dev_appserver.py in root, I keep getting ImportError: No module named backend
I created the virualenv and installed the requirements.txt packages inside the backend directory.
EDIT: I am unsure if this makes a difference, but I have already deployed my application when the app.yaml was inside the backend folder. I am guessing this should not matter since I am trying to test locally by moving the app.yaml in my project root and running dev_appserver.py app.yaml, but it seems to not work when I do this.

The directory containing the app.yaml file for a GAE service is the service's top-level directory. The content of this directory is what will be uploaded to GAE when you deploy the service. All paths referenced in the service's code or configurations are relative to this top level dir. So moving the app.yaml file around without updating the related code and configurations accordingly will break the app's functionality.
You don't seem to grasp the meaning of the script: statement very well. From Handlers element:
A script: directive must be a python import path, for example,
package.module.app that points to a WSGI application. The last
component of a script: directive using a Python module path is the
name of a global variable in the module: that variable must be a WSGI
app, and is usually called app by convention.
Note: just like for a Python import statement, each subdirectory that
is a package must contain a file named __init__.py
So, assuming your app.yaml file is located in your project dir, the
script: backend.pythonAttack.app
would mean:
having an __init__.py file inside the backend dir, to make backend a package
having a pythonAttack.py file in the backend dir, with an app variable pointing to your webapp2 application
According to your description you don't meet any of these conditions.
My recommendation:
keep the app.yaml inside the backend dir (which doesn't need to be a python package dir)
update its script line to match your code. Assuming the app variable for your webapp2 app is actually in the main.py file the line would be:
script: main.app
run the app locally by explicitly passing the app.yaml file as argument (in general a good habit and also the only way to run apps with multiple services and/or a dispatch.yaml file):
dev_appserver.py backend/app.yaml
store your service python dependencies inside a backend/lib directory (to follow the naming convention), separated from your virtualenv packages
store the env virtualenv package dir outside the backend directory to prevent unnecessarily uploading them to GAE when deploying the service (and potential interference with the app's operation). The goal of the virtualenv is to properly emulate the GAE sandbox locally so that you can run the development server correctly.
Potentially of interest for structuring a multi-service app: Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?

Related

While importing 'myapp.app' an import error was raised

I'm building a flask application, located in a subdirectory within my project called myapp. Running gunicorn --bind 0.0.0.0:$PORT myapp.app:app works fine, no errors, regardless of what FLASK_APP is set to. When I try to use Flask's CLI, however, it fails to find my app (reasonable), but when I set FLASK_APP to myapp.app., it appears to be doubling up the import path, resulting in an import error:
FLASK_APP=myapp.app flask run
* Serving Flask app 'myapp.app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
Usage: flask run [OPTIONS]
Try 'flask run --help' for help.
Error: While importing 'myapp.myapp.app', an ImportError was raised.
How can I solve this? Is this a bug in Flask?
There was a __init__.py in my project directory that was causing this.
I dug into the source code of flask in cli.py and found the following code in prepare_import:
# move up until outside package structure (no __init__.py)
while True:
path, name = os.path.split(path)
module_name.append(name)
if not os.path.exists(os.path.join(path, "__init__.py")):
break
Because I had an __init__.py in my project directory (also called myapp), this made flask try to import myapp.app from the ancestor of my project, resulting in myapp.myapp.app.

Receiving error 502 BAD CONNECTION nginx when deploying django app on Google app engine

Ive been trying to my django website but I've been having the same 502 BAD CONNECTION nginx error every time and I'm not sure how to troubleshoot it.
When I run the program using py.manager runserver, it works perfectly fine, but but I only receive an error once I run gcloud app deploy, looking at the logs I haven't received any errors, just one warning stating that Container called exit(1).
I've tried one solution from the Google's website: https://cloud.google.com/endpoints/docs/openapi/troubleshoot-response-errors and I added resources: memory_gb: 4 to my app.yaml file and I still end up with an error.
I am not sure this is of much help my app.yaml currently looks like this:
runtime: python39
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static/
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
resources:
memory_gb: 4
And my gcloudignore looks like this:
# This file specifies files that are *not* uploaded to Google Cloud Platform
# using gcloud. It follows the same syntax as .gitignore, with the addition of
# "#!include" directives (which insert the entries of the given .gitignore-style
# file at that point).
#
# For more information, run:
# $ gcloud topic gcloudignore
#
.gcloudignore
# If you would like to upload your .git directory, .gitignore file or files
# from your .gitignore file, remove the corresponding line
# below:
.git
.gitignore
secret_variables.py
.venv
Pipfile
Pipfile.lock
# Python pycache:
__pycache__/
# Ignored by the build system
/setup.cfg
Thank you in advance for the help
Cloned and tried running the code locally after running pip install -r requirements.txt. This is the error I got:
ModuleNotFoundError: No module named 'django_filters'
This means that this requirement needs to be added before the code is deployed to the GAE.
add django-filter to the requirements.txt file to fix it.

What is the entry point that Apache mod_wsgi on OpenShift Online is looking for?

I've got an OpenShift Python 2.7 application that I am guessing uses mod_wsgi.
Is it possible to ssh in to OpenShift Online and view the .conf files located somewhere like:
/etc/apache2/sites-available/
I am wanting to see which .wsgi file Apache looks for as defined in WSGIScriptAlias.
Perhaps it just looks for /wsgi/application?
A few posts indicate that changes have been made recently to the structure of Python applications, but they may not effect my older version:
How to change or override openshift.conf in Python 3.3 cartridge
https://blog.openshift.com/openshift-online-march-2014-release-blog/
WSGI Application not found on OpenShift
Ideally, I'm trying to comprehend the order in which these files are executed and their functions:
/wsgi/application
/wsgi/my-bottle-application
setup.py
setup.pyc
setup.pyo
UPDATE
This indicates the entry point is wsgi/application:
https://github.com/openshift/origin-server/search?utf8=%E2%9C%93&q=OPENSHIFT_PYTHON_WSGI_APPLICATION
I'd still be interested to know the order of execution of the above files and exactly what setup.py does and how it is executed - ie there are no references to it in application so how is it 'called'?.
According to this you can set your entry point: https://blog.openshift.com/openshift-online-march-2014-release-blog/
Python
For python app’s we’ve made some similar changes:
We got rid of wsgi/, wsgi/static/, data/ and libs/ directories.
You can use wsgi.py instead of wsgi/application as the default WSGI entry-point.
We’ve discarded the README.md file that can often conflict with an upstream file of the same name.
New OPENSHIFT_PYTHON_WSGI_APPLICATION to set an alternative WSGI entry-point.
wsgi.py WSGI entry-point (configurable by $OPENSHIFT_PYTHON_WSGI_APPLICATION)
setup.py Standard setup.py, specify deps here
.openshift/ Location for OpenShift specific files
action_hooks/ See the Action Hooks documentation
markers/ See the Markers section below
For more information on environment variables on OpenShift Online: https://developers.openshift.com/en/managing-environment-variables.html
Example that worked for me using the OPENSHIFT_PYTHON_WSGI_APPLICATION variable:
rhc env-set OPENSHIFT_PYTHON_WSGI_APPLICATION="${OPENSHIFT_REPO_DIR}/server/wsgi.py" --app MyApp

Openshift wrapper vs. project repo

I've tried to find a concrete answer for this but failed. I'm rather new to openshift, django and git so I hope it's not too trivial and not a duplicate.
The situation is as follows:
1. I'm developing a Django based web application and source control it using git on a private bitbucket repo. I have my regular django application source tree as needed.
2. I wish to deploy my app on openshift which comes with a different directory tree. I've been able to successfully run this quickstart:
git://github.com/rancavil/django-openshift-quickstart.git
Basically what I try to achieve is a wrapper directory (and git project) which is based on the repo specified in #2. Within this wrapper one of the subdirectories (wsgi/my_project) should be based on my private repo specified in #1.
I wish to be able to pull recent changes from the private repo and then push and deploy to openshift these changes.
Any help would be greatly appreciated!
I was also struggling with openshift structure. But there is no need for doing the staff like you have proposed.
You can have any structure of a project you want. Openshift will need just wsgi/application or wsgi.py as an entry point. I think that having one more file in your project is hardly a problem.
My current structure for a project (got it running recently) is following:
|- .openshift (directory for build and deployment hooks like syncdb or collectstatic commands)
|- custom_project (directory that is you django project)
\- templates
|- settings.py
|- urls.py
|- manage.py
|- requirements.txt
|- wsgi.py
There is really no need to have project under wsgi folder. However there are few more pitfalls to be aware of. Like creating static folder or setting .htaccess for static files.
Including my wsgi.py:
#!/usr/bin/python
import os
import logging
try:
virtenv = os.environ['OPENSHIFT_PYTHON_DIR'] + '/virtenv/'
virtualenv = os.path.join(virtenv, 'bin/activate_this.py')
execfile(virtualenv, dict(__file__=virtualenv))
except (IOError, KeyError):
pass
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'custom_project.settings')
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
I have used a lot of resources to make my project running on openshift so I'm also including them for other.
bitbucket and openshit repos: Can I use my existing git repo with openshift?
Slides with necesary steps: http://appsembler.com/blog/paas-bakeoff-comparing-stackato-openshift-dotcloud-and-heroku-for-django-hosting-and-deployment/
And the most important one, the really well written step-by-step tutorial: http://www.paascheatsheet.com/#Introduction

location of settings.py for django project hosted in Google App Engine

I've problems running the simplest django project within GAE.
After running django-admin startproject myproj (Using django 1.4 from Google SDK), the folder hierarchy look as:
+ myproj (root folder of project)
- manage.py
+ myproj (a sub folder for the project files)
- __init__.py
- settings.py
- urls.py
- wsgi.py
I then add the usual app.yaml to the root folder - that is to the same folder where manage.py is.
application: u-pavarotti
version: 1
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: django
version: "1.4"
builtins:
- django_wsgi: on
syncdb, dbshell, shell ... works great. Even manage.py runserver works file. However, trying to run dev_appserver fails. The exception I get is:
ImportError: Could not import settings 'settings'.
Well, taking a look at google/appengine/ext/django/main (which django_wsgi is basically setting a URL handler for .* to the google.appengine.ext.django.main.app) is appears that it expect to find settings in the python path. But without doing anything, the module to import is myproj.settings (as myproj isn't in the path).
Now, adding .../myproj/myproj (the lower folder) to the PYTHONPATH solve everything on my dev machine, but as I have no control on the search path on the GAE deployment - this solution won't work.
I can move all files, or just settings.py up to the upper myproj, or move app.yaml down (kinda same thing), but it requires various changes form the default genereated settings.py - and I don't this is the way to go. Alternatively, I can write my own handler, which will instantiate django.core.handlers.wsge.WSGIHandler (old tutorials used to do that) - again, seems against the intention of at least the creators of the 'builtin' attribute of app.yaml.
I wonder how everyone else is solving this issue, and what is the right thing to do.
You can declare the environment variable DJANGO_SETTINGS_MODULE in your app.yaml:
env_variables:
DJANGO_SETTINGS_MODULE: 'myproj.settings'