Django 1.11 on passenger_wsgi not routing POST request - django

I'm trying to setup python on A2 shared hosting via passenger_wsgi.
The app is working fine when I run it via 'runserver'. I tested this both in my local PC, and via SSH tunnel.
However, when I try to set this up on passenger_wsgi, it can't seem to be able to route POST request.
1 import os
2 import sys
3
4 sys.path.insert(0, "/home/<username>/app")
5
6 import APP_CORE
7
8 # where is the python interpreter
9 INTERP = "/home/<username>/app/.virtualenv/bin/python"
10 if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)
11
12
13 os.environ['DJANGO_SETTINGS_MODULE'] = "APP_CORE.settings"
14
15 import APP_CORE.wsgi
16 application = APP_CORE.wsgi.application
Example: when I load the admin page (/admin/login), it can load the login page, but when submitting the credentials, it says that POST to /admin/login is not found - returning HTTP 404.
The SAME flow when I run via runserver works - I feel that I could be missing something in the django WSGI configuration. Any help would be appreciated !!
Edit/update: After diving into resolver.py and base.py:_get_response, I've noticed that somehow the /path/info truncates the first bit of the URL. Example, when I am requesting for /admin/login/, the path info only shows /login - but when I am using runserver, it is properly passed thru as /admin/login. To me this is clearly the issue on the web server setup and not on the django site. So will try to work it out with A2Hosting...

It looks like you may have solved this, but to followup for anyone that may stumble here. I have been using A2Hosting, Passenger, and CPanel with django (and wagtail). What I found was that during POST requests the wsgi SCRIPT_NAME was being set to a relative path and not the root of the application.
When I added logging to each application call, the correct GET request was:
{
'REQUEST_URI': '/admin/',
'PATH_INFO': '/admin/',
'SCRIPT_NAME': '',
'QUERY_STRING': '',
'REQUEST_METHOD': 'GET',
...
But on that page, a form was submitting a POST, which had the PATH_INFO incorrectly set:
{
'REQUEST_URI': '/admin/login/',
'PATH_INFO': '/login/',
'SCRIPT_NAME': '/admin',
'QUERY_STRING': '',
'REQUEST_METHOD': 'POST',
...
The workaround I ended up using was to create middleware which asserted a known SCRIPT_NAME and rebuilt the PATH_INFO from it.
# Set this to your root
SCRIPT_NAME = ''
class PassengerPathInfoFix(object):
"""
Sets PATH_INFO from REQUEST_URI since Passenger doesn't provide it.
"""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = get_wsgi_application()
application = PassengerPathInfoFix(application)
Related Reading:
http://alyalearningdjango.blogspot.com/2014/05/issue-360-passenger-doesnt-set-pathinfo.html
https://github.com/phusion/passenger/issues/460
https://www.python.org/dev/peps/pep-0333/#environ-variables

Related

Django Channels Websocket hanging - WebSocketProtocol took too long to shut down and was killed

Environment:
Ubuntu 16.04.6
conda 4.12.0
Apache/2.4.18 (Ubuntu)
python==3.8.1
Django==4.0.3
channels==3.0.5
asgi-redis==1.4.3
asgiref==3.4.1
daphne==3.0.2
I am attempting to create a websocket service that only relays messages from redis to an authenticated user. Users do not communicate with each other, therefore I don't need Channel layers and my understanding is that Channel layers are an entirely optional part of Channels.
I'm simply trying to broadcast messages specific to a user that has been authenticated through custom middleware. I have custom auth middleware that takes an incoming session id and returns the authenticated user dictionary along with a user_id. I'm also attaching this user_id to the scope.
I have elected to route all traffic through daphne via Apache2 using ProxyPass on port 8033. This is preferred since I'm using a single domain for this service.
However, I'm really struggling to maintain a connection to the websocket server, particularly if I refresh the browser. It will work on the first request, and fail after with the following message in journalctl:
Application instance <Task pending name='Task-22' coro=<ProtocolTypeRouter.__call__() running at /root/miniconda2/lib/python3.8/site-packages/channels/routing.py:71> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f658c03f670>()]>> for connection <WebSocketProtocol client=['127.0.0.1', 46010] path=b'/ws/userupdates/'> took too long to shut down and was killed.
After spending hours on the github for channels, and trying many of the suggestions (particularly found on https://github.com/django/channels/issues/1119), I'm still at a loss. The version of the code below is the best working version so far that at least establishes an initial connection, sends back the connection payload {"success": true, "user_id": XXXXXX, "message": "Connected"} and relays all redis messages successfully. But, if I refresh the browser, or close and reopen, it fails to establish a connection and posts the above journalctl message until I restart apache and daphne.
My hunch is that I'm not properly disconnecting the consumer or I'm missing proper use of async await. Any thoughts?
Relevant apache config
RewriteEngine On
RewriteCond %{HTTP:Connection} Upgrade [NC]
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteRule /(.*) ws://127.0.0.1:8033/$1 [P,L]
<Location />
ProxyPass http://127.0.0.1:8033/
ProxyPassReverse /
</Location>
app/settings.py
[...]
ASGI_APPLICATION = 'app.asgi.application'
ASGI_THREADS = 1000
CHANNEL_LAYERS = {}
[...]
app/asgi.py
import os
from django.core.asgi import get_asgi_application
from django.urls import path
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import AllowedHostsOriginValidator
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')
import websockets.routing
from user.models import UserAuthMiddleware
application = ProtocolTypeRouter({
'http': get_asgi_application(),
"websocket": AllowedHostsOriginValidator(
UserAuthMiddleware(
URLRouter(websockets.routing.websocket_urlpatterns)
)
),
})
user/models.py::UserAuthMiddleWare
Pulls user_id from custom authentication layer and attaches user_id to scope.
class UserAuthMiddleware(CookieMiddleware):
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
# Check this actually has headers. They're a required scope key for HTTP and WS.
if "headers" not in scope:
raise UserSessionError(
"UserAuthMiddleware was passed a scope that did not have a headers key "
+ "(make sure it is only passed HTTP or WebSocket connections)"
)
# Go through headers to find the cookie one
for name, value in scope.get("headers", []):
if name == b"cookie":
cookies = parse_cookie(value.decode("latin1"))
break
else:
# No cookie header found - add an empty default.
cookies = {}
# now gather user data from session
try:
req = HttpRequest()
req.GET = QueryDict(query_string=scope.get("query_string"))
setattr(req, 'COOKIES', cookies)
setattr(req, 'headers', scope.get("headers")),
session = UserSession(req)
scope['user_id'] = session.get_user_id()
except UserSessionError as e:
raise e
return await self.app(scope, receive, send)
websockets/routing.py
from django.urls import re_path
from . import consumers
websocket_urlpatterns = [
re_path(r'ws/userupdates/', consumers.UserUpdatesConsumer.as_asgi())
]
websockets/consumers.py::UserUpdatesConsumer
from channels.generic.websocket import JsonWebsocketConsumer
import json, redis
class UserUpdatesConsumer(JsonWebsocketConsumer):
def connect(self):
self.accept()
self.redis = redis.Redis(host='127.0.0.1', port=6379, db=0, decode_responses=True)
self.p = self.redis.pubsub()
if 'user_id' not in self.scope:
self.send_json({
'success': False,
'message': 'No user_id present'
})
self.close()
else:
self.send_json({
'success': True,
'user_id': self.scope['user_id'],
'message': 'Connected'
})
self.p.psubscribe(f"dip_alerts")
self.p.psubscribe(f"userupdates_{self.scope['user_id']}*")
for message in self.p.listen():
if message.get('type') == 'psubscribe' and message.get('data') in [1,2]:
continue
if message.get('channel') == "dip_alerts":
self.send_json({
"key": "dip_alerts",
"event": "dip_alert",
"data": json.loads(message.get('data'))
})
else:
self.send(message.get('data'))

why does my django site url reveal the directory path on the server?

I am deploying a django site for the first time to a cpanel server with phusion passenger on cloudlinux and i have discovered a weird phenomenon that can't be normal.
When I go to the site homepage it comes up as expected and when I click on any link the page loads fine but the url displays the full directory path to the app location on the server.
So, for example, rather than -
https://website.net/django_app
I get -
https://website.net/home/userdir/django_site/django_app
I don't even know where to begin to troubleshoot this as all the deployment settings were straightforward but I have clearly done something wrong somewhere.
Here is my passenger_wsgi.py file which I got from this site (https://www.a2hosting.co.uk/kb/developer-corner/python/installing-and-configuring-django-on-linux-shared-hosting) as the cpanel automatically generated passenger_wsgi did not work ;
1 import os
2 import sys
3
4 import django.core.handlers.wsgi
5 from django.core.wsgi import get_wsgi_application
6
7 # Set up paths and environment variables
8 sys.path.append(os.getcwd())
9 os.environ['DJANGO_SETTINGS_MODULE'] = 'django_site.deployment_settings'
10
11 # Set script name for the PATH_INFO fix below
12 SCRIPT_NAME = os.getcwd()
13
14 class PassengerPathInfoFix(object):
15 """
16 Sets PATH_INFO from REQUEST_URI because Passenger doesn't provide it.
17 """
18 def __init__(self, app):
19 self.app = app
20
21 def __call__(self, environ, start_response):
22 from urllib.parse import unquote
23 environ['SCRIPT_NAME'] = SCRIPT_NAME
24 request_uri = unquote(environ['REQUEST_URI'])
25 script_name = unquote(environ.get('SCRIPT_NAME', ''))
26 offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
27 environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
28 return self.app(environ, start_response)
29
30 # Set the application
31 application = get_wsgi_application()
32 application = PassengerPathInfoFix(application)
I hope somebody can help
Thanks.

Django Azure AD Integration

I'm currently integrating SSO using Azure AD for a Django Project. I'm currently using the package: https://github.com/leibowitz/django-azure-ad-auth . I have followed the docs to setup the Azure AD Authentication . On entering the application url, it takes me to the microsoft login page and after entering the credentials it's redirected to the application. But on redirection to the application after the Azure Auth, the code checks in the session for 'nonce' & 'state' variables , which are strangely returned as None and hence the application redirects to the failure url.
#never_cache
def auth(request):
backend = AzureActiveDirectoryBackend()
redirect_uri = request.build_absolute_uri(reverse(complete))
nonce = str(uuid.uuid4())
request.session['nonce'] = nonce
state = str(uuid.uuid4())
request.session['state'] = state
login_url = backend.login_url(
redirect_uri=redirect_uri,
nonce=nonce,
state=state
)
return HttpResponseRedirect(login_url)
#never_cache
#csrf_exempt
def complete(request):
backend = AzureActiveDirectoryBackend()
method = 'GET' if backend.RESPONSE_MODE == 'fragment' else 'POST'
original_state = request.session.get('state')
state = getattr(request, method).get('state')
if original_state == state:
token = getattr(request, method).get('id_token')
nonce = request.session.get('nonce')
user = backend.authenticate(token=token, nonce=nonce)
if user is not None:
login(request, user)
return HttpResponseRedirect(get_login_success_url(request))
return HttpResponseRedirect('failure')
This is the code used for authentication.
Settings.py sample is given below:
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend',
'azure_ad_auth.backends.AzureActiveDirectoryBackend',
)
LOGIN_REDIRECT_URL = '/login_successful/'
AAD_TENANT_ID = 'd472b4f4-95c5-4eb3-8a9a-3615c837eada'
AAD_CLIENT_ID = '75e38b53-8174-4dc6-a8f6-bb7a913f1565'
SESSION_EXPIRE_AT_BROWSER_CLOSE = True
SESSION_SAVE_EVERY_REQUEST = True
SESSION_COOKIE_AGE = 86400 # sec
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_NAME = 'DSESSIONID'
SESSION_COOKIE_SECURE = True
Traceback
TypeError at /TypeError at /project/azure/complete/
must be str, not NoneType
Request Method: POST
Request URL: http://testdomain.com/project/azure/complete/
Django Version: 2.2.4
Exception Type: TypeError
Exception Value:
must be str, not NoneType
Exception Location: /home/project/azure_auth/views.py in complete, line 57
Python Executable: /home/project/app/venv/bin/python3
Python Version: 3.6.8
Python Path:
['/home/project/app/project',
'/home/project/app/venv/bin',
'/home/project/app/venv/lib64/python36.zip',
'/home/project/app/venv/lib64/python3.6',
'/home/project/app/venv/lib64/python3.6/lib-dynload',
'/usr/lib64/python3.6',
'/usr/lib/python3.6',
'/home/project/app/venv/lib/python3.6/site-packages']
Server time: Tue, 19 Nov 2019 05:21:10 +0000/azure/complete/
must be str, not NoneType
Request Method: POST
Request URL: http://testdomain.com/project/azure/complete/
Django Version: 2.2.4
Exception Type: TypeError
Exception Value:
must be str, not NoneType
Exception Location: /home/project/app/project/azure_auth/views.py in complete, line 57
Python Executable: /home/project/app/venv/bin/python3
Python Version: 3.6.8
Python Path:
['/home/project/app/project',
'/home/project/app/venv/bin',
'/home/project/app/venv/lib64/python36.zip',
'/home/project/app/venv/lib64/python3.6',
'/home/project/app/venv/lib64/python3.6/lib-dynload',
'/usr/lib64/python3.6',
'/usr/lib/python3.6',
'/home/project/app/venv/lib/python3.6/site-packages']
Server time: Tue, 19 Nov 2019 05:21:10 +0000
/home/project/app/project/azure_auth/views.py in complete
f.write("nonce -->"+nonce+"\n") …
▼ Local vars
Variable Value
backend
<azure_auth.backends.AzureActiveDirectoryBackend object at 0x7f5c688dce80>
data
['82aff4f9-2cc0-4521-aea7-ad3281d20774\n',
'ba821364-86c9-4233-881f-bdc772f7c488\n']
f
<_io.TextIOWrapper name='t1.txt' mode='w' encoding='UTF-8'>
method
'POST'
n
'82aff4f9-2cc0-4521-aea7-ad3281d20774'
nonce
None
original_state
None
request
<WSGIRequest: POST '/project/azure/complete/'>
state
'fd93da6a-9009-4363-9640-9364df7f64df'
token
'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkJCOENlRlZxeWFHckdOdWVoSklpTDRkZmp6dyIsImtpZCI6IkJCOENlRlZxeWFHckdOdWVoSklpTDRkZmp6dyJ9.eyJhdWQiOiI0MDMyODJjZi1kYjlmLTQ1OTYtOWM1My0wMmI1MTA2ZDA0MDIiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9mYmM0OTNhOC0wZDI0LTQ0NTQtYTgxNS1mNGNhNThlOGMwOWQvIiwiaWF0IjoxNTc0MTQwNTY5LCJuYmYiOjE1NzQxNDA1NjksImV4cCI6MTU3NDE0NDQ2OSwiYWlvIjoiNDJWZ1lCQSt4TFhqNEdVTjRRWEJiU3ZOZmF4NXBHY2NPenFoNzFuNDlXMmxnYTYzMjIwQiIsImFtciI6WyJwd2QiXSwiZmFtaWx5X25hbWUiOiJFIEsgUyIsImdpdmVuX25hbWUiOiJTdXNyZWV0aGEiLCJpbl9jb3JwIjoidHJ1ZSIsImlwYWRkciI6IjE4Mi43NS4xNjcuMTg4IiwibmFtZSI6IkUgSyBTLFN1c3JlZXRoYSIsIm5vbmNlIjoiODgyNTg4ZjgtMGM3MC00Y2JlLTk4MTktY2JkNjUyZmI0MDQ5Iiwib2lkIjoiNTU0ZjYzZWEtOTg4Yi00MmMwLTk4NjUtMTIxMDNkZTdhZTBmIiwib25wcmVtX3NpZCI6IlMtMS01LTIxLTYwMzE5MzI1LTExNjA5ODI5NTEtMTYwMTc3MzkwNy02NTg1OTMiLCJzdWIiOiI2aERUa1hYYkN3Wm5rcHEwSU9wRTRVWk56RHZlRFhvVjM3RGV5U3dkaDZRIiwidGlkIjoiZmJjNDkzYTgtMGQyNC00NDU0LWE4MTUtZjRjYTU4ZThjMDlkIiwidW5pcXVlX25hbWUiOiJTRTA3NTA0OEBjZXJuZXIubmV0IiwidXBuIjoiU0UwNzUwNDhAY2VybmVyLm5ldCIsInV0aSI6IkZTQmhnVDg4UTAyUHNfU293ZDdtQUEiLCJ2ZXIiOiIxLjAifQ.Rvc6xcPRZ01iebYtEyAWeyDnQEUVtqV1L1mapr658jLog-_yIASyEm3kMrkt6dIWWEO3dJSe3k05xOJlbnHqcjaR5LKAwOZzGR_oBmyIyB8-IvuEankNVpwYtcz8mY7kFr6AqQmIsx7xLLgv4grp-bSy4eRqjk36VeLX_LwMBuM_U6V70w0gXN1vvFCj0tjsv-VtTAmNgvdxS0ltzdD3rzZ87DoXbPWmoozLtO9WBRsJvMuvn-frBtYUYkIhs3I-eVAO9ZG2IWEuLQx6k7RBmzX6HgFi9SVpyEhNru7fmwO-qj5uRj9FQa45lCZluUV25o_AV1NQ94d5lnFyeMh7uw'
user
None
I got the above error while trying to write the session variables to file (for debugging.)
I know this question is a bit old, but the session won't be able to be retrieved (and with it the original state and nonce), and will fail the comparison if the cookie is not being sent by the browser.
The cookie is not sent by default in django 2.1+, since the default settings add SameSite=Lax
The cookies used for django.contrib.sessions, django.contrib.messages,
and Django’s CSRF protection now set the SameSite flag to Lax by
default. Browsers that respect this flag won’t send these cookies on
cross-origin requests. If you rely on the old behavior, set the
SESSION_COOKIE_SAMESITE and/or CSRF_COOKIE_SAMESITE setting to None.
https://docs.djangoproject.com/en/3.0/releases/2.1/#samesite-cookies
In theory this should still send the cookie (from what I understand), but for some reason chrome doesn't seem to. There's something I clearly do not understand, so if anyone knows better please comment.
Anyway, changing the setting via SESSION_COOKIE_SAMESITE = None should work.

Why is my hosted Django app showing my username in the URL?

I've written a Django app which I'm trying to get set up on shared web hosting (A2). It's working, except that when I go to:
http://example.com/terms/
the URL changes in the browser bar to:
http://example.com/home/myusername/myappfolder/myappname/terms/
showing the full path to where my app is on disk.
This doesn't happen with static files - e.g. http://example.com.com/static/image.png works normally.
The app is running in a virtual environment. I'm using python 3.6.8 and Django 2.1.4.
I followed these instructions to set up my app, which include setting up this passenger.wsgi file, that looks like this:
import myapp.wsgi
SCRIPT_NAME = '/home/username/myapp'
class PassengerPathInfoFix(object):
"""
Sets PATH_INFO from REQUEST_URI because Passenger doesn't provide it.
"""
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
from urllib.parse import unquote
environ['SCRIPT_NAME'] = SCRIPT_NAME
request_uri = unquote(environ['REQUEST_URI'])
script_name = unquote(environ.get('SCRIPT_NAME', ''))
offset = request_uri.startswith(script_name) and len(environ['SCRIPT_NAME']) or 0
environ['PATH_INFO'] = request_uri[offset:].split('?', 1)[0]
return self.app(environ, start_response)
application = myapp.wsgi.application
application = PassengerPathInfoFix(application)
I'd be grateful for any pointers as to where to look to solve this.
Got it working!
In my modified passenger_wsgi.py, I changed the line
SCRIPT_NAME = os.getcwd()
to
SCRIPT_NAME = ''
One thing I should point out is that the absolute path was getting inserted on redirects - so if I visited
http://example.com/terms
it would redirect to
http://example.com/terms/
and insert the path in the URL.
As you're debugging I recommend disabling the cache, as that threw me for several loops when changes I made didn't seem to take effect.
Thanks to this question for getting me on the right track.

Django POST/GET, GET works, POST doesn't from MATLAB (urlread). Both work using django.test.client

I am learning Django so I've set up a very simple form/view/url example
Django Version 1.5.1
MATLAB Version R2012A
forms.py
from django import forms
import json
class json_input(forms.Form):
jsonfield = forms.CharField(max_length=1024)
def clean_jsonfield(self):
jdata = self.cleaned_data['jsonfield']
try:
json_data = json.loads(jdata)
except:
raise forms.ValidationError("Invalid data in jsonfield")
return jdata
views.py
from django.http import HttpResponse
from rds.forms import json_input
def testpost(request):
if request.method == 'GET':
form = json_input(request.GET)
if form.is_valid():
return HttpResponse('Were Good Get',mimetype='text/plain')
elif request.method == 'POST':
form = json_input(request.POST)
if form.is_valid():
return HttpResponse('Were Good Post',mimetype='text/plain')
else:
return HttpResponse('Not GET or POST.',mimetype='text/plain')
This view is mapped to the url in urls.py
url(r'^test2$','rds.views.testpost'),
So when I jump into the python manage.py shell on the local machine django is on I can issue the following commands and get the expected responses:
>>> from django.test.client import Client
>>> c = Client()
>>> r = c.post('/test2',{'jsonfield': '{"value":100}'})
>>> print r
Content-Type: text/plain
Were Good Post
>>> r = c.get('/test2',{'jsonfield': '{"value":100}'})
>>> print r
Content-Type: text/plain
Were Good Get
However when I jump into MATLAB on an external machine and issue the following commands. (Note doing this from MATLAB is a project requirement)
json = '{"value":100}';
% GET METHOD FOR JSON FORM
[gresponse,gstatus]=urlread('http://aq-318ni07.home.ku.edu/django/test2','Get',{'jsonfield' json});
% POST METHOD FOR JSON FORM
[presponse,pstatus]=urlread('http://aq-318ni07.home.ku.edu/django/test2','Post',{'jsonfield' json});
>> gresponse
gresponse =
Were Good Get
>> presponse
presponse =
''
I have searched around for a solution and really cant find anything. I've hit on it potentially being an issue with the CSRF (which I am still figuring out). Any hints or thoughts would be much appreciated.
Thank you.
EDIT:
Django is exposed through Apache, here is the configuration.
################################################
# Django WSGI Config
################################################
WSGIScriptAlias /django /var/django/cdp/cdp/wsgi.py
WSGIPythonPath /var/django/cdp
<Directory /var/django/cdp/cdp>
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>
################################################
how are you exposing your django app for MATLAB? The very first thing is to check your access logs, is your server even getting a request? If so anything in its error logs?
Are you using the built in developkment server?
python manage.py runserver 0.0.0.0:8000
If so make sure that you can accept requests on that port
If you are serving it through another server i believe you have to whitelist the IP that you are making request from MATLAB, by adding it to ALLOWED_HOSTS