django and nginx - urls are all the same - django

Right now, any url just brings up a project default page ("welcome to django").
No matter what I put (example.com, example.com/hello, example.com/asdfjkasdf(&$(#$$#)
I'm new to django and am following a simple tutorial.
My nginx.conf has this:
location / {
# host and port to fastcgi server
fastcgi_pass 127.0.0.1:8801;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
}
My site files are stored in /var/www/firstsite/
My views.py has this:
from django.http import HttpResponse
def hello(request):
return HttpResponse("Hello world")
And my urls.py has this:
from django.conf.urls.defaults import *
from firstsite.views import hello
# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()
urlpatterns = patterns('',
('^hello/$', hello),
# Example:
# (r'^firstsite/', include('firstsite.foo.urls')),
# Uncomment the admin/doc line below and add 'django.contrib.admindocs'
# to INSTALLED_APPS to enable admin documentation:
# (r'^admin/doc/', include('django.contrib.admindocs.urls')),
# Uncomment the next line to enable the admin:
# (r'^admin/', include(admin.site.urls)),
)
Do I need to restart the fcgi instance with each change(I wouldn't think so). I've been using: python manage.py runfcgi method="thread" host=127.0.0.1 port=8080
So yeah, how can I get urls working? Is there a way I can debug using django? For example, maybe print out the data it's receiving to make sure nginx is behaving correctly?

Don't start by trying to set Django up with FastCGI. Follow the actual tutorial, and use the built-in development server. Once you've got a grip on how the basic framework works, then you can move to understanding how to deploy it.
And why would you say you wouldn't think you would have to restart the instance with each change? That is precisely what you need to do.

Related

How to change the Django Rest Framework's default url to a custom

Question says it almost all.
E.g. changing default url (http://127.0.0.1:8000) to a custom (https://api.example.com/v1)
I'm using HyperlinkedModels and everything seems to work properly in development. Moving the app to another server with custom url is giving me problems.
How do I change the default url:
To a custom one, let's say:
https://api.example.org/v1/
You are mixing two questions in one:
How to run django-rest-framework project on a different domain
How to change URL path of API
To answer the first one I'd say, "Just do it". Django's reverse uses request's domain to build absolute URL.
UPDATE: don't forget to pass Host header from nginx/apache. Below is a sample nginx config:
server {
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8000;
}
}
The second (path, mount point) is set in the urls.py:
from django.conf.urls import url, include
from django.contrib import admin
from rest_framework import routers
from quickstart import views
router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework')),
url(r'^v1/', include(router.urls)), # <-------------- HERE
]

Django URLs repeat when using runfcgi

I have the following urlpatterns:
urlpatterns = patterns('',
url(r'^$', 'opeiaa.views.home', name='home'),
url(r'^store/$', 'opeiaa.views.store', name='store'),
url(r'^resume/$', 'opeiaa.views.resume', name='resume'),
url(r'^contact/$', 'opeiaa.views.contact', name='contact'),
url(r'^gallery/', include('gallery.urls')),
url(r'^admin/', include(admin.site.urls)),
)
... and am using this kind of template tag:
<a class='nav-link' href='{% url 'contact' %}'>Contact</a>
The URL gets rendered in the page as http://localhost:8000/contact/. Everything works fine, when using ./manage.py runserver for testing...
... but then I run ./manage.py runfcgi - then when I navigate to the contact page, the URL in the navigation points to http://localhost:8000/contact/contact/! I have tried putting a slash at the start to make the URL absolute, but the URLs appear to be absolute without it.
I am using nginx as a frontend, and the relevant config from there is:
location / {
include fastcgi_params;
fastcgi_pass unix:/tmp/django.sock;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
}
I am using Django 1.6 & Python 2.7.4. Anyone have any insight?
8 months later I had figured this out when it happened on another one of my sites, because SCRIPT_NAME was being set before include fastcgi_params!
The final working config snippet:
location / {
include fastcgi_params;
fastcgi_param SCRIPT_NAME "";
fastcgi_pass unix:/tmp/django.sock;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
}

Celery Flower Security in Production

I am looking to use Flower (https://github.com/mher/flower) to monitor my Celery tasks in place of the django-admin as reccomended in their docs (http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monitor). However, because I am new to this I am a little confused about the way Flower's page is only based on HTTP, and not HTTPS. How can I enable security for my Celery tasks such that any old user can't just visit the no-login-needed website http://flowerserver.com:5555 and change something?
I have considered Celery's own documentation on this, but they unfortunately there is no mention of how to secure Flower's api or web ui. All it says: [Need more text here]
Thanks!
Update: My question is in part a duplicate of here: How do I add authentication and endpoint to Django Celery Flower Monitoring?
However, I clarify his question here by asking how to run it using an environment that includes nginx, gunicorn, and celery all on the same remote machine. I too am wondering about how to set up Flower's outside accessible url, but also would prefer something like https instead of http if possible (or some way of securing the webui and accessing it remotely). I also need to know if leaving Flower running is a considerable security risk for anyone who may gain access to Flower's internal API and what the best way for securing this could be, or if it should just be disabled altogether and used just on an as-needed basis.
You can run flower with --auth flag, which will authenticate using a particular google email:
celery flower --auth=your.email#gmail.com
Edit 1:
New version of Flower requires couple more flags and a registered OAuth2 Client with Google Developer Console:
celery flower \
--auth=your.email#gmail.com \
--oauth2_key="client_id" \
--oauth2_secret="client_secret" \
--oauth2_redirect_uri="http://example.com:5555/login"
oauth2_redirect_uri has to be the actual flower login url, and it also has to be added to authorized redirect url's in Google Development Console.
Unfortunately this feature doesn't work properly in current stable version 0.7.2, but it is now fixed in development version 0.8.0-dev with this commit.
Edit 2:
You can configure Flower using basic authentication:
celery flower --basic_auth=user1:password1,user2:password2
Then block 5555 port for all but localhost and configure reverse proxy for nginx or for apache:
ProxyRequests off
ProxyPreserveHost On
ProxyPass / http://localhost:5555
Then make sure proxy mod is on:
sudo a2enmod proxy
sudo a2enmod proxy_http
In case you can't set it up on a separate subdomain, ex: flower.example.com (config above), you can set it up for example.com/flower:
run flower with url_prefix:
celery flower --url_prefix=flower --basic_auth=user1:password1,user2:password2
in apache config:
ProxyPass /flower http://localhost:5555
Of course, make sure SSL is configured, otherwise there is no point :)
I have figured out it using proxy on Django side https://pypi.org/project/django-revproxy/. So Flower is hidden behind Django auth which is more flexible than basic auth. And you don't need rewrite rule in NGINX.
Flower 0.9.5 and higher
URL prefix must be moved into proxy path: https://github.com/mher/flower/pull/766
urls.py
urlpatterns = [
FlowerProxyView.as_url(),
...
]
views.py
class FlowerProxyView(UserPassesTestMixin, ProxyView):
# `flower` is Docker container, you can use `localhost` instead
upstream = 'http://{}:{}'.format('flower', 5555)
url_prefix = 'flower'
rewrite = (
(r'^/{}$'.format(url_prefix), r'/{}/'.format(url_prefix)),
)
def test_func(self):
return self.request.user.is_superuser
#classmethod
def as_url(cls):
return re_path(r'^(?P<path>{}.*)$'.format(cls.url_prefix), cls.as_view())
Flower 0.9.4 and lower
urls.py
urlpatterns = [
re_path(r'^flower/?(?P<path>.*)$', FlowerProxyView.as_view()),
...
]
views.py
from django.contrib.auth.mixins import UserPassesTestMixin
from revproxy.views import ProxyView
class FlowerProxyView(UserPassesTestMixin, ProxyView):
# `flower` is Docker container, you can use `localhost` instead
upstream = 'http://flower:5555'
def test_func(self):
return self.request.user.is_superuser
I wanted flower on a subdirectory of my webserver, so my nginx reverse proxy configuration looked like this:
location /flower/ {
proxy_pass http://localhost:5555/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd;
}
Now I can get to flower (password-protected) via www.example.com/flower
Most of this is derived from the Flower documentation page about configuring an nginx reverse proxy:
http://flower.readthedocs.org/en/latest/reverse-proxy.html
I followed #petr-přikryl's approach using a proxy view. However I couldn't get it to verify authentication (I don't think test_func is ever called). Instead I chose to embed this in the Django Admin views and use AdminSite.admin_view() (as described here) to wrap the view with Django Admin authentication.
Specifically, I made the following changes:
# Pipfile
[packages]
...
django-revproxy="*"
# admin.py
class MyAdminSite(admin.AdminSite):
# ...
def get_urls(self):
from django.urls import re_path
# Because this is hosted in the root `urls.py` under `/admin` this
# makes the total prefix /admin/flower
urls = super().get_urls()
urls += [
re_path(
r"^(?P<path>flower.*)$",
self.admin_view(FlowerProxyView.as_view()),
)
]
return urls
# views.py
from __future__ import annotations
from django.urls import re_path
from revproxy.views import ProxyView
class FlowerProxyView(ProxyView):
# Need `/admin/` here because the embedded view in the admin app drops the
# `/admin` prefix before sending the URL to the ProxyView
upstream = "http://{}:{}/admin/".format("localhost", 5555)
Lastly, we need to make sure that --url_prefix is set when running flower, so I set it to run like this in our production and dev environments:
celery flower --app=my_app.celery:app --url_prefix=admin/flower
To offload the django app, I suggest you use the X-Accel-Redirect header in order to use nginx to proxy the Flower server. It goes as follow:
the user requests the flower path (e.g. /task)
nginx proxy_pass the request to your app, as usual
your django app chooses to accept or reject the request (e.g. based on authentification)
if your app accepts the request, it returns a response with X-Accel-Redirect HTTP-header together with a string of an internal location, i.e. a path that cannot be accessed directly by the user
nginx intercepts the response instead of forwarding it to the user and uses it as a new path with the possibility this time to access internal locations, in our case the Flower server
If the request is rejected, simply do not use X-Accel-Redirect and handle the case as any other rejected request you'd implement.
nginx.conf:
upstream celery_server {
server /var/run/celery/flower.sock;
}
upstream app_server {
server /var/run/gunicorn/asgi.sock;
}
server {
listen 80;
location /protected/task {
internal; # returns 404 if accessed directly
proxy_http_version 1.1;
proxy_redirect off;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_pass http://celery_server/task;
}
location / {
proxy_http_version 1.1;
proxy_redirect off;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
proxy_pass http://app_server;
}
}
views.py:
from django.contrib.admin.views.decorators import staff_member_required
from django.views.decorators.csrf import csrf_exempt
from django.http import HttpResponse
class XAccelRedirectResponse(HttpResponse):
def __init__(self, path, *args, **kwargs):
super().__init__(*args, **kwargs)
self['X-Accel-Redirect'] = '/protected' + path
del self['Content-Type'] # necessary
# I chose to only allow staff members, i.e. whose who can access the admin panel
#staff_member_required
#csrf_exempt
def task_app(request, path):
query_str = request.META['QUERY_STRING'] # you must keep the query string
return XAccelRedirectResponse(f'/task/{path}?{query_str}')
urls.py:
from django.urls import re_path
from app import views
urlpatterns = [
re_path('task/(?P<path>.*)', views.task_app, name='task'),
]
Flower
It is important to change the url-prefix of Flower:
celery flower --unix-socket="/var/run/celery/flower.sock" --url-prefix="task"
Yep there's not auth on flower, since it's just talking to the broker, but if you run it over SSL then basic auth should be good enough.
How would HTTP and HTTPS affect Celery security? What user logins are you referring to?
Flower monitors to a Celery queue by attaching to the workers. When setting up Flower you need to provide connection string [broker]://[user_name]:[password]#[database_address]:[port]/[instance]. User name and password are the credential to log into the database of your choice.
If you're referring to this login, wouldn't simply disable/remove their logins be suffice?
This is a reply to Petr Přikryl's post. django-revproxy fails to work on my Django 4.1.x project. I am encountering error AttributeError: 'HttpResponse' object has no attribute '_headers'. Many others are facing the same issue. brianmay in the issue thread claims, "I think this project is basically dead, sorry."
I went with a different library to serve as a workaround.
Install django-proxy
This is what my code looks like.
# urls.py
from django.urls import re_path
from myapp.views import flower
urlpatterns = [
re_path("flower/(?P<path>.*)", flower),
]
# views.py
from django.views.decorators.csrf import csrf_exempt
from proxy.views import proxy_view
#csrf_exempt
def flower(request, path):
extra_requests_args = {}
remoteurl = f"http://localhost:5555/flower/" + path
return proxy_view(request, remoteurl, extra_requests_args)
Then run celery with
$ celery --app myproject flower --loglevel INFO --url_prefix=flower
You can then view it in your browser, served through Django, at http://localhost:8000/flower/.
Additional notes:
--url_prefix= is important because this will allow the proxy to serve the static files that flower requests.
If you are using docker compose, then you will likely need to change the hostname in the remoteurl string in the flower function to reflect the same of the service. For example, my service is appropriately called flower in my docker-compose.yaml file. Therefore, I would change the string from f"http://localhost:5555/flower/" to f"http://flower:5555/flower/"

Django admin panel throws 404 when passing any url parameters

Django admin panel throws 404 when passing any url parameters
Example url:
/admin/app/model/ - OK
/admin/app/model/?foo=bar - 404
/admin/app/model/?p=1 - 404
Nginx+uwsgi
Project urls file (admin,admin_tools,application urls)
from django.conf.urls import patterns, include, url
from django.views.generic import TemplateView
import restaurant.views
import club.views
import hotel.views
import custom_app.views
import cart.views
from django.views.decorators.csrf import csrf_exempt
from django.contrib import admin
import settings
handler404 = 'custom_app.views.handler404'
admin.autodiscover()
urlpatterns = patterns('',
url(r'^admin_tools/', include('admin_tools.urls')),
(r'^$', custom_app.views.Index.as_view()),
(r'^restaurant$', restaurant.views.Index.as_view()),
(r'^which$', csrf_exempt(TemplateView.as_view(template_name='custom_app/which.html'))),
(r'^cart/add', cart.views.AddToCart.as_view()),
(r'^cart/delete', cart.views.RemoveFromCart.as_view()),
(r'^cart/send', cart.views.SendCart.as_view()),
(r'^cart/delivery', cart.views.SetDelivery.as_view()),
(r'^restaurant/about', restaurant.views.About.as_view()),
(r'^restaurant/payment', restaurant.views.Payment.as_view()),
(r'^restaurant/vip', restaurant.views.Vip.as_view()),
(r'^restaurant/menu/(?P<slug>.+)', restaurant.views.Menu.as_view()),
(r'^restaurant/menu/', restaurant.views.Menu.as_view()),
(r'^restaurant/cart/', cart.views.Basket.as_view()),
(r'^restaurant/tables/', restaurant.views.Tables.as_view()),
(r'^restaurant/success_ordering', restaurant.views.SuccesTableOrder.as_view()),
(r'^vacancy', custom_app.views.Vacancy.as_view()),
(r'^hotel$', hotel.views.Main.as_view()),
(r'^hotel/services', hotel.views.Services.as_view()),
(r'^hotel/room/(?P<room>\d+)$', hotel.views.Main.as_view()),
(r'^hotel/order/option/toggle$', hotel.views.ToggleOption.as_view()),
(r'^hotel/order/date/toggle$', hotel.views.ToggleDate.as_view()),
(r'^hotel/order/send$', hotel.views.Send.as_view()),
(r'^club/events/old/(?P<year>\d+)/(?P<month>\d+)', club.views.OldEvents.as_view()),
(r'^club/events/old/', club.views.OldEvents.as_view()),
(r'^club/about', club.views.About.as_view()),
(r'^club/event/(?P<pk>\d+)', club.views.DetailEvent.as_view()),
(r'^club$', club.views.Main.as_view()),
url(r'^admin/', include(admin.site.urls)),
(r'^ckeditor/', include('ckeditor.urls')),
)
if settings.DEBUG:
urlpatterns += patterns('',
url(r'^media/(?P<path>.*)$', 'django.views.static.serve', {
'document_root': settings.MEDIA_ROOT,
}),
)
Nginx configuration file:
server
{
listen %server_ip%;
server_name custom.ru www.custom.ru;
root /home/custom/www/;
location ~* ^/resize/([\d\-]+)/([\d\-]+)/(.+)$ {
alias /home/custom/www/custom/$3;
image_filter resize $1 $2;
image_filter_buffer 2M;
error_page 415 = /empty;
}
location ~* ^/crop/([\d\-]+)/([\d\-]+)/(.+)$ {
alias /home/custom/www/custom/$3;
image_filter crop $1 $2;
image_filter_buffer 2M;
error_page 415 = /empty;
}
location = /empty {
empty_gif;
}
location /
{
root /home/custom/www/custom/;
uwsgi_pass unix:///home/custom/tmp/uwsgi.sock;
include uwsgi_params;
uwsgi_param UWSGI_PASS unix:///home/custom/tmp/uwsgi.sock;
uwsgi_param UWSGI_CHDIR /home/custom/www/custom/;
uwsgi_param UWSGI_SCRIPT wsgi;
}
location /static
{
alias /home/custom/www/custom/static/;
}
location /media
{
alias /home/custom/www/custom/media/;
}
location = /favicon.ico { alias /home/custom/www/favicon.ico; }
}
Try
/admin/app/action?foo=bar
instead of /?
The extra / is looking for a resource at that location

The current URL, , didn't match any of these - using multiple sites

I wish to have multiple django installations. One at / (which is working fine) and one at /adam. The one at slash adam is redirected by apache correctly, until you try and visit an app. E.g. /admin works but /adam/admin does not work. I get the error:
Page not found (404)
Request Method: GET
Request URL: http://[CENSORED]/adam/
Using the URLconf defined in bms.urls, Django tried these URL patterns, in this order:
^admin/doc/
^admin/
The current URL, , didn't match any of these.
Notice the empty commas. The apache virtual host is:
<VirtualHost *:80>
ServerName [CENSORED]
DocumentRoot /home/user/bms
Alias /static/admin/ /usr/local/lib/python2.7/site-packages/Django-1.3-py2.7.egg/django/contrib/admin/media/
<Directory /home/user/bms/apache>
Order allow,deny
Allow from all
</Directory>
<Directory /home/ajt1g09/bms/apache>
Order allow,deny
Allow from all
</Directory>
WSGIDaemonProcess bms user=user group=user processes=2 threads=25 python-path=/usr/local/lib/python2.7/site-packages
WSGIProcessGroup bms
WSGIScriptAliasMatch ^/adam(.*) /home/ajt1g09/bms/apache/django.wsgi
WSGIScriptAlias / /home/user/bms/apache/django.wsgi
</VirtualHost>
And the django.wsgi file in ajt1g09/bms/apache:
import os
import sys
path = '/home/ajt1g09/bms'
if path not in sys.path:
sys.path.append(path)
sys.path.append('/usr/local/lib/python2.7/site-packages')
sys.path.append('/home/ajt1g09')
os.environ['DJANGO_SETTINGS_MODULE'] = 'bms.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
And finally, the urls.py file in ajt1g09/bms (clearly showing /admin is there):
from django.conf.urls.defaults import
patterns, include, url
#Uncomment the next two lines to enable the admin: from django.contrib
import admin
admin.autodiscover()
urlpatterns = patterns('',
# Examples:
# url(r'^$', 'bms.views.home', name='home'),
# url(r'^bms/', include('bms.foo.urls')),
# Uncomment the admin/doc line below to enable admin documentation:
url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
# Uncomment the next line to enable the admin:
url(r'^admin/', include(admin.site.urls)), )
I have no idea what the problem is.
You shouldn't be using:
WSGIScriptAliasMatch ^/adam(.*) /home/ajt1g09/bms/apache/django.wsgi
Just use:
WSGIScriptAlias /adam /home/ajt1g09/bms/apache/django.wsgi
The WSGIScriptAliasMatch will not work as written because you haven't re substituted the matched part from back into last argument. Ie.,
WSGIScriptAliasMatch ^/adam(.*) /home/ajt1g09/bms/apache/django.wsgi$1
You should though simply not be using WSGIScriptAliasMatch. That is for advanced use cases only and requires you be very careful in using it because how you use it can impact what SCRIPT_NAME/PATH_INFO are set to when passed to application and it is those that urls.py matching is based off.