Cannot overwrite Django authentication templates in AWS - django

I have developped a Django application and I have been using the django-authtools module to be able to login using email. It is working great on my laptop but when I tried to deploy it in production in AWS using Beanstalk, it seems Django does not recognize the overwrite of the authentication module and is forcing redirection to the django built-in authentication module. Everything else seems to work fine (from a deployment and application point of view).
It results in a 500 error:
xxx.xxx.xxx.xxx (-) - - [04/Jul/2017:19:07:54 +1000] "GET /accounts/login/ HTTP/1.1" 500 7807 "http://<removed>.ap-southeast-2.elasticbeanstalk.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
Internal Server Error: /accounts/login/
Traceback (most recent call last):
File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 217, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/python/run/venv/lib/python3.4/site-packages/django/core/handlers/base.py", line 215, in _get_response
response = response.render()
File "/opt/python/run/venv/lib/python3.4/site-packages/django/template/response.py", line 107, in render
self.content = self.rendered_content
File "/opt/python/run/venv/lib/python3.4/site-packages/django/template/response.py", line 82, in rendered_content
template = self.resolve_template(self.template_name)
File "/opt/python/run/venv/lib/python3.4/site-packages/django/template/response.py", line 64, in resolve_template
return select_template(template, using=self.using)
File "/opt/python/run/venv/lib/python3.4/site-packages/django/template/loader.py", line 53, in select_template
raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)
django.template.exceptions.TemplateDoesNotExist: registration/login.html
Again this is working perfectly fine on my laptop, but not on the AWS server. I cannot find any difference between the 2 and they are running same version of django and django-authools:
$ pip freeze
[...]
Django==1.11.2
django-authtools==1.5.0
django-extensions==1.7.6
django-phonenumber-field==1.3.0
django-qsstats-magic==0.7.2
django-simple-captcha==0.5.5
django-storages==1.6.1
[...]
The only difference seems to be the version of python, 3.4.3 on AWS and 3.5.2 on my laptop.
On the server, my settings.py file is the same as on my laptop:
myproject/settings.py:
INSTALLED_APPS = [
'myapp.apps.MyappConfig',
'authtools',
'captcha',
'storages',
'phonenumber_field',
'django_extensions',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
AUTH_USER_MODEL = "authtools.User"
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': ['templates'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
myapp/urls.py:
from authtools import urls
[...]
url(r'^accounts/', include('authtools.urls')),
[...]
authtools/url.py:
from django.conf.urls import url
from authtools import views as authools_views
urlpatterns = [
url(r'^login/$', authools_views.LoginView.as_view(), name='login'),
url(r'^logout/$', authools_views.LogoutView.as_view(), {'next_page': '/accounts/login'}, name='logout'),
url(r'^password_change/$', authools_views.PasswordChangeView.as_view(), name='password_change'),
url(r'^password_change/done/$', authools_views.PasswordChangeDoneView.as_view(), name='password_change_done'),
url(r'^password_reset/$', authools_views.PasswordResetView.as_view(), name='password_reset'),
url(r'^password_reset/done/$', authools_views.PasswordResetDoneView.as_view(), name='password_reset_done'),
url(r'^reset/done/$', authools_views.PasswordResetCompleteView.as_view(), name='password_reset_complete'),
url(r'^reset/(?P<uidb36>[0-9A-Za-z]{1,13})-(?P<token>[0-9A-Za-z]{1,13}-[0-9A-Za-z]{1,20})/$',
authools_views.PasswordResetConfirmView.as_view()),
url(r'^reset/(?P<uidb64>[0-9A-Za-z_\-]+)/(?P<token>[0-9A-Za-z]{1,13}-[0-9A-Za-z]{1,20})/$',
authools_views.PasswordResetConfirmView.as_view(),
name='password_reset_confirm'),
]
I have also tried to pass the template in the url statement directly (LoginView.as_view(template_name='myapp/login.html')) but same result.
Extract of authtools/view.py:
class LoginView(AuthDecoratorsMixin, WithCurrentSiteMixin, WithNextUrlMixin, FormView):
print('ENTERING LOGIN VIEW')
form_class = AuthenticationForm
template_name = 'myapp/login.html'
allow_authenticated = True
success_url = resolve_url_lazy(settings.LOGIN_REDIRECT_URL)
# BBB: This is deprecated (See LoginView.get_allow_authenticated)
disallow_authenticated = None
The template showed in the error "registration/login.html" is the built-in template and not the one provided by the authtools module which should overwrite it.
Also when navigating to the /admin/ of my website, I can login correctly with my super user and when going back to the site I am then detected as being authenticated. The logout action though redirects me to the default logout page and is not using my custom template provided in the authtools module (so same behavior).
If any one would have a solution or any idea of where to investigate that would be greatly appreciated!
Thanks!
I am adding my beanstalk config file just in case it helps, even though I don't believe this is a problem related to beanstalk itself.
.ebextensions# cat 001_set_env.config
option_settings:
"aws:elasticbeanstalk:application:environment":
PYTHONPATH: /opt/python/current/app/myapp:/opt/python/current/app/authtools:$PYTHONPATH
"aws:elasticbeanstalk:container:python":
WSGIPath: "myproject/wsgi.py"
.ebextensions# cat 002_deploy.config
commands:
01_update_pip:
command: "/opt/python/run/venv/bin/pip install --upgrade pip"
02_set_time_zone:
command: ln -f -s /usr/share/zoneinfo/Australia/Sydney /etc/localtime
container_commands:
01_makemigration:
command: "source /opt/python/run/venv/bin/activate && python manage.py makemigrations --noinput"
leader_only: true
02_migrate:
command: "source /opt/python/run/venv/bin/activate && python manage.py migrate --noinput"
leader_only: true
03_initialize_db:
command: "source /opt/python/run/venv/bin/activate && python manage.py initializedb"
leader_only: true
04_create_superuser:
command: "source /opt/python/run/venv/bin/activate && python manage.py createsu"
leader_only: true

I suggest you stop editing the code in authtools. It is not clear how you are doing it, and since the error is showing the missing template is registration/login.html, it is now working in production.
If you rename your template from myapp/templates/myapp/login.html to myapp/templates/registration/login.html, then the app directories loader should find your template.
Finally, as an aside, your DIRS setting looks incorrect.
'DIRS': ['templates'],
If you want to include a myproject/templates directory, then change it to
'DIRS': [os.path.join(BASE_DIR, 'templates')]

For those interested I have finally found the issue to this problem. I had the authools module installed both in site_packages and locally in my project. On my local laptop, it seems the order of precedence was looking at my local django project first (where I had the module with the right template paths), whereas on AWS, the python path order listed site_packages first, thus looking into an unmodified version of authtools.
I have therefore removed the authtools version installed in my project and I am now using the original version installed in site_packages, while over-writting the authools template names in my app urls.py directly.
Thank you all for helping and putting me on the right track.

Related

Cannot push to heroku with cloudinary storing my staticfiles

I am trying to use cloudinary to store my media and static files for my django project. I am using django-cloudinary-storage package from here. First I changed my settings so that media files are stored on cloudinary and pushed it to heroku and it worked fine except the staticfiles were not loading. So I changed my settings again so that static files are also stored in cloudinary. It works fine locally with DEBUG=False. When I see the page source of my html locally, it is getting the css and javascript files from cloudinary. So, then I tried pushing it to heroku using git push heroku master. But I get the below error in my heroku logs:
Step 11/14 : RUN python manage.py collectstatic --noinput
---> Running in 5b062a8f7c52
Traceback (most recent call last):
File "/usr/src/app/manage.py", line 22, in <module>
main()
File "/usr/src/app/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 425, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 373, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 417, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 161, in handle
if self.is_local_storage() and self.storage.location:
File "/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 215, in is_local_storage
return isinstance(self.storage, FileSystemStorage)
File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 248, in inner
self._setup()
File "/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/storage.py", line 465, in _setup
self._wrapped = get_storage_class(settings.STATICFILES_STORAGE)()
File "/usr/local/lib/python3.10/site-packages/cloudinary_storage/storage.py", line 275, in __init__
super(HashCloudinaryMixin, self).__init__(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/storage.py", line 400, in __init__
self.hashed_files = self.load_manifest()
File "/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/storage.py", line 410, in load_manifest
content = self.read_manifest()
File "/usr/local/lib/python3.10/site-packages/cloudinary_storage/storage.py", line 319, in read_manifest
with self.manifest_storage.open(self.manifest_name) as manifest:
File "/usr/local/lib/python3.10/site-packages/django/core/files/storage.py", line 38, in open
return self._open(name, mode)
File "/usr/local/lib/python3.10/site-packages/cloudinary_storage/storage.py", line 47, in _open
url = self._get_url(name)
File "/usr/local/lib/python3.10/site-packages/cloudinary_storage/storage.py", line 78, in _get_url
return cloudinary_resource.url
File "/usr/local/lib/python3.10/site-packages/cloudinary/__init__.py", line 305, in url
return self.build_url(**self.url_options)
File "/usr/local/lib/python3.10/site-packages/cloudinary/__init__.py", line 315, in build_url
return self.__build_url(**options)[0]
File "/usr/local/lib/python3.10/site-packages/cloudinary/__init__.py", line 312, in __build_url
return utils.cloudinary_url(public_id, **combined_options)
File "/usr/local/lib/python3.10/site-packages/cloudinary/utils.py", line 736, in cloudinary_url
raise ValueError("Must supply cloud_name in tag or in configuration")
ValueError: Must supply cloud_name in tag or in configuration
The command '/bin/sh -c python manage.py collectstatic --noinput' returned a non-zero code: 1
I already have CLOUD_NAME, API_KEY and API_SECRET as environment variables on heroku.
It also shows The command '/bin/sh -c python manage.py collectstatic --noinput' returned a non-zero code: 1 which is in my DockerFile:
FROM python:3.10
# setting work directory
WORKDIR /usr/src/app
# env variables
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWEITEBYTECODE 1
ENV DEBUG 0
# install psycopg dependencies
RUN apt-get update && apt-get install -y \
build-essential \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*
# install dependencies
RUN pip install --upgrade pip
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
RUN python manage.py collectstatic --noinput
RUN adduser -u 99 myuser
USER myuser
CMD gunicorn ramrobazar.wsgi:application --bind 0.0.0.0:$PORT
And here's my settings.py:
INSTALLED_APPS = [
'phonenumber_field',
'ramrobazar.account.apps.AccountConfig',
'ramrobazar.dashboard.apps.DashboardConfig',
'ramrobazar.inventory.apps.InventoryConfig',
'ramrobazar.drf.apps.DrfConfig',
'ramrobazar.demo.apps.DemoConfig',
'mptt',
'rest_framework',
'rest_framework.authtoken',
'rest_framework_simplejwt.token_blacklist',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'cloudinary_storage',
'django.contrib.staticfiles',
'cloudinary',
'corsheaders',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
STATIC_URL = 'static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_STORAGE = 'cloudinary_storage.storage.StaticHashedCloudinaryStorage'
MEDIA_URL = 'media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
CLOUDINARY_STORAGE = {
'CLOUD_NAME': os.environ.get('CLOUD_NAME'),
'API_KEY': os.environ.get('API_KEY'),
'API_SECRET': os.environ.get('API_SECRET'),
}
DEFAULT_FILE_STORAGE = 'cloudinary_storage.storage.MediaCloudinaryStorage'
This is my first time using heroku and cloudinary. If anyone has any idea about what the problem might be, please share it. If any more information is required, please ask.

Django 1.11 ManifestStaticFilesStorage - all hashed static files return 404 [duplicate]

Am building an app using Django as my workhorse. All has been well so far - specified db settings, configured static directories, urls, views etc. But trouble started sneaking in the moment I wanted to render my own beautiful and custom 404.html and 500.html pages.
I read the docs on custom error handling, and set necessary configurations in UrlsConf, created corresponding views and added the 404.html and the 500.html to my app's template directory (specified in the settings.py too).
But the docs say you can actually view custom error views until Debug is Off, so I did turn it off to test my stuff, and that's when stuff goes berserk!
Not only do I fail to view the custom 404.html (actually, it loads, but because my error pages each contain a graphic error message -as some nice image), the source of the error page loads, but nothing else loads! Not even linked CSS or Javascript!
Generally, once I set DEBUG = False, all views will load, but any linked content (CSS, Javascript, Images, etc) wont load! What's happening? Is there something am missing, concerning static files and the DEBUG setting?
If you still need to server static locally (e.g. for testing without debug) you can run devserver in insecure mode:
manage.py runserver --insecure
With debug turned off Django won't handle static files for you any more - your production web server (Apache or something) should take care of that.
In urls.py I added this line:
from django.views.static import serve
add those two urls in urlpatterns:
url(r'^media/(?P<path>.*)$', serve,{'document_root': settings.MEDIA_ROOT}),
url(r'^static/(?P<path>.*)$', serve,{'document_root': settings.STATIC_ROOT}),
and both static and media files were accesible when DEBUG=FALSE.
You can use WhiteNoise to serve static files in production.
Install:
pip install WhiteNoise==2.0.6
And change your wsgi.py file to this:
from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise
application = get_wsgi_application()
application = DjangoWhiteNoise(application)
And you're good to go!
Credit to Handlebar Creative Blog.
BUT, it's really not recommended serving static files this way in production. Your production web server(like nginx) should take care of that.
Johnny's answer is great, but still didn't work for me just by adding those lines described there. Based on that answer, the steps that actually worked for me where:
Install WhiteNoise as described:
pip install WhiteNoise
Create the STATIC_ROOT variable and add WhiteNoise to your MIDDLEWARE variable in settings.py:
#settings.py
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware', #add whitenoise
'django.contrib.sessions.middleware.SessionMiddleware',
...
]
#...
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') ##specify static root
Then, modify your wsgi.py file as explained in Johnny's answer:
#wsgi.py
from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise
application = get_wsgi_application()
application = DjangoWhiteNoise(application)
After that, deploy your changes to your server (with git or whatever you use).
Finally, run the collectstatic option from your manage.py on your server. This will copy all files from your static folders into the STATIC_ROOT directory we specified before:
$ python manage.py collectstatic
You will now see a new folder named staticfiles that contains such elements.
After following these steps you can now run your server and will be able to see your static files while in Production mode.
Update: In case you had version < 4 the changelog indicates that it's no longer necessary to declare the WSGI_APPLICATION = 'projectName.wsgi.application' on your settings.py file.
If you are using the static serve view in development, you have to have DEBUG = True :
Warning
This will only work if DEBUG is True.
That's because this view is grossly
inefficient and probably insecure.
This is only intended for local
development, and should never be used
in production.
Docs: serving static files in developent
Updated link, and this
EDIT: You could add some urls just to test your 404 and 500 templates, just use the generic view direct_to_template in your urls.
from django.views.generic.simple import direct_to_template
urlpatterns = patterns('',
('^404testing/$', direct_to_template, {'template': '404.html'})
)
You actually can serve static files in a production Django app, securely and without DEBUG=True.
Rather than using Django itself, use dj_static in your WSGI file (github):
requirements.txt:
...
dj-static==0.0.6
YOURAPP/settings.py:
...
STATIC_ROOT = 'staticdir'
STATIC_URL = '/staticpath/'
YOURAPP/wsgi.py:
...
from django.core.wsgi import get_wsgi_application
from dj_static import Cling
application = Cling(get_wsgi_application())
You can debug this in many different ways. Here's my approach.
localsettings.py:
DEBUG = False
DEBUG404 = True
urls.py:
from django.conf import settings
import os
if settings.DEBUG404:
urlpatterns += patterns('',
(r'^static/(?P<path>.*)$', 'django.views.static.serve',
{'document_root': os.path.join(os.path.dirname(__file__), 'static')} ),
)
Be sure to read the docs ;)
https://docs.djangoproject.com/en/2.0/howto/static-files/#limiting-use-to-debug-true
Ultimate solution:-
So basically when you make debug = False, Django doesn't want to take care of your static files.
So we want something that can take care of our files.
The answer is whitenoise.
pip install whitenoise in your environment
Add 'whitenoise.middleware.WhiteNoiseMiddleware' in your middleware list in settings.py.
This should be added just below the 'django.middleware.security.SecurityMiddleware' and above all the remaining middleware. So that your middleware list will look like this:-
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
# add it exactlyhere
'django.contrib.sessions.middleware.SessionMiddleware',
'...'
]
Add 'whitenoise.runserver_nostatic' on top of your installed apps
So that your installed apps list will look like this:-
INSTALLED_APPS = [
'whitenoise.runserver_nostatic',
'django.contrib.admin',
'django.contrib.auth',
'...'
]
Done, you will be able to serve static files in production now!!
From here I took help by mixing a few answers. Here, I am adding my whole parts. [I am doing this for a beginners help and for my future use as well]
Well at first the question is why Debug=False needed!
I put my project in AWS and it was being connection timeout after few hours because of memory leaking.
At first I thought for celery. [of course I am just a beginner]
Then I put DEBUG=False from DEBUG=True As we can see the security warning in settings.py
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
Once I did that my staticfiles were not loading successfully in webpages.
Then I searched everywhere and at first tried from here the --insecure command to runserver.
python manage.py runserver --insecure
Which is successful but I don't want the insecure mode in my project when it is in production.
And as the proper solution [according to me] I followed the steps below.
At first, I correct the static URL,root, and dir in settings.py
STATIC_URL = '/static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static')]
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
Then collect the static files by command
python manage.py collectstatic
Now the second step, [which also provided here]
At first install whitenoise in your project directory in the command line
pip install whitenoise
Then Add 'whitenoise.middleware.WhiteNoiseMiddleware' in your middleware list in settings.py.
This should be added just below the 'django.middleware.security.SecurityMiddleware' and above all the remaining middleware. So that your middleware list will look like this:-
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware', #after this line
'whitenoise.middleware.WhiteNoiseMiddleware', #add it exactlyhere
'django.contrib.sessions.middleware.SessionMiddleware', #before this
'...'
]
Add 'whitenoise.runserver_nostatic' on top of your installed apps So that your installed apps list will look like this:-
INSTALLED_APPS = [
'whitenoise.runserver_nostatic',
'django.contrib.admin',
'django.contrib.auth',
'...'
]
Done, you will be able to serve static files in production now!! [I did on my local environment as well]
Just use the runserver command as always no insecure or anything needed.
python manage.py runserver
Boom!!! It's working for me.
Hahaha. I know kinda childish nature but I am so happy now.
Thanks to everyone who provided answers here and help my work.
This is Exactly you must type on terminal to run your project without DEBUG = TRUE
and then you see all assets (static) file is loading correctly On local server .
python manage.py runserver --insecure
--insecure : it means you can run server without security mode
For last versions of Django please look at the answer here: https://stackoverflow.com/a/7639983/6180987
For django version below 1.10 the solution should work:
Just open your project urls.py, then find this if statement.
if settings.DEBUG:
urlpatterns += patterns(
'django.views.static',
(r'^media/(?P<path>.*)','serve',{'document_root': settings.MEDIA_ROOT}), )
You can change settings.DEBUG on True and it will work always. But if your project is a something serious then you should to think about other solutions mentioned above.
if True:
urlpatterns += patterns(
'django.views.static',
(r'^media/(?P<path>.*)','serve',{'document_root': settings.MEDIA_ROOT}), )
In django 1.10 you can write so:
urlpatterns += [ url(r'^media/(?P<path>.*)$', serve, { 'document_root': settings.MEDIA_ROOT, }), url(r'^static/(?P<path>.*)$', serve, { 'document_root': settings.STATIC_ROOT }), ]
when i make DEBUG = True my static are doesn't work.
if i run my project in python manage.py runserver --insecure . By this i got my static as well.
Solution 1:
python manage.py runserver --insecure
Solution 2:
But I Need Permanent Solution. then i install pip install dj-static==0.0.6 and add some code to my wsgi.py file:
from django.core.wsgi import get_wsgi_application
from dj_static import Cling
application = Cling(get_wsgi_application())
and then i added some in setting.py:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, '/static/')
STATICFILES_DIRS = [
BASE_DIR / "static",
]
I agree with Marek Sapkota answer; But you can still use django URFConf to reallocate the url, if static file is requested.
Step 1: Define a STATIC_ROOT path in settings.py
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
Step 2: Then collect the static files
$ python manage.py collectstatic
Step 3: Now define your URLConf that if static is in the beginning of url, access files from the static folder staticfiles. NOTE: This is your project's urls.py file:
from django.urls import re_path
from django.views.static import serve
urlpattern += [
re_path(r'^static/(?:.*)$', serve, {'document_root': settings.STATIC_ROOT, })
]
I got this problem today and this fixed it while on development, If you still need to server static locally (e.g. for testing without debug) you can run devserver in insecure mode:
manage.py runserver --insecure
Don't worry because when in production, this hosting platform (Apache, Heroku E.T.C ) would handle serving the static files for you.
Note: Heroku Doesn't server static files, you'd want to put it on AWS or MS Azure
nginx,settings and url configs
If you're on linux this may help.
nginx file
your_machn:/#vim etc/nginx/sites-available/nginxfile
server {
server_name xyz.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /var/www/your_prj;
}
location /media/ {
root /var/www/your_prj;
}
...........
......
}
urls.py
.........
.....
urlpatterns = [
path('admin/', admin.site.urls),
path('test/', test_viewset.TestServer_View.as_view()),
path('api/private/', include(router_admin.urls)),
path('api/public/', include(router_public.urls)),
]
if settings.DEBUG:
import debug_toolbar
urlpatterns += static(settings.MEDIA_URL,document_root=settings.MEDIA_ROOT)
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
settings.py
.....
........
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static/')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
.....
....
Ensure to run:
(venv)yourPrj$ ./manage.py collectstatic
yourSys# systemctrl daemon-reload
This is normal and intended behavior.
Warning
This will only work if DEBUG is True.
you can actually view custom error views until Debug is Off
If Django is just reading from the filesystem and sending out a file, then it has no advantage over a normal web server, all web servers are capable to server the files on it's own.
Furthermore, if you serve static files with Django, you will keep the Python process busy for the duration of the request and it will be unable to serve the dynamic requests to which it is more suited.
For these reasons, the Django static view is designed only for use during development and will not work if your DEBUG setting is False.
Since during development we only usually have one person accessing the site at a time (the
developer), Django is fine to serve static files.
Support for string view arguments to url() is deprecated and will be removed in Django 1.10
My solution is just small correction to Conrado solution above.
from django.conf import settings
import os
from django.views.static import serve as staticserve
if settings.DEBUG404:
urlpatterns += patterns('',
(r'^static/(?P<path>.*)$', staticserve,
{'document_root': os.path.join(os.path.dirname(__file__), 'static')} ),
)
I did the following changes to my project/urls.py and it worked for me
Add this line :
from django.conf.urls import url
and add :
url(r'^media/(?P<path>.*)$', serve, {'document_root': settings.MEDIA_ROOT, }),
in urlpatterns.
Although it's not safest, but you can change in the source code. navigate to Python/2.7/site-packages/django/conf/urls/static.py
Then edit like following:
if settings.DEBUG or (prefix and '://' in prefix):
So then if settings.debug==False it won't effect on the code, also after running try python manage.py runserver --runserver to run static files.
NOTE: Information should only be used for testing only

Unable to set up Jinja2 with Django

I've installed Django 1.9.7, and I have Pythons 3.4.3 and 2.7.10 on Ubuntu.
These are the steps I've followed:
Made a new project with django-admin startproject testproject
cd testproject/testproject
Made an app within the project with django-admin startapp testapp
Made a directory for templates in that app with mkdir testapp/templates and added a very basic index.html template in there
Edited settings.py to change the template backend to django.template.backends.jinja2.Jinja2, by editing line 57 of the default settings file, and to add testproject.testapp to INSTALLED_APPS; the TEMPLATES section is therefore like this:
TEMPLATES = [
{
'BACKEND': 'django.template.backends.jinja2.Jinja2',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
Edited urls.py, adding from testproject.testapp import views and a URL pattern url(r'^$', views.index),
Edited testapp/views.py adding
def index(request):
return render(request, 'index.html')
cd ..
Ran the server with python3 manage.py runserver, or python manage.py runserver -- very similar effect
Take a browser to http://localhost:3000
I get an error. On the command line I get this:
Internal Server Error: /
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/django/template/utils.py", line 86, in __getitem__
return self._engines[alias]
KeyError: 'jinja2'
Followed by another exception caused "during handling of the above exception", which matches the exception I see in the browser:
Environment:
Request Method: GET
Request URL: http://localhost:3000/
Django Version: 1.9.7
Python Version: 3.4.3
Installed Applications:
['django.contrib.staticfiles', 'testproject.testapp', 'webpack_loader']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback:
File "/usr/local/lib/python3.4/dist-packages/django/template/utils.py" in __getitem__
86. return self._engines[alias]
During handling of the above exception ('jinja2'), another exception occurred:
File "/usr/local/lib/python3.4/dist-packages/django/core/handlers/base.py" in get_response
174. response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.4/dist-packages/django/core/handlers/base.py" in get_response
172. response = response.render()
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in render
160. self.content = self.rendered_content
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in rendered_content
135. template = self._resolve_template(self.template_name)
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in _resolve_template
90. new_template = self.resolve_template(template)
File "/usr/local/lib/python3.4/dist-packages/django/template/response.py" in resolve_template
80. return select_template(template, using=self.using)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader.py" in select_template
55. engines = _engine_list(using)
File "/usr/local/lib/python3.4/dist-packages/django/template/loader.py" in _engine_list
143. return engines.all() if using is None else [engines[using]]
File "/usr/local/lib/python3.4/dist-packages/django/template/utils.py" in all
110. return [self[alias] for alias in self]
File "/usr/local/lib/python3.4/dist-packages/django/template/utils.py" in <listcomp>
110. return [self[alias] for alias in self]
File "/usr/local/lib/python3.4/dist-packages/django/template/utils.py" in __getitem__
101. engine = engine_cls(params)
File "/usr/local/lib/python3.4/dist-packages/django/template/backends/jinja2.py" in __init__
35. self.env = environment_cls(**options)
Exception Type: TypeError at /
Exception Value: __init__() got an unexpected keyword argument 'context_processors'
I get very similar traces with Python 2.
I found this question which has a similar error message (KeyError: 'jinja2') but seems to be a separate problem, and this bug report which has the same error again whose solution is to install jinja2, but jinja2 is definitely installed. At least, I can run python or python3 and then import jinja2. pip says jinja2 is up to date.
I must be missing something crucial -- any ideas?
The error about context_processors is because the Jinja2 backend does not support that argument.
You should add an additional backend to your TEMPLATES setting, rather than replacing the existing backend from django to jinja2. See this answer for more information. If you replace the existing backend, then apps that require Django templates will not work, including the admin.
Finally, the jinja2 backend will look for templates in testapp/jinja2, not testapp/templates.

Virtualenv package issue on Heroku deploy (Django app)

I have deployed a Django project to Heroku, but I've run into an issue: there is a package, that's inside the virtualenv folder, that gives me an error.
This is the traceback
Environment:
Request Method: POST
Request URL: https://pin-a-voyage.herokuapp.com/login/
Django Version: 1.8
Python Version: 2.7.6
Installed Applications:
('django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'blog',
'custom_user',
'django_markdown',
'parsley')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware')
Traceback:
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
132. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/app/blog/views.py" in login
147. auth_login(request, user)
File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/auth/__init__.py" in login
117. user_logged_in.send(sender=user.__class__, request=request, user=user)
File "/app/.heroku/python/lib/python2.7/site-packages/django/dispatch/dispatcher.py" in send
201. response = receiver(signal=self, sender=sender, **named)
File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/auth/models.py" in update_last_login
26. user.save(update_fields=['last_login'])
File "/app/custom_user/models.py" in save
62. new_image = resizeimage.resize_width(pil_image_obj, 300)
File "/app/.heroku/python/lib/python2.7/site-packages/resizeimage/resizeimage.py" in wrapper
31. validator(image, size)
File "/app/.heroku/python/lib/python2.7/site-packages/resizeimage/resizeimage.py" in _width_is_big_enough
47. raise ImageSizeError(image.size[0], width)
Exception Type: ImageSizeError at /login/
Exception Value: 'Image is too small, Image size : 300, Required size : 300'
In a nutshell: I upload a profile pic, but I run into this error, because I have modified the package on the virtualenv folder, but the virtualenv folder isn't pushed to Heroku.
Since the virtualenv folder has to be put in .gitignore when deploying to Heroku (isn't it?), how can I avoid this problem?
If you need to make local edits to a package ( but consider instead contributing or forking a new version ), then you may want to copy that package into your Django project and use it as if you would have written it yourself. I.e. As a regular Django app in your project.
In most cases one should not edit files in a virtual environment. This is because virtual environment holds packages installed with pip only in the local system. However the developer can edit code of the installed app if requirement cannot by achieved by using the options/attributes provided with the package.
You can push customized code of your installed app and can use it in your production environment by using overriding file with custom code. See reusable apps for an example.
First make sure updated requirements.txt is pushed to your production environment. From the stack trace /site-packages/resizeimage/ is causing you issues. If you are using PIL update requirements.txt to include pil and install PIL in your heroku server too.
then override the necessary files of your package. To do this you can create a directory with same name as of the pip package and then create a file with the same name of the one you have to edit. After this push this directory.

Django deployment - can't import app.urls

I just moved a django project to a deployment server from my dev server, and I'm having some issues deploying it. My apache config is as follows:
<Location "/">
Order allow,deny
Allow from all
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE project.settings
PythonDebug On
PythonPath "['/home/django/'] + sys.path"
</Location>
Django does work, since it renders the Django debug views, but I get the following error:
ImportError at /
No module named app.urls
And here is all the information Django gives me:
Request Method: GET
Request URL: http://myserver.com/
Django Version: 1.1.1
Python Version: 2.6.5
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.admin',
'django.contrib.admindocs',
'project.app']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware')
Traceback:
File "/usr/lib64/python2.6/site-packages/django/core/handlers/base.py" in get_response
83. request.path_info)
File "/usr/lib64/python2.6/site-packages/django/core/urlresolvers.py" in resolve
218. sub_match = pattern.resolve(new_path)
File "/usr/lib64/python2.6/site-packages/django/core/urlresolvers.py" in resolve
216. for pattern in self.url_patterns:
File "/usr/lib64/python2.6/site-packages/django/core/urlresolvers.py" in _get_url_patterns
245. patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/lib64/python2.6/site-packages/django/core/urlresolvers.py" in _get_urlconf_module
240. self._urlconf_module = import_module(self.urlconf_name)
File "/usr/lib64/python2.6/site-packages/django/utils/importlib.py" in import_module
35. __import__(name)
Exception Type: ImportError at /
Exception Value: No module named app.urls
Any ideas as to why I get an import error?
Add the project directory to sys.path.
My guess is that if you simply change your url configuration to reference "project.app.urls" instead of "app.urls", your problem will be fixed.
It seems that you've listed "project.app" in INSTALLED_APPS in your project's settings.py file, but you've referenced "app.urls" in your urls.py. You need to standardize and either always reference "app", and change your PythonPath to include the project directory, or always reference "project.app".
I would recommend to use virtualenv along with mod_python. Some instructions here: http://mydjangoblog.com/2009/03/30/django-mod_python-and-virtualenv/
It has the advantage of solving all your path problems, but also to allow to install extra modules (or even other versions of django) very easily.