I'm in the middle of migrating a Django project to Python 3 and updating several dependencies along the way.
Right now, I'm using Django 2.2.3.
After putting the code on the staging server I noticed that all responses are returned as bytestring literals:
b'<html>\n...'
This was very hard to narrow down because I first noticed it only on the staging server. Luckily I found out that this has nothing to do with NGINX nor Gunicorn but that DEBUG=True is actually the culprit.
The question is: what does DEBUG=True trigger that is messing up the response?
It took me a several hour long train ride to figure out but I finally found the root cause:
Going over my settings file looking for something where processing changes drastically between DEBUG=False and DEBUG=True, django-pipeline's MinifyHTMLMiddleware caught my eye. Disabling it does help indeed.
An issue about it has been opened back in May already, but I couldn't find it via Google. Hopefully this answer will help someone out there.
Related
In a Django project of mine, I've written a middleware that performs an operation for every app user.
I've noticed that the response time balloons up if I write the following at the start of the middleware module:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE","myproject.settings")
It's about 10 times less if I omit these lines. Being a beginner, I'm trying to clarify why there's such a large differential between the respective response times. Can an expert explain it? Have you seen something like it before?
p.s. I already know why I shouldn't modify the environment variable for Django settings inside a middleware, so don't worry about that.
The reason will likely have to do something with django reloading your settings configuration for every request rather than once per server thread/process (and thus, also, re-instantiating/connecting to your database, cache, etc.). You will want to confirm this with profiling. this behavior is also very likely dependent on which app server you are running.
If you really want this level of control for your settings, it is much easier for you to add this line to manage.py, wsgi.py or whatever file/script you use to launch your app server.
P.S. If you already know you shouldn’t do it, why are you doing it?
Django CMS = 3.1.1, 3.1.2, 3.1.3
Django = 1.6.5, 1.6.11
The problem is:
a user with proper access rights, who previously successfully edited a page, returns to a page, and attempts to edit it.
he logs in
he selects the page - in no-edit view, it loads fine, it has a collection of content
he goes to edit view - there is absolutely no page content - static content is there, but all content relevant to that particular page is missing.
Reverting versions does no good - any version is blank.
A user who is not stuck in this state can view and edit the page just fine.
When the problem occurs, it is stuck there for many repetitions.
Things that don't fix the problem:
server restarts
dumping and reloading the database
logging out logging in
upgrading django cms across the 3.1.X space
removing reversion
flushing the mysql cache
We have the same basic installation in vagrant dev boxes, a test server emulating all settings of the live (but with older data), and the live server - all three installations have seen this problem across 3 different variants of our other code bases.
Our data dumps from a MUCH smaller database taken from 9/13 do not have this problem
We haven't been able to find a trigger or a root cause.
The problem IS coming in intermittently, but over longish time spans. For example:
2 weeks ago, all privileged users could edit the page
4 days ago - several users experienced the problem. It was repeatable on more than 1/2 the pages of the site, but 2 pages worked fine for those users. Problem was repeatable for hours.
today - in the morning the problem persisted in the 4 day ago state.
6 hours later - the problem is gone on the live site, but persists elsewhere.
5 days later - it's back and it's affecting ALL users, including those who had not previously been affected.
I feel like I'm chasing a phantom here. Any help would be appreciated. I'm glad to give data dumps or anything else requested to see what can be done or if there's a best way to turn on debug logging to trace this.
I really need help here please.
I made a rails application with the default gemfile and then followed the instructions for installation in comfy's wiki. Ran my app in the local host with webrick and sqlite3 getting no issues really, even though I am learning as I develop, but when the app was kind of ready (I judge it so it probably wasn't) for getting it up and running on heroku it failed, I checked the logs I migrated the database and tried with new apps, even tried installing psql and using unicorn as a web server (tried is emphasized cause neither of em worked for various reasons).
I've spent 6 hours (more or less) going through wikis and stuff but there really isn't much material for this CMS, please help, I even created a stack overflow account because of this issue.
I need a more detailed guide of the configuration needed for heroku.
And I have another question, what happens to the info generated by the users in heroku, how can I backup that. If I push something the differences in the cloud get erased? (Obviously the answer is no, but I want to know how and why)
Thanks a lot, this is driving me crazy.
FYI I'm an amateur.
Pd I can post some of my code's app if required.
What I'm looking for is a piece of Django middleware that catches a FileNotFound exception and attempts to rsync the file from the production webserver. This way you can develop your site with a copy of production data without having to continually rsync down all the uploaded files.
I'm sure I've seen a solution to this problem on the internets, but hours of Googleing have so far produced nothing. Anyone know of where to find this?
The code I'd seen before is this:
http://menendez.com/blog/using-django-as-pass-through-image-proxy/
which doesn't catch the FileNotFound error, but the Http404 error, and then loads images from the live server. Unfortunately this doesn't fix the issue of FileNotFound when trying to read images off disk (for sizes etc).
I'm occasionally but quite often getting an unhandled exception in cursor.execute (django1.1/db/models/sql/query.py line 2369), using psycopg2 with postgresql.
Looks like the database drops connection in some way, so Django crashes. For unhandled exception there is a ticket in Django's bugtrack (#11015), but I'm rather interested in reasons why db drops connection, not why Django doesn't catches it.
Using django's dev. server this error never happens (it runs db requests in order, concurrency never happens), so it's like it has something to do with db requests concurrency or what.
I have no access to postgresql config. or logs.
Any suggestions welcomed, maybe some postgresql tweaking, or some thoughts on how to debug this issue.
upd: looks like this question - Django + FastCGI - randomly raising OperationalError - addresses the same problem, but no solution provided :-(
The problem could be mainly with Imports. Atleast thats what happened to me. I wrote my own solution after finding nothing from the web. Please check my blogpost here: Simple Python Utility to check all Imports in your project
Ofcourse this will only help you to get to the solution of the original issue pretty quickly and not the actual solution for your problem by itself.