I am using a custom Django command to generate XML sitemaps for a site with about 3-4 million data entries (./manage.py generate_sitemaps). This seems to work, but eats way too much memory when DEBUG is enabled in settings.py.
I usually have the DEBUG option enabled during development and frequently forget to disable it before starting the sitemap creation. If that happens, the memory starts filling up until the script crashes after about 2-3 hours. Very annoying.
Is there a way to temporarily disable the debug setting for the execution of a Django command? I thought of importing the settings module and overriding the option, but I don't think that'll work.
I think you have a couple of options here:
Import settings and throw an error to remind yourself to turn debug off.
Use the --settings= and set that equal to a file (e.g. gen_settings.py) file specifically for your generate_sitemaps command where DEBUG=False. Then create an alias for ./manage.py generate_sitemaps --settings=gen_settings
http://docs.djangoproject.com/en/dev/topics/settings/ warns specifically to not change the settings at runtime
I've used the second option before and it worked fairly well. Better than being annoyed after 2-3 hours =)
I am not really sure it helps you, but you can try:
from django.conf import settings
tmp = settings.DEBUG
settings.DEBUG = False
# some your actions
# ...
# ...
settings.DEBUG = tmp
Alternatively you can use separated settings file and set it in command line like
./manage.py your_command --settings=another_settings.py
And in that another_settings.py:
from .settings import *
DEBUG = False
Related
I have faced a strange problem.
I have a button that creates xlsx file. For example, firstly file is empty and when I press the button it becomes full of information from the database( PostgreSQL ).
And now what is wrong:
When DEBUG is TRUE in settings.py file everything works pretty fine and the document creates. When DEBUG is FALSE it do not change the file.
I really appreciate all answers, thanks!
One of the main features of debug mode is the display of detailed error pages. If your app raises an exception when DEBUG is True, Django will display a detailed traceback, including a lot of metadata about your environment, such as all the currently defined Django settings (from settings.py).
It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you’re debugging, but it’ll rapidly consume memory on a production server.
Finally, if DEBUG is False, you also need to properly set the ALLOWED_HOSTS setting. Failing to do so will result in all requests being returned as “Bad Request (400)”.
source: https://docs.djangoproject.com/en/dev/ref/settings/#debug ^_^
I still cant figure out, how to debug in django best way. For example, I created a dict and now I would like to check all the data that is arranged in that dict. How can I force django to show a debugging page in browser and print out the dict in a comfortable way?
To get the variables value in browser using Django is possible naturally only if:
DEBUG=True is set for the app and some error occurs.
Instead you can use one of the following(if you already know the line/variables to debug):
print(dict_var)
From your view: return HttpResponse(dict_var)
Other way to debug is to use Python Debugger.
Add following line in your code to put a breakpoint and then debug line by line.
import pdb; pdb.set_trace()
And run your app using
python -m pdb manage.py runserver
PyCharm
You can use the debugger of PyCharm an ide developed by JetBrains.
It's free and it helps you in so many other way.
And if you are a student the premium version it's free.
Download Link
Edit:
Then run the program in debug mode and set a breakpoint.
To set a breakpoint just click the number of line in the left of the view.
I've inherited a Django application that I need to modify using a custom template filter. I'm absolutely new to Django and am quite mystified by it. I thought I had followed the instructions exactly, and also followed all the advice from other posts on the subject, but I still get an error when I include the following line in my template:
{% load mlgb_custom_filters %}
My directory structure is as follows:
mysite (i.e. the project)
__init__.py
mlgb/ (i.e. the app)
__init__.py
templatetags/
__init__.py
mlgb_custom_filters.py
The code of mlgb_custom_filters.py is as follows:
from django import template
from django.template.defaultfilters import stringfilter
register = template.Library()
#register.filter(name='fix_dashes')
#stringfilter
def fix_dashes( value ):
return value.replace( '--', 'DASH' )
if __name__ == "__main__":
testvar = fix_dashes( "ouch -- ow -- I hate django" )
print testvar
As you can see, I've added a 'name = main' section to let me run it in standalone mode, just to check that there are no errors in that particular file, and it's fine when run in standalone mode.
Based on someone else's advice, I've also tried importing it into another file, just to see whether there was an import error, and once again, it was fine if I added this to the end of settings.py (while using the dev server):
try:
import mlgb.templatetags.mlgb_custom_filters
except Exception, exc:
print 'error importing mlgb_custom_filters'
print exc
Also, INSTALLED_APPS in settings.py includes the line 'mysite.mlgb', and I have also tried putting just 'mlgb' instead of 'mysite.mlgb' there, as yet another person suggested. And I restarted the dev server every time I made a change.
I think I have tried every single suggestion that I have found on the web until now. Does anyone have any new ideas? Could it be anything to do with the fact that I have inherited a directory structure where the template directory is not within the same structure as the application, i.e. it is not under mysite? Scraping the barrel for ideas here! I hope someone can help.
OK, in the situation that I was in when first posting this question, it seems all I actually needed to do was touch a wsgi file under my appname/apache directory to force the application to be refreshed. Yesterday's initial answer was a red herring. Basically I should have touched the file myproject/myapp/apache/myapp.wsgi. Then maybe restart Apache for good measure? But the confusion was caused by the fact that apparently it wasn't enough either simply to restart Apache or to manually recompile the Python. In order to pick up my changes, seems like I needed to touch that wsgi file. Then all is well.
I can now post an answer to my own question thanks to the help of my colleague Masud Khokhar, whose brilliant detective work has saved the day. To recap, my application worked fine until I added a 'load' statement to one of my template files, to load a 'custom filters' module. Masud identified that now I needed to use a full/absolute path to the template file in urls.py instead of a relative one as I had before (and which worked before, until it needed to load the custom filters module). So, in urls.py, I had a section of code as follows:
url(r'^book/(?P<object_id>\d+)/$', 'list_detail.object_detail',
kwargs={
'queryset':Book.objects.all(),
'template_name' : 'mlgb/mlgb_detail.html'
},
name='mlgb_detail'
),
Instead of this:
'template_name' : 'mlgb/mlgb_detail.html'
I needed something like this:
'template_name' : '/THE_FULL_PATH/mlgb/templates/mlgb/mlgb_detail.html'
Made that change - sorted! Thank you once again, Masud.
I have this line in my wsgi.conf file:
WSGIScriptAlias /trunk "c:/app/trunk/app.wsgi"
Inside of my django settings file, I need to know the alias "/trunk" to get LOGIN_URL to work properly. How can I retrieve this value from my apache settings?
Thanks!
Pete
Access the original WSGI environ dictionary for a specific request and lookup the 'SCRIPT_NAME' variable. The value of this is the notional mount point for the WSGI application as specified when using WSGIScriptAlias. Getting it through the per request environment is the only real way of doing it automatically. You cannot get to it from outside of a request and there should be no real need for you to do that.
By rights, Django should provide a way of automatically having the mount point of the application inserted into configured URLs such as that. You should perhaps bring up the issue on the official Django users list instead if you cannot find the way of doing it as perhaps a change in Django is needed.
Since you want to obtain a value from the Apache configuration, I guess the only thing you can do is read the file and process it.
Something like (assuming your settings.py lives in the same directory as wsgi.conf):
try:
f = open('wsgi.conf', 'r')
LOGIN_URL=[line for line in f.readlines() if 'WSGIScriptAlias' in line][0].split()[1]
finally:
f.close()
Catching an exception if the file is not there might be a good idea too.
Edit after your comment: Ah, I see what you are trying to do. This thread may be helpful in understanding why using os.environ won't work. The alternative they present won't help you though:
In a nutshell, the apache SetEnv isn't setting values in the process
environment that os.environ represents. Instead SetEnv is setting
values in the context of the WSGI request.
Within your code you can reference that context at request.environ:
def myView(request):
tier = request.environ['TIER']
It's me again. Because of what Apache's SetEnv is doing, I don't think you will have access to the variable in settings.py. It seems parsing the file is still the only option.
Further Edit: Another alternative - can you base your decision on the host name?
#settings.py
import socket
production_servers = { 'server1.com': '/trunk...',
'server2.com': '/trunk_2...' }
LOGIN_URL=production_servers[ socket.gethostname() ]
This completely sidesteps the information contained in apache configuration.
Is there a way to somehow trace importing of the views ? I want to find which one is broken and doesn't import in some situations (which leads to the fact that all url resolving schema in django stops working).
Pretty amazing that no one has suggested pdb. Place the following in a strategic point in your code:
import pdb;pdb.set_trace()
When execution reaches that point, the dev server will drop into a shell where you can check values of variables, trace the execution, etc.
It works like a standard shell (use any python commands you like), but there's also special commands that let you control the execution. For example next will go to the next line (processing the previous line). continue will continue execution until the next break point, etc. (full list of pdb commands)
Don't you get a stack trace? Is DEBUG set to True?
Ok, well it's possible to just write
python -v manage.py <whatevercommand>
and search for the error in the produced logs.
I'm assuming this means you're getting a 501 server error?
If you're using the Apache web server, you can set it to log python errors in the config for the site using the ErrorLog directive:
ErrorLog /tmp/django_errors.log
Then in the terminal (or via ssh):
tail -f /tmp/djanogo_errors.log
And then try to load the webpage in question. You should then be able to see what the error is and fix it.