I currently have a sizable django project. In the view, I do a large computation that is shared between views to save time on later requests. To activate the site, I have a python script that kicks off a number of scripts through manage.py for things like creating symlinks, collecting static files, etc. When each of the commands runs, it loads all the apps in the project, which does this large computation. I want to prevent this. Ideally, I'd like the precomputation to happen only once when I activate. I think I need to either:
Prevent manage.py from actually loading the view (not sure this is possible)
Lazily initialize the computation (not ideal since the first person to request a resouce will take multiple minutes)
What are my options? Any help is appreciated!
You can use AppConfig for this, more specifically AppConfig.ready()
See this SO answer or documentation on how to do this.
Edit:
One option is to run a script manually from django shell. You can do this by typing python manage.py shell in terminal. You can then import models, views and similar to shell and execute you script or function.
Another option is to bind the computation to a view that is not used for anything else, and adding an url-path to run this function. Then you can manually go to this url and the calculation will be run.
Would these methods be suitable?
Related
I heard of about Django Custom Management script in lots of community and i am not getting why should i write custom management script as django manage.py a lot of useful command? I am very new in django...
Can anyone tell me some usecase of custom management script?
Thanks
The documentation on Writing custom django-admin commands mentions:
Applications can register their own actions with manage.py. For example, you might want to add a manage.py action for a Django app that you’re distributing
Usually those commands are used for "maintenance" activities. Or to give administrators more convenient tooling.
For example, you can use django-unused-media [GitHub]. This will add an extra admin command, that you can use like:
./manage.py cleanup_unused_media --noinput
This will then remove media files that are no longer referenced. You can then make a cronjob that for example each week calls that command to clean up media files. It thus here can help you to reduce disk space usage, without having to interfere yourself.
Another use case is to make management of your server more convenient. In fact makemigrations [GitHub], migrate [GitHub], makemessages [GitHub] and runserver [GitHub] are managment commands as well. So you probably use these commands all the time.
You put a certain amount of logic into these that is useful, and that you want to repeat several times. So instead of defining migration files yourself each time you change your model, you can implement the logic once to write such commands, and then the developer/administrator, can run a convenient script to do some work.
In a Django project of mine, I've written a middleware that performs an operation for every app user.
I've noticed that the response time balloons up if I write the following at the start of the middleware module:
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE","myproject.settings")
It's about 10 times less if I omit these lines. Being a beginner, I'm trying to clarify why there's such a large differential between the respective response times. Can an expert explain it? Have you seen something like it before?
p.s. I already know why I shouldn't modify the environment variable for Django settings inside a middleware, so don't worry about that.
The reason will likely have to do something with django reloading your settings configuration for every request rather than once per server thread/process (and thus, also, re-instantiating/connecting to your database, cache, etc.). You will want to confirm this with profiling. this behavior is also very likely dependent on which app server you are running.
If you really want this level of control for your settings, it is much easier for you to add this line to manage.py, wsgi.py or whatever file/script you use to launch your app server.
P.S. If you already know you shouldn’t do it, why are you doing it?
I want to build a Django application in which each "set of users" can see different data, as they were using different DBs. Is there any best practice to achieve this?
E.g.
user A, after logging, sees and manages his own data, with no possibility to see other's data.
I could use Django multi-db feature to define several dbs and create an automatic router to point to the correct db according to the user of current request (keeping auth_user on a common db).
As last chance, I can make a different application folder for each group (the only difference being the database name), but I prefer a smarter solution.
You could consider reusing one of the many per-object permission apps for django.
Another possibility is to make different settings files. For example, settings_groupA.py:
from settings import *
DATABASES = ...
And then you could use management commands, like syncdb, or runserver, etc, etc ... with the --settings option:
--settings=SETTINGS The Python path to a settings module, e.g.
"myproject.settings.main". If this isn't provided, the
DJANGO_SETTINGS_MODULE environment variable will be
used.
Examples:
./manage.py syncdb --settings=settings_groupA
./manage.py runserver --settings=settings_groupA
The advantage of using one database per set of users is that you can keep your code easier and let them have native support for django.contrib.admin with no hack. But then if you're going to have per-object permissions within a group anyway then you might as well go straight for the first solution.
I asked a previous question getting a django command to run on a schedule. I got a solution for that question, but I still want to get my commands to run from the admin interface. The obstacle I'm hitting is that my custom management commands aren't getting recognized once I get to the admin interface.
I traced this back to the __init__.py file of the django/core/management utility. There seems to be some strange behavior going on. When the server first comes up, a dictionary variable _commands is populated with the core commands (from django/core/management/commands). Custom management commands from all of the installed apps are also pushed into the _commands variable for an overall dictionary of all management commands.
Somehow, though between when the server starts and when django-chronograph goes to run the job from the admin interface, the _commands variable loses the custom commands; the only commands in the dictionary are the core commands. I'm not sure why this is. Could it be a path issue? Am I missing some setting? Is it a django-chronograph specific problem? So forget scheduling. How might I run a custom management command from the django admin graphical interface to prove that it can indeed be done? Or rather, how can I make sure that custom management commands available from said interface?
I'm also using django-chronograph
and for me it works fine. I did also run once into the problem that my custom commands were not regognized by the auto-discovery feature. I think the first reason was because the custom command had an error in it. Thus it might be an idea to check whether your custom commands run without problems from the command line.
The second reason was indeed some strange path issue. I might check back with my hosting provider to provide you with a solution. Will post back to you in a few days..
i am the "unix-guy" mentioned above by tom tom.
as far as i remember there were some issues in the cronograph code itself, so it would be a good idea to use the code tom tom posted in the comments.
where on the filesystem is django-cronograph stored (in you app-folder, in an extra "lib-folder" or in your site-packages?
when you have it in site-packages or another folder that is in your "global pythonpath" pathing should be no issue.
the cron-process itself DOES NOT USE THE SAME pythonpath, as your django app. remember: you start the cron-process via your crontab - right? so there are 2 different process who do not "know" each other: the cron-process AND the django-process (initialized by the webserver) so i would suggest to call the following script via crontab and export pythonpath again:
#!/bin/bash
PYTHONPATH=/path/to/libs:/path/to/project_root:/path/to/other/libs/used/in/project
export PYTHONPATH
python /path/to/project/manage.py cron
so the cron-started-process has the same pythonpath-information as your project.
greez from vienna/austria
berni
How to make Django execute something automatically at a particular time.?
For example, my django application has to ftp upload to remote servers at pre defined times. The ftp server addresses, usernames, passwords, time, day and frequency has been defined in a django model.
I want to run a file upload automatically based on the values stored in the model.
One way to do is to write a python script and add it to the crontab. This script runs every minute and keeps an eye on the time values defined in the model.
Other thing that I can roughly think of is maybe django signals. I'm not sure if they can handle this issue. Is there a way to generate signals at predefined times (Haven't read indepth about them yet).
Just for the record - there is also celery which allows to schedule messages for the future dispatch. It's, however, a different beast than cron, as it requires/uses RabbitMQ and is meant for message queues.
I have been thinking about this recently and have found django-cron which seems as though it would do what you want.
Edit: Also if you are not specifically looking for Django based solution, I have recently used scheduler.py, which is a small single file script which works well and is simple to use.
I've had really good experiences with django-chronograph.
You need to set one crontab task: to call the chronograph python management command, which then runs other custom management commands, based on an admin-tweakable schedule
The problem you're describing is best solved using cron, not Django directly. Since it seems that you need to store data about your ftp uploads in your database (using Django to access it for logs or graphs or whatever), you can make a python script that uses Django which runs via cron.
James Bennett wrote a great article on how to do this which you can read in full here: http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/
The main gist of it is that, you can write standalone django scripts that cron can launch and run periodically, and these scripts can fully utilize your Django database, models, and anything else they want to. This gives you the flexibility to run whatever code you need and populate your database, while not trying to make Django do something it wasn't meant to do (Django is a web framework, and is event-driven, not time-driven).
Best of luck!