I have a Django project that includes code for processes that are scheduled to run (via cron) independently from the website. The processes update the database using the models from one of my apps so I guess the code for these processes could be considered part of that app even though it's not part of the website. Should I create a package inside the app directory to hold these modules?
If the code you're supposed to run is tied to models in a certain app, you can write a custom management command for it.
The code lives inside your app (in myapp/management/commands/command_name.py) and you'll be able to call it using manage.py or django-admin.py, which allows you to add an entry to cron very easily.
Related
I'm using a 3rd party dependency modeltranslation-lokalise which requires me to run python manage.py makemigration. This will generate a migration file that sits within the package directory once generated, and therefore are not source controlled.
For all other migration files I'm keeping them in source control and would like to do the same for these, as it will allow us to track changes to the database in the environments we use, ensuring they can be easily replicated and that operations are not duplicated on the same database (e.g. by having the same operation exist in differently-named migrations).
My question: How are these 3rd party migrations meant to be handled when building and deploying to production (e.g. via a Dockerfile in a build pipeline?
My guess would be to find a way to reliably extract the migration files from the package and add it to source control.
The alternative would be: run makemigrations within the Dockerfile that builds the deployed image, however that could result in the same migration being run twice on the database should the name of that migration change (e.g. if there's a timestamp in the name). So I'm thinking best to avoid that.
Note: this question is similar but it considers the use of the third part optional and doesn't seem to consider the above alternatives.
I solve this using the answer of this question:
Django migration file in an other app?
By using the MIGRATIONS_MODULE settings, I was able to point django to create the migrations in my source directory, and thereby commit them to source control.
So I have an assignment to build a web interface for a smart sensor,
I've already written the python code to read the data from the sensor and write it into sqlite3, control the sensor etc.
I've built the HTML, CSS template and implemented it into Django.
My goal is to run the sensor reading script pararel to the Django interface on the same server, so the server will do all the communication with the sensor and the user will be able to read and configure the sensor from the web interface. (Same logic as modern routers - control and configure from a web interface)
Q: Where do I put my sensor_ctl.py script in my Django project and how I make it to run independent on the server. (To read sensor data 24/7)
Q: Where in my Django project I use my classes and method from sensor_ctl.py to write/read data to my djangos database instead of the local sqlite3 database (That I've used to test sensor_ctl.py)
Place your code in app/appname/management/commands folder. Use Official guide for management commands. Then you will be able to use your custom command like this:
./manage getsensorinfo
So when you will have this command registered, you can just put in in cron and it will be executed every minute.
Secondly you need to rewrite your code to use django ORM models like this:
Stat.objects.create(temp1=60,temp2=70) instead of INSERT into....
I am building a small financial web app with django. The app requires that the database has a complete history of prices, regardless of whether someone is currently using the app. These prices are freely available online.
The way I am currently handling this is by running simultaneously a separate python script (outside of django) which downloads the price data and records it in the django database using the sqlite3 module.
My plan for deployment is to run the app on an AWS EC2 instance, change the permissions of the folder where the db file resides, and separately run the download script.
Is this a good way to deploy this sort of app? What are the downsides?
Is there a better way to handle the asynchronous downloads and the deployment? (PythonAnywhere?)
You can write the daemon code and follow this approach to push data to DB as soon as you get it from Internet. Since your daemon would be running independently from the Django, you'd need to take care of data synchronisation related issues as well. One possible solution could be to use DateTimeField in your Django model with auto_now_add = True, which will give you idea of time when data was entered in DB. Hope this helps you or someone else looking for similar answer.
I asked a previous question getting a django command to run on a schedule. I got a solution for that question, but I still want to get my commands to run from the admin interface. The obstacle I'm hitting is that my custom management commands aren't getting recognized once I get to the admin interface.
I traced this back to the __init__.py file of the django/core/management utility. There seems to be some strange behavior going on. When the server first comes up, a dictionary variable _commands is populated with the core commands (from django/core/management/commands). Custom management commands from all of the installed apps are also pushed into the _commands variable for an overall dictionary of all management commands.
Somehow, though between when the server starts and when django-chronograph goes to run the job from the admin interface, the _commands variable loses the custom commands; the only commands in the dictionary are the core commands. I'm not sure why this is. Could it be a path issue? Am I missing some setting? Is it a django-chronograph specific problem? So forget scheduling. How might I run a custom management command from the django admin graphical interface to prove that it can indeed be done? Or rather, how can I make sure that custom management commands available from said interface?
I'm also using django-chronograph
and for me it works fine. I did also run once into the problem that my custom commands were not regognized by the auto-discovery feature. I think the first reason was because the custom command had an error in it. Thus it might be an idea to check whether your custom commands run without problems from the command line.
The second reason was indeed some strange path issue. I might check back with my hosting provider to provide you with a solution. Will post back to you in a few days..
i am the "unix-guy" mentioned above by tom tom.
as far as i remember there were some issues in the cronograph code itself, so it would be a good idea to use the code tom tom posted in the comments.
where on the filesystem is django-cronograph stored (in you app-folder, in an extra "lib-folder" or in your site-packages?
when you have it in site-packages or another folder that is in your "global pythonpath" pathing should be no issue.
the cron-process itself DOES NOT USE THE SAME pythonpath, as your django app. remember: you start the cron-process via your crontab - right? so there are 2 different process who do not "know" each other: the cron-process AND the django-process (initialized by the webserver) so i would suggest to call the following script via crontab and export pythonpath again:
#!/bin/bash
PYTHONPATH=/path/to/libs:/path/to/project_root:/path/to/other/libs/used/in/project
export PYTHONPATH
python /path/to/project/manage.py cron
so the cron-started-process has the same pythonpath-information as your project.
greez from vienna/austria
berni
How to make Django execute something automatically at a particular time.?
For example, my django application has to ftp upload to remote servers at pre defined times. The ftp server addresses, usernames, passwords, time, day and frequency has been defined in a django model.
I want to run a file upload automatically based on the values stored in the model.
One way to do is to write a python script and add it to the crontab. This script runs every minute and keeps an eye on the time values defined in the model.
Other thing that I can roughly think of is maybe django signals. I'm not sure if they can handle this issue. Is there a way to generate signals at predefined times (Haven't read indepth about them yet).
Just for the record - there is also celery which allows to schedule messages for the future dispatch. It's, however, a different beast than cron, as it requires/uses RabbitMQ and is meant for message queues.
I have been thinking about this recently and have found django-cron which seems as though it would do what you want.
Edit: Also if you are not specifically looking for Django based solution, I have recently used scheduler.py, which is a small single file script which works well and is simple to use.
I've had really good experiences with django-chronograph.
You need to set one crontab task: to call the chronograph python management command, which then runs other custom management commands, based on an admin-tweakable schedule
The problem you're describing is best solved using cron, not Django directly. Since it seems that you need to store data about your ftp uploads in your database (using Django to access it for logs or graphs or whatever), you can make a python script that uses Django which runs via cron.
James Bennett wrote a great article on how to do this which you can read in full here: http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/
The main gist of it is that, you can write standalone django scripts that cron can launch and run periodically, and these scripts can fully utilize your Django database, models, and anything else they want to. This gives you the flexibility to run whatever code you need and populate your database, while not trying to make Django do something it wasn't meant to do (Django is a web framework, and is event-driven, not time-driven).
Best of luck!