I've run manage.py collectstatic and Whitenoise has post-processed all of the static files. I'm not quite sure what should i do now if i want to change/update some of the files, for example, my .css stylesheet? Should i run manage.py collectstatic every time files have been changed? I'm asking this because my development server takes about 45 minutes to finish that task, and i'm not sure if that's normal because i have only 550 static files, 250Mb total.
Secondly, as Whitenoise doesn't support serving media files i use Amazon CloudFront for that. How can i control cache busting with those media files that users have uploaded? This is very important for me.
Yes, you'll need to run collectstatic every time your files change.
It's very unusual to have 250MB of static files. Also, because Django's cache busting creates a copy of each file with a unique name you'll end up with two copies of each file so that's 500MB already. On top of this WhiteNoise will be creating gzip-compressed versions of each file so you could be heading towards 1GB of files.
One quick way of speeding up this process would be to tell WhiteNoise not to compress your PDF files, which you can do by adding .pdf to the WHITENOISE_SKIP_COMPRESS_EXTENSIONS setting.
It sounds like your brochures would be better stored as user media though, rather than static assets.
To control caching you should make your code generate a unique name for each file when it's uploaded (adding a random string as a prefix to the filename should do the trick). You can then set caching headers on these files for as long as you like.
Related
How can i get django to process media files on production when DEBUG = False on heroku server?
I know that it’s better not to do this and that this will lead to a loss of performance, but my application is used only by me and my cat, so I don't think that this will be unjustified in my case.
The reason this won't work is because the Heroku file system is ephemeral, meaning any files uploaded after your app code is pushed will be overwritten anytime your app is restarted. This will leave your app with image links in the DB which lead to non-existent files.
You can read more about it here:
https://help.heroku.com/K1PPS2WM/why-are-my-file-uploads-missing-deleted
Your best bet is using a bucket like Amazon S3 to upload your files to. It costs almost nothing for small use, and is very reliable.
https://blog.theodo.com/2019/07/aws-s3-upload-django/
I need to serve some content that should be preprocessed before getting served. There are huge volume of files (500,000 files with sizes around 1GB for example.) My server is written in Django. I can do the preprocess in python (a Django view for example) but I can't do it in Nginx or other static file servers. Is there anyway to implement a Django view that serves these files efficiently with random access? Is there any modules for this purpose? What should I take care of to implement it?
P.S. The files are not saved anywhere, there are around 2000 files and all other files can get generated by these 2000 files in the python code (Django view). I don't wanna buy that much hard disk to preprocess and store all 500,000 final files.
For a production Django website with several applications that generate graphs/images based on user input data, how should these images be handled?
Currently each image is stored to the local folders (e.g. app1/static/app1, app2/static/app2) folder upon generation.
Images are then copied using manage.py collectstatic to a central folder (e.g. main_app/static/app1) from which they are served (i.e. this folder is STATIC_ROOT).
The issue is that the images are obviously not found in the main static folder when dynamically generated as I'm relying on collectstatic to move them to the STATIC_ROOT.
Questions:
Should dynamic images be served from the local main_app/static/app1 folder in production (i.e. change STATIC_ROOT)?
Should the dynamic images be saved to the main directory folder (e.g. main_app/static/app1) instead of relying on collectstatic?
Should dynamic images be handled in some other way entirely in production?
Directory structure and image locations for clarity:
main_app
settings.py
models.py
view.py
manage.py
/static/app1 (images copied here using `collectstatic`)
/static/app2 (images copied here using `collectstatic`)
/app1
/static/app1 (images saved here in production)
/app2
/static/app2 (images saved here in production)
The basis for the above is based on https://docs.djangoproject.com/en/1.8/howto/static-files/deployment/ , but my spider sense tells me I'm doing this completely wrong. Using nginx and gunicorn for production.
As a neophyte I haven't much on this so I'd appreciate some tips on terminology (i.e. these don't seem like they are static images at all, but I've found very little on dynamic images in the documentation).
I suppose it takes time to generate the images, so it's a good idea to create a task for image generation in your view and pass it to some task queue, e.g. Celery. Let it create the image and save it to media folder, e.g. local MEDIA_ROOT or some remote storage like Amazon, and add a link to it to your django model object, that uses this image. Then when the view is requested it will return the link to these images amongst its other content.
See Managing Files.
Your images are not really static files, especially if they get 'recalculated' over time. So IMO using collectstatic is not the best decision in this case.
Hi I have an issue that is preventing some additional security on my website,
Bearing in mine, when run locally this all works perfectly.
Basically when it is being used my site will zip/ rename/ re-zip files (mostly image files)every three minuets after a certain request is made. But since deploying on heroku this does not work. it says it cant fine the file after zipping or re-naming it.
I am using Django 1.7
Heroku has a read-only filesystem. You can only write to ./tmp and ./log directories:
https://devcenter.heroku.com/articles/read-only-filesystem
There are two directories that are writeable: ./tmp and ./log (under your application root). If you wish to drop a file temporarily for the duration of the request, you can write to a filename like #{RAILS_ROOT}/tmp/myfile_#{Process.pid}. There is no guarantee that this file will be there on subsequent requests (although it might be), so this should not be used for any kind of permanent storage.
we have just started using a git account of our Django website project so that the team can collaborate on the source code.
I have heard different things concerning what should be done with the /media directory. We currently keep the /static directory under version control so that the whole project can be cloned and recreated. However, the website also contains a large amount (>400mb) of uploaded images for galleries which will likely grow over time.
Should this be under git also? Is there a reasonable size limit to be aware of when using GIT? And is there some other method for dealing with the /media folder which is used by the Django community?
Any guidance would be much appreciated.
You should exclude your media folder in the .gitignore. There are some problems.
When you check in the files its possible that they are modified (Upload script) on the server. Then you cannot pull.
when you need your sources you have to download the whole media files.
You must commit new files everytime on your server.
So we use it without media files. But if you have do automatic deployment and enough time you can to it.
Definitely don't put all your uploaded files from the live site in the source code. It's not where they belong. At the very least you should back up your /media directory to an external location e.g. another server, a local NAS, some backup provider etc.
If your development team wants access to the files during development, you should consider putting a small subset of these files in your source tree and using fixtures to create a standard set of test data for the development environment.