I want to serve downloadable files located in the /tmp directory via django. However I don't want these files copied or moved to the static_dir. The whole point of having the files in /tmp is that a cronjob will delete them every night.
I realized I could just set /tmp as my static_dir in the django settings, but I already have a static_dir set that I am using in my project.
My question is what is the easiest way to allow an end user to download files located in the /tmp directory?
If I hit this url:
http://localhost:8000/api/v2/tmp/testfile.zip
I would expect to download /tmp/testfile.zip
Can this be done via just a single entry in urls.py? Or am I going to have to create a new view?
Related
I am running a django application on Heroku, and currently using AWS S3 to serve my static files. We store our static files both in static folders per app, and also in a static/ folder at the root of the directory. The static/ folder at the root is about 40Mb large.
Whenever we deploy our app to Heroku, the static files are included in the Heroku slug, so that
heroku run python manage.py collectstatic --no-input can be run from the Dyno itself, which then copies any changed/new static files to our S3 bucket so that they can be served.
The issue is after we go through this process, we now have a static/ folder on the Dyno which takes up about 40Mb of space, and is seemingly useless since our files are being served from our S3 bucket!
Is there a better way to go about deploying our application, and collecting our static files to our S3 bucket but not copying the static files to Heroku?
One way I was thinking was to add all static files to Heroku's .slugignore file, and then configure a way to upload static files to our S3 bucket without using Heroku at all. I'm not sure if this is the correct way to go about it, however, and would appreciate advice on this.
The reason we have been looking into this is our Heroku slug size is starting to grow far too large (~450Mb), and we need to start reducing it.
After some more digging, I found examples of people doing exactly what I was describing above, which is uploading static files directly to S3 without using any intermediary storage. This article shows how to configure Django and S3 so that running python manage.py collectstatic on your local machine will copy the static files directly to S3.
This configuration, in combination with disabling collectstatic on Heroku (https://devcenter.heroku.com/articles/django-assets#disabling-collectstatic) and adding our static files to .slugignore, would be exactly what I was looking for, which was to upload static files directly to S3 without uploading them first to Heroku.
More reading from Django' docs
We are facing a very weird issue. We ship a django application in a docker container through Github Actions on each push. Everything is working fine except collectstatic.
We have the following lines at the end of our CD github action:
docker exec container_name python manage.py migrate --noinput
docker exec container_name python manage.py collectstatic --noinput
migrate works perfectly fine, but collectstatic just keeps on waiting if ran through the github action. If I run the command directly on the server then it works just fine and completes with in few minutes.
Can someone please help me figuring out what could be the issue?
Thanks in advance.
Now I am far from the most experienced but I did this recently and I have some suggestions of where to look. I'm definitely not the greatest authority though.
I wasn't using docker so I can't say anything about that. From the issues, I had here are some suggestions I can recommend to try.
Take note that all of this was for a self-hosted runner. Things would be very different otherwise.
Check to make sure STATIC_ROOT and MEDIA_ROOT variables are set correctly in the settings file.
If the STATIC and MEDIA root variables are environment variables make sure you are serving the correct environment variables file like a .env file which I used.
I used django-environ to serve my environment variables. From the docs, it says to have the .env file in the same directory as the settings file. Well if you are putting the project on a production server with github actions, you won't be able to put the .env file anywhere in the project because it will get overwritten every time new code is pushed.
So to fix that you need to specify the correct .env file from somewhere else on the server. Do that by specifying ENV_PATH.
https://django-environ.readthedocs.io/en/latest/
Under the section Multiple env files
Another resource that was helpful:
https://github.com/joke2k/django-environ/issues/143
I set up my settings file like how they did there.
I put my .env file in a proj directory I made in the virtualenvironment folder for the project.
I don't know if it's a good place to put it but that's how I did it. I didn't find much great info online for this stuff. Had to figure out a lot on my own.
Make sure the user which is running the github action has permissions to read the .env file.
Also like .env file, if you have the static files being collected into the base directory of your project you might have an issue with github actions overwriting those files every time new code is pushed. If you have a media directory where the user uploads files to then that will really be an issue because those files won't get overwritten. They'll just disappear.
Now if this was an issue it shouldn't cause github actions to just get stuck on the collect static command. It would just cause files to get overwritten every time the workflow runs and the media files will disappear.
If you do change the directory of where the static and media files are located as stated before, make sure all the variables for the paths are correct in the settings file and the .env file.
You will also need to update the nginx config file for the static and media root directories if you used nginx. Not sure about how apache does this.
You can do that with this command:
sudo nano /etc/nginx/sites-available/myproject
Don't forget to restart the nginx server after doing that.
If you are writing static and media files at a different location from the base project directory on the server, also check permissions on those directories. Make sure the user running the github action has permissions to write to those directories. I suspect that might cause it to hang but it very well might just cause an error.
Check all the syntax in the github actions yml file. Make sure everything is correct and it's not hanging cause it had an incomplete command or something like that.
But yeah, that's some things I had to take a look at. Honestly, none of this might be relevant for you. All of these issues should cause an error somewhere for the most part.
I couldn't really offer many external resources for you to look deeper into this because I'm just speaking from personal experience.
Hope I could help.
Heres my github repo for the project I did: https://github.com/pkudlanov/personal-portfolio-django
I hosted it on digitalocean on a linux server using nginx and gunicorn.
I deployed the server with Ubuntu 18, Django, Gunicorn, Nginx
And I ran into this problem:
everything works great but,
When I upload large pictures files in Django, Nginx gives 403 Error Forbidden.
I updated the permissions to the folder with static files on 755. It works!
But when I upload other files, the rights do not work.
I added the user root and user www-data to the folder owner’s group, but nothing has changed.
I understand that Nginx has no permissions, but how can I implement the inheritance permissions of new files from the parent folder
or will you suggest another solution?
You need to add FILE_UPLOAD_PERMISSIONS=0o644 variable to you settings.py file.
This is the numeric mode (i.e. 0o644) to newly uploaded files to.
For more information, please read this doc.
Try use this
chown -R www-data:www-data 'your project folder'
I am using a combination of django-storages and ManifestStaticFilesStorage to server static files and media from S3.
class StaticStorage(ManifestFilesMixin, S3BotoStorage):
location = settings.STATICFILES_LOCATION
When I run collectstatic I can see the newest version of my JS file on S3 with the correct timestamp.
I can also see that file being referenced in the staticfiles.json manifest.
However looking at the site in the browser I am still seeing the old JS being pulled down, not the one in the manifest
What could be going wrong?
The staticfiles.json seems to be loaded once when the server starts up (from the S3 instance). If you run collectstatic while the server is running it has no way of knowing that there were changes made to S3. You need to restart the server after running collectstatic if changes have been made.
You can read this post for more infomation. In short:
By default staticfiles.json will reside in STATIC_ROOT which is the
directory where all static files are collected in. We host all our
static assets on an S3 bucket which means staticfiles.json by default
would end up being synced to S3.
So if your staticfiles.json being cached, your static files will be the old ones.
There are 2 ways to fix this:
Versionize staticfiles.json like you're already done with your static files
Keep staticfiles.json in local instead of S3
How can I collect media files from different django project applications into one folder to be easier to deploy in production with Nginx/Apache?
No, there isn't a collectstatic equivalent for media files.
Media files are meant to be uploaded by the user as the site runs, so it doesn't make sense to collect them before deployment.