Maintain the media files after build with Docker Container - Django - django

I am using Django as a Web Framework. Azure NGINEX as WebServer. My Project is deployed with Docker Containers.
In my Django project root structure will be as follows:
root:
- app1
- app2
- media
whenever saving images, it will correctly save under media folder. But whenever doing "docker-compose up" it will replaces the source code, so that my media folder will be cleaned up everytime.
In my settings.py file, I have added as follows:
MEDIA_ROOT = os.path.join(BASE_DIR,'media')
MEDIA_URL = 'media/'
Kindly help me to maintain the media files with Docker based Environment

As mentioned in the previous answer, this isn't a best practice to handle media and static inside your project or app directory rather you can use a file or file storage server. But I am trying to give the answer to your question here.
Suppose you have a Django project directory named root and inside this, you are managing the media and static folders. As you are using Docker every time your container gets restarted, it wipes out the contents from both these folders. So what you have to do here is mount your media and static folders from inside of your container to your local storage i.e /var/lib/docker/volumes/{volume_name}/_data to persist your media and static files in between restarts of your container.
I am describing here the docker-compose version:
version: "3.8"
services:
app:
image: {your django project image}
#build: {direct build of your Dockerfile}
volumes:
- media:/src/media/
- static:/src/static/
volumes:
media:
static:
My goal here is to point out volume mount, so in the above code, you have to define volumes this way, where /src is the working directory defined in your Dockerfile using the WORKDIR directive. And in my case media and static are the direct children of the src folder. Now you can run docker volume ls to see your volume names and using the name you can inspect your volumes using this command docker volume inspect {volume_name}. Usually, you will find your volumes i.e media and static here -
/var/lib/docker/volumes/{{container_name]_media}/_data
/var/lib/docker/volumes/{{container_name]_static}/_data
Hope this clears the question.

You have to basically do 2 things:
Changing the directory to keep the media files to somewhere out of your source code directory - it's not any good practice to keep them in the source directory because of several reasons,
Using docker volumes in your docker-compose configuration file to persist your media directory. You can find the detailed documentation on the way to configure volumes in docker-compose here.

Related

Set default media files in Docker using Django

I am deploying my Django web application using Docker and Docker Compose, Nginx and Postgresql. I have already done the deployment and it works well. But I have a problem with the default media files.
When I was deploying in local and in my server, I add those default media files manually to the media folder, but now with Docker I can't do that.
One of my models would be that one, which has a default image "dictionary/default.png" and obviously when I run my docker image doesn't load because it doesn't exist anywhere.
class Dictionary(models.Model):
dictionary_name = models.CharField(max_length=200)
dictionary_picture = models.ImageField(upload_to='dictionary', default='dictionary/default.png')
I was thinking to have these default media files locally and copy them to the Docker media folder in the Dockerfile, but I do not know if this is possible.
There is a COPY instruction in the Dockerfile.
For example, copy all files from the dictionary to the media:
...
COPY dictionary/ /app/media/
...
If you already set WORKDIR /app:
...
COPY dictionary/ ./media/
...

Django collectstatic keeps waiting when run through Github Action

We are facing a very weird issue. We ship a django application in a docker container through Github Actions on each push. Everything is working fine except collectstatic.
We have the following lines at the end of our CD github action:
docker exec container_name python manage.py migrate --noinput
docker exec container_name python manage.py collectstatic --noinput
migrate works perfectly fine, but collectstatic just keeps on waiting if ran through the github action. If I run the command directly on the server then it works just fine and completes with in few minutes.
Can someone please help me figuring out what could be the issue?
Thanks in advance.
Now I am far from the most experienced but I did this recently and I have some suggestions of where to look. I'm definitely not the greatest authority though.
I wasn't using docker so I can't say anything about that. From the issues, I had here are some suggestions I can recommend to try.
Take note that all of this was for a self-hosted runner. Things would be very different otherwise.
Check to make sure STATIC_ROOT and MEDIA_ROOT variables are set correctly in the settings file.
If the STATIC and MEDIA root variables are environment variables make sure you are serving the correct environment variables file like a .env file which I used.
I used django-environ to serve my environment variables. From the docs, it says to have the .env file in the same directory as the settings file. Well if you are putting the project on a production server with github actions, you won't be able to put the .env file anywhere in the project because it will get overwritten every time new code is pushed.
So to fix that you need to specify the correct .env file from somewhere else on the server. Do that by specifying ENV_PATH.
https://django-environ.readthedocs.io/en/latest/
Under the section Multiple env files
Another resource that was helpful:
https://github.com/joke2k/django-environ/issues/143
I set up my settings file like how they did there.
I put my .env file in a proj directory I made in the virtualenvironment folder for the project.
I don't know if it's a good place to put it but that's how I did it. I didn't find much great info online for this stuff. Had to figure out a lot on my own.
Make sure the user which is running the github action has permissions to read the .env file.
Also like .env file, if you have the static files being collected into the base directory of your project you might have an issue with github actions overwriting those files every time new code is pushed. If you have a media directory where the user uploads files to then that will really be an issue because those files won't get overwritten. They'll just disappear.
Now if this was an issue it shouldn't cause github actions to just get stuck on the collect static command. It would just cause files to get overwritten every time the workflow runs and the media files will disappear.
If you do change the directory of where the static and media files are located as stated before, make sure all the variables for the paths are correct in the settings file and the .env file.
You will also need to update the nginx config file for the static and media root directories if you used nginx. Not sure about how apache does this.
You can do that with this command:
sudo nano /etc/nginx/sites-available/myproject
Don't forget to restart the nginx server after doing that.
If you are writing static and media files at a different location from the base project directory on the server, also check permissions on those directories. Make sure the user running the github action has permissions to write to those directories. I suspect that might cause it to hang but it very well might just cause an error.
Check all the syntax in the github actions yml file. Make sure everything is correct and it's not hanging cause it had an incomplete command or something like that.
But yeah, that's some things I had to take a look at. Honestly, none of this might be relevant for you. All of these issues should cause an error somewhere for the most part.
I couldn't really offer many external resources for you to look deeper into this because I'm just speaking from personal experience.
Hope I could help.
Heres my github repo for the project I did: https://github.com/pkudlanov/personal-portfolio-django
I hosted it on digitalocean on a linux server using nginx and gunicorn.

File uploads in Heroku deployment with Django

So I was finally able to set up local + prod test project I'm working on.
# wsgi.py
from dj_static import Cling, MediaCling
application = Cling(MediaCling(get_wsgi_application()))
application = DjangoWhiteNoise(application)
I set up static files using whitenoise (without any problems) and media (file uploads) using dj_static and Postgres for local + prod. Everything works fine at first... static files, file uploads.
But after the Heroku dynos restart I lose all the file uploads. My question is, --- Since I'm serving the media files from the Django app instead of something like S3, does the dyno restart wipe all that out too?
PS: I'm aware I can do this with AWS, etc, but I just want to know if thats the reason I'm losing all the uploads.
Since I'm serving the media files from the Django app instead of something like S3, does the dyno restart wipe all that out too?
Yes!. That's right. According to the Heroku docs:
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code.
See, also this answer and this answer.
Conclusion: For media files (the uploaded ones), you must use some external service (like S3 or something). whitenoise is just for static files. See here why whitenoise is not suitable for serving user-uploaded (media) files.

running nginx/wusgi/mysql/django in docker container

I have a docker image for running a django app. If I mount the dir containing the django app when I create the container it works fine. But I want to make the image self-contained and not dependent on the local file system. So I changed the Dockerfile to copy the dir containing the django app from the host machine into the image. But then, when I create the container (without mounting the dir) I get permission denied on all accesses to that dir (e.g. the socket, the static files, ...). Everything is world readable and executable. Anyone have any clues as to what could be causing this?
I ended up fixing it. Turned out one of the dirs in the path was not readable. That is, the django app was in /foo/bar/baz and although /foo and /foo/bar/baz were readable, /foo/bar was not. Once I chmod-ed that all was well.

Heroku deleting files upon git push?

I have a media folder that stores all my uploaded images during development and I'm pushing my django(1.5.1) project to a dev server on heroku. Inside the media folder I have
media/
# cache and images were commited before .gitignored was added
cache/ # store thumbnails
images/ # store images
.gitignore
the .gitignore has
*
!.gitignore
The problem is whenever I git push to heroku, all my testing uploads are wiped out by git. Is there a way to deal with this?
This isn't an issue with Git, but rather with Heroku. Heroku's file system is ephemeral, and is reset between deploys. Use a service like Amazon S3 to store uploaded files.