I have a site which enables the user to download certain files. However I want to keep a download count for each file so going the usual way by putting the static files on a different subdomain and then letting apache do the heavy lifting is not a way as well as HttpResponseRedirecting the user to a subdomain isn't good because then the user 'sees' the proper download url and can therefore download the file without incrementing the download count. I could just build a view which then serve()s the file however i am worried about that "big fat disclaimer". How would you/did you implement this? I am quite shure I am not the only one with that problem.
About the Platform: I am using apache and mod_wsgi.
Thank you
We've implemented a system where we needed to control download access to (largish) static files, naturally not wanting Django to serve them itself. We came up with a scheme whereby the Django app, after validating that the user was allowed to download the file (or increment a counter, in your case) we would create a randomly-named symlink to the file, which Apache had access to (be careful: make sure directory indexing is off etc), and then redirect the user to that symlink to be served by Apache.
We have a "cleanup" cronjob that cleans up symlink a minute after they're created, so if they want to download it again, they have to go through Django and have it counted again. Now, theoretically they could download it more than once in that time, but is that likely to happen? You could clean up more than every minute: Apache just needs the symlink to exist at the beginning of the download, not throughout the whole thing.
I'd be curious to know how others address this problem, as I agree with the OP that it is a common scenario.
psj's answer is definitely one viable option. Another option you should investigate is putting a reverse-proxy server in-front of apache like Perlbal which supports "X-REPROXY-URL" headers.
Once you have the reverse-proxy server in place, instead of sending the user a redirect response, you can send a response with the "X-REPROXY-URL" header set to a URL where the proxy server can access but the user can't. The proxy server will then read in the file from the location you sent in the header, and then serve it out to your client. They'll do so in an efficient way and since all your Django app server needs to send is a response with a header set, it is free to handle another request.
The easiest way to do this is to use Apache's X-Sendfile header. Just set the value of the header to the file path and Apache will send the file for you. This blog post has some more details: http://francoisgaudin.com/2011/03/13/serving-static-files-with-apache-while-controlling-access-with-django/ .
I did this with django-counter not to long ago. Lets you keep track of the counts in the admin.
http://github.com/svetlyak40wt/django-counter/
Related
I am providing sensitive username and password file to the authenticated user. I want user to download the file via file_url in template through model.
File_link = models.FileField(upload_to='SAFE_DIRECTORY_PATH')
I don't feel it safe storing it in media directory
Any suggestions keeping them safe ,web app will be generating the link.
Some security notes first.
This is probably a bad idea. Storing sensitive information in plain files is probably not the correct security approach, especially if you plan to use Django's media storage backend for doing that. It leaves all files out-in-the-open.
If however you really, really, and I mean really need to do that, you should encrypt the file first before saving in Django.
Again though, if at all possible I would recommend to store sensitive information in db. In your case of storing passwords, you can use Django techniques to store that information relatively-safely such as correctly hashing passwords via pbkdf function (e.g. pbkdf or bcrypt, etc). If users will need to download that information, you can always generate the file on the fly for them for download.
Some suggestions for uploading files.
I usually assign random filenames to the uploaded files. This way at least its more challenging for the users to guess the filenames to download them. Not very security since this relies on security by obfuscation but its better then nothing. If you need a Django field which does that automatically, you can do that by making upload_to a callable (there are also 3rd party libs for doing that such as django-auxilium although for full disclosure Im the author of that lib).
Now that files are stored with random filenames, you probably never want to provide direct download links to the users for download but instead authenticate them first and then use something like X-Accel in nginx or X-Sendfile in Apache to actually serve the file to the user. The idea being that you first authenticate user in Django. Then however instead of Django serving the file, you return a special header which nginx/apache catches which contains a filepath to the file nginx/apache should serve to the user. This way you dont have to waste resources in Django to serve the file however you still get the advantage of being able to authenticate the request. There are a number of 3rd party apps for doing that as well.
Finally to protect users from downloading the media files you can use nginx (and I imagine apache) by restricting certain parts of the media folder:
location /media/protected {
internal;
alias /var/www/files;
}
In this case nginx will refuse direct user requests to /media/protected and will only allow to serve those files via X-Accel-Redirect header sent by Django. Then all you have to configure in Django is to store files in that path to make them protected:
models.FileField(upload_to='protected/myfiles')
I was looking for a solution to serve files only to authorized users and came across this post. I think it it is top google result for "django storing and providing secure files"
As the answer is rather old I wanted to share my finding:
django-private-storage (https://pypi.org/project/django-private-storage/) seems to be a good solution to this problem.
I want to make more secure my django virtual host by disabling the access of files that do not belong to my project. So basically I'm looking for similiar like "php_admin_value open_basedir".
Is there a simple way to do this?
Thanks!
There's no need for a setting. As the Django docs state, your code should not be in the docroot anyway:
Where should this code live?
If your background is in plain old PHP (with no use of modern
frameworks), you’re probably used to putting code under the Web
server’s document root (in a place such as /var/www). With Django, you
don’t do that. It’s not a good idea to put any of this Python code
within your Web server’s document root, because it risks the
possibility that people may be able to view your code over the Web.
That’s not good for security.
Put your code in some directory outside of the document root, such as
/home/mycode.
The only thing that should be served is the wsgi file. So the rest of your files are safe.
I currently have Django set up to upload files to:
/path/to/project/uploads
This works great. This folder is in the root folder of the project so the files cannot be served directly from a web URL, which is what I want, the files are "CVs" uploaded by users.
I've had a look at a third-party django app called filetransfers which would do the job, but I'm wondering if there is a way with Django core to serve files from outside the media folder.
Any help would be great.
Andy
Depending on what web server you are using I would recommend using X-sendfile if you use Apache or X-accel-redirect if you use Nginx. But remember you will need to change setting in your web server. But this is far more efficient way of serving files than using Django to do it.
If what you want is to keep control on how your files are served / who can see them etc, then the simplest solution is to write a custom view serving theses files. You just have to provide the file's content as the response body and set the appropriate response headers (file type, content length etc). Reading the FineManual(tm) part about the Response object should be a good starting point.
Resolved using FileWrapper().
Thanks anyway.
I am building a system that allows users to generate a documents and then download them. The documents are PDFs (not that it matters for the sake of this question) and when they are generated I store them on my local file system that the web server is running on with uuid file names
c7d43358-7532-4812-b828-b10b26694f0f.pdf
but I know "security through obscurity" is not the right solution ...
I want to restrict access to they files on a per account basis if possible. One thing I think I could do is upload them to S3 and provide a signed URL, but I want to avoid that for now if possible.
I am using Nginx/Django/Gunicorn/EC2/S3
What are some other solutions?
If you are serving small files, you can indeed use Django to serve them directly, writing the file into the HttpResponse object.
If you're serving large files however, you might want to leave that task to your webserver, you can use the X-Accel-Redirect header on Nginx (and X-Sendfile for Apache & Lighttpd) to have your webserver serve the file for you.
You can find more information about the header itself in Nginx's documentation here, and you could find some inspiration as to how to use that in Django here.
Once you're done sending files through Django views, enforcing user authentication should be pretty straightfoward using Django's auth framework.
How about enforcing user==owner at the view level, preventing access to the files, storing them as FileFields, and only retrieving the file if that condition is met.
e.g. You could use the #login_required decorator on the view to allow access only if logged in. This could be refined using request.user to check against the owner of the file. The User Auth section of the Django documentation is likely to be helpful here.
The other option, as you mention is via S3 itself, generating urls within Django which have a querystring allowing an authenticated user access to download a particular s3 object with a time limit. Details on that can be found at the s3 documentation. A similar question has been asked before here on SO.
I've used django-private-files with great success, it enforces protection at the view level and uses differente backends to do the actual file transfer.
Ok, I know that serving media files through Django is a not recommended. However, I'm in a situation where I'd like to serve "static" files using fine-grained access control through Django models.
Example: I want to serve my movie library to myself over the web. I'm often travelling and I'd like to be able to view any of my movies wherever I am, provided I have internet access. So I rip my DVDs, upload them to my server and build this simple Django application coupled with some embeddable video player.
To avoid any legal repercussions, I'd like to ensure that only logged-on users with the proper permissions (i.e. myself and people living in the same household, which can, like me, access the real DVDs at their convenience), but denies it to other users (i.e. people who posted comments on my blog) and returns an HTTP 404.
Now, serving these files directly using Apache and mod_wsgi is rather troublesome because when an HTTP request for the media files (i.e. http://video.mywebsite.com/my-favorite-movie/) comes in, I need to validate against my user database that the person at the other end has the proper permissions.
Question: can I achieve this effect without serving the media files directly through a Django view? What are my options?
One thing I did think of is to write a simple script that takes a session ID and a video's slug and returns some boolean indicating if the user may (or may not) access the video file. Then, somehow request mod_wsgi to execute this script before accessing the requested URL and return an HTTP 404 if the script failed. However, I don't have a clue if this is even possible.
Edit: Posting this question clarified some of my ideas for search and I've come across mod_python's file wrapper extension. Does anyone have enough experience with that to validate that it is a viable solution?
Yes, you can hook into Django's authentication from Apache. See this how-to:
Authenticating against Django’s user database from Apache