I am building a system that allows users to generate a documents and then download them. The documents are PDFs (not that it matters for the sake of this question) and when they are generated I store them on my local file system that the web server is running on with uuid file names
c7d43358-7532-4812-b828-b10b26694f0f.pdf
but I know "security through obscurity" is not the right solution ...
I want to restrict access to they files on a per account basis if possible. One thing I think I could do is upload them to S3 and provide a signed URL, but I want to avoid that for now if possible.
I am using Nginx/Django/Gunicorn/EC2/S3
What are some other solutions?
If you are serving small files, you can indeed use Django to serve them directly, writing the file into the HttpResponse object.
If you're serving large files however, you might want to leave that task to your webserver, you can use the X-Accel-Redirect header on Nginx (and X-Sendfile for Apache & Lighttpd) to have your webserver serve the file for you.
You can find more information about the header itself in Nginx's documentation here, and you could find some inspiration as to how to use that in Django here.
Once you're done sending files through Django views, enforcing user authentication should be pretty straightfoward using Django's auth framework.
How about enforcing user==owner at the view level, preventing access to the files, storing them as FileFields, and only retrieving the file if that condition is met.
e.g. You could use the #login_required decorator on the view to allow access only if logged in. This could be refined using request.user to check against the owner of the file. The User Auth section of the Django documentation is likely to be helpful here.
The other option, as you mention is via S3 itself, generating urls within Django which have a querystring allowing an authenticated user access to download a particular s3 object with a time limit. Details on that can be found at the s3 documentation. A similar question has been asked before here on SO.
I've used django-private-files with great success, it enforces protection at the view level and uses differente backends to do the actual file transfer.
Related
I am providing sensitive username and password file to the authenticated user. I want user to download the file via file_url in template through model.
File_link = models.FileField(upload_to='SAFE_DIRECTORY_PATH')
I don't feel it safe storing it in media directory
Any suggestions keeping them safe ,web app will be generating the link.
Some security notes first.
This is probably a bad idea. Storing sensitive information in plain files is probably not the correct security approach, especially if you plan to use Django's media storage backend for doing that. It leaves all files out-in-the-open.
If however you really, really, and I mean really need to do that, you should encrypt the file first before saving in Django.
Again though, if at all possible I would recommend to store sensitive information in db. In your case of storing passwords, you can use Django techniques to store that information relatively-safely such as correctly hashing passwords via pbkdf function (e.g. pbkdf or bcrypt, etc). If users will need to download that information, you can always generate the file on the fly for them for download.
Some suggestions for uploading files.
I usually assign random filenames to the uploaded files. This way at least its more challenging for the users to guess the filenames to download them. Not very security since this relies on security by obfuscation but its better then nothing. If you need a Django field which does that automatically, you can do that by making upload_to a callable (there are also 3rd party libs for doing that such as django-auxilium although for full disclosure Im the author of that lib).
Now that files are stored with random filenames, you probably never want to provide direct download links to the users for download but instead authenticate them first and then use something like X-Accel in nginx or X-Sendfile in Apache to actually serve the file to the user. The idea being that you first authenticate user in Django. Then however instead of Django serving the file, you return a special header which nginx/apache catches which contains a filepath to the file nginx/apache should serve to the user. This way you dont have to waste resources in Django to serve the file however you still get the advantage of being able to authenticate the request. There are a number of 3rd party apps for doing that as well.
Finally to protect users from downloading the media files you can use nginx (and I imagine apache) by restricting certain parts of the media folder:
location /media/protected {
internal;
alias /var/www/files;
}
In this case nginx will refuse direct user requests to /media/protected and will only allow to serve those files via X-Accel-Redirect header sent by Django. Then all you have to configure in Django is to store files in that path to make them protected:
models.FileField(upload_to='protected/myfiles')
I was looking for a solution to serve files only to authorized users and came across this post. I think it it is top google result for "django storing and providing secure files"
As the answer is rather old I wanted to share my finding:
django-private-storage (https://pypi.org/project/django-private-storage/) seems to be a good solution to this problem.
Do any of you know of a Django app out there for allowing users to browse for files, and upload new ones? The ones I have found seem to be built as an add-on for the admin interface (django-filebrowser, django-filer).
Not aiming for anything incredibly complex, just something that allows a user to upload files and then browse between folders (either specific directories on the server, or artificially generated "folders" out of some model field).
I recall the admin tutorial mentioned "The admin isn’t intended to be used by site visitors. It’s for site managers."
My thought from above is that it would be bad practice to simply allow users to see content via the admin interface, and that it would be better to create an app to allow for this.
To get the convenience that your end users usually expect in the age of google drive, you really want a complex javascript filebrowser that plays nicely with django.
I'm using yawd-elfinder which is a great django backend for elfinder to manage my students' association's website with great success ( About 1500 users with different groups and privileges).
Features:
Yawd-elfinder can manage local files but also use Django filesystem storages to connect to remote filesystems. A set of django options allows control over file and directory permissions, accepted mime types, max file sizes etc.
It allows you to have fairly complex management of files and different permissions for different users by activating different roots and/or mapping them differently based on the user.
Furthermore you have capabilities like drag and drop, upload by drag and drop and it's very customizable.
I'm not sure it's maintained anymore, but you can find working code here with the relevant views and templates.
I have an existing site with a number of documents being served staticly. Client wants to add login protection to the site - not a problem using django_auth. However, the files being served from apache are still downloadable?
Is there a way to restrict access?
Ideally, this would require the path to these docs not changing on the site.
Was thinking of removing the alias from the apache config and having that route to a view that has the #login_required decorator on it, and then forwarding on.
See Having Django serve downloadable files on how to set up Django to work with Apache X-Sendfile. You can wrap the X-Sendfile header sending with some authentication checks and you should be good to go.
Ok, I know that serving media files through Django is a not recommended. However, I'm in a situation where I'd like to serve "static" files using fine-grained access control through Django models.
Example: I want to serve my movie library to myself over the web. I'm often travelling and I'd like to be able to view any of my movies wherever I am, provided I have internet access. So I rip my DVDs, upload them to my server and build this simple Django application coupled with some embeddable video player.
To avoid any legal repercussions, I'd like to ensure that only logged-on users with the proper permissions (i.e. myself and people living in the same household, which can, like me, access the real DVDs at their convenience), but denies it to other users (i.e. people who posted comments on my blog) and returns an HTTP 404.
Now, serving these files directly using Apache and mod_wsgi is rather troublesome because when an HTTP request for the media files (i.e. http://video.mywebsite.com/my-favorite-movie/) comes in, I need to validate against my user database that the person at the other end has the proper permissions.
Question: can I achieve this effect without serving the media files directly through a Django view? What are my options?
One thing I did think of is to write a simple script that takes a session ID and a video's slug and returns some boolean indicating if the user may (or may not) access the video file. Then, somehow request mod_wsgi to execute this script before accessing the requested URL and return an HTTP 404 if the script failed. However, I don't have a clue if this is even possible.
Edit: Posting this question clarified some of my ideas for search and I've come across mod_python's file wrapper extension. Does anyone have enough experience with that to validate that it is a viable solution?
Yes, you can hook into Django's authentication from Apache. See this how-to:
Authenticating against Django’s user database from Apache
I'm serving "sensitive" information in downloadable PDF's and Spreadsheets within a user registration section of a site.
Is there a way to allow the django authentication to secure this media without serving it (and not have to manually login using basic auth)?
I'm guessing theres (fingers crossed) not a way to do it with the psuedo code below, but it helps better illustrate the end goal.
#urls.py
(r'^protected_media/(?P<filename>.*)$', 'protected_media')
#views.py
from django.contrib.auth.decorators import login_required
#login_required
def protected_media(request, filename):
# #login_required bounces you out to the login url
# if logged in, serve "filename" from Apache
It seems to me that the method you outlined in your code should work. It's really no different than any other protected resource: your views can serve files from disks, records from databases, rendered templates or anything. Just as the login_required decorator prevents unauthorized access to other views, it will prevent such access to your view serving protected media.
Am I missing something from your question here? Please clarify if that's the case.
EDIT: With regard to the django doc link in your comment: that's the method for simply serving any request file from a particular directory. So, in that example URLS like /site_media/foo.jpg, /site_media/somefolder/bar.jpg will automatically look for files foo.jpg and somefolder/bar.jpg under document_root. Basically, every thing under document_root will be publicly available. That's obviously insecure. So you avoid that with your method.
It's also considered inefficient because django is just adding a lot of unnecessary overhead when all you need is something like Apache to take a URL request and map it to a file on the hard drive. (You don't need django sessions, request processing, etc.)
In your case, this may not be such a big concern. First, you've secured the view. Second, it depends on your usage patterns. How many requests do you anticipate for these files? You're only using django for authentication -- does that justify other overhead? If not, you can look into serving those files with Apache and using an authentication provider. For more on this, see the mod_wsgi documentation:
http://code.google.com/p/modwsgi/wiki/AccessControlMechanisms
see the section "Apache Authentication Provider" and search for django
There are similar mechanisms available under mod_python I believe. (Update: just noticed the other answer. Please see Andre's answer for the mod_python method.)
EDIT 2: With regard to the code for serving a file, please see this snippet:
http://www.djangosnippets.org/snippets/365/
The send_file method uses a FileWrapper which is good for sending large static files back (it doesn't read the entire file into memory). You would need to change the content_type depending on the type of file you're sending (pdf, jpg, etc).
Read this Django ticket for more info. Start at the bottom to save yourself some time. Looks like it just missed getting into Django 1.2, and I assume also isn't in 1.3.
For Nginx, I found this Django snippet that takes advantage of the X-Accel-Redirect header, but haven't tried it yet.
If I understand your question correctly you want to restrict access to files that are not being served by Django, for example, with an Apache server?
What you would then require is some way for this Apache server to use Django as an authentication source.
This django snippet describes such a method. It creates an access handler in Django which is used by Apache when a request for a static file comes in that needs to be protected:
<Location "/protected/location">
PythonPath "['/path/to/proj/'] + sys.path"
PythonOption DJANGO_SETTINGS_MODULE myproj.settings
PythonOption DjangoPermissionName '<permission.codename>'
PythonAccessHandler my_proj.modpython #this should point to accesshandler
SetHandler None
</Location>
Hope this helps, the snippet was posted a while ago, so things might have changed between Django versions :)
More efficient serving of static files through Django is being looked at currently as part of Google SOC project. For WSGI this will use wsgi.file_wrapper extensions for WSGI if available, as it is for mod_wsgi, and req.sendfile() if using mod_python. It will also support returning of headers such as 'Location', 'X-Accel-Redirect' and others, which different web hosting mechanisms and proxy front ends accept as a means of serving up static files where location is defined by a backend web application, which isn't as effecient as front end for serving static files.
I am not sure if there is a project page for this in Django wiki somewhere or not, but the code changes are being committed into the branches/soc2009/http-wsgi-improvements branch of Django source code repository.
You needn't strictly wait for that stuff. It is just putting a clean and portable interface in place across the different mechanisms. If using nginx as front end in front of Apache/mod_wsgi, you could use X-Accel-Redirect now. If using Apache/mod_wsgi 3.0 and daemon mode, you could use Location now, but do need to ensure you set up Apache correct. Alternatively, you could implement your own WSGI middleware wrapper around the Django application which looks for some response header of your own to indicate file to be returned and which uses wsgi.file_wrapper to return that instead of actual response returned from Django.
BTW, the authentication hook mechanisms listed for both mod_python and mod_wsgi by others would use HTTP basic authentication, which isn't what you wanted. This is presuming you want files to be protected by Django form based login mechanism using cookies and backend sessions.