I've looked over the questions in SO and none can explain the proper usage of Django's FilePathField. The Django documentation about it is a little short. A web search does not yield good tutorials about it as well. To add do non uploaded files must be collected to the static directory,reside inside apps where they are used, or at the project level?
If you want to handle uploaded files consider using FileField instead of FilePathField. FileField also stores path to the uploaded file, but it is designed to handle new file creation for uploads. FilePathField is just used to point to a path in your file system.
Related
I'm creating a website where the user is able to upload large files. Around 100-300Mb. I'm using django but I want to know what is the best option to save the files. Should I just add them to my user database? Or should I create a media folder and save the files there. If so then this would mean that I have to save the name of the file in my user database?
Please let me know how you would tackle this issue.
django does not store files in the database, but in media folder. The FileField contains the path to the file(in the media folder). You can change that behavior but it is not recommended(as general practice). What you should consider however is that files of that size will require some more work both in django and the front-end since the upload process will freeze the server. A potential solution to the problem is this: https://pypi.org/project/django-chunked-upload/
I am providing sensitive username and password file to the authenticated user. I want user to download the file via file_url in template through model.
File_link = models.FileField(upload_to='SAFE_DIRECTORY_PATH')
I don't feel it safe storing it in media directory
Any suggestions keeping them safe ,web app will be generating the link.
Some security notes first.
This is probably a bad idea. Storing sensitive information in plain files is probably not the correct security approach, especially if you plan to use Django's media storage backend for doing that. It leaves all files out-in-the-open.
If however you really, really, and I mean really need to do that, you should encrypt the file first before saving in Django.
Again though, if at all possible I would recommend to store sensitive information in db. In your case of storing passwords, you can use Django techniques to store that information relatively-safely such as correctly hashing passwords via pbkdf function (e.g. pbkdf or bcrypt, etc). If users will need to download that information, you can always generate the file on the fly for them for download.
Some suggestions for uploading files.
I usually assign random filenames to the uploaded files. This way at least its more challenging for the users to guess the filenames to download them. Not very security since this relies on security by obfuscation but its better then nothing. If you need a Django field which does that automatically, you can do that by making upload_to a callable (there are also 3rd party libs for doing that such as django-auxilium although for full disclosure Im the author of that lib).
Now that files are stored with random filenames, you probably never want to provide direct download links to the users for download but instead authenticate them first and then use something like X-Accel in nginx or X-Sendfile in Apache to actually serve the file to the user. The idea being that you first authenticate user in Django. Then however instead of Django serving the file, you return a special header which nginx/apache catches which contains a filepath to the file nginx/apache should serve to the user. This way you dont have to waste resources in Django to serve the file however you still get the advantage of being able to authenticate the request. There are a number of 3rd party apps for doing that as well.
Finally to protect users from downloading the media files you can use nginx (and I imagine apache) by restricting certain parts of the media folder:
location /media/protected {
internal;
alias /var/www/files;
}
In this case nginx will refuse direct user requests to /media/protected and will only allow to serve those files via X-Accel-Redirect header sent by Django. Then all you have to configure in Django is to store files in that path to make them protected:
models.FileField(upload_to='protected/myfiles')
I was looking for a solution to serve files only to authorized users and came across this post. I think it it is top google result for "django storing and providing secure files"
As the answer is rather old I wanted to share my finding:
django-private-storage (https://pypi.org/project/django-private-storage/) seems to be a good solution to this problem.
My setup is: Django 1.3/Python 2.7.2/Win Server 2008 R2/IIS 7.5/MS SQL Server 2008 R2. I am developing an application whose main function is to analyze uploaded files and produce a report.
Reading over the documentation for django-filetransfers, I believe this is a solution to a problem I've been trying to solve for a while (i.e. form-based file uploads completely block all Django responses until the file-transfer finishes...horror for even moderate-sized files).
The documentation talks about piping uploads to S3 or Blobstore, and that might be what I end up doing eventually, but during development I thought maybe I could just set up my own "poor-man's S3" on a server that I control. This would basically just be another Django instance (or possibly a simple ASP.NET app) whose sole purpose is to receive uploaded files. This sounds like it should be possible with django-filetransfers and would solve the problem of Django responsiveness (???).
But I am missing some bits of understanding how this works in general, as well as some specifics. Maybe an example will help: let's say I have MyMainDjangoServer and MyFileUploadServer. MyMainDjangoServer will serve the views, including the upload form. MyFileUploadServer will "catch" the uploaded files. My questions/confusion are as follows:
My upload form will contain additional fields beyond just the file(s)...do I understand correctly that MyMainDjangoServer will somehow still get that form data, minus the file data (basically: request.POST), and the file data gets shunted over to MyFileUploadServer? How does this work? Will MyMainDjangoServer still block during the upload to MyFileUploadServer?
I assume that what I would need to do on MyFileUploadServer is have a view/URL that handles the form request and sucks out the request.FILES data. What else needs to happen? What happens to the rest of the form data?
How would I set up my settings.py for this scenario? The django-filetransfers examples seem to assume either S3 or GAE/Blobstore but maybe I am missing some basics.
Any advice/answers appreciated...this is a confusing and frustrating area of Django for me.
"MyMainDjangoServer will somehow still get that form data, minus the file data (basically: request.POST), and the file data gets shunted over to MyFileUploadServer? How does this work? Will MyMainDjangoServer still block during the upload to MyFileUploadServer?"
I know the GAE Blobstore, presumably S3 as well, handles this by requiring you to give it a success_url. In your case that would be the url on MyMainDjangoServer where your file receiving view on MyFileUploadServer would re-post the non-files form data to once the upload is complete.
Have a look at the create_upload_url method here: https://developers.google.com/appengine/docs/python/blobstore/functions
You need to recreate this functionality in some form (see below).
"How would I set up my settings.py for this scenario?"
You'd need to create your own filetransfers backend which would be a file with a prepare_upload function in it.
You can see the App Engine one here:
https://github.com/django-nonrel/djangoappengine/blob/develop/storage.py
The prepare_upload method just wraps the GAE create_upload_url method mentioned above.
So in your settings.py you'd have something like:
PREPARE_UPLOAD_BACKEND = 'myapp.filetransfers_backend.prepare_upload'
(i.e. the import path to your prepare_upload function)
For the rest you can start with the ones provided by filetransfers already:
SERVE_FILE_BACKEND = 'filetransfers.backends.url.serve_file'
# if you need it:
PUBLIC_DOWNLOAD_URL_BACKEND = 'filetransfers.backends.url.public_download_url'
These rely on the file_field.url being set (see Django docs) and since your files will be on a separate server you probably need to look into writing a custom storage backend for Django too. (the S3 and GAE cases assume you're using the custom Django storage backends from here)
I am developing two web sites using the Django framework.
The thing is - one site is sharing part of the content from the other one.
They both use different amazon WS buckets to store images, etc.
So for the site which shares some content with another one I need to specify a different MEDIA_URL, but it seems impossible cause 'upload_to' and 'storage' parameters of the FileField only influence the file being uploaded.
Is there any way to use another storage when displaying image after it was uploaded?
I am farily new to the topic, but I am trying to combine both Django and Pyjamas. What would be the smart way to combine the two? I am not asking about communication, but rather about the logical part.
Should I just put all the Pyjamas generated JS in the base of the domain, say http://www.mysite.com/something and setup Django on a subdirectory, or even subdomain, so all the JSON calls will go for http://something.mysite.com/something ?
As far as I understand now in such combination theres not much point to create views in Django?
Is there some solution for clean urls in Pyjamas, or that should be solved on some other level? How? Is it a standard way to pass some arguments as GET parameteres in a clean url while calling a Pyjamas generated JS?
You should take a look at the good Django With Pyjamas Howto.
I've managed to get the following to work, but it's not ideal. Full disclosure: I haven't figured out how to use the django's template system to get stuff into the pyjamas UI elements, and I have not confirmed that this setup works with django's authentication system. The only thing I've confirmed is that this gets the pyjamas-generated page to show up. Here's what I did.
Put the main .html file generated by pyjamas in django's "templates" directory and serve it from your project the way you'd serve any other template.
Put everything else in django's "static" files directory.
Make the following changes to the main .html file generated by pyjamas: in the head section find the meta element with name="pygwt:module" and change the content="..." attribute to content="/static/..." where "/static/" is the static page URL path you've configured in django; in the body section find the script element with src="bootstrap.js" and replace the attribute with src="/static/bootstrap.js".
You need to make these edits manually each time you regenerate the files with pyjamas. There appears to be no way to tell pyjamas to use a specific URL prefix when generating together its output. Oh well, pyjamas' coolness makes up for a lot.
acid, I'm not sure this is as much an answer as you would hope but I've been looking for the same answers as you have.
As far as I can see the most practical way to do it is with an Apache server serving Pyjamas output and Django being used as simply a service API for JSONrpc calls and such.
On a side note I am starting to wonder if Django is even the best option for this considering using it simply for this feature is not utilizing most of it's functionality.
The issue so far as I have found with using Django to serve Pyjamas output as Django Views/Templates is that Pyjamas loads as such
Main html page loads "bootstrap.js" and depending on the browser used bootstrap.js will load the appropriate app page. Even if you appropriately setup the static file links using the Django templating language to reference and load "bootstrap.js", I can't seem to do the same for bootstrap.js referencing each individual app page.
This leaves me sad since I do so love the "cruftless URLS" feature of Django.