I am new to Google App Engine. I have deploy the pure django application in google app engine. It is working fine. But I wrote the django file upload functionality as shown below,
def forhandler(list, project_id, task_id, request):
for i in list:
filename = i
extension = filename.content_type
print i
newdoc = FileUpload(project_id=project_id, task_id=task_id, file=filename, filetype=extension)
newdoc.save()
When I run the ./manage.py runserver. The above functionality working correctly and uploading files correctly. When I use google app engine means dev_appserver.py my_project. It is perfect but when I upload the file using above functionality It gives an error as shown below,
Exception Value: [Errno 30] Read-only file system: u'/home/nyros/Desktop/projectstat/projectstat/media/documents/2013/05/24/1354676051_chasm_fishing_w1.jpeg'
How do I upload the file using django with google app Engine ? Please solve my problem.Thanks......
The problem here is that App Engine uses a read-only filesystem, and the default Django file upload mechanism wants to store files on disk. (Regardless of whether you're using App Engine or not, storing images in an ordinary database is a bad idea.)
Instead, you should use AppEngine's Blobstore API to save the image. This is special storage App Engine provides for storing large data uploaded by users.
The good news is there's a plugin that takes care of all of this for you: http://www.allbuttonspressed.com/projects/django-filetransfers
Just follow the instructions there and you should be in business.
(Note: I've only tried this with django-nonrel. I've never tried with with vanilla Django.)
The best way to upload files in google app engine with python is using the blobstorehandler.
class Upload(blobstore_handlers.BlobstoreUploadHandler):
for upload in self.get_uploads():
try:
img = Image()
img.primary_image = upload.key()
img.put()
except:
pass
Related
I am trying to download zip file which contains multiple files in same or different formats.
This zip file get downloaded after clicking on a button "Download".
This functionality works perfectly on local development server. But after deploying the web app on Google cloud it throws
"NotImplementedError at /myfile/download/8"
This backend doesn't support absolute paths.
...
Cloud storage has respective path with the file still it is not working, why?
Everything is working fine on local machine, but fails on production, why?
Please help! Thanks in advance.
I think that calling the path() property of the FileSystemStorage class, or the url() property where the contents of the file referenced by name can be accessed and then using absolute URLs for downloading files from either of the two is what's flagging this error:
“For storage systems that aren’t accessible from the local filesystem,
this will raise NotImplementedError instead.”
You should try and avoid saving to absolute paths; there is a File Storage API which abstracts these types of operations for you. Looking at the documentation, it appears that the save() function supports passing a file-like object instead of a path.
Also, if you need to serve a zip file download in Django, you just need to serve it as attachment using Django's HttpResponse:
from django.http.response import HttpResponse
def zip_file_view(request):
response = HttpResponse(open('/path/to/your/zipfile.zip', 'rb'), content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=any_name_you_like.zip'
return response
I have a Django app running on a server. Currently user uploads are stored on the server filesystem.
I already know how to set up S3 storage. What is the best way to migrate existing uploads to S3 without breaking the API and having existing uploads still available?
These files are served to the front end in two ways:
Directly through a /media/path/to/upload endpoint:
/media/<path> django.views.static.serve
Through a viewset:
/api/v1/users/<user_pk>/items/<item_pk>/attachments/<pk>/ project.app.views.ItemAttachmentsViewSet
Does the following make sense:
Change the storage backend
Go through all model objects
Save each model object to get the files uploaded
Have /media/path go to a new view that will serve the files similar to how ItemAttachmentsViewSet does it.
? Or is there a better way?
The procedure outlined in the question was what I ended up doing, with the exception of step 4 which turned out to be unnecessary.
I have a Django 2.x with python 3.6 site in Google Cloud, the app is in app engine flex. (my first app :)
My app has an upload page, where I am asking the user upload a JSON file (that is never kept in the site), what I do is open it and generate another file from it
I know that django depending on the size of the file it goes into memory but I was never able to use this functionality, so what I did in local env, was creating a folder that I called, temp_reports, so I created the files here, uploaded them into a bucket and then deleted them, from temp_reports.
So I was thinking, as the site is already in gcloud, if I can directly create these files into the bucket? or do I still need to generate them in the site and then upload them?
Now if it is from my site I keep getting the following error:
Exception Value:
[Errno 2] No such file or directory: '/home/vmagent/app/temp_reports/file_516A3E1B80334372ADB440681BB5F030.xlsx
I had in my app.yaml
handlers:
- url: /temp_reports
static_dir: temp_reports
Is there something I am missing? in order to use temp_reports?
Or how can I create a file directly into my bucket?
You can certainly use the Storage Bucket without having to upload the file manually. This can be done by Google Cloud Storage client library (Preferred Method) . It allows you to store and retrieve data directly from the Storage Bucket. Secondly, you can use Cloud Storage API to do the same functionality but requires more efforts to set it up.
You want to use the upload_from_string method from google.cloud.storage.blob.Blob.
upload_from_string(data, content_type='text/plain', client=None,
predefined_acl=None)
So to create a text file directly on the bucket you could do this:
storage_client = storage.Client()
bucket = storage_client.get_bucket(‘mybucket’)
blob = bucket.blob(‘mytextfile.txt’)
blob.upload_from_string('Text file contents', content_type='text/plain')
For more information you can refer to the following page:
https://googleapis.github.io/google-cloud-python/latest/storage/blobs.html#google.cloud.storage.blob.Blob.upload_from_string
I am using Django 1.4 on Google App Engine.
I have a model called Media, where the admins can upload files to use in their website. It has a field:
file = models.FileField(upload_to='/uploads/%Y/%m/%d')
It works perfectly with images (although the URL provided is weird), but that is not a problem.
The problem is when they try to upload a PDF (or anything else). Everything seems to work, but when you go to edit it, it doesn't contain any file - there is no "Currently" or anything else.
If I go to Google App Engine dashboard, the file is in Blob Viewer, and the record is also saved and available through the Datastore Viewer, with the correct blob key.
Why Django is not recognizing it? And how I can fix it?
djangoappengine contains a storage provider for the blobstore.
To me it doesn't seem like a full-featured solution, just something to get file uploads running.
I had to add this to get some functionality I needed (urls to the files).
It might make the admin work too:
https://github.com/dragonx/djangoappengine/commit/6a472a80c25bc077816c9104b45d5a64f3063273
I've been trying to find a way to upload a large file to GAE's datastore using Django's admin interface, but haven't found an answer that specifically addresses this issue. I'm fairly new to Python/Django, so there might be an angle that I'm not looking at.
I've been looking at the django-filetransfers solution, but I'm not sure how to integrate that into the admin interface. Any suggestions would be great!
File uploads to the appengine are handled using the blobstore and not the datastore.
https://developers.google.com/appengine/docs/python/blobstore/overview#Complete_Sample_App
The datastore filesize limit is 1 megabyte. So, attempting to upload a large file into the appengine's non-relational database isn't going to work. This is by design.
The documentation I provided above shows you how to implement large file uploads.