I am creating a site that downloads videos from other sites and converts them to GIF when requested.
The problem is that it takes too long to download and convert videos.
This causes a 504 timeout error.
How can I avoid timeout?
Currently, I am downloading using celery when we receive a request.
While downloading, django redirects right away.
def post(self, request):
form = URLform(request.POST)
ctx = {'form':form}
....
t = downloand_video.delay(data)
return redirect('gif_to_mp4:home')
This prevents transferring the files to the user.
Because celery cannot return file to user or response.
How can I send the file to the user?
Related
I have a Django app where users can upload PDF files. The PDF files will be saved on my cloud provider. After successfully submitting the PDF, I want to send an email to the user with the URL to the PDF on my cloud. I've been trying to do it by overriding form_valid() but at that point, the URL is not yet generated. The URL also isn't hardcoded, so I can't just point to a hard coded URL in form_valid()
Any ideas on how to solve this?
Can you please provide us the code within form_valid() ? You need to insert your logic after super.form_valid() e.g.:
def form_valid(self, form):
ret = super().form_valid(form)
instance = form.instance
# get the filename and send an email
...
return ret
I have two requests, which are called from react front end, one request is running in a loop which is returning image per request, now the other request is registering a user, both are working perfect, but when the images request is running in a loop, at the same time I register user from other tab,but that request status shows pending, if I stops the images request then user registered,How can I run them parallel at the same time.
urls.py
url(r'^create_user/$', views.CreateUser.as_view(), name='CreateAbout'),
url(r'^process_image/$', views.AcceptImages.as_view(), name='AcceptImage'),
Views.py
class CreateUser(APIView):
def get(self,request):
return Response([UserSerializer(dat).data for dat in User.objects.all()])
def post(self,request):
payload=request.data
serializer = UserSerializer(data=payload)
if serializer.is_valid():
instance = serializer.save()
instance.set_password(instance.password)
instance.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
class AcceptImages(APIView):
def post(self,request):
global video
stat, img = video.read()
frame = img
retval, buffer_img = cv2.imencode('.jpg', frame)
resdata = base64.b64encode(buffer_img)
return Response(resdata)
These endpoints I am calling from react,the second endpoint is being calling in a loop and the same time from other tab I register user but it shows status pending and If I stop the image endpoint it then register the user,how can I make these two request run parallel.
I have researched a lot but can't find appropriate solution, there is one solution I using celery, but did not whether it solves my problem if it solves how can I implement above scenerio
You should first determine whether the bottleneck is the frontend or the backend.
frontend: Chrome can make up to 6 requests for the same domain at a time. (Up to HTTP/1.1)
backend: If you use python manage.py runserver, consider using gunicorn or uWSGI. As the Django documentation says, the manage.py command should not be used in production. Modify the process and thread count settings in the gunicorn or uWSGI to 2 or higher and try again.
I have a django app that serves images when a certain page is loaded. The images are stored on S3 and I retrieve them using boto and send the image content as an HttpResponse with the appropriate content type.
The problem here is, this is a blocking call. Sometimes, it takes a long time (few secs for the image of few hundred KBs) to retrieve the images and serve them to the client.
I tried doing converting this process to a celery task (async, non-blocking), but I am not sure how I can send back the data (images) when they are done downloading. Just returning HttpResponse from a celery task does not work. I found docs related to http callback tasks in an old celery docs here, but this is not supported in the newer celery versions.
So, should I use polling in the js? (I have used celery tasks in other parts of my website, but all of them are socket based) or is this even the right way to approach the problem?
Code:
Django views code that fetches the images (from S3 using boto3): (in views.py)
#csrf_protect
#ensure_csrf_cookie
def getimg(request, public_hash):
if request.user.is_authenticated:
query = img_table.objects.filter(public_hash=public_hash)
else:
query = img_table.objects.filter(public_hash=public_hash, is_public=True)
if query.exists():
item_dir = construct_s3_path(s3_map_thumbnail_folder, public_map_hash)
if check(s3, s3_bucket, item_dir): #checks if file exists
item = s3.Object(s3_bucket, item_dir)
item_content = item.get()['Body'].read()
return HttpResponse(item_content, content_type="image/png",status=200)
else: #if no image found, return a blank image
blank = Image.new('RGB', (1000,1000), (255,255,255))
response = HttpResponse(content_type="image/jpeg")
blank.save(response, "JPEG")
return response
else: #if request image corresponding to hash is not found in db
return render(request, 'core/404.html')
I call the above django view in a page like this:
<img src='/api/getimg/123abc' alt='img'>
In urls.py I have:
url(r'^api/getimg/(?P<public_hash>[a-zA-Z0-9]{6})$', views.getimg, name='getimg')
I'm currently writing a small personal-use website served with nginx, utilizing uwsgi, django, and bootstrap. All is going smoothly except for one issue that I cannot figure out. I have download links (buttons) on the site, and when clicked, should initiate a file download. Here is the view that is executed when a button is pressed:
#login_required
def download_file(request):
'''
downloads a file given a specified UUID
'''
if request.method == 'GET':
file_uuid = request.GET['file_id']
file_row = Files.objects.get(uuid=file_uuid)
file_name = file_row.file_name
response = HttpResponse()
response['Content-Disposition'] = 'attachment; filename=%s' % file_name
response['X-Accel-Redirect'] = '/media/files/%s' % file_name
return response
else:
return redirect('/files')
/media/files is served directly by nginx as a internal location:
location /media/files/ {
internal;
alias /mnt/files/;
}
How this view is being called with an onclick event assigned to each button:
$('.download_btn').on('click',function(){
download_file(this.id);
})
function download_file(uuid){
$('.file_id').val(uuid);
$('.get_file').submit();
}
I have a form with a single hidden field. This gets set to the id (uuid) of the button that is pressed.
Pretty simple right? My issue is that when the download button is pressed, the download is not initiated correctly. The user is not prompted with a save dialog, nor does the file begin automatically downloading (Chrome or Safari). Instead, in debug tools, I can see the file downloading to what seems to be local storage in the browser, or some memory location (these are large files; > 1GB). I see the memory ballooning, and eventually the browser will crash. Any clue what I'm doing wrong here? Based on what I've been reading, this should be working without issue.
I have a view to which I am trying to submit multiple ajax uploads via raw post data (e.g. via an octet-stream). These requests are submitted one after the other so that they process in parallel. The problem is that django thinks that only the last request is valid. For example, if I submit 5 files, the first four give:
Upload a valid image. The file you uploaded was either not an image or a corrupted image.
I'm guessing this occurs because somehow the requests overlap? And so the image isn't completely loaded before the form attempts to validate it?
And the last one works fine.
My upload view:
def upload(request):
form = UploadImageForm(request.POST, request.FILES)
print form
if form.is_valid():
# ..process image..
And my upload image form:
class UploadImageForm(forms.Form):
upload = forms.ImageField()
To submit the requests I'm using the html5uploader js pretty much right out of the box.
On a different not, have you tried https://github.com/blueimp/jQuery-File-Upload/ - is a pretty good non-flash based file uploader with progress bar.