how to stream file to client in django - django

I want to know how can I stream data to client using django.
The Goal
The user submits a form, the form data is passed to a web service which returns a string. The string is tarballed (tar.gz) and the tarball is sent back to the user.
I don't know what's the way. I searched and I found this, but I just have a string and I don't know if it is the thing I want, I don't know what to use in place of filename = __file__ , because I don't have file - just a string. If I create a new file for each user, this won't be a good way. so please help me. (sorry I'm new in web programming).
EDIT:
$('#sendButton').click(function(e) {
e.preventDefault();
var temp = $("#mainForm").serialize();
$.ajax({
type: "POST",
data: temp,
url: 'main/',
success: function(data) {
$("#mainDiv").html(data.form);
????
}
});
});
I want to use ajax, so what should i do in success of ajac function and in return of view. really thanks.
my view.py:
def idsBackup(request):
if request.is_ajax():
if request.method == 'POST':
result = ""
form = mainForm(request.POST)
if form.is_valid():
form = mainForm(request.POST)
//do form processing and call web service
string_to_return = webserviceString._result
???
to_json = {}
to_json['form'] = render_to_string('main.html', {'form': form}, context_instance=RequestContext(request))
to_json['result'] = result
???return HttpResponse(json.dumps(to_json), mimetype='application/json')
else:
form = mainForm()
return render_to_response('main.html', RequestContext(request, {'form':form}))
else:
return render_to_response("ajax.html", {}, context_instance=RequestContext(request))

You can create a django file instance of ContentFile using a string content instead of actual file and then send it as a response.
Sample code:
from django.core.files.base import ContentFile
def your_view(request):
#your view code
string_to_return = get_the_string() # get the string you want to return.
file_to_send = ContentFile(string_to_return)
response = HttpResponse(file_to_send,'application/x-gzip')
response['Content-Length'] = file_to_send.size
response['Content-Disposition'] = 'attachment; filename="somefile.tar.gz"'
return response

You can modify send_zipfile from the snippet to suit your needs. Just use StringIO to turn your string into a file-like object which can be passed to FileWrapper.
import StringIO, tempfile, zipfile
...
# get your string from the webservice
string = webservice.get_response()
...
temp = tempfile.TemporaryFile()
# this creates a zip, not a tarball
archive = zipfile.ZipFile(temp, 'w', zipfile.ZIP_DEFLATED)
# this converts your string into a filelike object
fstring = StringIO.StringIO(string)
# writes the "file" to the zip archive
archive.write(fstring)
archive.close()
wrapper = FileWrapper(temp)
response = HttpResponse(wrapper, content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=test.zip'
response['Content-Length'] = temp.tell()
temp.seek(0)
return response

Related

Data in request.body can't be found by request.data - Django Rest Framework

I'm writing a django application. I am trying to call my django rest framework from outside, and expecting an answer.
I use requests to send some data to a function in the DRF like this:
j=[i.json() for i in AttachmentType.objects.annotate(text_len=Length('terms')).filter(text_len__gt=1)]
j = json.dumps(j)
url = settings.WEBSERVICE_URL + '/api/v1/inference'
headers = {
'Content-Disposition': f'attachment; filename={file_name}',
'callback': 'http://localhost',
'type':j,
'x-api-key': settings.WEBSERVICE_API_KEY
}
data = {
'type':j
}
files = {
'file':file
}
response = requests.post(
url,
headers=headers,
files=files,
json=data,
)
In the DRF, i use the request object to get the data.
class InferenceView(APIView):
"""
From a pdf file, extract infos and return it
"""
permission_classes = [HasAPIKey]
def post(self, request):
print("REQUEST FILE",request.FILES)
print("REQUEST DATA",request.data)
callback = request.headers.get('callback', None)
# check correctness of callback
msg, ok = check_callback(callback)
if not ok: # if not ok return bad request
return build_json_response(msg, 400)
# get zip file
zip_file = request.FILES.get('file', None)
parsed = json.loads(request.data.get('type', None).replace("'","\""))
The problem is that the data in the DRF are not received correctly. Whatever I send from the requests.post is not received.
I am sending a file and a JSON together. The file somehow is received, but other data are not.
If I try to do something like
request.data.update({"type":j})
in the DRF, the JSON is correctly added to the data, so it is not a problem with the JSON I'm trying to send itself.
Another thing, request.body shows that the JSON is somehow present in the body, but request.data can't find it.
I don't want to use request.body directly because I can't understand why it is present in the body but not visible with request.data.
In this line
response = requests.post(
url,
headers=headers,
files=files,
json=data,
)
replace json=data with data=data
like this:
response = requests.post(
url,
headers=headers,
files=files,
data=data,
)

How to serve a PDF file created by pdfkit.from_string() with Flask send_file()

I have a pdfkit PDF that is working fine as a Sendgrid attachment, created by the following function:
def wish_lists_pdf(user=current_user):
pdf_heading = "Thank you!"
pdf_subheading = "Please find the Wish Lists you signed up to sponsor listed below."
pdf_context = {
'heading': pdf_heading,
'subheading': pdf_subheading,
'user': user,
}
css = os.path.join(basedir, 'static/main.css')
pdf_content = render_template(
'partials/email_lists_pdf.html', **pdf_context)
path_wkhtmltopdf = app.config['WKHTMLTOPDF_EXE']
config = pdfkit.configuration(wkhtmltopdf=path_wkhtmltopdf)
pdf_file = pdfkit.from_string(
pdf_content, False, configuration=config, css=css)
bytes_file = BytesIO(pdf_file)
return bytes_file
Actually, sendgrid needs this line instead of the bytes encoding:
encoded_file = base64.b64encode(pdf_attachment).decode()
I tried it with this encoding and the b64 encoding as different tutorials have suggested. I don't really understand the purpose of the encoding, so that may be some of the cause for my error. In any case, here is a route that I want to serve the PDF file:
#bp.route('download_lists_pdf/<int:user_id>', methods=['GET'])
def download_lists_pdf(user_id):
user = User.query.filter_by(id=user_id).first()
pdf_file = wish_lists_pdf(user=user)
return send_file(
pdf_file,
as_attachment=True,
attachment_filename="Wish List Reminder Page.pdf",
mimetype='application/pdf',
)
This downloads a completely blank, 0kb PDF file. Can someone help me understand how to use send_file() in a way that will allow me to serve this PDF from pdfkit? Again, as a Sendgrid attachment the file works fine.
Here's the sendgrid attachment config if that's helpful...
context = {
'heading': heading,
'subheading': subheading,
'user': user,
}
message = Mail(
from_email=app.config['ADMIN_EMAIL'],
to_emails=app.config['EMAIL_RECIPIENTS'],
subject=email_subject,
html_content=render_template('partials/email_lists.html', **context),
)
encoded_file = base64.b64encode(pdf_attachment).decode()
attached_file = Attachment(
FileContent(encoded_file),
FileName('Wish List Reminder Page.pdf'),
FileType('application/pdf'),
Disposition('attachment')
)
message.attachment = attached_file
sg = SendGridAPIClient(app.config['SENDGRID_API_KEY'])
response = sg.send(message)
Thank you in advance for your assistance.
Edit: tried below and it didn't work
bytes_file = BytesIO(pdf_file)
return bytes(bytes_file), 200, {
'Content-Type': 'application/pdf',
'Content-Disposition': 'inline; filename="Wish List reminder sheet.pdf"'}
If you have the PDF as bytes_file,
return bytes(byte_file), 200, {
'Content-Type': 'application/pdf',
'Content-Disposition': 'inline; filename="nameofyourchoice.pdf"'}
should do the trick.
If you are looking to return PDF file as a response then you can try this-
pdf = BytesIO(pdf_attachment)
response = pdf.getvalue()
pdf.close()
return response, 200, {
'Content-Type': 'application/pdf',
'Content-Disposition': 'inline; filename="name_of_file.pdf"'}
I solved this by creating a blank template.pdf file in my templates directory then did this in the from_string function
pdfkit.from_string(
pdf_content, "templates/template.pdf", configuration=config, css=css
)
i.e pass the path to your blank template.pdf as the output instead of False
You will then invoke send_file like so:
return send_file(
'templates/template.pdf',
mimetype="pdf",
download_name="Wish List Reminder Page.pdf",
as_attachment=True,
)
These lines were useful for me, to redirect to a new browser tab with the pdf view. Thanks!
return bytes(byte_file), 200, {
'Content-Type': 'application/pdf',
'Content-Disposition': 'inline; filename="nameofyourchoice.pdf"'
}
I found a solution. It uses the PDF file returned directly from pdfkit and then uses a Flask Response for serving the file.
Here is the function that returns the PDF file:
def wish_lists_pdf(user=current_user):
pdf_heading = "Thank you!"
pdf_subheading = "Please find the Wish Lists you signed up to sponsor listed below."
pdf_context = {
'heading': pdf_heading,
'subheading': pdf_subheading,
'user': user,
}
css = os.path.join(basedir, 'static/main.css')
pdf_content = render_template(
'partials/email_lists_pdf.html', **pdf_context)
path_wkhtmltopdf = app.config['WKHTMLTOPDF_EXE']
config = pdfkit.configuration(wkhtmltopdf=path_wkhtmltopdf)
pdf_file = pdfkit.from_string(
pdf_content, False, configuration=config, css=css)
return pdf_file
And here is the view:
#bp.route('download_lists_pdf/<int:user_id>', methods=['GET'])
def download_lists_pdf(user_id):
user = User.query.filter_by(id=user_id).first()
pdf_file = wish_lists_pdf(user=user)
response = Response(pdf_file)
response.headers['Content-Disposition'] = "inline; 'Wish List reminder page'"
response.mimetype = 'application/pdf'
return response
I think it would be more ideal if it would open in a new tab, or if it downloaded without opening in the browser, but this is good enough for now.

Generate multiple PDFs and zip them for download, all in a single view

I am using xhtml2pdf to generate PDFs in my Django View. The idea is to loop over all the instances that are there in the query, then for each instance create a PDF, then add all the generated PDFs to one zip File for download. The xtml2pdf logic is working okay but the looping logic is what gives me headache.
So this is my function so far:
def bulk_cover_letter(request, ward_id, school_cat_id, cheque_number):
school_type = SchoolType.objects.get(id=school_cat_id)
schools_in_school_type = Applicant.objects.filter(
school_type=school_type, ward_id=ward_id, award_status='awarded'
).order_by().values_list('school_name', flat=True).distinct()
for school in schools_in_school_type:
beneficiaries = Applicant.objects.filter(school_type=school_type, ward_id=ward_id, award_status='awarded', school_name=school)
total_amount_to_beneficiaries = Applicant.objects.filter(school_type=school_type, ward_id=ward_id, award_status='awarded', school_name=school).aggregate(total=Sum('school_type__amount_allocated'))
context = {
'school_name' : school,
'beneficiaries' : beneficiaries,
'total_amount_to_beneficiaries' : total_amount_to_beneficiaries,
'title' : school + ' Disbursement Details',
'cheque_number': cheque_number
}
response = HttpResponse('<title>Cover Letter</title>', content_type='application/pdf')
filename = "%s.pdf" %(cheque_number)
content = "inline; filename=%s" %(filename)
response['Content-Disposition'] = content
template = get_template('cover_letter.html')
html = template.render(context)
result = io.BytesIO()
pdf = pisa.CreatePDF(
html, dest=response, link_callback=link_callback)
if not pdf.error:
# At this point I can generate a single PDF.
# But no idea on what to do next.
# The zipping logic should follow here after looping all the instances - (schools)
From that Point I have no idea on what to do next. Any help will be highly appreciated.
Try this:
Utils.py
def render_to_pdf(template_src, context_dict={}):
template = get_template(template_src)
html = template.render(context_dict)
buffer = BytesIO()
p = pisa.pisaDocument(BytesIO(html.encode("ISO-8859-1")), buffer)
pdf = buffer.getvalue()
buffer.close()
if not p.err:
return pdf#HttpResponse(result.getvalue(), content_type='application/pdf')
return None
def generate_zip(files):
mem_zip = BytesIO()
with zipfile.ZipFile(mem_zip, mode="w",compression=zipfile.ZIP_DEFLATED) as zf:
for f in files:
zf.writestr(f[0], f[1])
return mem_zip.getvalue()
Views.py
def generate_attendance_pdf(modeladmin, request, queryset):
template_path = 'student/pdf_template.html'
files = []
for q in queryset:
context = {
'firstname': q.firstname,
'lastname': q.lastname,
'p_firstname': q.bceID.firstname
}
pdf = render_to_pdf(template_path, context)
files.append((q.firstname + ".pdf", pdf))
full_zip_in_memory = generate_zip(files)
response = HttpResponse(full_zip_in_memory, content_type='application/force-download')
response['Content-Disposition'] = 'attachment; filename="{}"'.format('attendnace.zip')
return response
Obviously, you have to modify the context/names to what you need.
Credit to -> Neil Grogan https://www.neilgrogan.com/py-bin-zip/
If you need to generate several PDF files and send them as a response in a zip file then you can store the reports in memory and set it as dest when you call pisa.CreatePDF. Then have a list of reports in memory, zip them, and send as a Django response specifying another content type.
For example:
reports = tempfile.TemporaryDirectory()
report_files = {}
for school in schools_in_school_type:
# ... same code that renerates `html`
mem_fp = BytesIO()
pisa.CreatePDF(html, dest=mem_fp)
report_files[filename] = mem_fp
mem_zip = BytesIO()
with zipfile.ZipFile(mem_zip, mode="w") as zf:
for filename, content in report_files.items():
zf.write(filename, content)
response = HttpResponse(mem_zip, content_type='application/force-download')
response['Content-Disposition'] = 'attachment; filename="{}"'.format('cover_letters.zip')
This still generates an error of [Errno 2] No such file or directory: 'cheque_number.pdf'.

Django : Calling a Url with get method

I have the url and corresponding view as follows.
url(r'^(?P<token>.*?)/ack$', views.api_ACK, name='device-api_ack')
def api_ACK(request, token):
"""
Process the ACK request comming from the device
"""
logger.info('-> api_ACK', extra={'request': request, 'token' : token, 'url': request.get_full_path()})
logger.debug(request)
if request.method == 'GET':
# verify the request
action, err_msg = api_verify_request(token=token, action_code=Action.AC_ACKNOWLEDGE)
return api_send_answer(action, err_msg)
I want to call api_ACK function with request method as GET from another view api_send_answer
I am creating one url in /device/LEAB86JFOZ6R7W4F69CBIMVBYB9SFZVC/ack in api_send_answer view as follows..
def api_send_answer(action, err_msg, provisional_answer=None):
last_action = create_action(session,action=Action.AC_ACKNOWLEDGE,token=last_action.next_token,timer=500)
url = ''.join (['/device/',last_action.next_token ,'/',Action.AC_ACKNOWLEDGE])
logger.debug('Request Url')
logger.debug(url)
response = api_ACK(request=url,token=last_action.next_token) # This is wrong
Now from api_send_answer it is redirecting to api_ACK view, but how to call api_ACK with request method as GET?
Please help..Any suggestions would be helpful to me
This line
response = api_ACK(request=url,token=last_action.next_token) is wrong because view expects HttpRequest object and you give him url instead.
if you need to return view response to user, you can use redirect:
def api_send_answer(action, err_msg, provisional_answer=None):
last_action = create_action(session,action=Action.AC_ACKNOWLEDGE,token=last_action.next_token,timer=500)
url = ''.join (['/device/',last_action.next_token ,'/',Action.AC_ACKNOWLEDGE])
logger.debug('Request Url')
logger.debug(url)
return HttpResponseRedirect(url)
if you need to do something else with view response you have to use HttpRequest object not url as parameter.

Is it possible to return an HttpResponse in django with text & a json object?

In my view function, I'd like to return a json object (data1) and some text/html (form). Is this possible?
Here is part of my views.py:
if request.is_ajax() and request.method == 'POST':
...
if form.is_valid():
answer = form.cleaned_data['answer'] # Answer extracted from form is also a string
a1 = ques1.correct_answer
if a1 == answer:
test1 = question_list.get(id=nextid)
form = AnswerForm(test1)
ques1 = question_list.filter(id=nextid) # Filter next question as <qs>
data1 = serializers.serialize("json",ques1) # Json-ize
# ********EDITED HERE **********
variables1 = Context({
'form' : form,
'q1' : data1,
})
#response = HttpResponse()
#response['data1'] = response.write(data1)
#response['form'] = response.write(form)
if nextid <= qsnlen:
return HttpResponse(variables1, mimetype="application/json")
#return HttpResponse(response)
else:
...
I'd like to send back both the form html and the ques1 json object. How can I do this? Thanks in advance.
Just put both pieces of data in a JSON container, one key with the form data and one with the HTML as a rendered string. In the browser, you can just pull both keys out & do your thing.
In your view:
form_json_data = get_form_json_data()
rendered_html = get_the_html()
return HttpResponse(json.dumps({
"formdata": form_json,
"html": rendered_html}),
content_type="application/json")
In js:
$.post(foo, postdata, function(data){
var formdata = data.formdata
var html = data.html;
$(".html-target").replaceWith(html);
do_whatever(formdata);
})
Use JsonResponse
from django.http import JsonResponse
response_data = {put your data into a dict}
return JsonResponse(response_data, status=201)
To do this with one response; you need to send the JSON as a plain text in the context of your template response (HTML).
If you need to send JSON as as a separate JSON object, with its own mime type, then you need to write two views; one that sends back the JSON as application/json and the other that sends back the form (HTML).
EDIT:
You are not returning JSON objects, but you are turning a dictionary that has two items of two different types. As I explained in the comments, in one request/response cycle; you can only return one response which has a specific mime type that is based on the content and how you want the browser to handle it. Most of the time the content type is 'text/html'.
In your scenario, if you want to return both the HTML (which is your form), and the JSON response (which is a string), you need to return HTML.
If you want to return JSON to Jquery as a JSON object; you need to detect the request type. In your front end (the templates), you will initiate two requests - one from the browser, which will return back the form. The other from jQuery, which will return the appropriate JSON object.
Here is a possible approach to this:
def foo(request):
if request.is_ajax():
ctx = dict()
ctx['hello'] = 'world'
return HttpResponse(json.dumps(ctx),content_type='application/json')
else:
return HttpResponse('hello world')