I'm writing a web app for customer/order handling in python with asana integration.
For a registered incoming order, an invoice is created as .pdf. This file I want to send to asana as an email attachment using mandrill, because the asana python API doesn't provide attachments yet.
Because mandrill wants the content of the attachment as a base64-encoded string, I create a binary of the pdf using this function:
def binaryFile(self, pathToFile):
binary_obj = xmlrpclib.Binary( open(pathToFile).read() )
return binary_obj
Together with the path of the file, I throw this into mandrill like so:
'attachments': [{'content': binaryFile,
'name': pathOfFile,
'type': 'application/pdf'}]
When I try to send the whole thing, this is what I get:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site- packages/mandrill.py", line 1215, in send
return self.master.call('messages/send', _params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mandrill.py", line 131, in call
params = json.dumps(params)
OverflowError: Overlong 3 byte UTF-8 sequence detected when encoding string
Who can hint at what I am doing wrong?
Thank you.
All right ok, I found the mistake myself, it's in the base64-encoding of course. I'm now doing it like so:
import base64
def filetobase64(self, inputfilename):
return base64.b64encode(open(inputfilename, 'rb').read())
Everything works fine now.
No harm meant!
Related
I am receiving an http request from a desktop application with a screenshot. I cannot speak with the developer or see source code, so all I have is the http request I am getting.
The file isn't in request.FILES, it is in request.POST.
#csrf_exempt
def create_contract_event_handler(request, contract_id, event_type):
keyboard_events_count = request.POST.get('keyboard_events_count')
mouse_events_count = request.POST.get('mouse_events_count')
screenshot_file = request.POST.get('screenshot_file')
barr2 = bytes(screenshot_file.encode(encoding='utf8'))
with open('.test/output.jpeg', 'wb') as f:
f.write(barr2)
f.close()
The file is corrupted.
The binary starts like this, I don't know if that helps:
����JFIFHH��C
%# , #&')*)-0-(0%()(��C
(((((((((((((((((((((((((((((((((((((((((((((((((((�� `"��
Also, if I try to open the image with PIL, I get the following error:
from PIL import Image
im = Image.open('./test/output.jpg')
#OSError: cannot identify image file './test/output.jpg'
Finally, I managed to touch the code in the other hand, the 'filename' was missing in the header and for that reason I was getting the file in the POST instead of in the FILES dictionary.
I have a setup that lets users download files that are stored in the DB as BYTEA data. Everything works OK, except the download speed is very slow...it seems to download in 33KB chunks, one chunk per second.
Is there a setting I can specify to speed this up?
views.py
from django.http import FileResponse
def getFileResponse(filedata, filename, filesize, contenttype):
response = FileResponse(filedata, content_type=contenttype)
response['Content-Disposition'] = 'attachment; filename=%s' % filename
response['Content-Length'] = filesize
return response
return getFileResponse(
filedata = myfile.filedata, # Binary data from DB
filename = myfile.filename + myfile.fileextension,
filesize = myfile.filesize,
contenttype = myfile.filetype
)
Previously, I had the binary data returned as an HttpResponse and it downloaded like a normal file, with normal speeds. This worked fine locally, but when I pushed to Heroku, it wouldn't download the file -- instead displaying <Memory at XXX> in the download file.
And another side issue...when I include a text file with non-ASCII data (i.e. á), I get an error as well:
UnicodeEncodeError: 'ascii' codec can't encode characters...: ordinal not in range(128)
How can I handle files with Unicode data?
Update
Anyone know why the download speed gets so slow when changing from HTTPResponse to FileResponse? Or alternatively, why the HTTPResponse to return a file doesn't work on Heroku?
Update - Google Drive
I re-worked my application and hooked it up with a Google Drive back-end for serving files. It employs BytesIO() suggested by Eric below:
def download_file(self, fileid, mimetype=None):
# Get binary file data
request = self.get_file(fileid=fileid, mediaflag=True)
stream = io.BytesIO()
downloader = MediaIoBaseDownload(stream, request)
done = False
# Retry if we received HTTPError
for retry in range(0, 5):
try:
while done is False:
status, done = downloader.next_chunk()
print("Download %d%%." % int(status.progress() * 100))
return stream.getvalue()
except (HTTPError) as error:
return ('API error: {}. Try # {} failed.'.format(error.response, retry))
I think the difference you observe between HttpResponse vs. FileResponse is caused by the spec: https://www.python.org/dev/peps/pep-3333/#buffering-and-streaming
In your previous code, an HttpResponse was created with one huge byte string containing your whole file, and the first iteration pass returned the complete response body. With a a FileResponse, the file is iterated in chunks (of 4kb, 8kb or other depending on your WSGI app server), which (I think) are streamed immediately upstream (to the reverse proxy then client), which may add overhead (more communication over process boundaries?).
It would help to know the app server used (uwsgi, gunicorn, waitress, other) and its relevant config. Also more details about the heroku error in case that can be solved!
why you store whole file in database.
best case is to store file on hard and store only path on database
then according to your web server you can let web server to serve file.
web services serve file better than Django.
if files have no access check store them on media
if your files have access control you according to your web server you can use some response headers
if you use Nginx must use X-Accel-Redirect and use any alternative on other web services tutorial on https://wellfire.co/learn/nginx-django-x-accel-redirects/
I cannot find any example on how to attach files(pdf) that are within my root folder of the site in python (google app engine) send_mail function.
url_test = "https://mywebsite.com/pdf/test.pdf"
test_file = urlfetch.fetch(url_test)
if test_file.status_code == 200:
test_document = test_file.content
mail.send_mail(sender=EMAIL_SENDER,
to=['test#test.com'],
subject=subject,
body=theBody,
attachments=[("testing",test_document)])
Decided to try it with EmailMessage:
message = mail.EmailMessage( sender=EMAIL_SENDER,
subject=subject,body=theBody,to=['myemail#gmail.com'],attachments=
[(attachname, blob.archivoBlob)])
message.send()
The above blob attachment is successfully sending however attaching a file with relative path always says "invalid attachment"
new_file = open(os.path.dirname(__file__) +
'/../pages/pdf/test.PDF').read()
message = mail.EmailMessage( sender=EMAIL_SENDER,
subject=subject,body=theBody,to=['myemail#gmail.com'],attachments=
[('testing',new_file )])
message.send()
In debugging I have also tried to see if the file is being read by doing this:
logging.info(new_file)
It seems to be reading the file as it outputs some unicode characters
Please help why am I not able to attach a PDF while I can attach a blob
When calling the attachments, the File type has to be indicated on the file title, for example attachments= [('testing.pdf',new_file )]). View this link
Language: Python 2.7
Hello all. I found a really helpful script here: Python to Slack Web Hook
that shows how to send messages to a Slack web hook.
import json
import requests
# Set the webhook_url to the one provided by Slack when you create the webhook at https://my.slack.com/services/new/incoming-webhook/
webhook_url = 'https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX'
slack_data = {"text": "<https://alert-system.com/alerts/1234|Click here> for details!"}
response = requests.post(
webhook_url, data=json.dumps(slack_data),
headers={'Content-Type': 'application/json'}
)
if response.status_code != 200:
raise ValueError(
'Request to slack returned an error %s, the response is:\n%s'
% (response.status_code, response.text)
)
It works flawlessly when I run .py file.
Now, I have a file that has many lines of messages that I want to send to Slack. I have it formatted correctly already in the file, no spaces etc.. It's just a matter of grabbing it and passing it so slack_data = line1 etc..
So, I modify the file with something like this:
with open('export.txt', 'r') as e:
for line in e:
slack_data = line
Now if I do a print slack_data right after that, the information returns on the screen exactly as it should be, so I'm thinking it's good. I haven't began to get it working for each line yet, because it's not even working on the first line.
I get an invalid payload 400 when I run it.
EDIT: Slack support said the what they were receiving has escape characters inserted into for some reason.
"{\"text\": \"<https://alert-system.com/alerts/1234|Click here> for details!"}\n"
Any direction or assistance is appreciated.
Thanks!!
Just posting as it might help somebody. For me the below snippet worked:
data = json.dumps(slack_data)
response = requests.post(
URL, json={"text": data},
headers={'Content-Type': 'application/json'}
)
As #Geo pointed out the final payload that we are going to send should have keyword "text", else it will fail.
Moreover, in post method I have to replace data= with json= else it kept throwing error for invalid payload with 400
Since I already had the data preformatted in the file as JSON already, it was just a matter of removing json.dumps out of the code.
OLD:
#response = requests.post(webhook_url, data=json.dumps(slack_data), headers={'Content-Type': 'application/json'})
NEW:
response = requests.post(webhook_url, data=slack_data, headers={'Content-Type': 'application/json'})
Once I did that, everything worked like a charm.
If you change the code to this:
with open('export.txt', 'r') as e:
slack_data = e.read()
do you still get the 400?
I looked at the various questions similar to mine, but I could not find anything a fix for my problem.
In my code, I want to serve a freshly generated excel file residing in my app directory in a folder named files
excelFile = ExcelCreator.ExcelCreator("test")
excelFile.create()
response = HttpResponse(content_type='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
return response
So when I click on the button that run this part of the code, it sends to the user an empty file. By looking at my code, I can understand that behavior because I don't point to that file within my response...
I saw some people use the file wrapper (which I don't quite understand the use). So I did like that:
response = HttpResponse(FileWrapper(excelFile.file),content_type='application/vnd.ms-excel')
But then, I receive the error message from server : A server error occurred. Please contact the administrator.
Thanks for helping me in my Django quest, I'm getting better with all of your precious advices!
First, you need to understand how this works, you are getting an empty file because that is what you are doing, actually:
response = HttpResponse(content_type='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
HttpResponse receives as first arg the content of the response, take a look to its contructor:
def __init__(self, content='', mimetype=None, status=None, content_type=None):
so you need to create the response with the content that you wish, is this case, with the content of your .xls file.
You can use any method to do that, just be sure the content is there.
Here a sample:
import StringIO
output = StringIO.StringIO()
# read your content and put it in output var
out_content = output.getvalue()
output.close()
response = HttpResponse(out_content, mimetype='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
I would recommend you use:
python manage.py runserver
to run your application from the command line. From here you will see the console output of your application and any exceptions that are thrown as it runs. This may provide a quick resolution to your problem.