I have an IICS taskflow with a mapping task and a notification task. The target of the mapping task is a csv stored in a server location.
With the notification task, I want to send an email with the csv attached. Do you know if this is possible or if is there another way to get to send the target csv by email??
Not the ideal solution, but you can write a simple python program to send the csv as an email, just provide the path of the csv to python program. and then, execute the python script from informatica as a command task. (IF Success) it is super easy to use a python script.
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.base import MIMEBase
from email import encoders
import os
import shutil
def send_mail(body_text, fromaddr, recipient_list, smtp_login, smtp_pass, file_path):
msg = MIMEMultipart()
msg['From'] = fromaddr
msg['To'] = ', '.join(recipient_list)
msg['Subject'] = 'your Subject variable'
msg.attach(MIMEText(body_text, 'plain'))
filename = os.path.basename(file_path)
attachment = open(file_path, "rb")
part = MIMEBase('multipart', 'mixed; name=%s' % filename)
part.set_payload(attachment.read())
encoders.encode_base64(part)
part.add_header('Content-Disposition', "attachment; filename= %s" % filename)
msg.attach(part)
server = smtplib.SMTP(host="your.mailserver.com", port=123)
# eg smtp.google.com
server.starttls()
server.login(smtp_login, smtp_pass)
text = msg.as_string()
server.set_debuglevel(1)
server.sendmail(fromaddr, recipient_list, text)
server.quit()
mainDir = '/path/to/file'
sendFileName = 'yourfilename' + '.csv'
sourceFilePath = mainDir + 'file_send/filename.csv'
body_text = '''
Dear All,
Please find attached document
Thank You.
'''
smtp_login = "emailusername"
smtp_pass = "emaipassword"
recipient_list = ['abc#company.com', 'def#company.com']
file_path = os.path.abspath(sendFilePath)
fromaddr = 'emailusername#company.com'
send_mail(body_text=body_text, fromaddr=fromaddr, recipient_list=recipient_list, smtp_login=smtp_login, smtp_pass=smtp_pass,
file_path=file_path)
This code may needs some modifications but thought it might help someone.
I'm trying to return a zip file with HttpResponse, using StringIO() because i'm not storing in DB or Harddrive.
My issue is that my response is returning 200 when i request the file, but the OS never ask me if i want to save the file, or the file is never saved. i think that the browser is reciving the file because i have seen on the Network Activity (inspect panel) and it says than a 6.4 MB file type zip is returned.
I'm taking a .step file (text file) from a DB's url, extracting the content, zipping and returning, that's all.
this my code:
def function(request, url_file = None):
#retrieving info
name_file = url_file.split('/')[-1]
file_content = urllib2.urlopen(url_file).read()
stream_content = StringIO(file_content)
upload_name = name_file.split('.')[0]
# Create a new stream and write to it
write_stream = StringIO()
zip_file = ZipFile(write_stream, "w")
try:
zip_file.writestr(name_file, stream_content.getvalue().encode('utf-8'))
except:
zip_file.writestr(name_file, stream_content.getvalue().encode('utf-8', 'ignore'))
zip_file.close()
response = HttpResponse(write_stream.getvalue(), mimetype="application/x-zip-compressed")
response['Content-Disposition'] = 'attachment; filename=%s.zip' % upload_name
response['Content-Language'] = 'en'
response['Content-Length'] = write_stream.tell()
return response
I am using pandas tool to create downloadable excel file.Excel will get it data from SQL table , which in turn will get populated depending on values entered by user.I have attached one download button in my web page to download the excel.
On clicking download button it's generating a blank excel file with sheet name and file name as Consistency Report. Can anyone point out what I am doing wrong here.
Thanks in advance
views.py
def export_excel(request):
response = HttpResponse(content_type="application/vnd.ms-excel")
response['Content-Disposition'] = 'attachment; filename=Consistency Report.xls'
fname = 'Consistency Report.xls'
cnxn = pyodbc.connect('DRIVER={SQL Server Native Client 10.0};SERVER=******;DATABASE=testing;UID=***;PWD=******')
cursor = cnxn.cursor()
cursor.execute("select * from dhm_consis_report_tbl")
columns = [column[0] for column in cursor.description]
data=cursor.fetchall()
cursor.commit()
print(columns)
for i in range(0,len(data)):
data[i]=tuple(data[i])
df = ps.DataFrame(data=data,columns=columns)
writer = ps.ExcelWriter('Consistency Report.xls',engine='xlwt')
df.to_excel(writer,sheet_name='Report')
writer.save()
return response
So, I edited my views.py and am able to generate and download excel file but the problem is that the file is now downloading at two locations; one at my project folder and one at my download folder.Also the file at download folder is empty while the one at my project folder contains data.Can anybody explain why this is happening?
Modified views.py
def export_excel(request):
response = HttpResponse(content_type="application/vnd.ms-excel;charset=utf-8")
response['Content-Disposition'] = 'attachment; filename=Consistency Report.xls'
df = ps.DataFrame.from_records(DHM_Consis_Report.objects.values('conquery_source','conquery_name','conquery_count','conquery_desc','criticality','sp_status','con_rundate','instance_id'))
print(df)
writer = ps.ExcelWriter('Consistency Report.xls',encoding='utf-8')
df.to_excel(writer,sheet_name='Report')
writer.save()
return response
Finally, I was able to generate required excel.Earlier I was creating and writing the excel with data but my HttpResponse was not getting the required data.I put the logic regarding generating excel in separate function and logic regarding downloading the generated file in separate function.This is what I did.
I know it's crude and inefficient but this works for me.If anyone has better way to do this please share.
views.py
#login_required
def consisreports(request):
cust= customername
username=None
if request.user.is_authenticated():
username=request.user.username
print(username)
table=ConsisReport(DHM_Consis_Report.objects.all())
RequestConfig(request,paginate={"per_page": 25}).configure(table)
todays=date.today()
todays=todays.strftime("%d-%m-%y")
filename="Consistency Report %s as on %s %s.xls"%(cust,str(todays),username)
colname=['Customer','Query ','Count','Desc','Criticality','Status','Rundate','Instance ID']
df = ps.DataFrame.from_records(DHM_Consis_Report.objects.values('conquery_source','conquery_name','conquery_count','conquery_desc','criticality','sp_status','con_rundate','instance_id'))
df=df[['conquery_source','conquery_name','conquery_count','conquery_desc','criticality','sp_status','con_rundate','instance_id']]
print(df)
#df.save('C:/Users/P1097/Desktop')
writer = ps.ExcelWriter(filename)
df.to_excel(writer,sheet_name='Report',index=False,engine='xlsxwriter',header=colname)
writer.save()
return render(request, 'consistency/consresult.html', {'table': table,'customername':cust})
#login_required
def export_excel(request):
custname=customername
username=None
if request.user.is_authenticated():
username=request.user.username
todays=date.today()
todays=todays.strftime("%d-%m-%y")
filename="Consistency Report %s as on %s %s.xls"%(custname,str(todays),username)
wrapper=open(filename,"rb")
cont=wrapper.read()
response = HttpResponse(cont,content_type="application/vnd.ms-excel;charset=utf-8")
response['Content-Length']=os.path.getsize(filename)
size=os.path.getsize(filename)
print(size)
wrapper.close()
response['Content-Disposition'] = 'attachment; filename= %s'%filename
return response
In my django app, I have a view which accomplishes file upload.The core snippet is like this
...
if (request.method == 'POST'):
if request.FILES.has_key('file'):
file = request.FILES['file']
with open(settings.destfolder+'/%s' % file.name, 'wb+') as dest:
for chunk in file.chunks():
dest.write(chunk)
I would like to unit test the view.I am planning to test the happy path as well as the fail path..ie,the case where the request.FILES has no key 'file' , case where request.FILES['file'] has None..
How do I set up the post data for the happy path?Can somebody tell me?
I used to do the same with open('some_file.txt') as fp: but then I needed images, videos and other real files in the repo and also I was testing a part of a Django core component that is well tested, so currently this is what I have been doing:
from django.core.files.uploadedfile import SimpleUploadedFile
def test_upload_video(self):
video = SimpleUploadedFile("file.mp4", "file_content", content_type="video/mp4")
self.client.post(reverse('app:some_view'), {'video': video})
# some important assertions ...
In Python 3.5+ you need to use bytes object instead of str. Change "file_content" to b"file_content"
It's been working fine, SimpleUploadedFile creates an InMemoryFile that behaves like a regular upload and you can pick the name, content and content type.
From Django docs on Client.post:
Submitting files is a special case. To POST a file, you need only
provide the file field name as a key, and a file handle to the file
you wish to upload as a value. For example:
c = Client()
with open('wishlist.doc') as fp:
c.post('/customers/wishes/', {'name': 'fred', 'attachment': fp})
I recommend you to take a look at Django RequestFactory. It's the best way to mock data provided in the request.
Said that, I found several flaws in your code.
"unit" testing means to test just one "unit" of functionality. So,
if you want to test that view you'd be testing the view, and the file
system, ergo, not really unit test. To make this point more clear. If
you run that test, and the view works fine, but you don't have
permissions to save that file, your test would fail because of that.
Other important thing is test speed. If you're doing something like
TDD the speed of execution of your tests is really important.
Accessing any I/O is not a good idea.
So, I recommend you to refactor your view to use a function like:
def upload_file_to_location(request, location=None): # Can use the default configured
And do some mocking on that. You can use Python Mock.
PS: You could also use Django Test Client But that would mean that you're adding another thing more to test, because that client make use of Sessions, middlewares, etc. Nothing similar to Unit Testing.
I do something like this for my own event related application but you should have more than enough code to get on with your own use case
import tempfile, csv, os
class UploadPaperTest(TestCase):
def generate_file(self):
try:
myfile = open('test.csv', 'wb')
wr = csv.writer(myfile)
wr.writerow(('Paper ID','Paper Title', 'Authors'))
wr.writerow(('1','Title1', 'Author1'))
wr.writerow(('2','Title2', 'Author2'))
wr.writerow(('3','Title3', 'Author3'))
finally:
myfile.close()
return myfile
def setUp(self):
self.user = create_fuser()
self.profile = ProfileFactory(user=self.user)
self.event = EventFactory()
self.client = Client()
self.module = ModuleFactory()
self.event_module = EventModule.objects.get_or_create(event=self.event,
module=self.module)[0]
add_to_admin(self.event, self.user)
def test_paper_upload(self):
response = self.client.login(username=self.user.email, password='foz')
self.assertTrue(response)
myfile = self.generate_file()
file_path = myfile.name
f = open(file_path, "r")
url = reverse('registration_upload_papers', args=[self.event.slug])
# post wrong data type
post_data = {'uploaded_file': i}
response = self.client.post(url, post_data)
self.assertContains(response, 'File type is not supported.')
post_data['uploaded_file'] = f
response = self.client.post(url, post_data)
import_file = SubmissionImportFile.objects.all()[0]
self.assertEqual(SubmissionImportFile.objects.all().count(), 1)
#self.assertEqual(import_file.uploaded_file.name, 'files/registration/{0}'.format(file_path))
os.remove(myfile.name)
file_path = import_file.uploaded_file.path
os.remove(file_path)
I did something like that :
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
from django.core.urlresolvers import reverse
from django.core.files import File
from django.utils.six import BytesIO
from .forms import UploadImageForm
from PIL import Image
from io import StringIO
def create_image(storage, filename, size=(100, 100), image_mode='RGB', image_format='PNG'):
"""
Generate a test image, returning the filename that it was saved as.
If ``storage`` is ``None``, the BytesIO containing the image data
will be passed instead.
"""
data = BytesIO()
Image.new(image_mode, size).save(data, image_format)
data.seek(0)
if not storage:
return data
image_file = ContentFile(data.read())
return storage.save(filename, image_file)
class UploadImageTests(TestCase):
def setUp(self):
super(UploadImageTests, self).setUp()
def test_valid_form(self):
'''
valid post data should redirect
The expected behavior is to show the image
'''
url = reverse('image')
avatar = create_image(None, 'avatar.png')
avatar_file = SimpleUploadedFile('front.png', avatar.getvalue())
data = {'image': avatar_file}
response = self.client.post(url, data, follow=True)
image_src = response.context.get('image_src')
self.assertEquals(response.status_code, 200)
self.assertTrue(image_src)
self.assertTemplateUsed('content_upload/result_image.html')
create_image function will create image so you don't need to give static path of image.
Note : You can update code as per you code.
This code for Python 3.6.
from rest_framework.test import force_authenticate
from rest_framework.test import APIRequestFactory
factory = APIRequestFactory()
user = User.objects.get(username='#####')
view = <your_view_name>.as_view()
with open('<file_name>.pdf', 'rb') as fp:
request=factory.post('<url_path>',{'file_name':fp})
force_authenticate(request, user)
response = view(request)
As mentioned in Django's official documentation:
Submitting files is a special case. To POST a file, you need only provide the file field name as a key, and a file handle to the file you wish to upload as a value. For example:
c = Client()
with open('wishlist.doc') as fp:
c.post('/customers/wishes/', {'name': 'fred', 'attachment': fp})
More Information: How to check if the file is passed as an argument to some function?
While testing, sometimes we want to make sure that the file is passed as an argument to some function.
e.g.
...
class AnyView(CreateView):
...
def post(self, request, *args, **kwargs):
attachment = request.FILES['attachment']
# pass the file as an argument
my_function(attachment)
...
In tests, use Python's mock something like this:
# Mock 'my_function' and then check the following:
response = do_a_post_request()
self.assertEqual(mock_my_function.call_count, 1)
self.assertEqual(
mock_my_function.call_args,
call(response.wsgi_request.FILES['attachment']),
)
if you want to add other data with file upload then follow the below method
file = open('path/to/file.txt', 'r', encoding='utf-8')
data = {
'file_name_to_receive_on_backend': file,
'param1': 1,
'param2': 2,
.
.
}
response = self.client.post("/url/to/view", data, format='multipart')`
The only file_name_to_receive_on_backend will be received as a file other params received normally as post paramas.
In Django 1.7 there's an issue with the TestCase wich can be resolved by using open(filepath, 'rb') but when using the test client we have no control over it. I think it's probably best to ensure file.read() returns always bytes.
source: https://code.djangoproject.com/ticket/23912, by KevinEtienne
Without rb option, a TypeError is raised:
TypeError: sequence item 4: expected bytes, bytearray, or an object with the buffer interface, str found
from django.test import Client
from requests import Response
client = Client()
with open(template_path, 'rb') as f:
file = SimpleUploadedFile('Name of the django file', f.read())
response: Response = client.post(url, format='multipart', data={'file': file})
Hope this helps.
Very handy solution with mock
from django.test import TestCase, override_settings
#use your own client request factory
from my_framework.test import APIClient
from django.core.files import File
import tempfile
from pathlib import Path
import mock
image_mock = mock.MagicMock(spec=File)
image_mock.name = 'image.png' # or smt else
class MyTest(TestCase):
# I assume we want to put this file in storage
# so to avoid putting garbage in our MEDIA_ROOT
# we're using temporary storage for test purposes
#override_settings(MEDIA_ROOT=Path(tempfile.gettempdir()))
def test_send_file(self):
client = APIClient()
client.post(
'/endpoint/'
{'file':image_mock},
format="multipart"
)
I am using Python==3.8.2 , Django==3.0.4, djangorestframework==3.11.0
I tried self.client.post but got a Resolver404 exception.
Following worked for me:
import requests
upload_url='www.some.com/oaisjdoasjd' # your url to upload
with open('/home/xyz/video1.webm', 'rb') as video_file:
# if it was a text file we would perhaps do
# file = video_file.read()
response_upload = requests.put(
upload_url,
data=video_file,
headers={'content-type': 'video/webm'}
)
I am using django rest framework and I had to test the upload of multiple files.
I finally get it by using format="multipart" in my APIClient.post request.
from rest_framework.test import APIClient
...
self.client = APIClient()
with open('./photo.jpg', 'rb') as fp:
resp = self.client.post('/upload/',
{'images': [fp]},
format="multipart")
I am using GraphQL, upload for test:
with open('test.jpg', 'rb') as fp:
response = self.client.execute(query, variables, data={'image': [fp]})
code in class mutation
#classmethod
def mutate(cls, root, info, **kwargs):
if image := info.context.FILES.get("image", None):
kwargs["image"] = image
TestingMainModel.objects.get_or_create(
id=kwargs["id"],
defaults=kwargs
)
I've a got a django app that currently generates pdfs using a canvas that the user can download. I create a StringIO buffer, do some stuff and then send call response.write.
# Set up response
response = HttpResponse(mimetype='application/pdf')
response['Content-Disposition'] = 'attachment; filename=menu-%s.pdf' % str(menu_id)
# buffer
buff = StringIO()
# Create the pdf object
p = canvas.Canvas(buff)
# Add some elements... then
p.showPage()
p.save()
# Get the pdf from the buffer and return the response
pdf = buff.getvalue()
buff.close()
response.write(pdf)
I now want to build my pdf using platypus and SimpleDocTemplate and have written this
# Set up response
response = HttpResponse(mimetype='application/pdf')
pdf_name = "menu-%s.pdf" % str(menu_id)
response['Content-Disposition'] = 'attachment; filename=%s' % pdf_name
menu_pdf = SimpleDocTemplate(pdf_name, rightMargin=72,
leftMargin=72, topMargin=72, bottomMargin=18)
# container for pdf elements
elements = []
styles=getSampleStyleSheet()
styles.add(ParagraphStyle(name='centered', alignment=TA_CENTER))
# Add the content as before then...
menu_pdf.build(elements)
response.write(menu_pdf)
return response
But this doesn't work, it creates a bad pdf that cannot be opened. I presume the line
response.write(menu_pdf)
is incorrect.
How do I render the pdf?
Your error is actually a pretty simple one. It's just a matter of trying to write the wrong thing. In your code, menu_pdf is not a PDF, but a SimpleDocTemplate, and the PDF has been stored in pdf_name, although here I suspect pdf_name is a path name rather than a file object. To fix it, change your code to use a memory file like you did in your original code:
# Set up response
response = HttpResponse(mimetype='application/pdf')
pdf_name = "menu-%s.pdf" % str(menu_id)
response['Content-Disposition'] = 'attachment; filename=%s' % pdf_name
buff = StringIO()
menu_pdf = SimpleDocTemplate(buff, rightMargin=72,
leftMargin=72, topMargin=72, bottomMargin=18)
# container for pdf elements
elements = []
styles=getSampleStyleSheet()
styles.add(ParagraphStyle(name='centered', alignment=TA_CENTER))
# Add the content as before then...
menu_pdf.build(elements)
response.write(buff.getvalue())
buff.close()
return response
I'm not sure if using file objects rather than paths with Platypus is mentioned in the documentation, but if you dig into the code you'll see that it is possible.
For people who are working with python3 and django 1.7+ some changes to the answer need to be done.
from django.shortcuts import HttpResponse
import io
from reportlab.platypus import SimpleDocTemplate, BaseDocTemplate
def view(request):
buffer = io.BytesIO()
doc = # ... create your SimpleDocTemplate / BaseDocTemplate
# create the usual story
story = []
# ...
doc.build(story)
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment; filename=your_name.pdf'
response.write(buffer.getvalue())
buffer.close()
return response