I'm quite new to using Django. As first project I wrote a little tool to create M3U8-Playlists.
My problem is, that long playlists are not transferred completely.
This is the stripped-down code:
def create(request):
# just a dummy filelist
playlist = ["#EXTM3U"] + 5 * ["/home/pi/music/" + 5 * "äöü0123456789á/" + "xyz.mp3"]
file_to_send = ContentFile("")
for item in playlist:
file_to_send.write("{}\n".format(item.replace("/home/pi/Music", r"\\raspberry\music").replace("/", "\\")))
response = HttpResponse(file_to_send, "audio/x-mpegurl")
response["Content-Length"] = file_to_send.size
response["Content-Disposition"] = f"attachment; filename=\"playlist.m3u8\""
# print some debug info
print("lines:", len(playlist), "chars (no linebreaks)", sum([len(entry) for entry in playlist]),
"filesize:", file_to_send.size)
return response
The problem seems to lie in the non-ascii chars in playlist entries (äöüá). When there are no such characters, the file is transferred intact. I assume that these are characters that use two bytes in UTF-8, but writing strings to the ContentFile like I do is probably not correct.
Found the answer, while working on the problem description.
This works:
def create(request):
# just a dummy filelist
playlist = ["#EXTM3U"] + 5 * ["/home/pi/music/" + 5 * "äöü0123456789á/" + "xyz.mp3"]
joined_playlist = "\n".join([item.replace("/home/pi/Music", r"\\raspberry\music").replace("/", "\\") for item in playlist])
file_to_send = ContentFile(joined_playlist.encode("UTF-8"))
response = HttpResponse(file_to_send, "audio/x-mpegurl")
response["Content-Length"] = file_to_send.size
response["Content-Disposition"] = f"attachment; filename=\"playlist.m3u8\""
# print some debug info
print("lines:", len(playlist), "chars (no linebreaks)", sum([len(entry) for entry in playlist]),
"filesize:", file_to_send.size)
return response
The important difference is, that I don't write Strings to the ContentFile any longer, but a byte array, which I got through encoding the String in UTF-8.
HTH
Related
I have the following script to open ABAQUS ODB file and get displacements and coordinates of a specific node set. I can get these to print on screen but need help to write them to a file (.xlsx, .cvs, .dat or .txt) for postprocess. I'm new to scripting with abaqus so any help would be great appreciated. Code is currently as follows:
from odbAccess import *
from numpy import array
odb = openOdb(path='Test_3.odb')
lastFrame = odb.steps['Step-1'].frames[1]
displacement = lastFrame.fieldOutputs['U']
coords=lastFrame.fieldOutputs['COORD']
NodeSet_x = odb.rootAssembly.instances['CFRP_SKIN_TS-1'].nodeSets['NODE_SET_X_AXIS']
NodeSet_y = odb.rootAssembly.instances['CFRP_SKIN_TS-1'].nodeSets['NODE_SET_Y_AXIS']
centerDisplacement_x = displacement.getSubset(region=NodeSet_x)
NodeCoord_x = coords.getSubset(region=NodeSet_x)
centerDisplacement_y = displacement.getSubset(region=NodeSet_y)
NodeCoord_y = coords.getSubset(region=NodeSet_y)
for v in centerDisplacement_x.values:
disp_out = v.nodeLabel, v.data[2]
print (disp_out)
for c in NodeCoord_x.values:
coord_out = c.nodeLabel, c.data[0], c.data[1], c.data[2]
print (coord_out)
odb.close()
I think, it just basic file read write thing. But anyways.
For more details on how to write the data in text file in python refer below links.
Click here to know about opening and closing files in python.
Click here to know about writing format in python.
Please follow below simple lines of code which works for any number of node sets.
node_sets = ['NODE_SET_X_AXIS','NODE_SET_Y_AXIS']
for node_set in node_sets:
fileName = '%s.dat'%node_set
fout = open(fileName,'w')
nset = odb.rootAssembly.instances['CFRP_SKIN_TS-1'].nodeSets[node_set]
field = odb.steps['Step-1'].frames[1].fieldOutputs['U'].getSubset(region=nset)
for val in field.values:
data = val.data
node_label = val.nodeLabel
node = odb.rootAssembly.instances['CFRP_SKIN_TS-1'].getNodeFromLabel(label=node_label)
coords = node.coordinates
fout.write('%10d%14.4E%14.4E%14.4E%16.4E%16.4E%16.4E\n'%tuple([node_label,]+list(coords)+list(data)))
fout.close()
This code creates a separate text file for each node set.
I'm using Unity 2018.3. and try to HTTP POST Request with image from Unity to Django.
try to take a picture in Unity and then POST the image to Django server.
I want to receive in server and make it to image file.
well, when i send for image file to byte[] using EncodeToPNG() or EncodeToJPG(), the server got it but when i print it the data it was crash seems like encoding error. so it can't write to image format. (the take a picture code is working right.)
Django server result image
i saw a bunch of things about this issue so i tried to other way like use WWWform or JSON but anything couldn't works..
how to get image form Unity to Django?
All help appreciated! Thanks, all.
take a picture
void TakeSnapshot()
{
Texture2D snap = new Texture2D(frontCam.width, frontCam.height);
snap.SetPixels(frontCam.GetPixels());
snap.Apply();
_SavePath = pathForDocumentsFile("photo");
System.IO.File.WriteAllBytes(_SavePath + ".png", snap.EncodeToPNG());
bytes = snap.EncodeToPNG();
//bytes = snap.EncodeToJPG();
UnityEngine.Object.Destroy(snap);
path = _SavePath + ".png";
StartCoroutine(ServerThrows());
}
POST to server
IEnumerator ServerThrows()
{
List<IMultipartFormSection> formData = new List<IMultipartFormSection>();
formData.Add(new MultipartFormDataSection("photo", bytes, "byte[]"));
//UnityWebRequest www = UnityWebRequest.Post(url, null, bytes);
UnityWebRequest www = UnityWebRequest.Post(url, formData);
www.chunkedTransfer = false;
yield return www.SendWebRequest();
if (www.isNetworkError || www.isHttpError)
{
Debug.Log(www.error);
}
else
{
Debug.Log("Form upload complete!" + www.downloadHandler.text);
}
}
response in server
def post(self, request, format=None):
print('get the data')
print('request.POST: ', request.POST)
print('request.headers: ', request.headers)
data_test = request.POST.get('photo', '')
print('data test is : ', data_test)
print('type of data_test is : ', type(data_test))
print('length of data_test is : ', len(data_test))
print('finish to get ')
# data_test = data_test.decode('utf-8').encode('euc_kr','replace')
f = open('./test.png','wb')
f.write(data_test.encode())
f.close()
#
data = open('./test.png', 'rb')
return HttpResponse("post !")
I have a script which pulls XML hosted online and saves it locally. The script then goes through the local file and replaces/adds certain text. However, for some reason, when I use the "&" symbol, there is an extra space inserted along with it within the element text. Here is a sample of the XML elements I am parsing:
<TrackingEvents>
<Tracking event="rewind">
http://www.example.com/rewind_1.png?test=rewind_test
</Tracking>
<Tracking event="pause">
http://www.example.com/pause_1.png?test=rewind_test
</Tracking>
However, after running my script to add the additional test to my elements, the text is added with an additional space, like this:
<TrackingEvents>
<Tracking event="rewind">
http://www.example.com/rewind_1.png?test=rewind_test &cb={CACHEBUSTER}
</Tracking>
<Tracking event="pause">
http://www.example.com/pause_1.png?test=rewind_test &cb={CACHEBUSTER}
</Tracking>
I have tried everything but I don't know why this is occurring or what I can do to prevent this space from being added. I have even tried to strip the white space as well. When I look at the XML that is saved locally before uploading it, everything looks fine (& is for the "&" symbol) as seen here from the source:
<Tracking event="rewind">
http://www.example.com/rewind_1.png?test=rewind_test
&cb={CACHEBUSTER}</Tracking>
<Tracking event="pause">
http://www.example.com/pause_1.png?test=rewind_test
&cb={CACHEBUSTER}</Tracking>
Here is what the code from my script looks like:
for URL, xml_name, original_server in tqdm(XML_tags):
response = requests.get(URL)
with open(xml_name, 'wb') as file:
file.write(response.content)
with open(xml_name) as saved_file:
tree = ET.parse(saved_file)
root = tree.getroot()
for element in root.iter(tag=ET.Element):
if element.text != None:
if ".png" in element.text:
if "?" in element.text:
element.text = element.text + "&cb={CACHEBUSTER}"
element.text = element.text.strip()
else:
element.text = element.text + "?cb={CACHEBUSTER}"
element.text = element.text.strip()
else:
pass
server = "example.server: ../sample/sample/" + original_server
tree.write(xml_name, xml_declaration=True, method='xml',
encoding='utf8')
server_upload = subprocess.Popen(["scp", xml_name, server])
upload_wait = os.waitpid(server_upload.pid, 0)
I can definitely use some help with this. Thanks.
Update: Actually, it appears that this has nothing to do with using the "&". Here is a sample when I just add different text:
<TrackingEvents>
<Tracking event="rewind">
http://www.example.com/rewind_1.png?test=rewind_test test123
</Tracking>
<Tracking event="pause">
http://www.example.com/pause_1.png?test=rewind_test test123
</Tracking>
</TrackingEvents>
The whitespace was in the original XML even before you add anything to element.text; it is the newline between the last letter in the .text and the closing tag. So you should have removed the whitespace before appending text instead of after appending as you did in your code above :
....
if "?" in element.text:
element.text = element.text.strip() + "&cb={CACHEBUSTER}"
else:
element.text = element.text.strip() + "?cb={CACHEBUSTER}"
....
I'm trying to troncate several element.text on a xml file. I succeed to get two list, the first one regroup the formers too long element.text as str (long_name) and the second regroup the same after a troncation (short_name).
Now i want to replace the element.text on my xml, i tried some script but i surrended to work with the function readlines(), i want to find a similar solution with lxml as this code :
txt = open('IF_Generic.arxml','r')
Lines = txt.readlines()
txt.close()
txt = open('IF_Genericnew.arxml','w')
for e in range(len(long_name)) :
for i in range(len(Lines)) :
if (long_name[e] in Lines[i]) == True :
Lines[i] = Lines[i].replace(long_name[e],short_name[e])
for i in Lines :
txt.write(i)
txt.close()
I tried this, but it doesn't work :
f = open('IF_Generic.arxml')
arxml = f.read()
f.close()
tree = etree.parse(StringIO(arxml))
for e,b in enumerate(long_name) :
context = etree.iterparse(StringIO(arxml))
for a,i in context:
if not i.text:
pass
else:
if (b in i.text) == True :
i.text = short_name[e]
obj_arxml = etree.tostring(tree,pretty_print=True)
f = open('IF_Genericnew.arxml','w')
f.write(obj_arxml)
f.close()
Let's say the first element of the list long_name is RoutineServices_EngMGslLim_NVMID03
<BALISE_A>
<BALISE_B>
<SHORT-NAME>RoutineServices_EngMGslLim_NVMID03</SHORT-NAME>
</BALISE_B>
</BALISE_A>
<BALISE_C>
<POSSIBLE-ERROR-REF DEST="APPLICATION-ERROR">/Interfaces/RoutineServices_EngMGslLim_NVMID03/E_NOT_OK</POSSIBLE-ERROR-REF>
<SHORT-NAME>Blah_Bleh_Bluh</SHORT-NAME>
</BALISE_C>
The first element of the list short_name is RoutineServices_EngMGslLim_NV
<BALISE_A>
<BALISE_B>
<SHORT-NAME>RoutineServices_EngMGslLim_NV</SHORT-NAME>
</BALISE_B>
</BALISE_A>
<BALISE_C>
<POSSIBLE-ERROR-REF DEST="APPLICATION-ERROR">/Interfaces/RoutineServices_EngMGslLim_NV/E_NOT_OK</POSSIBLE-ERROR-REF>
<SHORT-NAME>Blah_Bleh_Bluh</SHORT-NAME>
</BALISE_C>
I want this
P.S: I use python 2.7.9
Thanks in advance everyone !
Don't open XML files like text files. I have explained in this answer why this is a bad idea.
Simply let etree read and write the file. It's also less code to write.
from lxml import etree
# read the file and load it into a DOM tree
tree = etree.parse('IF_Generic.arxml')
for elem in tree.iterfind("//*"):
# find elements that contain only text
if len(elem) == 0 and elem.text and elem.text.strip() > '':
# do your replacements ...
elem.text = "new text"
# serialize the DOM tree and write it to file
tree.write('IF_Genericnew.arxml', pretty_print=True)
Instead of going over all elements, which is what "//*" does, you can use more specific XPath to narrow down the elements you want to work on.
For example, something like "//SHORT-NAME | //POSSIBLE-ERROR-REF" would help to reduce the overall work load.
Using ReportLab 2.7, Python 2.7:
I can create a pdf with clickable bookmarks in the table of contents (toc), or I can create a pdf with "x of y" page numbering, but the bookmarkpage(key) appears to break when I try to do both.
Example code:
class MyDocTemplate(BaseDocTemplate):
def __init__(self, filename, **kw):
self.allowSplitting = 0
BaseDocTemplate.__init__(self, filename, **kw)
template = PageTemplate('normal', [Frame(2.5*cm, 2.5*cm, 15*cm, 25*cm)]) #, id='F1'
self.addPageTemplates(template)
def afterFlowable(self, flowable):
if isinstance(flowable, Paragraph):
txt = flowable.getPlainText()
style = flowable.style.name
if style == 'Heading1':
key = 'h1-%s' % self.seq.nextf('heading1')
self.canv.bookmarkPage(key)
self.notify('TOCEntry', (0, txt, self.page))
elif style == 'Heading2':
key = 'h2-%s' % self.seq.nextf('heading2')
print key
self.canv.bookmarkPage(key)
self.notify('TOCEntry', (1, txt, self.page, key))
class NumberedCanvas(canvas.Canvas):
def __init__(self, *args, **kwargs):
canvas.Canvas.__init__(self, *args, **kwargs)
self._saved_page_states = []
def showPage(self):
self._saved_page_states.append(dict(self.__dict__))
self._startPage()
def save(self):
"""add page info to each page (page x of y)"""
num_pages = len(self._saved_page_states)
for state in self._saved_page_states:
self.__dict__.update(state)
self.draw_page_number(num_pages)
canvas.Canvas.showPage(self)
canvas.Canvas.save(self)
def draw_page_number(self, page_count):
self.setFont('Times-Bold',14)
self.drawRightString(7.6*inch,.5*inch,
"Page %d of %d" % (self._pageNumber, page_count))
h1 = PS(name = 'Heading1',
fontSize = 14,
leading = 16)
h2 = PS(name = 'Heading2',
fontSize = 12,
leading = 14,
leftIndent = 25)
#Build story.
story = []
toc = TableOfContents()
#For conciseness, using the same styles for headings and TOC entries
toc.levelStyles = [h1, h2]
story.append(toc)
story.append(PageBreak())
story.append(Paragraph('First heading', h1))
story.append(Paragraph('Text in first heading', PS('body')))
story.append(Paragraph('First sub heading', h2))
story.append(Paragraph('Text in first sub heading', PS('body')))
story.append(PageBreak())
story.append(Paragraph('Second sub heading', h2))
story.append(Paragraph('Text in second sub heading', PS('body')))
story.append(Paragraph('Last heading', h1))
doc = MyDocTemplate("mypdf.pdf")
doc.multiBuild(story, canvasmaker=NumberedCanvas)
Any comments would be greatly appreciated.
found this initially looking for a solution to no avail, however, figured something out which works for me and wish to share as follows:
With the starting point of the reportlab example as regards to creating a toc containing document -which I assume you used in the first place- proceed as follows.
Modify the template assignment in the init function by adding the options onPage and onPageEnd defining calls to functions for drawing a header and a footer onto the page as follows, guess you could do the same within one function only, however, I used two in order to separate the header/footer section in my usecase:
template = PageTemplate('normal', [Frame(2.5*cm, 2.5*cm, 15*cm, 25*cm, id='F1')],onPage=header,onPageEnd=footer)
add the named header/footer functions to your class as shown below with illustrating some options for clarification including the page number in the footer function to make sure the question is answered:
def header(canvas, doc):
canvas.saveState()
canvas.drawImage('some_image.gif', 5,780, width=None,height=None,mask=None)
canvas.setLineWidth(1)
canvas.line(5,780,120,780)
canvas.setFont('Times-Bold',16)
canvas.drawString(108, 108, 'blah')
canvas.restoreState()
def footer(canvas, doc):
canvas.saveState()
canvas.setFont('Times-Roman',9)
canvas.drawImage('py.jpg', inch,inch, width=None,height=None,mask=None)
canvas.drawString(inch, 0.75 * inch, "Page %d " % doc.page)
canvas.restoreState()
Whatever you did with the added 'numbered canvas class' and why it breaks I don't know, but since your provided code reminds me strongly of the reportlab toc example I guess you can start with that, apply my suggested changes/additions and see how it goes.
That should do the trick, hope that helps!