Converting python2 byte/string encoding to python3 - python-2.7

I'm communicating over a serial port, and currently using python2 code which I want to convert to python3. I want to make sure the bytes I send over the wire are the same, but I'm having trouble verifying that that's the case.
In the original code the commands are sent like this:
serial.Serial().write("\xaa\xb4" + chr(2))
If I print "\xaa\xb4" in python2 I get this: ��.
If I print("\xaa\xb4") in python3 I get this: ª´
Encoding and decoding seem opposite too:
Python2: print "\xaa".decode('latin-1') -> ª
Python3: print("\xaa".encode('latin-1')) -> b'\xaa'
To be crude, what do I need to send in serial.write() in python3 to make sure exactly the same sequence of 1s and 0s are sent down the wire?

Use a bytes sequence.
ser.write(b'\xaa\xb4')

Related

Problem using magenta to generate song: SyntaxError: (unicode error) 'unicodeescape'

I want to generate music with magenta and a neural network model for a project.
I found this simple example and wanted to try it first to understand how it works: https://www.twilio.com/blog/training-a-neural-network-on-midi-music-data-with-magenta-and-python
Apparently i have to modify the type of my inital data (which is midi file) "in note to sequences"
Here is what i have:
convert_dir_to_note_sequences \
--input_dir == 'C:\Users\mista\Downloads\CLEANED_DATA\CLEANED_DATA' \
--output_file = tmp/notesequences.tfrecord \
--recursive
and here is the error i get:
File "C:\Users\mista\AppData\Local\Temp/ipykernel_28328/3757950315.py", line 3
--output_file = tmp/notesequences.tfrecord
^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
I saw some people saying that you could use an 'r' before your path to solve this but i've tried many ways, i'm still stuck
I am following the same tutorial and after a couple hours of bashing my head against the keyboard, I managed to run the neural net properly. It seems like the problem is a lot of conflicting dependencies and deprecations, so you may have to play around with what version of Python you're using once you set everything else up properly (I used Python 3.7 and did pip install magenta).
I used Powershell (right click on Windows home button --> run Powershell as admin). You'll want to set up a virtual environment in your current working directory, as the tutorial advises. And now we get to your problem in particular - make sure you're running the command all in one line, as such:
convert_dir_to_note_sequences --input_dir == 'C:\Users\mista\Downloads\CLEANED_DATA\CLEANED_DATA' --output_file = tmp/notesequences.tfrecord --recursive
That should start converting all the midi files to NoteSequences objects. If you have any more trouble, please follow up and I'll see what I can do to help.

Receiving back string of lenght 0 from os.popen('cmd').read()

I am working with a command line tool called 'ideviceinfo' (see https://github.com/libimobiledevice) to help me to quickly get back serial, IMEI and battery health information from the iOS device I work with daily. It executes much quicker than Apple's own 'cfgutil' tools.
Up to know I have been able to develop a more complicated script than the one shown below in PyCharm (my main IDE) to assign specific values etc to individual variables and then to use something like to pyclip and pyautogui to help automatically paste these into the fields of the database app we work with. I have also been able to use the simplified version of the script both in Mac OS X terminal and in the python shell without any hiccups.
I am looking to use AppleScript to help make running the script as easy as possible.
When I try to use Applescript's "do shell script 'python script.py'" I just get back a string of lenght zero when I call 'ideviceinfo'. The exact same thing happens when I try to build an Automator app with a 'Run Shell Script' component for "python script.py".
I have tried my best to isolate the problem down. When other more basic commands such as 'date' are called within the script they return valid strings.
#!/usr/bin/python
import os
ideviceinfoOutput = os.popen('ideviceinfo').read()
print ideviceinfoOutput
print len (ideviceinfoOutput)
boringExample = os.popen('date').read()
print boringExample
print len (boringExample)
I am running Mac OS X 10.11 and am on Python 2.7
Thanks.
I think I've managed to fix it on my own. I just need to be far more explicit about where the 'ideviceinfo' binary (I hope that's the correct term) was stored on the computer.
Changed one line of code to
ideviceinfoOutput = os.popen('/usr/local/bin/ideviceinfo').read()
and all seems to be OK again.

Reading hexascii in Python 2.7 vs Python 3.5x

I have a function built that reads in hex-ascii encoded data, I built that in Python 2.7. I am changing my code over to run on 3.x and hit an unforeseen issue. The function worked flawlessly under 2.7. Here is what I have:
# works with 2.7
data = open('hexascii_file.dat', 'rU').read()
When I run that under 3.x I get a UnicodeError:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x85 in position 500594: invalid start byte
I thought the default codec under Python 2.7 was ascii, so I tried the following under 3.x:
data = open('hexascii_file.dat', 'rU', encoding='ascii')
This did not work (same error as above, but specifying 'ascii' instead of 'utf-8'. However, when I use the latin-1 codec all works well.
data = open('hexascii_file.dat', 'rU', encoding='latin-1')
I guess I am looking for a quick sanity check here to ensure I have made the proper change to the script. Does this change make sense?

Interoperability problems python2 python3

Two uServices are communicating via a message queue (RabbitMQ). The data is encoded using message pack.
I have the following scenarios:
python3 -> python3: working fine
python2 -> python3: encoding issues
Encoding is done with:
umsgpack.packb(data)
Decoding with:
umsgpack.unpackb(body)
When doing encoding and decoding in python3 I get:
data={'sender': 'producer-big-red-tiger', 'json': '{"msg": "hi"}', 'servicename': 'echo', 'command': 'run'}
When doing encoding in python2 and decoding on python3 I get:
data={b'command': b'run', b'json': b'{"msg": ""}', b'servicename': b'echo', b'sender': b'bla-blah'}
Why is the data non "completely" decoded? What should I do on the sender / receiver to achieve compatibility between python2 and python3?
Look at the "Notes" section of the README from msgpack-python;
msgpack can distinguish string and binary type for now. But it is not like Python 2. Python 2 added unicode string. But msgpack renamed raw to str and added bin type. It is because keep compatibility with data created by old libs. raw was used for text more than binary.
Currently, while msgpack-python supports new bin type, default setting doesn't use it and decodes raw as bytes instead of unicode (str in Python 3).
You can change this by using use_bin_type=True option in Packer and encoding="utf-8" option in Unpacker.
>>> import msgpack
>>> packed = msgpack.packb([b'spam', u'egg'], use_bin_type=True)
>>> msgpack.unpackb(packed, encoding='utf-8')
['spam', u'egg']

Can't create a packet on scapy

I'm new to scapy.
I'm trying to combine scapy in a script, and not use it through bash.
I'm intending to forge a packet.
By all the tutorials and .pdf's I saw, all explaining how to do it inside the scapy tool(in bash).
I tried the following code:
from scapy.all import *
a=IP()
a.src="10.0.0.30"
a.dst="10.0.0.138"
print a
I tried this just to get a print output like I would get in bash, that will look like:
<IP src=10.0.0.20 dst=10.0.0.138|>
but insted I get unknown symbols, something like:
"EXX#f#
?"
My question is how do I get the same output that I got in bash??? I mean viewing the fields that I inserted.
Thanks for the responders.
The print statement in Python when used on a Scapy object will output the raw bytes of a packet.
You should use the .show() function for the packet you created to get the nice output you are expecting. Try the following in your python program.
a.show()
a.show2()
See this Send and Receive example in the Scapy docs for more.
A list of for examining the contents of the packet is also found here - just scroll down to the "Command/Effect" table.