process a string of ascii Hex values into a numerical hex value - python-2.7

I'm still fairly new at python so forgive me if this is a fairly easy question, but I didn't find anything obvious through searching.
I've got a string of Ascii hex in the form of:
7F 9D AA 3E F7 0E 9C 75 7C 37
What I'm trying to do is extract a number from the table (for example 7F) and then convert it into a hex value that I can then perform a mathematical operation on.
How would I go about doing this?

If you simply need to convert a string representation of a hex value to an non-string version, you can use binascii to convert your string representation to its hex equivalent:
import binascii
h = "7F 9D AA 3E F7 0E 9C 75 7C 37"
binascii.unhexlify(''.join(h.split()))
# >> '\x7f\x9d\xaa>\xf7\x0e\x9cu|7'
If you need integers, convert your string into a list (so that you are able to iterate over it) and then use int() to convert your base-16 hexadecimal numbers back to integers:
hex_list = "7F 9D AA 3E F7 0E 9C 75 7C 37"
# Convert your string list to a list: e.g. ['7F', '9D', 'AA'...
hex_list = hex_list.split()
for hex in hex_list:
print int(hex, 16) #hex is base 16
To convert just a single list value, specify its index in your list:
# Convert 7F back to an integer:
print int(hex_list[0], 16)
To convert an int back to hex, simply pass into hex().

Related

converting a string read from binary file to integer

I have a binary file. i am reading 16 bytes at a time it using fstream.
I want to convert it to an integer. I tried atoi. but it didnt work.
In python we can do that by converting to byte stream using stringobtained.encode('utf-8') and then converting it to int using int(bytestring.hex(),16). Should we follow such an elloborate steps as done in python or is there a way to convert it directly?
ifstream file(binfile, ios::in | ios::binary | ios::ate);
if (file.is_open())
{
size = file.tellg();
memblock = new char[size];
file.seekg(0, ios::beg);
while (!file.eof())
{
file.read(memblock, 16);
int a = atoi(memblock); // doesnt work 0 always
cout << a << "\n";
memset(memblock, 0, sizeof(memblock));
}
file.close();
Edit:
This is the sample contents of the file.
53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00
04 00 01 01 00 40 20 20 00 00 05 A3 00 00 00 47
00 00 00 2E 00 00 00 3B 00 00 00 04 00 00 00 01
I need to read it as 16 byte i.e. 32 hex digits at a time.(i.e. one row in the sample file content) and convert it to integer.
so when reading 53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00, i should get, 110748049513798795666017677735771517696
But i couldnt do it. I always get 0 even after trying strtoull. Am i reading the file wrong, or what am i missing.
You have a number of problems here. First is that C++ doesn't have a standard 128-bit integer type. You may be able to find a compiler extension, see for example Is there a 128 bit integer in gcc? or Is there a 128 bit integer in C++?.
Second is that you're trying to decode raw bytes instead of a character string. atoi will stop at the first non-digit character it runs into, which 246 times out of 256 will be the very first byte, thus it returns zero. If you're very unlucky you will read 16 valid digits and atoi will start reading uninitialized memory, leading to undefined behavior.
You don't need atoi anyway, your problem is much simpler than that. You just need to assemble 16 bytes into an integer, which can be done with shifting and or operators. The only complication is that read wants a char type which will probably be signed, and you need unsigned bytes.
ifstream file(binfile, ios::in | ios::binary);
char memblock[16];
while (file.read(memblock, 16))
{
uint128_t a = 0;
for (int i = 0; i < 16; ++i)
{
a = (a << 8) | (static_cast<unsigned int>(memblock[i]) & 0xff);
}
cout << a << "\n";
}
file.close();
It the number is binary what you want is:
short value ;
file.read(&value, sizeof (value));
Depending upon how the file was written and your processor, you may have to reverse the bytes in value using bit operations.

c++ replacing order of items in byte array

I'm writing a code for Arduino C++.
I have a byte array with hex byte values, for example:
20 32 36 20 E0 EC 20 F9 F0 E9 E9 E3 F8 5C 70 5C 70 5C 73 20 E3 E2 EC 20 F8 E0 E5 E1 EF 20 39 31 5C
There are four ASCII digits in these bytes:
HEX 0x32 is number 2 in ascii code
HEX 0x35 is number 5 in ascii code
HEX 0x39 is number 9 in ascii code
and so on....
https://www.ascii-codes.com/cp862.html
So the hex values 32, 36 represent the number 26, and 39, 31 represent 91.
I want to find these numbers and reverse each group, so that (in this example) 62 and 19 are represented instead of 26 and 91.
The output would thus have to look like this:
20 36 32 20 E0 EC 20 F9 F0 E9 E9 E3 F8 5C 70 5C 70 5C 73 20 E3 E2 EC 20 F8 E0 E5 E1 EF 20 31 39 5C
The numbers don't have to be two digits but could be anything in 0-1000
I also know that each group of such numbers is preceded by the hex value 20, if that helps.
I have done this in C# (with some help of Stack overflow users :-) ):
string result = Regex.Replace(HexMessage1,
#"(?<=20\-)3[0-9](\-3[0-9])*(?=\-20)",
match => string.Join("-", Transform(match.Value.Split('-'))));
private static IEnumerable<string> Transform(string[] items)
{
// Either terse Linq:
// return items.Reverse();
// Or good old for loop:
string[] result = new string[items.Length];
for (int i = 0; i < items.Length; ++i)
result[i] = items[items.Length - i - 1];
return result;
}
Can someone help me make it work on C++?
Loop over the array, element by element, looking for 0x32 or 0x39. If found, check the next byte (if within bounds) to see if it matches 0x36 or 0x31 (respectively). If it does then swap the current and the next byte. Continue the loop, skipping over the current and the next byte.

C++ Read Binary File with "ieee-be" machinefmt

I basically try to convert Matlab Code to C++, reading a binary file I do not really know how it looks like.
The Matlab Code is simplified as follows:
x=zeros(48,32);
fid=fopen('pres_00.bin','r','ieee-be');
fseek(fid,ipos,'bof');
x(1:4:48,:)=fread(fid,[12,32],'single');
in the end we basically get double numbers in the x array (row 1, 5,..)
How can I read the *.bin file in C++? I tried:
file1.seekg(0, ios::end);
int length = file1.tellg();
file1.seekg(ipos, ios_base::beg);
lenght = lenght - ipos;
char * buffer = new char[length];
file1.read(buffer, length);
double* double_values = (double*)buffer;
double test = double_values[0];
file1.close();
Sadly "test" is not similar to the number matlab is encoding out of the binary file. How can I implement the information with the ieee-be encoding into c++?
Unfortunately I'm not that familiar with binary files...
Cheers and thanks for your help!
//edit:
Maybe it helps:
In my case
ipos = 0
the first hex row (offset0) (32) :
44 7C CD 35 44 7C AD 89 44 7C E9 F2 44 7D F7 10 44 7D 9C F9 44 7B F9 E4 44 7B 3E 1D 44 7B 6C CE
ANSI: D|Í5D|.‰D|éòD}÷.D}œùD{ùäD{>.D{lÎ
First value in Matlab: 1.011206359863281e+03
What my Code reads in buffer: D|Í5D|-‰.D|éòD}÷.\x10D}œùD{ùäD{>\x1dD{lÎ......
double test = -4.6818882332480884e-262
There are two parts to this problem. First, the representation is IEEE 32 bit floating point; since most processors use IEEE floating point, all you need is a simple cast to do the conversion. This won't be portable to all processors though. The second part is the be in the ieee-be specification, it means that the bytes are stored big-endian. Since many processors (i.e. Intel/AMD) are little-endian, you need to do a byte swap before the conversion.
void byteswap4(char *p)
{
std::swap(p[0], p[3]);
std::swap(p[1], p[2]);
}
float to_float(char *p)
{
return *((float*)p);
}
See it in action: https://ideone.com/IrDEJF

Binary File interpretation

I am reading in a binary file (in c++). And the header is something like this (printed in hexadecimal)
43 27 41 1A 00 00 00 00 23 00 00 00 00 00 00 00 04 63 68 72 31 FFFFFFB4 01 00 00 04 63 68 72 32 FFFFFFEE FFFFFFB7
when printed out using:
std::cout << hex << (int)mem[c];
Is there an efficient way to store 23 which is the 9th byte(?) into an integer without using stringstream? Or is stringstream the best way?
Something like
int n= mem[8]
I want to store 23 in n not 35.
You did store 23 in n. You only see 35 because you are outputting it with a routine that converts it to decimal for display. If you could look at the binary data inside the computer, you would see that it is in fact a hex 23.
You will get the same result as if you did:
int n=0x23;
(What you might think you want is impossible. What number should be stored in n for 1E? The only corresponding number is 31, which is what you are getting.)
Do you mean you want to treat the value as binary-coded decimal? In that case, you could convert it using something like:
unsigned char bcd = mem[8];
unsigned char ones = bcd % 16;
unsigned char tens = bcd / 16;
if (ones > 9 || tens > 9) {
// handle error
}
int n = 10*tens + ones;

8-bit char to Hex representation

I'm trying to convert 8 bit char into hex view which looks like this:
00 03 80 45 E5 93 00 18 02 72 3B 90 88 64 11 00
45 FF 00 36 00 FF 45 00 00 34 7B FE 40 00 40 02
But some characters contain negative values which makes a larger hex value of more than 2 digits. how would i get each one as represented above?
I don't know what you are using for formatting, but make sure that you make your byte holding variable an unsigned char (assuming that char is 8-bits on your platform, which it is on all sane platforms), before formatting. If your platform has a sane BYTE typedef, use that. You can also use the boost::uint8_t type to store the byte and avoid these sorts of issues. For example:
char c=-25; // Oh no, this is one of those pesky "negative" characters
unsigned char byteVal=static_cast<unsigned char>(c); // FTFY
// Do the formatting with byteVal
"negative byte values" is an oxymoron, a byte is a number of bits without any sign typically an unsigned char which, when being 8 bits. can contain values 0-255 or in hex 00 to FF.