Reading Hex Values from file (C++) - c++

I've been struggling with this one for a while now. I'm trying to read a file with hexadecimal information in it, lets say the contents of the file looks like this.
00 00 e0 3a 12 16 00 ff fe 98 c4 cc ce 14 0e 0a aa cf
I'm looking for a method of reading each 'byte' of information (whilst ignoring whitespace) in C++.
Since the code I'm writing is part of a class I've created a small program to demonstrate what I'm currently doing.
int main(void)
{
int i = 0;
unsigned char byte;
unsigned char Memory[50];
ifstream f_in;
f_in.open("file_with_hex_stuff.txt");
f_in >> skipws; // Skip Whitespace
while(f_in >> hex >> byte)
{
Memory[i++] = byte;
}
return 0;
}
(apologies if the code above doesn't compile, it's just so you can get a feel for what I'm wanting.
I expected the array to be like this:
Memory[0] => 0x00;
Memory[1] => 0x00;
Memory[2] => 0xe0;
Memory[3] => 0x3a;
Memory[4] => 0x12;
etc....
But instead it loads each of the numbers into it's own position in the array like this:
Memory[0] => 0x00; // 2 LOCATIONS USED FOR 0x00
Memory[1] => 0x00; //
Memory[2] => 0x00; // 2 LOCATIONS USED FOR 0x00
Memory[3] => 0x00; //
Memory[4] => 0x0e; // 2 LOCATIONS USED FOR 0xe0
Memory[5] => 0x00; //
Memory[6] => 0x03; // 2 LOCATIONS USED FOR 0x3a
Memory[7] => 0x0a; //
etc...
Applogies if this post makes no sense. Any feedback appreciated.
Thanks.

From your output, your program is reading one hex digit at a time.
Try reading into an unsigned int instead:
unsigned int value;
while (f_in >> hex >> value)
{
byte = ((unsigned char) (value & 0xff);
Memory[i++] = byte;
//...
}
Also, try using a debugger to explore your code. For example, you could add value and byte as watch variables and single step through your code.
Demo.

Related

CRC-16 specific example for serial connection

While working with the following PDF, there is an example in
Section 4: CRC-16 Code and Example
(page 95 or 91) that shows a serial packet with a CRC16 value of 133 (LSB) and 24 (MSB).
However, I have tried different calculators, for example:
Lammert
Elaborate calculator
CRC calc
but I cannot get the CRC16 values that the PDF indicates, regardless of the byte combination I use.
How can I correctly calculate the CRC16 in the example, preferably using one of these calculators? (otherwise, C/C++ code should work).
Thanks.
This particular CRC is CRC-16/ARC. crcany generates the code for this CRC, which includes this simple bit-wise routine:
#include <stddef.h>
#include <stdint.h>
uint16_t crc16arc_bit(uint16_t crc, void const *mem, size_t len) {
unsigned char const *data = mem;
if (data == NULL)
return 0;
for (size_t i = 0; i < len; i++) {
crc ^= data[i];
for (unsigned k = 0; k < 8; k++) {
crc = crc & 1 ? (crc >> 1) ^ 0xa001 : crc >> 1;
}
}
return crc;
}
The standard interface is to do crc = crc16arc_bit(0, NULL, 0); to get the initial value (zero in this case), and then crc = crc16arc_bit(crc, data, len); with successive portions of the message to compute the CRC.
If you do that on the nine-byte message in the appendix, {1, 2, 1, 0, 17, 3, 'M', 'O', 'C'}, the returned CRC is 0x1885, which has the least significant byte 133 in decimal and most significant byte 24 in decimal.
Faster table-driven routines are also generated by crcany.
If you give 01 02 01 00 11 03 4d 4f 43 as hex to Lammert Bies' calculator, the very first one, called "CRC-16" gives 0x1885.
If you give 0102010011034d4f43 to crccalc.com and hit Calc-CRC16, the second line is CRC-16/ARC, with the result 0x1885.

C++ write nibbles in file

Good evening,
I'm new to C++ and encountered a problem that I wasn't able to solve despite reading numerous pages here. I've got a file with hexvalues that need to be read and compressed, then written in a new file. An example sequence looks like this:
C9 CB FF 01 06 (each byte [8 bit] represent a number)
Compression starts with the first number, then only writing the difference to the next number (differences are a nibble [4 bit]). Example from C9 to CB: difference = 2. If the difference is greater than 7, thus can't be represented by a nibble, we use a 0x8 to mark a new start. 0xFF-0xCB > 7 so the sequence would look like this (entire compressed code):
C9 28 FF 15 (mixture of entire bytes (0xC9 and 0xFF) representing numbers and nibbles representing differences to the next number. Now to my problem. I'm using fstream and put to write bytes to a new file, nibbles are stored to combine with an other nibble to a byte which can be written to the file. However it only works with bytes smaller than 128 so I can't write values greater than 0x7F into a file. I prepared a file with notepad++ starting with the value 0xFF - reading that value works great but dest.put(source.get()); doesn't in that specific case. How can I work with (signed) nibbles [for negative differences] and binary presentations of numbers in C++? By the way using negative numbers in file.put() results in strange behavior as 2 bytes are written rather than one. Here's my code, I hope you understand my problem and I really appreciate your help
int lastValue = s.get();
d.put((char)lastValue);
char highNibble = 0;
bool nibbleSet = false;
int diff = 0;
for (int c = s.get(); c != -1; c = s.get()) {
diff = (char)((unsigned char)c - (unsigned char)lastValue);
if (abs(diff) > 7) {
if (nibbleSet) {
d.put(highNibble << 4 | 8);
d.put((char)c);
nibbleSet = false;
}
else {
cout << (8 << 4 | (c & 0xF0) >> 4) << endl;
d.put(8 << 4 | (c & 0xF0) >> 4);
highNibble = c & 0x0F;
nibbleSet = true;
}
}
else {
if (nibbleSet) {
d.put(((char)highNibble << 4) & 0xF0 | ((char)diff) & 0x0F);
nibbleSet = false;
}
else {
highNibble = (char)diff;
nibbleSet = true;
}
}
lastValue = c;
}

Increase Byte Values in NSData

I have an NSMutableData holding random ASCII bytes.
I would like to shift the values of those bytes by a value (X).
So let us say I have something like this:
02 00 02 4e 00
I now want to increase every single byte by 0x01 to get:
03 01 03 4f 01
What is the best approach for this?
Use mutableBytes to get a pointer to your bytes, and treat them like a normal C array.
uint8_t originalBytes[] = {0x02, 0x00, 0x02, 0x4e, 0x00};
NSMutableData * myData = [NSMutableData dataWithBytes:originalBytes length:5];
uint8_t * bytePtr = [myData mutableBytes];
for(int i = 0; i < [myData length]; i++) {
bytePtr[i] += 0x01;
}
NSLog(#"%#", myData);
There's more info in the Binary Data Programming Guide article, "Working With Mutable Binary Data".
Also, what you're doing is not "shifting" but merely adding 0x01. "Shifting" typically refers to "bit shifting".
something like this:
NSString *_chars = #"abcd";
NSMutableData *_data = [NSMutableData dataWithBytes:[_chars UTF8String] length:_chars.length];
NSLog(#"Bef. : %#", _data);
for (int i = 0; i < _data.length; i++) ((char *)[_data mutableBytes])[i]++;
NSLog(#"Aft. : %#", _data);
the log shows the result:
Bef. : <61626364>
Aft. : <62636465>

C++ OpenSSL HMACSHA1 works but not how I want it to

This HMACSHA1 code below works for converting "Password" and "Message" to AFF791FA574D564C83F6456CC198CBD316949DC9 as evidence by http://buchananweb.co.uk/security01.aspx.
My question is, Is it possible to have:
BYTE HMAC[] = {0x50,0x61,0x73,0x73,0x77,0x6F,0x72,0x64};
BYTE data2[] = {0x4D,0x65,0x73,0x73,0x61,0x67,0x65};
And still get the same value: AFF791FA574D564C83F6456CC198CBD316949DC9.
For example, if I was on a server and received the packet:
[HEADER] 08 50 61 73 73 77 6F 72 64 00
[HEADER] 07 4D 65 73 73 61 67 65 00
And I rip 50 61 73 73 77 6F 72 64 & 4D 65 73 73 61 67 65 from the packet and used this for my HMACSHA1. How would I go about doing that to get the correct HMACSHA1 value?
BYTE HMAC[] = "Password";
BYTE data2[] = "Message";
//BYTE HMAC[] = {0x50,0x61,0x73,0x73,0x77,0x6F,0x72,0x64};
//BYTE data2[] = {0x4D,0x65,0x73,0x73,0x61,0x67,0x65};
HMAC_CTX ctx;
result = (unsigned char*) malloc(sizeof(char) * result_len);
ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
HMAC_CTX_init(&ctx);
HMAC_Init_ex(&ctx, HMAC, strlen((const char*)HMAC), EVP_sha1(), NULL);
HMAC_Update(&ctx, data2, strlen((const char*)(data2)));
HMAC_Final(&ctx, result, &result_len);
HMAC_CTX_cleanup(&ctx);
std::cout << "\n\n";
for(int i=0;i<result_len;i++)
std::cout << setfill('0') << setw(2) << hex << (int)result[i];
int asd;
std::cin >> asd;
// AFF791FA574D564C83F6456CC198CBD316949DC9
EDIT:
It works by doing this:
BYTE HMAC[] = {0x50,0x61,0x73,0x73,0x77,0x6F,0x72,0x64, 0x00};
BYTE data2[] = {0x4D,0x65,0x73,0x73,0x61,0x67,0x65, 0x00};
By adding 0x00, at the end. But, my question is more towards ripping it from data, and using it... would it still be fine?
The issue is the relation ship between arrays, strings, and the null char.
When you declare "Password", the compiler logically treats the string literal as a nine byte array, {0x50,0x61,0x73,0x73,0x77,0x6F,0x72,0x64, 0x00}. When you call strlen, it will count the number of bytes until it encounters the first 0x00. strlen("Password") will return 8 even though there are technically nine characters in the array of characters.
So when you declare an array of 8 bytes as follows without a trailing null byte:
BYTE HMAC[] = {0x50,0x61,0x73,0x73,0x77,0x6F,0x72,0x64};
The problem is that "strlen(HMAC)" will count at least 8 bytes, and keep counting while traversing undefined memory until it finally (if ever) hits a byte that is zero. At best, you might get lucky because the stack memory always has a zero byte padding your array declaration. More likely it will return a value much larger than 8. Maybe it will crash.
So when you parse the HMAC and MESSAGE field from your protocol packet, you count the number of bytes actually parsed (not including the terminating null). And pass that count into the hmac functions to indicate the size of your data.
I don't know your protocol code, but I hope you aren't using strlen to parse the packet to figure out where the string inside the packet ends. A clever attacker could inject a packet with no null terminator and cause your code do bad things. I hope you are parsing securely and carefully. Typical protocol code doesn't include the null terminating byte in the strings packed inside. Usually the "length" is encoded as an integer field followed by the string bytes. Makes it easier to parse and determine if the length would exceed the packet size read in.

Using bitwise operators in C++ to change 4 chars to int

What I must do is open a file in binary mode that contains stored data that is intended to be interpreted as integers. I have seen other examples such as Stackoverflow-Reading “integer” size bytes from a char* array. but I want to try taking a different approach (I may just be stubborn, or stupid :/). I first created a simple binary file in a hex editor that reads as follows.
00 00 00 47 00 00 00 17 00 00 00 41
This (should) equal 71, 23, and 65 if the 12 bytes were divided into 3 integers.
After opening this file in binary mode and reading 4 bytes into an array of chars, how can I use bitwise operations to make char[0] bits be the first 8 bits of an int and so on until the bits of each char are part of the int.
My integer = 00 00 00 00
+ ^ ^ ^ ^
Chars Char[0] Char[1] Char[2] Char[3]
00 00 00 47
So my integer(hex) = 00 00 00 47 = numerical value of 71
Also, I don't know how the endianness of my system comes into play here, so is there anything that I need to keep in mind?
Here is a code snippet of what I have so far, I just don't know the next steps to take.
std::fstream myfile;
myfile.open("C:\\Users\\Jacob\\Desktop\\hextest.txt", std::ios::in | std::ios::out | std::ios::binary);
if(myfile.is_open() == false)
{
std::cout &lt&lt "Error" &lt&lt std::endl;
}
char* mychar;
std::cout &lt&lt myfile.is_open() &lt&lt std::endl;
mychar = new char[4];
myfile.read(mychar, 4);
I eventually plan on dealing with reading floats from a file and maybe a custom data type eventually, but first I just need to get more familiar with using bitwise operations.
Thanks.
You want the bitwise left shift operator:
typedef unsigned char u8; // in case char is signed by default on your platform
unsigned num = ((u8)chars[0] << 24) | ((u8)chars[1] << 16) | ((u8)chars[2] << 8) | (u8)chars[3];
What it does is shift the left argument a specified number of bits to the left, adding zeros from the right as stuffing. For example, 2 << 1 is 4, since 2 is 10 in binary and shifting one to the left gives 100, which is 4.
This can be more written in a more general loop form:
unsigned num = 0;
for (int i = 0; i != 4; ++i) {
num |= (u8)chars[i] << (24 - i * 8); // += could have also been used
}
The endianness of your system doesn't matter here; you know the endianness of the representation in the file, which is constant (and therefore portable), so when you read in the bytes you know what to do with them. The internal representation of the integer in your CPU/memory may be different from that of the file, but the logical bitwise manipulation of it in code is independent of your system's endianness; the least significant bits are always at the right, and the most at the left (in code). That's why shifting is cross-platform -- it operates at the logical bit level :-)
Have you thought of using Boost.Spirit to make a binary parser? You might hit a bit of a learning curve when you start, but if you want to expand your program later to read floats and structured types, you'll have an excellent base to start from.
Spirit is very well-documented and is part of Boost. Once you get around to understanding its ins and outs, it's really mind-boggling what you can do with it, so if you have a bit of time to play around with it, I'd really recommend taking a look.
Otherwise, if you want your binary to be "portable" - i.e. you want to be able to read it on a big-endian and a little-endian machine, you'll need some sort of byte-order mark (BOM). That would be the first thing you'd read, after which you can simply read your integers byte by byte. Simplest thing would probably be to read them into a union (if you know the size of the integer you're going to read), like this:
union U
{
unsigned char uc_[4];
unsigned long ui_;
};
read the data into the uc_ member, swap the bytes around if you need to change endianness and read the value from the ui_ member. There's no shifting etc. to be done - except for the swapping if you want to change endianness..
HTH
rlc