I am writing code in c++ to read a wave file in. I am following the wave file specification I found here.
In the following code I am reading in the chunksize, which is stored in bytes 4,5,6,7.
According to the specification, this int is stores in little endian in these 4 bytes.
So if these 4 bytes held the unsigned value 2, I would think the would be as follows..
4 5 6 7
00000010 00000000 00000000 00000000
So if I am trying to read these 4 bytes as an int on windows, I don't need to do anything correct? Since windows it little endian. So this is what I did...
unsigned int chunk_size = (hbytes[4] << 24) + (hbytes[5] << 16) + (hbytes[6] << 8) + hbytes[7];
but that didn't work, it gave me an incorrect value. When I swapped the endian of the bytes, it did work....
unsigned int chunk_size = (hbytes[7] << 24) + (hbytes[6] << 16) + (hbytes[5] << 8) + hbytes[4];
Is this information I have about wavefiles correct? Is this int stored as little endian? Or are my assumptions about endianess incorrect?
You got everything right except the procedure to convert a little-endian stream.
Your diagram is correct: if the 4-byte field holds a 2, then the first byte (hbytes[4]) is 2 and the remaining bytes are 0. Why would you then want to left shift that byte by 24? The byte you want to left shift by 24 is the high-order byte, hbytes[7].
Related
What I want to do: read a series of 4 bytes e.g. 00000000 00000011 00000001 00000011 (this is a random example) from a binary file, and represent it as an integer in my program. What is the best way to do this?
EDIT SOLUTION I overlooked this part of the spec for the PNG file format here, hopefully this is useful to anyone that finds the question.
I am experimenting with the PNG image format and am having trouble extracting a 4 byte number. I Have succeeded in opening and printing the binary representation of the file, so I know that the data I am working with isn't corrupted or malformed.
I have reviewed questions like Reading 16-bit integers from binary file c++, and the 32 bit equivalent(s) but I cannot discern if they are reading integers that are in a binary file e.g. 00000000 72 00000000 or reading bytes as integers, which is what my goal is.
As an example, the first four bytes of the first chunk are 00000000 00000000 00000000 00001101 or 13.
Following the example of questions like the one above, this should == 13:
int test;
img.read( (char*) &test, sizeof(test));
yet it outputs 218103808
I also tried the approach of using a union with a character array and integer data member, and got the same output of 218103808
also, on my system sizeof(int) is equal to 4
And lastly, just to be sure that it wasn't a malformed PNG (which it wasn't I am rather sure) I used gimp to import it then export it as a new file, therefore natively created on my system.
EDIT
As I mentioned, after seekg(8) the next four bytes are 00000000 00000000 00000000 00001101 but when I decided to test the read function using
bitset<32> num;
img.read( (char*) &num, sizeof(int) );
it outputs 00001101 00000000 00000000 00000000
I am simply confused by this part, here. It's as if the bytes are reversed here. And this string of bytes equates to 218103808
Any insight would be appreciated
Notice that 218103808 is 0x0D000000 in hex. You might want to read about Endianess
That means the data you are reading is in big endian format, while your platform uses little endian.
Basically you need to reverse the 4 bytes, (and you likely want to use unsigned integers), so you get 0x0000000D, (13 decimal) which you can do like:
#define BSWAPUINT(x) ((((x) & 0x000000ff) << 24) |\
(((x) & 0x0000ff00) << 8) |\
(((x) & 0x00ff0000) >> 8) |\
(((x) & 0xff000000) >> 24))
unsigned int test;
img.read( (char*) &test, sizeof(test));
test = BSWAPUINT(test);
The above code will only work if the code runs on a little endian platform though.
To have your code be independent on whether your platform is big or little endian you can assemble the bytes to an integer yourself, given that you know the data format is big endian, you can do:
unsigned char buf[4];
unsigned int test;
img.read( (char*) &test, sizeof(test));
test = (unsigned int)buf[0] << 24;
test |= buf[1] << 16;
test |= buf[2] << 8;
test |= buf[3];
Or, on unix systems you can #include <arpa/inet.h> and use ntohl()
test = ntohl(test);
(Dealing with data in this manner, you are also better of using types such as uint32_t instead of int/unsigned int's , from stdint.h )
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 years ago.
Improve this question
I am an amateur at software programming.
However, I have a task to convert a dataframe that is 40 bytes long and in hex values to be converted to binary values and subsequently into decimal values. I tried converting the values from hex to binary after reading them byte by byte. It didn't work quite effectively as some of the data in the frame are not constituted of a single byte.
Let me explain a little in detail. I have a 40 byte long data frame that reads in hex like this:
0 40 ffffff82 2 0 0 28 6d
ffffffaf ffffffc8 0 41 0 8 78 8
72 17 16 16 0 42 0 2
1 2 1 16 ffffffff ffffffff 0 43
0 0 3 0 0 2 8 0
The reason I do not prefer converting these data by reading one byte at a time is because every byte displayed may not essentially imply a meaning. Please read on to understand what I mean by this.
For example:
1st to 6th byte represent data that are just 1 byte each. 1st byte is status, 2nd byte is unit voltage, 3rd being unit current and so forth.
Whereas when it comes to 7th and 8th byte represent a 2 byte data, unit SOC, meaning, unit SOC is a 16 bit data.
9th, 10th and 11th byte together indicate Module 1 cell failure information, i.e, the failure information is a 24 bit data.
12th,13th and 14th byte together indicate Module 2 cell failure information etc.
This being the case, how can I convert the incoming data frame into binary and subsequently to decimal without reading them byte after byte.
I would appreciate if this is something someone may be able to lend a helping hand with.
Suppose you have read your data frame into a buffer like this:
unsigned char inputbuffer[40];
Set a pointer pointing to the beginning of the buffer:
unsigned char *p = inputbuffer;
You can extract single-byte fields trivially:
int status = *p++; /* first byte */
int maxvoltage = *p++; /* second byte */
int current = *p++; /* third byte */
A two-byte field is only slightly more complicated:
int soc = *p++;
soc = (soc << 8) | *p++;
This reads two bytes for soc, concatenating them together as firstbyte+secondbyte. That assumes that the data frame uses what's called "big endian" byte order (that is, most-significant or "biggest" byte first). If that gives you crazy values, it's likely that the data uses "little endian" order, in which case you can flip the bytes around, yielding secondbyte+firstbyte, by reading them like this instead:
int soc = *p++;
soc = soc | *p++ << 8);
Alternatively, you can dispense with the pointer p, and access various bytes out of the inputbuffer array directly, although in that case you need to remember that arrays in C are 0-based:
int status = inputbuffer[0]; /* first byte */
int maxvoltage = inputbuffer[1]; /* second byte */
int current = inputbuffer[2]; /* third byte */
int soc = (inputbuffer[6] << 8) | inputbuffer[7];
or
int soc = inputbuffer[6] | (inputbuffer[7] << 8);
You can almost follow the same pattern for your 24-bit fields, except that for portability (and especially if you're on an old 16-bit machine) you need to take care to use a long int:
long int module_1_cell_failure = *p++;
module_1_cell_failure = (module_1_cell_failure << 8) | *p++;
module_1_cell_failure = (module_1_cell_failure << 8) | *p++;
or
long int module_1_cell_failure = *p++;
module_1_cell_failure |= (*p++ << 8);
module_1_cell_failure |= ((unsigned long)*p++ << 16);
or
long int module_1_cell_failure =
inputbuffer[8] | (inputbuffer[9] << 8) |
((unsigned long)inputbuffer[10] << 16);
The code that I'm using for reading .wav file data into an 2D array:
int signal_frame_width = wavHeader.SamplesPerSec / 100; //10ms frame
int total_number_of_frames = numSamples / signal_frame_width;
double** loadedSignal = new double *[total_number_of_frames]; //array that contains the whole signal
int iteration = 0;
int16_t* buffer = new int16_t[signal_frame_width];
while ((bytesRead = fread(buffer, sizeof(buffer[0]), signal_frame_width, wavFile)) > 0)
{
loadedSignal[iteration] = new double[signal_frame_width];
for(int i = 0; i < signal_frame_width; i++){
//value normalisation:
int16_t c = (buffer[i + 1] << 8) | buffer[i];
double normalisedValue = c/32768.0;
loadedSignal[iteration][i] = normalisedValue;
}
iteration++;
}
The problem is in this part, I don't exaclty understand how it works:
int16_t c = (buffer[i + 1] << 8) | buffer[i];
It's example taken from here.
I'm working on 16bit .wav files only. As you can see, my buffer is loading (for ex. sampling freq. = 44.1kHz) 441 elements (each is 2byte signed sample). How should I change above code?
The original example, from which you constructed your code, used an array where each individual element represented a byte. It therefore needs to combine two consecutive bytes into a 16-bit value, which is what this line does:
int16_t c = (buffer[i + 1] << 8) | buffer[i];
It shifts the byte at index i+1 (here assumed to be the most significant byte) left by 8 positions, and then ORs the byte at index i onto that. For example, if buffer[i+1]==0x12 and buffer[i]==0x34, then you get
buffer[i+1] << 8 == 0x12 << 8 == 0x1200
0x1200 | buffer[i] == 0x1200 | 0x34 == 0x1234
(The | operator is a bitwise OR.)
Note that you need to be careful whether your WAV file is little-endian or big-endian (but the original post explains that quite well).
Now, if you store the resulting value in a signed 16-bit integer, you get a value between −32768 and +32767. The point in the actual normalization step (dividing by 32768) is just to bring the value range down to [−1.0, 1.0).
In your case above, you appear to already be reading into a buffer of 16-bit values. Note that your code will therefore only work if the endianness of your platform matches that of the WAV file you are working with. But if this assumption is correct, then you don't need the code line which you do not understand. You can just convert every array element into a double directly:
double normalisedValue = buffer[i]/32768.0;
If buffer was an array of bytes, then that piece of code would interpret two consecutive bytes as a single 16-bit integer (assuming little-endian encoding). The | operator will perform a bit-wise OR on the bits of the two bytes. Since we wish to interpret the two bytes as a single 2-byte integer, then we must shift the bits of one of them 8 bits (1 byte) to the left. Which one depends on whether they are ordered in little-endian or big-endian order. Little-endian means that the least significant byte comes first, so we shift the second byte 8 bits to the left.
Example:
First byte: 0101 1100
Second byte: 1111 0100
Now shift second byte:
Second "byte": 1111 0100 0000 0000
First "byte": 0000 0000 0101 1100
Bitwise OR-operation (if either is 1, then 1. If both are 0, then 0):
16-bit integer: 1111 0100 0101 1100
In your case however, the bytes in your file have already been interpreted as 16-bit ints using whatever endianness the platform has. So you do not need this step. However, in order to correctly interpret the bytes in the file, one must assume the same byte-order as they were written in. Therefore, one usually adds this step to ensure that the code works independent of the endianness of the platform, instead relying on the expected byte-order of the files (as most file formats will specify what the byte-order should be).
I have a requirement, where 3 bytes (24 bits) need to be populated in a binary protocol. The original value is stored in an int (32 bits). One way to achieve this would be as follows:-
Technique1:-
long x = 24;
long y = htonl(x);
long z = y>>8;
memcpy(dest, z, 3);
Please let me know if above is the correct way to do it?
The other way, which i dont understand was implemented as below
Technique2:-
typedef struct {
char data1;
char data2[3];
} some_data;
typedef union {
long original_data;
some_data data;
} combined_data;
long x = 24;
combined_data somedata;
somedata.original_data = htonl(x);
memcpy(dest, &combined_data.data.data2, 3);
What i dont understand is, how did the 3 bytes end up in combined_data.data.data2 as opposed to first byte should go into combined_data.data.data1 and next 2 bytes should go into
combined_data.data.data2?
This is x86_64 platform running 2.6.x linux and gcc.
PARTIALLY SOLVED:-
On x86_64 platform, memory is addressed from right to left. So a variable of type long with value 24, will have following memory representation
|--Byte4--|--Byte3--|--Byte2--|--Byte1--|
0 0 0 0x18
With htonl() performed on above long type, the memory becomes
|--Byte4--|--Byte3--|--Byte2--|--Byte1--|
0x18 0 0 0
In the struct some_data, the
data1 = Byte1
data2[0] = Byte2
data2[1] = Byte3
data4[2] = Byte4
But my Question still holds, Why not simply right shift by 8 as shown in technique 1 ?
A byte takes 8 bits :-)
int x = 24;
int y = x<<8;
moving by 0 you are changing nothing. By 1 - *2, by 2 - *4, by 8 - *256.
if we are on the BIG ENDIAN machine, 4 bytes are put in memory as so: 2143. And such algorythms won't work for numbers greater than 2^15. On the other way, on the BIG ENDIAN machine you should define, what means " putting integer in 3 bytes"
Hmm. I think, the second proposed algorythm will be ok, but change the order of bytes:
You have them as 2143. You need 321, I think. But better check it.
Edit: I checked on wiki - x86 is little endian, they say, so algorythms are OK
I was searching for a way to efficiently pack my data in order to send them over a network.
I found a topic which suggested a way : http://www.sdltutorials.com/cpp-tip-packing-data
And I've also seen it being used in commercial applications. So I decided to give it a try, but the results weren't what I expected.
First of all , the whole point of "packing" your data is to save bytes. But I don't think that the algorithm mentioned above is saving bytes at all.
Because , without packing ... The server would send 4 bytes (Data) , after the packing the server sends a character array , 4 bytes long ... So it's pointless.
Aside from that , why would someone add 0xFF , it doesn't do anything at all.
The code snippet found in the tutorial mentioned above:
unsigned char Buffer[3];
unsigned int Data = 1024;
unsigned int UpackedData;
Buffer[0] = (Data >> 24) & 0xFF;
Buffer[1] = (Data >> 12) & 0xFF;
Buffer[2] = (Data >> 8) & 0xFF;
Buffer[3] = (Data ) & 0xFF;
UnpackedData = (Buffer[0] << 24) | (Buffer[1] << 12) | (Buffer[2] << 8) | (Buffer[3] & 0xFF);
Result:
0040 // 4 bytes long character
1024 // 4 bytes long
The & 0xFF is to make sure it's between 0 and 255.
I wouldn't place too much credence in that posting; aside from your objection, the code contains an obvious mistake. Buffer is only 3 elements long, but the code stores data in 4 elements.
For integers a simple method I found often useful is BER encoding. Basically for an unsigned integer you write 7 bits for each byte, using the 8th bit to mark if another byte is needed
void berPack(unsigned x, std::vector<unsigned char>& out)
{
while (x >= 128)
{
out.push_back(128 + (x & 127)); // write 7 bits, 8th=1 -> more needed
x >>= 7;
}
out.push_back(x); // Write last bits (8th=0 -> this ends the number)
}
for a signed integer you encode the sign in the least significant bit and the use the same encoding as before
void berPack(int x, std::vector<unsigned char>& out)
{
if (x < 0) berPack((unsigned(-x) << 1) + 1, out);
else berPack((unsigned(x) << 1), out);
}
With this approach small numbers will use less space. Another advantage is that this encoding is already architecture-neutral (i.e. data will be understood correctly independently on the endian-ness of the system) and that the same format can handle different integer sizes and you can send data from a 32 bit system to a 64 bit system without problems (assuming of course that the values themselves are not overflowing).
The price to pay is that for example unsigned values from 268435456 (1 << 28) to 4294967295 ((1 << 32) - 1) will require 5 bytes instead of 4 bytes of standard fixed 4-bytes packing.
Another reason for packing is to enforce a consistent structure, so that data written by one machine can be reliably read by another.
It's not "adding"; it's performing a bitwise-AND in order to mask out the LSB (least-significant byte). But it doesn't look necessary here.