How to byteswap a double? - c++

Question

I'm trying to write a byteswap routine for a C++ program running on Win XP. I'm compiling with Visual Studio 2008. This is what I've come up with:
int byteswap(int v) // This is good
{
return _byteswap_ulong(v);
}
double byteswap(double v) // This doesn't work for some values
{
union { // This trick is first used in Quake2 source I believe :D
__int64 i;
double d;
} conv;
conv.d = v;
conv.i = _byteswap_uint64(conv.i);
return conv.d;
}
And a function to test:
void testit() {
double a, b, c;
CString str;
for (a = -100; a < 100; a += 0.01) {
b = byteswap(a);
c = byteswap(b);
if (a != c) {
str.Format("%15.15f %15.15f %15.15f", a, c, a - c);
}
}
}
Getting these numbers not matching:
-76.789999999988126 -76.790000000017230 0.000000000029104
-30.499999999987718 -30.499999999994994 0.000000000007276
41.790000000014508 41.790000000029060 -0.000000000014552
90.330000000023560 90.330000000052664 -0.000000000029104
This is after having read through:
How do I convert between big-endian and little-endian values in C++?
Little Endian - Big Endian Problem
You can't use << and >> on double, by the way (unless I'm mistaken?)

Answer 1

Although a double in main memory is 64 bits, on x86 CPUs double-precision registers are 80 bits wide. So if one of your values is stored in a register throughout, but the other makes a round-trip through main memory and is truncated to 64 bits, this could explain the small differences you're seeing.
Maybe you can force variables to live in main memory by taking their address (and printing it, to prevent the compiler from optimizing it out), but I'm not certain that this is guaranteed to work.

Answer 2

b = byteswap(a);
That's a problem. After swapping the bytes, the value is no longer a proper double. Storing it back to a double is going to cause subtle problems when the FPU normalizes the value. You have to store it back into an __int64 (long long). Modify the return type of the method.

Answer 3

Try 3
Okay, found out there's a better way. The other way you have to worry about the order you pack/unpack stuff. This way you don't:
// int and float
static void swap4(void *v)
{
char in[4], out[4];
memcpy(in, v, 4);
out[0] = in[3];
out[1] = in[2];
out[2] = in[1];
out[3] = in[0];
memcpy(v, out, 4);
}
// double
static void swap8(void *v)
{
char in[8], out[8];
memcpy(in, v, 8);
out[0] = in[7];
out[1] = in[6];
out[2] = in[5];
out[3] = in[4];
out[4] = in[3];
out[5] = in[2];
out[6] = in[1];
out[7] = in[0];
memcpy(v, out, 8);
}
typedef struct
{
int theint;
float thefloat;
double thedouble;
} mystruct;
static void swap_mystruct(void *buf)
{
mystruct *ps = (mystruct *) buf;
swap4(&ps->theint);
swap4(&ps->thefloat);
swap8(&ps->thedouble);
}
Send:
char buf[sizeof (mystruct)];
memcpy(buf, &s, sizeof (mystruct));
swap_mystruct(buf);
Recv:
mystruct s;
swap_mystruct(buf);
memcpy(&s, buf, sizeof (mystruct));

Answer 4

Try 2
Okay, got it working! Hans Passant was right. They got me thinking with the "no longer a proper double" comment. So you can't byteswap a float into another float because then it might be in an improper format, so you have to byteswap to a char array and unswap back. This is the code I used:
int pack(int value, char *buf)
{
union temp {
int value;
char c[4];
} in, out;
in.value = value;
out.c[0] = in.c[3];
out.c[1] = in.c[2];
out.c[2] = in.c[1];
out.c[3] = in.c[0];
memcpy(buf, out.c, 4);
return 4;
}
int pack(float value, char *buf)
{
union temp {
float value;
char c[4];
} in, out;
in.value = value;
out.c[0] = in.c[3];
out.c[1] = in.c[2];
out.c[2] = in.c[1];
out.c[3] = in.c[0];
memcpy(buf, out.c, 4);
return 4;
}
int pack(double value, char *buf)
{
union temp {
double value;
char c[8];
} in, out;
in.value = value;
out.c[0] = in.c[7];
out.c[1] = in.c[6];
out.c[2] = in.c[5];
out.c[3] = in.c[4];
out.c[4] = in.c[3];
out.c[5] = in.c[2];
out.c[6] = in.c[1];
out.c[7] = in.c[0];
memcpy(buf, out.c, 8);
return 8;
}
int unpack(char *buf, int *value)
{
union temp {
int value;
char c[4];
} in, out;
memcpy(in.c, buf, 4);
out.c[0] = in.c[3];
out.c[1] = in.c[2];
out.c[2] = in.c[1];
out.c[3] = in.c[0];
memcpy(value, &out.value, 4);
return 4;
}
int unpack(char *buf, float *value)
{
union temp {
float value;
char c[4];
} in, out;
memcpy(in.c, buf, 4);
out.c[0] = in.c[3];
out.c[1] = in.c[2];
out.c[2] = in.c[1];
out.c[3] = in.c[0];
memcpy(value, &out.value, 4);
return 4;
}
int unpack(char *buf, double *value)
{
union temp {
double value;
char c[8];
} in, out;
memcpy(in.c, buf, 8);
out.c[0] = in.c[7];
out.c[1] = in.c[6];
out.c[2] = in.c[5];
out.c[3] = in.c[4];
out.c[4] = in.c[3];
out.c[5] = in.c[2];
out.c[6] = in.c[1];
out.c[7] = in.c[0];
memcpy(value, &out.value, 8);
return 8;
}
And a simple test function:
typedef struct
{
int theint;
float thefloat;
double thedouble;
} mystruct;
void PackStruct()
{
char buf[sizeof (mystruct)];
char *p;
p = buf;
mystruct foo, foo2;
foo.theint = 1;
foo.thefloat = 3.14f;
foo.thedouble = 400.5;
p += pack(foo.theint, p);
p += pack(foo.thefloat, p);
p += pack(foo.thedouble, p);
// Send or recv char array
p = buf;
p += unpack(p, &foo2.theint);
p += unpack(p, &foo2.thefloat);
p += unpack(p, &foo2.thedouble);
}

Answer 5

How to swap the bytes in any basic data type or array of bytes
ie: How to swap the bytes in place in any array, variable, or any other memory block, such as an int16_t, uint16_t, uint32_t, float, double, etc.:
Here's a way to improve the efficiency from 3 entire copy operations of the array to 1.5 entire copy operations of the array. See also the comments I left under your answer. I said:
Get rid of this: memcpy(in, v, 4); and just copy-swap straight into out from v, then memcpy the swapped values back from out into v. This saves you an entire unnecessary copy, reducing your copies of the entire array from 3 to 2.
There's also a further optimization to reduce the copies of the entire array from 2 to 1.5: copy the left half of the array into temporary variables, and the right-half of the array straight into the left-half, swapping as appropriately. Then copy from the temporary variables, which contain the old left-half of the array, into the right-half of the array, swapping as appropriately. This results in the equivalent of only 1.5 copy operations of the entire array, to be more efficient. Do all this in-place in the original array, aside from the temp variables you require for half of the array.
1. Here is my general C and C++ solution:
/// \brief Swap all the bytes in an array to convert from little-endian
/// byte order to big-endian byte order, or vice versa.
/// \note Works for arrays of any size. Swaps the bytes **in place**
/// in the array.
/// \param[in,out] byte_array The array in which to swap the bytes in-place.
/// \param[in] len The length (in bytes) of the array.
/// \return None
void swap_bytes_in_array(uint8_t * byte_array, size_t len)
{
size_t i_left = 0; // index for left side of the array
size_t i_right = len - 1; // index for right side of the array
while (i_left < i_right)
{
// swap left and right bytes
uint8_t left_copy = byte_array[i_left];
byte_array[i_left] = byte_array[i_right];
byte_array[i_right] = left_copy;
i_left++;
i_right--;
}
}
Usage:
// array of bytes
uint8_t bytes_array[16];
// Swap the bytes in this array of bytes in place
swap_bytes_in_array(bytes_array, sizeof(bytes_array));
double d;
// Swap the bytes in the double in place
swap_bytes_in_array((uint8_t*)(&d), sizeof(d));
uint64_t u64;
// swap the bytes in a uint64_t in place
swap_bytes_in_array((uint8_t*)(&u64), sizeof(u64));
2. And here is an optional C++ template wrapper around that to make it even easier to use in C++:
template <typename T>
void swap_bytes(T *var)
{
// Note that `sizeof(*var)` is the exact same thing as `sizeof(T)`
swap_bytes_in_array((uint8_t*)var, sizeof(*var));
}
Usage:
double d;
// Swap the bytes in the double in place
swap_bytes(&d);
uint64_t u64;
// swap the bytes in a uint64_t in place
swap_bytes(&u64);
Notes & unanswered questions
Note, however, that #Hans Passant seems to be onto something here. Although the above works perfectly on any signed or unsigned integer type, and seems to work on float and double for me too, it seems to be broken on long double. I think it's because when I store the swapped long double back into a long double variable, if it is determined to be not-a-valid long double representation anymore, something automatically changes a few of the swapped bytes or something. I'm not entirely sure.
On many 64-bit systems, long double is 16 bytes, so perhaps the solution is to keep the swapped version of the long double inside a 16-byte array and NOT attempt to use it or cast it back to a long double from the uint8_t 16-byte array until either A) it has been sent to the receiver (where the endianness of the system is opposite, so it's in good shape now) and/or B) byte-swapped back again so it's a valid long double again.
Keep the above in mind in case you see problems with float or double types too, as I see with only long double types.
Linux byteswap and endianness and host-to-network byte order utilities
Linux also has a bunch of built-in utilities via gcc GNU extensions that you can use. See:
https://man7.org/linux/man-pages/man3/bswap.3.html - #include <byteswap.h>
https://man7.org/linux/man-pages/man3/endian.3.html - #include <endian.h>
https://man7.org/linux/man-pages/man3/byteorder.3.html - #include <arpa/inet.h> - generally used for network sockets (Ethernet packets) and things; inet stands for "internet"