If I recall correctly, it would be undefined behavior to write to FastKey::key and then read from FastKey::keyValue:
struct Key {
std::array<uint8_t, 6> MACAddress;
uint16_t EtherType;
};
union FastKey {
Key key;
uint64_t keyValue;
};
However, I have been told that if I add char array to the union then the UB is cleared:
union FastKey {
Key key;
uint64_t keyValue;
char fixUB[sizeof(Key)];
};
Is this true?
Edit
As usual my understanding was wrong. With the new information I gathered, I think that I can get the key as a uint64_t value like this:
struct Key {
std::array<uint8_t, 6> MACAddress;
uint16_t EtherType;
};
union FastKey {
Key key;
unsigned char data[sizeof(Key)];
};
inline uint64_t GetKeyValue(FastKey fastKey)
{
uint64_t key = 0;
key |= size_t(fastKey.data[0]) << 56;
key |= size_t(fastKey.data[1]) << 48;
key |= size_t(fastKey.data[2]) << 40;
key |= size_t(fastKey.data[3]) << 32;
key |= size_t(fastKey.data[4]) << 24;
key |= size_t(fastKey.data[5]) << 16;
key |= size_t(fastKey.data[6]) << 8;
key |= size_t(fastKey.data[7]) << 0;
return key;
}
I suspect that this will be equally fast as the original version. Feel free to correct me.
Update
#Steve Jessop I implemented a quick benchmark to test the performance of memcpy vs my solution. I'm not a benchmarking expert, so there may be stupid errors in the code the lead to wrong results. However, if the code is right then it would seem that memcpy is much slower.
Note: It seems the benchmark is wrong because the time to calculate the time for fast key is always zero. I'll see if I can fix it.
No, reading a uint64_t if you have a Key object there is still UB. What isn't UB is to read a char, because there's an exception for char in the aliasing rules. Adding the array doesn't propagate the exception to the other types.
The version in the edit seems fine (though I'd use unsigned char), but now it is more complex than just using a reinterpret_cast from Key* to unsigned char* or a memcpy.
Related
#include <iostream>
int main(){
uint8_t memory[1024];
memory[0] = 1;
memory[1] = 1;
uint32_t *test = memory;
//is it possible to get a value for *test that would be in this example 257?
}
I want to create a uin32_t pointer to the same adress as the uint8_t pointer. Is this possible without using new(adress)? I don't want to lose the information at the adress. I know pointers are just adresses and therefor I should be able to just set the uint32_t pointer to the same adress.
This code produces an error:
invalid conversion from 'uint8_t*' to 'uint32_t*' in initialization
This would be a violation of so-called Strict Aliasing Rule, so it can not be done. Sad, but true.
Use memcpy to copy data and in many cases compilers will optimize memory copy and generate the same code as they would with cast, but in Standard-conforming way.
As already mentioned you cannot convert uint8_t * to uint32_t * due to strict aliasing rule, you can convert uint32_t * to unsigned char * though:
#include <iostream>
int main(){
uint32_t test[1024/4] = {}; // initialize it!
auto memory = reinterpret_cast<unsigned char *>( test );
memory[0] = 1;
memory[1] = 1;
std::cout << test[0] << std::endl;
}
this is not portable code due to Endianness, but at least it does not have UB.
This question completely ignores the concept of endian-ness; while your example has the lower and upper byte the same value, if the byte order is swapped it makes no difference; but in the case where it is; your number will be wrong unexpectedly.
As such, there's no portable way to use the resulting number.
You can do that with union. As mentioned above, you have to be aware of endianness of target device, but in most cases it will be little-endian. And there is also a bit of controversy about using unions in such way, but fwiw it's getting a job done and for some uses it's good enough.
#include <iostream>
int main(){
union {
uint8_t memory[1024] = {};
uint32_t test[1024/4];
};
memory[0] = 1;
memory[1] = 1;
std::cout << test[0]; // 257
}
uint32_t *test =(uint32_t*) memory;
uint32_t shows that the memory pointed by test should contain uint32_t .
I have to admit that I am a bit confused at the moment, so sorry if the question isnt quite clear or trivial (actually I hope it is the latter)....
I am sending an array of bytes across the network and would like to do something like this on the sender side:
size_t max_size = 100;
uint8_t buffer[size];
idontknowwhat_t x{buffer};
uint16_t size = 11; // total number of bytes in the buffer
uint16_t id_a,id_b,id_c; // some ids
uint8_t a,b,c; // some data
x << size << id_a << a << id_b << b << id_c << c;
someMethodToSend(buffer,size);
and on the receiver side something like this:
size_t max_size = 100;
uint8_t buffer[size];
someMethodToReceive(buffer);
idontknowwhat_t x{buffer};
uint16_t size;
x >> size;
for (uint16_t i=0; i<size-2; i++) {
uint16_t id;
uint8_t data;
x >> id >> data;
std::cout << id << " " << data;
}
So my aim is basically to avoid ugly casts and manually incrementing a pointer while being able to have uint8_t and uint16_t (and possibly also uint32_t) in the buffer. The data I put in the buffer here is just an example, and I am aware that I need to take care of the byte order when sending over the network (and it would be fine if I had to do this "manually").
Is there something that I can use in place of my hypothetical idontknowwhat_t ?
You cannot really avoid doing ugly casts, but at least you can hide them into the idontknowwhat_t class's operator>> and operator<< functions. And using templates, you could limit the number of casts in your code to the bare minimum.
class idontknowwhat_t
{
uint8_t* _data;
public:
idontknowwhat_t(uint8_t* buffer)
: _data(buffer)
{}
template<typename insert_type>
idontknowwhat_t& operator<<(insert_type value)
{
*reinterpret_cast<insert_type*>(_data) = value;
_data += sizeof(insert_type);
return *this;
}
template<typename extract_type>
idontknowwhat_t& operator>>(extract_type& value)
{
value = *reinterpret_cast<extract_type*>(_data);
_data += sizeof(extract_type);
return *this;
}
};
I think this will actually work directly with your code. In this example, the idontknowwhat_t class does not own the buffer and simply keeps a raw pointer to the next bit of data it expects to read or write. For real-life purposes I would recommend letting the idontknowwhat_t class manage the buffer memory.
In addition, none of the code on this page actually takes care of the data's endianness, which would definitely be the idontknowwhat_t class's responsibility. There is a boost library for that. I'm not documenting that library's use here, since I think it distracts from the questions real point.
Have you tried std::list? You could group the elements into types and put them into lists with the appropriate type. Then you could create an std::list of std::lists.
Let's say I have the following
struct S {
union {
uint8_t flags;
struct {
uint8_t flag2bits : 2;
uint8_t flag1bit : 1;
};
};
};
S s;
s.flag2bits = 2;
s.flag1bit = 1; // this will wipe out the values of other bits
What's the best way to assign value to a specific bit without affecting other bit fields?
I can shift around and then assign and then shift again but it means once someone changes the order of the bit fields, the code is broken....
I can shift around and then assign and then shift again but it means
once someone changes the order of the bit fields, the code is
broken....
No, it doesn't mean the code is broken. You can change the bitfields whatever (in any order/you can leave some of them unset) you like
In your example:
S s;
s.flag2bits = 2;
s.flag1bit = 1;
Changing flag2bits will not affect value stored in flag1bit.
However, your problem may be related to the union you hold in your struct. Changing the flags variable will affect both of the bitfields, as you are storing them in a separate struct.
I hope this example will explain the case here:
#include <iostream>
#include <cstdint>
struct S {
union {
uint8_t flags;
struct {
uint8_t flag2bits : 2;
uint8_t flag1bit : 1;
};
};
};
int main(int argc, char *argv[]) {
S s;
s.flag2bits = 2;
s.flag1bit = 1;
std::cout << int(s.flag2bits) << int(s.flag1bit) << std::endl;
s.flags = 4; // As you are using union, at this point you are overwriting
// values stored in your (nested) struct
std::cout << int(s.flag2bits) << int(s.flag1bit) << std::endl;
return 0;
}
EDIT: As #M.M points out, it's undefined behavior to read from the member of the union that wasn't most recently written. Though at least on clang-3.5, the code above would print:
21
01
which illustrates the point I am trying to make (i.e. overwriting of union fields).
I would consider removing union from your struct S code, though I may not see the whole picture of what you are trying to achieve.
The C++ compiler will manage the bits for you. You can just set the values as you have it. Only the appropriate bits will be set.
Did you try it?
So, there are a few questions on SO about this subject, but I haven't quite found something that exactly answers the question I have in mind. First some background:
I would like to have a uint32_t field, which I can also access as an array of bytes.
So the first thing that comes to mind is:
union U {
uint32_t u32;
uint8_t bytes[sizeof(uint32_t)];
};
Which allows me to do this:
// "works", but is UB as far as I understand
U u;
u.u32 = 0x11223344;
u.bytes[0] = 0x55;
OK, so undefined behavior (UB) is bad, therefore we don't want to do that. Similarly casts are UB and can sometimes be even worse due to alignment concerns (though not in this case because I'm using a char sized object for my array).
// "works", but is UB as far as I understand
uint32_t v = 0x11223344;
auto p = reinterpret_cast<uint8_t *>(&v);
p[0] = 0x55;
Once again, UB is bad, therefore we don't want to do that.
Some say that this is OK if we use a char* instead of a uint8_t*:
// "works", but maybe is UB?
uint32_t v = 0x11223344;
auto p = reinterpret_cast<char *>(&v);
p[0] = 0x55;
But I am honestly not sure about it... So getting creative.
So, I think I remember it being legal (as far as I know) to read the contents of a void* cast to a char* (this allows things like std::memcpy to not be UB). So maybe we can kinda play with this:
uint8_t get_byte(const void *p, size_t n) {
auto ptr = static_cast<const char *>(p);
return ptr[n];
}
void set_byte(void *p, size_t index, uint8_t v) {
auto ptr = static_cast<char *>(p);
ptr[index] = v;
}
// "works", is this UB?
uint32_t v = 0x11223344;
uint8_t v1 = get_byte(&v, 0); // read
set_byte(&v, 0, 0x55); // write
So my questions are:
Is the final example I came up with UB?
If it is, what is the "right" way to do this? I really hope the "correct" way isn't a memcpy to and from a byte array. That would be ridiculous.
(BONUS): suppose I want my get_byte to return a reference (like for implementing operator[]. Is it safe to use uint8_t instead of literal char when reading a the contents of a void *?
NOTE: I understand the concerns regarding endian and portability. They are not a problem for my use case. I think that it is acceptable for the result to be an "unspecified value" (in that it is compiler specific which byte it will read). My question is really focused on the UB aspects ("nasal demons" and similar).
Why not create a class for that ?
Something like:
class MyInt32 {
public:
std::uint32_t asInt32() const {
return b[0]
| (b[1] << 8)
| (b[2] << 16)
| (b[3] << 24);
}
void setInt32(std::uint32 i) {
b[0] = (i & 0xFF);
b[1] = ((i >> 8) & 0xFF);
b[2] = ((i >> 16) & 0xFF);
b[3] = ((i >> 24) & 0xFF);
}
const std::array<std::uint8_t, 4u>& asInt8() const { return b; }
std::array<std::uint8_t, 4u>& asInt8() { return b; }
void setInt8s(const std::array<std::uint8_t, 4u>& a) { b = a; }
private:
std::array<std::uint8_t, 4u> b;
};
So you don't have UB, you don't break aliasing rules, you manage endianess as you want.
It's perfectly legit (as long as the type is a POD), and uint8_t is not guaranteed to be legal so don't.
I'm looking for an efficient way to hash a 6-byte field so that it can be used for std::unordered_map .
I think this would be the conventional way of creating a hash:
struct Hash {
std::size_t operator()(const std::array<uint8_t, 6> & mac) const {
std::size_t key = 0;
boost::hash_combine(key, mac[0]);
boost::hash_combine(key, mac[1]);
boost::hash_combine(key, mac[2]);
boost::hash_combine(key, mac[3]);
boost::hash_combine(key, mac[4]);
boost::hash_combine(key, mac[5]);
return key;
}
};
However I noticed that I can make it a little faster (~20%) using this trick:
struct Hash {
std::size_t operator()(const std::array<uint8_t, 6> & mac) const {
std::size_t key = 0;
// Possibly UB?
boost::hash_combine(key, reinterpret_cast<const uint32_t&>(mac[0]));
boost::hash_combine(key, reinterpret_cast<const uint16_t&>(mac[4]));
return key;
}
};
And this was even faster:
struct Hash {
std::size_t operator()(const std::array<uint8_t, 6> & mac) const {
// Requires size_t to be 64-bit.
static_assert(sizeof(std::size_t) >= 6, "MAC address doesn't fit in std::size_t!");
std::size_t key = 0;
// Likely UB?
boost::hash_combine(key, 0x0000FFFFFFFFFFFF & reinterpret_cast<const uint64_t&>(mac[0]));
return key;
}
};
My question is two-fold:
Are these optimizations going to result in UB?
Is my first solution the way to go? Or is there a better way?
Your optimizations are breaking the strict aliasing rules, which leads (standardly speaking) to undefined behavior.
The last optimization worries me the most since you are essentially reading memory you ought not to, which may provoke traps if this memory happened to be protected.
Any reason you are not using boost::hash_range ?
Since boost::hash_range turns out not to be as fast as required, I would propose another solution, based on aliasing. Or rather, two solutions in one.
The first idea is that aliasing can be subdued using char* as a temporary type.
size_t key = 0;
char* k = &reinterpret_cast<char*>(&key);
std::copy(mac.begin(), mac.end(), k);
return key;
is therefore a valid implementation of the hash.
However, we can go one step further. Because of alignment and padding, storing a char[6] and char[8] are likely to use the same amount of memory within a map node. Therefore, we could enrich the type, by using union:
union MacType {
unsigned char value[8];
size_t hash;
};
Now, you can encapsulate this properly within a class (and make sure you always initialize the bytes 7 and 8 to 0), and implement the interface of std::array<unsigned char, 6> that you actually need.
I've used a similar trick for tiny strings (below 8 characters) for hashing and fast (non-alphabetic) comparisons and it's really sweet.