I am trying to perform arithmetic on an address. I need to make sure that it is on a 4 byte word boundary. This requires me to get an unsigned integer representation of the pointer to perform some math on. I have done this:
//memStartAddr is of type void* (from an external API function)
data32 tempAddr = reinterpret_cast<data32>(memStartAddr);
//adjust pointer to the next word boundary if needed
tempAddr = wordAlign(tempAddr);
//memStartAddress is a class variable of type unsigned char*
memStartAddress = reinterpret_cast<data8*>(tempAddr);
//calculate bytes lost due to above adjustment
data32 delta = (tempAddr - reinterpret_cast<data32>(memStartAddress));
//Take advantage of integer arithmetic to ensure usable size is a multiple
//of 4 bytes. Remainders are lost. Shrinks usable size if necessary.
memSize = ((size - delta) / 4) * 4;
This all works in my tests, however, using reinterpret_cast is considered a forbidden practice. Is there another way to do this? Or is an exception to the rule warranted here?
You're looking for memStartAddress = std::align(4, objSize, memStartAddr, objSize+3);. The standard function is a bit more flexible than you need, so you need to tell it that you want at most a +3 change in address.
a tricky approach might be to have pointer to initial address
then on the pointer perform + 1 --> this will automatically result in the next address according to the size in bytes of the variable type it's pointing to.
Note: make sure you perform the +1 on the pointer not on the address (int* initialAddress --> perform the operation on initialAddress not on *initialAddress)
then you can perform a subtraction between the two and you'll have the number of bytes.
I suggest you avoid division.
Try this:
unsigned int pointer_value = (unsigned int) pointer;
if ((pointer_value & 0x3U) == 0U)
{
// Pointer is on 4 byte boundary.
}
Some compilers may only optimize division by 4 and multiplication by 4 when the optimization levels are set on high. The embedded compiler I'm using will call the division function when dividing by 4 rather than right shifting, unless the optimization setting is cranked up to high.
Related
Please take a look at this snippet of code:
uint16_t buffer_array[2]; //Two uint16_t = 4 bytes
temp = swSerial.read(); // Populate with serial readings coming from the sensor
switch(received_counter)
{
case 3: //when we receive the fourth byte from the sensor
buffer_array[0] = temp;
break;
case 4: //fifth byte
buffer_array[0] += temp;
break;
case 5: //sixth byte
buffer_array[1] = temp;
break;
case 6: //seventh byte
buffer_array[1] += temp;
break;
default:
break;
}
This code populates the array I declared with readings from a source. When this ends, the array buffer_array[2] will have its 4 bytes populated (or 8 hexadecimal values). This is straightforward.
But now we have four raw bytes stored in the array. These four bytes represent a floating point value (i know that from documentation). So we want these 4 bytes (8 hex values) converter to a floating point value.
I can do this with:
float voltage;
memcpy(&voltage, &buffer_array, sizeof(float));
My question is why does this work? memcpy() copies n characters from memory area src to memory area dest.
Does this imply an implicit casting when the values in the memory area is not the same as the value in the destination variable? Am i missing something?
Does this imply an implicit casting when the values in the memory area is not the same as the value in the destination variable?
No, memcpy doesn't even know the source and destination types. It just takes two void pointers.
Afterwards, the destination contains the same bit pattern as the source - that's all.
Types are only how we interpret a bit pattern. Presumably your serial device has contrived to encode a valid IEEE 754 single-precision binary32 in the protocol it sends you.
This use of memcpy is the easiest correct way to convert between the type convenient for deserializing the bit pattern, and the type convenient for interpreting it. The bit pattern itself is unchanged.
If you just hexdump (or otherwise inspect in your debugger) the contents of buffer_array and voltage, you can verify this.
Note that there are older popular approaches, but they rely on type-punning and have problems:
union { uint16_t x[2]; float y; };
prohibited by strict aliasing (ie, allowing this to work would inhibit loads of other optimizations) and union subobject lifetime rules
float *f = reinterpret_cast<float *>(buffer_array);
problems with both strict aliasing and possibly alignment, if alignof(uint16_t[2]) < alignof(float)
memcpy() works strictly with bytes, it has no clue what they represent, or what is or isn't valid in the context of the destination. You point it at some memory, you point it at a destination, and you tell it how many bytes to copy. It does that as quickly as it can.
If you want to convert an arbitrary chunk of memory into a floating point value you can just recast it as such, at least if you're confident the result is valid.1 This is as easy as:
float x = *((float*) &buffer_array[0]);
Where buffer_array is a pointer to your assembled data.
By the way, I'm not sure the floating point value you're constructing is valid as the format is not split evenly into two 16-bit chunks. What you have there looks like some kind of fixed point representation.
--
1 Don't forget to account for any endian issues for both the source data, and your system's architecture.
I know it is an integer type that can be cast to/from pointer without loss of data, but why would I ever want to do this? What advantage does having an integer type have over void* for holding the pointer and THE_REAL_TYPE* for pointer arithmetic?
EDIT
The question marked as "already been asked" doesn't answer this. The question there is if using intptr_t as a general replacement for void* is a good idea, and the answers there seem to be "don't use intptr_t", so my question is still valid: What would be a good use case for intptr_t?
The primary reason, you cannot do bitwise operation on a void *, but you can do the same on a intptr_t.
On many occassion, where you need to perform bitwise operation on an address, you can use intptr_t.
However, for bitwise operations, best approach is to use the unsigned counterpart, uintptr_t.
As mentioned in the other answer by #chux, pointer comparison is another important aspect.
Also, FWIW, as per C11 standard, §7.20.1.4,
These types are optional.
There's also a semantic consideration.
A void* is supposed to point to something. Despite modern practicality, a pointer is not a memory address. Okay, it usually/probably/always(!) holds one, but it's not a number. It's a pointer. It refers to a thing.
A intptr_t does not. It's an integer value, that is safe to convert to/from a pointer so you can use it for antique APIs, packing it into a pthread function argument, things like that.
That's why you can do more numbery and bitty things on an intptr_t than you can on a void*, and why you should be self-documenting by using the proper type for the job.
Ultimately, almost everything could be an integer (remember, your computer works on numbers!). Pointers could have been integers. But they're not. They're pointers, because they are meant for different use. And, theoretically, they could be something other than numbers.
The uintptr_t type is very useful when writing memory management code. That kind of code wants to talk to its clients in terms of generic pointers (void *), but internally do all kinds of arithmetic on addresses.
You can do some of the same things by operating in terms of char *, but not everything, and the result looks like pre-Ansi C.
Not all memory management code uses uintptr_t - as an example, the BSD kernel code defines a vm_offset_t with similar properties. But if you are writing e.g. a debug malloc package, why invent your own type?
It's also helpful when you have %p available in your printf, and are writing code that needs to print pointer sized integral variables in hex on a variety of architectures.
I find intptr_t rather less useful, except possibly as a way station when casting, to avoid the dread warning about changing signedness and integer size in the same cast. (Writing portable code that passes -Wall -Werror on all relevant architectures can be a bit of a struggle.)
What is the use of intptr_t?
Example use: order comparing.
Comparing pointers for equality is not a problem.
Other compare operations like >, <= may be UB. C11dr §6.5.8/5 Relational operators.
So convert to intptr_t first.
[Edit] New example: Sort an array of pointers by pointer value.
int ptr_cmp(const void *a, const void *b) {
intptr_t ia = (intptr) (*((void **) a));
intptr_t ib = (intptr) (*((void **) b));
return (ia > ib) - (ia < ib);
}
void *a[N];
...
qsort(a, sizeof a/sizeof a[0], sizeof a[0], ptr_cmp);
[Former example]
Example use: Test if a pointer is of an array of pointers.
#define N 10
char special[N][1];
// UB as testing order of pointer, not of the same array, is UB.
int test_special1(char *candidate) {
return (candidate >= special[0]) && (candidate <= special[N-1]);
}
// OK - integer compare
int test_special2(char *candidate) {
intptr_t ca = (intptr_t) candidate;
intptr_t mn = (intptr_t) special[0];
intptr_t mx = (intptr_t) special[N-1];
return (ca >= mn) && (ca <= mx);
}
As commented by #M.M, the above code may not work as intended. But at least it is not UB. - just non-portably functionality. I was hoping to use this to solve this problem.
(u)intptr_t is used when you want to do arithmetic on pointers, specifically bitwise operations. But as others said, you'll almost always want to use uintptr_t because bitwise operations are better done in unsigned. However if you need to do an arithmetic right shift then you must use intptr_t1. It's usually used for storing data in the pointer, usually called tagged pointer
In x86-64 you can use the high 16/7 bits of the address for data, but you must do sign extension manually to make the pointer canonical because it doesn't have a flag for ignoring the high bits like in ARM2. So for example if you have char* tagged_address then you'll need to do this before dereferencing it
char* pointer = (char*)((intptr_t)tagged_address << 16 >> 16);
The 32-bit Chrome V8 engine uses smi (small integer) optimization where the low bits denote the type
|----- 32 bits -----|
Pointer: |_____address_____w1| # Address to object, w = weak pointer
Smi: |___int31_value____0| # Small integer
So when the pointer's least significant bit is 0 then it'll be right shifted to retrieve the original 31-bit signed int
int v = (intptr_t)address >> 1;
For more information read
Using the extra 16 bits in 64-bit pointers
Pointer magic for efficient dynamic value representations
Another usage is when you pass a signed integer as void* which is usually done in simple callback functions or threads
void* my_thread(void *arg)
{
intptr_t val = (intptr_t)arg;
// Do something
}
int main()
{
pthread_t thread1;
intptr_t some_val = -2;
int r = pthread_create(&thread1, NULL, my_thread, (void*)some_val);
}
1 When the implementation does arithmetic shift on signed types of course
2 Very new x86-64 CPUs may have UAI/LAM support for that
Is there a way in C/C++ to cast a char array to an int at any position?
I tried the following, bit it automatically aligns to the nearest 32 bits (on a 32 bit architecture) if I try to use pointer arithmetic with non-const offsets:
unsigned char data[8];
data[0] = 0; data[1] = 1; ... data[7] = 7;
int32_t p = 3;
int32_t d1 = *((int*)(data+3)); // = 0x03040506 CORRECT
int32_t d2 = *((int*)(data+p)); // = 0x00010203 WRONG
Update:
As stated in the comments the input comes in tuples of 3 and I cannot
change that.
I want to convert 3 values to an int for further
processing and this conversion should be as fast as possible.
The
solution does not have to be cross platform. I am working with a very
specific compiler and processor, so it can be assumed that it is a 32
bit architecture with big endian.
The lowest byte of the result does not matter to me (see above).
My main questions at the moment are: Why has d1 the correct value but d2 does not? Is this also true for other compilers? Can this behavior be changed?
No you can't do that in a portable way.
The behaviour encountered when attempting a cast from char* to int* is undefined in both C and C++ (possibly for the very reasons that you've spotted: ints are possibly aligned on 4 byte boundaries and data is, of course, contiguous.)
(The fact that data+3 works but data+p doesn't is possibly due to to compile time vs. runtime evaluation.)
Also note that the signed-ness of char is not specified in either C or C++ so you should use signed char or unsigned char if you're writing code like this.
Your best bet is to use bitwise shift operators (>> and <<) and logical | and & to absorb char values into an int. Also consider using int32_tin case you build to targets with 16 or 64 bit ints.
There is no way, converting a pointer to a wrongly aligned one is undefined.
You can use memcpy to copy the char array into an int32_t.
int32_t d = 0;
memcpy(&d, data+3, 4); // assuming sizeof(int) is 4
Most compilers have built-in functions for memcpy with a constant size argument, so it's likely that this won't produce any runtime overhead.
Even though a cast like you've shown is allowed for correctly aligned pointers, dereferencing such a pointer is a violation of strict aliasing. An object with an effective type of char[] must not be accessed through an lvalue of type int.
In general, type-punning is endianness-dependent, and converting a char array representing RGB colours is probably easier to do in an endianness-agnostic way, something like
int32_t d = (int32_t)data[2] << 16 | (int32_t)data[1] << 8 | data[0];
I found this post:
Why is memcpy slower than a reinterpret_cast when parsing binary data?
where somebody uses reinterpret_cast to convert binary data to an integer. However (I presume) the number they are converting is at the 0th element in the char* array.
How could I use the above for situations where the binary number I want to convert is offset N bytes from the beginning of the char array?
I want to convert the binary number in as fewer CPU cycles as possible, hence my interest in reinterpret_cast and the above SO question.
Just add a offset to the byte array address.
Instead of x, cast x+123
But: Have you read the first line of the question (bold edit)?
TLDR: I forgot to enable compiler optimizations. With the
optimizations enabled the performance is (nearly) identical.
If you have a *array; initialized with your binary data then you can simply do this:
for (int offset = 0; offset < sizeof (array); offset++)
{
... = *reinterpret_cast<const int*>(array + offset);
}
If there is for example a class that requires a pointer and a bool. For simplicity an int pointer will be used in examples, but the pointer type is irrelevant as long as it points to something whose size() is more than 1 .
Defining the class with { bool , int *} data members will result in the class having a size that is double the size of the pointer and a lot of wasted space
If the pointer does not point to a char (or other data of size(1)), then presumably the low bit will always be zero. The class could defined with {int *} or for convenience: union { int *, uintptr_t }
The bool is implemented by setting/clearing the low bit of the pointer as per the logical bool value and clearing the bit when you need to use the pointer.
The defined way:
struct myData
{
int * ptr;
bool flag;
};
myData x;
// initialize
x.ptr = new int;
x.flag = false;
// set flag true
x.flag = true;
// set flag false
x.flag = false;
// use ptr
*(x.ptr)=7;
// change ptr
x = y; // y is another int *
And the proposed way:
union tiny
{
int * ptr;
uintptr_t flag;
};
tiny x;
// initialize
x.ptr = new int;
// set flag true
x.flag |= 1;
// set flag false
x.flag &= ~1;
// use ptr
tiny clean=x; // note that clean will likely be optimized out
clean.flag &= ~1; // back to original value as assigned to ptr
*(clean.ptr)=7;
// change ptr
bool flag=x.flag;
x.ptr = y; // y is another int *
x.flag |= flag;
This seems to be undefined behavior, but how portable is this?
As long as you restore the pointer's low-order bit before trying to use it as a pointer, it's likely to be "reasonably" portable, as long as your system, your C++ implementation, and your code meet certain assumptions.
I can't necessarily give you a complete list of assumptions, but off the top of my head:
It assumes you're not pointing to anything whose size is 1 byte. This excludes char, unsigned char, signed char, int8_t, and uint8_t. (And that assumes CHAR_BIT == 8; on exotic systems with, say, 16-bit or 32-bit bytes, other types might be excluded.)
It assumes objects whose size is at least 2 bytes are always aligned at an even address. Note that x86 doesn't require this; you can access a 4-byte int at an odd address, but it will be slightly slower. But compilers typically arrange for objects to be stored at even addresses. Other architectures may have different requirements.
It assumes a pointer to an even address has its low-order bit set to 0.
For that last assumption, I actually have a concrete counterexample. On Cray vector systems (J90, T90, and SV1 are the ones I've used myself) a machine address points to a 64-bit word, but the C compiler under Unicos sets CHAR_BIT == 8. Byte pointers are implemented in software, with the 3-bit byte offset within a word stored in the otherwise unused high-order 3 bits of the 64-bit pointer. So a pointer to an 8-byte aligned object could have easily its low-order bit set to 1.
There have been Lisp implementations (example) that use the low-order 2 bits of pointers to store a type tag. I vaguely recall this causing serious problems during porting.
Bottom line: You can probably get away with it for most systems. Future architectures are largely unpredictable, and I can easily imagine your scheme breaking on the next Big New Thing.
Some things to consider:
Can you store the boolean values in a bit vector outside your class? (Maintaining the association between your pointer and the corresponding bit in the bit vector is left as an exercise).
Consider adding code to all pointer operations that fails with an error message if it ever sees a pointer with its low-order bit set to 1. Use #ifdef to remove the checking code in your production version. If you start running into problems on some platform, build a version of your code with the checks enabled and see what happens.
I suspect that, as your application grows (they seldom shrink), you'll want to store more than just a bool along with your pointer. If that happens, the space issue goes away, because you're already using that extra space anyway.
In "theory": it's undefined behavior as far as I know.
In "reality": it'll work on everyday x86/x64 machines, and probably ARM too?
I can't really make a statement beyond that.
It's very portable, and furthermore, you can assert when you accept the raw pointer to make sure it meets the alignment requirement. This will insure against the unfathomable future compiler that somehow messes you up.
Only reasons not to do it are the readability cost and general maintenance associated with "hacky" stuff like that. I'd shy away from it unless there's a clear gain to be made. But it is sometimes totally worth it.
Conform to those rules and it should be very portable.