I have the following question.
Given that a pointer holds the value of a memory address, why is it permitted to add an integer
data type value to a pointer variable but not a double data type?
My thoughts: Is it because we assume that the pointer is an int as well, or maybe because if we add a double will increase its length?
Thank you for your time.
You almost answered your question yourself: a pointer is a memory address. A memory address is an integer. You can add integers to integers and get integers as a result. Adding a float to an integer gives you a float, which cannot be used as a memory address.
For example, char *x = 0; is the address of a single byte; What would char *y = 0.5; mean? A byte that's somehow made up of the second half of the byte at address 0 and the first half of the byte at address 1?? This may make sense, but what about char *x = 3.1415926; or any similar floating-point number??
My thoughts: Is it because we assume that the pointer is an int as well, or maybe because if we add a double will increase its length?
If you look to documentation it says:
Certain addition, subtraction, increment, and decrement operators are defined for pointers to elements of arrays: such pointers satisfy the LegacyRandomAccessIterator requirements and allow the C++ library algorithms to work with raw arrays.
(emphasis is mine) and you should remember that:
*(ptr + 1)
is equal to:
ptr[1]
and indexes for arrays are integers so language does not define operations on pointers with floating point operands as that does not make any sense.
You can not add a double* (pointer) to an int* (pointer) via the conventions of C. A pointer holds a value of a memory address ["stores/points to the address of another variable"] that value in essence is determined by its type in this case int(4 byte-block of memory if I recall). A double is a double-precision, 64-bit floating-point data type. Just can't do it from the most "hardware" of levels.
Related
What I know about pointer is, it is used to point to specific location (memory address), so why do we even need the same data type of pointer as that of the variable we are trying to point.
Suppose I create a variable of integer, then I have to create a pointer to integer to point it. So why can't I create a void pointer or float pointer to point the value stored in that integer variable!
Am I missing some concepts of pointers?
So why can't I create a void pointer [...] to point the value stored in that integer variable
You can do that, no problem:
int x = 10;
double y = 0.4;
void* v = &x;
v = &y;
But now imagine a function like this:
void print(void* value)
How would this function know what to do with the memory at the pointer location? Is it an integer? Or a floating point number? A float or a double? Maybe it's a huge struct or an array of values? You must know this to dereference the pointer (i.e. read the memory) correctly, so it only makes sense to have different pointer types for pointers to different types:
void print(int* value)
This function knows that the pointer points to an int, so it can happily dereference it to get an int value.
The pointer type is also important when dealing with arrays, as arrays and pointers are interchangeable. When you increment a pointer (which is what indexing does), you need to know how big the type is (int, long, structure, class) in order to access the next item.
arr[5] == *(arr+5) but 5 what? This is determined by the type.
A small addition on Max Langhof's answer:
It is important to realise that in the end, variables are stored simply as a sequence of bits (binary digits), e.g. 01010101 00011101 11100010 11110000. How does your program know what this 'means'? It could be an integer (which is often 4 bytes on modern architectures), it could be a floating-point value. For the memory involved this makes no difference, but for your code the implications can be huge. Therefore, if you refer to this memory location (using a pointer), you will need to specify how the bytes there should be converted to decimal (or other) values.
Pointer arithmetic is the main reason - if p points to an object of type T, then p+1 points to the next object of that type. If p points to a 4-byte int, then p+1 points to the following 4-byte int. If p points to a 128-byte struct, then p+1 points to the following 128-byte struct. If p points to a 2 Kbyte array of double, then p+1 points to the next 2 Kbyte array of double, etc.
But it's also for the same reason we have different types in the first place - at the abstract machine level, we want to distinguish different types of data and operations that are allowed on that data. A pointer to an int is different from a pointer to a double because an int is different from a double.
You are right. Although int and float are different types, there should be no difference between their pointers int* and float*. In general, this is the case. However, the binary representation is different between int and float. Therefore accessing an int with a float* pointer leads to garbage being read from the RAM.
Furthermore, what you have on your machine is not the general case, but hardware and implementation dependent.
For example: float and int variables are usually 32bit long. However, there are systems where the int has only 16bit. What happens now if you try to read a float from a int* pointer? (Or, even if both are 32bit, what happens if you try to read a float from a char*?)
Memory accesses do not work without knowing what kind of data object you are dealing with.
Imagine some simple assignment:
int a, b=10;
float f;
a = b; // same type => Just copy the integer
f = b; // wrong type => Convert to float.
This works fine because the compiler knows that both variables are of a certain type and size and representation. If the types do not match, a proper conversion is applied.
Now the same with typed pointers:
int a = 10;
float f;
int *pa;
float *pf;
f = a; // Type conversion to float applied
*pa = a; // Just copy
*pf = a; // Type conversion
If you take away the knowledge about the memory location where the pointer points to, how would the compiler know if a conversion is required?
Or if some integer propagation is needed or is an integer is truncated into a shorter type?
More problems are waiting around the corner if you want to use a pointer to address elements of an array. Pointer arithmetics won't fly without types.
Types are essential. For variables as well as for pointers.
I'm testing some ways of calculating the size,in bytes of a function(I'm familiar with opcodes on x86). The code is quite self-explanatory:
void exec(void* addr){
int (WINAPI *msg)(HWND,LPCSTR,LPCSTR,UINT)=(int(WINAPI *)(HWND,LPCSTR,LPCSTR,UINT))addr;
msg(0,"content","title",0);
}
void dump(){};
int main()
{
cout<<(char*)dump-(char*)exec; // this is 53
return 0;
}
It is supposed to substract the address of 'exec' from 'dump'. This works but I noticed the values differ when using other types of pointers like DWORD*:
void exec(void* addr){
int (WINAPI *msg)(HWND,LPCSTR,LPCSTR,UINT)=(int(WINAPI *)(HWND,LPCSTR,LPCSTR,UINT))addr;
msg(0,"content","title",0);
}
void dump(){};
int main()
{
cout<<(DWORD*)dump-(DWORD*)exec; // this is 13
return 0;
}
From my understanding no matter the pointer type ,it is always the largest possible data type(so that it can handle large adresses),in my case of 4 bytes (x86 system). The only thing that changes between pointers is the data type it points to.
What is the explanation?
Pointer arithmetic in C/C++ is designed for accessing elements of an array. In fact, array indexing is merely a simpler syntax for pointer arithmetic. For example, if you have an array named array, array[1] is the same thing as *(array+1), regardless of the data type of the elements in array.
(I'm assuming here that no operator overloading is going on; that could change everything.)
If you have a char* or unsigned char*, the pointer points to a single byte, and incrementing the pointer advances it to the next byte.
In Windows, DWORD is a 32-bit value (four bytes), and DWORD* points to a 32-bit value. If you increment a DWORD*, the pointer is advanced by four bytes, just as array[1] gives you the second element of the array, which is four bytes (one DWORD) after the first element. Similarly, if you add 10 to a DWORD*, it advances 40 bytes, not 10 bytes.
Either way, incrementing or adding to a pointer is only valid if the resulting pointer points into the same array as the original one, or one element past the end. Otherwise it is undefined behavior.
Pointer subtraction works just like addition. When you subtract one pointer from another, they must be the same type, and must be pointers into the same array or one past the end.
What you're doing is counting the number of elements between the two pointers, as if they were pointers into the same array (or one past the end). But when the two pointers don't point into the same array (or again, one past the end), the result is undefined behavior.
Here is a reference from Carnegie Mellon University about this:
ARR36-C. Do not subtract or compare two pointers that do not refer to the same array - SEI CERT C Coding Standard
Pointer subtraction tells you the number of elements between the two addresses, so using DWORD * it will be in DWORD sized units.
You have:
cout<<(char*)dump-(char*)exec;
where dump and exec are the names of functions. Each cast converts a function pointer to char*.
I'm not sure about the status of such a conversion in C++. I think it either has undefined behavior or is illegal (making your program ill-formed). When I compile with g++ 4.8.4 with options -pedantic -std=c++11, it complains:
warning: ISO C++ forbids casting between pointer-to-function and pointer-to-object [-Wpedantic]
(There's a similar diagnostic for C, which I believe is not strictly correct, but that's another story.)
There's no guarantee that there's any meaningful relationship between object pointers and function pointers.
Apparently your compiler lets you get away with the casts, and presumably the result is a char* representation of the address of the function. Subtracting two pointers yields the distance between the two addresses in units of the type the pointers point to. Subtracting two char* pointers yields a ptrdiff_t result that is the difference in bytes. Subtracting two DWORD* pointers yields the difference in unit of sizeof (DWORD) (probably 4 bytes?). That explains why you get different results. If two DWORD pointers don't point to addresses that aren't a whole number of DWORDs apart in memory, the results are unpredictable, but in your example getting 13 rather than 53 (truncating) is plausible.
However, pointer subtraction is defined only when both pointer operands point to elements of the same array object, or just past the end of it. For any other operands, the behavior is undefined.
For an implementation that permits the casts, that uses the same representation for object pointers and for function pointers, and on which the value of a function pointer refers to a memory address in the same way that the value of an object pointer does, you can likely determine the size of a function by converting its address to char* and subtracting the result from the converted address of an adjacent function. But a compiler and/or linker is free to generate code for functions in any order it likes, including perhaps inserting code for other functions between two functions whose definitions are adjacent in your source code.
If you want to determine the size in bytes, use pointers to byte-sized types such as char. And be aware that the method you're using is not portable and is not guaranteed to work.
If you really need the size of a function, see if you can get your linker to generate some kind of map showing the allocated sizes and locations of your functions. There's no portable way to do it from within C++.
(related to my previous question)
In QT, the QMap documentation says:
The key type of a QMap must provide operator<() specifying a total order.
However, in qmap.h, they seem to use something similar to std::less to compare pointers:
/*
QMap uses qMapLessThanKey() to compare keys. The default
implementation uses operator<(). For pointer types,
qMapLessThanKey() casts the pointers to integers before it
compares them, because operator<() is undefined on pointers
that come from different memory blocks. (In practice, this
is only a problem when running a program such as
BoundsChecker.)
*/
template <class Key> inline bool qMapLessThanKey(const Key &key1, const Key &key2)
{
return key1 < key2;
}
template <class Ptr> inline bool qMapLessThanKey(const Ptr *key1, const Ptr *key2)
{
Q_STATIC_ASSERT(sizeof(quintptr) == sizeof(const Ptr *));
return quintptr(key1) < quintptr(key2);
}
They just cast the pointers to quintptrs (which is the QT-version of uintptr_t, that is, an unsigned int that is capable of storing a pointer) and compare the results.
The following type designates an unsigned integer type with the property that any valid pointer to void can be converted to this type, then converted back to a pointer to void, and the result will compare equal to the original pointer: uintptr_t
Do you think this implementation of qMapLessThanKey() on pointers is ok?
Of course, there is a total order on integral types. But I think this is not sufficient to conclude that this operation defines a total order on pointers.
I think that it is true only if p1 == p2 implies quintptr(p1) == quintptr(p2), which, AFAIK, is not specified.
As a counterexample of this condition, imagine a target using 40 bits for pointers; it could convert pointers to quintptr, setting the 40 lowest bits to the pointer address and leaving the 24 highest bits unchanged (random). This is sufficient to respect the convertibility between quintptr and pointers, but this does not define a total order for pointers.
What do you think?
The Standard guarantees that converting a pointer to an uintptr_t will yield a value of some unsigned type which, if cast to the original pointer type, will yield the original pointer. It also mandates that any pointer can be decomposed into a sequence of unsigned char values, and that using such a sequence of unsigned char values to construct a pointer will yield the original. Neither guarantee, however, would forbid an implementation from including padding bits within pointer types, nor would either guarantee require that the padding bits behave in any consistent fashion.
If code avoided storing pointers, and instead cast to uintptr_t every pointer returned from malloc, later casting those values back to pointers as required, then the resulting uintptr_t values would form a ranking. The ranking might not have any relationship to the order in which objects were created, nor to their arrangement in memory, but it would be a ranking. If any pointer gets converted to uintptr_t more than once, however, the resulting values might rank entirely independently.
I think that you can't assume that there is a total order on pointers. The guarantees given by the standard for pointer to int conversions are rather limited:
5.2.10/4: A pointer can be explicitly converted to any integral type large enough to hold it. The mapping function is
implementation-defined.
5.2.10/5: A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted to an integer
of sufficient size (...) and back to the same pointer type will have
its original value; mappings between pointers and integers are
otherwise implementation-defined.
From a practical point of view, most of the mainstream compilers will convert a pointer to an integer in a bitwise manner, and you'll have a total order.
The theoretical problem:
But this is not guaranteed. It might not work on past platforms (x86 real and protected mode), on exotic platform (embedded systems ?) , and -who knows- on some future platforms (?).
Take the example of segmented memory of the 8086: The real address is given by the combination of a segment (e.g. DS register for data segment, an SS for stack segment,...) and an offest:
Segment: XXXX YYYY YYYY YYYY 0000 16 bits shifted by 4 bits
Offset: 0000 ZZZZ ZZZZ ZZZZ ZZZZ 16 bits not sifted
------------------------
Address: AAAA AAAA AAAA AAAA AAAA 20 bits address
Now imagine that the compiler would convert the pointer to int, by simply doing the address math and put 20 bits in the integer: your safe and have a total order.
But another equally valid approach would be to store the segment on 16 upper bits and the offset on the 16 lower bits. In fact, this way would significantly facilitate/accelerate the load of pointer values into cpu registers.
This approach is compliant with standard c++ requirements, but each single address could be represented by 16 different pointers: your total order is lost !!
**Are there alternatives for the order ? **
One could imagine using pointer arithmetics. There are strong constraints on pointer arithmetics for elements in a same array:
5.7/6: When two pointers to elements of the same array object are subtracted, the result is the difference of the subscripts of the two
array elements.
And subscripts are ordered.
Array can be of maximum size_t elements. So, naively, if sizeof(pointer) <= sizof(size_t) one could assume that taking an arbitrary reference pointer and doing some pointer arithmetic should lead to a total order.
Unfortunately, here also, the standard is very prudent:
5.7.7: For addition or subtraction, if the expressions P or Q have type “pointer to cv T”, where T is different from the
cv-unqualified array element type, the behavior is undefined.
So pointer arithmetic won't do the trick for arbitrary pointers either. Again, back to the segmented memory models, helps to understand: arrays could have maximum 65535 bytes to fit completely in one segment. But different arrays could use different segments so that pointer arithmetic wouldn't be reliable for a total order either.
Conclusion
There's a subtle note in the standard on the mapping between pointer and interal value:
It is intended to be unsurprising to those who know the addressing
structure of the underlying machine.
This means that must be be possible to determine a total order. But keep in mind that it'll be non portable.
Now we know that doing out-of-bounds-pointer-arithmetic has undefined behavior as described in this SO question.
My question is: can we workaround such restriction by casting to std::uintptr_t for arithmetic operations and then cast back to pointer? is that guaranteed to work?
For example:
char a[5];
auto u = reinterpret_cast<std::uintptr_t>(a) - 1;
auto p = reinterpret_cast<char*>(u + 1); // OK?
The real world usage is for optimizing offsetted memory access -- instead of p[n + offset], I want to do offset_p[n].
EDIT To make the question more explicit:
Given a base pointer p of a char array, if p + n is a valid pointer, will reinterpret_cast<char*>(reinterpret_cast<std::uintptr_t>(p) + n) be guaranteed to yield the same valid pointer?
No, uintptr_t cannot be meaningfully used to avoid undefined behavior when performing pointer arithmetic.
For one thing, at least in C there is no guarantee that uintptr_t even exists. The requirement is that any value of type void* may be converted to uintptr_t and back again, yielding the original value without loss of information. In principle, there might not be any unsigned integer type wide enough to hold all pointer values. (I presume the same applies to C++, since C++ inherits most of the C standard library and defines it by reference to the C standard.)
Even if uintptr_t does exist, there is no guarantee that a given arithmetic operation on a uintptr_t value does the same thing as the corresponding operation on a pointer value.
For example, I've worked on systems (Cray vector systems, T90 and SV1) on which byte pointers are implemented in software. A native address is a 64-bit address that refers to a 64-bit word; there is no hardware support for byte addressing. A char* or void* pointer consists of a word pointer with a 3-bit offset stored in the otherwise unused high-order bits. Conversion between integers and pointers simply copies the bits. So incrementing a char* would advance it to point to the next 8-bit byte in memory; incrementing a uintptr_t obtained by converting a char* would advance it to point to the next 64-bit word.
That's just one example. More generally, conversions between pointers and integers are implementation-defined, and the language standard makes no guarantee about the semantics of those conversions (other than, in some cases, converting back to a pointer).
So yes, you can convert a pointer value to uintptr_t (if that type exists) and perform arithmetic on it without risking undefined behavior -- but the result may or may not be meaningful.
It happens that, on most systems, the mapping between pointers and integers is simpler, and you probably can get away with that kind of game. But you're better off using pointer arithmetic directly, and just being very careful to avoid any invalid operations.
Yes, that is legal, but you must reinterpret_cast exactly the same uintptr_t value back to char*.
(Therefore, what it you're intending to do is illegal; that is, converting a different value back to a pointer.)
5.2.10 Reinterpret cast
4 . A pointer can be explicitly converted to any integral type large enough to hold it. The mapping function is
implementation-defined.
5 . A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted
to an integer of sufficient size (if any such exists on the implementation) and back to the same pointer type
will have its original value;
(Note that there'd be no way, in general, for the compiler to know that you subtracted one and then added it back.)
I think I understand the semantics of pointer arithmetic fairly well, but I only ever see examples when dealing with arrays. Does it have any other uses that can't be achieved by less opaque means? I'm sure you could find a way with clever casting to use it to access members of a struct, but I'm not sure why you'd bother. I'm mostly interested in C, but I'll tag with C++ because the answer probably applies there too.
Edit, based on answers received so far: I know pointers can be used in many non-array contexts. I'm specifically wondering about arithmetic on pointers, e.g. incrementing, taking a difference, etc.
Pointer arithmetic by definition in C happens only on arrays. However, as every object has a representation consisting of an overlaid unsigned char [sizeof object] array, it's also valid to perform pointer arithmetic on this representation. For example:
struct foo {
int a, b, c;
} bar;
/* Equivalent to: bar.c = 1; */
*(int *)((unsigned char *)&bar + offsetof(struct foo, c)) = 1;
Actually char * would work just as well.
If you follow the language standard to the letter, then pointer arithmetic is only defined when pointing to an array, and not in any other case.
A pointer may point to any element of an array, or one step past the end of the array.
From the top of my head I know it's used in XOR linked-lists (very nifty) and I've seen it used in very hacky recursions.
On the other hand, it's very hard to find uses since according to the standard pointer arithmic is only defined if within the bounds of an array.
a[n] is "just" syntactic sugar for *(a + n). For lulz, try the following
int a[2];
0[a] = 10;
1[a] = 20;
So one could argue that indexing and pointer arithmetic are merely interchangeable syntax.
Pointer arithmetic is only defined on arrays. Adding an integer to a pointer that does not point to an array element produces undefined behavior.
In embedded systems, pointers are used to represent addresses or locations. There may not be an array defined. (Although one could say that all of memory is one huge array.)
For example, a stack (holding variables and addresses) is manipulated by adding or subtracting values from the stack pointer. (In this case, the stack could be said to be an array based stack.)
Here's a case for pointer arithmetic outside of (strictly defined) arrays:
double d = 0.5;
unsigned char *bytes = (void *)&d;
for(size_t i = 0; i < sizeof d; i++)
printf("Byte %zu of d is %hhu\n", i, bytes[i]);
Why would you do this? I don't know. But if you want to look at the bitwise representation of an object (useful for things like memcpy and memcmp), you'll need to cast their addresses to unsigned char *s (or signed char *s if you like) and work with them byte-by-byte. (If your task isn't too difficult you can even write the code to work word-by-word, which most memcpy implementations will do. It's the same principle, though, just replace char with int32_t.)
Note that, in the standard, the exact values (or the number of values) that are printed are implementation-defined, but that this will always work as a way to access an object's internal bytewise representation. (It is not required to work for larger integer types, but almost always will - no processor I know of has had trap representations for integers in quite some time).