Today I wrote something which looked like this:
void foo(std::vector<char>&v){
v.push_back('a');
char*front=&v.front();
char*back=&v.back();
size_t n1=back-front+1;
v.push_back('b');//This could reallocate the vector elements
size_t n2=back-front+1;//Is this line valid or Undefined Behavior ?
}
If a reallocation occures when I push 'b' back, may I still compute the difference of my two pointers ?
After reading the relevant passage of the standard a few times, I still cannot make my mind on this point.
C++11 5.7.6:
When two pointers to elements of the same array object are subtracted, the result is the difference of the
subscripts of the two array elements. The type of the result is an implementation-defined signed integral
type; this type shall be the same type that is defined as std::ptrdiff_t in the header (18.2). As
with any other arithmetic overflow, if the result does not fit in the space provided, the behavior is undefined.
In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object,
the expression (P)-(Q) has the value i − j provided the value fits in an object of type std::ptrdiff_t.
Moreover, if the expression P points either to an element of an array object or one past the last element of
an array object, and the expression Q points to the last element of the same array object, the expression
((Q)+1)-(P) has the same value as ((Q)-(P))+1 and as -((P)-((Q)+1)), and has the value zero if the
expression P points one past the last element of the array object, even though the expression (Q)+1 does not
point to an element of the array object. Unless both pointers point to elements of the same array object, or
one past the last element of the array object, the behavior is undefined.
Of course I know that it works, I just wonder if it is legal.
Pointers to deleted objects are toxic: don't touch then for anything other than giving them a new value. A memory tracking system may trap aby use of a reclaimed pointer value. I'm not aware if any such system in existence, however.
The relevant quote is 3.7.4.2 [basic.stc.dynamic.deallocation] paragraph 4:
If the argument given to a deallocation function in the standard library is a pointer that is not the null pointer value, the deallocation function shall deallocate the storage referenced by the pointer, rendering invalid all pointers to any part of the deallocated storage. The effect of using an invalid pointer value (including passing it to a deallocation function) is undefined.
When resizing a std::vector<...> it jumps through a number of hoops (allocators) and, by default, eventually calls a deallocation function.
Strictly speaking, it's UB. But you can always convert your char * pointers to uintptr_t (provided it is present) and then safely subtract the resulting integers.
void foo(std::vector<char>&v){
v.push_back('a');
auto front= uintptr_t (&v.front());
auto back = uintptr_t (&v.back());
size_t n1=back-front+1;
v.push_back('b');//This could reallocate the vector elements
size_t n2=back-front+1;
}
This particular case is safe but ugly and misleading.
Line v.push_back('b');//This could reallocate the vector elements can cause reallocation of your container. In this case next line will use a non existent front and back pointers. Computing difference of two addresses is safe even if are dangling pointers. What is not safe is dereferencing them.
The correct solution is to use vector::count() function the will be always in sync. If you (for some reason) don;t want to call vector::count() you should at leas use ++n1.
Related
AFAIK, although we cannot create a 0-sized static-memory array, but we can do it with dynamic ones:
int a[0]{}; // Compile-time error
int* p = new int[0]; // Is well-defined
As I've read, p acts like one-past-end element. I can print the address that p points to.
if(p)
cout << p << endl;
Although I am sure of we cannot dereference that pointer (past-last-element) as we cannot with iterators (past-last element), but what I am not sure of is whether incrementing that pointer p? Is an undefined behaviour (UB) like with iterators?
p++; // UB?
Pointers to elements of arrays are allowed to point to a valid element, or one past the end. If you increment a pointer in a way that goes more than one past the end, the behavior is undefined.
For your 0-sized array, p is already pointing one past the end, so incrementing it is not allowed.
See C++17 8.7/4 regarding the + operator (++ has the same restrictions):
f the expression P points to element x[i] of an array object x with n elements, the expressions P + J and J + P (where J has the value j) point to the (possibly-hypothetical) element x[i+j] if 0≤i+j≤n; otherwise, the behavior is undefined.
I guess you've already have the answer; If you look a bit deeper: You've said that incrementing an off-the-end iterator is UB thus: This answer is in what is an iterator?
The iterator is just an object that has a pointer and incrementing that iterator is really incrementing the pointer it has. Thus in many aspects an iterator is handled in terms of a pointer.
int arr[] = {0,1,2,3,4,5,6,7,8,9};
int *p = arr; // p points to the first element in arr
++p; // p points to arr[1]
Just as we can use iterators to traverse the elements in a vector, we can use pointers to traverse the elements in an array. Of course, to do so, we need to obtain pointers to the first and one past the last element. As we’ve just seen, we can obtain a pointer to the first element by using the array itself or by taking the address-of the first element. We can obtain an off-the-end pointer by using another special property of arrays. We can take the address of the nonexistent element one past the last element of an array:
int *e = &arr[10]; // pointer just past the last element in arr
Here we used the subscript operator to index a nonexisting element; arr has ten elements, so the last element in arr is at index position 9. The only thing we can do with this element is take its address, which we do to initialize e. Like an off-the-end iterator (§ 3.4.1, p. 106), an off-the-end pointer does not point to an element. As a result, we may not dereference or increment an off-the-end pointer.
This is from C++ primer 5 edition by Lipmann.
So it is UB don't do it.
In the strictest sense, this is not Undefined Behavior, but implementation-defined. So, although inadvisable if you plan to support non-mainstream architectures, you can probably do it.
The standard quote given by interjay is a good one, indicating UB, but it is only the second best hit in my opinion, since it deals with pointer-pointer arithmetic (funnily, one is explicitly UB, while the other isn't). There is a paragraph dealing with the operation in the question directly:
[expr.post.incr] / [expr.pre.incr]
The operand shall be [...] or a pointer to a completely-defined object type.
Oh, wait a moment, a completely-defined object type? That's all? I mean, really, type? So you don't need an object at all?
It takes quite a bit of reading to actually find a hint that something in there might not be quite so well-defined. Because so far, it reads as if you are perfectly allowed to do it, no restrictions.
[basic.compound] 3 makes a statement about what type of pointer one may have, and being none of the other three, the result of your operation would clearly fall under 3.4: invalid pointer.
It however doesn't say that you aren't allowed to have an invalid pointer. On the contrary, it lists some very common, normal conditions (e.g. end of storage duration) where pointers regularly become invalid. So that's apparently an allowable thing to happen. And indeed:
[basic.stc] 4
Indirection through an invalid pointer value and passing an invalid pointer value to a deallocation function have undefined behavior. Any other use of an invalid pointer value has implementation-defined behavior.
We are doing an "any other" there, so it's not Undefined Behavior, but implementation-defined, thus generally allowable (unless the implementation explicitly says something different).
Unluckily, that's not the end of the story. Although the net result doesn't change any more from here on, it gets more confusing, the longer you search for "pointer":
[basic.compound]
A valid value of an object pointer type represents either the address of a byte in memory or a null pointer. If an object of type T is located at an address A [...] is said to point to that object, regardless of how the value was obtained.
[ Note: For instance, the address one past the end of an array would be considered to point to an unrelated object of the array's element type that might be located at that address. [...]].
Read as: OK, who cares! As long as a pointer points somewhere in memory, I'm good?
[basic.stc.dynamic.safety]
A pointer value is a safely-derived pointer [blah blah]
Read as: OK, safely-derived, whatever. It doesn't explain what this is, nor does it say I actually need it. Safely-derived-the-heck. Apparently I can still have non-safely-derived pointers just fine. I'm guessing that dereferencing them would probably not be such a good idea, but it's perfectly allowable to have them. It doesn't say otherwise.
An implementation may have relaxed pointer safety, in which case the validity of a pointer value does not depend on whether it is a safely-derived pointer value.
Oh, so it may not matter, just what I thought. But wait... "may not"? That means, it may as well. How do I know?
Alternatively, an implementation may have strict pointer safety, in which case a pointer value that is not a safely-derived pointer value is an invalid pointer value unless the referenced complete object is of dynamic storage duration and has previously been declared reachable
Wait, so it's even possible that I need to call declare_reachable() on every pointer? How do I know?
Now, you can convert to intptr_t, which is well-defined, giving an integer representation of a safely-derived pointer. For which, of course, being an integer, it is perfectly legitimate and well-defined to increment it as you please.
And yes, you can convert the intptr_t back to a pointer, which is also well-defined. Only just, not being the original value, it is no longer guaranteed that you have a safely-derived pointer (obviously). Still, all in all, to the letter of the standard, while being implementation-defined, this is a 100% legitimate thing to do:
[expr.reinterpret.cast] 5
A value of integral type or enumeration type can be explicitly converted to a pointer. A pointer converted to an integer of sufficient size [...] and back to the same pointer type [...] original value; mappings between pointers and integers are otherwise implementation-defined.
The catch
Pointers are just ordinary integers, only you happen to use them as pointers. Oh if only that was true!
Unluckily, there exist architectures where that isn't true at all, and merely generating an invalid pointer (not dereferencing it, just having it in a pointer register) will cause a trap.
So that's the base of "implementation defined". That, and the fact that incrementing a pointer whenever you want, as you please could of course cause overflow, which the standard doesn't want to deal with. The end of application address space may not coincide with the location of overflow, and you do not even know whether there is any such thing as overflow for pointers on a particular architecture. All in all it's a nightmarish mess not in any relation of the possible benefits.
Dealing with the one-past-object condition on the other hand side, is easy: The implementation must simply make sure no object is ever allocated so the last byte in the address space is occupied. So that's well-defined as it's useful and trivial to guarantee.
I have a class that is just a raw pointer wrapper. I am wondering whether I can declare:
operator++()
operator--()
operator++(int)
operator--(int)
as noexcept. The only reason these operators could throw is because the raw pointer would throw. So the question is: what is happening when -- is called on a pointer that is already 0, and what is happening when ++ is called on a pointer that is at its maximum?
The standard specifies in §5.7/4 that:
When an expression that has integral type is added to or subtracted from a pointer, the result has the type of the pointer operand. If the pointer operand points to an element of an array object, and the array is large enough, the result points to an element offset from the original element such that the difference of the subscripts of the resulting and original array elements equals the integral expression. [...] Moreover, if the expression P points to the last element of an array object, the expression (P)+1 points one past the last element of the array object, and if the expression Q points one past the last element of an array object, the expression (Q)-1 points to the last element of the array object. If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behaviour is undefined.
(emphasis mine)
A similar quote can be found for subtraction in §5.7/5. So given that overflow and underflow are special cases of pointers that exceed the bounds of object to which they originally point to, the behaviour would simply be undefined.
I'd say that it's therefore possible for you to declare those functions noexcept.
As a side note, pointer-like types in the standard library, like std::unique_ptr, do not define the functions that could cause the above mentioned undefined behaviour as noexcept (for example operator[] for std::unique_ptr) due to the fact that some implementations decide to throw an exception in debug mode.
Raw pointer arithmetic doesn't throw anything. It'll just let you screw yourself. So yeah, you can.
Considering that C++ does not have bound checking for built-in type arrays, Is it possible that:
One array's off-the-end pointer points to another array's first element?
Yes, a pointer beyond the end of an array could point to another object. Dereferencing a pointer beyond the end of an array results in undefined behavior.
My opinion: yes, it is possible in C++. There have been several SO threads on this topic, none of which reached any solid conclusion. Here is one example.
In some cases we can be sure that there is actually a valid object in memory immediately after the end of the old object. One case is standard-layout structs; another is multi-dimensional arrays. I originally wrote this post with a multi-dimensional array, but I have edited it to use the standard layout struct case, to avoid any objections about what the term "array object" means in the Standard.
struct
{
int a[2];
int b[2];
} foo;
if ( sizeof foo == 4 * sizeof(int) )
{
int *p = &foo.a[0];
++p; // (1)
++p; // (2)
*p = 3; // (3)
++p; // (4)
*p = 5; // (5)
}
Which line causes undefined behaviour (if any)? p is (initially, anyway) a pointer into the array of type int[2] which is designated by foo.a.
After line (2), p is now a one-past-the-end pointer. Is this dereferenceable?
The case of incrementing the pointer is covered by the section on the + operator (it is defined to have the same effect on p as p = p + 1). Here is a quote from C++11 [expr.add]#7:
Unless both pointers point to elements of the same array object, or
one past the last element of the array object, the behavior is undefined.
Line (2) does not cause UB by this clause. What about line (3)?
As far as I can see, there is no clause in the C++ standard that says dereferencing a one-past-the-end pointer causes undefined behaviour. In several places it says that iterators "might not be dereferencable", or "the library does not assume that the iterator is dereferenceable". But it carefully avoids saying "the iterator is not dereferenceable".
From the fact that we proved there is no padding, and the rules about standard-layout structs saying that elements cannot be reordered; we can conclude that now p must hold the address of the element foo.b[0]. Therefore, p is a pointer into the subobject foo.b, as well as being a one-past-the-end pointer for foo.a.
Note that in C99 it is different. The text in C99 for the + operator has (emphasis mine):
If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the
behavior is undefined. If the result points one past the last element of the array object, it
shall not be used as the operand of a unary * operator that is evaluated.
So, in C99 line (3) causes undefined behaviour. However C++ deliberately omits the bolded line.
Rationale: I don't know what the actual rationale is. However, my "mental model" for C's pointers is that it permits the compiler to implement "fat pointers", i.e. bounds-checked pointers. A pointer may contain the bounds of the (sub-)object that it was pointed to; and so the executable can detect array bounds errors at runtime just based on the pointer value.
I believe the C99 text is compatible with this; and the compiler can produce an executable that aborts on line (3).
However , as already stated, C++ does not have equivalent text and I can find no justification in the C++ Standard for considering (3) to cause UB; nor (4) or (5).
Is it possible that:
One array's off-the-end pointer points to another array's first element?
I'm not sure by what you mean by off the end pointer. As c++ iterators use half open ranges, I'm assuming you mean the pointer that represents the end position in an iteration. As that is one past the end, yes, it might overlap a next array, and hence it may not be dereferenced.
When using pointers as iterators, addresses and not values are compared. End implies the next address beyond end.
Reading beyond the bound of an array might result in dirty read.
It could be possible you may hit another array body
but it could also be possible that you may hit an unallocated region or
in case of int pointer you may point to a 4 byte region shared by an array of two shorts.
Your pointer may try to access a region which does not belongs to your process. Fatal error!
Not recommended to go beyond the bounds.
Regards
Kajal
Some C or C++ programmers are surprised to find out that even storing an invalid pointer is undefined behavior. However, for heap or stack arrays, it's okay to store the address of one past the end of the array, which allows you to store "end" positions for use in loops.
But is it undefined behavior to form a pointer range from a single stack variable, like:
char c = 'X';
char* begin = &c;
char* end = begin + 1;
for (; begin != end; ++begin) { /* do something */ }
Although the above example is pretty useless, this might be useful in the event that some function expects a pointer range, and you have a case where you simply have a single value to pass it.
Is this undefined behavior?
This is allowed, the behavior is defined and both begin and end are safely-derived pointer values.
In the C++ standard section 5.7 ([expr.add]) paragraph 4:
For the purposes of these operators, a pointer to a nonarray object behaves the same as a pointer to the first element of an array of length one with the type of the object as its element type.
When using C a similar clause can be found in the the C99/N1256 standard section 6.5.6 paragraph 7.
For the purposes of these operators, a pointer to an object that is not an element of an array behaves the same as a pointer to the first element of an array of length one with the type of the object as its element type.
As an aside, in section 3.7.4.3 ([basic.stc.dynamic.safety]) "Safely-derived pointers" there is a footnote:
This section does not impose restrictions on dereferencing pointers to memory not allocated by ::operator new. This maintains the ability of many C++ implementations to use binary libraries and components written in other languages. In particular, this applies to C binaries, because dereferencing pointers to memory allocated by malloc is not restricted.
This suggests that pointer arithmetic throughout the stack is implementation-defined behavior, not undefined behavior.
I believe that legally, you may treat a single object as an array of size one. In addition, it is most definitely legal to take a pointer one past the end of any array as long as it's not de-referenced. So I believe that it is not UB.
It is not Undefined Behavior as long as you don't dereference the invalid iterator.
You are allowed to hold a pointer to memory beyond your allocation but not allowed to dereference it.
5.7-5 of ISO14882:2011(e) states:
When an expression that has integral type is added to or subtracted
from a pointer, the result has the type of the pointer operand. If the
pointer operand points to an element of an array object, and the array
is large enough, the result points to an element offset from the
original element such that the difference of the subscripts of the
resulting and original array elements equals the integral expression.
In other words, if the expression P points to the i-th element of an
array object, the expressions (P)+N (equivalently, N+(P)) and (P)-N
(where N has the value n) point to, respectively, the i + n-th and i −
n-th elements of the array object, provided they exist. Moreover, if
the expression P points to the last element of an array object, the
expression (P)+1 points one past the last element of the array object,
and if the expression Q points one past the last element of an array
object, the expression (Q)-1 points to the last element of the array
object. If both the pointer operand and the result point to elements
of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is
undefined.
Unless I overlooked something there, the addition only applies to pointers pointing to the same array. For everything else, the last sentence applies: "otherwise, the behaviour is undefined"
edit:
Indeed, when you add 5.7-4 it turns out that the operation you do is (virtually) on an array, thus the sentence does not apply:
For the purposes of these operators, a pointer to a nonarray object
behaves the same as a pointer to the first element of an array of
length one with the type of the object as its element type.
In general it would be undefined behaviour to point beyond the memory space, however there is an exception for "one past the end", which is valid according to the standard.
Therefore in the particular example, &c+1 is a valid pointer but cannot be safely dereferenced.
You could define c as an array of size 1:
char c[1] = { 'X' };
Then the undefined behavior would become defined behavior.
Resulting code should be identical.
In an algorithm I'm writing, I can have the following (simplified of course)
int a[3] = {1,2,3};
int b = a[3];
when the index used to fill b overflows, I never use the value of b. Is the code still incorrect? Do I have to make an explicit boundary check?
This code has Undefined Behavior whether or not you use b. Why? Because a[3] is equivalent to *(a+3) by definition. And here's a quote from the standard that proved that *(a+3) is in itself undefined, regardless of whether the value is stored, used, or left alone.
When an expression that has integral
type is added to or subtracted from a
pointer, the result has the type of
the pointer operand. If the pointer
operand points to an element of an
array object, and the array is large
enough, the result points to an
element offset from the original
element such that the difference of
the subscripts of the resulting and
original array elements equals the
integral expression. In other words,
if the expression P points to the i-th
element of an array object, the
expressions (P)+N (equivalently,
N+(P)) and (P)-N (where N has the
value n) point to, respectively, the
i+n-th and i−n-th elements of the
array object, provided they exist.
Moreover, if the expression P points
to the last element of an array
object, the expression (P)+1 points
one past the last element of the array
object, and if the expression Q points
one past the last element of an array
object, the expression (Q)-1 points to
the last element of the array object.
If both the pointer operand and the
result point to elements of the same
array object, or one past the last
element of the array object, the
evaluation shall not produce an
overflow; otherwise, the behavior is
undefined.
Still incorrect, still undefined behaviour. Do the bounds check.
int b = *(a + 3); // dereferencing beyond the array bound.
Reading a[3] already causes undefined behaviour. As undefined behaviour is never locally limited, this can already lead to your harddisk drive being formatted or your computer emerging to a giant, flesh-eating zombie.
In reality, it will usually just work. But it's easy to make up a case where the end of the array marks the end of a mapped memory region, so accessing one element beyond would cause a segmentation fault. This is certainly not the case for an array of int on the stack and neither with most heap implementations, but you shouldn't rely on it.
(Whether taking the address of &a[3] is undefined behaviour as well is heavily disputed.)
It's still incorrect yes, because you access the out-of-bounds memory location to get the value a[3] and store it in the variable b.
The fact that you never use b could mean that the compiler optimizes out that line of code, so you might not ever see any adverse effects from that line being there.
However, the compiler is not required to do so, and the code itself still has undefined behavior.
Yes.
You are using the value, by copying it into b.
More specifically, dereferencing (a+3) is not allowed since the expression (a+3) is not a valid pointer... and the expression a[3] is equivalent to *(a+3) (where a has decayed to a pointer-expression).
Yes, it is wrong to read a[3] which doesn't exits.
Using b would be wrong too, but it is already too late.