I am contemplating a fixed-point arithmetic library, and in order to decide on how much optimization should be done by the library itself (through expression templates) I started questioning how much will already be done by the optimizer. Take the following example for instance:
//This is a totally useless function to exemplify my point
void Compare(FixedPoint a, FixedPoint b) {
if(a/b>10) {
... do stuff
}
}
Now, in this function, a typical implementation of the FixedPoint class will cause
if( ( (a_<<N) / b_) > (10 <<N) ) {
... do stuff
}
Where N is the number of fractional bits. That expression could mathematically be transformed into:
(a_ > 10*b_)
even though this transformation will not result in the same behavior when you consider integer overflow. The users of my library will presumably care about the mathematical equivalence and would rather have the reduced version (possibly provided through expression templates).
Now, the question is: Will the optimizer dare do the optimization itself, even though the behavior is not strictly the same? Should I bother with such optimizations? Note that such optimizations aren't trivial. In reality, you rarely have to do any bit shifts when you're using fixed-point arithmetic if you actually do these optimizations.
That will depend on whether the a_ and b_ types are signed or unsigned.
In C and C++ signed overflow is technically undefined behavior, while unsigned overflow is done using two-complement arithmetic.
Nevertheless, some compilers refuse to optimize the that code because many programs rely on the two-complement behavior of the signed overflow.
Good modern compilers will have an option to enable/disable this particular assumption: that signed integers won't overflow. What option is the default will vary with the compiler.
With GCC, for example, see options -fstrict-overflow/-fno-strict-overflow and the related warning -Wstrict-overflow.
Related
I came to know through this answer that:
Signed overflow due to computation is still undefined behavior in C++20 while Signed overflow due to conversion is well defined in C++20(which was implementation defined for Pre-C++20).
And this change in the signed overflow due to conversion is because that from C++20 compilers are required use 2's complement.
My question is:
If compilers are required to use 2's complement from C++20, then why isn't signed overflow due to computation well-defined just like for signed overflow due to conversion?
That is, why(how) is there a difference between overflow due to computation and overflow due to conversion. Essentially, why these two kinds of overflows treated differently.
If non-two's-complement support had been the only concern, then signed arithmetic overflow could have been defined as having implementation defined result, just like converting an integer has been defined. There are reasons why it is UB instead, and those reasons haven't changed, nor have the rules of signed arithmetic overflow changed.
In case of any UB, there are essentially two primary reasons for it to exist:
Portability. Different systems behave in different ways and UB allows supporting all systems in an optimal way. In this case as Martin Rosenau
mentions in a comment, there are systems that don't simply produce a "wrong" value.
Optimisation. UB allows a compiler to assume that it doesn't happen, which allows for optimisations based on that assumption. Jarod42 shows an example in a comment. Another example is that with UB overflow, it is possible to deduce that adding two positive numbers never produces a negative number, nor a number that is smaller than either of the positive numbers.
The question is clear.
I wonder why they even thought this would be handy, as clearly negative indices are unusable in the containers that would be used with them (see for example QList's docs).
I thought they wanted to allow that for some crazy form of indexing, but it seems unsupported?
It also generates a ton of (correct) compiler warnings about casting to and comparing of signed/unsigned types (on MSVC).
It just seems incompatible with the STL by design for some reason...
Although I am deeply sympathetic to Chris's line of reasoning, I will disagree here (at least in part, I am playing devil's advocate). There is nothing wrong with using unsigned types for sizes, and it can even be beneficial in some circumstances.
Chris's justification for signed size types is that they are naturally used as array indices, and you may want to do arithmetic on array indices, and that arithmetic may create temporary values that are negative.
That's fine, and unsigned arithmetic introduces no problem in doing so, as long as you make sure to interpret your values correctly when you do comparisons. Because the overflow behavior of unsigned integers is fully specified, temporary overflows into the negative range (or into huge positive numbers) do not introduce any error as long as they are corrected before a comparison is performed.
Sometimes, the overflow behavior is even desirable, as the overflow behavior of unsigned arithmetic makes certain range checks expressible as a single comparison that would require two comparisons otherwise. If I want to check if x is in the range [a,b] and all the values are unsigned, I can simply do:
if (x - a < b - a) {
}
That doesn't work with signed variables; such range checks are pretty common with sizes and array offsets.
I mentioned before that a benefit is that overflow arithmetic has defined results. If your index arithmetic overflows a signed type, the behavior is implementation defined; there is no way to make your program portable. Use an unsigned type and this problem goes away. Admittedly this only applies to huge offsets, but it is a concern for some uses.
Basically, the objections to unsigned types are frequently overstated. The real problem is that most programmers don't really think about the exact semantics of the code they write, and for small integer values, signed types behave more nearly in line with their intuition. However, data sizes grow pretty fast. When we deal with buffers or databases, we're frequently way outside of the range of "small", and signed overflow is far more problematic to handle correctly than is unsigned overflow. The solution is not "don't use unsigned types", it is "think carefully about the code you are writing, and make sure you understand it".
Because, realistically, you usually want to perform arithmetic on indices, which means that you might want to create temporaries that are negative.
This is clearly painful when the underlying indexing type is unsigned.
The only appropriate time to use unsigned numbers is with modulus arithmetic.
Using "unsgined" as some kind of contract specifier "a number in the range [0..." is just clumsy, and too coarse to be useful.
Consider: What type should I use to represent the idea that the number should be a positive integer between 1 and 10? Why is 0...2^x a more special range?
C++20 will specify that signed integral types must use two's complement. This doesn't seem like a big change given that (virtually?) every implementation currently uses two's complement.
But I was wondering if this change might shift some "undefined behaviors" to be "implementation defined" or even "defined."
Consider, the absolute value function, std::abs(int) and some of its overloads. The C++ standard includes this function by reference to the C standard, which says that the behavior is undefined if the result cannot be represented.
In two's complement, there is no positive counterpart to INT_MIN:
abs(INT_MIN) == -INT_MIN == undefined behavior
In sign-magnitude representation, there is:
-INT_MIN == INT_MAX
Thus it seemed reasonable that abs() was left with some undefined behavior.
Once two's complement is required, it would seem to make sense that abs(INT_MIN)'s behavior could be fully specified or, at least, implementation defined, without any issue of backward compatibility. But I don't see any such change proposed.
The only drawback I see is that the C++ Standard would need to specify abs() explicitly rather than referencing the C Standard's description of abs(). (As far as I know, C is not mandating two's complement.)
Was this just not a priority for the committee or are there still reasons not to take advantage of the simplification and certainty that the two's complement mandate provides?
One of the specific questions considered by the committee was what to do about -INT_MIN, and the results of that poll were:
addition / subtraction / multiplication and -INT_MIN overflow is currently undefined behavior, it should instead be:
4: wrap
6: wrap or trap
5: intermediate values are mathematical integers
14: status quo (remain undefined behavior)
This was explicitly considered and people felt that the best option was keeping it undefined behavior.
To clarify on "intermediate values are mathematical integers", there is a other part of the paper which clarifies that means that (int)a + (int)b > INT_MAX might be true.
Note that implementations are free to define specific behavior in these cases if they so choose. I don't know if any of them do.
The Committee that wrote C89 deliberately avoided any judgments about things that quality implementations "should" do when practical. The published Rationale indicates that they expected implementations to behave usefully in circumstances beyond those required by the Standard (and in the case of integer overflow, even documents some very specific expectations), but for whatever reason the Committee deliberately avoided saying such things within the Standard itself.
When later C or C++ committees added new features, they were willing to consider the possibility that they might be supportable on some platforms and unsupportable on others, but there has almost never been any effort to revisit questions of whether the Standard should recognize cases where many implementations would process code in the same useful and consistent fashion even though the Standard had imposed no requirements, and provide a means by which a program could test whether an implementation supports such behavior, refuse to compile on one that doesn't, and have defined behavior on those that do.
The net effect is that something like: unsigned mul_mod_65536(unsigned short x, unsigned short y) { return (x*y) & 0xFFFFu; } may arbitrarily disrupt the behavior of calling code if the arithmetical value of x*y is between INT_MAX+1u and UINT_MAX even though that would be a situation that the authors of the Standard said they expected to be processed consistently by most implementations. The recent Standard have eliminated the main reason the authors of C89 would have expected that some implementations might process the aforementioned function strangely, but that doesn't mean that implementations haven't decided to treat it weirdly in ways the authors of C89 could never have imagined, and would never knowingly have allowed.
(I guess this question could apply to many typed languages, but I chose to use C++ as an example.)
Why is there no way to just write:
struct foo {
little int x; // little-endian
big long int y; // big-endian
short z; // native endianness
};
to specify the endianness for specific members, variables and parameters?
Comparison to signedness
I understand that the type of a variable not only determines how many bytes are used to store a value but also how those bytes are interpreted when performing computations.
For example, these two declarations each allocate one byte, and for both bytes, every possible 8-bit sequence is a valid value:
signed char s;
unsigned char u;
but the same binary sequence might be interpreted differently, e.g. 11111111 would mean -1 when assigned to s but 255 when assigned to u. When signed and unsigned variables are involved in the same computation, the compiler (mostly) takes care of proper conversions.
In my understanding, endianness is just a variation of the same principle: a different interpretation of a binary pattern based on compile-time information about the memory in which it will be stored.
It seems obvious to have that feature in a typed language that allows low-level programming. However, this is not a part of C, C++ or any other language I know, and I did not find any discussion about this online.
Update
I'll try to summarize some takeaways from the many comments that I got in the first hour after asking:
signedness is strictly binary (either signed or unsigned) and will always be, in contrast to endianness, which also has two well-known variants (big and little), but also lesser-known variants such as mixed/middle endian. New variants might be invented in the future.
endianness matters when accessing multiple-byte values byte-wise. There are many aspects beyond just endianness that affect the memory layout of multi-byte structures, so this kind of access is mostly discouraged.
C++ aims to target an abstract machine and minimize the number of assumptions about the implementation. This abstract machine does not have any endianness.
Also, now I realize that signedness and endianness are not a perfect analogy, because:
endianness only defines how something is represented as a binary sequence, but now what can be represented. Both big int and little int would have the exact same value range.
signedness defines how bits and actual values map to each other, but also affects what can be represented, e.g. -3 can't be represented by an unsigned char and (assuming that char has 8 bits) 130 can't be represented by a signed char.
So that changing the endianness of some variables would never change the behavior of the program (except for byte-wise access), whereas a change of signedness usually would.
What the standard says
[intro.abstract]/1:
The semantic descriptions in this document define a parameterized nondeterministic abstract machine.
This document places no requirement on the structure of conforming implementations.
In particular, they need not copy or emulate the structure of the abstract machine.
Rather, conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below.
C++ could not define an endianness qualifier since it has no concept of endianness.
Discussion
About the difference between signness and endianness, OP wrote
In my understanding, endianness is just a variation of the same principle [(signness)]: a different interpretation of a binary pattern based on compile-time information about the memory in which it will be stored.
I'd argue signness both have a semantic and a representative aspect1. What [intro.abstract]/1 implies is that C++ only care about semantic, and never addresses the way a signed number should be represented in memory2. Actually, "sign bit" only appears once in the C++ specs and refer to an implementation-defined value.
On the other hand, endianness only have a representative aspect: endianness conveys no meaning.
With C++20, std::endian appears. It is still implementation-defined, but let us test the endian of the host without depending on old tricks based on undefined behaviour.
1) Semantic aspect: an signed integer can represent values below zero; representative aspect: one need to, for example, reserve a bit to convey the positive/negative sign.
2) In the same vein, C++ never describe how a floating point number should be represented, IEEE-754 is often used, but this is a choice made by the implementation, in any case enforced by the standard: [basic.fundamental]/8 "The value representation of floating-point types is implementation-defined".
In addition to YSC's answer, let's take your sample code, and consider what it might aim to achieve
struct foo {
little int x; // little-endian
big long int y; // big-endian
short z; // native endianness
};
You might hope that this would exactly specify layout for architecture-independent data interchange (file, network, whatever)
But this can't possibly work, because several things are still unspecified:
data type size: you'd have to use little int32_t, big int64_t and int16_t respectively, if that's what you want
padding and alignment, which cannot be controlled strictly within the language: use #pragma or __attribute__((packed)) or some other compiler-specific extension
actual format (1s- or 2s-complement signedness, floating-point type layout, trap representations)
Alternatively, you might simply want to reflect the endianness of some specified hardware - but big and little don't cover all the possibilities here (just the two most common).
So, the proposal is incomplete (it doesn't distinguish all reasonable byte-ordering arrangements), ineffective (it doesn't achieve what it sets out to), and has additional drawbacks:
Performance
Changing the endianness of a variable from the native byte ordering should either disable arithmetic, comparisons etc (since the hardware cannot correctly perform them on this type), or must silently inject more code, creating natively-ordered temporaries to work on.
The argument here isn't that manually converting to/from native byte order is faster, it's that controlling it explicitly makes it easier to minimise the number of unnecessary conversions, and much easier to reason about how code will behave, than if the conversions are implicit.
Complexity
Everything overloaded or specialized for integer types now needs twice as many versions, to cope with the rare event that it gets passed a non-native-endianness value. Even if that's just a forwarding wrapper (with a couple of casts to translate to/from native ordering), it's still a lot of code for no discernible benefit.
The final argument against changing the language to support this is that you can easily do it in code. Changing the language syntax is a big deal, and doesn't offer any obvious benefit over something like a type wrapper:
// store T with reversed byte order
template <typename T>
class Reversed {
T val_;
static T reverse(T); // platform-specific implementation
public:
explicit Reversed(T t) : val_(reverse(t)) {}
Reversed(Reversed const &other) : val_(other.val_) {}
// assignment, move, arithmetic, comparison etc. etc.
operator T () const { return reverse(val_); }
};
Integers (as a mathematical concept) have the concept of positive and negative numbers. This abstract concept of sign has a number of different implementations in hardware.
Endianness is not a mathematical concept. Little-endian is a hardware implementation trick to improve the performance of multi-byte twos-complement integer arithmetic on a microprocessor with 16 or 32 bit registers and an 8-bit memory bus. Its creation required using the term big-endian to describe everything else that had the same byte-order in registers and in memory.
The C abstract machine includes the concept of signed and unsigned integers, without details -- without requiring twos-complement arithmetic, 8-bit bytes or how to store a binary number in memory.
PS: I agree that binary data compatibility on the net or in memory/storage is a PIA.
That's a good question and I have often thought something like this would be useful. However you need to remember that C aims for platform independence and endianness is only important when a structure like this is converted into some underlying memory layout. This conversion can happen when you cast a uint8_t buffer into an int for example. While an endianness modifier looks neat the programmer still needs to consider other platform differences such as int sizes and structure alignment and packing.
For defensive programming when you want find grain control over how some variables or structures are represented in a memory buffer then it is best to code explicit conversion functions and then let the compiler optimiser generate the most efficient code for each supported platform.
Endianness is not inherently a part of a data type but rather of its storage layout.
As such, it would not be really akin to signed/unsigned but rather more like bit field widths in structs. Similar to those, they could be used for defining binary APIs.
So you'd have something like
int ip : big 32;
which would define both storage layout and integer size, leaving it to the compiler to do the best job of matching use of the field to its access. It's not obvious to me what the allowed declarations should be.
Short Answer: if it should not be possible to use objects in arithmetic expressions (with no overloaded operators) involving ints, then these objects should not be integer types. And there is no point in allowing addition and multiplication of big-endian and little-endian ints in the same expression.
Longer Answer:
As someone mentioned, endianness is processor-specific. Which really means that this is how numbers are represented when they are used as numbers in the machine language (as addresses and as operands/results of arithmetic operations).
The same is "sort of" true of signage. But not to the same degree. Conversion from language-semantic signage to processor-accepted signage is something that needs to be done to use numbers as numbers. Conversion from big-endian to little-endian and reverse is something that needs to be done to use numbers as data (send them over the network or represent metadata about data sent over the network such as payload lengths).
Having said that, this decision appears to be mostly driven by use cases. The flip side is that there is a good pragmatic reason to ignore certain use cases. The pragmatism arises out of the fact that endianness conversion is more expensive than most arithmetic operations.
If a language had semantics for keeping numbers as little-endian, it would allow developers to shoot themselves in the foot by forcing little-endianness of numbers in a program which does a lot of arithmetic. If developed on a little-endian machine, this enforcing of endianness would be a no-op. But when ported to a big-endian machine, there would a lot of unexpected slowdowns. And if the variables in question were used both for arithmetic and as network data, it would make the code completely non-portable.
Not having these endian semantics or forcing them to be explicitly compiler-specific forces the developers to go through the mental step of thinking of the numbers as being "read" or "written" to/from the network format. This would make the code which converts back and forth between network and host byte order, in the middle of arithmetic operations, cumbersome and less likely to be the preferred way of writing by a lazy developer.
And since development is a human endeavor, making bad choices uncomfortable is a Good Thing(TM).
Edit: here's an example of how this can go badly:
Assume that little_endian_int32 and big_endian_int32 types are introduced. Then little_endian_int32(7) % big_endian_int32(5) is a constant expression. What is its result? Do the numbers get implicitly converted to the native format? If not, what is the type of the result? Worse yet, what is the value of the result (which in this case should probably be the same on every machine)?
Again, if multi-byte numbers are used as plain data, then char arrays are just as good. Even if they are "ports" (which are really lookup values into tables or their hashes), they are just sequences of bytes rather than integer types (on which one can do arithmetic).
Now if you limit the allowed arithmetic operations on explicitly-endian numbers to only those operations allowed for pointer types, then you might have a better case for predictability. Then myPort + 5 actually makes sense even if myPort is declared as something like little_endian_int16 on a big endian machine. Same for lastPortInRange - firstPortInRange + 1. If the arithmetic works as it does for pointer types, then this would do what you'd expect, but firstPort * 10000 would be illegal.
Then, of course, you get into the argument of whether the feature bloat is justified by any possible benefit.
From a pragmatic programmer perspective searching Stack Overflow, it's worth noting that the spirit of this question can be answered with a utility library. Boost has such a library:
http://www.boost.org/doc/libs/1_65_1/libs/endian/doc/index.html
The feature of the library most like the language feature under discussion is a set of arithmetic types such as big_int16_t.
Because nobody has proposed to add it to the standard, and/or because compiler implementer have never felt a need for it.
Maybe you could propose it to the committee. I do not think it is difficult to implement it in a compiler: compilers already propose fundamental types that are not fundamental types for the target machine.
The development of C++ is an affair of all C++ coders.
#Schimmel. Do not listen to people who justify the status quo! All the cited arguments to justify this absence are more than fragile. A student logician could find their inconsistence without knowing anything about computer science. Just propose it, and just don't care about pathological conservatives. (Advise: propose new types rather than a qualifier because the unsigned and signed keywords are considered mistakes).
Endianness is compiler specific as a result of being machine specific, not as a support mechanism for platform independence. The standard -- is an abstraction that has no regard for imposing rules that make things "easy" -- its task is to create similarity between compilers that allows the programmer to create "platform independence" for their code -- if they choose to do so.
Initially, there was a lot of competition between platforms for market share and also -- compilers were most often written as proprietary tools by microprocessor manufacturers and to support operating systems on specific hardware platforms. Intel was likely not very concerned about writing compilers that supported Motorola microprocessors.
C was -- after all -- invented by Bell Labs to rewrite Unix.
Many people frequently point out in discussions of the right-shift operator that the C standard explicitly states that the effect of right-shifting a negative number is implementation defined. I can understand the historical basis for that statement, given that C compilers have been used to generate code for a variety of platforms which do not use two's-complement arithmetic. All new-product development that I'm aware of, however, has centered around processors which have no inherent support for any kind of integer arithmetic other than two's-complement.
If code wishes to perform a floored signed integer division by a power of two, and it is only going to be run for current or future architectures, is there any realistic danger that any future compiler is going to interpret the right-shift operator as doing anything else? If there is a realistic possibility, is there any good way to provide for it without adversely affecting readability, performance, or both? Are there any other dependencies which would justify making an outright assumption of the operator's behavior (e.g. code will be useless on implementations that don't support function X, and implementations are unlikely to support X if they don't use sign-extended right shifts)?
Note: I ask under the C99 and C11 tags because I would expect that newer language features would be among the things which, if supported, would suggest that a platform is probably going to use a right-shift which is arithmetically equivalent to floored division, and would be interested in knowing of any C99 or C11 compilers which implement right-shift any other way.
This is just one of many reasons why this is so, but consider the signal processing case:
1111 0001 >> 1
0000 1111 >> 1
In the form of shift right arithmetic (SRA) you refer to you would get the following:
1111 0001 >> 1 = 1111 1000
OR
-15 >> 1 = -8
0000 1111 >> 1 = 0000 0111
OR
15 >> 1 = 7
So what's the problem? Consider a digital signal with an amplitude of 15 "units". Dividing this signal by 2 should yield equivalent behavior regardless of sign. However, with a SRA as above, the positive signal of 15 would result in a signal with 7 amplitude, while a negative signal of 15 would result in a signal with 8 amplitude. This unevenness results in a DC bias in the output. For this reason, some DSP processors choose to implement a "round to 0" shift right arithmetic, or other methods altogether. Because the C99 standard is worded as it is, these processors can still be compliant.
On these processors, -1 >> 1 == 0
Related Wiki
Theoretically, nowadays there are subtleties in compiler implementations that could abuse the so-called "undefined behavior", beyond what the backend cpu would do to the actual integers on registers (or "files" or memory locations or whatever):
Cross compilers are usual stuff: the compiler may abuse implementation dependent specs when executing simple calculations itself. Consider the case where the target architecture implements this one way and the hosting the other. In your particular example, compile-time constants could end up as 1, even if whatever assembly output in the target architecture would otherwise gives 0 (I can't think of no such architecture). And then again, vice-versa. There would be no requirement (other than user-base complaining) for a compiler implementer to otherwise care.
Consider CLANG and other compilers that generate intermediate abstract code. There's nothing preventing the type mechanics to optimize some operations up to the last bit at intermediate time on some code paths (i.e. when it can reduce code to constants, loop folding comes to mind), while leaving the assembly backend to resolve this at runtime in other paths. In other words, you could see mixed behavior. In this kind of abstract, there's no obligation from the implementer to obey any standards other than whatever the C language expects. Think the case where all integer math is done by arbitrary precision arithmetics libraries instead of direct mapping to the host cpu integers. The implementation may decide for whatever reason that this is undefined and would return 0. It can do so for any of the signed arithmetic undefined behavior, and there's lots of then in the ISO C standard, specially the wrapping and such.
Consider the (theoretical) case where instead of emitting the full instruction to do the low-level op, the compiler hijacks a sub-operation. An example of this is the ARM with barrel shifter: an explicit instruction (i.e. add or whatever) could have a range and semantics, but the sub-operation could operate with slightly different limits. The compiler could exploit this up to limits where behavior could differ, for instance one case can set result flags and the other not. I can't think of a concrete case where it matters, but it's a possibility that some weird instruction can only deal with subsets of "otherwise normal behavior" and the compiler may assume it's a nice optimization since undefined behavior is supposed to really means undefined :-)
Apart from weird architectures where you would actually have weird behavior at runtime, these are some of the reasons I can think of why you can't assume anything beyond the undefined behavior.
Having said all that, however, we must also consider:
You asked for a C99 compiler. Most weird architectures (i.e. embedded targets) don't have a C99 compiler.
Most "large scale" compiler implementers deals with very large user code bases and generally speaking face support nightmares by over-optimizing minor subtleties. So they don't. Or they do it the way other players are doing.
In the particular case of signed integer "undefined behavior", normally the complementary unsigned operation is a defined operation, i.e. I've seen code casting signed to unsigned only to do the op and then casting the result back.
I think the best straight answer I could give is "you might assume all this is irrelevant, but perhaps you shouldn't".