I would like to store an unsigned char into a char by means of a shift. As the two data types have the same length (1 byte on my machine), I would have expected the following code to work:
#include <iostream>
#include <cstring>
#include <cstdio>
using namespace std;
int main () {
printf ("%d\n", sizeof(char));
printf ("%d\n", sizeof(unsigned char));
unsigned char test = 49;
char testchar = (char) (test - 127);
printf ("%x\n", testchar);
return 0;
}
but it doesn't. In particular, I got the following output:
1
1
ffffffb2
that suggests that the char has been casted to int. Does anybody has an explanation and, hopefully, a solution?
%x is a specifier for a 4-byte int. To print one byte char use %hhx.
printf typecasts its arguments according to the format specifiers passed to it.That is why testchar was type promoted to int.
printf is a variable argument function, and as such it's arguments are subject to default promotion rules. For this case, your char is promoted to an int, and in that process is sign extended.
A 2's complement int of 4 bytes with the binary pattern 0xffffffb2 is -78. Print it as a char with the %hhx specifier.
See also Which integral promotions do take place when printing a char?
%x is only for printing unsigned int, however you supply a char.
Using %x with a negative value of char causes undefined behaviour.
Aside: The C Standard specification of printf is not particularly clear; some feel that passing anything except exactly an unsigned int causes undefined behaviour. Others (including myself) feel that it's OK to pass arguments that are not specifically unsigned int, but after the default argument promotions, have type int with a non-negative value. The standard does guarantee that non-negative ints have the same representation as the unsigned int with the same value.
Some of the other answers suggest %hhx, but that is not any better than %x. The standard (on a sensible interpretation) specifies that %hhx only be used with an unsigned char argument, and %hhd only be used with a signed char argument. There is actually no modifier for plain char.
Either way you look at it, nowhere can printf be used to convert negative values to positive representations in a well-defined manner. You must convert the argument yourself and then use a matching format specifier. In this case:
printf ("%hhx\n", (unsigned char)testchar);
would be one option. IMO %x could be used here, but as mentioned above, some disagree.
NB. The wrong format specifier is used in printf ("%d\n", sizeof(char)); and the line following that. The specifier for size_t is %zu. So you could either use %zu, or cast the argument to int, or even better:
printf("1\n");
What happens is !!!!
1) unsigned char test = 49; // hex value 31 gets assigned
2) char testchar = (char) (test - 127); // 49-127 = -78 ie; 0xb2 (as unsigned),converting it to signed char results F padding before b2 to indicate it as negative
3) printf ("%x\n", testchar); //Since %x is a specifier for a 4-byte int (as #Don't You Worry Child said) ffffffb2, 4 byte output is obtained
So try as per #Don't You Worry Child said
I would have expected the following code to work:
It won't.
Ignoring the issues other people have pointed out with how you're printing the character, there is no guarantee in the standard that your code will work. Why?
Because char does not have to be signed. Whether char is signed or unsigned is implementation-dependent. Some implementations make char signed, others make it unsigned.
As such, there's no guarantee that (char) (test - 127) will produce a value that can be represented by char.
C++(14) does allow lossless conversion between unsigned char and char. The stadnard says (3.9.1/1):
For each value i of type unsigned char in the range 0 to 255 inclusive, there exists a value j of type char such that the result of an integral conversion (4.7) from i to char is j, and the result of an integral conversion from j to unsigned char is i.
Related
I am trying to print char as positive value:
char ch = 212;
printf("%u", ch);
but I get:
4294967252
How I can get 212 in the output?
Declare your ch as
unsigned char ch = 212 ;
And your printf will work.
This is because in this case the char type is signed on your system*. When this happens, the data gets sign-extended during the default conversions while passing the data to the function with variable number of arguments. Since 212 is greater than 0x80, it's treated as negative, %u interprets the number as a large positive number:
212 = 0xD4
When it is sign-extended, FFs are pre-pended to your number, so it becomes
0xFFFFFFD4 = 4294967252
which is the number that gets printed.
Note that this behavior is specific to your implementation. According to C99 specification, all char types are promoted to (signed) int, because an int can represent all values of a char, signed or unsigned:
6.1.1.2: If an int can represent all values of the original type, the value is converted to an int; otherwise, it is converted to an unsigned int.
This results in passing an int to a format specifier %u, which expects an unsigned int.
To avoid undefined behavior in your program, add explicit type casts as follows:
unsigned char ch = (unsigned char)212;
printf("%u", (unsigned int)ch);
* In general, the standard leaves the signedness of char up to the implementation. See this question for more details.
There are two bugs in this code. First, in most C implementations with signed char, there is a problem in char ch = 212 because 212 does not fit in an 8-bit signed char, and the C standard does not fully define the behavior (it requires the implementation to define the behavior). It should instead be:
unsigned char ch = 212;
Second, in printf("%u",ch), ch will be promoted to an int in normal C implementations. However, the %u specifier expects an unsigned int, and the C standard does not define behavior when the wrong type is passed. It should instead be:
printf("%hhu", ch);
(For %hhu, printf expects an unsigned char that has, in normal C implementations, been promoted to int.)
In case you cannot change the declaration for whatever reason, you can do:
char ch = 212;
printf("%d", (unsigned char) ch);
The range of char is 127 to -128. If you assign 212, ch stores -44 (212-128-128) not 212.So if you try to print a negative number as unsigned you get (MAX value of unsigned int)-abs(number) which in this case is 4294967252
So if you want to store 212 as it is in ch the only thing you can do is declare ch as
unsigned char ch;
now the range of ch is 0 to 255.
Because char is by default signed declared that means the range of the variable is
-127 to +127>
your value is overflowed. To get the desired value you have to declared the unsigned modifier. the modifier's (unsigned) range is:
0 to 255
to get the the range of any data type follow the process 2^bit example charis 8 bit length to get its range just 2 ^(power) 8.
printf has a series of "length modifier" flags you can pass in the format specifications, for example %d is for an int argument, but %hhd is for a signed char (8-bit) and %lld is a long long (64-bit).
I can see why this is necessary for larger-than-int types, but smaller types are implicitly promoted to int anyway.
In what situation would %hhd give a different output to %d, assuming you actually do pass a signed char?
The promotion will preserve the signedness. The compiler knows the type of the variable, but the printf does not. If you try to print a signed variable with an unsigned conversation specifier, it'll treat the variable differently (i.e, printing out sign-extended F's). The length modifier hh says you explicitly want to print the signed char with an unsigned format. Live example:
#include <stdio.h>
int main(void) {
char* s = "\u00d1"; /* Ñ */
for (char* p = s; *p; ++p) printf("%02X (%02hhX)\n", *p, *p);
return 0;
}
Output:
FFFFFFC3 (C3)
FFFFFF91 (91)
You need the length modifiers for reading shorter data with scanf. Because printf and scanf shares the mostly same set of formatting strings, including modifiers, it's retained primarily of consistence concerns.
#include <stdio.h>
int main() {
int i,n;
int a = 123456789;
void *v = &a;
unsigned char *c = (unsigned char*)v;
for(i=0;i< sizeof a;i++) {
printf("%u ",*(c+i));
}
char *cc = (char*)v;
printf("\n %d", *(cc+1));
char *ccc = (char*)v;
printf("\n %u \n", *(ccc+1));
}
This program generates the following output on my 32 bit Ubuntu machine.
21 205 91 7
-51
4294967245
First two lines of output I can understand =>
1st Line : sequence of storing of bytes in memory.
2nd Line : signed value of the second byte value (2's complement).
3rd Line : why such a large value ?
please explain the last line of output. WHY three bytes of 1's are added
because (11111111111111111111111111001101) = 4294967245 .
Apparently your compiler uses signed characters and it is a little endian, two's complement system.
123456789d = 075BCD15h
Little endian: 15 CD 5B 07
Thus v+1 gives value 0xCD. When this is stored in a signed char, you get -51 in signed decimal format.
When passed to printf, the character *(ccc+1) containing value -51 first gets implicitly type promoted to int, because variadic functions like printf has a rule stating that all small integer parameters will get promoted to int (the default argument promotions). During this promotion, the sign is preserved. You still have value -51, but for a 32 bit signed integer, this gives the value 0xFFFFFFCD.
And finally the %u specifier tells printf to treat this as an unsigned integer, so you end up with 4.29 bil something.
The important part to understand here is that %u has nothing to do with the actual type promotion, it just tells printf how to interpret the data after the promotion.
-51 store in 8 bit hex is 0xCD. (Assuming 2s compliment binary system)
When you pass it to a variadic function like printf, default argument promotion takes place and char is promoted to int with representation 0xFFFFFFCD (for 4 byte int).
0xFFFFFFCD interpreted as int is -51 and interpreted as unsigned int is 4294967245.
Further reading: Default argument promotions in C function calls
please explain the last line of output. WHY three bytes of 1's are
added
This is called sign extension. When a smaller signed number is assigned (converted) to larger number, its signed bit get's replicated to ensure it represents same number (for example in 1s and 2s compliment).
Bad printf format specifier
You are attempting to print a char with specifier "%u" which specifies unsigned [int]. Arguments which do not match the conversion specifier in printf is undefined behavior from 7.19.6.1 paragraph 9.
If a conversion specification is invalid, the behavior is undefined. If
any argument is not the correct type for the corresponding conversion
specification, the behavior is undefined.
Use of char to store signed value
Also to ensure char contains signed value, explicitly use signed char as char may behave as signed char or unsigned char. (In latter case, output of your snippet may be 205 205). In gcc you can force char to behave as unsigned char with -funsigned-char option.
I want to use a function that expects data like this:
void process(char *data_in, int data_len);
So it's just processing some bytes really.
But I'm more comfortable working with "unsigned char" when it comes to raw bytes (it somehow "feels" more right to deal with positive 0 to 255 values only), so my question is:
Can I always safely pass a unsigned char * into this function?
In other words:
Is it guaranteed that I can safely convert (cast) between char and unsigned char at will, without any loss of information?
Can I safely convert (cast) between pointers to char and unsigned char at will, without any loss of information?
Bonus: Is the answer same in C and C++?
The short answer is yes if you use an explicit cast, but to explain it in detail, there are three aspects to look at:
1) Legality of the conversion
Converting between signed T* and unsigned T* (for some type T) in either direction is generally possible because the source type can first be converted to void * (this is a standard conversion, §4.10), and the void * can be converted to the destination type using an explicit static_cast (§5.2.9/13):
static_cast<unsigned char*>(static_cast<void *>(data_in))
This can be abbreviated (§5.2.10/7) as
reinterpret_cast<unsigned char *>(data_in)
because char is a standard-layout type (§3.9.1/7,8 and §3.9/9) and signedness does not change alignment (§3.9.1/1). It can also be written as a C-style cast:
(unsigned char *)(data_in)
Again, this works both ways, from unsigned* to signed* and back. There is also a guarantee that if you apply this procedure one way and then back, the pointer value (i.e. the address it's pointing to) won't have changed (§5.2.10/7).
All of this applies not only to conversions between signed char * and unsigned char *, but also to char */unsigned char * and char */signed char *, respectively. (char, signed char and unsigned char are formally three distinct types, §3.9.1/1.)
To be clear, it doesn't matter which of the three cast-methods you use, but you must use one. Merely passing the pointer will not work, as the conversion, while legal, is not a standard conversion, so it won't be performed implicitly (the compiler will issue an error if you try).
2) Well-definedness of the access to the values
What happens if, inside the function, you dereference the pointer, i.e. you perform *data_in to retrieve a glvalue for the underlying character; is this well-defined and legal? The relevant rule here is the strict-aliasing rule (§3.10/10):
If a program attempts to access the stored value of an object through a glvalue of other than one of the following types the behavior is undefined:
[...]
a type that is the signed or unsigned type corresponding to the dynamic type of the object,
[...]
a char or unsigned char type.
Therefore, accessing a signed char (or char) through an unsigned char* (or char) and vice versa is not disallowed by this rule – you should be able to do this without problems.
3) Resulting values
After derefencing the type-converted pointer, will you be able to work with the value you get? It's important to bear in mind that the conversion and dereferencing of the pointer described above amounts to reinterpreting (not changing!) the bit pattern stored at the address of the character. So what happens when a bit pattern for a signed character is interpreted as that of an unsigned character (or vice versa)?
When going from unsigned to signed, the typical effect will be that for values between 0 and 128 nothing happens, and values above 128 become negative. Similar in reverse: When going from signed to unsigned, negative values will appear as values greater than 128.
But this behaviour isn't actually guaranteed by the Standard. The only thing the Standard guarantees is that for all three types, char, unsigned char and signed char, all bits (not necessarily 8, btw) are used for the value representation. So if you interpret one as the other, make a few copies and then store it back to the original location, you can be sure that there will be no information loss (as you required), but you won't necessarily know what the values actually mean (at least not in a fully portable way).
unsigned char or signed char is just interpretation: there is no conversion happening.
Since you are processing bytes, to show intent, it would be better to declare as
void process(unsigned char *data_in, int data_len);
[As noted by an editor: A plain char may be either a signed or an unsigned type. The C and C++ standards explicitly allow either (it is always a separate type from either unsigned char or signed char, but has the same range as one of them)]
Yes, you can always convert from char to unsigned char & vice versa without problems. If you run the following code, and compare it with an ASCII table (ref. http://www.asciitable.com/), you can see a proof by yourself, and how the C/C++ deal with the conversions - they deal exactly in the same way:
#include "stdio.h"
int main(void) {
//converting from char to unsigned char
char c = 0;
printf("%d byte(s)\n", sizeof(char)); // result: 1byte, i.e. 8bits, so there are 2^8=256 values that a char can store.
for (int i=0; i<256; i++){
printf("int value: %d - from: %c\tto: %c\n", c, c, (unsigned char) c);
c++;
}
//converting from unsigned char to char
unsigned char uc = 0;
printf("\n%d byte(s)\n", sizeof(unsigned char));
for (int i=0; i<256; i++){
printf("int value: %d - from: %c\tto: %c\n", uc, uc, (char) uc);
uc++;
}
}
I will not post the output because it has too many lines! It can be noticed in the output that in the first half of each section, i.e. from i=0:127, the conversion from chars to unsigned chars and vice-versa works well, without any modification or loss.
However, from i=128:255 the chars and the unsigned chars cannot be casted, or you would have different outputs, because unsigned char saves the values from [0:256] and char saves the values in the interval [-128:127]). Nevertheless, the behaviour in this 2nd half is irrelevant, because in C/C++, in general, you only lead with chars/unsigned chars as ASCII characters, whose can take only 128 different values and the other 128 values (positive for chars or negative for unsigned chars) are never used.
If you never put a value in a char that doesn't represent a character, and you never put a value in an unsigned char that doesn't represent a character, everything will be OK!
extra: even if you use UTF-8 or other encodings (for special characters) in your strings with C/C++, everything with this kind of casts would be OK, for instance, using UTF-8 encoding (ref. http://lwp.interglacial.com/appf_01.htm):
char hearts[] = {0xe2, 0x99, 0xa5, 0x00};
char diamonds[] = {0xe2, 0x99, 0xa6, 0x00};
char clubs[] = {0xe2, 0x99, 0xa3, 0x00};
char spades[] = {0xe2, 0x99, 0xa0, 0x00};
printf("hearts (%s)\ndiamonds (%s)\nclubs (%s)\nspades (%s)\n\n", hearts, diamonds, clubs, spades);
the output of that code will be:
hearts (♥)
diamonds (♦)
clubs (♣)
spades (♠)
even if you cast each of its chars to unsigned chars.
so:
"can I always safely pass a unsigned char * into this function?"
yes!
"is it guaranteed that I can safely convert (cast) between char and unsigned char at will, without any loss of information?"
yes!
"can I safely convert (cast) between pointers to char and unsigned char at will, without any loss of information?"
yes!
"is the answer same in C and C++?"
yes!
Semantically, passing between unsigned char * and char * are safe, and even though casting between them, so as in c++.
However, consider the following sample code:
#include "stdio.h"
void process_unsigned(unsigned char *data_in, int data_len) {
int i=data_len;
unsigned short product=1;
for(; i--; product*=data_in[i])
;
for(i=sizeof(product); i--; ) {
data_in[i]=((unsigned char *)&product)[i];
printf("%d\r\n", data_in[i]);
}
}
void process(char *data_in, int data_len) {
int i=data_len;
unsigned short product=1;
for(; i--; product*=data_in[i])
;
for(i=sizeof(product); i--; ) {
data_in[i]=((unsigned char *)&product)[i];
printf("%d\r\n", data_in[i]);
}
}
void main() {
unsigned char
a[]={1, -1},
b[]={1, -1};
process_unsigned(a, sizeof(a));
process(b, sizeof(b));
getch();
}
output:
0
255
-1
-1
All the code inside process_unsigned and process are just IDENTICAL. The only difference is unsigned and signed. This sample shows that the code in the black box, do be affected by the SIGN, and nothing is guaranteed between the callee and caller.
Thus I would say that, it's applicable of passing only, but none of any other possibilities is guaranteed.
You can pass a pointer to a different kind of char, but you may need to explicitly cast it. The pointers are guaranteed to be the same size and the same values. There isn't going to be any information loss during the conversion.
If you want to convert char to unsigned char inside the function, you just assign a char value to an unsigned char variable or cast the char value to unsigned char.
If you need to convert unsigned char to char without data loss, it's a bit harder, but still possible:
#include <limits.h>
char uc2c(unsigned char c)
{
#if CHAR_MIN == 0
// char is unsigned
return c;
#else
// char is signed
if (c <= CHAR_MAX)
return c;
else
// ASSUMPTION 1: int is larger than char
// ASSUMPTION 2: integers are 2's complement
return c - CHAR_MAX - 1 - CHAR_MAX - 1;
#endif
}
This function will convert unsigned char to char in such a way that the returned value can be converted back to the same unsigned char value as the parameter.
You really need to view the code to process() to know if you can safely pass in unsigned characters. If the function uses the characters as an index into an array, then no, you can't use unsigned data.
Is it safe to convert, say, from an unsigned char * to a signed char * (or just a char *?
The access is well-defined, you are allowed to access an object through a pointer to signed or unsigned type corresponding to the dynamic type of the object (3.10/15).
Additionally, signed char is guaranteed not to have any trap values and as such you can safely read through the signed char pointer no matter what the value of the original unsigned char object was.
You can, of course, expect that the values you read through one pointer will be different from the values you read through the other one.
Edit: regarding sellibitze's comment, this is what 3.9.1/1 says.
A char, a signed char, and an unsigned char occupy the same amount of storage and have the same alignment requirements (3.9); that is, they have the same object representation. For character types, all bits of the object representation participate in the value representation. For unsigned character types, all possible bit patterns of the value representation represent numbers.
So indeed it seems that signed char may have trap values. Nice catch!
The conversion should be safe, as all you're doing is converting from one type of character to another, which should have the same size. Just be aware of what sort of data your code is expecting when you dereference the pointer, as the numeric ranges of the two data types are different. (i.e. if your number pointed by the pointer was originally positive as unsigned, it might become a negative number once the pointer is converted to a signed char* and you dereference it.)
Casting changes the type, but does not affect the bit representation. Casting from unsigned char to signed char does not change the value at all, but it affects the meaning of the value.
Here is an example:
#include <stdio.h>
int main(int args, char** argv) {
/* example 1 */
unsigned char a_unsigned_char = 192;
signed char b_signed_char = b_unsigned_char;
printf("%d, %d\n", a_signed_char, a_unsigned_char); //192, -64
/* example 2 */
unsigned char b_unsigned_char = 32;
signed char a_signed_char = a_unsigned_char;
printf("%d, %d\n", b_signed_char, b_unsigned_char); //32, 32
return 0;
}
In the first example, you have an unsigned char with value 192, or 110000000 in binary. After the cast to signed char, the value is still 110000000, but that happens to be the 2s-complement representation of -64. Signed values are stored in 2s-complement representation.
In the second example, our unsigned initial value (32) is less than 128, so it seems unaffected by the cast. The binary representation is 00100000, which is still 32 in 2s-complement representation.
To "safely" cast from unsigned char to signed char, ensure the value is less than 128.
It depends on how you are going to use the pointer. You are just converting the pointer type.
You can safely convert an unsigned char* to a char * as the function you are calling will be expecting the behavior from a char pointer, but, if your char value goes over 127 then you will get a result that will not be what you expected, so just make certain that what you have in your unsigned array is valid for a signed array.
I've seen it go wrong in a few ways, converting to a signed char from an unsigned char.
One, if you're using it as an index to an array, that index could go negative.
Secondly, if inputted to a switch statement, it may result in a negative input which often is something the switch isn't expecting.
Third, it has different behavior on an arithmetic right shift
int x = ...;
char c = 128
unsigned char u = 128
c >> x;
has a different result than
u >> x;
Because the former is sign-extended and the latter isn't.
Fourth, a signed character causes underflow at a different point than an unsigned character.
So a common overflow check,
(c + x > c)
could return a different result than
(u + x > u)
Safe if you are dealing with only ASCII data.
I'm astonished it hasn't been mentioned yet: Boost numeric cast should do the trick - but only for the data of course.
Pointers are always pointers. By casting them to a different type, you only change the way the compiler interprets the data pointed to.