String manipulation problem
http://www.ideone.com/qyTkL
In the above program (given in the book C++ Primer, Third Edition By Stanley B. Lippman, Josée Lajoie Exercise 3.14) the length of the Character pointer taken is len+1
char *pc2 = new char[ len + 1];
http://www.ideone.com/pGa6c
However, in this program the length of the Character pointer i have taken is len
char *pc2 = new char[ len ];
Why is there the need to take the length of new string as 1 greater when we get the same result. Please Explain.
Mind it the Programs i have shown here are altered slightly. Not exactly the same one as in the book.
To store a string of length n in C, you need n+1 chars. This is because a string in C is simply an array of chars terminated by the null character \0. Thus, the memory that stores the string "hello" looks like
'h' 'e' 'l' 'l' 'o' '\0'
and consists of 6 chars even though the word hello is only 5 letters long.
The inconsistency you're seeing could be a semantic one; some would say that length of the word hello is len = 5, so we need to allocate len+1 chars, while some would say that since hello requires 6 chars we should say its length (as a C string) is len=6.
Note, by the way, that the C way of storing strings is not the only possible one. For example, one could store a string as an integer (giving the string's length) followed by characters. (I believe this is what Pascal does?). If one doesn't use a length field such as this, one needs another way to know when the string stops. The C way is that the string stops whenever a null character is reached.
To get a feel for how this works, you might want to try the following:
char* string = "hello, world!";
printf("%s\n", string);
char* string2 = "hello\0, world!";
printf("%s\n", string2);
(The assignment char* string = "foo"; is just a shorthand way of creating an array with 4 elements, and giving the first the value 'f', the second 'o', the third 'o', and the fourth '\0').
It's a convention that the string is terminated by an extra null character so whoever allocates storage has to allocate len + 1 characters.
It causes problem. But, sometimes, when len isn't aligned, the OS adds some bytes after it, so the problem is hidden.
Related
Edited question
I understood my mistake in the code I had given in the original question, and the characters I was getting are garbage characters. Although, I still have a few questions about garbage characters in C:
Why can't the character be copied?
Do garbage characters have some pattern? Meaning that can you predict for an empty string what character can come, for an empty integer what will come, and so on.
When a variable is declared, why does it have a garbage character instead of being blank? Is there a specific reason of storing it with a garbage character?
For a string which is not null-terminated, will the same garbage character be printed on every OS? If yes, which one?
Are there the same garbage characters on every OS? Or are they different?
Is there a way to print these characters on the stdout buffer in C / C++?
If you see carefully in the character , there are some characters and numbers in it. Do they represent something?
Is there a list of garbage characters which can be printed in C / C++?
Original Question
Title of original question: Mysterious character output in C
I had come across this code in K & R:
int scanline (char str [], int lim) /* Line will be read in 'str []', while lim is the maximum characters to be read */
{
int c, len, j; /* 'len' will have the length of the read string */
j = 0; /* Initializing 'j' */
for (len = 0; (c = getchar ()) != EOF && c != '\n'; ++len) /* Reading a character one by one, till the user enters '\n', and checking for failure of 'getchar' */
{
if (len < (lim -2)) /* Checking that string entered has not gone beyond it's boundaries. '-2' for '\n' and '\0' */
{
str [j] = c; /* Copying read character into 'string [j]' */
++ j; /* Incrementing 'j' by 1 */
}
}
if (c == '\n') /* Checking if user has finished inputting the line */
{
str [j] = c; /* Copying newline into string */
++j;
++ len;
}
return len; /* Returning number of characters read */
}
In the K & R, it is known as getline, but I made changes, added comments, and thus defined it as scanline. To test this, I made a demo program:
#include <mocl/cancel.h>
int main (int argc, char **argv)
{
int len;
char str [50];
len = scanline (str, 50);
printf ("len = %d\n str = %s\n", len, str);
return 0;
}
The required headers and the function was in my own library, cancel.h. Then when I compiled my program, it was successful. Although, when I ran the executable, I got unexpected output (I cannot type it as I get a character which when I copy, it just gets pasted as 'm'):
The mysterious character is which when I copy, gets copied as the letter m. Also, when I run my program with different inputs, I get different mysterious outputs:
In another case, I get perfect output, just that a blank line is printed:
I had also come across this question, in which the user gets the same symbol.
What have I done till now?
I searched a lot, and I could not find any clue about this character, but if you see carefully, in the second image, I get more characters when I enter "hi this is ashish". One of them is the slash, and one is . But I get another character . I got this link which was showed how to reproduce it, and explained it, although I could not understand. When you run the code given there, you get a lot of characters, and one of them is . Although, even the author of that article could not copy it, and has not posted it. So here's the output:
That was the actual output, as that's not clear, here's a cut out version:
So basically I got to know that both the characters and are extended characters from a string. At that point, I actually figured out what was causing the problem in scanline.
The lines
if (c == '\n') /* Checking if user has finished inputting the line */
{
str [j] = c; /* Copying newline into string */
++j;
++ len;
}
were causing the problems as you were copying a newline into the string. It worked, but I'm not sure why, as doing that was just a guess. I searched but still could not find the reason.
My Questions
How does removing those lines make the program work properly?
What are the characters and ? What are they supposed to do and how did they appear over here?
Are there any more characters like this?
Why can't those characters be copied?
Is it Undefined Behavior?
There's some confusion here regarding the term garbage characters. What it refers to is any byte that resides in a variable that wasn't assigned in some well-defined way. The character A can be a garbage character if it happens to appear in (for example) a block of memory returned by malloc or an uninitialized char variable.
This is distinct from unprintable characters which are any character that does not have a well-defined representation when printed as characters. For example, ASCII codes 0 - 31 and 127 (0 - 1F and 7F hex) are control characters and therefore unprintable. There are also multibyte characters for which a particular terminal may not know how to render them.
To get into your specific questions:
Why can't the character (image) be copied?
As an unprintable character, its screen representation is not well defined. So attempting to copy and paste it from a terminal will yield unexpected results.
Do garbage characters have some pattern? Meaning that can you
predict for an empty string what character can come, for an empty
integer what will come, and so on.
The nature of garbage characters is that their contents are undefined. Trying to predict what uninitialized data will contain is a futile effort. The same piece of code compiled with two different compilers (or the same compiler with different optimization settings) can have completely different contents for any uninitialized data.
The standard doesn't say what values should go there, so implementations are free to handle it however they want. They could chose to leave whatever values happen to be at those memory addresses, they could choose to write 0 to all addresses, they could choose to write the values 0, 1, 2, 3, etc. in sequence. In other words, the contents are undefined.
When a variable is declared, why does it have a garbage character
instead of being blank? Is there a specific reason of storing it with
a garbage character?
Global variables and static local variables are initialized with all bytes zero, which is what the standard dictates. That is something that is done easily at compile time. Local variables on the other hand reside on the stack. So their values are whatever happens to be on the stack at the time the function is called.
Here's an interesting example:
void f1()
{
char str[10];
strcpy(str, "hello");
}
int main()
{
f1();
f1();
return 0;
}
Here is what a particular implementation might do:
The first time f1 is called, the local variable str is uninitialized. Then strcpy is called which copies in the string "hello". This takes up the first 6 bytes of the variable (5 for the string and 1 for the null terminator). The remaining 4 bytes are still garbage. When this functions returns, the memory that the variable str resided at is free to be used for some other purpose.
Now f1 gets called again immediately after the first call. Since no other function was called, the stack for this invocation of f1 happens to sit at the exact same place as the last invocation. So if you were to examine str at this time, you would find it contains h, e, l, l, o, and a null byte (i.e. the string "hello") for the first 6 bytes. But, this string is garbage. It wasn't specifically stored there. If some other function was called before calling f1 a second time, most likely those values would not be there.
Again, garbage means the contents are undefined. The compiler doesn't explicitly put "garbage" (or unprintable characters) in variables.
For a string which is not null-terminated, will the same garbage
character be printed on every OS? If yes, which one?
Here's one of those places you're confusing garbage and unprintable. In your specific case, the garbage character happens to be unprintable, but it doesn't have to be. Here's another example:
void f3()
{
char str1[5], str2[5];
strcpy(str1, "hello");
strcpy(str2, "test");
printf("str1=%s\n", str1);
}
Let's suppose the compiler decides to place str2 immediately after str1 in memory (although it doesn't have to). The first call to strcpy will write the string "hello" into str1, but this variable doesn't have enough room the the null terminating byte. So it gets written to the next byte in memory, which happens to be the first byte of str2. Then when the next call to strcpy runs, it puts the string "test" in str2 but in doing so it overwrites the null terminating byte put there when str1 was written to.
Then when printf gets called, you'll get this as output:
str1=hellotest
When printing str1, printf looks for the null terminator, but there isn't one inside of str1. So it keeps reading until it does. In this case there happens to be another string right after it, so it prints that as well until it finds the null terminator that was properly stored in that string.
But again, this behavior is undefined. A seemingly minor change in this function could result in str2 appearing in memory first. The compiler is free to do as it wishes in the regard, so there's no way to predict what will happen.
Are there the same garbage characters on every OS? Or are they
different?
I believe you're actually referring to unprintable characters in this case. This really depends on the character set of the OS and/or terminal in question. For example, Chinese characters are represented with multiple bytes. If your terminal can't print Chinese characters, you'll see some type of code similar to what you saw for each of the bytes. But if it can, it will display it in a well-defined manner.
Is there a way to print these characters on the stdout buffer in C /
C++?
Not as characters. You can however print out their numerical representations. For example:
void f4()
{
char c;
printf("c=%02hhX\n", (unsigned char)c);
}
The contents of c are undefined, but the above will print whatever value happens to be there in hexadecimal format.
If you see carefully in the character (image),
there are some characters and numbers in it. Do they represent
something?
Some terminals will display unprintable characters by printing a box containing the Unicode codepoint of the character so the reader can know what it is.
Unicode is a standard for text where each character is assigned a numerical code point. Besides the typical set of characters in the ASCII range, Unicode also defines other characters, such as accented letters, other alphabets like Greek, Hebrew, Cyrillic, Chinese, and Japanese, as well as various symbols. Because there are thousands of characters defined by Unicode, multiple bytes are needed to represent them. The most common encoding for Unicode is UTF-8, which allows regular ASCII characters to be encoded with one byte, and other characters to be encoded with two or more bytes as needed.
In this case, the codepoint in question is 007F. This is the DELETE control character, which is typically generated when the Delete key is pressed. Since this is a control character, your terminal is displaying it as a box with the Unicode point for the character instead of attempting to "print" it.
Is there a list of garbage characters which can be printed in C /
C++?
Again, assuming you really mean unprintable characters here, that has more to do with the terminal that's displaying the characters that with the language. Generally, control characters are unprintable, while certain multibyte characters may or may not display properly depending on the font / character set of the terminal.
For starters the function returns incorrect value of len. Let's assume that lim is equal to 2.
In this case in the loop there will be written nothing in the array due to the condition
if (len < (lim -2))
However after the first iteration of the loop len will be increased.
for (len = 0; (c = getchar ()) != EOF && c != '\n'; ++len)
^^^^^
In the second iteration again there will be written nothing in the array diue to the same condition
if (len < (lim -2))
but len will be increased.
for (len = 0; (c = getchar ()) != EOF && c != '\n'; ++len)
^^^^^
Thus nothing will be written in the array but len will be increased until for example the new line character will be encountered.
So the function is invalid. Moreover it is supposed that the function will append the read string with the terminating zero. But this is not done in the function. So you may not output the character array as a string.
The function can be written the following way
int scanline( char str [], int lim )
{
int len = 0;
int c;
while ( len < lim - 1 && ( c = getchar () ) != EOF && c != '\n' )
{
str[len++] = c;
}
if ( len < lim - 1 && c == '\n' ) str[len++] = c;
if ( len < lim ) str[len++] = '\0';
return len;
}
This question already has answers here:
How to strip all non alphanumeric characters from a string in c++?
(12 answers)
Closed 6 years ago.
I'm trying to remove all non alphabet characters from an inputed string in c++ and don't know how to. I know it probably involves ascii numbers because that's what we're learning about. I can't figure out how to remove them. We only learned up to loops and haven't started arrays yet. Not sure what to do.
If the string is Hello 1234 World&*
It would print HelloWorld
If you use std::string and STL, you can:
string s("Hello 1234 World&*");
s.erase(remove_if(s.begin(), s.end(), [](char c) { return !isalpha(c); } ), s.end());
http://ideone.com/OIsJmb
Note: If you want to be able to handle strings holding text in just about any language except English, or where programs use a locale other than the default, you can use isalpha(std::locale).
PS: If you use a c-style string such as char *, you can convert it to std::string by its constructor, and convert back by its member function c_str().
If you're working with C-style strings (e.g. char* str = "foobar") then you can't "remove" characters from a string trivially (as a string is just a sequence of characters stored sequentially in memory - removing a character means copying bytes forward to fill the empty space used by the deleted character.
You'd have to allocate space for a new string and copy characters into it as-needed. The problem is, you have to allocate memory before you fill it, so you'd over-allocate memory unless you do an initial pass to get a count of the number of characters remaining in the string.
Like so:
void BlatentlyObviousHomeworkExercise() {
char* str = "someString";
size_t strLength = ... // how `strLength` is set depends on how `str` gets its value, if it's a literal then using the `sizeof` operator is fine, otherwise use `strlen` (assuming it's a null-terminated string).
size_t finalLength = 0;
for(size_t i = 0; i < strLength; i++ ) {
char c = str[i]; // get the ith element of the `str` array.
if( IsAlphabetical(c) ) finalLength++;
}
char* filteredString = new char[ finalLength + 1 ]; // note I use `new[]` instead of `malloc` as this is C++, not C. Use the right idioms :) The +1 is for the null-terminator.
size_t filteredStringI = 0;
for(size_t i = 0; i < strLength; i++ ) {
char c = str[i];
if( IsAlphabetical(c) ) filteredString[ filteredStringI++ ] = c;
}
filteredString[ filteredStringI ] = '\0'; // set the null terminator
}
bool IsAlphabet(char c) { // `IsAlphabet` rather than `IsNonAlphabet` to avoid negatives in function names/behaviors for simplicity
return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z');
}
I do not want to spoil the solution so I will not type out the code, only describe the solution. For your problem think of iterating through your string. Start with that. Then you need to decide if the currently selected character is part of the alphabet or not. You can do this numerous different ways. Checking ASCII values? Comparing against a string of the alphabet? Once you decide if it is a letter, then you need to rebuild the new string with that letter plus the valid letters before and after that you found or will find. Finally you need to display your new string.
If you look at an ascii table, you can see that A-Z is between 65-90 and a-z is between 97-122.
So, assuming that you only need to remove those characters (not accentuated), and not other characters from other languages for example, not represented in ascii, all you would need to do is loop the string, verify if each char is in these values and remove it.
I have a fixed length character array I want to assign to a string. The problem comes if the character array is full, the assign fails. I thought of using the assign where you can supply n however that ignores \0s. For example:
std::string str;
char test1[4] = {'T', 'e', 's', 't'};
str.assign(test1); // BAD "Test2" (or some random extra characters)
str.assign(test1, 4); // GOOD "Test"
size_t len = strlen(test1); // BAD 5
char test2[4] = {'T', 'e', '\0', 't'};
str.assign(test2); // GOOD "Te"
str.assign(test2, 4); // BAD "Tet"
size_t len = strlen(test2); // GOOD 2
How can I assign a fixed length character array to a string correctly for both cases?
Use the "pair of iterators" form of assign.
str.assign(test1, std::find(test1, test1 + 4, '\0'));
Character buffers in C++ are either-or: either they are null terminated or they are not (and fixed-length). Mixing them in the way you do is thus not recommended. If you absolutely need this, there seems to be no alternative to manual copying until either the maximum length or a null terminator is reached.
for (char const* i = test1; i != test1 + length and *i != '\0'; ++i)
str += *i;
You want both NULL termination and fixed length? This is highly unusual and not recommended. You'll have to write your own function and push_back each individual character.
For the first case, when you do str.assign(test1) and str.assign(test2), you have to have /0 in your array, otherwise this is not a "char*" string and you can't assign it to std::string like this.
saw your serialization comment -- use std::vector<char>, std::array<char,4>, or just a 4 char array or container.
Your second 'bad' example - the one which prints out "Tet" - actually does work, but you have to be careful about how you check it:
str.assign(test2, 4); // BAD "Tet"
cout << "\"" << str << "\"" << endl;
does copy exactly four characters. If you run it through octal dump(od) on Linux say, using my.exe | od -c you'd get:
0000000 " T e \0 t " \n
0000007
I am a student learning C++, and I am trying to understand how null-terminated character arrays work. Suppose I define a char array like so:
char* str1 = "hello world";
As expected, strlen(str1) is equal to 11, and it is null-terminated.
Where does C++ put the null terminator, if all 11 elements of the above char array are filled with the characters "hello world"? Is it actually allocating an array of length 12 instead of 11, with the 12th character being '\0'? CPlusPlus.com seems to suggest that one of the 11 would need to be '\0', unless it is indeed allocating 12.
Suppose I do the following:
// Create a new char array
char* str2 = (char*) malloc( strlen(str1) );
// Copy the first one to the second one
strncpy( str2, str1, strlen(str1) );
// Output the second one
cout << "Str2: " << str2 << endl;
This outputs Str2: hello worldatcomY╗°g♠↕, which I assume is C++ reading the memory at the location pointed to by the pointer char* str2 until it encounters what it interprets to be a null character.
However, if I then do this:
// Null-terminate the second one
str2[strlen(str1)] = '\0';
// Output the second one again
cout << "Terminated Str2: " << str2 << endl;
It outputs Terminated Str2: hello world as expected.
But doesn't writing to str2[11] imply that we are writing outside of the allocated memory space of str2, since str2[11] is the 12th byte, but we only allocated 11 bytes?
Running this code does not seem to cause any compiler warnings or run-time errors. Is this safe to do in practice? Would it be better to use malloc( strlen(str1) + 1 ) instead of malloc( strlen(str1) )?
In the case of a string literal the compiler is actually reserving an extra char element for the \0 element.
// Create a new char array
char* str2 = (char*) malloc( strlen(str1) );
This is a common mistake new C programmers make. When allocating the storage for a char* you need to allocate the number of characters + 1 more to store the \0. Not allocating the extra storage here means this line is also illegal
// Null-terminate the second one
str2[strlen(str1)] = '\0';
Here you're actually writing past the end of the memory you allocated. When allocating X elements the last legal byte you can access is the memory address offset by X - 1. Writing to the X element causes undefined behavior. It will often work but is a ticking time bomb.
The proper way to write this is as follows
size_t size = strlen(str1) + sizeof(char);
char* str2 = (char*) malloc(size);
strncpy( str2, str1, size);
// Output the second one
cout << "Str2: " << str2 << endl;
In this example the str2[size - 1] = '\0' isn't actually needed. The strncpy function will fill all extra spaces with the null terminator. Here there are only size - 1 elements in str1 so the final element in the array is unneeded and will be filled with \0
Is it actually allocating an array of length 12 instead of 11, with the 12th character being '\0'?
Yes.
But doesn't writing to str2[11] imply that we are writing outside of the allocated memory space of str2, since str2[11] is the 12th byte, but we only allocated 11 bytes?
Yes.
Would it be better to use malloc( strlen(str1) + 1 ) instead of malloc( strlen(str1) )?
Yes, because the second form is not long enough to copy the string into.
Running this code does not seem to cause any compiler warnings or run-time errors.
Detecting this in all but the simplest cases is a very difficult problem. So the compiler authors simply don't bother.
This sort of complexity is exactly why you should be using std::string rather than raw C-style strings if you are writing C++. It's as simple as this:
std::string str1 = "hello world";
std::string str2 = str1;
The literal "hello world" is a char array that looks like:
{ 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '\0' }
So, yes, the literal is 12 chars in size.
Also, malloc( strlen(str1) ) is allocating memory for 1 less byte than is needed, since strlen returns the length of the string, not including the NUL terminator. Writing to str[strlen(str1)] is writing 1 byte past the amount of memory that you've allocated.
Your compiler won't tell you that, but if you run your program through valgrind or a similar program available on your system it'll tell you if you're accessing memory you shouldn't be.
I think you are confused by the return value of strlen. It returns the length of the string, and it should not be confused with the size of the array that holds the string. Consider this example :
char* str = "Hello\0 world";
I added a null character in the middle of the string, which is perfectly valid. Here the array will have a length of 13 (12 characters + the final null character), but strlen(str) will return 5, because there are 5 characters before the first null character. strlen just counts the characters until a null character is found.
So if I use your code :
char* str1 = "Hello\0 world";
char* str2 = (char*) malloc(strlen(str1)); // strlen(str1) will return 5
strncpy(str2, str1, strlen(str1));
cout << "Str2: " << str2 << endl;
The str2 array will have a length of 5, and won't be terminated by a null character (because strlen doesn't count it). Is this what you expected?
For a standard C string the length of the array that is storing the string is always one character longer then the length of the string in characters. So your "hello world" string has a string length of 11 but requires a backing array with 12 entries.
The reason for this is simply the way those string are read. The functions handling those strings basically read the characters of the string one by one until they find the termination character '\0' and stop at this point. If this character is missing those functions just keep reading the memory until they either hit a protected memory area that causes the host operating system to kill your application or until they find the termination character.
Also if you initialize a character array with the length 11 and write the string "hello world" into it will yield massive problems. Because the array is expected to hold at least 12 characters. That means the byte that follows the array in the memory is overwritten. Resulting in unpredictable side effects.
Also while you are working with C++, you might want to look into std:string. This class is accessible if you are using C++ and provides better handling of strings. It might be worth looking into that.
I think what you need to know is that char arrays starts from 0 and goes until array length-1 and on position array length has the terminator('\0').
In your case:
str1[0] == 'h';
str1[10] == 'd';
str1[11] == '\0';
This is why is correct str2[strlen(str1)] = '\0';
The problem with the output after the strncpy is because it copys 11 elements(0..10) so you need to put manually the terminator(str2[11] = '\0').
How to find the length of the string which also contains string terminating character in between. Example: ABC\0ABC. Is there any inbuilt function present which returns the length?
If the Input string is: ABC\0ABC
Output: 8
Since the end of a C-style string is only detectable because it ends in a '\0' character, what you're asking isn't solvable in a general way. If, like in your example, you know there's exactly one '\0' character in the string, then you can simply do:
x = strlen(string); // length of string before the first '\0'
x += strlen(string + x + 1) + 1; // plus the length of the string after
// the first '\0' plus the '\0' itself.