Related
I would like to concatenate 2 strings in C or C++ without new memory allocation and copying. Is it possible?
Possible C code:
char* str1 = (char*)malloc(100);
char* str2 = (char*)malloc(50);
char* str3 = /* some code that concatenates these 2 strings
without copying to occupy a continuous memory region */
Then, when I don't need them any more, I just do:
free(str1);
free(str2);
Or if possible, I would like to achieve the same in C++, using std::string or maybe char*, but using new and delete (possibly void operator delete ( void* ptr, std::size_t sz ) operator (C++14) on the str3).
There are a lot of questions about strings concatenation, but I haven't found one that asks the same.
No, it is not possible
In C, malloc operations return blocks of memory that have no relationship to each other. But in C, strings must be a continuous array of bytes. So there is no way to extend str1 without copying, let alone concatenate.
For C++, perhaps ropes may be of interest: See this answer.
Ropes are allocated in chunks that do not have to be contiguous. This supports O(1) concatenation. However, the accessors make it appear as a single string of bytes. I'm certain that to convert ropes back to std::string or C style strings will take a copy however, but this is probably the closest to what you want.
Also, it is probably a premature optimization to worry about the costs of copying a few strings around. Unless you are moving lots of data, it won't matter
Text concatenation is possible by writing your own string data structure. Easier in C++ than C.
struct My_String
{
std::vector<char *> text_fragments;
};
You would have to implement all the text manipulation and searching algorithms based on this data structure. Nothing in the C library could be applied to the My_String structure. The std::string in C++ would not be compatible.
One of the issues is how to handle text modification. If one of the text fragments is a constant literal (that can't be modified), it would need to be copied before it could be modified. But copying is against the requirements. :-(
A "string" in C is a an array of chars with a null char at the end. And an array is "a data structure that lets you store one or more elements consecutively in memory". GNU C reference
You cannot concatenate two arrays that are not in consecutive memory blocks without copying one of them. You can do it however without allocating new memory. E.g.
char* str1 = malloc(100); // size 100 bytes, uninitialised
str1[0] = '\0'; // string length 0, size of str1 100
strcat(str1, "a"); // string length 1, size of str1 still 100
strcat(str1, "b"); // string length 2, size of str1 still 100
You could if you want retrieve chars of 2 strings as if they were one without copying or reallocating. Here is an example function to do that (simple example, don't use in production code)
char* str1 = (char*)malloc(100);
char* str2 = (char*)malloc(50);
char get_char(int i) {
if (i > 0 && i < 100) {
return str1[i];
}
if (i >= 100 && i < 150) {
return str2[i-100];
}
return 0;
}
But in such a case you couldn't have a char* str3 to perform pointer arithmetic with and access all 150 chars.
Tags C and C++ are contradictory. In C, I'd recommend exploring realloc. You can code something along following lines:
char* str = malloc(50);
str = realloc(ptr, 55);
If you are lucky, the realloc call will not reallocate new memory and just 'extened' the already allocated segment, but there is no guarantee for this. This way you at at least have a shot of avoiding reallocations of the string. You will still have to copy contents of the second string into neweley allocated memory.
There is a function which sends data to the server:
int send(
_In_ SOCKET s,
_In_ const char *buf,
_In_ int len,
_In_ int flags
);
Providing length seems to me a little bit weird. I need to write a function, sending a line to the server and wrapping this one such that we don't have to provide length explicitly. I'm a Java-developer and in Java we could just invoke String::length() method, but now we're not in Java. How can I do that, unless providing length as a template parameter? For instance:
void sendLine(SOCKET s, const char *buf)
{
}
Is it possible to implement such a function?
Use std string:
void sendLine(SOCKET s, const std::string& buf) {
send (s, buf.c_str(), buf.size()+1, 0); //+1 will also transmit terminating \0.
}
On a side note: your wrapper function ignores the return value and doesn't take any flags.
you can retrieve the length of C-string by using strlen(const char*) function.
make sure all the strings are null terminated and keep in mind that null-termination (the length grows by 1)
Edit: My answer originally only mentioned std::string. I've now also added std::vector<char> to account for situations where send is not used for strictly textual data.
First of all, you absolutely need a C++ book. You are looking for either the std::string class or for std::vector<char>, both of which are fundamental elements of the language.
Your question is a bit like asking, in Java, how to avoid char[] because you never heard of java.lang.String, or how to avoid arrays in general because you never heard of java.util.ArrayList.
For the first part of this answer, let's assume you are dealing with just text output here, i.e. with output where a char is really meant to be a text character. That's the std::string use case.
Providing lenght seems to me a little bit wierd.
That's the way strings work in C. A C string is really a pointer to a memory location where characters are stored. Normally, C strings are null-terminated. This means that the last character stored for the string is '\0'. It means "the string stops here, and if you move further, you enter illegal territory".
Here is a C-style example:
#include <string.h>
#include <stdio.h>
void f(char const* s)
{
int l = strlen(s); // l = 3
printf(s); // prints "foo"
}
int main()
{
char* test = new char[4]; // avoid new[] in real programs
test[0] = 'f';
test[1] = 'o';
test[2] = 'o';
test[3] = '\0';
f(test);
delete[] test;
}
strlen just counts all characters at the specified position in memory until it finds '\0'. printf just writes all characters at the specified position in memory until it finds '\0'.
So far, so good. Now what happens if someone forgets about the null terminator?
char* test = new char[3]; // don't do this at home, please
test[0] = 'f';
test[1] = 'o';
test[2] = 'o';
f(test); // uh-oh, there is no null terminator...
The result will be undefined behaviour. strlen will keep looking for '\0'. So will printf. The functions will try to read memory they are not supposed to. The program is allowed to do anything, including crashing. The evil thing is that most likely, nothing will happen for a while because a '\0' just happens to be stored there in memory, until one day you are not so lucky anymore.
That's why C functions are sometimes made safer by requiring you to explicitly specify the number of characters. Your send is such a function. It works fine even without null-terminated strings.
So much for C strings. And now please don't use them in your C++ code. Use std::string. It is designed to be compatible with C functions by providing the c_str() member function, which returns a null-terminated char const * pointing to the contents of the string, and it of course has a size() member function to tell you the number of characters without the null-terminated character (e.g. for a std::string representing the word "foo", size() would be 3, not 4, and 3 is also what a C function like yours would probably expect, but you have to look at the documentation of the function to find out whether it needs the number of visible characters or number of elements in memory).
In fact, with std::string you can just forget about the whole null-termination business. Everything is nicely automated. std::string is exactly as easy and safe to use as java.lang.String.
Your sendLine should thus become:
void sendLine(SOCKET s, std::string const& line)
{
send(s, line.c_str(), line.size());
}
(Passing a std::string by const& is the normal way of passing big objects in C++. It's just for performance, but it's such a widely-used convention that your code would look strange if you just passed std::string.)
How can I do that, unless providing lenght as a template parameter?
This is a misunderstanding of how templates work. With a template, the length would have to be known at compile time. That's certainly not what you intended.
Now, for the second part of the answer, perhaps you aren't really dealing with text here. It's unlikely, as the name "sendLine" in your example sounds very much like text, but perhaps you are dealing with raw data, and a char in your output does not represent a text character but just a value to be interpreted as something completely different, such as the contents of an image file.
In that case, std::string is a poor choice. Your output could contain '\0' characters that do not have the meaning of "data ends here", but which are part of the normal contents. In other words, you don't really have strings anymore, you have a range of char elements in which '\0' has no special meaning.
For this situation, C++ offers the std::vector template, which you can use as std::vector<char>. It is also designed to be usable with C functions by providing a member function that returns a char pointer. Here's an example:
void sendLine(SOCKET s, std::vector<char> const& data)
{
send(s, &data[0], data.size());
}
(The unusual &data[0] syntax means "pointer to the first element of the encapsulated data. C++11 has nicer-to-read ways of doing this, but &data[0] also works in older versions of C++.)
Things to keep in mind:
std::string is like String in Java.
std::vector is like ArrayList in Java.
std::string is for a range of char with the meaning of text, std::vector<char> is for a range of char with the meaning of raw data.
std::string and std::vector are designed to work together with C APIs.
Do not use new[] in C++.
Understand the null termination of C strings.
I have got a const char which is made by concatenation like this:
const char *fileName = "background1";
std::stringstream sstm;
sstm << fileName << "-hd.png";
fileName = sstm.str().c_str();
My problem is that the following instruction:
printf("const char = %s size = %d", fileName, sizeof(fileName));
returns:
"const char = background1-hd.png size = 4"
whereas I would expect that it returns:
"const char = background1-hd.png size = 19"
For example, the following gives the convenient result (as there is no concatenation):
const char *fileName2 = "background1-hd";
printf("const char = %s size = %d", fileName2, sizeof(fileName2));
returns:
"const char = background1-hd.png size = 19"
How to avoid this issue and guarantee that the characters will be correctly counted in my concatenated char ?
Thanks !!
sizeof() returns the number of bytes the variable occupies in memory (in this case returns the size of the pointer fileName).
strlen() returns the length of the string (which is what you need).
You could as well try something like:
#include <iostream>
#include <cstdio>
int main()
{
std::string fileName("background1");
fileName.append("-hd.png");
printf("const char = %s size = %d", fileName.c_str(), fileName.length());
return 0;
}
sizeof returns the size of the variable you give to it; it's evaluated at compile time. The "4" is the size of a pointer on your system. You want to use strlen() to determine the length of a string.
The result of sizeof(fileName) is related to fileName being a pointer, not an array. It literally returns the size of a pointer to a constant character string, and on a 32-bit system, all pointers are 32 bits (so sizeof == 4).
What you should use instead is strlen or similar, which will count the characters in the string, up to the trailing null, and return that. The results with strlen in place of sizeof will be about what you expect.
Side-related, with const char strings there is only ever one character per "cell" (actually byte). There are character sets which make for multiple bytes per character, but packing multiple characters into a single byte is quite rare, at least in C-family languages.
sizeof calculates the size of the data type in bytes and not the size of its contents (what it points to). In your example you are calculating the sizeof char* which is 4 bytes on your system. To get the length of a C string use strlen.
There is a distinction in the language between arrays and pointers, even if this distinction seems diluted both by implicit conversions (arrays tend to decay into pointers quite easily), and common statements that they are the same.
How does this even relate to your code?
Well, a string literal is actually an array of constant characters, not a pointer to character(s). In the initialization const char *fileName = "background1"; you are creating a pointer variable that points to the first element of the array ("background1" is decaying into a pointer to the first element), and from there on the variable you are managing is pointer and not the literal.
If you mix this with the fact that sizeof will tell you the size of the variable, you get that in a platform with 32bit pointers and 8 bit chars, sizeof( const char* ) is always 4, regardless of the object that is pointed by that pointer (if there is even one).
Now, if you were treating the literal as what it actually is you would be having a bit more luck there:
const char filename[] = "background1";
assert( sizeof filename == 12 ); // note: NUL character is counted!
const char *fname = filename;
assert( sizeof filename == sizeof( void* ) );
In real code, you are not a so lucky and in many cases the literals have decayed into pointers well before you get a chance of getting the compile time size of the literal, so you cannot ask the compiler to tell you the size. In that case you need to calculate the length of the C style string, which can be done by calling strlen.
strlen has been suggested a number of times already, and for this case it's probably perfectly reasonable.
There is an alternative that will let you use sizeof though:
char fileName[] = "background1";
std::cout << sizeof(fileName) << "\n";
Since you're making fileName an array, it has all the characteristics of an array -- including the fact that your later attempt at assigning to it:
fileName = sstm.str().c_str();
...would fail (won't even compile when fileName is defined as an array). I should add, however, that it seems to me that you'd be better off just using std::string throughout:
std::string fileName("background1");
std::stringstream sstm;
sstm << fileName << "-hd.png";
fileName = sstm.str();
In this case, you can use string's size() or length() member.
std::strlen doesn't handle c strings that are not \0 terminated. Is there a safe version of it?
PS I know that in c++ std::string should be used instead of c strings, but in this case my string is stored in a shared memory.
EDIT
Ok, I need to add some explanation.
My application is getting a string from a shared memory (which is of some length), therefore it could be represented as an array of characters. If there is a bug in the library writing this string, then the string would not be zero terminated, and the strlen could fail.
You've added that the string is in shared memory. That's guaranteed readable, and of fixed size. You can therefore use size_t MaxPossibleSize = startOfSharedMemory + sizeOfSharedMemory - input; strnlen(input, MaxPossibleSize) (mind the extra n in strnlen).
This will return MaxPossibleSize if there's no \0 in the shared memory following input, or the string length if there is. (The maximal possible string length is of course MaxPossibleSize-1, in case the last byte of shared memory is the first \0)
C strings that are not null-terminated are not C strings, they are simply arrays of characters, and there is no way of finding their length.
If you define a c-string as
char* cowSays = "moo";
then you autmagically get the '\0' at the end and strlen would return 3. If you define it like:
char iDoThis[1024] = {0};
you get an empty buffer (and array of characters, all of which are null characters). You can then fill it with what you like as long as you don't over-run the buffer length. At the start strlen would return 0, and once you have written something you would also get the correct number from strlen.
You could also do this:
char uhoh[100];
int len = strlen(uhoh);
but that would be bad, because you have no idea what is in that array. It could hit a null character you might not. The point is that the null character is the defined standard manner to declare that the string is finished.
Not having a null character means by definition that the string is not finished. Changing that will break the paradigm of how the string works. What you want to do is make up your own rules. C++ will let you do that, but you will have to write a lot of code yourself.
EDIT
From your newly added info, what you want to do is loop over the array and check for the null character by hand. You should also do some validation if you are expecting ASCII characters only (especially if you are expecting alpha-numeric characters). This assumes that you know the maximum size.
If you do not need to validate the content of the string then you could use one of the strnlen family of functions:
http://msdn.microsoft.com/en-us/library/z50ty2zh%28v=vs.80%29.aspx
http://linux.about.com/library/cmd/blcmdl3_strnlen.htm
size_t safe_strlen(const char *str, size_t max_len)
{
const char * end = (const char *)memchr(str, '\0', max_len);
if (end == NULL)
return max_len;
else
return end - str;
}
Yes, since C11:
size_t strnlen_s( const char *str, size_t strsz );
Located in <string.h>
Get a better library, or verify the one you have - if you can't trust you library to do what it says it will, then how the h%^&l do you expect your program to?
Thats said, Assuming you know the length of the buiffer the string resides, what about
buffer[-1+sizeof(buffer)]=0 ;
x = strlen(buffer) ;
make buffer bigger than needed and you can then test the lib.
assert(x<-1+sizeof(buffer));
C11 includes "safe" functions such as strnlen_s. strnlen_s takes an extra maximum length argument (a size_t). This argument is returned if a null character isn't found after checking that many characters. It also returns the second argument if a null pointer is provided.
size_t strnlen_s(const char *, size_t);
While part of C11, it is recommended that you check that your compiler supports these bounds-checking "safe" functions via its definition of __STDC_LIB_EXT1__. Furthermore, a user must also set another macro, __STDC_WANT_LIB_EXT1__, to 1, before including string.h, if they intend to use such functions. See here for some Stack Overflow commentary on the origins of these functions, and here for C++ documentation.
GCC and Clang also support the POSIX function strnlen, and provide it within string.h. Microsoft too provide strnlen which can also be found within string.h.
You will need to encode your string. For example:
struct string
{
size_t len;
char *data;
} __attribute__(packed);
You can then accept any array of characters if you know the first sizeof(size_t) bytes of the shared memory location is the size of the char array. It gets tricky when you want to chain arrays this way.
It's better to trust your other end to terminate it's strings or roll your own strlen that does not go outside the bounderies of the shared memory segment (providing you know at least the size of that segment).
If you need to get the size of shared memory, try to use
// get memory size
struct shmid_ds shm_info;
size_t shm_size;
int shm_rc;
if((shm_rc = shmctl(shmid, IPC_STAT, &shm_info)) < 0)
exit(101);
shm_size = shm_info.shm_segsz;
Instead of using strlen you can use shm_size - 1 if you are sure that it is null terminated. Otherwise you can null terminate it by data[shm_size - 1] = '\0'; then use strlen(data);
a simple solution:
buff[BUFF_SIZE -1] = '\0'
ofc this will not tell you if the string originally was exactly BUFF_SIZE-1 long or it was just not terminated... so you need xtra logic for that.
How about this portable nugget:
int safeStrlen(char *buf, int max)
{
int i;
for(i=0;buf[i] && i<max; i++){};
return i;
}
As Neil Butterworth already said in his answer above: C-Strings which are not terminated by a \0 character, are no C-Strings!
The only chance you do have is to write an immutable Adaptor or something which creates a valid copy of the C-String with a \0 terminating character. Of course, if the input is wrong and there is an C-String defined like:
char cstring[3] = {'1','2','3'};
will indeed result in unexpected behavior, because there can be something like 123#4x\0 in the memory now. So the result of of strlen() for example is now 6 and not 3 as expected.
The following approach shows how to create a safe C-String in any case:
char *createSafeCString(char cStringToCheck[]) {
//Cast size_t to integer
int size = static_cast<int>(strlen(cStringToCheck)) ;
//Initialize new array out of the stack of the method
char *pszCString = new char[size + 1];
//Copy data from one char array to the new
strncpy(pszCString, cStringToCheck, size);
//set last character to the \0 termination character
pszCString[size] = '\0';
return pszCString;
}
This ensures that if you manipulate the C-String to not write on the memory of something else.
But this is not what you wanted. I know, but there is no other way to achieve the length of a char array without termination. This isn't even an approach. It just ensures that even if the User (or Dev) is inserting ***** to work fine.
I want to copy a string into a char array, and not overrun the buffer.
So if I have a char array of size 5, then I want to copy a maximum of 5 bytes from a string into it.
what's the code to do that?
This is exactly what std::string's copy function does.
#include <string>
#include <iostream>
int main()
{
char test[5];
std::string str( "Hello, world" );
str.copy(test, 5);
std::cout.write(test, 5);
std::cout.put('\n');
return 0;
}
If you need null termination you should do something like this:
str.copy(test, 4);
test[4] = '\0';
First of all, strncpy is almost certainly not what you want. strncpy was designed for a fairly specific purpose. It's in the standard library almost exclusively because it already exists, not because it's generally useful.
Probably the simplest way to do what you want is with something like:
sprintf(buffer, "%.4s", your_string.c_str());
Unlike strncpy, this guarantees that the result will be NUL terminated, but does not fill in extra data in the target if the source is shorter than specified (though the latter isn't a major issue when the target length is 5).
Use function strlcpybroken link, and material not found on destination site if your implementation provides one (the function is not in the standard C library), yet it is rather widely accepted as a de-facto standard name for a "safe" limited-length copying function for zero-terminated strings.
If your implementation does not provide strlcpy function, implement one yourself. For example, something like this might work for you
char *my_strlcpy(char *dst, const char *src, size_t n)
{
assert(dst != NULL && src != NULL);
if (n > 0)
{
char *pd;
const char *ps;
for (--n, pd = dst, ps = src; n > 0 && *ps != '\0'; --n, ++pd, ++ps)
*pd = *ps;
*pd = '\0';
}
return dst;
}
(Actually, the de-facto accepted strlcpy returns size_t, so you might prefer to implement the accepted specification instead of what I did above).
Beware of the answers that recommend using strncpy for that purpose. strncpy is not a safe limited-length string copying function and is not supposed to be used for that purpose. While you can force strncpy to "work" for that purpose, it is still akin to driving woodscrews with a hammer.
Update: Thought I would try to tie together some of the answers, answers which have convinced me that my own original knee-jerk strncpy response was poor.
First, as AndreyT noted in the comments to this question, truncation methods (snprintf, strlcpy, and strncpy) are often not a good solution. Its often better to check the size of the string string.size() against the buffer length and return/throw an error or resize the buffer.
If truncation is OK in your situation, IMHO, strlcpy is the best solution, being the fastest/least overhead method that ensures null termination. Unfortunately, its not in many/all standard distributions and so is not portable. If you are doing a lot of these, it maybe worth providing your own implementation, AndreyT gave an example. It runs in O(result length). Also the reference specification returns the number of bytes copied, which can assist in detecting if the source was truncated.
Other good solutions are sprintf and snprintf. They are standard, and so are portable and provide a safe null terminated result. They have more overhead than strlcpy (parsing the format string specifier and variable argument list), but unless you are doing a lot of these you probably won't notice the difference. It also runs in O(result length). snprintf is always safe and that sprintf may overflow if you get the format specifier wrong (as other have noted, format string should be "%.<N>s" not "%<N>s"). These methods also return the number of bytes copied.
A special case solution is strncpy. It runs in O(buffer length), because if it reaches the end of the src it zeros out the remainder of the buffer. Only useful if you need to zero the tail of the buffer or are confident that destination and source string lengths are the same. Also note that it is not safe in that it doesn't necessarily null terminate the string. If the source is truncated, then null will not be appended, so call in sequence with a null assignment to ensure null termination: strncpy(buffer, str.c_str(), BUFFER_LAST); buffer[BUFFER_LAST] = '\0';
Some nice libc versions provide non-standard but great replacement for strcpy(3)/strncpy(3) - strlcpy(3).
If yours doesn't, the source code is freely available here from the OpenBSD repository.
void stringChange(string var){
char strArray[100];
strcpy(strArray, var.c_str());
}
I guess this should work. it'll copy form string to an char array.
i think snprintf() is much safe and simlest
snprintf ( buffer, 100, "The half of %d is %d", 60, 60/2 );
null character is append it end automatically :)
The most popular answer is fine but the null-termination is not generic. The generic way to null-terminate the char-buffer is:
std::string aString = "foo";
const size_t BUF_LEN = 5;
char buf[BUF_LEN];
size_t len = aString.copy(buf, BUF_LEN-1); // leave one char for the null-termination
buf[len] = '\0';
len is the number of chars copied so it's between 0 and BUF_LEN-1.
std::string my_string("something");
char* my_char_array = new char[5];
strncpy(my_char_array, my_string.c_str(), 4);
my_char_array[4] = '\0'; // my_char_array contains "some"
With strncpy, you can copy at most n characters from the source to the destination. However, note that if the source string is at most n chars long, the destination will not be null terminated; you must put the terminating null character into it yourself.
A char array with a length of 5 can contain at most a string of 4 characters, since the 5th must be the terminating null character. Hence in the above code, n = 4.
std::string str = "Your string";
char buffer[5];
strncpy(buffer, str.c_str(), sizeof(buffer));
buffer[sizeof(buffer)-1] = '\0';
The last line is required because strncpy isn't guaranteed to NUL terminate the string (there has been a discussion about the motivation yesterday).
If you used wide strings, instead of sizeof(buffer) you'd use sizeof(buffer)/sizeof(*buffer), or, even better, a macro like
#define ARRSIZE(arr) (sizeof(arr)/sizeof(*(arr)))
/* ... */
buffer[ARRSIZE(buffer)-1]='\0';
char mystring[101]; // a 100 character string plus terminator
char *any_input;
any_input = "Example";
iterate = 0;
while ( any_input[iterate] != '\0' && iterate < 100) {
mystring[iterate] = any_input[iterate];
iterate++;
}
mystring[iterate] = '\0';
This is the basic efficient design.
If you always have a buffer of size 5, then you could do:
std::string s = "Your string";
char buffer[5]={s[0],s[1],s[2],s[3],'\0'};
Edit:
Of course, assuming that your std::string is large enough.