I got a string variable (contains passphrase) and would like to overwrite it's value with a sequence of '0' before the variable is released. I tought about doing something like:
void overwrite(std::string &toOverwrite){
if(toOverwrite.empty())
return;
else{
std::string removeString;
size_t length = toOverwrite.size();
for(int i = 0; i < length; i++){
removeString += "0";
}
toOverwrite = removeString;
}
}
But somehow this doesn't feel right.
First because it seems to produce much overhead in the for loop.
Moreover I'm not sure if the last line would really overwrite the string. I know that e.g. in Java strings are immutable and therefore can not be overwritten at all. They are not immutable in C++ (at least not std::string) but would toOverwrite = removeString really replace toOverwrite or just make that the "pointer" of toOverwrite will point to removeString?
Is it possible that my compiler will optimize the code and removes this overwriting?
Maybe I should use the std::string::replace method or change the datatype to char* / byte[]?
Chances are that will just swap and free pointers, leaving the passphrase somewhere in memory which is no longer pointed to. If you want to overwrite the string data, do:
std::fill(toOverwrite.begin(), toOverwrite.end(), '0');
And you don't need a test for an empty string either.
Related
I want to make a function that removes all the characters of ch in a c-string.
But I keep getting an access violation error.
Unhandled exception at 0x000f17ba in testassignments.exe: 0xC0000005: Access violation writing location 0x000f787e.
void removeAll(char* &s, const char ch)
{
int len=strlen(s);
int i,j;
for(i = 0; i < len; i++)
{
if(s[i] == ch)
{
for(j = i; j < len; j++)
{
s[j] = s[j + 1];
}
len--;
i--;
}
}
return;
}
I expected the c-string to not contain the character "ch", but instead, I get an access violation error.
In the debug I got the error on the line:
s[j] = s[j + 1];
I tried to modify the function but I keep getting this error.
Edit--
Sample inputs:
s="abmas$sachus#settes";
ch='e' Output->abmas$sachus#settes, becomes abmas$sachus#stts
ch='t' Output-> abmas$sachus#stts, becomes abmas$sachus#ss.
Instead of producing those outputs, I get the access violation error.
Edit 2:
If its any help, I am using Microsoft Visual C++ 2010 Express.
Apart from the inefficiency of your function shifting the entire remainder of the string whenever encountering a single character to remove, there's actually not much wrong with it.
In the comments, people have assumed that you are reading off the end of the string with s[j+1], but that is untrue. They are forgetting that s[len] is completely valid because that is the string's null-terminator character.
So I'm using my crystal ball now, and I believe that the error is because you're actually running this on a string literal.
// This is NOT okay!
char* str = "abmas$sachus#settes";
removeAll(str, 'e');
This code above is (sort of) not legal. The string literal "abmas$sachus#settes" should not be stored as a non-const char*. But for backward compatibility with C where this is allowed (provided you don't attempt to modify the string) this is generally issued as a compiler warning instead of an error.
However, you are really not allowed to modify the string. And your program is crashing the moment you try.
If you were to use the correct approach with a char array (which you can modify), then you have a different problem:
// This will result in a compiler error
char str[] = "abmas$sachus#settes";
removeAll(str, 'e');
Results in
error: invalid initialization of non-const reference of type ‘char*&’ from an rvalue of type ‘char*’
So why is that? Well, your function takes a char*& type that forces the caller to use pointers. It's making a contract that states "I can modify your pointer if I want to", even if it never does.
There are two ways you can fix that error:
The TERRIBLE PLEASE DON'T DO THIS way:
// This compiles and works but it's not cool!
char str[] = "abmas$sachus#settes";
char *pstr = str;
removeAll(pstr, 'e');
The reason I say this is bad is because it sets a dangerous precedent. If the function actually did modify the pointer in a future "optimization", then you might break some code without realizing it.
Imagine that you want to output the string with characters removed later, but the first character was removed and you function decided to modify the pointer to start at the second character instead. Now if you output str, you'll get a different result from using pstr.
And this example is only assuming that you're storing the string in an array. Imagine if you actually allocated a pointer like this:
char *str = new char[strlen("abmas$sachus#settes") + 1];
strcpy(str, "abmas$sachus#settes");
removeAll(str, 'e');
Then if removeAll changes the pointer, you're going to have a BAD time when you later clean up this memory with:
delete[] str; //<-- BOOM!!!
The I ACKNOWLEDGE MY FUNCTION DEFINITION IS BROKEN way:
Real simply, your function definition should take a pointer, not a pointer reference:
void removeAll(char* s, const char ch)
This means you can call it on any modifiable block of memory, including an array. And you can be comforted by the fact that the caller's pointer will never be modified.
Now, the following will work:
// This is now 100% legit!
char str[] = "abmas$sachus#settes";
removeAll(str, 'e');
Now that my free crystal-ball reading is complete, and your problem has gone away, let's address the elephant in the room:
Your code is needlessly inefficient!
You do not need to do the first pass over the string (with strlen) to calculate its length
The inner loop effectively gives your algorithm a worst-case time complexity of O(N^2).
The little tricks modifying len and, worse than that, the loop variable i make your code more complex to read.
What if you could avoid all of these undesirable things!? Well, you can!
Think about what you're doing when removing characters. Essentially, the moment you have removed one character, then you need to start shuffling future characters to the left. But you do not need to shuffle one at a time. If, after some more characters you encounter a second character to remove, then you simply shunt future characters further to the left.
What I'm trying to say is that each character only needs to move once at most.
There is already an answer demonstrating this using pointers, but it comes with no explanation and you are also a beginner, so let's use indices because you understand those.
The first thing to do is get rid of strlen. Remember, your string is null-terminated. All strlen does is search through characters until it finds the null byte (otherwise known as 0 or '\0')...
[Note that real implementations of strlen are super smart (i.e. much more efficient than searching single characters at a time)... but of course, no call to strlen is faster]
All you need is your loop to look for the NULL terminator, like this:
for(i = 0; s[i] != '\0'; i++)
Okay, and now to ditch the inner loop, you just need to know where to stick each new character. How about just keeping a variable new_size in which you are going to count up how long the final string is.
void removeAll(char* s, char ch)
{
int new_size = 0;
for(int i = 0; s[i] != '\0'; i++)
{
if(s[i] != ch)
{
s[new_size] = s[i];
new_size++;
}
}
// You must also null-terminate the string
s[new_size] = '\0';
}
If you look at this for a while, you may notice that it might do pointless "copies". That is, if i == new_size there is no point in copying characters. So, you can add that test if you want. I will say that it's likely to make little performance difference, and potentially reduce performance because of additional branching.
But I'll leave that as an exercise. And if you want to dream about really fast code and just how crazy it gets, then go and look at the source code for strlen in glibc. Prepare to have your mind blown.
You can make the logic simpler and more efficient by writing the function like this:
void removeAll(char * s, const char charToRemove)
{
const char * readPtr = s;
char * writePtr = s;
while (*readPtr) {
if (*readPtr != charToRemove) {
*writePtr++ = *readPtr;
}
readPtr++;
}
*writePtr = '\0';
}
Edit: Solutions must compile against Microsoft Visual Studio 2012.
I want to use a known string length to declare another string of the same length.
The reasoning is the second string will act as a container for operation done to the first string which must be non volatile with regards to it.
e.g.
const string messy "a bunch of letters";
string dostuff(string sentence) {
string organised NNN????? // Idk, just needs the same size.
for ( x = 0; x < NNN?; x++) {
organised[x] = sentence[x]++; // Doesn't matter what this does.
}
}
In both cases above, the declaration and the exit condition, the NNN? stands for the length of 'messy'.
How do I discover the length at compile time?
std::string has two constructors which could fit your purposes.
The first, a copy constructor:
string organised(sentence);
The second, a constructor which takes a character and a count. You could initialize a string with a temporary character.
string organised(sentence.length(), '_');
Alternatively, you can:
Use an empty string and append (+=) text to it as you go along, or
Use a std::stringstream for the same purpose.
the stringstream will likely be more efficient.
Overall, I would prefer the copy constructor if the length is known.
std::string isn't a compile time type (it can't be a constexpr), so you can't use it directly to determine the length at compile time.
You could initialize a constexpr char[] and then use sizeof on that:
constexpr char messychar[] = "a bunch of letters";
// - 1 to avoid including NUL terminator which std::string doesn't care about
constexpr size_t messylen = sizeof(messychar) / sizeof(messychar[0]) - 1;
const string messy(messychar);
and use that, but frankly, that's pretty ugly; the length would be compile time, but organized would need to use the count and char constructor that would still be performed on each call, allocating and initializing only to have the contents replaced in the loop.
While it's not compile time, you'd avoid that initialization cost by just using reserve and += to build the new string, which with the #define could be done in an ugly but likely efficient way as:
constexpr char messychar[] = "a bunch of letters";
constexpr size_t messylen = sizeof(messychar) / sizeof(messychar[0]) - 1;
// messy itself may not be needed, but if it is, it's initialized optimally
// by using the compile time calculated length, so there is no need to scan for
// NUL terminators, and it can reserve the necessary space in the initial alloc
const string messy(messychar, messylen);
string dostuff(string sentence) {
string organised;
organized.reserve(messylen);
for (size_t x = 0; x < messylen; x++) {
organised += sentence[x]++; // Doesn't matter what this does.
}
}
This avoids setting organised's values more than once, allocating more than once (well, possibly twice if initial construction performs it) per call, and only performs a single read/write pass of sentence, no full read followed by read/write or the like. It also makes the loop constraint a compile time value, so the compiler has the opportunity to unroll the loop (though there is no guarantee of this, and even if it happens, it may not be helpful).
Also note: In your example, you mutate sentence, but it's accepted by value, so you're mutating the local copy, not the caller copy. If mutation of the caller value is required, accept it by reference, and if mutation is not required, accept by const reference to avoid a copy on every call (I understand the example code was filler, just mentioning this).
I'm finding standard string addition to be very slow so I'm looking for some tips/hacks that can speed up some code I have.
My code is basically structured as follows:
inline void add_to_string(string data, string &added_data) {
if(added_data.length()<1) added_data = added_data + "{";
added_data = added_data+data;
}
int main()
{
int some_int = 100;
float some_float = 100.0;
string some_string = "test";
string added_data;
added_data.reserve(1000*64);
for(int ii=0;ii<1000;ii++)
{
//variables manipulated here
some_int = ii;
some_float += ii;
some_string.assign(ii%20,'A');
//then we concatenate the strings!
stringstream fragment;
fragment<<some_int <<","<<some_float<<","<<some_string;
add_to_string(fragment.str(),added_data);
}
return;
}
Doing some basic profiling, I'm finding that a ton of time is being used in the for loop. Are there some things I can do that will significantly speed this up? Will it help to use c strings instead of c++ strings?
String addition is not the problem you are facing. std::stringstream is known to be slow due to it's design. On every iteration of your for-loop the stringstream is responsible for at least 2 allocations and 2 deletions. The cost of each of these 4 operations is likely more than that of the string addition.
Profile the following and measure the difference:
std::string stringBuffer;
for(int ii=0;ii<1000;ii++)
{
//variables manipulated here
some_int = ii;
some_float += ii;
some_string.assign(ii%20,'A');
//then we concatenate the strings!
char buffer[128];
sprintf(buffer, "%i,%f,%s",some_int,some_float,some_string.c_str());
stringBuffer = buffer;
add_to_string(stringBuffer ,added_data);
}
Ideally, replace sprintf with _snprintf or the equivalent supported by your compiler.
As a rule of thumb, use stringstream for formatting by default and switch to the faster and less safe functions like sprintf, itoa, etc. whenever performance matters.
Edit: that, and what didierc said: added_data += data;
You can save lots of string operations if you do not call add_to_string in your loop.
I believe this does the same (although I am not a C++ expert and do not know exactly what stringstream does):
stringstream fragment;
for(int ii=0;ii<1000;ii++)
{
//variables manipulated here
some_int = ii;
some_float += ii;
some_string.assign(ii%20,'A');
//then we concatenate the strings!
fragment<<some_int<<","<<some_float<<","<<some_string;
}
// inlined add_to_string call without the if-statement ;)
added_data = "{" + fragment.str();
I see you used the reserve method on added_data, which should help by avoiding multiple reallocations of the string as it grows.
You should also use the += string operator where possible:
added_data += data;
I think that the above should save up some significant time by avoiding unecessary copies back and forth of added_data in a temporary string when doing the catenation.
This += operator is a simpler version of the string::append method, it just copies data directly at the end of added_data. Since you made the reserve, that operation alone should be very fast (almost equivalent to a strcpy).
But why going through all this, when you are already using a stringstream to handle input? Keep it all in there to begin with!
The stringstream class is indeed not very efficient.
You may have a look at the stringstream class for more information on how to use it, if necessary, but your solution of using a string as a buffer seems to avoid that class speed issue.
At any rate, stay away from any attempt at reimplementing the speed critical code in pure C unless you really know what you are doing. Some other SO posts support the idea of doing it,, but I think it's best (read safer) to rely as much as possible on the standard library, which will be enhanced over time, and take care of many corner cases you (or I) wouldn't think of. If your input data format is set in stone, then you might start thinking about taking that road, but otherwise it's premature optimization.
If you start added_data with a "{", you would be able to remove the if from your add_to_string method: the if gets executed exactly once, when the string is empty, so you might as well make it non-empty right away.
In addition, your add_to_string makes a copy of the data; this is not necessary, because it does not get modified. Accepting the data by const reference should speed things up for you.
Finally, changing your added_data from string to sstream should let you append to it in a loop, without the sstream intermediary that gets created, copied, and thrown away on each iteration of the loop.
Please have a look at Twine used in LLVM.
A Twine is a kind of rope, it represents a concatenated string using a
binary-tree, where the string is the preorder of the nodes. Since the
Twine can be efficiently rendered into a buffer when its result is used,
it avoids the cost of generating temporary values for intermediate string
results -- particularly in cases when the Twine result is never
required. By explicitly tracking the type of leaf nodes, we can also avoid
the creation of temporary strings for conversions operations (such as
appending an integer to a string).
It may helpful in solving your problem.
How about this approach?
This is a DevPartner for MSVC 2010 report.
string newstring = stringA & stringB;
i dont think strings are slow, its the conversions that can make it slow
and maybe your compiler that might check variable types for mismatches.
This is a fairly basic question and I am pretty sure I know the answer, but seeing as the consequence for being wrong is a segfault I figure I should ask. I have been using strlen() and the new char[] operator in the following way for quite some time now and just noticed something that threw up a red flag:
void genericCopy(char *somestring, char *someOtherString) {
someOtherString = new char[strlen(somestring)];
strcpy(someOtherString,somestring);
}
My question is, seeing as a string should be null terminated, should I be doing this as such:
void genericCopy(char *somestring, char *someOtherString) {
someOtherString = new char[strlen(somestring)+1];
strcpy(someOtherString,somestring);
someOtherString[strlen(someOtherString)] = '\0';
}
So far I have never had a problem with the first method, but that doesn't mean I'm doing it right. Since the length being return by strlen()is the number of characters in the string without the null terminator so new isn't reserving space for '/0'... At least I don't think it is.
First of all, you should know that this function of yours is pointless to write, just use strdup (if available on your system).
But yes, you need an additional byte to store the \0, so always do something like new char[strlen(somestring)+1];. However, there is no need to manually add the \0; strcpy already does this.
You should use something like Valgrind to discover this and similar bugs in your code.
There is however an additional problem in your code; your code will always leak someOtherString; it will not be returned to where you called it from. You either need to change your method to something like:
char *genericCopy(char *something) {
char *copy = new char[strlen(somestring)+1];
strcpy(copy,somestring);
return copy;
}
and then get the copy as follows:
copy = genericCopy(something);
Or you need to change your method to something like:
void genericCopy(char *something, char **copy) {
*copy = new char[strlen(somestring)+1];
strcpy(*copy,somestring);
}
and call it as:
genericCopy(something, ©);
If you'll be using C++ you could also just change the method prototype to:
void genericCopy(char* somestring, char*& someOtherString)
and call it as:
genericCopy(something, copy);
Then someOtherString will be passed as a reference, and the new value you allocate to it will propagate outside of your method.
Yes, your suspicion is correct. You should be allocating an additional character, and making sure the copied string is null-terminated. (strcpy() itself will do this, but when someone advises to you that you switch to strncpy(), as they no doubt will (it's safer!) you'll need to be extra careful, because it is NOT guaranteed to copy the '/0'.)
If you're already using C++, though, you may be well-advised to switch to using std::string. It's often an easier, less error-prone method of manipulating character arrays.
However, here's the further problem that you need to address. You are assigning your new character array to a COPY of someOtherString. You need to make some changes:
void genericCopy(char *somestring, char **someOtherString) {
*someOtherString = new char[strlen(somestring)+1];
strcpy(*someOtherString,somestring);
(*someOtherString)[strlen(somestring)] = '\0';
}
This way you will get back the new character buffer outside your function call.
My problem goes like this: I have a class called 'Register'. It has a string attribute called 'trainName' and its setter:
class Register {
private:
string trainName;
public:
string getTrainName();
};
As a matter of fact, it is longer but I want to make this simpler.
In other class, I copy several Register objects into a binary file, previously setting trainName.
Register auxRegister = Register();
auxRegister.setName("name");
for(int i = 0; i < 10; i++) {
file.write(reinterpret_cast<char*>(&auxRegister),sizeof(Register));
}
Later on, I try to retrieve the register from the binary file:
Register auxRegister = Register();
while(!file.eof()) { //I kwnow this is not right. Which is the right way?
file.read(reinterpret_cast<char*>(&auxRegister), sizeof(Register));
}
It occurs it does not work. Register does, in fact, have more attributes (they are int) and I retrieve them OK, but it's not the case with the string.
Am I doing something wrong? Should I take something into consideration when working with binary files and strings?
Thank you very much.
The std::string class contains a pointer to a buffer where the string is stored (along with other member variables). The string buffer itself is not a part of the class. So writing out the contents of an instance of the class is not going to work, since the string will never be part of what you dump into the file, if you do it that way. You need to get a pointer to the string and write that.
Register auxRegister = Register();
auxRegister.setName("name");
auto length = auxRegister.size();
for(int i = 0; i < 10; i++) {
file.write( auxRegister.c_str(), length );
// You'll need to multiply length by sizeof(CharType) if you
// use a wstring instead of string
}
Later on, to read the string, you'll have to keep track of the number of bytes that were written to the file; or maybe fetch that information from the file itself, depending on the file format.
std::unique_ptr<char[]> buffer( new char[length + 1] );
file.read( buffer, length );
buffer[length] = '\0'; // NULL terminate the string
Register auxRegister = Register();
auxRegister.setName( buffer );
You cannot write string this way, as it almost certainly contains pointers to some structs and other binary stuff that cannot be serialized at all.
You need to write your own serializing function, and write the string length + bytes (for example) or use complete library, for example, protobuf, which can solve serializing problem for you.
edit: see praetorian's answer. much better than mine (even with lower score at time of this edit).