I have implemented an algorithm 1) using ordinary string copy and 2) with boost::string_view to lookup a suffix part of a string after a delimiter. I have profiled the code and counter-intuitively, I have ended up seeing the boost::string_view performed poorly.
std::string algorithm1(const std::string &mine, const std::string& path) {
size_t startPos = mine.find(path);
std::string temp;
if (startPos != mine.npos) {
startPos += path.size();
temp = mine.substr(startPos);
} else {
std::cout << "not found" << std::endl;
return "";
}
return temp;
}
boost::string_ref algorithm2(boost::string_ref mine, boost::string_ref path) {
size_t startPos = mine.find(path);
boost::string_ref temp;
if (startPos != mine.npos) {
startPos += path.size();
temp = mine.substr(startPos);
} else {
std::cout << "not found" << std::endl;
}
return temp;
}
int main(int argc, char* argv[])
{
std::cout << "entered" << std::endl;
const std::string mine = "sth/aStr/theSuffixWeDesire";
const std::string path = "/aStr/";
for (size_t i = 0; i < 10; i++)
{
assert (algorithm1(mine, path) == "theSuffixWeDesire");
assert (algorithm2(mine, path) == "theSuffixWeDesire");
}
return 0;
}
When I have profiled the code with uftrace, I ended up with, for each iteration:
algorithm1 taking 2,083 ns, internally calls std::find
algorithm2 taking 11,835 ns with maximum time spent in calling std::search
Possibly, there could be three reasons I could think of:
CPU cache hit in algorithm1, whereas algorithm2 calling boost
library and therefore CPU cache miss and resulted lower speed
Std operations are optimized by the compiler whereas boost operations
are not.
std::search that boost uses is not a good substitute for
std::find.
Which of these explanations are more plausible? This have been an unexpected behavior, made me doubt using boost::string_ref in my codebase.
My system has gcc 5.4.0 (C++14), boost 1.66, no compiler optimization flags (beginning with -O).
Thank you,
Related
I have a function ls() which parses a vector of string and puts it into a comma-separated list, wrapped within parentheses ():
std::string ls(std::vector<std::string> vec, std::string wrap="()", std::string sep=", ") {
std::string wrap_open, wrap_close;
wrap_open = std::to_string(wrap[0]);
wrap_close = std::to_string(wrap[1]);
std::string result = wrap_open;
size_t length = vec.size();
if (length > 0) {
if (length == 1) {
result += vec[0];
result += wrap_close;
}
else {
for (int i = 0; i < vec.size(); i++) {
if (i == vec.size() - 1) {
result += sep;
result += vec[i];
result += wrap_close;
}
else if (i == 0) {
result += vec[i];
}
else {
result += sep;
result += vec[i];
}
}
}
}
else {
result += wrap_close;
}
return result;
}
If I pass this vector
std::vector<std::string> vec = {"hello", "world", "three"};
to the ls() function, I should get this string:
std::string parsed_vector = ls(vec);
// AKA
std::string result = "(hello, world, three)"
The parsing works fine, however the characters in the wrap string turn into numbers when printed.
std::cout << result << std::endl;
Will result in the following:
40hello, world, three41
When it should instead result in this:
(hello, world, three)
The ( is turned into 40, and the ) is turned into 41.
My guess is that the characters are being turned into the Unicode/ASCII number values or something like that, I do not know how this happened or what to do.
The problem here is std::to_string converts a number to a string. There is no specialization for char values. So here, you're converting the ASCII value to a string:
wrap_open = std::to_string(wrap[0]);
wrap_close = std::to_string(wrap[1]);
Instead, you could simply do:
std::string wrap_open(1, wrap[0]);
std::string wrap_close(1, wrap[1]);
Note that you can greatly simplify your function by using std::ostringstream:
std::ostringstream oss;
oss << wrap[0];
for (size_t i = 0; i < vec.size(); i++)
{
if (i != 0) oss << sep;
oss << vec[i];
}
oss << wrap[1];
return oss.str();
I won't be commenting on how you could improve the function and that passing a vector by value (as an argument in the function) is never a good idea, however I will tell you how to fix your current issue:
std::string ls(std::vector<std::string> vec, std::string wrap = "()", std::string sep = ", ") {
std::string wrap_open, wrap_close;
wrap_open = wrap.at(0); //<----
wrap_close = wrap.at(1); //<----
std::string result = wrap_open;
size_t length = vec.size();
if (length > 0) {
... // Rest of the code
You don't need to use std::to_string, just use one of std::string's constructors to create a string with one character from the wrap string. This constructor is invoked via the = operator.
I recommend reading about std::string, it is apparent that you aren't using the full potential of the STL library : std::string
EDIT: After discussing the usage of .at() vs [] operator in the comments. I've decided to add the bit into this answer:
The main difference between .at() and [] is the bounds checking feature. .at will throw an std::out_of_range exception because it is performing a bounds check. The [] operator (IMHO) is present in STL containers due to backwards compatibility (imagine refactoring old C code into a C++ project). Point being it behaves like you would expect [] to behave and doesn't do any bounds checking.
In general I recommend the usage of .at() especially to beginners and especially if you are relying on human input. The uncaught exception will produce an easy to understand error, while untested [] will either produce weird values or RAV (read access violation) depending on the type stored in the container and from experience beginners usually have a harder time debugging this.
Bare in mind that this is just an opinion of one programmer and opinions may vary (as is visible in the discussion).
Hope it helps!
I want to generate clean assembly like Compiler Explorer locally. Note that, I read How to remove “noise” from GCC/clang assembly output? before attempting this. The output using that method isn't as clean or dense compared to godbolt and still has a lot of asm directives and unused labels in it.
How can I get clean assembly output without any unused labels or directives?
For the record, it is possible (and apparently not too hard) to set up a local install of Matt Godbolt's Compiler Explorer stuff, so you can use that to explore asm output for files that are part of existing large projects with their #include dependencies and everything.
If you already have some asm output, #Waqar's answer looks useful. Or maybe that functionality can be used on its own from the Compiler Explorer repo via node.js, IDK.
According to the install info in the readme in https://github.com/compiler-explorer/compiler-explorer (Matt's repo), you can simply run make after cloning it on a machine that has node.js installed.
I also found https://isocpp.org/blog/2017/10/cpp-weekly-episode-83-installing-compiler-explorerjason-turner which might have more details (or be obsolete at this point, IDK).
I think Matt also mentions using a local clone of Compiler Explorer in his CppCon 2017 talk about Compiler Explorer (maybe replying to a question at the end), “What Has My Compiler Done for Me Lately? Unbolting the Compiler's Lid”, and recommends it for playing with code that uses lots of #include that would be hard to get onto https://godbolt.org/. (Or for closed-source code).
A while ago, I needed something like this locally so I wrote a small tool to make the asm readable.
It attempts to 'clean' and make the 'asm' output from 'gcc' readable using C++ itself. It does something similar to Compiler Explorer and tries to remove all the directives and unused labels, making the asm clean. Only standard library is used for this.
Some things I should mention:
Will only with gcc and clang
Only tested with C++ code
compile with -S -fno-asynchronous-unwind-tables -fno-dwarf2-cfi-asm -masm=intel, (remove -masm= if you want AT&T asm)
AT&T syntax will probably work but I didn't test it much. The other two options are to remove the .cfi directives. It can be handled using the code below but the compiler itself does a much better job of this. See the answer by Peter Cordes above.
This program can work as standalone, but I would highly recommend reading this SO answer to tune your asm output and then process it using this program to remove unused labels / directives etc.
abi::__cxa_demangle() is used for demangling
Disclaimer: This isn't a perfect solution, and hasn't been tested extensively.
The strategy used for cleaning the asm(There are probably better, faster more efficient ways to do this):
Collect all the labels
Go through the asm line by line and check if the labels are used/unused
If the labels are unused, they get deleted
Every line beginning with '.' gets deleted, unless it is a used somewhere
Update 1: Not all static data gets removed now.
#include <algorithm>
#include <cxxabi.h>
#include <fstream>
#include <iostream>
#include <regex>
#include <string>
#include <sstream>
#include <unordered_map>
// trim from both ends (in place)
std::string_view trim(std::string_view s)
{
s.remove_prefix(std::min(s.find_first_not_of(" \t\r\v\n"), s.size()));
s.remove_suffix(std::min(s.size() - s.find_last_not_of(" \t\r\v\n") - 1, s.size()));
return s;
}
static inline bool startsWith(const std::string_view s, const std::string_view searchString)
{
return (s.rfind(searchString, 0) == 0);
}
std::string demangle(std::string &&asmText)
{
int next = 0;
int last = 0;
while (next != -1) {
next = asmText.find("_Z", last);
//get token
if (next != -1) {
int tokenEnd = asmText.find_first_of(":,.#[]() \n", next + 1);
int len = tokenEnd - next;
std::string tok = asmText.substr(next, len);
int status = 0;
char* name = abi::__cxa_demangle(tok.c_str(), 0, 0, &status);
if (status != 0) {
std::cout << "Demangling of: " << tok << " failed, status: " << status << '\n';
continue;
}
std::string demangledName{name};
demangledName.insert(demangledName.begin(), ' ');
asmText.replace(next, len, demangledName);
free((void*)name);
}
}
return std::move(asmText);
}
std::string clean_asm(const std::string& asmText)
{
std::string output;
output.reserve(asmText.length());
std::stringstream s{asmText};
//1. collect all the labels
//2. go through the asm line by line and check if the labels are used/unused
//3. if the labels are unused, they get deleted
//4. every line beginning with '.' gets deleted, unless it is a used label
std::regex exp {"^\\s*[_|a-zA-Z]"};
std::regex directiveRe { "^\\s*\\..*$" };
std::regex labelRe { "^\\.*[a-zA-Z]+[0-9]+:$" };
std::regex hasOpcodeRe { "^\\s*[a-zA-Z]" };
std::regex numericLabelsRe { "\\s*[0-9]:" };
const std::vector<std::string> allowedDirectives =
{
".string", ".zero", ".byte", ".value", ".long", ".quad", ".ascii"
};
//<label, used>
std::unordered_map<std::string, bool> labels;
//1
std::string line;
while (std::getline(s, line)) {
if (std::regex_match(line, labelRe)) {
trim(line);
// remove ':'
line = line.substr(0, line.size() - 1);
labels[line] = false;
}
}
s.clear();
s.str(asmText);
line = "";
//2
while (std::getline(s, line)) {
if (std::regex_match(line, hasOpcodeRe)) {
auto it = labels.begin();
for (; it != labels.end(); ++it) {
if (line.find(it->first)) {
labels[it->first] = true;
}
}
}
}
//remove false labels from labels hash-map
for (auto it = labels.begin(); it != labels.end();) {
if (it->second == false)
it = labels.erase(it);
else
++it;
}
s.clear();
s.str(asmText);
line = "";
std::string currentLabel;
//3
while (std::getline(s, line)) {
trim(line);
if (std::regex_match(line, labelRe)) {
auto l = line;
l = l.substr(0, l.size() - 1);
currentLabel = "";
if (labels.find(l) != labels.end()) {
currentLabel = line;
output += line + "\n";
}
continue;
}
if (std::regex_match(line, directiveRe)) {
//if we are in a label
if (!currentLabel.empty()) {
auto trimmedLine = trim(line);
for (const auto& allowedDir : allowedDirectives) {
if (startsWith(trimmedLine, allowedDir)) {
output += line;
output += '\n';
}
}
}
continue;
}
if (std::regex_match(line, numericLabelsRe)) {
continue;
}
if (line == "endbr64") {
continue;
}
if (line[line.size() - 1] == ':' || line.find(':') != std::string::npos) {
currentLabel = line;
output += line + '\n';
continue;
}
line.insert(line.begin(), '\t');
output += line + '\n';
}
return output;
}
int main(int argc, char* argv[])
{
if (argc < 2) {
std::cout << "Please provide more than asm filename you want to process.\n";
}
std::ifstream file(argv[1]);
std::string output;
if (file.is_open()) {
std::cout << "File '" << argv[1] << "' is opened\n";
std::string line;
while (std::getline(file, line)) {
output += line + '\n';
}
}
output = demangle(std::move(output));
output = clean_asm(output);
std::string fileName = argv[1];
auto dotPos = fileName.rfind('.');
if (dotPos != std::string::npos)
fileName.erase(fileName.begin() + dotPos, fileName.end());
std::cout << "Asm processed. Saving as '"<< fileName <<".asm'";
std::ofstream out;
out.open(fileName + ".asm");
out << output;
return 0;
}
I checked Compiler Explorer to see if they had a specific set of compiler options to get their output. But they don't. Instead they filter the assembly listing with this function. There's also an additional processing step that merges the debug info into source and assembly highlighting.
To answer your question, I don't think it's possible with GCC itself right now (Sep 2021).
so I'm currently working on a brute force attacker project in C++. I've managed to get it working, but one problem that I'm facing is that if the program actually managed to get a correct guess, the function still goes on. I think the problem is that the program fails to return a guess. Take a look at my code:
(Sorry for the mess, by the way, I'm not that experienced in C++ - I used to code in Python/JS.)
#include <iostream>
#include <cstdlib>
#include <string>
std::string chars = "abcdefghijklmnopqrstuvwxyz";
std::string iterateStr(std::string s, std::string guess, int pos);
std::string crack(std::string s);
std::string iterateChar(std::string s, std::string guess, int pos);
int main() {
crack("bb");
return EXIT_SUCCESS;
}
// this function iterates through the letters of the alphabet
std::string iterateChar(std::string s, std::string guess, int pos) {
for(int i = 0; i < chars.length(); i++) {
// sets the char to a certain letter from the chars variable
guess[pos] = chars[i];
// if the position reaches the end of the string
if(pos == s.length()) {
if(guess.compare(s) == 0) {
break;
}
} else {
// else, recursively call the function
std::cout << guess << " : " << s << std::endl;
iterateChar(s, guess, pos+1);
}
}
return guess;
}
// this function iterates through the characters in the string
std::string iterateStr(std::string s, std::string guess, int pos) {
for(int i = 0; i < s.length(); i++) {
guess = iterateChar(s, guess, i);
if(s.compare(guess) == 0) {
return guess;
}
}
return guess;
}
std::string crack(std::string s) {
int len = s.length();
std::string newS(len, 'a');
std::string newGuess;
newGuess = iterateStr(s, newS, 0);
return newGuess;
}
Edit : Updated code.
The main flaw in the posted code is that the recursive function returns a string (the guessed password) without a clear indication for the caller that the password was found.
Passing around all the strings by value, is also a potential efficiency problem, but the OP should be worried by snippets like this:
guess[pos] = chars[i]; // 'chars' contains the alphabet
if(pos == s.length()) {
if(guess.compare(s) == 0) {
break;
}
}
Where guess and s are strings of the same length. If that length is 2 (OP's last example), guess[2] is outside the bounds, but the successive call to guess.compare(s) will compare only the two chars "inside".
The loop inside iterateStr does nothing useful too, and the pos parameter is unused.
Rather than fixing this attempt, it may be better to rewrite it from scratch
#include <iostream>
#include <string>
#include <utility>
// Sets up the variable and start the brute force search
template <class Predicate>
auto crack(std::string const &src, size_t length, Predicate is_correct)
-> std::pair<bool, std::string>;
// Implements the brute force search in a single recursive function. It uses a
// lambda to check the password, instead of passing it directly
template <class Predicate>
bool recursive_search(std::string const &src, std::string &guess, size_t pos,
Predicate is_correct);
// Helper function, for testing purpouse
void test_cracker(std::string const &alphabet, std::string const &password);
int main()
{
test_cracker("abcdefghijklmnopqrstuvwxyz", "dance");
test_cracker("abcdefghijklmnopqrstuvwxyz ", "go on");
test_cracker("0123456789", "42");
test_cracker("0123456789", "one"); // <- 'Password not found.'
}
void test_cracker(std::string const &alphabet, std::string const &password)
{
auto [found, pwd] = crack(alphabet, password.length(),
[&password] (std::string const &guess) { return guess == password; });
std::cout << (found ? pwd : "Password not found.") << '\n';
}
// Brute force recursive search
template <class Predicate>
bool recursive_search(std::string const &src, std::string &guess, size_t pos,
Predicate is_correct)
{
if ( pos + 1 == guess.size() )
{
for (auto const ch : src)
{
guess[pos] = ch;
if ( is_correct(guess) )
return true;
}
}
else
{
for (auto const ch : src)
{
guess[pos] = ch;
if ( recursive_search(src, guess, pos + 1, is_correct) )
return true;
}
}
return false;
}
template <class Predicate>
auto crack(std::string const &src, size_t length, Predicate is_correct)
-> std::pair<bool, std::string>
{
if ( src.empty() )
return { length == 0 && is_correct(src), src };
std::string guess(length, src[0]);
return { recursive_search(src, guess, 0, is_correct), guess };
}
I've tried your code even with the modified version of your iterateStr() function. I used the word abduct as it is quicker to search for. When stepping through the debugger I noticed that your iterateChar() function was not returning when a match was found. Also I noticed that the length of string s being passed in was 6 however the guess string that is being updated on each iteration had a length of 7. You might want to step through your code and check this out.
For example at on specific iteration the s string contains: abduct but the guess string contains aaaabjz then on the next iteration the guess string contains aaaabkz. This might be your concerning issue of why the loop or function continues even when you think a match is found.
The difference in lengths here could be your culprit.
Also when stepping through your modified code:
for ( size_t i = 0; i < s.length(); i++ ) {
guess = iterCh( s, guess, i );
std::cout << "in the iterStr loop\n";
if ( guess.compare( s ) == 0 ) {
return guess;
}
}
return guess;
in your iterateStr() function the recursion always calls guess = iterCh( s, guess, i ); and the code never prints in the iterStr loop\n";. Your iterateChar function is completing through the entire string or sequence of characters never finding and return a match. I even tried the word abs as it is easier and quicker to step through the debugger and I'm getting the same kind of results.
When I did some performance test in my app I noticed a difference in the following code (Visual Studio 2010).
Slower version
while(heavyloop)
{
if(path+node+"/" == curNode)
{
do something
}
}
This will cause some extra mallocs for the resulting string to be generated.
In order to avoid these mallocs, I changed it in the following way:
std::string buffer;
buffer.reserve(500); // Big enough to hold all combinations without the need of malloc
while(heavyloop)
{
buffer = path;
buffer += node;
buffer += "/";
if(buffer == curNode)
{
do something
}
}
While the second version looks a bit more awkward compared to the first version it's still readable enough. What I was wondering though is, wether this kind of optimization is an oversight on part of the compiler, or if this always has to be done manually. Since it only changes the order of allocations I would expect that the compiler could also figure it out on it's own. On the other hand, certain conditions have to be met, to really make it an optimization, which may not neccessarily be fullfilled, but if the conditions are not, the code would at least perform as good as the first version. Are newer versions of Visual Studio better in this regard?
A more complete version which shows the difference (SSCE):
std::string gen_random(std::string &oString, const int len)
{
static const char alphanum[] =
"0123456789"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz";
oString = "";
for (int i = 0; i < len; ++i)
{
oString += alphanum[rand() % (sizeof(alphanum) - 1)];
}
return oString;
}
int main(int argc, char *argv[])
{
clock_t start = clock();
std::string s = "/";
size_t adds = 0;
size_t subs = 0;
size_t max_len = 0;
s.reserve(100000);
for(size_t i = 0; i < 1000000; i++)
{
std::string t1;
std::string t2;
if(rand() % 2)
{
// Slow version
//s += gen_random(t1, (rand() % 15)+3) + "/" + gen_random(t2, (rand() % 15)+3);
// Fast version
s += gen_random(t1, (rand() % 15)+3);
s += "/";
s += gen_random(t2, (rand() % 15)+3);
adds++;
}
else
{
subs++;
size_t pos = s.find_last_of("/", s.length()-1);
if(pos != std::string::npos)
s.resize(pos);
if(s.length() == 0)
s = "/";
}
if(max_len < s.length())
max_len = s.length();
}
std::cout << "Elapsed: " << clock() - start << std::endl;
std::cout << "Added: " << adds << std::endl;
std::cout << "Subtracted: " << subs << std::endl;
std::cout << "Max: " << max_len << std::endl;
return 0;
}
On my system I get about 1 second difference between the two (tested with gcc this time but there doesn't seem to be any notable difference to Visual Studio there):
Elapsed: 2669
Added: 500339
Subtracted: 499661
Max: 47197
Elapsed: 3417
Added: 500339
Subtracted: 499661
Max: 47367
Your slow version may be rewritten as
while(heavyloop)
{
std::string tempA = path + node;
std::string tempB = tempA + "/";
if(tempB == curNode)
{
do something
}
}
Yes, it is not a full analog, but makes temporary objects more visible.
See two temporary objects: tempA and tempB. They are created because std::string::operator+ always generates new std::string object. This is how std::string is designed. A compiler won't be able to optimize this code.
There is a technique in C++ called expression templates to address this issue, but again, it it done on library level.
For class types (like std::string) there is no requirement that the conventional relationship between operator + and operator += be honoured like you expect. There is certainly no requirement that a = a + b and a += b have the same net effect, since operator=(), operator+() and operator+=() can all potentially be implemented individually, and not work together in tandem.
As such, a compiler would be semantically incorrect if it replaced
if(path+node+"/" == curNode)
with
std::string buffer = path;
buffer += node;
buffer += "/";
if (buffer == curNode)
If there was some constraint in the standard, for example a fixed relationship between overloaded operator+() and overloaded operator+=() then the two fragments of code would have the same net effect. However, there is no such constraint, so the compiler is not permitted to do such substitutions. The result would be changing meaning of the code.
path+node+"/" will allocate a temp variable string to compare with curNode,it's the c++ implement.
I have an inline function
string myFunction() { return ""; }
Compared with
string myFunction() { return string(); }
does it has a performance sacrifice?
Tested it on VC2012 release with std::string and QString (although on QString, the two returns different results : thanks to DaoWen). Both show the second version is about 3 times faster than the first. Thanks to all your answers and comments. Tested code is attached below
#include <iostream>
#include <string>
#include <ctime>
using namespace std;
inline string fa() { return ""; }
inline string fb() { return string(); }
int main()
{
int N = 500000000;
{
clock_t begin = clock();
for (int i = 0; i < N; ++i)
fa();
clock_t end = clock();
double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC;
cout << "fa: " << elapsed_secs << "s \n";
}
{
clock_t begin = clock();
for (int i = 0; i < N; ++i)
fb();
clock_t end = clock();
double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC;
cout << "fb: " << elapsed_secs << "s \n";
}
return 0;
}
They will each cause different std::string constructors.
The std::string() -- will create an empty object.
The "" will construct using std::string(char*)
The later will internally do a strlen+strcpy which is not needed in the first, so very small difference.
In the example you posted, return ""; will be automatically translated to return string(""); at compile time. Unless string("") is significantly slower than string(), then there shouldn't be much of a difference.
In your comment you mention that you're actually using QString, not std::string. Note that QString() constructs a null string (isNull() and isEmpty() both return true), whereas QString("") constructs an empty string (isEmpty() returns true but isNull() returns false). They're not the same thing! You can think of QString() like char * str = NULL;, and QString("") like char * str = "";.
Use return string(); as that will use the default constructor. A good Standard Library implementation will probably not even initialise the string buffer at that point.
The constructor from const char* must take a string copy. So I think return ""; will be slower.
But, to be really sure, race your horses.