How do I compare a section of a string without copying? - c++

Question

I have a long string that I'm iterating through, and at each iteration I compare a section of the string to a constant and store some parts of the string. In my actual code, this code runs millions of times and is the main bottleneck. I think it's due to the excessive use of std::string::substr.
#include <iostream>
#include <map>
#include <string>
#include <vector>
int main() {
std::string str("0=My,1=comma,2=separated,3=string,0=with,3=repeated,7=IDs");
std::vector<std::string> out0;
std::map<std::string, std::string> out;
size_t pos = str.find(',');
// loop over the string, collecting "key=value" pairs
while (pos < str.size() - 1) {
if (str.substr(pos + 1, 2) == "0=") {
auto newPos = str.find(',', pos + 3);
out0.push_back(str.substr(pos + 3, newPos - pos - 3);
pos = newPos;
} else {
size_t eqPos = str.find('=', pos + 1);
auto newPos = str.find(',', eqPos + 1);
out[str.substr(pos + 1, eqPos - pos - 1)] = str.substr(eqPos + 1, newPos - eqPos - 1);
}
}
// print out the data structures (this doesn't happen in my actual code)
std::cout << "out0:";
for (auto& entry : out0) {
std::cout << ' ' << entry;
}
std::cout << std::endl;
std::cout << "out:";
for (auto it : out) {
std::cout << ' ' << it->first << '=' << it->second;
}
}
Here are my questions:
How can I perform comparisons on the string without performing a copy and without writing the comparison for each character, e.g. str[pos + 1] == '0' && str[pos + 2] == '=' && ...?
How can I store references to substrings, instead of making copies every time I add to out0 and out?
This may be a great case for the use of char *, but I've never used it before.
Edit:
Unfortunately, I've only got C++11; otherwise, std::string_view is the best answer. Is there a way to accomplish the storage of references without std::string_view?

Answer 1

If you have C++17, you can use string_view thus: (untested code):
string_view sv{str.data() + pos, 2};
if (sv == "0=") ...
No copies. Or even (all in one go):
if (string_view{str.data() + pos, 2} == "0=") ...
If you don't have string_view, you can use char_traits:
if (std::char_traits<char>::compare(str.data() + pos, "0=", 2) == 0) ...

Answer 2

Since people have posted std::string_view, here is the plain old C pointers version.
(Didn't test though, but it'll give you the idea)
See below:
std::string str("0=My,1=comma,2=separated,3=string,0=with,3=repeated,7=IDs");
std::string substr("test");
.
. Inside some function
.
const char *str_p = str.c_str(); // String you want to compare with a substring
const char *substr_p = substr.c_str(); // Your substring
size_t str_len = str.length();
size_t substr_len = substr.length();
bool comparison_result = true;
for(size_t i = 0; i < str_len - substr_len; i++) {
for(size_t j = 0; j < substr_len; j++) {
if(*(str_p + i + j) != *(substr_p + j)) {
comparison_result = false;
break;
}
if (j == substr_len - 1) { // We can only reach here when substring is hit
comparison_result = true;
i = str_len - substr_len;
break;
}
}
}
return comparison_result;
EDIT:
Due to #Toby Speight's suggestion in the comments (which I find very nice), I'm implementing a std::memcmp() version as well. In that case, the inner loop becomes:
.
. Inside some function
.
const char *str_p = str.c_str(); // String you want to compare with a substring
const char *substr_p = substr.c_str(); // Your substring
size_t str_len = str.length();
size_t substr_len = substr.length();
bool comparison_result = false;
for(size_t i = 0; i < str_len - substr_len; i++) {
if(std::memcmp(str_p + i, substr_p, substr_len) == 0) {
comparison_result = true;
break;
}
}
return comparison_result;
EDIT:
We got another request, this time from #Alexander Zhang, let's implement it:
.
. Inside some function
.
const char *str_p = str.c_str(); // String you want to compare with a substring
const char *substr_p = substr.c_str(); // Your substring
size_t str_len = str.length();
size_t substr_len = substr.length();
bool comparison_result = false;
for(size_t i = 0; i < str_len - substr_len; i++) {
if(std::memcmp(&str_p[i], &substr_p[0], substr_len) == 0) {
comparison_result = true;
break;
}
}
return comparison_result;

Answer 3

Use a std::string_view instead of std::string for the key and value of of out. std::string_view holds a pointer to the string, and a size of the string, so it is very light weight. This lets you extract the information you need, but without having to copy any of the characters in string and any potential memory allocations of creating those strings.
What you'll need to do is get a string_view from the std::string, and then use that string_view to get all of the sub strings you need.

Answer 4

std::string has compare() methods that take a const char* substring as input. You don't need to use std::string::substr() to compare substrings, eg:
#include <iostream>
#include <map>
#include <string>
#include <vector>
int main() {
std::string str("0=My,1=comma,2=separated,3=string,0=with,3=repeated,7=IDs");
std::vector<std::string> out0;
std::map<std::string, std::string> out;
size_t startPos = 0, delimPos, nameStart, nameEnd, valueStart, valueEnd;
// loop over the string, collecting "key=value" pairs
while (startPos < str.size()){
nameStart = startPos;
delimPos = str.find_first_of("=,", startPos, 2);
if (delimPos == std::string::npos) {
nameEnd = valueStart = valueEnd = str.size();
}
else {
nameEnd = delimPos;
if (str[delimPos] == '=') {
valueStart = nameEnd + 1;
valueEnd = str.find(',', valueStart);
if (valueEnd == std::string::npos) {
valueEnd = str.size();
}
}
else {
valueStart = valueEnd = nameEnd;
}
}
// TODO: if needed, adjust name(Start|End) and value(Start|End) to
// ignore leading/trailing whitespace around the name and value
// substrings...
if (str.compare(nameStart, nameEnd - nameStart, "0", 1) == 0) {
out0.push_back(str.substr(valueStart, valueEnd - valueStart));
} else {
out[str.substr(nameStart, nameEnd - nameStart)] = str.substr(valueStart, valueEnd - valueStart);
}
startPos = valueEnd + 1;
}
// print out the data structures
std::cout << "out0:";
for (auto& entry : out0) {
std::cout << ' ' << entry;
}
std::cout << std::endl;
std::cout << "out:";
for (auto it : out) {
std::cout << ' ' << it->first << '=' << it->second;
}
}
Output:
out0: My with
out: 1=comma 2=separated 3=repeated 7=IDs
Live Demo
You could take this a step further to eliminate the use of substr() altogether by not storing std::string values in your std::vector and std::map at all, but rather store std::pair<char*, size_t>:
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include <utility>
using StrView = std::pair<const char*, size_t>;
StrView makeStrView(const char *str, size_t size) {
return std::make_pair(str, size);
}
struct compareStrView {
bool operator()(const StrView &lhs, const StrView &rhs) const {
if (lhs.second == rhs.second)
return (std::char_traits<char>::compare(lhs.first, rhs.first, lhs.second) < 0);
return (lhs.second < rhs.second);
}
};
std::ostream& operator<<(std::ostream &os, const StrView &rhs) {
return os.write(rhs.first, rhs.second);
}
int main() {
std::string str("0=My,1=comma,2=separated,3=string,0=with,3=repeated,7=IDs");
std::vector<StrView> out0;
std::map<StrView, StrView, compareStrView> out;
size_t startPos = 0, delimPos, nameStart, nameEnd, valueStart, valueEnd;
// loop over the string, collecting "key=value" pairs
while (startPos < str.size()){
nameStart = startPos;
delimPos = str.find_first_of("=,", startPos, 2);
if (delimPos == std::string::npos) {
nameEnd = valueStart = valueEnd = str.size();
}
else {
nameEnd = delimPos;
if (str[delimPos] == '=') {
valueStart = nameEnd + 1;
valueEnd = str.find(',', valueStart);
if (valueEnd == std::string::npos) {
valueEnd = str.size();
}
}
else {
valueStart = valueEnd = nameEnd;
}
}
// TODO: if needed, adjust nameStart/End and valueStartEnd to
// ignore leading/trailing whitespace around the name and value
// substrings...
if (str.compare(nameStart, nameEnd - nameStart, "0", 1) == 0) {
out0.push_back(makeStrView(&str[valueStart], valueEnd - valueStart));
} else {
out[makeStrView(&str[nameStart], nameEnd - nameStart)] = makeStrView(&str[valueStart], valueEnd - valueStart);
}
startPos = valueEnd + 1;
}
// print out the data structures
std::cout << "out0:";
for (auto& entry : out0) {
std::cout << ' ' << entry;
}
std::cout << std::endl;
std::cout << "out:";
for (auto &it : out) {
std::cout << ' ' << it.first << '=' << it.second;
}
}
Output:
out0: My with
out: 1=comma 2=separated 3=repeated 7=IDs
Live Demo
In C++17, you can use std::string_view instead:
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include <string_view>
int main() {
std::string str("0=My,1=comma,2=separated,3=string,0=with,3=repeated,7=IDs");
std::string_view sv(str);
std::vector<std::string_view> out0;
std::map<std::string_view, std::string_view> out;
size_t startPos = 0, delimPos, nameStart, nameEnd, valueStart, valueEnd;
// loop over the string, collecting "key=value" pairs
while (startPos < sv.size()){
nameStart = startPos;
delimPos = sv.find_first_of("=,", startPos, 2);
if (delimPos == std::string_view::npos) {
nameEnd = valueStart = valueEnd = sv.size();
}
else {
nameEnd = delimPos;
if (sv[delimPos] == '=') {
valueStart = nameEnd + 1;
valueEnd = sv.find(',', valueStart);
if (valueEnd == std::string_view::npos) {
valueEnd = sv.size();
}
}
else {
valueStart = valueEnd = nameEnd;
}
}
// TODO: if needed, adjust nameStart/End and valueStartEnd to
// ignore leading/trailing whitespace around the name and value
// substrings...
if (sv.compare(nameStart, nameEnd - nameStart, "0", 1) == 0) {
out0.push_back(sv.substr(valueStart, valueEnd - valueStart));
} else {
out[sv.substr(nameStart, nameEnd - nameStart)] = sv.substr(valueStart, valueEnd - valueStart);
}
startPos = valueEnd + 1;
}
// print out the data structures
std::cout << "out0:";
for (auto& entry : out0) {
std::cout << ' ' << entry;
}
std::cout << std::endl;
std::cout << "out:";
for (auto &it : out) {
std::cout << ' ' << it.first << '=' << it.second;
}
}

Answer 5

You can try to use Regex to split the value pair tuples.
Although haven't tested if any faster
This expression should do the trick, just get all the match (all the pairs)
(?:(\d)+=(?:([^,]*),?))*?
https://regex101.com/r/PDZMq0/1