String overflow in C++ - c++

Question

I'm converting large graph files using a simple C++ program. Basically I mainly want to switch the delimiters used in the file. These graph files contain extremely long lines. My converting works fine for small files, but I lose edges in the output file for the large files. So I checked what the part of my code that reads actually returns and it seems the error is already in there. Using the following, I do not receive the full lines from the graph in curLineMetis or ssMetis (My guess is that there's an overflow somewhere and I lose the last part of the line thats too big for the allocated memory)
using namespace std;
void exampleFunc(const char* inputfileMetis) {
ifstream metisFile;
metisFile.open(inputfileMetis, ios::in);
if(!metisFile.is_open()) {
cout << "Could not open input file";
return;
}
long nMet = 0;
long mMet = 0;
string curLineMetis;
// skip first line
getline(metisFile, curLineMetis);
while(getline(metisFile, curLineMetis)) {
nMet++;
long targetMetis;
stringstream ssMetis(curLineMetis);
while(ssMetis >> targetMetis) {
mMet++;
}
}
}
I checked on google and it said std::string managed its own memory and would not overflow? I also tried it without the stringstream, just using an iterator over chars from the string before and the same happened. I'm using c++11 if that makes a difference. (the error expresses itself e.g. in mMet being different from the number of edges stated in the graph file itself, while the number of vertices nMet is the same, and the number of edges is also correct for smaller graphs)

Answer 1

I am not quite sure what is going wrong for you with your code. The standard library functions and types should be able to handle long lines. However, to reduce memory usage and to work around your problem you might want to process a part of a line at a time. Since the file format seemed interesting I wrote this program to read the graph into a vector<vector<int>>. It reproduces the graph file from that representation and produces a file identical to the input. Perhaps something in here will be of use to you. Good luck.
#include <cassert>
#include <fstream>
#include <iostream>
#include <sstream>
#include <vector>
int
main(int argc, char* argv[])
{
size_t const buf_size = 40; // big enough for a 64bit integer and then some
char buf[buf_size];
assert(argc >= 2);
std::ifstream is (argv[1]);
assert(is);
std::vector<std::vector<int>> graph;
std::vector<int> line;
std::string header;
std::getline(is, header);
{ std::istringstream is_header (header);
size_t n;
is_header >> n;
graph.reserve(n);
}
char delim;
for (;;) {
is.get(buf, buf_size, ' ');
if (!is) break;
is.get(delim);
line.push_back(std::atoi(buf));
while (is.peek() == '\n') {
graph.emplace_back(std::move(line));
(void) is.get();
}
}
std::cout << graph.size() << '\n';
size_t max_row = graph[0].size();
size_t row_sum = graph[0].size();
for (size_t i = 1, I = graph.size(); i < I; ++i) {
max_row = std::max(max_row, graph[i].size());
row_sum += graph[i].size();
}
std::cout << max_row << '\n';
std::cout << row_sum << '\n';
std::ofstream os ("dup.graph");
os << header << '\n';
for (size_t i = 0, I = graph.size(); i < I; ++i) {
for (size_t j = 0, J = graph[i].size(); j < J; ++j)
os << graph[i][j] << ' ';
os << '\n';
}
return 0;
}