Properly handling input when file has less numbers than expected - c++

Question

I'm attempting to read in a file with 3 floating point numbers per line. Right now, I have this implemented as:
std::ifstream inFile(inName.c_str());
if (!inFile) {
prterr("in ASCII file could not be opened!\n");
return -1;
}
std::vector<double> xData, yData, zData;
xData.resize(nPoints);
yData.resize(nPoints);
zData.resize(nPoints);
inFile.precision(std::numeric_limits<double>::digits10+1);
for (int i = 0; i < nPoints; ++i) {
inFile >> xData[i] >> yData[i] >> zData[i];
inFile.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
}
so that the program successfully runs even if the user inputs more than 3 numbers per line. However, sometimes a user tries to run the program with <3 numbers per line. When this happens, the parser will obviously store the data incorrectly.
I would like to either (a) throw an error if the file has less than 3 numbers per line, or (b) Only store the first N numbers per line in their respective vectors if only N numbers per line are present in the file. The trick is, I want to do this as quickly as possible, as my datasets can be several GBs. I can be guaranteed that my file has the exact same amount of numbers per line.
Is there a graceful and efficient way to perform (b)? I know I could implement (a) just by reading the first line as a string separately before the for loop, but that seems quite ugly. Is there a better way to do this?

Answer 1

I know you dd not want to read in the first line as a std::string but you need someway to find out how many white space separated columns there are and unfortunately a newline is treated like white space. If you are okay with doing that though then you can see how man columns you have with
std::ifstream inFile(inName.c_str());
std::vector<int> columns_in_file;
std::string temp;
std::getline(inFile, temp);
std::stringstream ss(temp);
int number;
while (ss >> number)
columns_in_file.push_back(number);
Then what we need to do is set up a 2d vector that will have the correct number of columns and rows.
// get number of columns. 3 max
int columns = columns_in_file.size() <= 3 ? columns_in_file.size() : 3;
std::vector<std::vector<int>> data(nPoints, std::vector<int>(columns));
// now we add the data we already read
for (int i = 0; i < columns; i++)
data[0][i] = columns_in_file[i];
Now we have a vector that is the same size as the file unless the file has more then 3 columns and has the first line of data in it. Now we have a decision to make, since you will only ever need to call
inFile.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
while reading if columns_in_file.size() > 3, then we don't want to call it if it is not needed. We could either have the reading code in two different functions or in two different blocks in an else if statement. The latter is what I will show but know you could refactor it into function calls. So to actually read the file we would have something like
if (columns <= 3)
{
for (int i = 0; i < nPoints; i++)
{
for(int j = 0; j < columns; j++)
{
infile >> data[i][j];
}
}
}
else
{
for (int i = 0; i < nPoints; i++)
{
for(int j = 0; j < columns; j++)
{
infile >> data[i][j];
inFile.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
}
}
}

Answer 2

You can first get the number of columns in the file while reading the first set of values,as a string, and then use the count with another loop inside the first for loop:
[EDIT] As per the comments given(and the learning continues again), instead of making the all vectors resize initially, you can resize them depending on the available columns. this will avoid unnecessary space consumption for the unused vectors.
std::vector<double> Data[3];//the x,y,z data set(Assuming the maximum number of columns can't be >3)
//you can decide which of the vectors(x,y,z) are used by looking at the column count
inFile.precision(std::numeric_limits<double>::digits10+1);
int count=0;//count the number of columns
string first_line;
double temp;
getline(inFile,first_line);
istringstream ss(first_line);
while(ss>>temp && count<3)
{
Data[count].resize(nPoints);
Data[count][0]=temp;
count++;
}
for(int i=1; i<nPoints&& inFile.peek() != EOF ; i++)
{
for(int j=0;j<count;j++)
{
inFile>>temp;
Data[j][i]=temp;
}
inFile.ignore(std::numeric_limits<std::streamsize>::max(), '\n');
}