Text parsing a log file effectively in c++ - c++

Question

I want to build a log browser. For which I need to code effectively. Given is a simple code for parsing.
Please let me know if this code is okay or any improvements to be given.
Also strtok(o,delim) function in the below given program is not clear. So please explain me about its functionality.
// parsing ex.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
using std::cout;
using std::endl;
#include <fstream>
using std::ifstream;
#include <cstring>
const int MAX_CHARS_PER_LINE = 512;
const int MAX_TOKENS_PER_LINE = 20;
const char* const DELIMITER = " ";
int main()
{
// create a file-reading object
ifstream fin;
fin.open("C:\\Personal\\data.txt"); // open a file
if (!fin.good())
return 1; // exit if file not found
// read each line of the file
while (!fin.eof())
{
// read an entire line into memory
char buf[MAX_CHARS_PER_LINE];
fin.getline(buf, MAX_CHARS_PER_LINE);
// parse the line into blank-delimited tokens
int n = 0; // a for-loop index
// array to store memory addresses of the tokens in buf
const char* token[MAX_TOKENS_PER_LINE] = {}; // initialize to 0
// parse the line
token[0] = strtok(buf, DELIMITER); // first token
if (token[0]) // zero if line is blank
{
for (n = 1; n < MAX_TOKENS_PER_LINE; n++)
{
token[n] = strtok(0, DELIMITER); // subsequent tokens
if (!token[n]) break; // no more tokens
}
}
// process (print) the tokens
for (int i = 0; i < n; i++) // n = #of tokens
cout << "Token[" << i << "] = " << token[i] << endl;
cout << endl;
}
}

Answer 1

Your code works, except there are no boundary checks. It will fail if a line in the file is longer than MAX_CHARS_PER_LINE. while (!fin.eof()){...} is prone to other errors as well.
You can easily solve this problem with std::string
The code also fails if a line contains more than MAX_TOKENS_PER_LINE tokens. You can solve this by using std::vector
For improvements, use std::string instead of character arrays.
Use std::vector instead of C-style arrays.
Use std::stringstream instead of strtok
The advantage is that you don't have to worry about maximum line length, or maximum number of tokens per line.
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <vector>
const char CDELIMITER = ' ';
int main()
{
...
std::string buf;
//read the file line by line
while (std::getline(fin, buf))
{
//convert the line in to stream:
std::istringstream ss(buf);
//declare vector of string (instead of fixed array)
std::vector<std::string> vec;
//read the line, word by word
while (std::getline(ss, buf, CDELIMITER))
vec.push_back(buf);
for (size_t i = 0; i < vec.size(); i++)
std::cout << "Token[" << i << "] = " << vec[i] << "\n";
std::cout << "\n";
}
return 0;
}