I have written a program to read a CSV file but I'm having some trouble in extracting data from that CSV file in c++. I want to count the no. of columns starting from the 5th column in the 1st row until the last column of the 1st row of the CSV file. I have written the following code to read a CVS file, but I am not sure how shall I count the no. of columns as I have mentioned before.
Will appreciate it if anyone could please tell me how shall I go about it?
char* substring(char* source, int startIndex, int endIndex)
{
int size = endIndex - startIndex + 1;
char* s = new char[size+1];
strncpy(s, source + startIndex, size); //you can read the documentation of strncpy online
s[size] = '\0'; //make it null-terminated
return s;
}
char** readCSV(const char* csvFileName, int& csvLineCount)
{
ifstream fin(csvFileName);
if (!fin)
{
return nullptr;
}
csvLineCount = 0;
char line[1024];
while(fin.getline(line, 1024))
{
csvLineCount++;
};
char **lines = new char*[csvLineCount];
fin.clear();
fin.seekg(0, ios::beg);
for (int i=0; i<csvLineCount; i++)
{
fin.getline(line, 1024);
lines[i] = new char[strlen(line)+1];
strcpy(lines[i], line);
};
fin.close();
return lines;
}
I have attached a few lines from the CSV file:-
Province/State,Country/Region,Lat,Long,1/22/20,1/23/20,1/24/20,
,Afghanistan,33.0,65.0,0,0,0,0,0,0,0,
,Albania,41.1533,20.1683,0,0,0,0
What I need is, in the 1st row, the number of dates after Long.
To answer your question:
I have attached a few lines from the CSV file:-
Province/State,Country/Region,Lat,Long,1/22/20,1/23/20,1/24/20, ,Afghanistan,33.0,65.0,0,0,0,0,0,0,0, ,Albania,41.1533,20.1683,0,0,0,0
What I need is, in the 1st row, the number of dates after Long.
Yeah, not that difficult - that's how I would do it:
#include <iostream>
#include <string>
#include <fstream>
#include <regex>
#define FILENAME "test.csv" //Your filename as Macro
//(The compiler just sees text.csv instead of FILENAME)
void read(){
std::string n;
//date format pattern %m/%dd/%YY
std::regex pattern1("\\b\\d{1}[/]\\d{2}[/]\\d{2}\\b");
//date format pattern %mm/%dd/%YY
std::regex pattern2("\\b\\d{2}[/]\\d{2}[/]\\d{2}\\b");
std::smatch result1, result2;
std::ifstream file(FILENAME, std::ios::in);
if ( ! file.is_open() )
{
std::cout << "Could not open file!" << '\n';
}
do{
getline(file,n,',');
//https://en.cppreference.com/w/cpp/string/basic_string/getline
if(std::regex_search(n,result1,pattern1))
std::cout << result1.str(1) << n << std::endl;
if(std::regex_search(n,result2,pattern2))
std::cout << result2.str(1) << n << std::endl;
}
while(!file.eof());
file.close();
}
int main ()
{
read();
return 0;
}
The file test.csv contains the following for testing:
Province/State,Country/Region,Lat,Long,1/22/20,1/23/20,1/24/20, ,Afghanistan,33.0,65.0,0,0,0,0,0,0,0, ,Albania,41.1533,20.1683,0,0,0,0
Province/State,Country/Region,Lat,Long,1/25/20,12/26/20,1/27/20, ,Bfghanistan,33.0,65.0,0,0,0,0,0,0,0, ,Blbania,41.1533,20.1683,0,0,0,0
It actually is pretty simple:
getline takes the open file and "escapes" at a so called escape-charachter,
in your case a comma ','.
(That is the very best way I found in reading csv - you can replace it with whatever you want, for example: ';' or ' ' or '...' - guess you get the drill)
After this you got all data nicely separated underneath one another without a comma.
Now you can "filter" out what you need. I use regex - but use what ever you want.
(Just fyi: For c++ tagged questions you shouldn't use c-style like strncpy..)
I gave you an example for 1.23.20 (m/dd/yy) and to make it simple if your file contains a november or december like 12.22.20 (mm/dd/yy) to make
the regex pattern more easy to read/understand in 2 lines.
you can/may have to expand the regex pattern if the data somehow matches
your date format in the file, really good explained here and not as complicated as it looks.
From that point you can put all the printed stuff f.e. in a vector (some more convenient array) to handle and/or pass/return data - that's up to you.
If you need more explaining I am happy to help you out and/or expand this example, just leave a comment.
You basically want to search for the seperator substring within your line (normally it is ';').
If you print out your lines it should look like this:
a;b;c;d;e;f;g;h
There are several ways to achieve what you want, I would look for a strip or split upon character function. Something along the example below should work. If you use std you can go with str.IndexOf instead of a loop.
int rows(char* line,char seperator, int count) {
unsigned length = strlen(line);
for (int i=pos; i<length;i++){
if(strcmp(line[i],seperator)) break;
}
count++;
if (i<length-1) return rows(substring(line,i,length-i),seperator,count);
else return count;
}
The recursion can obviously be replaced by one loop ;)
int countSign(char* line, char* sign){
unsigned l = strlen(line);
int count = 0;
for (int i=0; i < l; i++) {
if(strcmp(line[i],sign)) count++;
}
}
Given a file with some empty lines, some lines containing only integers, how would I make an array containing all of the integers? I have found methods for strings, but I need a list of integers. I want to do this using getline, but getline gives a string for "line"
A nonfunctioning example which returns the number of integers in the file and modifies a given array:
int getLinesFromFile(string fileName, int arr[], int arrLen) {
ifstream userFile;
userFile.open(fileName);
if (userFile.is_open()) {
int line;
int arrCount = 0;
while (getline(userFile, line)) {
if (tline.length() != 0 && arrCount < arrLen) {
arr[arrCount] = line;
arrCount++;
}
}
return arrCount;
}
else {
return -1;
}
userFile.close();
}
You can just use the >>-operator to read values. It will ignore any whitespace, including empty lines, between values. Here is a modified version of your function that uses it:
int getLinesFromFile(std::string fileName, int arr[], int arrLen) {
std::ifstream userFile(fileName);
int count = 0;
while(count < arrLen) {
int value;
userFile >> value;
if (!userFile.good())
return -1;
arr[count++] = value;
}
return count;
}
Note that you don't need to open and close the file manually, RAII will take care of that for you. Also, if the file could not be opened successfully, or if any other error occured while reading the file, userFile.good() will return false, so you can use that to detect and return an error. It's unclear if your function is supposed to read exactly arrLen values or if less is also valid. But at least you should take care not to write past the end of the provided array.
I would like to read in a file like this:
13.3027 29.2191 2.39999
13.3606 29.1612 2.39999
13.3586 29.0953 2.46377
13.4192 29.106 2.37817
It has more than 1mio lines.
My current cpp code is:
loadCloud(const string &filename, PointCloud<PointXYZ> &cloud)
{
print_info("\nLoad the Cloud .... (this takes some time!!!) \n");
ifstream fs;
fs.open(filename.c_str(), ios::binary);
if (!fs.is_open() || fs.fail())
{
PCL_ERROR(" Could not open file '%s'! Error : %s\n", filename.c_str(), strerror(errno));
fs.close();
return (false);
}
string line;
vector<string> st;
while (!fs.eof())
{
getline(fs, line);
// Ignore empty lines
if (line == "")
{
std::cout << " this line is empty...." << std::endl;
continue;
}
// Tokenize the line
boost::trim(line);
boost::split(st, line, boost::is_any_of("\t\r "), boost::token_compress_on);
cloud.push_back(PointXYZ(float(atof(st[0].c_str())), float(atof(st[1].c_str())), float(atof(st[2].c_str()))));
}
fs.close();
std::cout<<" Size of loaded cloud: " << cloud.size()<<" points" << std::endl;
cloud.width = uint32_t(cloud.size()); cloud.height = 1; cloud.is_dense = true;
return (true);
}
Reading this file currently takes really long. I would like to speed this up any ideas how to do that?
You can just read the numbers instead of the whole line plus parsing, as long as the numbers always come in sets of three.
void readFile(const std::string& fileName)
{
std::ifstream infile(fileName);
float vertex[3];
int coordinateCounter = 0;
while (infile >> vertex[coordinateCounter])
{
coordinateCounter++;
if (coordinateCounter == 3)
{
cloud.push_back(PointXYZ(vertex[0], vertex[1], vertex[2]));
coordinateCounter = 0;
}
}
}
Are you running optimised code? On my machine your code reads a million values in 1800ms.
The trim and the split are probably taking most of the time. If there is white space at the beginning of the string trim has to copy the whole string contents to erase the first characters. split is creating new string copies, you can optimise this by using string_view to avoid the copies.
As your separators are white space you can avoid all the copies with code like this:
bool loadCloud(const string &filename, std::vector<std::array<float, 3>> &cloud)
{
ifstream fs;
fs.open(filename.c_str(), ios::binary);
if (!fs)
{
fs.close();
return false;
}
string line;
vector<string> st;
while (getline(fs, line))
{
// Ignore empty lines
if (line == "")
{
continue;
}
const char* first = &line.front();
const char* last = first + line.length();
std::array<float, 3> arr;
for (float& f : arr)
{
auto result = std::from_chars(first, last, f);
if (result.ec != std::errc{})
{
return false;
}
first = result.ptr;
while (first != last && isspace(*first))
{
first++;
}
}
if (first != last)
{
return false;
}
cloud.push_back(arr);
}
fs.close();
return true;
}
On my machine this code runs in 650ms. About 35% of the time is used by getline, 45% by parsing the floats, the remaining 20% is used by push_back.
A few notes:
I've fixed the while(!fs.eof()) issue by checking the state of the stream after calling getline
I've changed the result to an array as your example wasn't a mcve so I didn't have a definition of PointCloud or PointXYZ, its possible that these types are the cause of your slowness.
If you know the number of lines (or at least an approximation) in advance then reserving the size of the vector would improve performance
I am trying to read in the first 7 chars of a file named "board.txt" into a vector<'char> but I am having issues for some reason. I am not too familiar with C++ so any advice would be appreciated, here is the code I have so far
//rack
int charCount = 0;
char ch;
ifstream rackIn("board.txt");
while(rackIn.get(ch) && charCount < 7){
this->getMyRack().push_back(ch);
}
And here is the function getMyRack used in the code above:
vector<char> board::getMyRack(){
return this->myRack;
}
myRack is a char vector
I tried to test this in my main using this:
for (int i = 0; i < test->getMyRack().size(); ++i){
cout << test->getMyRack().at(i);
}
but it does not output anything, why are the chars i am reading in not being added into my char vectors?
Because you don't put char in your vector. Your function getMyRack() returns vector but not address of your vector. You can add method to your class board for adding char, for example:
void board::addChar(char c){
this->myRack.push_back(c);
}
And then call this function:
while(rackIn.get(ch) && charCount < 7){
this->addChar(ch);
}
Or change the return type of your function.
read line one or (how much lines required) from file to a string
create substring of 7 chars from beginning
std::ifstream file("board.txt");
std::string str;
// to read single line
std::getline(file, str);
// to read 7 chars
str= str.substr(0,7);
vector<char> char_buf;
for(size_t i =0; i <= str.size();i++)
{
char_buf.push_back(str[i])
}
// use the char_buf
easier or second way is use
#include<fstream> // for ifstream
#include <cstdlib> // for exit()
std::string file_name ="board.txt";
std::ifstream input_stream;
std::vector<char> char_buf;
input_stream.open(file_name);
if(input_stream.fail()) { exit(0);}
int char_no=0;
while(i<=7)
{
char c = input_stream.get();
char_buf.push_back(c);
i++;
}
// use char_buf
std::string str;
int char_count=0;
// Read the next line from File untill it reaches the 7.
while (std::getline(in, str)&& char_count!=7)
{
// Line contains string of length > 0 then save it in vector
if (str.size() > 0)
your_char_vector.push_back(str);
char_count++;
if(char_count==7)
break;
}
I have three string file that will be stored into dynamic array, but I just try one of three file to test if this succed, so I'll do the same way to handle the three file i have.
the goal i'll shown the string that I get from the file to a ListView
this my code.
void __fastcall TFrmNewPeta::showDefaultRute() {
std::string lineDataAwal;
std::ifstream ifs_Awal;
int tempIndexAwal = 0;
ifs_Awal.open("DefaultDataAwal");
/*counting the line*/
while(std::getline(ifs_Awal,lineDataAwal)){++tempIndexAwal;}
/*use dynamic array to stored string*/
std::string *s = new std::string[tempIndexAwal];
for(int dx=0;dx<tempIndexAwal;dx++)
{
while(std::getline(ifs_Awal,lineDataAwal))
s[dx] = lineDataAwal[dx++];
}
for(int dex =0;dex<tempIndexAwal;++dex)
{
ItemDefult = ListView1->Items->Add();
ItemDefult->Caption = String(IntToStr(dex + 1));
ItemDefult->SubItems->Add(s[dex].c_str());
}
ifs_Awal.close();
delete []s;
s = NULL;
}
there's no errors during compile, but the result ListView just showing the number with this code ItemDefult->Caption = String(IntToStr(dex + 1));
can anyone show me how the best way for i do.
You are reading the file, leaving it open, and expecting to read it again. That won't work because the cursor in the file is at the end of the file (so your second while loop does nothing).
A much better approach would be:
std::vector<std::string> lines;
std::string line;
std::ifstream fin("Youfilename");
while (std::getline(fin, line))
{
lines.push_back(line);
}
fin.close();
// add data to your list view
its easier if you use std::vector for dynamic arrays and don't forget to first include the file header with #include<vector>
void __fastcall TFrmNewPeta::showDefaultRute() {
std::string lineDataAwal;
std::ifstream ifs_Awal;
std::vector<std::string> vec;
ifs_Awal.open("DefaultDataAwal");
/*get the string of lineDataAwal */
while(std::getline(ifs_Awal,lineDataAwal))
{ vec.push_back(lineDataAwal);}
for(int dex =0;dex<vec.size();++dex)
{
ItemDefult = ListView1->Items->Add();
ItemDefult->Caption = String(IntToStr(dex + 1));
ItemDefult->SubItems->Add(vec.at(dex).c_str());
}
ifs_Awal.close();
}
Hope this helps