I am new to file-handling...
I am writing a program that saves data in text-files in the following format:
3740541120991
Syed Waqas Ali
Rawalpindi
Lahore
12-12-2012
23:24
1
1
(Sorry for the bad alignment, it's NOT a part of the program)
Now I'm writing a delete function for the program that would delete a record.
So far this is my code:
void masterSystem::cancelReservation()
{
string line;
string searchfor = "3740541120991";
int i=0;
ifstream myfile("records.txt");
while (getline(myfile, line))
{
cout << line << endl;
if (line==searchfor)
{
// DELETE THIS + THE NEXT 8 LINES
}
}
}
I've done a bit of research and have found out that there is no easy way to access the line of a text file so we have to create another text file.
But the problem arises that how do I COPY the records/data before the record being deleted into the NEW text file?
Open the input file; read one line at a time from the input file. If you decide to want to keep that line, write it to the output file. On the other hand, if you want to 'delete' that line, don't write it to the output file.
You could have a record per line and make even more easy for example:
3740541120991|Syed Waqas Ali|Rawalpindi|Lahore|12-12-2012|23:24|1|1
and the | character saparating each field. This is a well known technic knows as CSV (Comma separated Values)
This way you don't have to worry about reading consecutive lines for erase a record and add a record access the file only once.
So your code becoms into:
void masterSystem::cancelReservation()
{
string line;
string searchfor = "3740541120991";
ifstream myfile("records.txt");
while (getline(myfile, line))
{
// Here each line is a record
// You only hace to decide if you will copy
// this line to the ouput file or not.
}
}
Don't think only about removing a record, there are others operations you will need to do against this file save a new record, read into memory and search.
Think a moment about search, and having your current desing in mind, try to answer this: How many reservations exists for date 12-12-2012 and past 12:00 AM?
In your code you have to access the file 8 times per record even if the other data is irrelevant to the question. But, if you have each record in a line you only have to access file 1 time per record.
With a few reservations the diference is near 0, but it grows exponentially (n^8).
Related
I'm writing a function which reads a CSV file using getline() and converts data to the vector of vectors. To test it I've tried to read two files with the same delimiter: one imported from the internet and second exported from R datasets. The first few lines of each looks like:
File1.csv
User ID,Category 1,Category 2,Category 3,Category 4,Category 5,Category 6,Category 7,Category 8,Category 9,Category 10
User 1,0.93,1.8,2.29,0.62,0.8,2.42,3.19,2.79,1.82,2.42
User 2,1.02,2.2,2.66,0.64,1.42,3.18,3.21,2.63,1.86,2.32
User 3,1.22,0.8,0.54,0.53,0.24,1.54,3.18,2.8,1.31,2.5
User 4,0.45,1.8,0.29,0.57,0.46,1.52,3.18,2.96,1.57,2.86
File2.csv
"","Sepal.Length","Sepal.Width","Petal.Length","Petal.Width"
"1",5.1,3.5,1.4,0.2
"2",4.9,3,1.4,0.2
"3",4.7,3.2,1.3,0.2
"4",4.6,3.1,1.5,0.2
However getline() works only for the first one. In second case it simply returns white space. The function performs similar even if I copy single lines from one file to another (of course adding or removing additional colums) -- the rows from file1 will be always properly read while those from file2 never. I've even tried removing " chars, but without much improvement. However switching from comas to '\t' solves the problem.
I'm curious what's the difference between those two files that makes such different outcome?
The source code of my function:
vector<vector<string>> readData(string fileName,int firstLine,char delimeter){
//Open data file
fstream fin;
fin.open(fileName, ios::in);
//Data stored in 2d vector of strings
vector<vector<string>> data;
vector<string> row;
string line,word,temp;
//Read data
int i=0;
while(fin>>temp){
row.clear();
//Read line and store in 'line'
getline(fin,line);
//Don't read first n lines
if (i<firstLine){
i++;
continue;
}
cout<<line<<endl;
//Break words
stringstream s(line);
//Read every column and store in in 'word;
while(getline(s,word,delimeter)){
row.push_back(word);
}
//Append row to the data vector
data.push_back(row);
}
//Close file
fin.close();
return data;
}
The problem is here:
while(fin>>temp){
row.clear();
//Read line and store in 'line'
getline(fin,line);
fin >> temp reads everything till the first space or newline. It is not clear why you do that as only with getline(fin,line) you then try to read the full line and you are not using temp. In the first file fin>>temp consumes only "User", in the second file it consumes the full line, because there are no spaces.
If you look at the read data from the first file you will also notice that the first part of each line is missing.
Tip: Use more meaningful names for your variables. I didn't manage to fully understand your logic, because variables named s and the presence of row and line at the same time causes me headaces.
I order to simulate my network I am using a trace file (csv file) with a size between 5 to 30 GB.
The csv file is a row based, where each row contains multiple fields delimited by a space and forming teh information to form a network packet:
3 53 4 12 1 1 2 6
Since the file's size could reach several GBs (millions of lines), is it better to divided it into small chunks myfile00.csv, myfile01.csv..., or I can process the entire file on the hard drive without being loaded into the memory?
I want to read the file line by line at a specific time, which is the clock cycle of the simulation, and get all information in the line to create an omnet++ message.
packet MyTrace::getpacket() {
int id; // first field
int cycle; // second field
int source; // third field
int destination; // fourth field
int numberofDep; // fifth field
std::list<int> listofDep; // remaining fields
if (!traceFile.is_open()) {
// get id
// get cycle
// ....
}
Any suggestion would be helpful.
EDIT:
string line;
ifstream myfile ("BlackSmall.csv");
int currentline=0 ;
if (myfile.is_open())
{
while (getline(myfile, line)) {
istringstream ss(line);
string request;
int id, cycle, source , dest, srcType, destType, packetSize, dependency;
int listdep;
std::list<int> dep;
ss >> id;
ss>> cycle;
ss>> source;
ss>> dest;
ss>>request;
ss>> srcType;
ss>> destType;
ss>> packetSize;
ss>> dependency;
while (ss >> listdep) dep.push_back(listdep);
// Create my packet
}
myfile.close();
}
else cout << "Unable to open file";
With the above code, I can get all information that I need from a line.
The problem is that I need to use this code inside a class, which when I call it returns just one line's information. Is there a way how to point to a specific line when I call this class?
It seems like your application seems to require a single sequential pass through the input, so processing a file that is 1GB or 100GB is perhaps just a matter of patience and perhaps parallelism.
The approach should be to translate records line-by-line. You should avoid strategies that attempt to read the entire file into memory. The STL offers the easy-to-use std::ifstream class with a built-in getline method, which returns a std::string containing the line to be converted.
If you are feeling more ambitious and want to control the amount of data read or buffered more carefully then you would not be the first developer to roll-your-own code to implement a buffered reader. This is a fairly empowering exercise and will help you think through some corner cases with reading partial lines and such. But in the end, it probably will not give you a significant boost toward your goal. I suspect the ifstream approach will get you up and running without the hassle and will not ultimately be the bottleneck in processing these files.
If you were really concerned about optimizing execution time then having multiple files might help you launch parallel processing tasks.
// define a class to hold your custom record
class Record {
};
// create a parser function to convert a line of text into the record
bool parse(std::string const &line, Record &record) {
}
// create a translator method to convert a record into the desired output
bool write(Record const &record, std::ofstream &os) {
}
// actually open input stream for the input file
std::ifstream is;
std::ofstream os;
std::string line;
while (std::getline(is,line)) {
Record record;
if (!parse(line,record)) break;
if (!write(record,os)) break;
}
You can re-use the Record instance by moving it outside the while loop so long as you are careful to reset the variable so that information from preceding records does not taint the current record. You can also dive head first into the C++ ecosystem by producing stream input and output operator ("<<",">>") but I personally find this approach to be more confusion than it is worth.
Perhaps best approach for you would be to import your CSV file into SQLite database.
Once you import it and add some indexes, you can easily and very efficiently query necessary rows from that database. SQLite has lots of ready-to-use C/C++ client libraries available, you can start with default one at https://www.sqlite.org/cintro.html.
I just wondering cause i have a text file containing STATUS:USERID:PASSWORD in accounts.txt
example it would look like this:
OPEN:bob:askmehere:
OPEN:john:askmethere:
LOCK:rob:robmypurse:
i have a user input in my main as such user can login 3x else status will change from OPEN to LOCK
example after 3 tries of john
before:
OPEN:bob:askmehere:
OPEN:john:askmethere:
LOCK:rob:robmypurse:
after:
OPEN:bob:askmehere:
LOCK:john:askmethere:
LOCK:rob:robmypurse:
what i have done is:
void lockUser(Accounts& in){
// Accounts class consist 3 attributes (string userid, string pass, status)
ofstream oFile;
fstream iFile;
string openFile="accounts.txt";
string status, userid, garbage;
Accounts toupdate;
oFile.open(openFile);
iFile.open(openFile);
while(!iFile.eof()){
getline(iFile, status, ':');
getline(iFile, userid, ':');
getline(iFile, garbage, '\n');
if(userid == in.getUserId()){
toupdate.setUserId(in.getuserId());
toupdate.setPassword(in.getPassword());
toupdate.setStatus("LOCK");
break;
}
//here i should update the account.txt how do i do that?
ofile.open(openFile);
ofile<<toupdate.getStatus()<<":"<<toupdate.getUserId()":"<<toupdate.getPassword()<<":"<<endl;
}
There are two common ways to replace or otherwise modify a file. The first and the "classic" way is to read the file, line by line, check for the line(s) that needs to be modified, and write to a temporary file. When you reach the end of the input file you close it, and rename the temporary file as the input file.
The other common way is when the file is relatively small, or you have a lot of memory, is to read it all into memory, do the modification needed, and then write out the contents of the memory to the file. How to store it in memory can be different, like a vector containing lines from the file, or a vector (or other buffer) containing all characters from the file without separation.
Your implementation is flawed because you open the output file (which is the same as the input file) inside the loop. The first problem with this is that the operating system may not allow you to open a file for writing if you already have it open for reading, and as you don't check for failure from opening the files you will not know about this. Another problem is if the operating system allows it, then your call to open will truncate the existing file, causing you to loose all but the very first line.
Simple pseudo-ish code to explain
std::ifstream input_file("your_file");
std::vector<std::string> lines;
std::string input;
while (std::getline(input_file, input))
lines.push_back(input);
for (auto& line : lines)
{
if (line_needs_to_be_modified(line))
modify_line_as_needed(line);
}
input_file.close();
std::ofstream output_file("your_file");
for (auto const& line : lines)
output_file << line << '\n';
Use ReadLine and find the line you wanna replace, and use replace to replace the thing you wanna replace. For example write:
string Example = "Text to find";
openFile="C:\\accounts.txt"; // the path of the file
ReadFile(openFile, Example);
OR
#include <fstream>
#include <iostream>
#include <string>
int main() {
ifstream openFile;
string ExampleText = BOB;
openFile("accounts.txt");
openFile >> ExampleText;
openFile.replace(Example, "Hello");
}
How do i find a string in a file? In my code, i want to find the name of a person in the file. and do the actions in the comments. Here is my code:
int main(){
size_t found;
ofstream myfile;
cout << "Enter the name you wish to delete." << endl;
getline(cin, name);
myfile.open("database.dat");
found=myfile.find(name);
if (found!=string::npos){
number = myfile.tellg();
/*Delete current line and next line*/
}
}
Do you want to modify the file, or simply skip those two lines while reading?
Actually, the solution is the same for both, because removing data from the middle of the file requires reading everything after that and rewriting it with an offset (just like removing an element from the middle of an array).
So, read the entire file into memory except for any lines you determine need to be deleted (just skip those). After that, write the surviving lines back to disk. It's probably a good idea to use a temporary file which gets moved to the original name as a final step, so that data isn't destroyed if your process is aborted.
I've been stuck on this issue for a while. What I have here is a loop that will read a text file containing file names. The loop reads these lines one by one, and sets it into memory via the variable sFileName. sFileName is later called upon to load an image, and the program does this one by one as the loop loads it. The user then selects a tag for the image and loads the next one. The tag and the image file name are exported into another text file, named imgresults.txt. Now, the text file with the file names is a few thousand lines. So, in the case that the user has to exit the program and tries to continue later, the loop restarts, instead of leaving off at the point when the program was closed.
I am trying to find a way to have the loop start at that point. So far, I decided to use getline() to count how many lines are currently in imgresults.txt, as that will give the number of images that have already been run through the program. This number is stored in the variable "x". I've been doing a lot of research, but I just cannot find how to set a condition for the while loop to begin at line "x". Do you guys have any suggestions? Also, if you need any clarifications, please ask. I only included the code regarding the loop, as the code for loading the image and such is perfect fine.
int _tmain(int argc, _TCHAR* argv[])
{
int value = 0;
int nCounter = 0;
FILE* fIn = NULL;
char * sLine = new char[MAX_FILENAME_SIZE];
char * sFileName = new char [MAX_FILENAME_SIZE];
char * s = new char [MAX_FILENAME_SIZE];
#define ImgListFileName "path"
#define ImgRepository "path"
if ((fIn = fopen(ImgListFileName,"rt"))==NULL)
{
printf("Failed to open file: %s\n",ImgListFileName);
return nCounter;
}
ifstream imgresults;
imgresults.open ("imgresults.txt");
int x=0;
string line;
while(!imgresults.eof()) {
getline (imgresults, line);
x++;
}
srand (time(NULL));
cout << x;
while(!feof(fIn)){
memset(sLine,0,MAX_FILENAME_SIZE*sizeof(char));
memset(sFileName,0,MAX_FILENAME_SIZE*sizeof(char));
memset(s,0,MAX_FILENAME_SIZE*sizeof(char));
fgets(sLine,MAX_FILENAME_SIZE,fIn);
strncpy(s,sLine,65);
strcat(sLine,"\0");
strcat(sFileName,s);
printf (sFileName);
nCounter++;
}
Thanks in advance!
If you really want to use imgresults.txt as the information on where you should start from the input file, then the best you can do is to have a while loop to read x lines from the input file just before the while loop where you read the input file.
while (x--) {
fgets(sLine, MAX_FILENAME_SIZE, fIn);
}
Better solution would probably be to write state of processing to another file so that you would not have to read input file line by line, but you could immediately seek to a known offset in the file.
Use ifstream::tellg to retrieve and store the current position of the file when the programm was closed.
Use ifstream::seekg to restore that position when the porgramm restarts.
Before you read each line, save the current offsets in your input and output file to a separate file, seeking back to the beginning and overwriting the existing data each time.
When/if you restart, read the offsets from that file, seek to those points, and start working from there.
You can just read lines in parallel from both files and stop when you reach the end of results file. When the loop ends, you have already discarded the file names that were already processed.
ifstream results("results.txt");
ifstream names("names.txt");
if (results && names) {
std::string temp1, temp2;
while (getline(results, temp1) && getline(names, temp2)) ; /* do nothing */
}
if (names) {
// process the rest
}
Not the most efficient solution, but it saves you the hassle of saving offsets. Just make sure that before the very first processing the results file is completely empty (or doesn't exist at all), otherwise this code will skip the first line of names file.