this is my c++ code
const int num_of_file = 1024;
std::ifstream data("data.txt");
std::vector<std::ofstream> files(num_of_file);
for (int i = 0; i < num_of_file; ++i)
{
files[i].open(std::to_string(i) + ".txt");
if (files[i].is_open() == false)
{
std::cerr << "open " << std::to_string(i) << ".txt fail" << std::endl;
exit(0);
}
}
but i received "open 509.txt fail" when i run the code every time.
After a little bit of research the limitation seems to stem from the stream.
stream is built on top of C "streams" (fopen, fread, etc), and those functions use collection of shared tables of "file handles", those tables having a maximum size that's "burned into" the VC++ runtime library. I'm a bit surprised that you're hitting the limit at 509 files - near as I can tell, the limit should be closer to 2048 files - but I'd bet that's the limit you're hitting.
You should keep an internal buffer and after a certain limit has been reached open, write and close the file.
Related
I'm working on a project that requires me to get a touchscreen working on Scientific Linux 6.4 (Linux kernel 2.6.32). Although the kernel does not support the touchscreen fully, I am able to see multi-touch events being generated in the /dev/input/eventX location for the touchscreen when I touch the screen.
I'm trying to write a simple C++ program to read the data from the /dev/input/eventX file and parse it so I can manually deal with the multi-touch events, since that seems the only way I'll get this working.
So I wrote the following program:
std::ifstream input("/dev/input/event10");
if(input.is_open()) {
while(input.good()) {
int header;
input >> header;
cout << std::hex << header << " ";
int data[16] = {};
for(int i = 0; i < 16; i++) {
input >> data[i];
cout << std::hex << data[i] << " ";
}
cout << endl;
}
input.close();
} else cout << "Unable to open event handler for input polling..." << endl;
Now, I don't exactly know if my method of reading and parsing the input itself is correct, but when I use the following command in bash:
sudo cat /dev/input/event10 | hexdump -C
I get the input data in the form of a number of lines beginning with an 8-digit hex value followed by 16 2-digit hex values (bytes).
The problem I'm having though is that I always get the message "Unable to open event handler for input polling..." suggesting an issue with opening the file. At first, I thought maybe that because nothing is in that file until an event is generated, it might not be able to be opened as an ifstream. I also tried running the program as sudo just in case it was a permissions issue and got the same message, so I believe it has to do with how I'm opening the file.
Does anyone know the proper way to open and read from these files?
EDIT: My question is regarding why the file is unable to be opened, not necessarily just how to parse the data. The suggested "duplicate" questions don't provide any helpful information in this regard.
Nevermind... there was a trailing space in my filename (which was detected rather than hard-coded). I added a trim function and now it opens just fine.
I'm writing a program which fetches a large number of email files using libcurl and then writes the file to disk, and then generates a receipt.
My problem is that, whilst most of the receipts seem to get written, the majority of the emails aren't written to disk. Worse, even though the file doesn't get written, ofstream returns success - so the receipt gets written even if the file write didn't complete successfully.
My guess is that, because ofstream is asynchronous, if a write doesn't complete in time then it'll get dropped on the floor - only a certain number of writes being possible concurrently. I am just guessing here.
Perhaps I need to refactor my code to write synchronously - but I can't believe that that's necessary. Does anyone have any idea how I can make this work?
The email sizes range from a few KBytes to a couple of MBytes.
int write_file(string filename, string mail_item) {
ofstream out(filename.c_str());
out << mail_item;
out.close();
out.flush();
if (!out) {
return FUNCTION_FAILED;
}
return FUNCTION_SUCCESS;
}
This is part of another function, and has been cut out so that only the salient code for this question is shown.
vector<string> directory = curl_listroot(curl);
for (int i=0; i<directory.size(); i++) {
vector<int> mail_list = curl_search(curl,directory[i],make_vector<string>() << "SEEN" << "RECENT" << "NEW" << "ANSWERED" << "FLAGGED");
for (int j=0; j<mail_list.size(); j++) {
curl_reset(curl, imap.username, imap.password);
string mail_item = curl_fetch(curl,directory[i],mail_list[j]);
if (mail_item.compare("") != 0) {
string m_id = getMessageID(mail_item);
string filename = save_path+"/"+RECEIPTNAME+"/"+clean_filename(m_id) + ".eml";
if (!file_exists(filename)) {
string real_filename;
real_filename = save_path+"/"+INBOXNAME+"/"+clean_filename(m_id) + ".eml";
int success = write_file(real_filename, mail_item);
if (success == FUNCTION_SUCCESS) {
write_file(filename, ""); //write empty receipt
}
}
}
}
}
All suggestions gratefully received! Thank you!
Okay. I've found an answer - there may be better answers - but this one works for me. The problem seems to be in the OS (Linux, in this case) - ofstream completes, having handed the responsibility for writing the file to the OS, but the file hasn't actually been written yet (so whilst ofstream may be synchronous the end to end write of the file, from data to file safely written to disk, isn't). Given that I'm banging away with a huge number of writes in quick succession (potentially thousands), this won't necessarily work. The OS may throw its hands in the air and drop a significant number of the files writes on the floor (hence my original request for a synchronous way of writing the files - end to end).
My solution is to pause after each write to give the OS time to catch up. It's inelegant though, and not as performant as it should be - it doesn't take half a second to write an empty file. Additionally, on slow storage, half a second might not be enough time. I'd welcome any clever suggestions for how to improve my code.
int write_file(string filename, string mail_item) {
ofstream out(filename.c_str());
if (!out) {
return FUNCTION_FAILED;
}
out << mail_item << endl;
out.flush();
usleep(500000); //wait for half a second to give the OS time to output the file
if (!out) {
return FUNCTION_FAILED;
}
out.close();
if (!out) {
return FUNCTION_FAILED;
}
return FUNCTION_SUCCESS;
}
As part of a bigger application I am working on a class for reading input from a text file for use in the initialization of the program. Now I am myself fairly new to programming, and I only started to learn C++ in December, so I would be very grateful for some hints and ideas on how to get started! I apologise in advance for a rather long wall of text.
The text file format is "keyword-driven" in the following way:
There are a rather small number of main/section keywords (currently 8) that need to be written in a given order. Some of them are optional, but if they are included they should adhere to the given ordering.
Example:
Suppose there are 3 potential keywords ordered like as follows:
"KEY1" (required)
"KEY2" (optional)
"KEY3" (required)
If the input file only includes the required ones, the ordering should be:
"KEY1"
"KEY3"
Otherwise it should be:
"KEY1"
"KEY2"
"KEY3"
If all the required keywords are present, and the total ordering is ok, the program should proceed by reading each section in the sequence given by the ordering.
Each section will include a (possibly large) amount of subkeywords, some of which are optional and some of which are not, but here the order does NOT matter.
Lines starting with characters '*' or '--' signify commented lines, and they should be ignored (as well as empty lines).
A line containing a keyword should (preferably) include nothing else than the keyword. At the very least, the keyword must be the first word appearing there.
I have already implemented parts of the framework, but I feel my approach so far has been rather ad-hoc. Currently I have manually created one method per section/main keyword , and the first task of the program is to scan the file for to locate these keywords and pass the necessary information on to the methods.
I first scan through the file using an std::ifstream object, removing empty and/or commented lines and storing the remaining lines in an object of type std::vector<std::string>.
Do you think this is an ok approach?
Moreover, I store the indices where each of the keywords start and stop (in two integer arrays) in this vector. This is the input to the above-mentioned methods, and it would look something like this:
bool readMAINKEY(int start, int stop);
Now I have already done this, and even though I do not find it very elegant, I guess I can keep it for the time being.
However, I feel that I need a better approach for handling the reading inside of each section, and my main issue is how should I store the keywords here? Should they be stored as arrays within a local namespace in the input class or maybe as static variables in the class? Or should they be defined locally inside relevant functions? Should I use enums? The questions are many!
Now I've started by defining the sub-keywords locally inside each readMAINKEY() method, but I found this to be less than optimal. Ideally I want to reuse as much code as possible inside each of these methods, calling upon a common readSECTION() method, and my current approach seems to lead to much code duplication and potential for error in programming. I guess the smartest thing to do would simply be to remove all the (currently 8) different readMAINKEY() methods, and use the same function for handling all kinds of keywords. There is also the possibility for having sub-sub-keywords etc. as well (i.e. a more general nested approach), so I think maybe this is the way to go, but I am unsure on how it would be best to implement it?
Once I've processed a keyword at the "bottom level", the program will expect a particular format of the following lines depending on the actual keyword. In principle each keyword will be handled differently, but here there is also potential for some code reuse by defining different "types" of keywords depending on what the program expects to do after triggering the reading of it. Common task include e.g. parsing an integer or a double array, but in principle it could be anything!
If a keyword for some reason cannot be correctly processed, the program should attempt as far as possible to use default values instead of terminating the program (if reasonable), but an error message should be written to a logfile. For optional keywords, default values will of course also be used.
In order to summarise, therefore, my main questions are the following:
1. Do you think think my approach of storing the relevant lines in a std::vector<std::string> to be reasonable?
This will of course require me to do a lot of "indexing work" to keep track of where in the vector the different keywords are located. Or should I work more "directly" with the original std::ifstream object? Or something else?
2. Given such a vector storing the lines of the text file, how I can I best go about detecting the keywords and start reading the information following them?
Here I will need to take account of possible ordering and whether a keyword is required or not. Also, I need to check if the lines following each "bottom level" keyword is in the format expected in each case.
One idea I've had is to store the keywords in different containers depending on whether they are optional or not (or maybe use object(s) of type std::map<std::string,bool>), and then remove them from the container(s) if correctly processed, but I am not sure exactly how I should go about it..
I guess there is really a thousand different ways one could answer these questions, but I would be grateful if someone more experienced could share some ideas on how to proceed. Is there e.g. a "standard" way of doing such things? Of course, a lot of details will also depend on the concrete application, but I think the general format indicated here can be used in a lot of different applications without a lot of tinkering if programmed in a good way!
UPDATE
Ok, so let my try to be more concrete. My current application is supposed to be a reservoir simulator, so as part of the input I need information about the grid/mesh, about rock and fluid properties, about wells/boundary conditions throughout the simulation and so on. At the moment I've been thinking about using (almost) the same set-up as the commercial Eclipse simulator when it comes to input, for details see
http://petrofaq.org/wiki/Eclipse_Input_Data.
However, I will probably change things a bit, so nothing is set in stone. Also, I am interested in making a more general "KeywordReader" class that with slight modifications can be adapted for use in other applications as well, at least it can be done in a reasonable amount of time.
As an example, I can post the current code that does the initial scan of the text file and locates the positions of the main keywords. As I said, I don't really like my solution very much, but it seems to work for what it needs to do.
At the top of the .cpp file I have the following namespace:
//Keywords used for reading input:
namespace KEYWORDS{
/*
* Main keywords and corresponding boolean values to signify whether or not they are required as input.
*/
enum MKEY{RUNSPEC = 0, GRID = 1, EDIT = 2, PROPS = 3, REGIONS = 4, SOLUTION = 5, SUMMARY =6, SCHEDULE = 7};
std::string mainKeywords[] = {std::string("RUNSPEC"), std::string("GRID"), std::string("EDIT"), std::string("PROPS"),
std::string("REGIONS"), std::string("SOLUTION"), std::string("SUMMARY"), std::string("SCHEDULE")};
bool required[] = {true,true,false,true,false,true,false,true};
const int n_key = 8;
}//end KEYWORDS namespace
Then further down I have the following function. I am not sure how understandable it is though..
bool InputReader::scanForMainKeywords(){
logfile << "Opening file.." << std::endl;
std::ifstream infile(filename);
//Test if file was opened. If not, write error message:
if(!infile.is_open()){
logfile << "ERROR: Could not open file! Unable to proceed!" << std::endl;
std::cout << "ERROR: Could not open file! Unable to proceed!" << std::endl;
return false;
}
else{
logfile << "Scanning for main keywords..." << std::endl;
int nkey = KEYWORDS::n_key;
//Initially no keywords have been found:
startIndex = std::vector<int>(nkey, -1);
stopIndex = std::vector<int>(nkey, -1);
//Variable used to control that the keywords are written in the correct order:
int foundIndex = -1;
//STATISTICS:
int lineCount = 0;//number of non-comment lines in text file
int commentCount = 0;//number of commented lines in text file
int emptyCount = 0;//number of empty lines in text file
//Create lines vector:
lines = std::vector<std::string>();
//Remove comments and empty lines from text file and store the result in the variable file_lines:
std::string str;
while(std::getline(infile,str)){
if(str.size()>=1 && str.at(0)=='*'){
commentCount++;
}
else if(str.size()>=2 && str.at(0)=='-' && str.at(1)=='-'){
commentCount++;
}
else if(str.size()==0){
emptyCount++;
}
else{
//Found a non-empty, non-comment line.
lines.push_back(str);//store in std::vector
//Start by checking if the first word of the line is one of the main keywords. If so, store the location of the keyword:
std::string fw = IO::getFirstWord(str);
for(int i=0;i<nkey;i++){
if(fw.compare(KEYWORDS::mainKeywords[i])==0){
if(i > foundIndex){
//Found a valid keyword!
foundIndex = i;
startIndex[i] = lineCount;//store where the keyword was found!
//logfile << "Keyword " << fw << " found at line " << lineCount << " in lines array!" << std::endl;
//std::cout << "Keyword " << fw << " found at line " << lineCount << " in lines array!" << std::endl;
break;//fw cannot equal several different keywords at the same time!
}
else{
//we have found a keyword, but in the wrong order... Terminate program:
std::cout << "ERROR: Keywords have been entered in the wrong order or been repeated! Cannot continue initialisation!" << std::endl;
logfile << "ERROR: Keywords have been entered in the wrong order or been repeated! Cannot continue initialisation!" << std::endl;
return false;
}
}
}//end for loop
lineCount++;
}//end else (found non-comment, non-empty line)
}//end while (reading ifstream)
logfile << "\n";
logfile << "FILE STATISTICS:" << std::endl;
logfile << "Number of commented lines: " << commentCount << std::endl;
logfile << "Number of non-commented lines: " << lineCount << std::endl;
logfile << "Number of empty lines: " << emptyCount << std::endl;
logfile << "\n";
/*
Print lines vector to screen:
for(int i=0;i<lines.size();i++){
std:: cout << "Line nr. " << i << " : " << lines[i] << std::endl;
}*/
/*
* So far, no keywords have been entered in the wrong order, but have all the necessary ones been found?
* Otherwise return false.
*/
for(int i=0;i<nkey;i++){
if(KEYWORDS::required[i] && startIndex[i] == -1){
logfile << "ERROR: Incorrect input of required keywords! At least " << KEYWORDS::mainKeywords[i] << " is missing!" << std::endl;;
logfile << "Cannot proceed with initialisation!" << std::endl;
std::cout << "ERROR: Incorrect input of required keywords! At least " << KEYWORDS::mainKeywords[i] << " is missing!" << std::endl;
std::cout << "Cannot proceed with initialisation!" << std::endl;
return false;
}
}
//If everything is in order, we also initialise the stopIndex array correctly:
int counter = 0;
//Find first existing keyword:
while(counter < nkey && startIndex[counter] == -1){
//Keyword doesn't exist. Leave stopindex at -1!
counter++;
}
//Store stop index of each keyword:
while(counter<nkey){
int offset = 1;
//Find next existing keyword:
while(counter+offset < nkey && startIndex[counter+offset] == -1){
offset++;
}
if(counter+offset < nkey){
stopIndex[counter] = startIndex[counter+offset]-1;
}
else{
//reached the end of array!
stopIndex[counter] = lines.size()-1;
}
counter += offset;
}//end while
/*
//Print out start/stop-index arrays to screen:
for(int i=0;i<nkey;i++){
std::cout << "Start index of " << KEYWORDS::mainKeywords[i] << " is : " << startIndex[i] << std::endl;
std::cout << "Stop index of " << KEYWORDS::mainKeywords[i] << " is : " << stopIndex[i] << std::endl;
}
*/
return true;
}//end else (file opened properly)
}//end scanForMainKeywords()
You say your purpose is to read initialization data from a text file.
Seems you need to parse (syntax analyze) this file and store the data under the right keys.
If the syntax is fixed and each construction starts with a keyword, you could write a recursive descent (LL1) parser creating a tree (each node is a stl vector of sub-branches) to store your data.
If the syntax is free, you might pick JSON or XML and use an existing parsing library.
I have a program that reads a set of files, closes them, and then attempt to delete them.
Sometimes (not always, but pretty often) the delete fails with 'sharing violation' error.
Using sysinternals process monitor I saw that in these cases the close operation wasn't reflected in the process monitor.
It appears that sometimes the close system call is skipped for no apparent reason, and without any exception.
This is happening on a windows 7 64bit machine using visual studio 2010.
Code sample;
void readFile(string file)
{
ifstream stream(file);
string line;
while(getline(stream, line))
{
cout << line << endl:
}
stream.close(); // this is redundant
}
// calling code:
readFile(file);
if(remove(file.c_str()) != 0)
{
cout << "file deletion failed" << endl;
}
Firsty your code is lacking a ;. Change this cout << line << endl: to this cout << line << endl;
Here's a similar problem : Any reason why an std::ofstream object won't close properly?
This could happen if you are creating processes in-between using CreateProcess with bInheritHandles=true. The new process will inherit the file handle and the file won't be closed by your main process as there is still an outstanding handle. This may explain why you can't see the close operation in Process Monitor, the OS will close the file once all handles have been released.
I'm trying to to the most basic of things .... write a file in C++, but the file is not being written. I don't get any errors either. Maybe I'm missing something obvious ... or what?
I thought there was something wrong with my code, but I also tried a sample I found on the net and still no file is created.
This is the code:
ofstream myfile;
myfile.open ("C:\\Users\\Thorgeir\\Documents\\test.txt");
myfile << "Writing this to a file.\n";
myfile.close();
I've also tried creating the file manually beforehand, but it's not updated at all.
I'm running Windows 7 64bit if that has got something to do with this. It's like file-write operations are completely forbidden and no error messages or exceptions are shown.
You need to open the file in write mode:
myfile.open ("C:\\Users\\Thorgeir\\Documents\\test.txt", ios::out);
Make sure to look at the other options for that second argument, as well. If you're writing binary data you'll need ios::binary for example.
You should also be checking the stream after opening it:
myfile.open(...
if (myfile.is_open())
...
Update:
AraK is right, I forgot that an ofstream is in write mode by default, so that's not the problem.
Perhaps you simply don't have write/create permissions to the directory? Win7 defaults a lot of directories with special permissions of "deny all". Or perhaps that file already exists and is read-only?
Start off by turning that slash around.
Even Windows understands the slash being the other way around.
ofstream myfile("C:/Users/Thorgeir/Documents/test.txt");
You could test if there are any errors:
if (!myfile)
{
std::cout << "Somthing failed while opening the file\n";
}
else
{
myfile << "Writing this to a file.\n";
myfile.close();
}
Make sure the directory exists.
If the file exists make sure it is writeable (by you)
Check the directory you are writing into is writeable (by you)
Have you read about UAC (User Account Control) and UAC Virtualization / Data Redirection in Windows Vista and 7? It's possible that your file is actually in the Virtual Store.
User Account Control Data Redirection
Your example output directory is in Users, so I wouldn't think this would be the issue, but it's a possibility worth mentioning and something that can be very frustrating if you're not looking out for it!
Hope this helps.
This code should catch any error. Most likely it's a permissions thing if any errors are encountered. Make sure you can read/write to the folder you're creating the file in.
#include "stdafx.h"
#include <fstream>
#include <iostream>
bool CheckStreamErrorBits(const std::ofstream& ofile);
int _tmain(int argc, _TCHAR* argv[]) {
std::ofstream ofile("c:\\test.txt");
if(ofile.is_open()) {
CheckStreamErrorBits(ofile);
ofile << "this is a test" << std::endl;
if(CheckStreamErrorBits(ofile)) {
std::cout << "successfully wrote file" << std::endl;
}
}else {
CheckStreamErrorBits(ofile);
std::cerr << "failed to open file" << std::endl;
}
ofile.close();
return 0;
}
//return true if stream is ok. return false if stream has error.
bool CheckStreamErrorBits(const std::ofstream& ofile) {
bool bError=false;
if(ofile.bad()) {
std::cerr << "error in file stream, the bad bit is set" << std::endl;
bError=true;
}else if(ofile.fail()) {
std::cerr << "error in file stream, the fail bit is set" << std::endl;
bError=true;
}else if(ofile.eof()) {
std::cerr << "error in file stream, the eof bit is set" << std::endl;
bError=true;
}
return !bError;
}
Update:
I just test my code under Windows 7 Enterprize and it failed the first time (fail bit was set). Then I turn off User Account Control (UAC) and tested again and it wrote the file. That is probably the same problem you're seeing. To turn off UAC go to:
Control Panel (view by Small icons) | User Accounts | Change User Account Control settings. Set it to Never notify then click OK button. You will have to restart for the changes to take affect.
I'm curious how to make it work with UAC on, i'll look into that.
Try this:
if( ! myfile)
{
cerr << "You have failed to open the file\n";
//find the error code and look up what it means.
}
Use FileMon and look for failed WriteFile calls from your process.