The following code is basically everything I'm doing - opening an XML file, processing it and (trying to) write it back. But writing back fails, every time. I tried to find a solution wrote code, Googled, but got no answer.
xml_parse_result result = doc.load_file("data.xml");
//I checked the value of result, it is equal to status_ok, so the file opened fine.
//...
//some XML processing
//...
bool b = doc.save_file("data.xml"); //b is always false
So, is it like pugi doesn't close the file after taking in the input or what? That doesn't seem to be the case as I can delete the file while the program is running. Does anyone know why my program reads the file but doesn't write the modifications back into it?
Try loading the file from an ifstream. This way you have control over the file, and can be sure when it is closed.
// Initialization code
{
std::ifstream stream("data.xml");
pugi::xml_parse_result result = doc.load(stream);
// Check validity
} // Input stream implicitly destructed and file closed.
// Processing
{
std::ofstream stream("data.xml");
doc.save(stream);
} // Output stream implicitly destructed and file closed.
As to why this happens... The Documentation isn't explicit about it, so it's hard to tell. It seems that it should close the file after loading, but the only way to be sure is by looking at the source code. BTW, if you're on a linux OS, you should be able to delete opened files.
Related
Consider the following code,
auto fin = ifstream("address", ios::binary);
if(fin.is_open())
fin.close()
for(auto i = 0; i < N; ++i){
fin.open()
// ....
// read (next) b bytes...
// ....
fin.close()
// Some delay
}
The code above can't be implemented in the C++ I know, but I'd like to know if it is possible to do so?
Here are my requirements:
When reopening the file, there would be no need to pass the parameters (path and mode) again.
When reopening the stream, it continues from the point in the stream that it was when got closed.
Clarification
The files I work with are big in size and in a point of time other threads from third party libraries may decide to (re)move them. An open stream will prevent such actions.
Continuously reading a big file will slow down the system.
The need
Indeed, a file can't be deleted by another process as long as a stream keeps it open.
I suppose you have already asked yourself these questions, but fo the recors I have to suggest you to think about it:
Can't the file be read into (virtual) memory and discarded when no longer needed ?
Can't the file processing be pipelined asynchronously, to read it at once and process it without unnecessary delays ?
What to do if the file can no longer be opened because it was deleted by the other process ? What to do if the location can't be found, because the file was modified (e.g. shortened) ?
If you would have the perfect solution to your issue, what would be the effect if the other process would try to delete the file when it is open (only for a short time, but nevertheless open and blocking the deletion) ?
The solution
Unfortunately, you can't achieve the desired behavior with standard streams. You could emulate it by keeping track of the filename and of the position (and more generally of the state):
auto mypos = ifs.tellg(); // saves position.
// Should flag be saved as well ? and what about gcount ?
ifs.close();
...
if (! ifs.is_open()) {
ifs.open(myfilename, myflags); // open again !
if (! ifs) {
// ouch ! file disapeared ==> process error
}
ifs.seekg(mypos); // restore position
if (! ifs) {
// ouch ! position no longer reachable ==> process error
}
}
Of course, you wouldn't like to repeat this code ever and ever. And it would not be so nice having all the sudden a lot of global variables to keep track of the stream's state. But you could very easily encapsulate it in a wrapper class that would take care of saving and restoring the stream's state using existing standard operations.
I have been scratching my head around this problem for a while, and I couldn't find any answer through surfing the web either.
The problem is that I call system("csvtojson someFile.csv 1> someOtherFile.json") inside my program to produce a JSON file. After this line I want to open, read, and process the JSON file. Although, I can see that the file is created, but fopen() returns NULL.
I read that system() is synchronized so I think the rest of my program will not get executed until the system call is finished, and so the file will be created.
I suspect the problem is somehow related to redirecting the output stream using "1>"; not sure, though.
Any help or hint will be much appreciated.
Thanks! :)
P.S. I don't want to use a library to convert csv to JSON, and I can't perform the conversion outside the program because there are tons of very large csv files and the only way for me is to convert each to a JSON file inside the program, run my algorithm, and move to the next csv file ( converting it to JSON and saving it in the very same JSON file). So in total I have only one JSON file, being like a buffer for my csv files. Having said that, if anyone has a better design approach that can be implemented quickly, that would be also great.
UPDATE : Actual code that exhibits the problem, copied from the OP's answer:
int main(){
system("csvtojson Test_Trace.csv 1> ~/Traces/Test_Trace.json");
FILE* traceFile = fopen("~/Traces/Test_Trace.json", "r");
if(traceFile == NULL)
perror("Error in Openning the trace file");
else
cout << "Successfull openning of the trace file!" << endl;
return 0;
}
Thank you guys for your answers. I had to be more detailed in my question as the problem seemed to be somewhere that wasn't clear from my question.
I figured out what was the problem, and would like to share it here (not a super interesting finding, but worth mentioning).
I wrote a simple program to find the problem:
int main(){
system("csvtojson Test_Trace.csv 1> ~/Traces/Test_Trace.json");
FILE* traceFile = fopen("~/Traces/Test_Trace.json", "r");
if(traceFile == NULL)
perror("Error in Openning the trace file");
else
cout << "Successfull openning of the trace file!" << endl;
return 0;
}
If you run this program you will get the error message No such file or directory, but if you replace the address string with the absolute location, i.e., /home/USER_ID/Traces/Test_Trace.json, in both system(...) and fopen(...) calls, your code will work fine. Interestingly, myself suspected that this could be the problem and I changed just the one for system(...) but still it wasn't working (though the file was being created in the location that was passed to fopen(...)).
EDIT: Thanks to #Peter's comment, this problem was because system() call takes care of ~, but fopen() does not and need an absolute path. So there is really no need to have both functions been given the absolute path.
Anyhow,
Thanks Again. :)
Perhaps the reason for this is because the system command hasn't finished executing by the time your program continues to the next instructions where it tries to read from the file that hasn't been created yet.
Although, this isn't the best way, putting in a short pause might make the difference, or at least let you know if that is the issue.
My program is always writing data to a file but when I close it before the program fully stops, the end result is nothing being written to the file. I would really like to be able to close it without completing it fully, so how can I fix this to make it constantly saving the file?
ofstream outfile;
outfile.open("text.txt", std::ios::app);
bool done = false;
int info;
while (done == false){
cin>>info;
outfile<<info;
cout<<info<<"Choose different info";
if(info == 100){
done = true;
}
}
outfile.close();
This is obviously just an example, but it is very similar to my actual code.
Edit: When i say closing I mean killing it (Hitting red X at top right of console)
You likely need to flush your std::ofstream when you have done "enough" work.
"enough" work here is going to depend on your application.
Perhaps
...
outfile<<info;
outfile.flush();
...
The operation system doesn't write to the file when you call the write function to save time, it wait to check if you want to write anything else or for a time which will be "good" to write. You write to a buffer and the operating system will write this buffer to the file.
When you close the function it write anything that left in the buffer to the file. You can force your code to write to the file using flush method. Just flush your file after every time you write and you will be ok.
flush: http://www.cplusplus.com/reference/ostream/ostream/flush/
outfile << n;
outfile.flush();
I have a C++ program that creates an output file "A" with ofstream. This file is then read by some legacy C code that opens the file with _iobuf. The legacy code then creates its own output file "B" using _iobuf, and this file is then read by the C++ program using ifstream. This sequence is iterated many times, with the same file names for A and B in each iteration.
Occasionally, the C++ program cannot open the output file A for writing, and I must try several times before it succeeds. This occurs nondeterministically, and maybe once in a thousand iterations. Note that the C program never has to wait to open its input or output file, nor does the C++ program ever have to wait to open its input file. This informal observation is based on hundreds of thousands of iterations.
I'm wondering if this has something to do with mixing ofstream and _iobuf in the same program? Both the C++ code and the C code are linked into the same program. And the legacy C code is technically C++ code, but was written in a very C-like style. Is there anything I can do to eliminate this waiting to open the ofstream file? I do not want to change the legacy code if I can possibly avoid it.
Pseudo code (not compiled):
void someObject::someMethod()
{
for (int count = 0; count < someLimit; ++count)
{
newerObject::firstMethod();
olderObject::secondMethod();
newerObject::thirdMethod();
}
}
void newerObject::firstMethod()
{
// do some processing first
// then write the results of the processing to a file
ofstream A;
A.open("A", ofstream::out); // this sometimes must be tried multiple times
// write data to file A
A.close();
}
void olderObject::secondMethod()
{
FILE* f;
f = fopen("A", "rt"); // this always works the first time
// read data from file A
fclose(f);
// do some processing
f = fopen("B", "w");
// write data to file B
fclose(f);
}
void newerObject::thirdMethod()
{
ifstream B;
B.open("B"); // this always works the first time
// read data from file B
B.close();
// do some processing
}
Currently, as a work around, I put the ofstream::open in a do-while loop. I would love to get rid of this awkwardness. Thanks in advance for any advice you can give.
First off, the problem is almost certainly not the use of different methods to access the files: under the hood, the C and C++ I/O functions use the same system I/O facilities. You seem to be using Windows (on other systems files typically can be open multiple times simultaneously) and I don't know much about the system but I would suspect that the file system hasn't been updated to reflect that the file is closed when you try to open it. This may have to do with the "t" open flag: I don't know what this is about.
On UNIXes you can force the I/O operations to wait until the actual change on disk completed. Something like this could help avoiding the problem but has the significant cost that operations become hideously slow. On UNIXes one approach would be to blow away the file system entry the moment the file was opened successfully (after all, at this point its name isn't used anymore):
if (FILE* fp = fopen("file", "r")) {
remove("file");
// do processing
}
However, if I recall correctly on Windows you can neither remove the file nor rename it. Personally, in solving the problem I would proceed as follows:
Determine under which situations the file can't be opened, e.g. by keeping the file open and trying to open it. This is mainly intended to create a setup where the problem is reproducible so you can verify later that you indeed found a solution.
Once I found a way to reproduce the problem I would probably a better idea of the actual root cause and possibly googling would help. In any case this is the point where researching the root cause comes in.
Once the cause is understood it is hopefully easy to devise a solution. If not, opening the file multiple times under it is successful may very well be the right solution.
void bot_manager_item::create_games()
{
games.clear();
std::ifstream paths_in("C:\\Users\\bill hank\\Documents\\bot_plugins\\directory_listing.txt", std::ios::in);
while (paths_in.good())
{
send_message("The path was good.");
char q[5000];
paths_in.getline(q, 5000);
send_message(q);
games.push_back(qanda(q));
}
paths_in.close();
}
The file I'm loading exists, what else might be wrong? paths_in.good keeps failing.
Edit: I figured it out. Wow am I annoyed by the answer to this. Basically Windows lets you say whether you want to show file extensions or not. This windows installation is set to say that the extension shouldn't be shown. So when I'm checking the file again and again I'm seeing: directory.txt and thinking that this means that everything is fine with the directory when in reality the filename was directory.txt.txt
If paths_in.good() keeps failing then it means that some of the stream error flags are set (badbit, eofbit or failbit).
eofbit - end of file was reached
badbit - error with the stream buffer such as memory shortage or an exception inside the stream buffer is cast
failbit - some other error beside eof was reached
In order to find out what happened, you need to check which errorbit is set first, and then find out more about the specific error, and what can cause it.
Out of curiosity, does this code output the contents of the file correctly? If this code works, then the problem is something else. If this code doesn't work, then that likely means that the file either isn't where you specified, or you don't have read permissions on it.
void bot_manager_item::create_games() {
std::ifstream paths_in("C:\\Users\\bill hank\\Documents\\bot_plugins\\directory_listing.txt");
char q[5000];
while (paths_in.getline(q, 5000)) {
std::cout << q << std::endl;
}
}
This code does a few minor things differently.
std::ios::in doesn't need to be explicitly specified for std::ifstream.
it doesn't use is_good, while that should be fine, you can just treat the std::ifstream as a bool which will be true when it is in a good state.
getline() returns a reference to the stream it operated on, so you can just put that whole line in the condition.
cosmetic, but no need to explicitly close the ifstream if it is about to go out of scope.