I'm saving a file on an USB drive and need to make sure that it's completely written to avoid corruption in case the USB drive is not removed properly.
Well I've done some research and it seems this is possible via calling the FlushFileBuffers Win32 function.
But the problem is, I'm saving using boost::serialization and thus don't have access to the actual file HANDLE.
I wonder what is the proper way to flush the file? Thanks!
Call ostream::flush on the output stream you created your archive object with:
// create and open a character archive for output
std::ofstream ofs("filename");
boost::archive::text_oarchive oa(ofs);
...
ofs.flush();
You could also just let the objects go out of scope which should flush everything:
{
// create and open a character archive for output
std::ofstream ofs("filename");
boost::archive::text_oarchive oa(ofs);
// going out of scope flushes the data
}
Note, you still need to properly unmount your USB device. Flushing the data just makes sure it gets from userland into the kernel, but the kernel can also do it's own buffering.
Related
(My previous questions was closed as a duplicate of Are there binary memory streams in C++ which is ridiculous, since i can't change the implementation of the library I'm using)
I'm using a library (Poco) to create zip files. It takes ostream as an input and writes the data of the zip file into it. Something like:
std::ofstream ofs("file.zip", std::ios::binary);
Compress compress(ofs);
// add data to compress ...
compress.close();
// now file.zip contains added file
This works. But I want to be able to create a zip in memory without creating a file. I tried using stringstream instead of ofstream, i get additionl newline characters in the data in the zip file is corrupted. Is there any other stream i can use?
(If someone still thinks it's a duplicate, I'm gonna need an explanation, since I don't see how this other question is helpful for me)
Use a std::stringstream -- that will create an in-memory string that you can write to as an ostream, and WILL NOT add extra newlines. If you later copy the string to an fstream that was opened in text mode (such as std::cout), then that process may add extra CR characters that are not in the string (nor in the original output).
If you are seeing extra characters corrupting your stream, they are coming from somewhere else -- something besides you compress call/lib is writing to the stream, or something with how you are looking at your stream is doing something.
If you're on linux, how about creating an anonymous file using memfd_create? You can then open /proc/self/fd/<fd> and do your stuff. Some implementations of std::ofstream may even provide a constructor that takes a FILE*, you can check if that's the case on your system.
I'm writing a mini database by using C++. In log module, I use fstream to append data to log file and I need data to be immediately written to disk. In http://www.cplusplus.com/doc/tutorial/files/, I find
When the buffer is flushed, all the data contained in it is written to the physical medium.
But in http://www.cplusplus.com/forum/general/7343/
I've tried the flush() call, but that just flushes the program's buffers out into the file system's buffers, and does not guarantee that the data is physically written to disk.
I'm confused if the flush() or sync() call guarantee the data has been written to the disk, for example
std::fstream file;
file.open("...", std::ios::out | std::ios::app); // open a file
file << "..."; //append log
file.flush(); //or file.sync()
//file.flush() has returned
//system crashed(not only program crashed) or an interruption in the supply of electricity heppened
then when I restart the computer, whether the log is lost? If log will lost, does fstream have any way to force data write to the disk?
I am attempting to serialize some EEG data from a command line application using the Boost library for the serialization and sending that serialized data over a named pipe to a user interface Form built in Visual Studio C++ 2010.
From the boost library tutorial I am able to serialize my data structure, and, http://www.boost.org/doc/libs/1_48_0/libs/serialization/doc/tutorial.html#simplecase
from this tutorial on Win32 named pipes, I can construct pipes and send text between applications.
http://avid-insight.co.uk/joomla/component/k2/item/589-introduction-to-win32-named-pipes-cpp?tmpl=component&print=1
The boost library tutorial serializes for a text file:
std::ofstream ofs("filename");
// create class instance
const gps_position g(35, 59, 24.567f);
// save data to archive
{
boost::archive::text_oarchive oa(ofs);
// write class instance to archive
oa << g;
// archive and stream closed when destructors are called
}
I want to know what do I need to serialize to in order to send my data structure over a named pipe? The IOstream c++ library seems to always need a file to stream to/from?http://www.cplusplus.com/reference/iostream/
I don't want o serialize to a file and I am not sure what to serialize to? I would really appreciate it if you could tell me what I need to serialize to and it would be great if you could tell me whether another boost command will be required other than boost::archive::text_oarchive , as I have been unable to find an alternative.
Thank you for your time! It is really appreciated!
(This question has been asked before: Serialize and send a data structure using Boost? , but the person was told not to use boost as for his simple data structure boost would have too much overhead, so it really is still floating.)
Thanks ForEveR, very simple, don't know how I missed it! :) The solution in context with the two tutorials posted above:
const EEG_Info g(35, 59, 24.567f);
std::stringstream MyStringStream ;
boost::archive::text_oarchive oa(MyStringStream);
// write class instance to archive
oa << g;
// This call blocks until a client process reads all the data
string strData;
strData = MyStringStream.str();
DWORD numBytesWritten = 0;
result = WriteFile(
pipe, // handle to our outbound pipe
strData.c_str(), // data to send
strData.length(), // length of data to send (bytes)
&numBytesWritten, // will store actual amount of data sent
NULL // not using overlapped IO
);
When I construct an iostream when say opening a file will this always read the entire file from the hard disk and then put it into memory, or is it streamed in and buffered by the OS on demand?
I ask because one way to check if a file exists is to see if opening it fails, but I fear if the files I am opening are very large then this take a long time if iostream must read the entire file in on open.
To check whether a file exists can be done like this if you want to use boost.
#include <boost/filesystem.hpp>
bool fileExists = boost::filesystem::exists("foo.txt");
No, it will not read the entire file into memory when you open it. It will read your file in chunks though, but I believe this process will not start until you read the first byte. Also these chunks are relatively small (on the order of 4-128 kibibytes in size), and the fact it does this will speed things up greatly if you are reading the file sequentially.
In a test on my Linux box (well, Linux VM) simply opening the file only results in the OS open system call, but no read system call. It doesn't start reading anything from the file until the first attempt to read from the stream. And then it reads 8191 (why 8191? that seems a very strange number) byte chunks as I read the file in.
Opening a file is a bad way of testing if the file exists - all it does is tell you if you can open it. Opening might fail for a number of reasons, typically because you don't have read permission, but the file will still exist. It is usually better to use an operating system specific function to test for existence. And no, opening an fstream will not cause the contents to be read.
What I think is, when you open a file, the corresponding data structures for the process opening the file are populated which include file pointer, file descriptor, v node etc.
Now one can read and write to a file using buffered streams (fwrite , fread) or using system calls (read and write).
When we use buffered streams, we buffer the data and then write or read it[This is done for efficiency puposes]. This statement itself means that the whole file is not read into memory but certain bytes are read into buffer and then made available.
In case of sys calls such as read and write , kernel level buffering is done (using fsync one can flush out kernel buffer too), but data is actually read and written to the device .file
checking existance of file
#include < sys/stat.h >
int main(){
struct stat file_i;
std::string f("myfile.txt");
if (stat(f.c_str(),&file_i) != 0){
cout << "File not found" << endl;
}
return 0;
}
Hope this clarifies a bit.
I'm trying to read data from a text file, clear it, and then write to it, in that order using the fstream class.
My question is how to clear a file after reading from it. I know that I can open a file and clear it at the same time, but is there some function I can call on the stream to clear its contents?
You should open it, perform your input operations, and then close it and reopen it with the std::fstream::trunc flag set.
#include <fstream>
int main()
{
std::fstream f;
f.open("file", std::fstream::in);
// read data
f.close();
f.open("file", std::fstream::out | std::fstream::trunc);
// write data
f.close();
return 0;
}
If you want to be totally safe in the event of a crash or other disastrous event, you should do the write to a second, temporary file. Once finished, delete the first file and rename the temporary file to the first file. See the Boost Filesystem library for help in doing this.