Suppose you declare an instance of std::ifstream or std::ofstream but is_open() returns 0
Example:
std::ifstream file("myfile.txt");
if (!file.is_open()) {
printf("Could not open file\n");
return;
}
Since the file never opened, do I still need to call file.close() after the printf statement?
No, you can only close an open file (similar to how you cannot close an already closed door - there is nothing to be done).
Extra note: Please do not combine the C I/O library (Xprintf family of functions) with the C++ I/O library (iostreams).
Consider using code like this:
std::ifstream file("myfile.txt");
if (!file.is_open()) {
std::cerr << "Could not open file\n";
return;
}
Edit (reasons not to use C IO API and C++ IO API together):
Using both APIs imposes synchronization between them, with priority towards the C API (i.e. the C api remains as fast, but IO streams will become slower, due to synchronization requirements).
It is inconsistent, using two very different concepts/abstraction levels for the same task. In more complex code, you will have to write twice the error handling (they impose different styles of error handling in client code), have both their limitations and combine their bad aspects (C API is prone to buffer overflows/security issues on reading and fails silently, unless you pay lots of attention to writing/maintenance of each API call, C++ API calls can be verbose).
They do not need to be particularly close (or far) appart, it is simply a bad programming practice.
This is similar to using a generic ODBC C API for reading table1 of your database, and ActiveX Data Objects for reading table2 in the same database, and the same program, or using Qt for your development, then hacking over it with raw WinAPI calls: you have twice the problems to solve (and many times end up implementing the solution to the same problems twice).
Related
Apologies if this question is overly broad. I'm new to C++ and trying to understand different stream types and why they matter (or doesn't matter).
I'm learning by coding a simple program that launch a child process, and process the output. I'm following the Boost process synchronous IO example: https://www.boost.org/doc/libs/1_75_0/doc/html/boost_process/tutorial.html#boost_process.tutorial.io.
One of the example can be reduce to this:
#include <boost/process.hpp>
using namespace std;
using namespace boost::process;
int main(int argc, char *argv[]) {
opstream in;
ipstream out;
child c("c++filt", std_out > out, std_in < in);
in << "_ZN5boost7process8tutorialE" << endl;
in.pipe().close(); // This will help c++filt quit, so we don't hang at wait() forever
c.wait();
return 0;
}
My question is:
Why do we have to use a boost opstream? Can I use istringstream instead (besides that it doesn't compile)? Can make it compile with istringstream?
Boost document said:
Boost.process provides the pipestream (ipstream, opstream, pstream) to wrap around the pipe and provide an implementation of the std::istream, std::ostream and std::iostream interface.
Does being a pipe matter, i.e. does pipe have significant implication here?
What Are Processes, How Do They Talk?
Programs interact with their environment in various ways. One set of channels are the standard input, output and error streams.
These are often tied to a terminal or files by a shell (cmd.exe, sh, bash etc).
Now if programs interact with eachother, like:
ls | rev
to list files and send the output to another program (rev, which reverses each line), this is implemented with pipes. Pipes are an operating system feature, not a boost idea. All major operating systems have them.
Fun fact: the | operator used in a most shells to indicate this type of output/input redirection between processes is called the PIPE symbol.
What Is A Pipe, Then?
Pipes are basically "magic" file-descriptors that refer to an "IO channel" rather than a file. Pipes have two ends: One party can writes to one end, the other party reads from the other.
Why?
Two reasons that come to mind right away
Files require disk IO and syncing, making it slow
Another fun fact: MSDOS has implemented pipes in terms of temporary files (on disk) for a very long time:
MS-DOS 2.0 introduced the ability to pipe the output of one program as the input of another. Since MS-DOS was a single-tasking operating system, this was simulated by redirecting the first program’s output to a temporary file and running it to completion, then running the second program with its input redirected from that temporary file. Now all of a sudden, MS-DOS needed a location to create temporary files! For whatever reason, the authors of MS-DOS chose to use the TEMP variable to control where these temporary files were created.
The pipe enables asynchronous IO. This can be important in case processes have two-way (full duplex) IO going on.
Okay Do I Care?
Yes, no, maybe.
You mostly don't. The ipstream/opstream classes are 100% compatible with std::istream/std::ostream, so if you had a function that expects them:
void simulate_input(std::ostream& os)
{
for (int i = 0; i < 10; ++i) {
os << "_ZN5boost7process8tutorialE" << std::endl;
}
}
You can perfectly use it in your sample:
bp::opstream in;
bp::ipstream out;
bp::child c("c++filt", bp::std_out > out, bp::std_in < in);
simulate_input(in);
in.close();
c.wait();
When You Definitely Need It
In full-duplex situations where you could easily induce a deadlock where both programs are waiting for input from the other end because they're doing the IO synchronously.
You can find examples + solution here:
How to reproduce deadlock hinted to by Boost process documentation?
boost::process::child will not exit after closing input stream
Boost::Process Pipe Streams and Unit Test
I have a bunch of legacy code that uses C-style file I/O (using FILE* streams, fprintf, fscanf, etc.) that I'd like to update to operate off of other data sources (e.g., downloading from or posting to the web, or to or from a custom data transfer cable).
If it used iostreams, this would be an easy update, but it doesn't.
Are there any C or C++ libraries for I/O that support different kinds of streams but that are source-compatible or mostly source-compatible with C's FILE* stream functions?
If source compatibility is too much to ask, I'd settle for "able to be search-and-replaced with a Perl script." fprintf and fscanf seem to be the major obstacles here; I haven't found any iostream-compatible versions of these.
I'm looking for a solution that will work in both Windows and Linux.
Example
As a trivial example, I have code like the following:
FILE *input = fopen(filename, "rt");
fscanf(input, "%d,%d,%d,%f", &f_id, &f_units, &h_index, &h_default);
fclose(input);
I'd like to update it to handle data that's of the same format but may come from other sources, without having to worry about the intricacies of how exactly to replicate fscanf's semantics myself (since I have many, many fscanf and fprintf calls that would need updating). Something like this would work, except that I've been unable to find anything like istream_scanf:
std::istream *input = make_http_istream(url);
istream_scanf(*input, &f_id, &f_units, &h_index, &h_default);
delete input;
I'm using fstream to open a file for write. I don't want to overwrite an existing file so after some searching, I found ios::noreplace. But when I compile this:
#include <fstream>
using namespace std;
//......Did something else.
ofstream fout;
fout.open(outputFile,ios::noreplace);//outputFile is a C string
I get an error
error: ‘noreplace’ is not a member of ‘std::ios’
I'm just wondering is there any std:: subsitution for ios::noreplace?
Some searching on the internet reveals that you can add an existence check manually by attempting to open in "input" mode:
std::fstream myfile("thefile.txt", std::ios::in);
if (myfile)
{
// error, file exists!
}
else
{
myfile.close();
myfile.open("thefile.txt", std::ios::out); // OK now
}
noreplace never got into the standard. About four seconds of googling yields:
http://www.devx.com/tips/Tip/14544
In pre-standard C++, certain implementations of offered the flags ios::nocreate and ios::noreplace for controlling file creation. These flags were too platform-specific and never made it into the standard library, which supersedes the deprecated, pre-standard header. However, you can achieve the functionality of these obsolete flags rather easily.
fstream fs(fname, ios_base::in);// attempt open for read
if (!fs)
{
// file doesn't exist; create a new one
fs.open(fname, ios_base::out);
}
else //ok, file exists; close and reopen in write mode
{
// Should throw an error
}
The suggested answers are risky since they have race conditions.
Unless you can guarantee nobody will ever create that file while your are running this test, you should not use it.
As a workaround, use the non-portable method (on Linux for example open with O_CREAT|O_EXCL).
You can either use the resulting handle with code like boost to wrap it into an ofstream, or in this case use open() only to check and then create a new ofstream on the file (the latter assumes nobody deletes/renames the file in-between and thus might still have a race condition).
C++ not providing ANY safe way to create a file is a bad joke and likely the cause of quite a few security holes. You have to love standards that encourage bad practices by making writing correct code impossible.
The complaints are addressed! C++23 finally standardises the std::ios_base::noreplace flag to open a file for writing in exclusive mode, i.e. to fail if that file already exists.
Paper: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2467r1.html
Common standard library implementations are already supporting this in C++23 mode, including libstdc++ as bundled with GCC/g++.
fopen_s <--> OpenFile
fclose <--> CloseFile
Is my assumption correct?
I wonder what is better to use, OpenFile or CreateFile. The latter gives more freedom, but is it faster?
I would use neither in Delphi – I would use streams. Low level file handling is messy and error-prone, it's much better to use higher level routines if you can.
You ask which is faster, OpenFile or CreateFile. They are basically the same, but any method of opening a file is going to map onto the system call anyway so the performance will be the same no matter how you do it. What's more, when does performance for opening a file matter, it's when reading or writing that time is expended.
Any questions about performance are hard to answer without context. The answer for an app which reads thousands of small text files is different from one which streams backups to a tape drive, for example.
Anyway, to stress my original point, take advantage of the excellent high-level framework that Delphi provides, use streams, avoid low-level I/O and enjoy!
So, how does one use a Delphi stream? I'll try to illustrate this with a made up example of writing some text, in a string, to a file.
procedure SaveTextToFile(FileName, Text: string);
var
Stream: TFileStream;
begin
Stream := TFileStream.Create(FileName, fmCreate);
Try
if Length(Text)>0 then
Stream.WriteBuffer(Text[1], Length(Text)*SizeOf(Char));
Finally
Stream.Free;
End;
end;
It's pretty self-explanatory. The second parameter to the TFileStream constructor determines the file mode. Here we want to create a brand new file and so if any contents exist, they are removed. You can also specify file sharing with this parameter.
The code to write the buffer out has a little boiler-plate but again is very simple.
Loading it back results in an almost identical routine:
function LoadTextFromFile(FileName: string): string;
var
Stream: TFileStream;
begin
Stream := TFileStream.Create(FileName, fmOpenRead);
Try
SetLength(Result, Stream.Size div SizeOf(Char));
if Length(Result)>0 then
Stream.ReadBuffer(Result[1], Length(Result)*SizeOf(Char));
Finally
Stream.Free;
End;
end;
If you wish to seek around the file then you can set the Position property of the stream, or call the Seek() method. The advantage of the latter is that you can seek from current position or end position.
Streams are idiomatic Delphi. They are used pervasively in the RTL and VCL and by 3rd party libraries. They signal errors with exceptions in the native Delphi manner. There are many different stream classes that all derive from a common ancestor and many routines accept this common ancestor.
Low-level Delphi file handling is done like this:
procedure Proc;
var
f: file; // or f: TextFile;
begin
FileMode := fmOpenRead; // or fmOpenWrite or fmOpenReadWrite
AssignFile(f, 'C:\file.txt');
try
// Reset/Rewrite
// A number of BlockRead/BlockWrite/ReadLn/WriteLn...
finally
CloseFile(f);
end;
end;
This is the classic way of working with files in Delphi, and this is what corresponds to the C++ functions.
OpenFile and CreateFile are not Delphi functions, so they cannot correspond to the C++ functions. Instead, these are functions of the Windows API, which is available in all (Windows) programming languages. The former, OpenFile, is not recommended. Use CreateFile instead. But if you use the Windows API file-handling functions to open/create a file, you should also use these to read/write the file, e.g. the ReadFile function, and you must finish by using the CloseHandle function.
Notice in particular that OpenFile is a function of the Windows API, whereas CloseFile is a Delphi RTL function, so you cannot even use these together! Delphi: AssignFile->CloseFile; Windows API: CreateFile->CloseHandle.
You should also know, that there are high-level functions for managing files in the Delphi RTL (run-time library). I am sure other users will promote these.
It's been a long while since I have done any Delphi programming, but I remember that file IO were much better served using TStream suite of classes (TFileStream for file IO). They are essentially the equivalent mechanism of C++'s IO streams library, which is, of course, the preferred way of doing file IO in C++. See this simple example and this wiki.
While troubleshooting some performance problems in our apps, I found out that C's stdio.h functions (and, at least for our vendor, C++'s fstream classes) are threadsafe. As a result, every time I do something as simple as fgetc, the RTL has to acquire a lock, read a byte, and release the lock.
This is not good for performance.
What's the best way to get non-threadsafe file I/O in C and C++, so that I can manage locking myself and get better performance?
MSVC provides _fputc_nolock, and GCC provides unlocked_stdio and flockfile, but I can't find any similar functions in my compiler (CodeGear C++Builder).
I could use the raw Windows API, but that's not portable and I assume would be slower than an unlocked fgetc for character-at-a-time I/O.
I could switch to something like the Apache Portable Runtime, but that could potentially be a lot of work.
How do others approach this?
Edit: Since a few people wondered, I had tested this before posting. fgetc doesn't do system calls if it can satisfy reads from its buffer, but it does still do locking, so locking ends up taking an enormous percentage of time (hundreds of locks to acquire and release for a single block of data read from disk). Not doing character-at-a-time I/O would be a solution, but C++Builder's fstream classes unfortunately use fgetc (so if I want to use iostream classes, I'm stuck with it), and I have a lot of legacy code that uses fgetc and friends to read fields out of record-style files (which would be reasonable if it weren't for locking issues).
I'd simply not do IO a char at a time if it is sensible performance wise.
fgetc is almost certainly not reading a byte each time you call it (where by 'reading' I mean invoking a system call to perform I/O). Look somewhere else for your performance bottleneck, as this is probably not the problem, and using unsafe functions is certainly not the solution. Any lock handling you do will probably be less efficient than the handling done by the standard routines.
The easiest way would be to read the entire file in memory, and then provide your own fgetc-like interface to that buffer.
Why not just memory map the file? Memory mapping is portable (except in Windows Vista which requires you to jump through hopes to use it now, the dumbasses). Anyhow, map your file into memory, and do you're own locking/not-locking on the resulting memory location.
The OS handles all the locking required to actually read from the disk - you'll NEVER be able to eliminate this overhead. But your processing overhead, on the otherhand, won't be affected by extraneous locking other than that which you do yourself.
the multi-platform approach is pretty simple. Avoid functions or operators where standard specifies that they should use sentry. sentry is an inner class in iostream classes which ensures stream consistency for every output character and in multi-threaded environment it locks the stream related mutex for each character being output. This avoids race conditions at low level but still makes the output unreadable, since strings from two threads might be output concurrently as the following example states:
thread 1 should write: abc
thread 2 should write: def
The output might look like: adebcf instead of abcdef or defabc. This is because sentry is implemented to lock and unlock per character.
The standard defines it for all functions and operators dealing with istream or ostream. The only way to avoid this is to use stream buffers and your own locking (per string for example).
I have written an app, which outputs some data to a file and mesures the speed. If you add here a function which ouptuts using the fstream directly without using the buffer and flush, you will see the speed difference. It uses boost, but I hope it is not a problem for you. Try to remove all the streambuffers and see the difference with and without them. I my case the performance drawback was factor 2-3 or so.
The following article by N. Myers will explain how locales and sentry in c++ IOStreams work. And for sure you should look up in ISO C++ Standard document, which functions use sentry.
Good Luck,
Ovanes
#include <vector>
#include <fstream>
#include <iterator>
#include <algorithm>
#include <iostream>
#include <cassert>
#include <cstdlib>
#include <boost/progress.hpp>
#include <boost/shared_ptr.hpp>
double do_copy_via_streambuf()
{
const size_t len = 1024*2048;
const size_t factor = 5;
::std::vector<char> data(len, 1);
std::vector<char> buffer(len*factor, 0);
::std::ofstream
ofs("test.dat", ::std::ios_base::binary|::std::ios_base::out);
noskipws(ofs);
std::streambuf* rdbuf = ofs.rdbuf()->pubsetbuf(&buffer[0], buffer.size());
::std::ostreambuf_iterator<char> oi(rdbuf);
boost::progress_timer pt;
for(size_t i=1; i<=250; ++i)
{
::std::copy(data.begin(), data.end(), oi);
if(0==i%factor)
rdbuf->pubsync();
}
ofs.flush();
double rate = 500 / pt.elapsed();
std::cout << rate << std::endl;
return rate;
}
void count_avarage(const char* op_name, double (*fct)())
{
double av_rate=0;
const size_t repeat = 1;
std::cout << "doing " << op_name << std::endl;
for(size_t i=0; i<repeat; ++i)
av_rate+=fct();
std::cout << "average rate for " << op_name << ": " << av_rate/repeat
<< "\n\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n"
<< std::endl;
}
int main()
{
count_avarage("copy via streambuf iterator", do_copy_via_streambuf);
return 0;
}
One thing to consider is to build a custom runtime. Most compilers provide the source to the runtime library (I'd be surprised if it weren't in the C++ Builder package).
This could end up being a lot of work, but maybe they've localized the thread support to make something like this easy. For example, with the embedded system compiler I'm using, it's designed for this - they have documented hooks to add the lock routines. However, it's possible that this could be a maintenance headache, even if it turns out to be relatively easy initially.
Another similar route would be to talk to someone like Dinkumware about using a 3rd party runtime that provides the capabilities you need.