I have a java 8 stream with underlying IOStream and I want to make sure that my method closes that stream.
Is there any way to check it with unit test?
I think providing a handler to Stream's onClose method is the easiest way to do this.
AtomicBoolean wasClosed = new AtomicBoolean(false);
Stream<> stream = Stream.of(foo, bar).onClose(() -> wasClosed.set(true));
// ...code under test that uses the stream
assertThat(wasClosed.get()).isTrue();
For what it's worth, I legitimately needed to test this, as we make Streams from JDBC ResultSets, and the actual Stream relies on onClose to close the ResultSet.
Streams have a BaseStream.close() method and implement AutoCloseable, but nearly all stream instances do not actually need to be closed after use. Generally, only streams whose source is an IO channel (such as those returned by Files.lines(Path, Charset)) will require closing. Most streams are backed by collections, arrays, or generating functions, which require no special resource management. (If a stream does require closing, it can be declared as a resource in a try-with-resources statement.)
https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html
Related
I'm connecting to a websocket using the boost/beast libraries and writing the data into a beast::flat_buffer. My issue is that I'm having trouble getting the data from buffer. I have a thread-safe channel object that I can to write to, but I'm not sure how to pull the most recently received message from the buffer.
beast::flat_buffer buffer;
// send through socket
send_socket(
ws, get_subscribe_string(trade_tickers, bar_tickers, quote_tickers));
// read into buffer
while (channel.running) {
ws.read(buffer); // need to write the most recently received message into the channel
}
I can write to the channel with channel.write(std::string arg). Any ideas on how to pull from the buffer?
The interface for flat_buffer is documented here: https://www.boost.org/doc/libs/1_77_0/libs/beast/doc/html/beast/ref/boost__beast__flat_buffer.html
As you can see it is a rich interface that lends itself to a number of different usage patterns, including reading and writing blocks in FIFO style.
Now, if you are using websockets, your protocol is already message-oriented rather than stream oriented. You might just want to access all of the data as one "body". In my opinion, the safe, expressive and flexible way to do this is using the data() member. This models the general Asio Buffer concept meaning that you can use the Buffer Iterators on it, without worrying about any of the buffer implementation details:
std::string text(buffers_begin(buffer), buffers_end(buffer));
For an example of this in actual use (to receive JSON or msgpack) see this recent answer: I would like to parse a boost::beast::flat_buffer with msgpack data using nlohmann:json
Think Outside The Box
Note the implication though: The flat_buffer is not mandatory. In fact, it's just one (simple) implementation modeling the DynamicBuffer concept.
You could use any model, so instead you can receive directly into a string:
std::string str;
auto buf = boost::asio::dynamic_buffer(str);
ws.read(str);
If you reuse str instance, e.g. just using str.clear() it may not be bad in terms of allocations.
According to the boost reference for Boost.Iostreams (In section 3.6, at the very bottom):
http://www.boost.org/doc/libs/1_64_0/libs/iostreams/doc/index.html
Although the Boost.Iostreams Filter and Device concepts can
accommodate non-blocking i/o, the C++ standard library stream and
stream buffer interfaces cannot, since they lack a means to
distinguish between temporary and permanent failures to satisfy a read
or write request
However, the function std::istream::readsome appears to be non-blocking, in that the available characters will be immediately returned, without a blocking (except for a RAM copy) wait. My understanding is that:
std::istream::read will block until eof or number of characters read.
std::istream::readsome will return immediately with characters copied from the internal buffer.
I agree with you that readsome is not a blocking operation. However, as specified, it is wholly inadequate as an interface for performing what is usually called "non-blocking I/O".
First, there is no guarantee that readsome will ever return new data, even if it is available. So to guarantee you actually make progress, you must use one of the blocking interfaces eventually.
Second, there is no way to know when readsome will return data. There is no way to "poll" the stream, or to get a "notification" or "event" or "callback". A usable non-blocking interface needs at least one of these.
In short, readsome appears to be a half-baked and under-specified attempt to provide a non-blocking interface to I/O streams. But I have never seen it used in production code, and I would not expect to.
I think the Boost documentation overstates the argument, because as you observe, readsome is certainly capable of distinguishing temporary from permanent failure. But their conclusion is still correct for the reasons above.
When looking into non-blocking portability, I didn't find anything in the C++ standard library that looked like it did what you think it does.
If your goal is portability, my interpretation was that the section that mattered most was this:
http://en.cppreference.com/w/cpp/io/basic_istream/readsome
For example, when used with std::ifstream, some library
implementations fill the underlying filebuf with data as soon as the
file is opened (and readsome() on such implementations reads data,
potentially, but not necessarily, the entire file), while other
implementations only read from file when an actual input operation is
requested (and readsome() issued after file opening never extracts any
characters).
This says that different implementations that use the iostream interface are allowed to do their work lazily, and readsome() doesn't guarantee that the work even gets kicked off.
However, I think your interpretation that readsome is guaranteed not to block is true.
I have an std::istream to work with. Is it possible to somehow pass it on to multiple readers which will potentially seek to and read from different positions?
If not, what if I restrict it to the case of an std::ifstream?
You already answered your question. If it is filestream (ifstream) you get random access (read only; you can set open mode), there should be no problem with multiple threads accessing the same file by opening multiples ifstreams each for one thread. The C++ standard said nothing about thread-safeness about ifstream. For the generic istream (socket, cin), if you use the get() method you will be consuming input stream. I don't see any document for thread-safe of istream. the peek() method will not consume the input stream but will still change the internal state of the istream. If multiple threads doing seek() on the same istream, the behavior is undefined. You are not assured of an internal lock by the C++ language. The seek() is basically dereferencing some sort of pointer to an internal buffer.
I would suggest that you have one thread reading the istream into some buffer (constructed objects (the producer), or simple raw memory), then, multiple threads can consume the result (consumer). This is typical consumer/producer synchronization; any multi-threading text book will teach you how to do it.
In C++, I know I can use read or write file using system function like read or write and I can also do that with fstream's help.
Now I'm implementing a disk management which is a component of DBMS. For simplicity I only use disk management to manage the space of a Unix file.
All I know is fstream wrap system function like read or write and provide some buffer.
However I was wondering whether this will affect atomicity and synchronization or not?
My question is which way should I use and why?
No. Particularly not with Unix. A DBM is going to want contiguous files. That means either a unix variant that support them or creating a disk partition.
You're also going to want to handle the buffering; not following the C++ library's buffering.
I could go on but streams are for - - streams of data -- not secure, reliable structured data.
The following information about synchronization and thread safety of 'fstream' can be found from ISO C++ standard.
27.2.3 Thread safety [iostreams.threadsafety]
Concurrent access to a stream object (27.8, 27.9), stream buffer
object (27.6), or C Library stream (27.9.2) by multiple threads may
result in a data race (1.10) unless otherwise specified (27.4). [
Note: Data races result in undefined behavior (1.10). —end note ]
If one thread makes a library call a that writes a value to a stream
and, as a result, another thread reads this value from the stream
through a library call b such that this does not result in a data
race, then a’s write synchronizes with b’s read.
C/C++ file I/O operation are not thread safe by default. So if you are planning to use fstream of open/write/read system call, then you would have to use synchronization mechanism by yourself in your implementation. You may use 'std::mutex' mechanism provided in new C++ standard(.i.e C++11) to synchronize your file I/O.
I have threads in my program and I want to put character into stream and read it in another thread, but after std::cin.putback() I need to write something from keyboard to "wake up" std::cin in function main. Can I do something to read automatically?
That's not how streams work. The std::cin reads data that come from outside your program to standard input and the putback only allows keeping a character that you actually just read back to the buffer for re-parsing next time you invoke operator>> (or get or getline or other read method).
If you want to communicate between threads, you should use a message queue from some threading library, e.g. Boost provides a decent portable one.
It is not possible to use streams, at least those provided by standard library, because stringstream is not thread-safe and fistream/fostream can't be created from raw file handle, so you can't combine them with POSIX pipe function. It would be possible to wrap a message queue in a stream (and boost gives you enough tools to do it), but the raw message queue API will probably be suitable.