Thread safe cout technique. Am I missing something? - c++

I'm working with some multithreaded code for a game project, and got a bit tired of sorting through the stdout vomit created by two threads using cout for debuging messages at the same time. I did some research and stared at a wall for an hour or two before coming up with "something". The following code uses SFML for time keeping and threading. SFML mutexes are just wrapped critical sections in windows.
Header:
#include <SFML\System.hpp>
#include <iostream>
class OutputStreamHack
{
public:
OutputStreamHack();
~OutputStreamHack();
ostream& outputHijack(ostream &os);
private:
sf::Clock myRunTime;
sf::Mutex myMutex;
};
static OutputStream OUTHACK;
ostream& operator<<(ostream& os, const OutputStreamHack& inputValue);
Implementation:
#include <SFML\System.hpp>
#include <iostream>
#include "OutputStreamHack.h"
using namespace std;
OutputStreamHack::OutputStreamHack()
{
myMutex.Unlock();
myRunTime.Reset();
}
OutputStreamHack::~OutputStreamHack()
{
myMutex.Unlock();
myRunTime.Reset();
}
ostream& OutputStreamHack::outputHijack(ostream &os)
{
sf::Lock lock(myMutex);
os<<"<"<<myRunTime.GetElapsedTime()<<","<<GetCurrentThreadId()<<"> "<<flush;
return os;
}
ostream& operator<<(ostream& os, const OutputStreamHack& inputValue)
{
OUTHACK.outputHijack(os);
return os;
}
Usage:
cout<<OUTHACK<<val1<<val2<<val3....<<endl;
Ok, so the way this works is through an overloaded insertion operator that imposes thread safety by locking an iterator in a static object, then flushing the buffer. If I understand the process correctly (I am mostly a self taught programmer), cout processes elements of its insertion chain from the end to the beginning, passing an ostream variable down the chain for each element to be prepended to the stream. Once it reaches the OUTHACK element, the overloaded operator is called, the mutex is locked, and the stream is flushed.
I've added some time/thread id debugging info to the output for verification purposes. So far, my testing shows that this method works. I have several threads pounding cout with multiple arguments, and everything is coming out in the right order.
From what I have read while researching this issue, lack of thread safety in cout seems to be a pretty common problem that people run into while venturing into threaded programming. What I am trying to figure out is if the technique I am using is a simple solution to the problem, or me thinking that I am clever but missing something important.
In my experience, the word clever when used to describe programming is just a code word for delayed pain. Am I on to something here, or just chasing lousy hacks around in circles?
Thanks!

What is not threadsafe here is not cout per se. It's calling two function calls in sequence. std::cout << a << b is roughly equivalent to calling operator<<(std::cout, a) followed by operator<<(std::cout, b). Calling two functions in sequence carries no guarantee that they will be executed in an atomic fashion.
As is, only the output of the time and thread id is protected by the mutex. It's perfectly possible to get another thread sneak in between the insertion of OUTHACK and val1, because the lock is no longer held after OUTHACK is inserted.
You can have operator<< for your OutputStreamHack return by value an object that unlocks in the destructor. Since temporaries live until the end of each full expression, the code would hold the lock "until the semicolon". However, because copies may be involved, this could be problematic without a move constructor (or a custom copy constructor in C++03, similar to auto_ptr's gasp).
Another option is to use the existing thread-safety of cout (guaranteed by the language in C++11, but many implementations were threadsafe before). Make an object that streams everything into a std::stringstream member and then write it all out at once when it is destroyed.
class FullExpressionAccumulator {
public:
explicit FullExpressionAccumulator(std::ostream& os) : os(os) {}
~FullExpressionAccumulator() {
os << ss.rdbuf() << std::flush; // write the whole shebang in one go
}
template <typename T>
FullExpressionAccumulator& operator<<(T const& t) {
ss << t; // accumulate into a non-shared stringstream, no threading issues
return *this;
}
private:
std::ostream& os;
std::stringstream ss;
// stringstream is not copyable, so copies are already forbidden
};
// using a temporary instead of returning one from a function avoids any issues with copies
FullExpressionAccumulator(std::cout) << val1 << val2 << val3;

Related

Is the C++ ostream at the start of a line?

In C++, how do I detect if my std::ostream os is at the start of a line, in other words (I think) the most recent thing written to os is either os<<'\n' or os<<std::endl(), or else nothing has yet been written to os?
At first glance, this sounds unnecessary, because I can just keep track of the state myself. But a common scenario is the following, where keeping track would involve altering every os<<thing statement which could possibly be called from the try block, in a very unnatural way.
try {
do_something_which_writes_to(std::cout);
}
catch(const My_error&error) {
print_a_newline_if_necessary(std::cout);
std::cout<<error<<"\n";
}
(In reality, of course, we want to write error to std::cerr but that usually gets mixed in with std::cout unless one of them is redirected, so we still want to terminate the std::cout line before printing to std::cerr. I have deliberately simplified the example to avoid this distraction.)
You might imagine that os.tellp() would be the answer, but tellp() seems to work only on std::ofstream. For me at least, std::cout.tellp() always returns -1, indicating that it is not supported.
At least as I'm reading things, what you really want isn't the ability to get the position in the current line. Rather, what you really want is to be able to print something that is guaranteed to be at the start of a line--the current line if the immediately previous character was a newline (and, I'd guess also if it was a carriage return), but otherwise print a newline, then whatever follows.
Here's some code to do that:
#include <iostream>
class linebuf : public std::streambuf
{
std::streambuf* sbuf;
bool need_newline;
int sync() {
return sbuf->pubsync();
}
int overflow(int c) {
switch (c) {
case '\r':
case '\n': need_newline = false;
break;
case '\v':
if (need_newline) {
need_newline = false;
return sbuf->sputc('\n');
}
return c;
default:
need_newline = true;
break;
}
return sbuf->sputc(c);
}
public:
linebuf(std::streambuf* sbuf)
: sbuf(sbuf)
, need_newline(true)
{}
std::streambuf *buf() const { return sbuf; }
~linebuf() { sync(); }
};
class linestream : public std::ostream {
linebuf buf;
std::ostream &os;
public:
linestream(std::ostream& out)
: buf(out.rdbuf())
, std::ios(&buf)
, std::ostream(&buf)
, os(out)
{
out.rdbuf(&buf);
}
~linestream() { os.rdbuf(buf.buf()); }
};
void do_stuff() {
std::cout << "\vMore output\v";
}
int main() {
{
linestream temp(std::cout);
std::cout << "\noutput\n";
std::cout << "\voutput";
do_stuff();
std::cout << "\voutput\n";
std::cout << "\voutput\v";
}
std::cout << "\voutput\v";
}
Since it's almost never used otherwise, I've hijacked the vertical tab ('\v') to signify the special behavior.
To use it, you simply create a temporary object of type linestream (sorry, I'm too tired to think of a good name right now), passing it an ostream object that will get the new behavior when a \v gets written to it. When that temporary object goes out of scope, the stream will be restored to its original behavior (I doubt anybody uses \v often enough to care, but who knows maybe somebody care--it's mostly just a side effect of cleaning up after itself anyway).
In any case, the special behavior remains in place when do_stuff is called, so it's not just local to the function where the local linestream object is created, or anything like that--once it's created, the special behavior remains in effect until it's destroyed.
One other point though: when/if you mix output from cout and cerr, this won't help much. In particular, neither will be at all aware of the other's state. You'd probably much need some hooks into the output terminal (or something on that order) to be able to deal with that, since output redirection is normally handled by the OS, so inside the program there's no way to even guess whether data written to cout and cerr are going to the same place or not.

std::for_each seems to be cleaning std::string

I'm making a simple snake game in c++17 with STL, and I have a problem with parsing board (std::vector) to std::string. I'm using std::for_each with my custom functor. Here's parsing method:
std::string Board::parseBoardToString()
{
boardToString.resetStringBoard();
std::for_each(v_board.begin(), v_board.end(), boardToString);
return boardToString.getStringBoard();
}
Functor prototype (some of these method are for testing, doesn't matter):
class BoardToString
{
public:
BoardToString(unsigned _width);
void operator()(char _cell);
void resetStringBoard();
std::string getStringBoard();
void printStringBoard();
private:
bool isItLastLine();
void resetCharsToNewLine();
std::string stringBoard;
const unsigned U_WIDTH;
unsigned charsToNewLine;
};
And the operator() implementation:
void BoardToString::operator()(char _cell)
{
stringBoard.push_back(_cell);
//std::cout << _cell;
//std::cout << stringBoard.back();
if (isItLastLine())
{
stringBoard.push_back('\n');
//std::cout << stringBoard.back();
//std::cout << '\n';
resetCharsToNewLine();
}
}
After some tests I'm pretty sure that stringBoard gets cleared when for_each ends his job. GDB shows properly string size when for_each is working, and 0 after that. Both commented couts are working properly. Anyway I don't know why it goes like that. I know I could easily fix it by making a little bit different implementation, but I want to do it right way.
Change
std::for_each(v_board.begin(), v_board.end(), boardToString);
to
std::for_each(v_board.begin(), v_board.end(), std::ref(boardToString));
Function objects passed to std algorithms are copied; std ref wraps them up so they are not copied.
So the problem is that your functor object is being copied and the modifications are happening to the copy. This is why it seems to reset when you've finished.
You could change your functor to hold references to the variables you need to modify.
You could use a lambda instead, again holding references to the variables that need to be modified.
But to me the whole approach is too elaborate. Just write a for loop. Nothing gets copied, everything is transparent and direct.

How to overload << operator for thread-safe logging in C++?

I'm writing a sketch of a simple thread-safe logging library and some things came on my mind. Here's the code:
#ifndef SimpleLogger_H
#define SimpleLogger_H
#include <iostream>
#include <mutex>
class SimpleLogger
{
public:
template <typename T>
static void log(T message)
{
mutex.lock();
std::cout << message;
mutex.unlock();
}
private:
static std::mutex mutex;
}LOG;
template <typename T>
SimpleLogger &operator<<(SimpleLogger &simpleLogger, T message)
{
simpleLogger.log(message);
return simpleLogger;
}
#endif //SimpleLogger_H
My idea is to use it like this:
LOG << "hello" << " world " << 8 << " I can mix integers and strings";
I understand that the line above is like the following:
auto a1 = LOG.operator<<("hello");
auto a2 = a1.operator<<(" world ");
//Another thread may want to use LOG here, and would print in the middle of my message
auto a3 = a2.operator<<(8);
auto a4 = a3.operator<<(" I can mix integers and strings");
As you can see, because << is broken into several funciton calls, there's a risk that a thread can use the LOG object in the middle of my message (I consider a message the entire cascade of << on one line)
Also, is there a way to automatically add an std::endl for the last << call? I couldn't think of a way to do this, but I saw that some logging libraries have this functionality
How do I solve these two problems?
I know it'd be preferable to use a logging library but I want to mix android, desktop and ios logging in one simple lib without need for high performance, and I'm also puzzled by how I can overcome the difficulties I encountered while writing my own
As others already mentioned, you need a local buffer to collect message before sending to the log file. In the example below, SimpleLoggerBuffer objects are designed to be used as temporary variable only. I.e. it gets destroyed at the end of the expression. The destructor flushes the buffer into the log so that you don't have to explicitly call a flush function (you may add endl there as well if you wish)
#include <iostream>
#include <sstream>
#include <mutex>
using namespace std;
class SimpleLogger
{
public:
template <typename T>
static void log(T& message)
{
mutex.lock();
std::cout << message.str();
message.flush();
mutex.unlock();
}
private:
static std::mutex mutex;
}LOG;
std::mutex SimpleLogger::mutex;
struct SimpleLoggerBuffer{
stringstream ss;
SimpleLoggerBuffer() = default;
SimpleLoggerBuffer(const SimpleLoggerBuffer&) = delete;
SimpleLoggerBuffer& operator=(const SimpleLoggerBuffer&) = delete;
SimpleLoggerBuffer& operator=(SimpleLoggerBuffer&&) = delete;
SimpleLoggerBuffer(SimpleLoggerBuffer&& buf): ss(move(buf.ss)) {
}
template <typename T>
SimpleLoggerBuffer& operator<<(T&& message)
{
ss << std::forward<T>(message);
return *this;
}
~SimpleLoggerBuffer() {
LOG.log(ss);
}
};
template <typename T>
SimpleLoggerBuffer operator<<(SimpleLogger &simpleLogger, T&& message)
{
SimpleLoggerBuffer buf;
buf.ss << std::forward<T>(message);
return buf;
}
int main() {
LOG << "hello" << " world " << 8 << " I can mix integers and strings";
}
You could create a helper class that collects all the output and prints on destruction. Outline:
#include <string>
#include <iostream>
struct Msg;
struct Log {
void print(const Msg &m);
};
struct Msg {
std::string m;
Log &l;
Msg(Log &l) : l(l) {}
~Msg() {
// Print the message on destruction
l.print(*this);
}
};
void Log::print(const Msg &m) {
// Logger specific printing... here, append newline
std::cout << m.m << std::endl;
}
Msg &&operator << (Msg &&m, const std::string &s) {
// Append operator
m.m += s;
return std::move(m);
}
// Helper to log on a specific logger. Just creates the initial message
Msg log(Log &l) { return Msg(l); }
int main()
{
Log l;
log(l) << "a" << "b" << "c";
return 0;
}
As the Msg is local, other threads will not interfere with it. Any necessary locking can be done in the Log.print method, which will receive the complete message
A simple solution is to write into files instead of standard output, and specifically, separate file for each thread. That way no locking or any other synchronization is needed. The files can later be merged if lines have parseable format.
Another solution is to write logs asynchronously from a single thread, and initially store the messages in a thread safe (possibly lock free) queue.
Also, is there a way to automatically add an std::endl for the last << call?
Unless I misunderstand, you can simply do stream << message << std::endl.
I think that you can simply use std::clog. It is thread safe and, in contrary of std::cout, designed to output instantly for logging.
From the reference page :
Unless sync_with_stdio(false) has been issued, it is safe to
concurrently access these objects from multiple threads for both
formatted and unformatted output.
I recommend you this Jason Turner's video about cout, clog and cerror.
The simplest approach is to return a temporary proxy from the first << - this can either log your stream for the duration (and unlock on destruction), or simply build a local ostringstream and flush it in a single call (again, on destruction).
Holding a lock while logging isn't great for performance (although it's better than your existing log method, which should use std::lock_guard for exception safety).
Building and discarding a temporary ostringstream is probably better, but if you care about performance you'll need to benchmark, and may well end up requiring something more elaborate (per-thread circular buffers, mmapped files or something).

How to detect if a ptr is still referencing a valid reference after that reference goes out of scope

I am toying around with streams for a bit and can't get my head around the following.
Here we have a basic ostream ptr that is set to different output streams, whether it is cout, cerr or a file.
// ostream ptr
std::ostream* outstream;
// set output ostream
void setOutput(std::ostream & os)
{
outstream = &os;
}
// write message to ostream
void writeData(const std::string & msg)
{
*outstream << msg << '\n';
}
int main (int argc, char * const argv[])
{
// init to std out
setOutput(std::cout);
writeData("message to cout");
setOutput(std::cerr);
writeData("message to cerr");
std::ofstream fileout("test.txt", std::ofstream::out | std::ofstream::app);
setOutput(fileout);
writeData("message to file");
//fileout.close();
setOutput(std::cout);
writeData("message2 to cout");
return 0;
}
The above works perfectly and shows the strength of the c++ iostream implementation. Perfect.
However, since the setOutput is set by reference the referenced object has to stay in scope. This is where the issue emerges. I want to figure out a way to default the output to std::cout if the ofstream or any other ostream is invalidated. That is, referenced object is or went out of scope.
For example:
// write message to ostream
void writeData(const std::string & msg)
{
if (/*stream or memory is invalid*/)
setOutput(std::cout);
*outstream << msg << '\n';
}
// local fileout goes out of scope
void foo()
{
std::ofstream fileout("test.txt", std::ofstream::out | std::ofstream::app);
setOutput(fileout);
writeData("message to file");
}
int main (int argc, char * const argv[])
{
setOutput(std::cout);
writeData("message to cout");
foo();
/* problem the local fileout is no longer referenced by the ostream ptr*/
/* the following should be redirected to std::cout cuz of default*/
writeData("message2 to cout");
return 0;
}
The above is fine until the foo() returns to the main function. There it goes horrible wrong because the locally defined ofstream is not reachable anymore.
Obviously this is not advisable and the user should realise this. However I want to wrap all this in a logging class and thus keep the state of the object valid even thought this misuse might happen. It will cause an invalidate access violation which can be hard to find.
Concrete question. Is there any way to figure out whether an ostream ptr or any ptr for that matter is still referencing a valid object or memory location?
ps: I could use heap memory and do something with smart pointers but frankly I'd want to keep it like this if possible
Concrete question. Is there any way to figure out whether an ostream ptr or any ptr for that matter is still referencing a valid object or memory location?
No. There is no way to figure that out with raw pointers. Not in standard c++ at least.
You will need to guarantee that the pointed object stays alive as long as it's pointed to.
A common pattern that is used to provide that guarantee is RAII, as detailed in other answers. Another approach to guaranteeing validity of of a pointer is to use a smart pointer instead of a raw one. However, those are not compatible with automatic variables.
It would be OK to keep pointing to dead objects as long as you could guarantee that the pointer is not dereferenced. Which is often difficult to guarantee, because, as already stated, there is no way to test whether the pointed object exists.
This sounds like a great use case for RAII.
Write a class that takes a filename and a std::ostream** as parameters to its constructor. In the constructor of the said class, construct the ofstream (as a member), and set the pointer to the ofstream. In the destructor, revert to stdout.
Then, replace the first two lines of the following function with a declaration of the new class.
void foo()
{
std::ofstream fileout("test.txt", std::ofstream::out | std::ofstream::app);
setOutput(fileout);
writeData("message to file");
}
You should use RAII to force the stream to be set correctly and then set back to std::cout if the object is destroyed.
class OutputStream
{
protected:
static std::ostream*& internalGlobalStateOfOutputStream()
{
static std::ostream* out = &std::cout;
return out;
}
public:
static std::ostream& getOutputStream()
{
return *internalGlobalStateOfOutputStream();
}
};
template<typename T>
class OutputStreamOwner: public OutputStream
{
T ownedStream;
public:
OutputStreamOwner(T&& obj)
: ownedStream(std::move(obj))
{
internalGlobalStateOfOutputStream() = &ownedStream;
}
template<typename... Args>
OutputStreamOwner(Args... args)
: ownedStream(args...)
{
internalGlobalStateOfOutputStream() = &ownedStream;
}
~OutputStreamOwner()
{
internalGlobalStateOfOutputStream() = & std::cout;
}
// Delete copy
OutputStreamOwner(OutputStreamOwner const&) = delete;
OutputStreamOwner& operator(OutputStreamOwner const&) = delete;
};
The usage is:
void foo()
{
OutputStreamOwner<std::ofstream> output("test.txt", std::ofstream::out | std::ofstream::app);
writeData("message to file");
}
A possible approach is to create a RAII class that wraps the stream before passing it into setOutput. This class should be designed to work like shared_ptr such that it maintains a shared ref count. writeData then checks to see if it has the only remaining reference and if so then destroys the ostream and defaults to cout.
You could avoid these complications altogether with a function that takes the stream as input.
void writeData(std::ostream& os, const std::string & msg)
{
os << msg << '\n';
}
You can further refine it by returning the stream, to allow one to chain calls to it:
std::ostream& os writeLine(std::ostream& os, const std::string & msg)
{
os << msg << '\n';
return os;
}
// declare stream
stream << writeLine(stream, "Foo") << writeLine(stream, "Bar");
In fact this function is nicer and easier to maintain, as you don't have to remember which stream is set at any given time. For large programs, this is an important quality.
Concrete question. Is there any way to figure out whether an ostream
ptr or any ptr for that matter is still referencing a valid object or
memory location?
ps: I could use heap memory and do something with smart pointers but
frankly I'd want to keep it like this if possible
No, there is no standard way to test if a raw pointer or reference is still referring to a valid object.
RAII is the standard C++ solution to this type of problem so you should be looking at smart pointers, in my opinion. I am not aware of any library provided smart pointer that would solve this particular problem but an RAII solution based on shared ownership seems the best solution here.

Ways to write a vector of a class to a file

I'm not sure how I should word this, so I'll attempt to put it in code. (This has many errors I know it will not compile it is simply to show what I want to do because I can't put it in words)
using namespace std; //for correctness sake
class foo {
public:
foo(int a=0, int n=0);
void stuff(int a);
void function(int n);
const int get_function() {return n;}
const int get_stuff(){return a;}
private:
int n, a;
};
struct Store{
public: get_foo(){return vector<f.foo>;} //I'm not too sure of the syntax here but you get the idea
private:
foo f;
}
Basically I want to take all the information that is returned in class foo, and output this, formatted, to a file. Thing is, I need to make many of these within the file and it has to be able to read it back for it to be worth anything.
So just appending each consecutive foo class to the file won't work(at least I don't see how).
I tried using ostream to overload the << operator, but I'm not sure how to call it to write it to the file. Any suggestions are welcome! Thanks.
I think your Store should be like this:
struct Store
{
public:
std::vector<foo> get_foo()
{
return f;
}
private:
std::vector<foo> f;
};
To overload << of std::ostream:
std::ostream& operator<<(std::ostream& out, const Store& f)
{
for (auto &x : f.get_foo())
out << x.get_function() << ", " << x.get_stuff() << "\n";
return out;
}
//Without auto
std::ostream& operator<<(std::ostream& out, const Store& f)
{
std::vector<foo> q = f;
for (int i=0; i<q.size(); i++)
out << q[i].get_function() << ", " << q[i].get_stuff() << "\n";
return out;
}
There are many tings wrong with your code.
So many that it's clear that you never read any C++ book and are just experimenting with a compiler.
Don't do that. C++ is really the worst language in the world to approach that way for many independent reasons.
No matter how smart you are.
Actually being smart is sort of a problem in certain areas because many C++ rules are not the result of a coherent logical design, but of historical evolution and committee decisions. Not even Hari Seldon would be able to foresee correctly what a committee would decide, you cannot deduce history.
Just pick a good book and read it cover to cover. There is no other sensible way to learn C++.
About writing structs to a file the topic is normally called "serialization" and takes care of the slightly more general problem of converting live objects into a dead sequence of bytes (written to a file or sent over the network) and the inverse problem "deserialization" of converting the sequence of bytes back into live objects (on the same system, on another identical system or even on a different system).
There are many facets of this problem, for example if your concern is about portability between systems, speed, size of the byte sequence, ability to reload bytes sequences that were saved back when your classes were slightly different because you evolved the program (versioning).
The simplest thing you can do is just fwrite things to a file, but this is most often simply nonsense in C++ and is a terrible way for many reasons even when it's technically possible. For example you cannot directly fwrite an std::vector object and hope to read it back.
I think you need something like this:
template<typename T>
std::ostream& operator << (std::ostream& out, const std::vector<T*>& elements)
{
for (size_t i = 0; i < elements.size(); i++)
out << elements[i] << ", ";
return out << std::endl;
}