A line-based thread-safe std::cerr for C++ - c++

Question

What is the easiest way to create my own std::cerr so that it is line-by-line thread-safe.
I am preferably looking for the code to do it.
What I need is so that a line of output (terminated with std::endl) generated by one thread stays as a line of output when I actually see it on my console (and is not mixed with some other thread's output).
Solution: std::cerr is much slower than cstdio. I prefer using fprintf(stderr, "The message") inside of a CriticalSectionLocker class whose constructor acquires a thread-safe lock and the destructor releases it.

Answer 1

If available, osyncstream (C++20) solves this problem:
#include <syncstream> // C++20
std::osyncstream tout(std::cout);
std::osyncstream terr(std::cerr);
If the above feature is not available, here is a drop-in header file containing two macros for thread-safe writing to std::cout and std::cerr (which must share a mutex in order to avoid interleaving of output). These are based on two other answers, but I have made some changes to make it easy to drop into an existing code base. This works with C++11 and forward.
I've tested this with 4 threads on a 4-core processor, with each thread writing 25,000 lines per second to tout and occasional output to terr, and it solves the output interleaving problem. Unlike a struct-based solution, there was no measurable performance hit for my application when dropping in this header file. The only drawback I can think of is that since this relies on macros, they can't be placed into a namespace.
threadstream.h
#ifndef THREADSTREAM
#define THREADSTREAM
#include <iostream>
#include <sstream>
#include <mutex>
#define terr ThreadStream(std::cerr)
#define tout ThreadStream(std::cout)
/**
* Thread-safe std::ostream class.
*
* Usage:
* tout << "Hello world!" << std::endl;
* terr << "Hello world!" << std::endl;
*/
class ThreadStream : public std::ostringstream
{
public:
ThreadStream(std::ostream& os) : os_(os)
{
// copyfmt causes odd problems with lost output
// probably some specific flag
// copyfmt(os);
// copy whatever properties are relevant
imbue(os.getloc());
precision(os.precision());
width(os.width());
setf(std::ios::fixed, std::ios::floatfield);
}
~ThreadStream()
{
std::lock_guard<std::mutex> guard(_mutex_threadstream);
os_ << this->str();
}
private:
static std::mutex _mutex_threadstream;
std::ostream& os_;
};
std::mutex ThreadStream::_mutex_threadstream{};
#endif
test.cc
#include <thread>
#include <vector>
#include <iomanip>
#include "threadstream.h"
void test(const unsigned int threadNumber)
{
tout << "Thread " << threadNumber << ": launched" << std::endl;
}
int main()
{
std::locale mylocale(""); // get global locale
std::cerr.imbue(mylocale); // imbue global locale
std::ios_base::sync_with_stdio(false); // disable synch with stdio (enables input buffering)
std::cout << std::fixed << std::setw(4) << std::setprecision(5);
std::cerr << std::fixed << std::setw(2) << std::setprecision(2);
std::vector<std::thread> threads;
for (unsigned int threadNumber = 0; threadNumber < 16; threadNumber++)
{
std::thread t(test, threadNumber);
threads.push_back(std::move(t));
}
for (std::thread& t : threads)
{
if (t.joinable())
{
t.join();
}
}
terr << std::endl << "Main: " << "Test completed." << std::endl;
return 0;
}
compiling
g++ -g -O2 -Wall -c -o test.o test.cc
g++ -o test test.o -pthread
output
./test
Thread 0: launched
Thread 4: launched
Thread 3: launched
Thread 1: launched
Thread 2: launched
Thread 6: launched
Thread 5: launched
Thread 7: launched
Thread 8: launched
Thread 9: launched
Thread 10: launched
Thread 11: launched
Thread 12: launched
Thread 13: launched
Thread 14: launched
Thread 15: launched
Main: Test completed.

Answer 2

Here's a thread safe line based logging solution I cooked up at some point. It uses boost mutex for thread safety. It is slightly more complicated than necessary because you can plug in output policies (should it go to a file, stderr, or somewhere else?):
logger.h:
#ifndef LOGGER_20080723_H_
#define LOGGER_20080723_H_
#include <boost/thread/mutex.hpp>
#include <iostream>
#include <cassert>
#include <sstream>
#include <ctime>
#include <ostream>
namespace logger {
namespace detail {
template<class Ch, class Tr, class A>
class no_output {
private:
struct null_buffer {
template<class T>
null_buffer &operator<<(const T &) {
return *this;
}
};
public:
typedef null_buffer stream_buffer;
public:
void operator()(const stream_buffer &) {
}
};
template<class Ch, class Tr, class A>
class output_to_clog {
public:
typedef std::basic_ostringstream<Ch, Tr, A> stream_buffer;
public:
void operator()(const stream_buffer &s) {
static boost::mutex mutex;
boost::mutex::scoped_lock lock(mutex);
std::clog << now() << ": " << s.str() << std::endl;
}
private:
static std::string now() {
char buf[64];
const time_t tm = time(0);
strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S", localtime(&tm));
return buf;
}
};
template<template <class Ch, class Tr, class A> class OutputPolicy, class Ch = char, class Tr = std::char_traits<Ch>, class A = std::allocator<Ch> >
class logger {
typedef OutputPolicy<Ch, Tr, A> output_policy;
public:
~logger() {
output_policy()(m_SS);
}
public:
template<class T>
logger &operator<<(const T &x) {
m_SS << x;
return *this;
}
private:
typename output_policy::stream_buffer m_SS;
};
}
class log : public detail::logger<detail::output_to_clog> {
};
}
#endif
Usage looks like this:
logger::log() << "this is a test" << 1234 << "testing";
note the lack of a '\n' and std::endl since it's implicit. The contents are buffered and then atomically output using the template specified policy. This implementation also prepends the line with a timestamp since it is for logging purposes. The no_output policy is stricly optional, it's what I use when I want to disable logging.

Answer 3

This:
#define myerr(e) {CiriticalSectionLocker crit; std::cerr << e << std::endl;}
works on most compilers for the common case of myerr("ERR: " << message << number).

Answer 4

Why not just create a locking class and use it where ever you want to do thread-safe IO?
class LockIO
{
static pthread_mutex_t *mutex;
public:
LockIO() { pthread_mutex_lock( mutex ); }
~LockIO() { pthread_mutex_unlock( mutex ); }
};
static pthread_mutex_t* getMutex()
{
pthread_mutex_t *mutex = new pthread_mutex_t;
pthread_mutex_init( mutex, NULL );
return mutex;
}
pthread_mutex_t* LockIO::mutex = getMutex();
Then you put any IO you want in a block:
std::cout <<"X is " <<x <<std::endl;
becomes:
{
LockIO lock;
std::cout <<"X is " <<x <<std::endl;
}

Answer 5

An improvement (that doesn't really fit in a comment) on the approach in unixman's comment.
#define LOCKED_ERR \
if(ErrCriticalSectionLocker crit = ErrCriticalSectionLocker()); \
else std::cerr
Which can be used like
LOCKED_ERR << "ERR: " << message << endl;
if ErrCriticalSectionLocker is implemented carefully.
But, I would personally prefer Ken's suggestion.