How to make thread safe Log class that supports `<<` operations? - c++

So I have such log class:
#include <iostream>
#include <sstream>
#include <boost/circular_buffer.hpp>
#include <boost/foreach.hpp>
class FlushInternal;
class Log
{
public:
static FlushInternal* endl;
Log(int log_length)
{
i = 0;
messages_buffer = new boost::circular_buffer<std::string>(log_length);
}
template <class T>
Log &operator<<(const T &v)
{
current_message << v;
return *this;
}
Log &operator<<(std::ostream&(*f)(std::ostream&))
{
current_message << *f;
return *this;
}
Log &operator<<(FlushInternal*)
{
++i;
messages_buffer->push_back(current_message.str());
clean_stringstream(current_message);
is_filled();
return *this;
}
boost::circular_buffer<std::string> *messages_buffer;
private:
int i;
std::stringstream current_message;
void is_filled()
{
if (i >= messages_buffer->capacity())
{
i = 0;
BOOST_FOREACH(std::string s, *messages_buffer)
{
std::cout << ++i << ": " << s << " ;" << std::endl;
}
i = 0;
}
}
void clean_stringstream(std::stringstream &message)
{
message.flush();
message.clear();
message.seekp(0);
message.str("");
}
};
FlushInternal* Log::endl = 0;
And I can Use it like this:
#include <log.h>
int main()
{
Log l(2);
l << "message one: " << 1 << Log::endl;
l << "message two:" << " " << 2 << Log::endl;
l << "message " << "three: " << 3 << Log::endl;
l << "message" << " " << "four: " << 4 << Log::endl;
std::cin.get();
}
This would output:
1: message one: 1 ;
2: message two: 2 ;
1: message three: 3 ;
2: message four: 4 ;
As you can see I can have as many << as I want inside each log message. I want to be capable to use one instance of Log class from many threads at the same time. So I would have something like (pseudocode that compiles, runs but traces nothing.):
#include <boost/thread.hpp>
#include <log.h>
Log *l;
void fun_one()
{
*l << "message one: " << 1 << Log::endl;
*l << "message two:" << " " << 2 << Log::endl;
}
void fun_two()
{
*l << "message " << "three: " << 3 << Log::endl;
*l << "message" << " " << "four: " << 4 << Log::endl;
}
int main()
{
l = new Log(2);
boost::thread(fun_one);
boost::thread(fun_two);
std::cin.get();
}
So as you can see I want messages to be inserted into log in multythreaded function. Lo I wonder - how to make my log cclass support this?

The approach linked by trojanfoe is pretty much the canonical one. Basically create some temporary thing for the leftmost << operator, accumulate everything, and output the message in the destructor for the temporary thing.
The only question is the exact mechanics of this accumulator. The example used ostringstream, but I've seen the ofstream for the log file used directly as well (requires locking to ensure the output ends up on one line).
Creating ostringstreams is relatively expensive on some platforms, because they may need to lock and copy some internal locale related things. You could re-implement also the << operator for interesting types, but I'd test the ostringstream approach first.
A useful optimization is determine at the point of the construction of the temporary whether the trace will be emitted (e.g., whether tracing is enabled at that particular level), and not create the guts of the temporary at all in that case - all the insertion operations will be no-ops.

Here's one approach:
http://drdobbs.com/cpp/201804215
It basically creates a new ostringstream object each time you perform logging, which makes it thread safe. I can't say I'm that keen on that, as it seems a little clumsy to me.
You might have a look at the Qt logging classes as they support the << operator, however I'm not sure about thread safety.

Related

How to automatically set stream mode back to default [duplicate]

This question already has answers here:
Restore the state of std::cout after manipulating it
(9 answers)
Closed 4 years ago.
C++ steam objects have state. If one write a piece of code like
using namespace std;
cout << hex << setw(8) << setfill('0') << x << endl;
forgetting setting the stream state back. This will cause problems in some other unrelated codes. It's tedious to do "set" and "set back" pair matching. Besides from that, it seems to me it's also against convention behind RAII.
My question is: is it possible, with only a thin layer of wrapping, to make those state manipulations RAII-like. That is, right after the end of an expression by semicolon, stream state is automatically set back to default.
Update: Following the link provided by #0x499602D2, one workaround might be something like
#include <boost/io/ios_state.hpp>
#include <ios>
#include <iostream>
#include <ostream>
#define AUTO_COUT(x) {\
boost::io::ios_all_saver ias( cout );\
x;\
}while(0)
Then one can use the macro like
AUTO_COUT(cout << hex << setw(8) << setfill('0') << x << endl);
BTW, it might be a good idea to add a lock field to those saver class of boost::io::ios_state, in case funny things occur in a multi-threading program. Or they have already done so?
I'm going to suggest an alternative approach. The manipulators apply to the std::[i|o]stream instance, but they do nothing with regards to the std::[i|o]streambuf which is managed by that std::[i|o]stream.
Therefore, you can create your own std::[i|o]stream, which will have its own formatting state, but writing to the same buffer std::cout uses:
#include <iostream>
#include <iomanip>
int main()
{
std::cout << std::hex << 32 << "\n";
std::ostream os(std::cout.rdbuf());
os << 32 << "\n" << std::hex;
std::cout << std::dec;
os << 32 << "\n";
std::cout << 32 << "\n";
}
Output:
20
32
20
32
Live on Coliru
This uses only features from standard library, and since the original stream is not touched, applying manipulators is trivially thread safe (because each thread operates on a different stream). Now, the actual writes and reads' thread safety depends on the thread safety of the managed stream buffer.
I once wrote a utility class for my personal use. (I don't know whether it is as perfect as the boost code probably is but it worked for me – so, I dare to share.)
#include <iostream>
#include <iomanip>
/** provides a helper class to work with streams.
*
* It stores current format states of a stream in constructor and
* recovers these settings in destructor.
*
* Example:
* <pre>
* { // backup states of std::cout
* IOSFmtBackup stateCOut(std::cout);
* // do some formatted output
* std::cout
* << "dec: " << std::dec << 123 << std::endl
* << "hex: " << std::hex << std::setw(8) << std::setfill('0')
* << 0xdeadbeef << std::endl;
* } // destruction of stateCOut recovers former states of std::cout
* </pre>
*/
class IOSFmtBackup {
// variables:
private:
/// the concerning stream
std::ios &_stream;
/// the backup of formatter states
std::ios _fmt;
// methods:
public:
/// #name Construction & Destruction
//#{
/** constructor.
*
* #param stream the stream for backup
*/
explicit IOSFmtBackup(std::ios &stream):
_stream(stream), _fmt(0)
{
_fmt.copyfmt(_stream);
}
/// destructor.
~IOSFmtBackup() { _stream.copyfmt(_fmt); }
// disabled:
IOSFmtBackup(const IOSFmtBackup&) = delete;
IOSFmtBackup& operator=(const IOSFmtBackup&) = delete;
//#}
};
int main()
{
{ // backup states of std::cout
IOSFmtBackup stateCOut(std::cout);
// do some formatted output
std::cout
<< "dec: " << std::dec << 123 << std::endl
<< "hex: " << std::hex << std::setw(8) << std::setfill('0')
<< 0xdeadbeef << std::endl
<< "123 in current: " << 123 << std::endl;
} // destruction of stateCOut recovers former states of std::cout
// check whether formatting is recovered
std::cout << "123 after recovered: " << 123 << std::endl;
return 0;
}
Compiled and tested on ideone (life demo).
Output:
dec: 123
hex: deadbeef
123 in current: 7b
123 after recovered: 123

My application is crashing using Do/While loop to parse table

I have 2 QCheckbox tables, each contains 11 elements.
I declare them as following in my class :
QCheckBox *sectionTable[10];
QCheckBox *postTable[10];
For each QCheckBox, I do this
QCheckBox* checkboxA = new QCheckBox("A");
sectionTable[0] = checkboxA;
Through my test method, I would like to return the content of each element of my QCheckbox tables.
To do so, I've done this test :
/** TEST() **/
void VGCCC::test()
{
sectionTable[0]->setText("A");
sectionTable[1]->setText("B");
sectionTable[2]->setText("C");
sectionTable[3]->setText("D");
postTable[0]->setText("E");
postTable[1]->setText("F");
postTable[2]->setText("G");
postTable[3]->setText("H");
int i=0;
do
{
m_testTextEdit->insertPlainText(sectionTable[i]->text());
std::cout << "SECTION TABLE " << sectionTable[i]->text().toStdString() << "\n" << std::endl;
i++;
}
while(!sectionTable[i]->text().isNull());
do
{
m_testTextEdit->insertPlainText(postTable[i]->text());
std::cout << "POST TABLE " << postTable[i]->text().toStdString() << "\n" << std::endl;
i++;
}
while(!postTable[i]->text().isEmpty());
}
My application is compiling, and also running. But when I call the test function, my application crash.
How can we explain this problem ?
I would like to notify that I get a result in my console. It seems my test is half working, but is crashing at the end of the 1st do/while loop, when I get out of my condition.
With regard to the 11 elements: QCheckBox *sectionTable[10]; defines only 10 slots (0 through 9) for elements.
int i=0;
do
{
m_testTextEdit->insertPlainText(sectionTable[i]->text());
std::cout << "SECTION TABLE " << sectionTable[i]->text().toStdString() << "\n" << std::endl;
i++;
}
while(!sectionTable[i]->text().isNull());
Has the potential to reach past ten or eleven elements. Unless the terminating condition is found earlier, there is nothing to stop sectionTable[i] from trying to read sectionTable[11] to call its text method. If it manages to survive the call to the out-of-range sectionTable[11]->text(), it will then try calling sectionTable[11]->text().isNull(). Possibly this will be survivable as well and not be NULL. In this case sectionTable[12] will be tested. This will continue until the program hits really bad memory and crashes, a null is found, or pigs become the terror of the airways we all know they truly wish to be.
Note that i is not set back to 0 after this loop, so the first postTable to be inspected in the next loop will be at the same index as the last sectionTable.
So if sectionTable[5]->text().isNull() was NULL, postTable[5] will be the first postTable indexed and inspected.
do
{
m_testTextEdit->insertPlainText(postTable[i]->text());
std::cout << "POST TABLE " << postTable[i]->text().toStdString() << "\n" << std::endl;
i++;
}
while(!postTable[i]->text().isEmpty());
This loop has the same error in the exit condition as the sectionTable loop.
I find out how to solve the problem. As said in the answer before (#user4581301), I didn't set back my iterator i to 0.
Also, to avoid the "out of range" crash, I put a second condition which is i<sizeof(sectionTable[i]);
This is my fonctional test function :
/** TEST() **/
void VGCCC::test()
{
int i = 0;
do
{
m_testTextEdit->insertPlainText(sectionTable[i]->text());
std::cout << "SECTION TABLE " << m_materialMap[sectionTable[i]].c_str() << "\n" << std::endl;
i++;
}
while(!sectionTable[i]->text().isNull() && i<sizeof(sectionTable[i]));
i = 0;
do
{
m_testTextEdit->insertPlainText(postTable[i]->text());
std::cout << "POST TABLE " << postTable[i]->text().toStdString() << "\n" << std::endl;
std::cout << "POST TABLE " << m_materialMap[postTable[i]].c_str() << "\n" << std::endl;
i++;
}
while(!postTable[i]->text().isEmpty() && i<sizeof(postTable[i]));
}

Logging function which uses operator <<

I'd like to write a function for logging which should be used like this:
log(__FILE__) << "My message containing integer: " << 123 << " and double: " << 1.2;
This should print the following line, add endl and flush immediately:
main.cpp: My message containing integer: 123 and double: 1.2
My (simplified) attempt for the implementation of the function:
class Writer
{
public:
template<typename T>
Writer & operator<<(T t)
{
cout << t << endl;
cout.flush();
return (*this);
}
};
Writer log(const char* fileName)
{
cout << fileName << ": ";
return Writer();
}
int main(int argc, const char *argv[])
{
log(__FILE__) << "My message containing integer: " << 123 << "and double: " << 1.2;
return 0;
}
My problem is that because of L-R associativity of the operator<< the output is:
main.cpp: My message containing integer:
123
and double:
1.2
Is there any way how to implement the function or is my requirement for its usage unrealizable?
Ideally I'd like to use plain C++03 (i.e. no C++11 features, boost and non-standard libraries).
L-R associativity is not related to your problem (if you talk about line breaks). The problem is because you use endl after each write. You don't need it (and if you do that, then you don't need flush, because endl already flushes the output).
The easy solution to your problem:
class Writer
{
public:
template<typename T>
Writer & operator<<(T t)
{
cout << t;
return (*this);
}
~Writer()
{
try {
cout << endl;
}
catch (...) {
// You have to make sure that no
// exception leaves destructor
}
}
};
It is also worth to notice, that your approach is not really scalable: it is impossible to use your code in multi-threaded environment. Assume that two threads are writing into your logging:
Thread 1: log(__FILE__) << "a" << "b" << "c";
Thread 2: log(__FILE__) << "a" << "b" << "c";
Here you can easily get a message "aabbcc\n\n" in your logfile, which is highly undesirable.
In order to avoid that, you can have a static mutex object inside log() function, which you pass into Writer constructor. Then you have to lock it in the constructor and unlock it in the destructor. It will guarantee the synchronization of concurrent writing of different entries.

multiple threads writing to std::cout or std::cerr

I have OpenMP threads that write to the console via cout and cerr. This of course is not safe, since output can be interleaved. I could do something like
#pragma omp critical(cerr)
{
cerr << "my variable: " << variable << endl;
}
It would be nicer if could replace cerr with a thread-safe version, similar to the approach explained in the valgrind DRD manual (http://valgrind.org/docs/manual/drd-manual.html#drd-manual.effective-use) which involves deriving a class from std::ostreambuf. Ideally in the end I would just replace cerr with my own threaded cerr, e.g. simply:
tcerr << "my variable: " << variable << endl;
Such a class could print to the console as soon as it encounters an "endl". I do not mind if lines from different threads are interleaved, but each line should come only from one thread.
I do not really understand how all this streaming in C++ works, it is too complicated. Has anybody such a class or can show me how to create such a class for that purpose?
As others pointed out, in C++11, std::cout is thread-safe.
However if you use it like
std::cout << 1 << 2 << 3;
with different threads, the output can still be interleaved, since every << is a new function call which can be preceeded by any function call on another thread.
To avoid interleaving without a #pragma omp critical - which would lock everything - you can do the following:
std::stringstream stream; // #include <sstream> for this
stream << 1 << 2 << 3;
std::cout << stream.str();
The three calls writing 123 to the stream are happening in only one thread to a local, non-shared object, therefore aren't affected by any other threads. Then, there is only one call to the shared output stream std::cout, where the order of items 123 is already fixed, therefore won't get messed up.
You can use an approach similar to a string builder. Create a non-template class that:
offers templated operator<< for insertion into this object
internally builds into a std::ostringstream
dumps the contents on destruction
Rough approach:
class AtomicWriter {
std::ostringstream st;
public:
template <typename T>
AtomicWriter& operator<<(T const& t) {
st << t;
return *this;
}
~AtomicWriter() {
std::string s = st.str();
std::cerr << s;
//fprintf(stderr,"%s", s.c_str());
// write(2,s.c_str(),s.size());
}
};
Use as:
AtomicWriter() << "my variable: " << variable << "\n";
Or in more complex scenarios:
{
AtomicWriter w;
w << "my variables:";
for (auto & v : vars) {
w << ' ' << v;
}
} // now it dumps
You will need to add more overloads if you want manipulators, you can use write better than fprintf for the atomic write in the destructor, or std::cerr, you can generalize so that the destination is passed to the constructor (std::ostream/file descriptor/FILE*),
I don't have enough reputation to post a comment, but I wanted to post my addition to the AtomicWriter class to support std::endl and allow for other streams to be used besides std::cout. Here it is:
class AtomicWriter {
std::ostringstream st;
std::ostream &stream;
public:
AtomicWriter(std::ostream &s=std::cout):stream(s) { }
template <typename T>
AtomicWriter& operator<<(T const& t) {
st << t;
return *this;
}
AtomicWriter& operator<<( std::ostream&(*f)(std::ostream&) ) {
st << f;
return *this;
}
~AtomicWriter() { stream << st.str(); }
};
Put the following code in header file atomic_stream_macro.h
#ifndef atomic_stream_macro_h
#define atomic_stream_macro_h
#include <mutex>
/************************************************************************/
/************************************************************************/
extern std::mutex print_mutex;
#define PRINT_MSG(out,msg) \
{ \
std::unique_lock<std::mutex> lock (print_mutex); \
\
out << __FILE__ << "(" << __LINE__ << ")" << ": " \
<< msg << std::endl; \
}
/************************************************************************/
/************************************************************************/
#endif
Now the macro can be used from a file as follows.
#include <atomic_stream_macro.h>
#include <iostream>
int foo (void)
{
PRINT_MSG (std::cout, "Some " << "Text " << "Here ");
}
Finally, in the main.cxx, declare the mutex.
#include <mutex>
std::mutex print_mutex;
int main (void)
{
// launch threads from here
}
You could do it by inheriting std::basic_streambuf, and override the correct functions to make it threadsafe. Then use this class for your stream objects.

Avoid using container to call a list of functions?

I have a list of functions that return bools. I want to iterate through the list of functions and write a message for each one "Test 1 passed", "Test 2 failed" etc.
My current solution is to create a vector of function pointers, push back each function and then loop through the vector. Code below. Is there a way to avoid the container without repeating the generic message (pass/fail) code for each test (imagine there would be hundreds of tests). It feels as if the vector is unnecessary or that there must be a more elegant solution for this.
typedef bool (*Tests)();
std::vector<Tests> tests;
tests.push_back(FASTA_FILE_READER_TEST);
tests.push_back(EXACT_MATCH_TEST);
for (int i = 0; i < tests.size(); i++) {
std::cout << "Test " << i + 1
<< (tests[i]() ? " PASSED" : " FAILED")
<< std::endl;
}
Is there anything stopping you using an array?
#include <iostream>
bool FASTA_FILE_READER_TEST() { return false; }
bool EXACT_MATCH_TEST() { return false; }
int main()
{
typedef bool (*Tests)();
Tests tests[] = {FASTA_FILE_READER_TEST, EXACT_MATCH_TEST};
for (int i = 0; i < sizeof(tests)/sizeof(Tests); i++) {
std::cout << "Test " << i + 1
<< (tests[i]() ? " PASSED" : " FAILED")
<< std::endl;
}
}
You could use a function to do that:
template<typename Functor>
void test(Functor& functor){
static int i = 0;
bool ret = functor();
if(ret){
std::cout << "Test " << i++ << " passed" << std::endl;
} else {
std::cout << "Test " << i++ << " failed" << std::endl;
}
}
void main(){
test(FASTA_FILE_READER_TEST);
test(EXACT_MATCH_TEST);
}
If you can use C++11 features:
#include <array>
#include <iterator>
#include <algorithm>
#include <iostream>
typedef bool (*Test)();
std::array<Test, 2> tests {{ FASTA_FILE_READER_TEST, EXACT_MATCH_TEST }};
void TestAll()
{
size_t i = 1;
std::for_each(std::begin(tests), std::end(tests),
[&i](Test& t)
{
std::cout << "Test " << i++ << (t() ? " PASSED" : " FAILED") << std::endl;
});
}
Demo.
It's another way of doing what you've already got (and your way is just fine, IMO). If the extra capacity a vector might have set aside bothers you, you can call shrink_to_fit() on it when you're done pushing back.
Create a class for each test. Then one static instance of each class.
Contructors of classes runs tests.
This of course may cause problems, because tests are executed before main() function is called.