glibc malloc deadlock during string reserve - c++

Question

From some time I am trying to catch a problem with my product hanging. This is the call stack I got after coredumping the process with gcore:
/lib64/libc.so.6
#1 0x00007f36fb339ba6 in _L_lock_12192 () from /lib64/libc.so.6
#2 0x00007f36fb337121 in malloc () from /lib64/libc.so.6
#3 0x00007f36fbbef0cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f36fbc4d7e9 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f36fbc4e42b in std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) () from /lib64/libstdc++.so.6
#6 0x00007f36fbc4e4d4 in std::string::reserve(unsigned long) () from /lib64/libstdc++.so.6
#7 0x00007f36fbc2b3c6 in std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow(int) () from /lib64/libstdc++.so.6
#8 0x00007f36fbc2fb36 in std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) () from /lib64/libstdc++.so.6
#9 0x00007f36fbc26885 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) () from /lib64/libstdc++.so.6
#10 0x00007f36fe85c83a in ?? ()
#11 0x00007f36fb2ffdff in vfprintf () from /lib64/libc.so.6
#12 0x000000000055c110 in fgetc#plt ()
#13 0xffffffffffffffa8 in ?? ()
#14 0xffffffffffffffa8 in ?? ()
#15 0x0000000000bc42a0 in ?? ()
#16 0x000000000055b8b0 in setsockopt#plt ()
#17 0x000000000055bf30 in log4cxx::NDC::push ()
This is multithreaded code executed after forking a process in log4cxx ndc push seemingly. All the other threads sleep on condition variables and this is the only one deadlocked. What could possibly make malloc deadlock?

Answer 1

You shouldn't fork processes that use threading.
If one of the other threads has a lock acquired (let's say, the lock in malloc) during the fork, the lock is cloned in locked state, but the thread that locked it isn't running in the forked process. This means that the lock will never be released.