gRPC v1.30.0
I created a grpc service and tried to run it. The execution goes smooth till the last return statement at server side.
Status theService(ServerContext *context, const Request* req, Response* res)
{
Status status = actualLogic(req,res);
//execution goes fine till here
return status;
}
Status actualLogic(req,res)
{
Response_NestedMsg msg;
msg.set_something(...);
res->mutable_nestedmsg()->CopyFrom(msg);
return Status::OK
}
//server startup code
ServerBuilder builder;
builder.AddListeningPort((address),grpc::InsecureServerCredentials());
builder.RegisterService(&serviceClassObj);
std::unique_ptr<Server> server(builder.BuildAndStart());
server->Wait();
Running this code, I get following runtime error
==14394==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x61b00000fcc8 in thread T5 (grpcpp_sync_ser)
#0 0x7fe9d35602c0 in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe12c0)
#1 0x55cb87299afd in __gnu_cxx::new_allocator<std::_List_node<grpc_impl::Server const*> >::deallocate(std::_List_node<grpc_impl::Server const*>*, unsigned long) (/home/john/Desktop/my_executable+0xd0afd)
#2 0x55cb87297ba1 in std::allocator_traits<std::allocator<std::_List_node<grpc_impl::Server const*> > >::deallocate(std::allocator<std::_List_node<grpc_impl::Server const*> >&, std::_List_node<grpc_impl::Server const*>*, unsigned long) (/home/john/Desktop/my_executable+0xceba1)
#3 0x55cb8729448d in std::__cxx11::_List_base<grpc_impl::Server const*, std::allocator<grpc_impl::Server const*> >::_M_put_node(std::_List_node<grpc_impl::Server const*>*) (/home/john/Desktop/my_executable+0xcb48d)
#4 0x55cb8728bb5a in std::__cxx11::_List_base<grpc_impl::Server const*, std::allocator<grpc_impl::Server const*> >::_M_clear() (/home/john/Desktop/my_executable+0xc2b5a)
#5 0x55cb87287307 in std::__cxx11::_List_base<grpc_impl::Server const*, std::allocator<grpc_impl::Server const*> >::~_List_base() (/home/john/Desktop/my_executable+0xbe307)
#6 0x55cb87278d29 in std::__cxx11::list<grpc_impl::Server const*, std::allocator<grpc_impl::Server const*> >::~list() (/home/john/Desktop/my_executable+0xafd29)
#7 0x55cb87278e2c in grpc_impl::CompletionQueue::~CompletionQueue() (/home/john/Desktop/my_executable+0xafe2c)
#8 0x7fe9d1826998 in grpc_impl::Server::SyncRequest::CallData::ContinueRunAfterInterception() (/usr/local/lib/libgrpc++.so.1+0x6f998)
#9 0x7fe9d18278ee in grpc_impl::Server::SyncRequestThreadManager::DoWork(void*, bool, bool) (/usr/local/lib/libgrpc++.so.1+0x708ee)
#10 0x7fe9d182c4ca in grpc::ThreadManager::MainWorkLoop() (/usr/local/lib/libgrpc++.so.1+0x754ca)
#11 0x7fe9d182c68b in grpc::ThreadManager::WorkerThread::Run() (/usr/local/lib/libgrpc++.so.1+0x7568b)
#12 0x7fe9cf5a78d2 in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::{lambda(void*)#1}::_FUN(void*) (/usr/local/lib/libgpr.so.10+0x118d2)
#13 0x7fe9d1eef6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
#14 0x7fe9d0ba8a3e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x121a3e)
0x61b00000fcc8 is located 72 bytes inside of 1448-byte region [0x61b00000fc80,0x61b000010228)
allocated by thread T5 (grpcpp_sync_ser) here:
#0 0x7fe9d355f448 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe0448)
#1 0x7fe9d18274f2 in grpc_impl::Server::SyncRequestThreadManager::DoWork(void*, bool, bool) (/usr/local/lib/libgrpc++.so.1+0x704f2)
Thread T5 (grpcpp_sync_ser) created by T0 here:
#0 0x7fe9d34b6d2f in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37d2f)
#1 0x7fe9cf5a7a92 in grpc_core::Thread::Thread(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&) (/usr/local/lib/libgpr.so.10+0x11a92)
SUMMARY: AddressSanitizer: bad-free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe12c0) in operator delete(void*)
==14394==ABORTING
None of my code tries to free any pointer and error seems to be coming from some auto generated file only. Please suggest if some more code/details needed.
I briefly checked the error message and code but it looks strange to me because both allocation and destruction were done by C++ new & delete. This is also consistent with your error message.
### Destruction (with operator delete)
#0 0x7fe9d35602c0 in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe12c0)
### Allocation (with operator new)
#0 0x7fe9d355f448 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe0448)
This might be caused by other issues like buggy ASAN or customized memory allocator.
Related
I build with brew version of clang++ and use adress sanitizer to look for memory leaks and it gives memory leak on every program even programs without any leak.
clang++ -fsanitize=thread main.cpp -g
int main() {
auto *p = new int;
delete p; // no leak
return 0;
}
I have been using the following commands. I expect there shouldn’t be any leaks however it shows the leaks from system libraries couldn't make any sense.
clang++ -fsanitize=address main.cpp -g
export ASAN_OPTIONS=detect_leaks=1
export MallocNanoZone=0
./a.out
=================================================================
==74341==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1952 byte(s) in 61 object(s) allocated from:
#0 0x1066d25e5 in wrap_calloc+0xa5 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x475e5) (BuildId: e487ca41363b3ac1b8e9e49fecb969fb2400000010000000000a0a0000000d00)
#1 0x7ff81bb972ee in realizeClassWithoutSwift(objc_class*, objc_class*)+0x85 (libobjc.A.dylib:x86_64h+0x52ee) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#2 0x7ff81bb95646 in map_images_nolock+0x160e (libobjc.A.dylib:x86_64h+0x3646) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#3 0x7ff81bb93fda in map_images+0x42 (libobjc.A.dylib:x86_64h+0x1fda) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#4 0x7ff81bbe04c2 in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*), void (*)(mach_header const*, void*, mach_header const*, void const*), void (*)(unsigned int, _dyld_objc_notify_mapped_info const*))+0x27c (dyld:x86_64+0xfffffffffff7e4c2) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#5 0x7ff81bbdaffe in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x2e
Direct leak of 1952 byte(s) in 61 object(s) allocated from:
#0 0x1066d25e5 in wrap_calloc+0xa5 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x475e5) (BuildId: e487ca41363b3ac1b8e9e49fecb969fb2400000010000000000a0a0000000d00)
#1 0x7ff81bb972ee in realizeClassWithoutSwift(objc_class*, objc_class*)+0x85 (libobjc.A.dylib:x86_64h+0x52ee) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#2 0x7ff81bb973ac in realizeClassWithoutSwift(objc_class*, objc_class*)+0x143 (libobjc.A.dylib:x86_64h+0x53ac) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#3 0x7ff81bb95646 in map_images_nolock+0x160e (libobjc.A.dylib:x86_64h+0x3646) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#4 0x7ff81bb93fda in map_images+0x42 (libobjc.A.dylib:x86_64h+0x1fda) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#5 0x7ff81bbe04c2 in invocation function for block in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*), void (*)(mach_header const*, void*, mach_header const*, void const*), void (*)(unsigned int, _dyld_objc_notify_mapped_info const*))+0x27c (dyld:x86_64+0xfffffffffff7e4c2) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#6 0x7ff81bbdaffe in dyld4::RuntimeState::withLoadersReadLock(void () block_pointer)+0x2e (dyld:x86_64+0xfffffffffff78ffe) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#7 0x7ff81bbe023f in dyld4::RuntimeState::setObjCNotifiers(void (*)(unsigned int, char const* const*, mach_header const* const*), void (*)(char const*, mach_header const*), void (*)(char const*, mach_header const*), void (*)(mach_header const*, void*, mach_header const*, void const*), void (*)(unsigned int, _dyld_objc_notify_mapped_info const*))+0x5f (dyld:x86_64+0xfffffffffff7e23f) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#8 0x7ff81bc045e3 in dyld4::APIs::_dyld_objc_register_callbacks(_dyld_objc_callbacks const*)+0x89 (dyld:x86_64+0xfffffffffffa25e3) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#9 0x7ff81bb93e3e in _objc_init+0x4f6 (libobjc.A.dylib:x86_64h+0x1e3e) (BuildId: aca7ef61285336998c1f1c0ab93ad6be32000000200000000100000000000d00)
#10 0x7ff81bd850bf in _os_object_init+0xc (libdispatch.dylib:x86_64+0x20bf) (BuildId: 817339a1d03e3e549c47acacf69f619332000000200000000100000000000d00)
#11 0x7ff81bd92d34 in libdispatch_init+0x16a (libdispatch.dylib:x86_64+0xfd34) (BuildId: 817339a1d03e3e549c47acacf69f619332000000200000000100000000000d00)
#12 0x7ff827b2d894 in libSystem_initializer+0xed (libSystem.B.dylib:x86_64+0x1894) (BuildId: 862b6758852e3e89a4fed564a7163e2532000000200000000100000000000d00)
#13 0x7ff81bbea617 in invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const+0xab (dyld:x86_64+0xfffffffffff88617) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#14 0x7ff81bc29de8 in invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const+0xf1 (dyld:x86_64+0xfffffffffffc7de8) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#15 0x7ff81bc1def6 in invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const+0x22c (dyld:x86_64+0xfffffffffffbbef6) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
...
#23 0x7ff81bbd5368 in dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*)+0xe9e (dyld:x86_64+0xfffffffffff73368) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
#24 0x7ff81bbd4280 in start+0x8f0 (dyld:x86_64+0xfffffffffff72280) (BuildId: 28fd207157f3387387bfe4f674a82de632000000200000000100000000000d00)
SUMMARY: AddressSanitizer: 4288 byte(s) leaked in 134 allocation(s).
and my clang++ version:
Homebrew clang version 15.0.6
Target: x86_64-apple-darwin22.1.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
on macOS 13.0.1
Same situation here as the OP. I don't know what the root cause is (false positive vs. true memory leak in realizeClassWithoutSwift()), but you can configure LeakSanitizer to suppress detected leaks in the memory leak report:
Create a file lsan.supp with
leak:realizeClassWithoutSwift
and then use it when running your binary:
ASAN_OPTIONS=detect_leaks=1 LSAN_OPTIONS=suppressions=lsan.supp my_binary
More details at https://clang.llvm.org/docs/AddressSanitizer.html#suppressing-memory-leaks.
Answer courtesy to GitHub user's willmcpherson2 comment at google/sanitizers/issues/1501.
I'm experimenting with coroutines in C++ using cppcoro. But Address Sanitizer seems always unhappy that I'm leaking memory. Why? I've never allocated anything using raw new/delete and have never cause a circular dependency with shared_ptr.
#include <cppcoro/task.hpp>
#include <cppcoro/sync_wait.hpp>
#include <iostream>
class A {
public:
A() { std::cerr << "A ctor\n"; }
~A() { std::cerr << "A dtor\n"; }
};
int main()
{
auto getACoro = []() -> cppcoro::task<std::shared_ptr<A>> {
co_return std::make_shared<A>();
};
auto foo = [getACoro]() -> cppcoro::task<> {
co_await getACoro();
co_return;
};
auto coro = [&foo]() -> cppcoro::task<> {
co_await foo();
};
cppcoro::sync_wait(coro());
}
Compile with: g++ -std=c++20 -focorutines main.cpp -o main -lcppcoro -g -O -fsanitize=address
Then address sanitizer complains with:
=================================================================
==274677==ERROR: LeakSanitizer: detected memory leaks
Indirect leak of 88 byte(s) in 1 object(s) allocated from:
#0 0x7ff558d7af41 in operator new(unsigned long) /build/gcc/src/gcc/libsanitizer/asan/asan_new_delete.cpp:99
#1 0x560128e69267 in operator() /home/marty/Documents/experiments/cpp/main.cpp:47
#2 0x560128e69267 in main::{lambda()#3}::operator()() const [clone .actor] /home/marty/Documents/experiments/cpp/main.cpp:50
Indirect leak of 80 byte(s) in 1 object(s) allocated from:
#0 0x7ff558d7af41 in operator new(unsigned long) /build/gcc/src/gcc/libsanitizer/asan/asan_new_delete.cpp:99
#1 0x560128e6878e in operator() /home/marty/Documents/experiments/cpp/main.cpp:43
#2 0x560128e6878e in main::{lambda()#2}::operator()() const [clone .actor] /home/marty/Documents/experiments/cpp/main.cpp:45
#3 0x560128f7b997 (/home/marty/Documents/experiments/cpp/main+0x145997)
Indirect leak of 24 byte(s) in 1 object(s) allocated from:
#0 0x7ff558d7af41 in operator new(unsigned long) /build/gcc/src/gcc/libsanitizer/asan/asan_new_delete.cpp:99
#1 0x560128e71168 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) /usr/include/c++/10.2.0/ext/new_allocator.h:115
#2 0x560128e71168 in std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long) /usr/include/c++/10.2.0/bits/allocator.h:173
#3 0x560128e71168 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) /usr/include/c++/10.2.0/bits/alloc_traits.h:460
#4 0x560128e71168 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<A, std::allocator<A>, (__gnu_cxx::_Lock_policy)2> >&) /usr/include/c++/10.2.0/bits/allocated_ptr.h:97
#5 0xbe0b47d5ffffffff (<unknown module>)
SUMMARY: AddressSanitizer: 192 byte(s) leaked in 3 allocation(s).
As far I can tell. The shared_ptr should have destructed right after co_await getACoro();. Why is it considered leaked? And why are the coroutine frames leaking?
Compiler: GCC 10.2
OS: Linux
I compiled my code with -fsanitize=address and I get this error:
==11760==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x6020000050d0 in thread T5:
object passed to delete has wrong type:
size of the allocated type: 8 bytes;
size of the deallocated type: 1 bytes.
#0 0x7f25f2aee938 in operator delete(void*, unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libasan.so.4+0xe1938)
#1 0x7f25f1c94de6 in sql::mysql::MySQL_ParamBind::clearParameters() (/usr/lib64/libmysqlcppconn.so.7+0x6bde6)
#2 0x7f25f1c8f0f2 in sql::mysql::MySQL_Prepared_Statement::closeIntern() (/usr/lib64/libmysqlcppconn.so.7+0x660f2)
#3 0x7f25f1c91f5c in sql::mysql::MySQL_Prepared_Statement::~MySQL_Prepared_Statement() (/usr/lib64/libmysqlcppconn.so.7+0x68f5c)
#4 0x7f25f1c92198 in sql::mysql::MySQL_Prepared_Statement::~MySQL_Prepared_Statement() (/usr/lib64/libmysqlcppconn.so.7+0x69198)
#5 0x555e98524892 in Job::process_job(sql::mysql::MySQL_Connection*) (/src/build/game_server+0x27c892)
#6 0x555e9842ecd5 in DbWorker::run_jobs_thread(int) /src/DbWorker.cpp:136
#7 0x555e984307d2 in DbWorker::run_thread(int) /src/DbWorker.cpp:115
#8 0x555e98425b94 in void std::__invoke_impl<void, void (DbWorker::*)(int), DbWorker*, int>(std::__invoke_memfun_deref, void (DbWorker::*&&)(int), DbWorker*&&, int&&) /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/bits/invoke.h:73
#9 0x555e98425b94 in std::__invoke_result<void (DbWorker::*)(int), DbWorker*, int>::type std::__invoke<void (DbWorker::*)(int), DbWorker*, int>(void (DbWorker::*&&)(int), DbWorker*&&, int&&) /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/bits/invoke.h:95
#10 0x555e98425b94 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)(), (_S_declval<2ul>)())) std::thread::_Invoker<std::tuple<void (DbWorker::*)(int), DbWorker*, int> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/thread:234
#11 0x555e98425b94 in std::thread::_Invoker<std::tuple<void (DbWorker::*)(int), DbWorker*, int> >::operator()() /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/thread:243
#12 0x555e98425b94 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (DbWorker::*)(int), DbWorker*, int> > >::_M_run() /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/g++-v7/thread:186
#13 0x7f25f16fcf9d (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libstdc++.so.6+0xe0f9d)
#14 0x7f25f1edb979 in start_thread (/lib64/libpthread.so.0+0x7979)
#15 0x7f25f0df23ee in clone (/lib64/libc.so.6+0x1013ee)
0x6020000050d0 is located 0 bytes inside of 8-byte region [0x6020000050d0,0x6020000050d8)
allocated by thread T5 here:
#0 0x7f25f2aed3b8 in operator new(unsigned long) (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libasan.so.4+0xe03b8)
#1 0x7f25f1c94272 in sql::mysql::MySQL_Prepared_Statement::setString(unsigned int, sql::SQLString const&) (/usr/lib64/libmysqlcppconn.so.7+0x6b272)
Thread T5 created by T0 here:
#0 0x7f25f2a44b2f in pthread_create (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libasan.so.4+0x37b2f)
#1 0x7f25f16fd2c4 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libstdc++.so.6+0xe12c4)
#2 0x7ffe077cd9f7 (<unknown module>)
SUMMARY: AddressSanitizer: new-delete-type-mismatch (/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/libasan.so.4+0xe1938) in operator delete(void*, unsigned long)
The code that generates that is:
bool Job::process_job(sql::Connection *c) {
try {
std::unique_ptr<sql::PreparedStatement> pstmt;
std::unique_ptr<sql::ResultSet> res;
pstmt.reset(c->prepareStatement(sql_select1));
pstmt->setString(1, player_id);
res.reset(pstmt->executeQuery());
while (res->next()) {
// Processing results...
}
} catch(sql::SQLException &) {
return false;
}
return true;
}
It seems like it happens when unique_ptr goes out of scope and deletes pstmt.
But what is it? Am I doing something wrong?
I use:
x86_64-pc-linux-gnu-7.3.0
mysql-connector-c++ 1.1.6
glibc 2.26-r7
The most interesting fact, that the same code works fine without problems and memory leaks with the same compile options on system:
x86_64-pc-linux-gnu-7.3.0
mysql-connector-c++ 1.1.6
glibc 2.23-r4
I have some weird multithreading bugs, so I wanted to use the gcc thread sanitizer to find them. However, when I compile with -sanitize=thread the resulting binary segfaults immediatly with the following stacktrace:
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff5911cee in ?? () from /usr/lib/x86_64-linux-gnu/libtsan.so.0
#2 0x00007ffff593408d in ?? () from /usr/lib/x86_64-linux-gnu/libtsan.so.0
#3 0x00007ffff58eeb98 in ?? () from /usr/lib/x86_64-linux-gnu/libtsan.so.0
#4 0x00007ffff58ef0ce in __interceptor___cxa_atexit ()
from /usr/lib/x86_64-linux-gnu/libtsan.so.0
#5 0x00007ffff55d1e56 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007ffff7de74ea in call_init (l=<optimised out>, argc=argc#entry=1,
argv=argv#entry=0x7fffffffdeb8, env=env#entry=0x7fffffffdec8)
at dl-init.c:72
#7 0x00007ffff7de75fb in call_init (env=0x7fffffffdec8, argv=0x7fffffffdeb8,
argc=1, l=<optimised out>) at dl-init.c:30
#8 _dl_init (main_map=0x7ffff7ffe168, argc=1, argv=0x7fffffffdeb8,
env=0x7fffffffdec8) at dl-init.c:120
#9 0x00007ffff7dd7cfa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#10 0x0000000000000001 in ?? ()
#11 0x00007fffffffe22b in ?? ()
#12 0x0000000000000000 in ?? ()
I tried adding the -static-libtsan attribute, but now I get the following error:
FATAL: ThreadSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_allocator.h:1203 "((IsAligned(p, page_size_))) != (0)" (0x0, 0x0)
#0 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (sql+0x0000004aeb53)
#1 __tsan::TsanCheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (sql+0x0000004aeb5b)
#2 __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) <null> (sql+0x0000004b3af3)
#3 __tsan::user_free(__tsan::ThreadState*, unsigned long, void*, bool) <null> (sql+0x00000049e6fc)
#4 operator delete(void*) <null> (sql+0x00000045f6ec)
#5 __static_initialization_and_destruction_0(int, int) [clone .constprop.123] <null> (sql+0x00000044a4ab)
#6 __libc_csu_init <null> (sql+0x00000254468c)
#7 __libc_start_main <null> (libc.so.6+0x0000000207be)
#8 _start <null> (sql+0x000000459e18)
How can I fix this? What am I doing wrong here?
I haven't used TSAN w/gcc so can't help there, but you might want to give it a shot with clang (https://clang.llvm.org/). Since TSAN is part of clang, bugs tend to get fixed there first.
If you need help building clang, I've also written an article that might be useful: http://btorpey.github.io/blog/2015/01/02/building-clang/
Answer found here:
https://groups.google.com/forum/#!topic/thread-sanitizer/5cxyhIrl_SE
You don't need to give -ltsan (at least when using clang). You need to
give -fsanitize=thread both when compiling and linking.
-fsanitize=thread when linking will ensure that all proper libraries
get linked in.
This worked for me for both clang and gcc
From some time I am trying to catch a problem with my product hanging. This is the call stack I got after coredumping the process with gcore:
/lib64/libc.so.6
#1 0x00007f36fb339ba6 in _L_lock_12192 () from /lib64/libc.so.6
#2 0x00007f36fb337121 in malloc () from /lib64/libc.so.6
#3 0x00007f36fbbef0cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f36fbc4d7e9 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f36fbc4e42b in std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) () from /lib64/libstdc++.so.6
#6 0x00007f36fbc4e4d4 in std::string::reserve(unsigned long) () from /lib64/libstdc++.so.6
#7 0x00007f36fbc2b3c6 in std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow(int) () from /lib64/libstdc++.so.6
#8 0x00007f36fbc2fb36 in std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) () from /lib64/libstdc++.so.6
#9 0x00007f36fbc26885 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) () from /lib64/libstdc++.so.6
#10 0x00007f36fe85c83a in ?? ()
#11 0x00007f36fb2ffdff in vfprintf () from /lib64/libc.so.6
#12 0x000000000055c110 in fgetc#plt ()
#13 0xffffffffffffffa8 in ?? ()
#14 0xffffffffffffffa8 in ?? ()
#15 0x0000000000bc42a0 in ?? ()
#16 0x000000000055b8b0 in setsockopt#plt ()
#17 0x000000000055bf30 in log4cxx::NDC::push ()
This is multithreaded code executed after forking a process in log4cxx ndc push seemingly. All the other threads sleep on condition variables and this is the only one deadlocked. What could possibly make malloc deadlock?
You shouldn't fork processes that use threading.
If one of the other threads has a lock acquired (let's say, the lock in malloc) during the fork, the lock is cloned in locked state, but the thread that locked it isn't running in the forked process. This means that the lock will never be released.