LLVM IR codegen segfaults during exit only when method declarations have parameters - c++

Explanation
I am creating a compiler for a C-like language using yacc/bison, flex, and the LLVM toolchain (LLVM 12) using the LLVM C++ API.
I have been developing and testing on Ubuntu version 20.04.3 LTS (Focal Fossa) and macOS 11.6 Big Sur.
Currently, the issue is the program segfaulting when exiting the program when a method declaration has method parameters such as simply:
func test(x int) void {}
The LLVM IR will be printed properly as
; ModuleID = 'Test'
source_filename = "Test"
define void #test(i32 %x) {
entry:
%x1 = alloca i32, align 4
store i32 %x, i32* %x1, align 4
ret void
}
And will segfault immediately after.
A method declaration like
func test() int {
var x int;
x = 5;
return (x);
}
Will not segfault.
GDB reports that the segfault occurs during llvm::LLVMContextImpl::~LLVMContextImpl(). Valgrind reports ~LLVMContextImpl() doing an invalid read of size 8.
Edit: Valgrind output relating to invalid read
==10254== Invalid read of size 8
==10254== at 0x5553C30: llvm::LLVMContextImpl::~LLVMContextImpl() (in /usr/lib/x86_64-linux-gnu/libLLVM-12.so.1)
==10254== by 0x5552130: llvm::LLVMContext::~LLVMContext() (in /usr/lib/x86_64-linux-gnu/libLLVM-12.so.1)
==10254== by 0xA44AA26: __run_exit_handlers (exit.c:108)
==10254== by 0xA44ABDF: exit (exit.c:139)
==10254== by 0xA4280B9: (below main) (libc-start.c:342)
==10254== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==10254==
==10254==
==10254== Process terminating with default action of signal 11 (SIGSEGV)
==10254== Access not within mapped region at address 0x0
==10254== at 0x5553C30: llvm::LLVMContextImpl::~LLVMContextImpl() (in /usr/lib/x86_64-linux-gnu/libLLVM-12.so.1)
==10254== by 0x5552130: llvm::LLVMContext::~LLVMContext() (in /usr/lib/x86_64-linux-gnu/libLLVM-12.so.1)
==10254== by 0xA44AA26: __run_exit_handlers (exit.c:108)
==10254== by 0xA44ABDF: exit (exit.c:139)
==10254== by 0xA4280B9: (below main) (libc-start.c:342)
==10254== If you believe this happened as a result of a stack
==10254== overflow in your program's main thread (unlikely but
==10254== possible), you can try to increase the size of the
==10254== main thread stack using the --main-stacksize= flag.
==10254== The main thread stack size used in this run was 8388608.
I'm hoping that by asking here I can get some kind of hint for how to work towards solving this issue. I've been stuck on this for days.
Source Code Fragments
The sections of my code relating to method declarations and method parameters are as follow, I apologize for the length:
Bison grammar rule for program
program: extern_list decafpackage
{
ProgramAST *prog = new ProgramAST((decafStmtList*)$1, (PackageAST*)$2);
if (printAST) {
cout << getString(prog) << endl;
}
prog->Codegen();
delete prog;
}
;
Bison grammar rule for method declaration
method_decl: T_FUNC T_ID T_LPAREN params T_RPAREN method_type method_block
{
$$ = new Method(*$2, $6->str(), $4, $7);
delete $2;
delete $6;
}
Bison grammar rule for method parameter
param: T_ID type { $$ = new VarDef(*$1, $2->str()); delete $1; delete $2; }
;
C++ Method::Codegen() handling of parameters
llvm::Function *func = llvm::Function::Create(
llvm::FunctionType::get(returnTy, args, false),
llvm::Function::ExternalLinkage,
name,
TheModule
);
llvm::BasicBlock *BB = llvm::BasicBlock::Create(TheContext, "entry", func);
Builder.SetInsertPoint(BB);
. . .
for (auto &Arg : func->args()) {
llvm::AllocaInst* Alloca = CreateEntryBlockAlloca(func, Arg.getName().str());
Builder.CreateStore(&Arg, Alloca);
sTStack->enter_symtbl(Arg.getName().str(), Alloca);
}
C++ VarDef::Codegen()
llvm::Value *Codegen() {
llvm::Type* ty = getLLVMType(type);
llvm::AllocaInst* V = Builder.CreateAlloca(ty, 0, name);
V->setName(name);
sTStack->enter_symtbl(name, V);
return V;
return nullptr;
}
Bison main
int main() {
// Setup
llvm::LLVMContext &Context = TheContext;
TheModule = new llvm::Module("Test", Context);
FPM = std::make_unique<llvm::legacy::FunctionPassManager>(TheModule);
FPM->add(llvm::createPromoteMemoryToRegisterPass());
FPM->add(llvm::createInstructionCombiningPass());
FPM->add(llvm::createReassociatePass());
FPM->add(llvm::createGVNPass());
FPM->add(llvm::createCFGSimplificationPass());
FPM->doInitialization();
int retval = yyparse();
TheModule->print(llvm::errs(), nullptr);
return(retval >= 1 ? EXIT_FAILURE : EXIT_SUCCESS);
}

Solution:
The problem was in lines of code not included. llvm::Function::Create requires an llvm::FunctionType which can be provided by filling a vector with llvm::Type* objects. I wrote a function to do this:
void getLLVMTypes(vector<llvm::Type*>* v) {
for (auto* i : stmts) {
llvm::Type* type = getLLVMType(i->getType());
((llvm::Value*)(type))->setName(i->getName()); // Problem
v->push_back(type);
}
}
The issue was casting each llvm::Type* object to llvm::Value* and using llvm::Value::setName to set its name. I did this to counter a problem I had earlier with parameter names not being set. I'm not entirely sure what the issue was, I had trouble compiling LLVM from source with debug flags, but it's a gnarly looking line of code and removing it, along with using an alternative way to preserve method parameter names, solved the issue.

Related

in multithread program does bt a coredump always gives the culprit thread?

this is a little bit general question,
I have a segfault in a multithreaded program, and bt coredump shows below,
(gdb) bt full
#0 0x0000000000441540 in try_dequeue<std::shared_ptr<Frame> > (item=<synthetic pointer>, this=0xbe3c50) at /root/projects/active/user/include/third_party/concurrentqueue.h:1111
nonEmptyCount = 0
best = 0x0
bestSize = 0
#1 ConsumerNice::listening_nice (this=0xbe3c40) at /root/projects/active/user/include/concurrency/consumer_nice.h:45
frame = std::shared_ptr (empty) 0x0
#2 0x00000000004c0530 in execute_native_thread_routine ()
No symbol table info available.
#3 0x00007f3eb3f81e65 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#4 0x00007f3ead70a88d in clone () from /lib64/libc.so.6
No symbol table info available.
So I go to look at the source code,
my code as below
void listening_nice() {
while (true) {
std::shared_ptr<Frame> frame;
if (nice_queue.try_dequeue(frame)) {
on_frame_nice(frame);
}
}
}
and cameron314/concurrentqueue part look like below,
bool try_dequeue(U& item)
{
// Instead of simply trying each producer in turn (which could cause needless contention on the first
// producer), we score them heuristically.
size_t nonEmptyCount = 0;
ProducerBase* best = nullptr;
size_t bestSize = 0;
for (auto ptr = producerListTail.load(std::memory_order_acquire); nonEmptyCount < 3 && ptr != nullptr; ptr = ptr->next_prod()) {
auto size = ptr->size_approx();
if (size > 0) {
if (size > bestSize) {
bestSize = size;
best = ptr;
}
++nonEmptyCount;
}
}
It doesnt seem possible to cause segfault, therefore I am wondering, is bt always show the culprit thread? or there is a chance segfault is caused by some other problem in some other thread, or even the operating system?
Noted this program is running on 3 same configured machine, but only one machine crashes once a day, that is it runs for 3 straight hours on that one machine, then crashed.

Runtime free error while deleting user space stack

I am trying to create a user space stack and run functions using this stack.
Following are the steps I am doing.
Allocate a space for the stack.
Before calling a function, switch the stack pointer to the new location.
Also try to store previous or original stack pointer
Return of the called function
Restore the original stack pointer
I implemented the following based on the above points:
/* Ubuntu 18.04 x86_64 */
/* Compiler gcc version 7.5.0 */
#include <iostream>
#include <cassert>
#include <memory>
class Thread {
public:
/* stack will point to lowest addr of th User Stack */
unsigned char * stack = nullptr;
/*stack_ptr will point to the Highest addr of the User Stack */
unsigned char* stack_ptr = nullptr;
int stack_size = 0;
Thread() {
stack_size = 16*64; // 1024 B
stack = new unsigned char[stack_size];
/*stack_ptr points to the highest memory of this space*/
stack_ptr = stack + stack_size;
}
~Thread() {
std::cout <<"User stack deleted"<< std::endl;
if(stack != nullptr)
delete[] stack;
}
void func(int x,int y) {
std::cout <<"Hello from the func ["<<x<<"] ["<<y<<"]"<< std::endl;
}
};
int main() {
/* createa a User level Thread */
Thread t;
/* Some random local variable */
// int a[100];
unsigned char* stk = t.stack_ptr;
/* Note: rsp = current stack top */
uint64_t prev_stack = 0; /* for assert checking */
uint64_t x = 7; /*some random value, just for stack alighment of 16B */
asm("movq %%rsp, %%r11\n\t" /* save current stack ptr to r11(or rax etc.) */
"movq %%r11, %[out1]\n\t" /* just for assert check later */
"movq %[in1], %%rsp\n\t" /* change rsp to stk, i.e. to our own stack top */
"pushq %%r11\n\t" /* Push r11 or prev stack ptr to current stack */
"pushq %[in2]\n\t" /* for stack alignment only */
:[out1] "=m" (prev_stack) /*just for assert check later */
:[in1] "m" (stk),[in2] "m" (x) /* two inputs */
:);
// call the func, this func
// will will use our own stack
t.func(1,2);
/*just for checking x */
uint64_t y;
uint64_t _prev_stack;
asm(
"popq %[out1]\n\t" /* y = stack.pop() */
"popq %%rsp\n\t" /* restore stack: rsp = stack.pop() */
"movq %%rsp, %[out2]\n\t" /* just for checking */
:[out1] "=m" (y),[out2] "=m" (_prev_stack) /* two outputs */
:
:
);
assert(prev_stack == _prev_stack && "Main's stack not restored!");
std::cout <<std::hex<<"Stack restored = "<< prev_stack << std::endl;
assert(x == y && "x and y are not equal!!");
std::cout <<"Program Ended!" << std::endl;
}
I have added couple of extra checks, just for debug purpose.
The following output is generated:
Hello from the func [1] [2]
Stack restored = 7ffd55eb8ea0
Program Ended!
User stack deleted
munmap_chunk(): invalid pointer
[1] 27337 abort (core dumped) ./a.out
It seems that, I can safely (?) return to the original function, because the instruction pointer is restored automatically (due to callq and retq operation sequence, gcc compiler did this) and the program also gets back the original rsp or the stack top. The rbp pointer is also taken care of by the compiler.
But I get an error in the destructor of the Thread class.
I tried to debug on my own using gdb, but I could not find any reason for observed the error:
Here are the output of the gdb backtrace
_GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
>>> bt
#0 __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00001555545f2801 in __GI_abort () at abort.c:79
#2 0x000015555463b897 in __libc_message (action=action#entry=do_abort, fmt=fmt#entry=0x155554768b9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3 0x000015555464290a in malloc_printerr (str=str#entry=0x15555476a7a8 "munmap_chunk(): invalid pointer") at malloc.c:5350
#4 0x0000155554649ecc in munmap_chunk (p=0x614e60) at malloc.c:2846
#5 __GI___libc_free (mem=0x614e70) at malloc.c:3117
#6 0x0000000000400e2d in Thread::~Thread() ()
#7 0x0000000000400bdc in main ()
However valgrind shows no leaks.
==27982== HEAP SUMMARY:
==27982== in use at exit: 0 bytes in 0 blocks
==27982== total heap usage: 3 allocs, 3 frees, 74,752 bytes allocated
==27982==
==27982== All heap blocks were freed -- no leaks are possible
(Although valgrind warns : Warning: client switching stacks?, I think it is normal w.r.t to the above code)
Observation: If I comment out the if(stack != nullptr) delete[] stack line, error goes out
but valgrind shows the following:
=28019== 1,024 bytes in 1 blocks are definitely lost in loss record 1 of 1
==28019== at 0x4C3089F: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28019== by 0x108ED7: Thread::Thread() (test.cpp:20)
==28019== by 0x108BBD: main (test.cpp:36)
I am really confused with the memory free error and no leak status. What could be the cause of such memory deallocation error ? Am I doing any mistake while saving and restoring old and new stacks ?
Thanks.
Update
I simplified the code:
/* Ubuntu 18.04 x86_64 */
/* Compiler gcc version 7.5.0 */
#include <iostream>
#include <cassert>
#include <memory>
const int _stack_size = 16*64; // 1024B
void func(int x,int y) {
std::cout <<"Hello from the func ["<<x<<"] ["<<y<<"]"<< std::endl;
}
int main() {
unsigned char * stackBase = new unsigned char[_stack_size];
unsigned char * stackTop = stackBase + _stack_size;
std::cout <<"Before "<<std::hex <<(void*)(stackBase) << std::endl;
asm volatile("movq %%rsp, %%r11\n\t" /* save current stack ptr to r11(or rax etc.) */
"movq %[in1], %%r12\n\t"
"movq %%r12, %%rsp\n\t" /* change rsp to stk, i.e. to our own stack top */
"pushq %%r11\n\t" /* Push r11 or prev stack ptr to current stack */
// "pushq %%rbp\n\t"
: /*no input */
:[in1] "m" (stackTop) /* two inputs */
:"r11","r12","rsp");
func(1,2);
asm volatile(
"popq %%rsp\n\t" /* restore stack: rsp = stack.pop() */
:/* no outputs */
:/* no inputs */
:"rsp");
std::cout <<"Program Ended!" << std::endl;
std::cout <<"After "<<std::hex<<(void*)(stackBase) << std::endl;
delete[] stackBase;
}
But it still shows the double free error:
Before 0x555e80926e70
Hello from the func [1] [2]
Program Ended!
After 0x555e80926e70
double free or corruption (out)
[1] 4343 abort (core dumped) ./a.out
Error in runtine with -fsanitize=address option:
Before 0x619000000080
ASAN:DEADLYSIGNAL
=================================================================
==5605==ERROR: AddressSanitizer: stack-overflow on address 0x618ffffffb90 (pc 0x14b656131576 bp 0x619000000418 sp 0x618ffffffb98 T0)
#0 0x14b656131575 (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x51575)
#1 0x14b655e39039 in std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) (/usr/local/gcc-10/lib64/libstdc++.so.6+0x12c039)
#2 0x564202743e4b in func(int, int) (/home/deka/Academic/userLevelThread/a.out+0x2e4b)
#3 0x5642027445ca in main (/home/deka/Academic/userLevelThread/a.out+0x35ca)
#4 0x14b654c1cb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
#5 0x564202743d49 in _start (/home/deka/Academic/userLevelThread/a.out+0x2d49)
SUMMARY: AddressSanitizer: stack-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x51575)
==5605==ABORTING

cygwin exception when assigning value to vector of strings

I am having following exception during the course of the run of program:
0 [main] myFunction 5560 cygwin_exception::open_stackdumpfile: Dumping stack trace to myFunction.exe.stackdump
The contents of stackdump file are as follows:
Stack trace:
Frame Function Args
00000223800 0018006FB93 (0060007AE38, 00600083EC8, 00600083EF8, 00600083F28)
00000000006 0018007105A (0060007BB78, 00600000000, 0000000014C, 00000000000)
000002239E0 0018011C6A7 (00600083048, 00600083078, 006000830A8, 006000830D8)
00000000041 001801198DE (0060007DCB8, 0060007DCE8, 00000000000, 0060007DD48)
0060008F2B0 00180119DAB (0060007E1F8, 0060007E228, 0060007E258, 00000000006)
0060008F2B0 00180119F7C (0060007CB38, 0060007CB68, 0060007CB98, 0060007CBC8)
0060008F2B0 0018011A23F (00180115A0B, 0060007CCE8, 006000885B0, 00000000000)
0060008F2B0 00180148A65 (003FC4AA93D, 00600083900, 00100439102, 0060007B080)
0060008F2B0 001800C1DB3 (00000000000, 00000223EE0, 0010042A2BC, 00000223E90)
0060008F2B0 00180115A0B (00000223EE0, 0010042A2BC, 00000223E90, 00000000017)
0060008F2B0 00600000001 (00000223EE0, 0010042A2BC, 00000223E90, 00000000017)
End of stack trace
Let me describe in detail the peculiar problem which happens at runtime. I am not able to describe the problem with just words, so I am listing scenario when the program works and when it fails.
I have created a vector of string in my header file and initialised them in the constructor as follows :
std::vector <std::string> symbolMap,localSymbolMap;
for(int i=0;i<100;i++){
symbolMap.push_back(" ");
localSymbolMap.push_back(" ");
}
I have defined a function to assign appropriate value to these variables later in the program as follows :
void TestClient::setTickerMap(int j, std::string symbol, std::string localSymbol){
symbolMap[j] = symbol;
localSymbolMap[j]=localSymbol;
}
Now, in the main program, I call this function as follows:
TestClient client;
for(int j=0;j<27;j++){
std::cout<<j<<" "<<realTimeSymbols[j]<<" "<<getLocalSymbol(realTimeSymbols[j],date)<<std::endl;
client.setTickerMap(j,realTimeSymbols[j],getLocalSymbol(realTimeSymbols[j],date));
}
// Here, I have checked for each j, that values of realTimeSymbols and getLocalSymbol are proper.
When I run the program, I get the error described above. The program always crashed when j is equal to 24.
Now the following workaround is working as of now:
void TestClient ::setTickerMap(int j, std::string symbol, std::string localSymbol){
if(j==24){
// symbolMap[j]="SYNDIBANK";
// localSymbolMap[j]="SYNDIBANK15MARFUT";
}
else{
symbolMap[j] = symbol;
localSymbolMap[j]=localSymbol;
}
if(j==1){
symbolMap[24]="SYNDIBANK";
localSymbolMap[24]="SYNDIBANK15MARFUT";
}
}
Following 3 variations of the code are above workaround are not working and they result in the original error:
Variation 1:
void TestClient ::setTickerMap(int j, std::string symbol, std::string localSymbol){
if(j==24){
// symbolMap[j]="SYNDIBANK";
// localSymbolMap[j]="SYNDIBANK15MARFUT";
}
else{
symbolMap[j] = symbol;
localSymbolMap[j]=localSymbol;
}
if(j==25){
symbolMap[24]="SYNDIBANK";
localSymbolMap[24]="SYNDIBANK15MARFUT";
}
}
Variation 2:
void TestClient ::setTickerMap(int j, std::string symbol, std::string localSymbol){
if(j==24){
symbolMap[j]="SYNDIBANK";
localSymbolMap[j]="SYNDIBANK15MARFUT";
}
else{
symbolMap[j] = symbol;
localSymbolMap[j]=localSymbol;
}
}
Variation 3:
void TestClient ::setTickerMap(int j, std::string symbol, std::string localSymbol){
if(j==24){
symbolMap[j]="AB";
localSymbolMap[j]="SYNDIBANK15MARFUT";
}
else{
symbolMap[j] = symbol;
localSymbolMap[j]=localSymbol;
}
}
Now, if I assign a single character to symbolMap in variation 3 as follows :
symbolMap[j]="A";
then the code is able to run(although is the result is not correct).
I am not able to figure what exactly is causing this runtime error. I have checked the related question (Cygwin Exception : open stack dump file) and I do not have a separate session of cygwin running. I have restarted my PC just be extra sure. Still the problem persists. Any suggestions as to why this behaviour is seen on my PC.
UPDATE:
To be sure that the error is not related to out-of-index, the following call from main program works fine:
TestClient client;
for(int j=25;j<27;j++){
std::cout<<j<<" "<<realTimeSymbols[j]<<" "<<getLocalSymbol(realTimeSymbols[j],date)<<std::endl;
client.setTickerMap(j,realTimeSymbols[j],getLocalSymbol(realTimeSymbols[j],date));
}
The program also works fine when j is iterated from 24 to 27. But fails when the loop is iterated from any number before 24 to 27.
GDB OUTPUT
I do not have much experience with gdb but following is the output of the gdb if it helps:
GNU gdb (GDB) 7.8
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-cygwin".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from order_trading2632_limit.exe...done.
(gdb) run
Starting program: /cygdrive/e/eclipse_workspace/testClient/Debug/testClient.exe
[New Thread 4832.0x11e4]
[New Thread 4832.0x1798]
Attempt 1 of 10000
[New Thread 4832.0x1020]
Connection successful
Program received signal SIGABRT, Aborted.
0x00000003fc4ab0e3 in cygstdc++-6!_ZNSs6assignERKSs () from /usr/bin/cygstdc++-6.dll
(gdb) bt
#0 0x00000003fc4ab0e3 in cygstdc++-6!_ZNSs6assignERKSs () from /usr/bin/cygstdc++-6.dll
#1 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) set $pc=*(void **)$rsp
(gdb) set $rsp=$rsp+8
(gdb) bt
#0 0x000007fefd3110ac in WaitForSingleObjectEx () from /cygdrive/c/Windows/system32/KERNELBASE.dll
#1 0x000000018011c639 in sig_send(_pinfo*, siginfo_t&, _cygtls*) () from /usr/bin/cygwin1.dll
#2 0x00000001801198de in _pinfo::kill(siginfo_t&) () from /usr/bin/cygwin1.dll
#3 0x0000000180119dab in kill0(int, siginfo_t&) () from /usr/bin/cygwin1.dll
#4 0x0000000180119f7c in raise () from /usr/bin/cygwin1.dll
#5 0x000000018011a23f in abort () from /usr/bin/cygwin1.dll
#6 0x0000000180148a65 in dlfree () from /usr/bin/cygwin1.dll
#7 0x00000001800c1db3 in free () from /usr/bin/cygwin1.dll
#8 0x0000000180115a0b in _sigfe () from /usr/bin/cygwin1.dll
#9 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Note that stack trace is corrupted and I have used trick from following question to print the stacktrace (GDB corrupted stack frame - How to debug?). Please help me in debugging the program further.
UPDATE
It is not the case that the error happens only when index is 24. Before calling the said loop, I initialize various arrays of int, double and string. Changing the number of initialization affects the index when this error happens. Today, I initialised vectors of length 24 before running this loop, this time the error happened at index 3.
This is really frustrating to implement the workaround. I do not that if there are some other memory issues I am overlooking because of this. Please offer suggestions.
CODE
int main(int argc, char** argv) {
unsigned int port = 7900;
const char* host = "";
int clientId = 6;
int attempt = 0;
int MAX_ATTEMPTS=10000;
int NUMREALTIMESYMBOLS=37;
std::string realTimeSymbolsArr[]={"a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z","aa","bb","cc","dd","ee","ff","gg","hh","ii","jj","kk"};
std::vector <std::string> realTimeSymbols(realTimeSymbolsArr,realTimeSymbolsArr+NUMREALTIMESYMBOLS);
int isTradeable[]={1,0,0,0,1,0,1,1,1,1,1,0,0,1,0,1,1,0,0,0,0,1,0,0,0,0,0,1,1,1,0,1,0,1,1,0,1};
int numSubscriptions[]={2,2,1,4,1,1,2,6,3,1,1,1,2,1,3,1,1,2,1,3,1,1,1,10,4,1,6,1,1,9,4,2,1,3,1,1,2};
int subscriptionList[NUMREALTIMESYMBOLS][100];
int subscriptionIndex[NUMREALTIMESYMBOLS][100];
subscriptionList[0][0]=0;subscriptionIndex[0][0]=0;
subscriptionList[0][1]=2;subscriptionIndex[0][1]=2;
subscriptionList[1][0]=2;subscriptionIndex[1][0]=1;
subscriptionList[1][1]=0;subscriptionIndex[1][1]=3;
subscriptionList[2][0]=2;subscriptionIndex[2][0]=0;
subscriptionList[3][0]=4;subscriptionIndex[3][0]=2;
subscriptionList[3][1]=31;subscriptionIndex[3][1]=2;
subscriptionList[3][2]=13;subscriptionIndex[3][2]=3;
subscriptionList[3][3]=34;subscriptionIndex[3][3]=3;
subscriptionList[4][0]=4;subscriptionIndex[4][0]=0;
subscriptionList[5][0]=9;subscriptionIndex[5][0]=2;
subscriptionList[6][0]=6;subscriptionIndex[6][0]=0;
subscriptionList[6][1]=8;subscriptionIndex[6][1]=2;
subscriptionList[7][0]=7;subscriptionIndex[7][0]=0;
subscriptionList[7][1]=8;subscriptionIndex[7][1]=1;
subscriptionList[7][2]=31;subscriptionIndex[7][2]=1;
subscriptionList[7][3]=36;subscriptionIndex[7][3]=1;
subscriptionList[7][4]=13;subscriptionIndex[7][4]=2;
subscriptionList[7][5]=34;subscriptionIndex[7][5]=2;
subscriptionList[8][0]=8;subscriptionIndex[8][0]=0;
subscriptionList[8][1]=7;subscriptionIndex[8][1]=1;
subscriptionList[8][2]=21;subscriptionIndex[8][2]=3;
subscriptionList[9][0]=9;subscriptionIndex[9][0]=0;
subscriptionList[10][0]=10;subscriptionIndex[10][0]=0;
subscriptionList[11][0]=11;subscriptionIndex[11][0]=0;
subscriptionList[12][0]=28;subscriptionIndex[12][0]=3;
subscriptionList[12][1]=33;subscriptionIndex[12][1]=3;
subscriptionList[13][0]=13;subscriptionIndex[13][0]=0;
subscriptionList[14][0]=33;subscriptionIndex[14][0]=1;
subscriptionList[14][1]=28;subscriptionIndex[14][1]=2;
subscriptionList[14][2]=15;subscriptionIndex[14][2]=3;
subscriptionList[15][0]=15;subscriptionIndex[15][0]=0;
subscriptionList[16][0]=16;subscriptionIndex[16][0]=0;
subscriptionList[17][0]=0;subscriptionIndex[17][0]=1;
subscriptionList[17][1]=11;subscriptionIndex[17][1]=2;
subscriptionList[18][0]=7;subscriptionIndex[18][0]=2;
subscriptionList[19][0]=6;subscriptionIndex[19][0]=3;
subscriptionList[19][1]=8;subscriptionIndex[19][1]=3;
subscriptionList[19][2]=16;subscriptionIndex[19][2]=3;
subscriptionList[20][0]=9;subscriptionIndex[20][0]=1;
subscriptionList[21][0]=21;subscriptionIndex[21][0]=0;
subscriptionList[22][0]=9;subscriptionIndex[22][0]=3;
subscriptionList[23][0]=6;subscriptionIndex[23][0]=1;
subscriptionList[23][1]=10;subscriptionIndex[23][1]=1;
subscriptionList[23][2]=27;subscriptionIndex[23][2]=1;
subscriptionList[23][3]=29;subscriptionIndex[23][3]=1;
subscriptionList[23][4]=16;subscriptionIndex[23][4]=2;
subscriptionList[23][5]=21;subscriptionIndex[23][5]=2;
subscriptionList[23][6]=2;subscriptionIndex[23][6]=3;
subscriptionList[23][7]=4;subscriptionIndex[23][7]=3;
subscriptionList[23][8]=7;subscriptionIndex[23][8]=3;
subscriptionList[23][9]=36;subscriptionIndex[23][9]=3;
subscriptionList[24][0]=24;subscriptionIndex[24][0]=0;
subscriptionList[24][1]=24;subscriptionIndex[24][1]=1;
subscriptionList[24][2]=24;subscriptionIndex[24][2]=2;
subscriptionList[24][3]=24;subscriptionIndex[24][3]=3;
subscriptionList[25][0]=29;subscriptionIndex[25][0]=3;
subscriptionList[26][0]=21;subscriptionIndex[26][0]=1;
subscriptionList[26][1]=0;subscriptionIndex[26][1]=2;
subscriptionList[26][2]=10;subscriptionIndex[26][2]=2;
subscriptionList[26][3]=15;subscriptionIndex[26][3]=2;
subscriptionList[26][4]=27;subscriptionIndex[26][4]=2;
subscriptionList[26][5]=33;subscriptionIndex[26][5]=2;
subscriptionList[27][0]=27;subscriptionIndex[27][0]=0;
subscriptionList[28][0]=28;subscriptionIndex[28][0]=0;
subscriptionList[29][0]=29;subscriptionIndex[29][0]=0;
subscriptionList[29][1]=4;subscriptionIndex[29][1]=1;
subscriptionList[29][2]=13;subscriptionIndex[29][2]=1;
subscriptionList[29][3]=16;subscriptionIndex[29][3]=1;
subscriptionList[29][4]=34;subscriptionIndex[29][4]=1;
subscriptionList[29][5]=6;subscriptionIndex[29][5]=2;
subscriptionList[29][6]=36;subscriptionIndex[29][6]=2;
subscriptionList[29][7]=27;subscriptionIndex[29][7]=3;
subscriptionList[29][8]=31;subscriptionIndex[29][8]=3;
subscriptionList[30][0]=30;subscriptionIndex[30][0]=0;
subscriptionList[30][1]=30;subscriptionIndex[30][1]=1;
subscriptionList[30][2]=30;subscriptionIndex[30][2]=2;
subscriptionList[30][3]=30;subscriptionIndex[30][3]=3;
subscriptionList[31][0]=31;subscriptionIndex[31][0]=0;
subscriptionList[31][1]=29;subscriptionIndex[31][1]=2;
subscriptionList[32][0]=11;subscriptionIndex[32][0]=3;
subscriptionList[33][0]=33;subscriptionIndex[33][0]=0;
subscriptionList[33][1]=15;subscriptionIndex[33][1]=1;
subscriptionList[33][2]=28;subscriptionIndex[33][2]=1;
subscriptionList[34][0]=34;subscriptionIndex[34][0]=0;
subscriptionList[35][0]=11;subscriptionIndex[35][0]=1;
subscriptionList[36][0]=36;subscriptionIndex[36][0]=0;
subscriptionList[36][1]=10;subscriptionIndex[36][1]=3;
double a1[]={720,0.0,750,0.0,900,0.0,760,360,120,390,600,360,0.0,760,0.0,140,660,0.0,0.0,0.0,0.0,720,0.0,0.0,100,0.0,0.0,120,320,40,100,500,0.0,630,570,0.0,100};
double a2[]={0.5,0.0,1.3,0.0,0.6,0.0,0.45,0.15,0.45,0.4,0.25,1.4,0.0,0.55,0.0,0.2,0.8,0.0,0.0,0.0,0.0,0.6,0.0,0.0,0.4,0.0,0.0,0.25,0.4,0.25,0.4,0.35,0.0,0.4,0.5,0.0,0.4};
double a3[]={1350,0.0,1250,0.0,300,0.0,1150,1400,900,1200,850,900,0.0,600,0.0,1450,1450,0.0,0.0,0.0,0.0,1000,0.0,0.0,1200,0.0,0.0,1150,350,1400,1200,1350,0.0,1500,300,0.0,1200};
double a4[]={0.6,0.0,0.7,0.0,0.2,0.0,0.3,0.55,0.4,0.8,0.25,0.7,0.0,0.25,0.0,0.55,0.5,0.0,0.0,0.0,0.0,0.4,0.0,0.0,0.7,0.0,0.0,0.65,0.55,0.45,0.7,0.6,0.0,0.4,0.4,0.0,0.7};
double a5[]={300,0.0,1300,0.0,1350,0.0,200,1100,1200,650,1500,1350,0.0,1050,0.0,1300,550,0.0,0.0,0.0,0.0,250,0.0,0.0,150,0.0,0.0,1250,700,1150,150,1250,0.0,1500,1500,0.0,150};
double a6[]={0.3,0.0,0.8,0.0,0.6,0.0,0.5,0.6,0.6,0.3,0.35,0.7,0.0,0.55,0.0,0.45,0.35,0.0,0.0,0.0,0.0,0.3,0.0,0.0,0.5,0.0,0.0,0.55,0.3,0.35,0.5,0.75,0.0,0.2,0.5,0.0,0.5};
double a7[]={1500,0.0,1500,0.0,1050,0.0,750,1100,1350,1350,100,1350,0.0,550,0.0,1400,1000,0.0,0.0,0.0,0.0,1000,0.0,0.0,1350,0.0,0.0,350,550,350,1350,500,0.0,1350,1250,0.0,1350};
double a8[]={0.9,0.0,0.9,0.0,0.8,0.0,0.6,0.35,0.7,0.2,0.15,0.7,0.0,0.3,0.0,0.55,0.5,0.0,0.0,0.0,0.0,0,0.0,0.0,0.3,0.0,0.0,0.4,0.3,0.5,0.3,0.35,0.0,0.5,0.5,0.0,0.3};
double a9[]={0.008,0.0,0.009,0.0,0.01,0.0,0.01,0.009,0.009,0.007,0.009,0.01,0.0,0.009,0.0,0.01,0.008,0.0,0.0,0.0,0.0,0.01,0.0,0.0,0.006,0.0,0.0,0.008,0.009,0.01,0.006,0.009,0.0,0.008,0.009,0.0,0.006};
double a10[]={0.008,0.0,0.009,0.0,0.008,0.0,0.008,0.008,0.009,0.008,0.008,0.006,0.0,0.009,0.0,0.01,0.008,0.0,0.0,0.0,0.0,0.005,0.0,0.0,0.009,0.0,0.0,0.01,0.008,0.009,0.009,0.009,0.0,0.008,0.01,0.0,0.009};
double a11[]={0.4,0.0,0.2,0.0,0.1,0.0,0.3,0.4,0.2,0.1,0.7,0.2,0.0,0,0.0,0.1,0,0.0,0.0,0.0,0.0,0.1,0.0,0.0,0.2,0.0,0.0,0.3,0,0.2,0.2,0,0.0,0.7,0.1,0.0,0.2};
int a12[]={500,1000,8000,2000,4000,1000,1250,1000,1000,500,1000,125,2000,4000,1000,250,2000,250,1000,1250,500,2000,1000,500,0,250,500,4000,4000,1250,0,2000,500,500,4000,125,1000};
double a13[]={0.0013406,0.0020022,0.0018709,0.0018948,0.0017975,0.0014687,0.0011068,0.001355,0.0010891,0.00088151,0.0014294,0.0012989,0.0014205,0.0019711,0.0015365,0.0020505,0.0018961,0.00078672,0.0023114,0.0012203,0.0012849,0.0015674,0.0012844,0.0014197,0.0,0.00074657,0.00096164,0.0017109,0.0015385,0.00068178,0.0,0.0021815,0.00087359,0.00074349,0.0021645,0.001595,0.0014573};
int a14[]={14850,0,16500,0,13740,0,13740,24750,13740,14100,13740,30750,0,14400,0,13740,13740,0,0,0,0,14100,0,0,13500,0,0,13740,13740,25200,13500,13740,0,13740,13740,0,13740};
int a15[]={30750,0,35900,0,34950,0,35900,35900,35900,35900,30000,34950,0,26250,0,34000,35900,0,0,0,0,34500,0,0,13500,0,0,35900,34650,35900,13500,32700,0,35900,35900,0,33300};
Client client;
for (int i = 0; i < MAX_ATTEMPTS; i++) {
client.connect(host, port, clientId);
++attempt;
std::cout << "Attempt " << attempt << " of " << MAX_ATTEMPTS<< std::endl;
for (int j=0;j<NUMREALTIMESYMBOLS;j++){
if(j==24 || j==30)
continue;
std::cout<<j<<" "<<realTimeSymbols[j]<<" "<<getLocalSymbol(realTimeSymbols[j],date)<<std::endl;
client.setTickerMap(j,realTimeSymbols[j],getLocalSymbol(realTimeSymbols[j],date));
}
}
}
Constructor of Client:
Client::Client(){
for(int i=0;i<50;i++){
symbolMap.push_back(" ");
localSymbolMap.push_back(" ");
}
}
The above code fails at 24 and 30. Hence, the loop to continue when j is 24 or 30 as workaround.
std::string realTimeSymbolsArr[]={"a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z","aa","bb","cc","dd","ee","ff","gg","hh","ii","jj","kk"};
subscriptionList[24][1]=24;subscriptionIndex[24][1]=1;
subscriptionList[24][2]=24;subscriptionIndex[24][2]=2;
subscriptionList[24][3]=24;subscriptionIndex[24][3]=3;
subscriptionList[30][1]=30;subscriptionIndex[30][1]=1;
subscriptionList[30][2]=30;subscriptionIndex[30][2]=2;
subscriptionList[30][3]=30;subscriptionIndex[30][3]=3;
You have not posted the source for getLocalSymbol, but I must assume that it also uses the same data and has a flow of the following form:
getLocalSymbol(a, b) {
int i, j, old_i, old_j;
std::string value;
// Derive i and j from the parameters
// ...
// And build the String
do {
old_i = i;
old_j = j;
i = subscriptionList[old_i][old_j];
j = subscriptionIndex[old_i][old_j];
value += realTimeSymbolsArr[i];
} while(j > 0);
return value;
}
Got it right? That control flow, or something equivalent, appears to be part of it, either way.
This goes well for almost all values of i and j - except for the aforementioned values of 24 and 30 for i, and 1 to 3 for j.
With these values, i and j remain the same in every iteration and value becomes longer and longer, until eventually something breaks on the stack which overwrites both j (and thereby causes the loop to terminate) and corrupts value.
Either way, the std::string you returned is now corrupted as you exceeded some limit during that endless loop.
As for how to solve it, fix that infinite loop and fix your data.
For fixing the loop, limit the iteration count.
For fixing your data, well, now that you know why the data is causing the bug, you should be able to figure yourself how to fix it.
Remember, you have to fix BOTH. If you don't fix the data, you will get an unreasonable long return value. And if you don't fix the iteration limit, it will crash again as soon as someone repeats a similar mistake when updating the data.

How do I solve memory leaks that are reported to reside in STL string?

For a school project, we have to send big files across the network., we must use Poco::XML for our data.
After our files are send over the network, it appears that the memory does not free.
Here is an example for a file of ~9 Mb on the receiving part:
valgrind --leak-check=full --show-reachable=yes -v ourExecutable parms returns:
12,880,736 bytes in 37 blocks are definitely lost in loss record 101 of 101
at 0x4C2747E: operator new(unsigned long) (vg_replace_malloc.c:261)
by 0x5A3AC88: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib64/gcc/x86_64-pc-linux-gnu/4.4.4/libstdc++.so.6.0.13)
by 0x5A3BC4A: std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long) (in /usr/lib64/gcc/x86_64-pc-linux-gnu/4.4.4/libstdc++.so.6.0.13)
by 0x5A3C1BB: std::string::reserve(unsigned long) (in /usr/lib64/gcc/x86_64-pc-linux-gnu/4.4.4/libstdc++.so.6.0.13)
by 0x5A3C68E: std::string::append(std::string const&) (in /usr/lib64/gcc/x86_64-pc-linux-gnu/4.4.4/libstdc++.so.6.0.13)
by 0x5202359: Poco::XML::Element::innerText() const (in /home/tomwij/IGS/trunk/Project/external/lib/libPocoXML.so.8)
by 0x4145BF: NodeProtocol::getChildNodeStrValue(Poco::XML::Element*, std::string) (NodeProtocol.cpp:82)
by 0x41544F: NodeProtocol::deserialize(std::string const&) (NodeProtocol.cpp:200)
by 0x40B088: Node::handleClientPacket(PriorityElement*) (Node.cpp:760)
by 0x40A04C: Node::handlePackets() (Node.cpp:574)
by 0x4078EA: Node::run() (Node.cpp:162)
by 0x40772D: Node::activate() (Node.cpp:138)
LEAK SUMMARY:
definitely lost: 12,888,036 bytes in 190 blocks
indirectly lost: 644,979 bytes in 1,355 blocks
possibly lost: 10,089 bytes in 27 blocks
still reachable: 306,020 bytes in 43 blocks
suppressed: 0 bytes in 0 blocks
The function which is right before Poco is
const string NodeProtocol::getChildNodeStrValue(Element * elem, string child)
{
Element* tempNode = elem->getChildElement(child);
XMLString result(tempNode->innerText());
string ret = string(fromXMLString(result));
result.clear();
return ret;
}
which calls
XMLString Element::innerText() const
{
XMLString result;
Node* pChild = firstChild();
while (pChild)
{
result.append(pChild->innerText());
pChild = pChild->nextSibling();
}
return result;
}
(Note that XMLString is std::string)
Why is the append of STL string leaking memory?
If I just assign instead of using the copy constructors it gives the same problem.
EDIT:
I'm using the latest stable GNU GCC 4.4.4 on Gentoo x64 (linux-2.6.34-gentoo-r12).
More functions from the call stack (stripped irrelevant big chunks of code / if structures):
Command * NodeProtocol::deserialize(const string & msg)
{
DOMParser xmlParser;
// Get the root node.
AutoPtr<Document> doc = xmlParser.parseString(msg);
AutoPtr<Element> rootElement = doc->documentElement();
string root = fromXMLString(rootElement->nodeName());
string name = getChildNodeStrValue(rootElement, "name");
string data = getChildNodeStrValue(rootElement, "data");
return new PutCommand(name, data);
}
and
void Node::handleClientPacket(PriorityElement * prio)
{
Command * command = NodeProtocol::deserialize(prio->fPacket);
// CUT: Access some properties of command, let the command execute.
delete command;
}
and
void Node::handlePackets()
{
PriorityElement * prio = fQueue->top();
fQueue->pop();
if (prio->fSource == kCLIENT)
handleClientPacket(prio);
else if (prio->fSource == kNODE)
handleNodePacket(prio);
delete prio;
}
where fQueue is:
priority_queue< PriorityElement*, vector<PriorityElement*>, ComparisonFunction >
I would make this a comment, but apparently I don't have the rep. Have you remembered to make the destructor for Command virtual? If name or data are fields of PutCommand rather than Command and the Command destructor is not virtual, they may not be freed properly when you delete command in handleClientPacket.

How to get a stack trace for C++ using gcc with line number information?

We use stack traces in proprietary assert like macro to catch developer mistakes - when error is caught, stack trace is printed.
I find gcc's pair backtrace()/backtrace_symbols() methods insufficient:
Names are mangled
No line information
1st problem can be resolved by abi::__cxa_demangle.
However 2nd problem s more tough. I found replacement for backtrace_symbols().
This is better than gcc's backtrace_symbols(), since it can retrieve line numbers (if compiled with -g) and you don't need to compile with -rdynamic.
Hoverer the code is GNU licenced, so IMHO I can't use it in commercial code.
Any proposal?
P.S.
gdb is capable to print out arguments passed to functions.
Probably it's already too much to ask for :)
PS 2
Similar question (thanks nobar)
So you want a stand-alone function that prints a stack trace with all of the features that gdb stack traces have and that doesn't terminate your application. The answer is to automate the launch of gdb in a non-interactive mode to perform just the tasks that you want.
This is done by executing gdb in a child process, using fork(), and scripting it to display a stack-trace while your application waits for it to complete. This can be performed without the use of a core-dump and without aborting the application. I learned how to do this from looking at this question: How it's better to invoke gdb from program to print it's stacktrace?
The example posted with that question didn't work for me exactly as written, so here's my "fixed" version (I ran this on Ubuntu 9.04).
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
#include <sys/prctl.h>
void print_trace() {
char pid_buf[30];
sprintf(pid_buf, "%d", getpid());
char name_buf[512];
name_buf[readlink("/proc/self/exe", name_buf, 511)]=0;
prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, 0, 0, 0);
int child_pid = fork();
if (!child_pid) {
dup2(2,1); // redirect output to stderr - edit: unnecessary?
execl("/usr/bin/gdb", "gdb", "--batch", "-n", "-ex", "thread", "-ex", "bt", name_buf, pid_buf, NULL);
abort(); /* If gdb failed to start */
} else {
waitpid(child_pid,NULL,0);
}
}
As shown in the referenced question, gdb provides additional options that you could use. For example, using "bt full" instead of "bt" produces an even more detailed report (local variables are included in the output). The manpages for gdb are kind of light, but complete documentation is available here.
Since this is based on gdb, the output includes demangled names, line-numbers, function arguments, and optionally even local variables. Also, gdb is thread-aware, so you should be able to extract some thread-specific metadata.
Here's an example of the kind of stack traces that I see with this method.
0x00007f97e1fc2925 in waitpid () from /lib/libc.so.6
[Current thread is 0 (process 15573)]
#0 0x00007f97e1fc2925 in waitpid () from /lib/libc.so.6
#1 0x0000000000400bd5 in print_trace () at ./demo3b.cpp:496
2 0x0000000000400c09 in recursive (i=2) at ./demo3b.cpp:636
3 0x0000000000400c1a in recursive (i=1) at ./demo3b.cpp:646
4 0x0000000000400c1a in recursive (i=0) at ./demo3b.cpp:646
5 0x0000000000400c46 in main (argc=1, argv=0x7fffe3b2b5b8) at ./demo3b.cpp:70
Note: I found this to be incompatible with the use of valgrind (probably due to Valgrind's use of a virtual machine). It also doesn't work when you are running the program inside of a gdb session (can't apply a second instance of "ptrace" to a process).
Not too long ago I answered a similar question. You should take a look at the source code available on method #4, which also prints line numbers and filenames.
Method #4:
A small improvement I've done on method #3 to print line numbers. This could be copied to work on method #2 also.
Basically, it uses addr2line to convert addresses into file names and line numbers.
The source code below prints line numbers for all local functions. If a function from another library is called, you might see a couple of ??:0 instead of file names.
#include <stdio.h>
#include <signal.h>
#include <stdio.h>
#include <signal.h>
#include <execinfo.h>
void bt_sighandler(int sig, struct sigcontext ctx) {
void *trace[16];
char **messages = (char **)NULL;
int i, trace_size = 0;
if (sig == SIGSEGV)
printf("Got signal %d, faulty address is %p, "
"from %p\n", sig, ctx.cr2, ctx.eip);
else
printf("Got signal %d\n", sig);
trace_size = backtrace(trace, 16);
/* overwrite sigaction with caller's address */
trace[1] = (void *)ctx.eip;
messages = backtrace_symbols(trace, trace_size);
/* skip first stack frame (points here) */
printf("[bt] Execution path:\n");
for (i=1; i<trace_size; ++i)
{
printf("[bt] #%d %s\n", i, messages[i]);
/* find first occurence of '(' or ' ' in message[i] and assume
* everything before that is the file name. (Don't go beyond 0 though
* (string terminator)*/
size_t p = 0;
while(messages[i][p] != '(' && messages[i][p] != ' '
&& messages[i][p] != 0)
++p;
char syscom[256];
sprintf(syscom,"addr2line %p -e %.*s", trace[i], p, messages[i]);
//last parameter is the file name of the symbol
system(syscom);
}
exit(0);
}
int func_a(int a, char b) {
char *p = (char *)0xdeadbeef;
a = a + b;
*p = 10; /* CRASH here!! */
return 2*a;
}
int func_b() {
int res, a = 5;
res = 5 + func_a(a, 't');
return res;
}
int main() {
/* Install our signal handler */
struct sigaction sa;
sa.sa_handler = (void *)bt_sighandler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART;
sigaction(SIGSEGV, &sa, NULL);
sigaction(SIGUSR1, &sa, NULL);
/* ... add any other signal here */
/* Do something */
printf("%d\n", func_b());
}
This code should be compiled as: gcc sighandler.c -o sighandler -rdynamic
The program outputs:
Got signal 11, faulty address is 0xdeadbeef, from 0x8048975
[bt] Execution path:
[bt] #1 ./sighandler(func_a+0x1d) [0x8048975]
/home/karl/workspace/stacktrace/sighandler.c:44
[bt] #2 ./sighandler(func_b+0x20) [0x804899f]
/home/karl/workspace/stacktrace/sighandler.c:54
[bt] #3 ./sighandler(main+0x6c) [0x8048a16]
/home/karl/workspace/stacktrace/sighandler.c:74
[bt] #4 /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0x3fdbd6]
??:0
[bt] #5 ./sighandler() [0x8048781]
??:0
There is a robust discussion of essentially the same question at: How to generate a stacktrace when my gcc C++ app crashes. Many suggestions are provided, including lots of discussion about how to generate stack traces at run-time.
My personal favorite answer from that thread was to enable core dumps which allows you to view the complete application state at the time of the crash (including function arguments, line numbers, and unmangled names). An additional benefit of this approach is that it not only works for asserts, but also for segmentation faults and unhandled exceptions.
Different Linux shells use different commands to enable core dumps, but you can do it from within your application code with something like this...
#include <sys/resource.h>
...
struct rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
assert( setrlimit( RLIMIT_CORE, &core_limit ) == 0 ); // enable core dumps for debug builds
After a crash, run your favorite debugger to examine the program state.
$ kdbg executable core
Here's some sample output...
It is also possible to extract the stack trace from a core dump at the command line.
$ ( CMDFILE=$(mktemp); echo "bt" >${CMDFILE}; gdb 2>/dev/null --batch -x ${CMDFILE} temp.exe core )
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 22857]
#0 0x00007f4189be5fb5 in raise () from /lib/libc.so.6
#0 0x00007f4189be5fb5 in raise () from /lib/libc.so.6
#1 0x00007f4189be7bc3 in abort () from /lib/libc.so.6
#2 0x00007f4189bdef09 in __assert_fail () from /lib/libc.so.6
#3 0x00000000004007e8 in recursive (i=5) at ./demo1.cpp:18
#4 0x00000000004007f3 in recursive (i=4) at ./demo1.cpp:19
#5 0x00000000004007f3 in recursive (i=3) at ./demo1.cpp:19
#6 0x00000000004007f3 in recursive (i=2) at ./demo1.cpp:19
#7 0x00000000004007f3 in recursive (i=1) at ./demo1.cpp:19
#8 0x00000000004007f3 in recursive (i=0) at ./demo1.cpp:19
#9 0x0000000000400849 in main (argc=1, argv=0x7fff2483bd98) at ./demo1.cpp:26
Since the GPL licensed code is intended to help you during development, you could simply not include it in the final product. The GPL restricts you from distributing GPL licenses code linked with non-GPL compatible code. As long as you only use the GPL code inhouse, you should be fine.
Use the google glog library for it. It has new BSD licence.
It contains a GetStackTrace function in the stacktrace.h file.
EDIT
I found here http://blog.bigpixel.ro/2010/09/09/stack-unwinding-stack-trace-with-gcc/ that there is an utility called addr2line that translates program addresses into file names and line numbers.
http://linuxcommand.org/man_pages/addr2line1.html
Here's an alternative approach. A debug_assert() macro programmatically sets a conditional breakpoint. If you are running in a debugger, you will hit a breakpoint when the assert expression is false -- and you can analyze the live stack (the program doesn't terminate). If you are not running in a debugger, a failed debug_assert() causes the program to abort and you get a core dump from which you can analyze the stack (see my earlier answer).
The advantage of this approach, compared to normal asserts, is that you can continue running the program after the debug_assert is triggered (when running in a debugger). In other words, debug_assert() is slightly more flexible than assert().
#include <iostream>
#include <cassert>
#include <sys/resource.h>
// note: The assert expression should show up in
// stack trace as parameter to this function
void debug_breakpoint( char const * expression )
{
asm("int3"); // x86 specific
}
#ifdef NDEBUG
#define debug_assert( expression )
#else
// creates a conditional breakpoint
#define debug_assert( expression ) \
do { if ( !(expression) ) debug_breakpoint( #expression ); } while (0)
#endif
void recursive( int i=0 )
{
debug_assert( i < 5 );
if ( i < 10 ) recursive(i+1);
}
int main( int argc, char * argv[] )
{
rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
setrlimit( RLIMIT_CORE, &core_limit ); // enable core dumps
recursive();
}
Note: Sometimes "conditional breakpoints" setup within debuggers can be slow. By establishing the breakpoint programmatically, the performance of this method should be equivalent to that of a normal assert().
Note: As written, this is specific to the Intel x86 architecture -- other processors may have different instructions for generating a breakpoint.
A bit late, but you can use libbfb to fetch the filename and linenumber like refdbg does in symsnarf.c. libbfb is internally used by addr2line and gdb
here is my solution:
#include <execinfo.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
#include <iostream>
#include <zconf.h>
#include "regex"
std::string getexepath() {
char result[PATH_MAX];
ssize_t count = readlink("/proc/self/exe", result, PATH_MAX);
return std::string(result, (count > 0) ? count : 0);
}
std::string sh(std::string cmd) {
std::array<char, 128> buffer;
std::string result;
std::shared_ptr<FILE> pipe(popen(cmd.c_str(), "r"), pclose);
if (!pipe) throw std::runtime_error("popen() failed!");
while (!feof(pipe.get())) {
if (fgets(buffer.data(), 128, pipe.get()) != nullptr) {
result += buffer.data();
}
}
return result;
}
void print_backtrace(void) {
void *bt[1024];
int bt_size;
char **bt_syms;
int i;
bt_size = backtrace(bt, 1024);
bt_syms = backtrace_symbols(bt, bt_size);
std::regex re("\\[(.+)\\]");
auto exec_path = getexepath();
for (i = 1; i < bt_size; i++) {
std::string sym = bt_syms[i];
std::smatch ms;
if (std::regex_search(sym, ms, re)) {
std::string addr = ms[1];
std::string cmd = "addr2line -e " + exec_path + " -f -C " + addr;
auto r = sh(cmd);
std::regex re2("\\n$");
auto r2 = std::regex_replace(r, re2, "");
std::cout << r2 << std::endl;
}
}
free(bt_syms);
}
void test_m() {
print_backtrace();
}
int main() {
test_m();
return 0;
}
output:
/home/roroco/Dropbox/c/ro-c/cmake-build-debug/ex/test_backtrace_with_line_number
test_m()
/home/roroco/Dropbox/c/ro-c/ex/test_backtrace_with_line_number.cpp:57
main
/home/roroco/Dropbox/c/ro-c/ex/test_backtrace_with_line_number.cpp:61
??
??:0
"??" and "??:0" since this trace is in libc, not in my source
The one of solutions is to start a gdb with "bt"-script in failed assert handler. It is not very easy to integrate such gdb-starting, but It will give you both backtrace and args and demangle names (or you can pass gdb output via c++filt programm).
Both programms (gdb and c++filt) will be not linked into your application, so GPL will not require you to opensource complete application.
The same approach (exec a GPL programme) you can use with backtrace-symbols. Just generate ascii list of %eip's and map of exec file (/proc/self/maps) and pass it to separate binary.
You can use DeathHandler - small C++ class which does everything for you, reliable.
I suppose line numbers are related to current eip value, right?
SOLUTION 1:
Then you can use something like GetThreadContext(), except that you're working on linux. I googled around a bit and found something similar, ptrace():
The ptrace() system call provides a
means by which a parent process may
observe and control the execution of
another process, and examine and
change its core image and registers. [...]
The parent can initiate a trace by
calling fork(2) and having the
resulting child do a PTRACE_TRACEME,
followed (typically) by an exec(3).
Alternatively, the parent may commence
trace of an existing process using
PTRACE_ATTACH.
Now I was thinking, you can do a 'main' program which checks for signals that are sent to its child, the real program you're working on. after fork() it call waitid():
All of these system calls are used to
wait for state changes in a child of
the calling process, and obtain
information about the child whose
state has changed.
and if a SIGSEGV (or something similar) is caught call ptrace() to obtain eip's value.
PS: I've never used these system calls (well, actually, I've never seen them before ;) so I don't know if it's possible neither can help you. At least I hope these links are useful. ;)
SOLUTION 2:
The first solution is quite complicated, right? I came up with a much simpler one: using signal() catch the signals you are interested in and call a simple function that reads the eip value stored in the stack:
...
signal(SIGSEGV, sig_handler);
...
void sig_handler(int signum)
{
int eip_value;
asm {
push eax;
mov eax, [ebp - 4]
mov eip_value, eax
pop eax
}
// now you have the address of the
// **next** instruction after the
// SIGSEGV was received
}
That asm syntax is Borland's one, just adapt it to GAS. ;)
Here's my third answer -- still trying to take advantage of core dumps.
It wasn't completely clear in the question whether the "assert-like" macros were supposed to terminate the application (the way assert does) or they were supposed to continue executing after generating their stack-trace.
In this answer, I'm addressing the case where you want to show a stack-trace and continue executing. I wrote the coredump() function below to generate a core dump, automatically extract the stack-trace from it, then continue executing the program.
Usage is the same as that of assert(). The difference, of course, is that assert() terminates the program but coredump_assert() does not.
#include <iostream>
#include <sys/resource.h>
#include <cstdio>
#include <cstdlib>
#include <boost/lexical_cast.hpp>
#include <string>
#include <sys/wait.h>
#include <unistd.h>
std::string exename;
// expression argument is for diagnostic purposes (shows up in call-stack)
void coredump( char const * expression )
{
pid_t childpid = fork();
if ( childpid == 0 ) // child process generates core dump
{
rlimit core_limit = { RLIM_INFINITY, RLIM_INFINITY };
setrlimit( RLIMIT_CORE, &core_limit ); // enable core dumps
abort(); // terminate child process and generate core dump
}
// give each core-file a unique name
if ( childpid > 0 ) waitpid( childpid, 0, 0 );
static int count=0;
using std::string;
string pid = boost::lexical_cast<string>(getpid());
string newcorename = "core-"+boost::lexical_cast<string>(count++)+"."+pid;
string rawcorename = "core."+boost::lexical_cast<string>(childpid);
int rename_rval = rename(rawcorename.c_str(),newcorename.c_str()); // try with core.PID
if ( rename_rval == -1 ) rename_rval = rename("core",newcorename.c_str()); // try with just core
if ( rename_rval == -1 ) std::cerr<<"failed to capture core file\n";
#if 1 // optional: dump stack trace and delete core file
string cmd = "( CMDFILE=$(mktemp); echo 'bt' >${CMDFILE}; gdb 2>/dev/null --batch -x ${CMDFILE} "+exename+" "+newcorename+" ; unlink ${CMDFILE} )";
int system_rval = system( ("bash -c '"+cmd+"'").c_str() );
if ( system_rval == -1 ) std::cerr.flush(), perror("system() failed during stack trace"), fflush(stderr);
unlink( newcorename.c_str() );
#endif
}
#ifdef NDEBUG
#define coredump_assert( expression ) ((void)(expression))
#else
#define coredump_assert( expression ) do { if ( !(expression) ) { coredump( #expression ); } } while (0)
#endif
void recursive( int i=0 )
{
coredump_assert( i < 2 );
if ( i < 4 ) recursive(i+1);
}
int main( int argc, char * argv[] )
{
exename = argv[0]; // this is used to generate the stack trace
recursive();
}
When I run the program, it displays three stack traces...
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24251]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=2) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#6 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24259]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=3) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=2) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#6 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#7 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
Core was generated by `./temp.exe'.
Program terminated with signal 6, Aborted.
[New process 24267]
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#0 0x00007f2818ac9fb5 in raise () from /lib/libc.so.6
#1 0x00007f2818acbbc3 in abort () from /lib/libc.so.6
#2 0x0000000000401a0e in coredump (expression=0x403303 "i < 2") at ./demo3.cpp:29
#3 0x0000000000401f5f in recursive (i=4) at ./demo3.cpp:60
#4 0x0000000000401f70 in recursive (i=3) at ./demo3.cpp:61
#5 0x0000000000401f70 in recursive (i=2) at ./demo3.cpp:61
#6 0x0000000000401f70 in recursive (i=1) at ./demo3.cpp:61
#7 0x0000000000401f70 in recursive (i=0) at ./demo3.cpp:61
#8 0x0000000000401f8b in main (argc=1, argv=0x7fffc229eb98) at ./demo3.cpp:66
I had to do this in a production environment with many constraints, so I wanted to explain the advantages and disadvantages of the already posted methods.
attach GDB
+ very simple and robust
- Slow for large programs because GDB insists on loading the entire address to line # database upfront instead of lazily
- Interferes with signal handling. When GDB is attached, it intercepts signals like SIGINT (ctrl-c), which will cause the program to get stuck at the GDB interactive prompt? if some other process routinely sends such signals. Maybe there's some way around it, but this made GDB unusable in my case. You can still use it if you only care about printing a call stack once when your program crashes, but not multiple times.
addr2line. Here's an alternate solution that doesn't use backtrace_symbols.
+ Doesn't allocate from the heap, which is unsafe inside a signal handler
+ Don't need to parse output of backtrace_symbols
- Won't work on MacOS, which doesn't have dladdr1. You can use _dyld_get_image_vmaddr_slide instead, which returns the same offset as link_map::l_addr.
- Requires adding negative offset or else the translated line # will be 1 greater. backtrace_symbols does this for you
#include <execinfo.h>
#include <link.h>
#include <stdlib.h>
#include <stdio.h>
// converts a function's address in memory to its VMA address in the executable file. VMA is what addr2line expects
size_t ConvertToVMA(size_t addr)
{
Dl_info info;
link_map* link_map;
dladdr1((void*)addr,&info,(void**)&link_map,RTLD_DL_LINKMAP);
return addr-link_map->l_addr;
}
void PrintCallStack()
{
void *callstack[128];
int frame_count = backtrace(callstack, sizeof(callstack)/sizeof(callstack[0]));
for (int i = 0; i < frame_count; i++)
{
char location[1024];
Dl_info info;
if(dladdr(callstack[i],&info))
{
char command[256];
size_t VMA_addr=ConvertToVMA((size_t)callstack[i]);
//if(i!=crash_depth)
VMA_addr-=1; // https://stackoverflow.com/questions/11579509/wrong-line-numbers-from-addr2line/63841497#63841497
snprintf(command,sizeof(command),"addr2line -e %s -Ci %zx",info.dli_fname,VMA_addr);
system(command);
}
}
}
void Foo()
{
PrintCallStack();
}
int main()
{
Foo();
return 0;
}
I also want to clarify what addresses backtrace and backtrace_symbols generate and what addr2line expects.
addr2line expects FooVMA or if you're using --section=.text, then Foofile - textfile. backtrace returns Foomem. backtrace_symbols generates FooVMA somewhere.
One big mistake I made and saw in several other posts was assuming VMAbase = 0 or FooVMA = Foofile = Foomem - ELFmem, which is easy to calculate.
That often works, but for some compilers (i.e. linker scripts) use VMAbase > 0. Examples would be the GCC 5.4 on Ubuntu 16 (0x400000) and clang 11 on MacOS (0x100000000).
For shared libs, it's always 0. Seems VMAbase was only meaningful for non-position independent code. Otherwise it has no effect on where the EXE is loaded in memory.
Also, neither karlphillip's nor this one requires compiling with -rdynamic. That will increase the binary size, especially for a large C++ program or shared lib, with useless entries in the dynamic symbol table that never get imported
AFAICS all of the solutions provided so far won't print functions names and line numbers from shared libraries. That's what I needed, so i altered karlphillip's solution (and some other answer from a similar question) to resolve shared library addresses using /proc/id/maps.
#include <stdlib.h>
#include <inttypes.h>
#include <stdio.h>
#include <string.h>
#include <execinfo.h>
#include <stdbool.h>
struct Region { // one mapped file, for example a shared library
uintptr_t start;
uintptr_t end;
char* path;
};
static struct Region* getRegions(int* size) {
// parse /proc/self/maps and get list of mapped files
FILE* file;
int allocated = 10;
*size = 0;
struct Region* res;
uintptr_t regionStart = 0x00000000;
uintptr_t regionEnd = 0x00000000;
char* regionPath = "";
uintmax_t matchedStart;
uintmax_t matchedEnd;
char* matchedPath;
res = (struct Region*)malloc(sizeof(struct Region) * allocated);
file = fopen("/proc/self/maps", "r");
while (!feof(file)) {
fscanf(file, "%jx-%jx %*s %*s %*s %*s%*[ ]%m[^\n]\n", &matchedStart, &matchedEnd, &matchedPath);
bool bothNull = matchedPath == 0x0 && regionPath == 0x0;
bool similar = matchedPath && regionPath && !strcmp(matchedPath, regionPath);
if(bothNull || similar) {
free(matchedPath);
regionEnd = matchedEnd;
} else {
if(*size == allocated) {
allocated *= 2;
res = (struct Region*)realloc(res, sizeof(struct Region) * allocated);
}
res[*size].start = regionStart;
res[*size].end = regionEnd;
res[*size].path = regionPath;
(*size)++;
regionStart = matchedStart;
regionEnd = matchedEnd;
regionPath = matchedPath;
}
}
return res;
}
struct SemiResolvedAddress {
char* path;
uintptr_t offset;
};
static struct SemiResolvedAddress semiResolve(struct Region* regions, int regionsNum, uintptr_t address) {
// convert address from our address space to
// address suitable fo addr2line
struct Region* region;
struct SemiResolvedAddress res = {"", address};
for(region = regions; region < regions+regionsNum; region++) {
if(address >= region->start && address < region->end) {
res.path = region->path;
res.offset = address - region->start;
}
}
return res;
}
void printStacktraceWithLines(unsigned int max_frames)
{
int regionsNum;
fprintf(stderr, "stack trace:\n");
// storage array for stack trace address data
void* addrlist[max_frames+1];
// retrieve current stack addresses
int addrlen = backtrace(addrlist, sizeof(addrlist) / sizeof(void*));
if (addrlen == 0) {
fprintf(stderr, " <empty, possibly corrupt>\n");
return;
}
struct Region* regions = getRegions(&regionsNum);
for (int i = 1; i < addrlen; i++)
{
struct SemiResolvedAddress hres =
semiResolve(regions, regionsNum, (uintptr_t)(addrlist[i]));
char syscom[256];
sprintf(syscom, "addr2line -C -f -p -a -e %s 0x%jx", hres.path, (intmax_t)(hres.offset));
system(syscom);
}
free(regions);
}
C++23 <stacktrace>
Finally, this has arrived! More details/comparison with other systems at: print call stack in C or C++
stacktrace.cpp
#include <iostream>
#include <stacktrace>
void my_func_2(void) {
std::cout << std::stacktrace::current(); // Line 5
}
void my_func_1(double f) {
(void)f;
my_func_2(); // Line 10
}
void my_func_1(int i) {
(void)i;
my_func_2(); // Line 15
}
int main(int argc, char **argv) {
my_func_1(1); // Line 19
my_func_1(2.0); // Line 20
}
GCC 12.1.0 from Ubuntu 22.04 does not have support compiled in, so for now I built it from source as per: How to edit and re-build the GCC libstdc++ C++ standard library source? and set --enable-libstdcxx-backtrace=yes, and it worked!
Compile and run:
g++ -ggdb3 -O2 -std=c++23 -Wall -Wextra -pedantic -o stacktrace.out stacktrace.cpp -lstdc++_libbacktrace
./stacktrace.out
Output:
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(int) at /home/ciro/stacktrace.cpp:15
2# at :0
3# at :0
4# at :0
5#
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(double) at /home/ciro/stacktrace.cpp:10
2# at :0
3# at :0
4# at :0
5#
The trace is not perfect (missing main line) because of optimization I think. With -O0 it is better:
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(int) at /home/ciro/stacktrace.cpp:15
2# at /home/ciro/stacktrace.cpp:19
3# at :0
4# at :0
5# at :0
6#
0# my_func_2() at /home/ciro/stacktrace.cpp:5
1# my_func_1(double) at /home/ciro/stacktrace.cpp:10
2# at /home/ciro/stacktrace.cpp:20
3# at :0
4# at :0
5# at :0
6#
I don't know why the name main is missing, but the line is there.
The "extra" lines after main like:
3# at :0
4# at :0
5# at :0
6#
are probably stuff that runs before main and that ends up calling main: What happens before main in C++?