Segmentation Fault in Multithreaded program and incomplete information on gdb backtrace - c++

Question

I am writing a program which use both OS threads and user threads (fibers, I have written this user Threading program with context switching through assembly language). The problem is that the program sometimes ends with a segmentation fault but other times it doesn't.
The problem is due to a function getting called, with invalid arguments, which shouldn't get called. I think gdb backtrace isn't giving the proper information. Here is the output of my gdb program
#0 0x0000000000000000 in ?? ()
#1 0x0000555555555613 in thread_entry (fn=0x0, arg=0x0) at userThread2.cpp:243
#2 0x000055555555c791 in start_thread () at contextSwitch2.s:57
#3 0x0000000000000000 in ?? ()
fn is the function I want to run as a user thread, arg is the argument passed to that function.
I have a function Spawn in my user threading library code which push the two arguments (fn and arg) and the pointer to start_thread on the stack and thus start_thread, an assembly function, gets called which call the c++ function thread_entry to call the function fn with arguments arg.
I am not expecting a call to start_thread or thread_entry at the point of error so I am not sure how start_thread gets called. Even if it gets called then Spawn() should have called start_thread as it is the only function which calls start_thread. But Spawn is not shown in gdb backtrace.
Some online posts have mentioned the possibility of stack corruption or something similar the result of error and they have prescribed the use of "record btrace pt". I have spent considerable time setting up intel btrace pt support in the kernel/gdb but I was unable to set it up so I am not going through that route.
Here is a link to my code with compilation instructions:
https://github.com/smartWaqar/userThreading

Answer 1

I set a breakpoint on thread_entry, and observed:
...
[Thread 0x7ffff7477700 (LWP 203995) exited]
parentId: 1
OST 1 Hello A0 on CPU 2
current_thread_num 0 next_thread_num 1
After Thread Exit
After changeOSThread
OST 1 Hello C1 on CPU 2 ---------------
Before changeOSThread
**************** In changeOSThread **************
current_thread_num 1 next_thread_num 2
Thread 3 "a.out" hit Breakpoint 1, thread_entry (fn=0x0, arg=0x0) at userThread2.cpp:243
243 fn(arg) ;
(gdb) bt
#0 thread_entry (fn=0x0, arg=0x0) at userThread2.cpp:243
#1 0x000055555555c181 in start_thread () at context.s:57
#2 0x0000000000000000 in ?? ()
Conclusions:
GDB is giving you correct crash stack trace.
You do in fact call thread_entry with fn==0, which of course promptly crashes.
There is something racy going on, as this doesn't happen every time.
Even if it gets called then Spawn() should have called start_thread as it is the only function which calls start_thread
I've observed the following "call" to strart_thread:
Thread 2 "a.out" hit Breakpoint 1, start_thread () at context.s:53
53 push %rbp
(gdb) bt
#0 start_thread () at context.s:53
#1 0x0000555555555e4f in changeOSThread (parentId=<error reading variable>) at t.cc:196
#2 0x0000000000000000 in ?? ()
So I think your mental model of who calls start_thread is wrong.
This is a bit too much code for me to look at. If you want additional help, please reduce the test case to bare minimum.