I think I've broken the rules on thread affinity somewhere in my Qt C++ application.
I'm getting lots of messages like this:
QBasicTimer::stop: Failed. Possibly trying to stop from a different thread
QBasicTimer::start: Timers cannot be started from another thread
I am using DDD/GDB. Can I insert some kind of trap which will catch these errors and give me a backtrace? Ideally suspending all other threads in the meantime?
I tried to halt on writing to stdout (b write if 1==$rdi as per this) but that didn't work.
Related
While writing an x86 WinAPI-based debugger, I've encountered a rare condition when the debuggee (which usually works well) suddenly terminates with EXCEPTION_ACCESS_VIOLATION after I attach to it with my native debugger. I can stably reproduce this on any applications it seems (tried on .NET Hello World-styled application and on notepad.exe on multiple Windows 10 machines).
Essentially I've written a simple WaitForDebugEvent loop:
CreateProcessW(L"C:\\Windows\\SYSWOW64\\notepad.exe", […], CREATE_SUSPENDED, […]);
DebugActiveProcess(processId);
DEBUG_EVENT debugEvent = {};
while (WaitForDebugEvent(&debugEvent, INFINITE)) {
switch (debugEvent.dwDebugEventCode) {
// log all the events
}
ContinueDebugEvent(debugEvent.dwProcessId, debugEvent.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
}
DebugActiveProcessStop(processId);
(here's the full listing: I won't paste it all here, because there's some additional non-essential boilerplate there; the MCVE is 136 lines long)
For the sake of an example, I'll just log all the debugger events and detect whether the debuggee is ready to "proceed normally" or it will terminate due to an exception.
Most of the time, my debugging session looks like that:
CREATE_PROCESS_DEBUG_EVENT (which reports creation of both the process and its initial thread)
LOAD_DLL_DEBUG_EVENT (I was never able to get the name for this DLL, but this is documented in MSDN)
CREATE_THREAD_DEBUG_EVENT (which, I suspect, is a thread injected by debugger)
LOAD_DLL_DEBUG_EVENT […] — after this, many DLLs get loaded into the target process and everything looks okay, the process works as intended
But sometimes (in about 1.5% of all runs), the event sequence changes:
CREATE_PROCESS_DEBUG_EVENT
LOAD_DLL_DEBUG_EVENT
CREATE_THREAD_DEBUG_EVENT
EXCEPTION_DEBUG_EVENT: EXCEPTION_ACCESS_VIOLATION (which I never was able to gather details for: it reports a DEP violation, and the address is empty)
After that, I cannot proceed with debugging, because my debuggee is in exception state and will terminate soon. I was never able to catch notepad.exe crash without my debugger attached (and I doubt it is that bad and will crash for no reason), so I suspect that my debugger causes these exceptions.
One bizarre detail is that I could "fix" the situation by calling Sleep(1) immediately after WaitForDebugEvent. So, this is possibly some sort of race condition, but race condition between what? Between the debugger thread and other threads in the debuggee? Is it a thing? How are we supposed to debug other applications, then? How could actual debuggers work if it is a thing?
I couldn't reproduce the issue with the same code compiled for x64 CPU (and debugging an x64 process).
What could actually cause this erroneous behavior? I've carefully read the documentation about the API functions I call, and checked some other debugger examples online, but still wasn't able to find what's wrong with my debugger: it looks like I follow all the right conventions.
I have tried to debug my debuggee with WinDBG while it is still paused in my debugger, but had no luck doing that. First of all, it's difficult to attach to the debuggee with another debugger (WinDBG only allows to use non-intrusive mode, which is less functional it seems?), and the call stacks for the process' threads aren't usually meaningful.
Steps to reproduce
Checkout this repository, compile with MSVC and then execute in cmd:
Debug\NetRuntimeWaiter.exe > log.txt
It is important to redirect output to the log file and not show it in the terminal: without that, timings for the log writer get changed, and the issue won't reproduce (due to a possible race condition I mentioned earlier?).
Usually the program will start and terminate 1000 notepads in about 10 seconds, and 10-15 of 1000 invocations will hold the error condition (i.e. EXCEPTION_ACCESS_VIOLATION).
the DebugActiveProcess (and undocumented DbgUiDebugActiveProcess which is internally called by DebugActiveProcess) have serious design problem: after calling NtDebugActiveProcess it create remote thread in the target process, via DbgUiIssueRemoteBreakin call - as result new thread in target process is created - DbgUiRemoteBreakin - this thread call DbgBreakPoint and then RtlExitUserThread
all this not documented and explained, only this note from DebugActiveProcess:
After all of this is done, the system resumes all threads in the
process. When the first thread in the process resumes, it executes a
breakpoint instruction that causes an EXCEPTION_DEBUG_EVENT
debugging event to be sent to the debugger.
of course this is wrong. why is DbgUiRemoteBreakin first (??) thread ? and which thread resume first undefined. why not exactly write - we create additional (but not first) thread in process ? and this thread execute breakpoint.
however, when process already running - create this additional thread not create problems. but in case we create process in suspended state, and then just call DebugActiveProcess - the DbgUiRemoteBreakin really became first executing thread in process and process initialization was done on this thread, instead of created first thread. on xp this always lead to fail process initialize at connect to csrss phase. (csrss wait connect to it only on first created thread in process). on later systems this is fixed and process can execute as usual. but can and not, because thread on which it was initialized is exit. it can cause subtle problems.
solution here - not use DebugActiveProcess but NtDebugActiveProcess in it place.
the debug object we can create or via DbgUiConnectToDbg() and then get it via DbgUiGetThreadDebugObject() (system store debug object in thread TEB) or direct by call NtCreateDebugObject
also if we create debuggee process from another process(B) we can do next:
duplicate debug object from debugger process to this B process
call DbgUiSetThreadDebugObject(hDdg) just before call
CreateProcessW with DEBUG_ONLY_THIS_PROCESS or DEBUG_PROCESS
system will be use DbgUiGetThreadDebugObject() for get debug object
from your thread and pass it to low level process create api
remove debug object from your thread via
DbgUiSetThreadDebugObject(0)
really no matter who is create process with debug object. matter who is handle events posted to this debug object.
all undocumented api definitions you can take from ntdbg.h and then link with ntdll.lib or ntdllp.lib
I'm on some c++ mobile product, but I need my apps main thread is still running without any blocking when doing some heavy work on the background thread and run back on main thread. But I realized there is no runOnMainThread/runOnUIThread in c++ thread api. I trying to figure it out the issue and found that need to depend library, or create your own thread event queue. Although it is good, but i am thinking to have a behavior which can runOnUIThread.
How it does not work: the mentioned library creates a timer, installs a SIGALRM signal handler and dispatches queued tasks when signals are fired. This allows tasks being processed on the main thread even when it is busy. However POSIX permits only a small set of async-signal-safe functions to be invoked inside of signal handler. Running arbitrary с++ code inside of signal handler violates that restriction and leaves application in hopelessly doomed state.
After some research and development, I've created a library called NonBlockpp
it is a small c++ library to allow c++ mobile application able to process the heavy and time consuming task on background and back to Main thread again, It’s been tested and fired the main thread event.
It also allow to save the tasks and fire them later, all the task has no blocking each other and thread safety.
How it works:
If you found any query or suggestion, please don't hesitate to raise an issue and we can discuss it together.
The project has rectify from signal to pollEvent due to signal handler might not be safe to use.
Please take a look the new changed.
NonBlockpp
Usage
There is a tool written in C language, based on socket programming and multi-threading.
If I simply run the tool, there is no error and the tool does its work flawlessly.
But if I run the tool in debugging mode, switching between thread is raising SIGPIPE signal and thus data could not be transferred from the tool to its server.
Would handling the SIGPIPE using SIGNAL(SIGPIPE, SIG_IGN) make the thread work?
As far as I understand, this is being raised due to failing of Inter-Process communication between threads. If it is, then how can one debug such multi-threaded application?
Please suggest me the way to handle this issue.
This got resolved by carefully stepping into threads.
The tool not only creates processes but threads also thus improper gdbinit setting and stepping raises SIGPIPE signal.
This is illustrated in the other related post here.
I'm currently playing with the Asynchronous Agents Library in Microsoft's Concurrency Runtime. I have not yet found an obvious way to signal that a task is finished by using window messages, or some other means of notifying the UI thread that the work is finished.
I know I can pass window handles and message values (WM_xxx) along to the tasks, and have the task use PostMessage() to signal the UI thread. This is somewhat ugly in my opinion, and a source of error. If an exception occurs, I have to have a catch handler that signals my UI thread. This is easily forgotten, and the exception condition might not be run very often, so it's hard to spot it.
The documentation talks about how to move data back to the UI thread. It does not make use of window messages, but polling techniques. I find it silly to set up timers to poll if a task has finished, when there are "interrupt" methods available!
It's kind of odd that this isn't built into the library, as it's not a cross platform library. It's designed to run on Windows, and Windows only, from what I understand.
Is the functionality available in the library, or do I have to hand roll this?
You can create one monitor thread with sole function of monitoring an unbounded_buffer for a windows message and dispatching that message appropriately. Have your agents know about this buffer.
I am developing a simple WinAPI application and started from writing my own assertion system.
I have a macro defined like ASSERT(X) which would make pretty the same thing as assert(X) does, but with more information, more options and etc.
At some moment (when that assertion system was already running and working) I realized there is a problem.
Suppose I wrote a code that does some action using a timer and (just a simple example) this action is done while handling WM_TIMER message. And now, the situation changes the way that this code starts throwing an assert. This assert message would be shown every TIMER_RESOLUTION milliseconds and would simply flood the screen.
Options for solving this situation could be:
1) Totally pause application running (probably also, suspend all threads) when the assertion messagebox is shown and continue running after it is closed
2) Make a static counter for the shown asserts and don't show asserts when one of them is already showing (but this doesn't pause application)
3) Group similiar asserts and show only one for each assert type (but this also doesn't pause application)
4) Modify the application code (for example, Get / Translate / Dispatch message loop) so that it suspends itself when there are any asserts. This is good, but not universal and looks like a hack.
To my mind, option number 1 is the best. But I don't know any way how this can be achieved. What I'm seeking for is a way to pause the runtime (something similiar to Pause button in the debugger). Does somebody know how to achieve this?
Also, if somebody knows an efficient way to handle this problem - I would appreciate your help. Thank you.
It is important to understand how Windows UI programs work, to answer this question.
At the core of the Windows UI programming model is of course "the message" queue". Messages arrive in message queues and are retrieved using message pumps. A message pump is not special. It's merely a loop that retrieves one message at a time, blocking the thread if none are available.
Now why are you getting all these dialogs? Dialog boxes, including MessageBox also have a message pump. As such, they will retrieve messages from the message queue (It doesn't matter much who is pumping messages, in the Windows model). This allows paints, mouse movement and keyboard input to work. It will also trigger additional timers and therefore dialog boxes.
So, the canonical Windows approach is to handle each message whenever it arrives. They are a fact of life and you deal with them.
In your situation, I would consider a slight variation. You really want to save the state of your stack at the point where the assert happened. That's a particularity of asserts that deserves to be respected. Therefore, spin off a thread for your dialog, and create it without a parent HWND. This gives the dialog an isolated message queue, independent of the original window. Since there's also a new thread for it, you can suspend the original thread, the one where WM_TIMER arrives.
Don't show a prompt - either log to a file/debug output, or just forcibly break the debugger (usually platform specific, eg. Microsoft's __debugbreak()). You have to do something more passive than show a dialog if there are threads involved which could fire lots of failures.
Create a worker thread for your debugging code. When an assert happens, send a message to the worker thread. The worker thread would call SuspendThread on each thread in the process (except itself) to stop it, and then display a message box.
To get the threads in a process - create a dll and monitor the DllMain for Thread Attach (and Detach) - each call will be done in the context of a thread being created (or destroyed) so you can get the current thread id and create a handle to use with SuspendThread.
Or, the toolhelp debug api will help you find out the threads to pause.
The reason I prefer this approach is, I don't like asserts that cause side effects. Too often Ive had asserts fire from asynchronous socket processing - or window message - processing code - then the assert Message box is created on that thread which either causes the state of the thread to be corrupted by a totally unexpected re-entrancy point - MessageBox also discards any messages sent to the thread, so it messes up any worker threads using thread message queues to queue jobs.
My own ASSERT implementation calls DebugBreak() or as alternative INT 3 (__asm int 3 in MS VC++). An ASSERT should break on the debugger.
Use the MessageBox function. This will block until the user clicks "ok". After this is done, you could choose to discard extra assertion failure messages or still display them as your choice.