The program I'm working on has the following features:
Display the incoming data (thru an ethernet UDP socket)
Manipulate and calculate some data in the header and display
Save a # of frames upon user input
The program has the following structure:
a Main UI thread running (provided by MFC)
a thread which takes the dialog itself, this, as a parameter and is in charge of receiving the data through the UDP socket
a thread which also takes the dialog pointer, this and is in charge of displaying the data
The data receiving thread is using a buffer to store data and the buffer is a member of the dialog class. What makes this possible is, in the display thread, I can access the data stored in the buffer with the use of the dialog pointer and display it; such as StretchBlt(....., pdlg->vbuf[0].data, ....);.
It really isn't much complicated. The program runs smooth until there is a user input (windows messages, or more). It crashes at:
Most frequent crash point
CEXAMPLEAPP::InitInstance()
{
...
CEXAMPLEDlg* pdlg = new CEXAMPLEDlg;
m_pMainWnd = pdlg;
INT_PTR nResponse = pdlg->DoModal(); <--------------------------------------------
if( pdlg!=NULL )
{
delete pdlg;
m_pMainWnd = NULL;
}
...
}
nResponse turns out to have a huuuuge negative value (something like 19 decimal degits) which I don't think is a normal behaviour.
It also crashes at a couple different locations. Most of them are reading violations or dll errors which are hard to interpret and track the cause.
I have done some research, the possible causes of the crashes are:
i) The dynamically allocated dialog object is a local. Therefore the allocated memory isn't enough to process the infinitely incoming data. That's why it's causing reading violations at random locations.
ii) Another possible cause is that most of the variables, including the frame buffers, are member variables - the program is using two separate worker threads and they all are taking the dialog pointer as a parameter. Even though the reading and the writing to the buffer routine are done within CCriticalSection lock(), and unlock() inside the thread functions (global), the variables' ownership could be the possible cause.
This is where I'm at right now. Any ideas to fix the crashes? Any thoughts on using the variables as member variables of the dialog class?
The threads are communicating through SetEvents() and WaitforSingleObject() btw.
When you display the data (StretchBlt(....., pdlg->vbuf[0].data, ....);) in the display thread and this is accessing the display DC, you have to make sure you are doing it in sync with the main thread, for example, by using atomic locks. Or as an alternately better solution, do the display in main thread and remove the display thread altogether.
Related
Back to stackoverflow with another question after hours of trying on my own haha.
Thank you all for reading this and helping in advance.
Please note the console program has following functionalities:
connect to a frame grabber
apply some configs
store the incoming data (640 * 480 16-bit grayscale imgs) in a stream of buffers inside a while loop
Exits the while loop upon a key press.
disconnect from device
And I'm only adding the displaying the images functionality on the MFC GUI app. In short,
i) Converting a console app to an MFC app (dialog based)
ii) decided to use thread for displaying images, but DK how to properly exit from thread when there are certain tasks to be done (such as call disconnectFromDevice(); freeBuffers();, etc) before exiting the thread.
iii) have tried making the while loop condition false but didn't work
( I actually want this to be a callback function that's called repeatedly but IDK how to implement it inside a thread)
iv) forcing AfxEndThread didn't work and it's not even the way it should be done (I think).
So my question is,
1. Are you supposed to use a while loop to excuete a certain job that should repeatedly be done? If not, do you have to implement a callback inside a thread? Or use Windows message loop? Why and how? Please provide a hello-world-like sample code example
(for example, you are printing "hello world" repeatedly inside a thread with a condtion in an MFC GUI app. How do you update or check the condition to end the thread if you can't just AfxEndThread() inside the threadproc)
2. If it's ok with a while, how do you exit from the while loop, in other words how do you properly update the exit condition outside the thread the while loop's in?
Please refer to the source code in the provided link
ctrl+F OnBnClickedConnectButton, AcquireImages and OnBnClickedDisconnectButton
https://github.com/MetaCortex728/img_processing/blob/main/IR140Dlg.cpp
Worker threads do not have message-queues, the (typically one and only) UI one does. The message-queue for a thread is created by the first call of the GetMessage() function. Why use messages to control processing in a worker thread? You would have to establish a special protocol for this, defining custom messages and posting them to the queue.
Worker threads can be implemented as a loop. The loop can be terminated based on various conditions, like failures to retrieve any data or request from the user. You can simply exit the thread proc to terminate the thread's execution. If the thread doesn't respond it may have stuck (unless it performs a really lengthy operation) and the UI thread must provide some mechanism to kill it. That is first request termination and if it doesn't respond within some set time then kill it.
The condition mechanism to terminate should best be some synchronization object (I would recommend a manual-reset event), interlocked variable or a simple boolean which you should access and set using a critical section.
Some considerations:
You pass a parameter block to the thread. Make sure that it remains alive throughout the thread's lifetime. For example, it should NOT be a local variable in a function that exits before the thread's termination.
The loop must be "efficient", ie do not loop infinitely if data are not available. Consider using blocking functions with timeouts, if available.
Resource management (eg connecting/disconnecting, allocating/releasing etc) should best be performed by the same thread.
An alternative implementation can be APCs. Then the thread's proc function is a while(!bTerminate) { SleepEx(INFINITE, TRUE); } loop, and other threads issue requests using a the QueueUserAPC() function.
The AfxEndThread(0) call in OnBnClickedDisconnectButton() is wrong, it terminates the current thread, which in this case is the main (UI) thread. Check the documentation.
A sidenote, my suggestion about the project type is not a dialog-based application but instead a normal MFC application without a document class (uncheck the Document/View architecture support option), as it offers features like menus, toolbars and the like, and most importantly the ON_UPDATE_COMMAND_UI handlers.
I need to do a C++ assignment, but I'm pretty much new to Win32 IPC.
I did a little bit of research yesterday, but I cannot find the thing I'm searching for.
Basically, I need two programs, the first creates a FileMapping with paging file, waits a buffer, display the buffer, and closes it.
The second connects to the Communication Channel, writes the buffer to the first program, and then closes.
The closest thing I've come to is this resource:
IPC Communication
but the guy there uses pipes instead of communication channels using paging file.
Also, I found that I can open a FileMapping with paging file pretty much like this:
TCHAR szMapFileName[] = _T("Local\\HelloWorld");
HANDLE hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE,
NULL, // Default security
PAGE_READWRITE, // Read/write access
0, // Max. object size
BUFFER_SIZE, // Buffer size
szMapFileName // Name of mapping object);
If somebody can provide a little help that would be very valuable(maybe a skeleton of an app?). I tried to do some research yesterday, but in vain.
Thanks
CreateFileMapping(), OpenFileMapping() and MapViewOfFile() are indeed the functions you need to call to allocate a shared memory buffer.
The first app must:
Create the File Mapping.
Create a synchronization object, preferably an Event, to wait on it.
Create an additional thread, which will be waiting for the Event to be signaled. Doing this in the main (UI) thread would block the application.
When the Event is signaled, post a custom message (WM_APP+nnn) to the main thread, to display the contents of the buffer.
It's a matter of specs or design what to do after this point, eg exit the application, just not be waiting for the buffer to receive data anymore, clear the event and wait again, etc.
The second app must:
Open the File Mapping and the Event. If failed, display an error message or warning and exit.
Write the data to the shared memory buffer.
Signal the Event.
Exit.
It could be further improved, eg the second app may not write to the buffer if the event is not in the nonsignaled state.
[I might be wrong about these]
TIdTCPServer Server is multithreaded in Borland C++ builder. It handles all client's on seperate threads. This is written in help of Borland c++.
Now is my problem & question.
For example, ShowMessage(String ..) method should be called on main (gui) thread. But as I say above, TCPServer is multithreaded and handles OnExecute event on different threads. When I use ShowMessage method in OnExecute event (which is handled on a different thread than main thread), I get weird results. Sometimes ShowMessage() works as expected, sometimes it is shown without any text on it with different box sizes(infinite lengt, very long, normal, etc). The other user interface changes causes no problem(update TEdit, TMemo. Only ShowMessage() has problem for now)
I think this problem is the result of calling ShowMessage() method not on the main (gui) thread but on TCPServer's thread which is created for client connections internally by TIdTCPServer.
So how can I fix it?
ShowMessage() displays a VCL TForm, and as such is not thread-safe. You have to use TThread::Synchronize(), TThread::Queue(), TIdSync, TIdNotify, or any other inter-thread communication mechanism of your choosing to make ShowMessage() run on the main thread.
To display a popup message in a worker thread, use the Win32 API MessageBox() function instead. It is thread-safe, and can be called in any thread without synchronizing with the main thread.
All changes to the user interface should be done in the main thread. You can use the TThread::Queue function for executing a function in the main thread. It posts a message to the main message queue, and the TThreadMethod passed as a parameter gets executed when the main thread handles messages.
If you need to pass data to the main thread, e.g. the message to be shown, you'll have to do that separately, as function parameters cannot be passed via the Queue function.
Yes your issue has most probably nothing to do with TCP. Any VCL access must be done within the main thread. (do not forget that message dialogs are often called from VCL wrapers and not directly through winapi)
Yeas I know it sometimes works 'well' even when not, but then this issues appears:
always an exception an App close/exit
occasional App hang/exception (fatal or non fatal for App)
occasional corrupted VCL visual stuff (missing items in lists, missed events, etc)
occasional unpredictable behaviour (sometimes overwrite even App non VCL data)
Many occasional issues are repeatable dependent on:
UI complexity
source code complexity (bigger code more often issues)
number of windows/forms
Also beware of memory leaks. VCL is extremly unstable if memory manager is corrupted by invalid deletes etc... (do not know how about it in newer versions but in bds2006 is this very big issue)
PS. if you need just dialogs then use WINAPI interface it should work even in threads if your text data is not VCL related (for example AnsiString variable access is fine but DBGrid access is not)
I fear that my component is leaking handles.
I see that the number of handles is increasing gradually
Using handle utility i was able to find out the number of handles and the type of each handle.
It appears i am somehow leaking Event Handles. I don't create events at all. Maybe something underlying is causing event handles to be leaked. To narrow my search, i wanted to get the number of handles opened by each thread.
I also tried windbg htrace(snapshot and diff). Was not able to get much information about my threads.
Are there any utilities that can give me information
Per thread or
Name of the handles open
No, event handles are not associated with a thread at all. And they won't typically have a name, it is only used when they need to be shared across processes.
Consider setting a breakpoint on CreateEvent() so you'll at least have an idea where to start looking. Debug + New Breakpoint + Break at Function. Type __imp__CreateEventW#16 for the function name. Just in case, also add a breakpoint for the A version. Or catch them all with _NtCreateEvent#20, debugging symbols required.
My program does file loading and memcpy'ing in the background while the screen is meant to be updated interactively. The idea is to have async loading of files the program will soon need so that they are ready to be used when the main thread needs them. However, the loads/copies don't seem to happen in parallel with the main thread. The main thread pauses during the loading and will often wait for all loads (can be up to 8 at once) to finish before the next iteration of the main thread's main loop.
I'm using Win32, so I'm using _beginthread for creating the file-loading/copying thread.
The worker thread function:
void fileLoadThreadFunc(void *arglist)
{
while(true)
{
// s_mutex keeps the list from being updated by the main thread
s_mutex.lock(); // uses WaitForSingleObject INFINITE
// s_filesToLoad is a list added to from the main thread
while (s_filesToLoad.size() == 0)
{
s_mutex.unlock();
Sleep(10);
s_mutex.lock();
}
loadObj *obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
s_mutex.unlock();
obj->loadFileAndMemcpy();
}
}
main thread startup:
_beginThread(fileLoadThreadFunc, 0, NULL);
code in a class that the main thread uses to "kick" the thread for loading a file:
// I used the commented code to see if main thread was ever blocking
// but the PRINT never printed, so it looks like it never was waiting on the worker
//while(!s_mutex.lock(false))
//{
// PRINT(L"blocked! ");
//}
s_mutex.lock();
s_filesToLoad.push_back(this);
s_mutex.unlock();
Some more notes based on comments:
The loadFileAndMemcpy() function in the worker thread loads via Win32 ReadFile function - does this cause the main thread to block?
I reduced the worker thread priority to either THREAD_PRIORITY_BELOW_NORMAL and THREAD_PRIORITY_LOWEST, and that helps a bit, but when I move the mouse around to see how slowly it moves while the worker thread is working, the mouse "jumps" a bit (without lowering the priority, it was MUCH worse).
I am running on a Core 2 Duo, so I wouldn't expect to see any mouse lag at all.
Mutex code doesn't seem to be an issue since the "blocked!" never printed in my test code above.
I bumped the sleep up to 100ms, but even 1000ms doesn't seem to help as far as the mouse lag goes.
Data being loaded is tiny - 20k .png images (but they are 2048x2048).. they are small size since this is just test data, one single color in the image, so real data will be much larger.
You will have to show the code for the main thread to indicate how it is notified that it a file is loaded. Most likely the blocking issue is there. This is really a good case for using asynchronous I/O instead of threads if you can work it into your main loop. If nothing else you really need to use conditions or events. One to trigger the file reader thread that there is work to do, and another to signal the main thread a file has been loaded.
Edit: Alright, so this is a game, and you're polling to see if the file is done loading as part of the rendering loop. Here's what I would try: use ReadFileEx to initiate an overlapped read. This won't block. Then in your main loop you can check if the read is done by using one of the Wait functions with a zero timeout. This won't block either.
Not sure on your specific problem but you really should mutex-protect the size call as well.
void fileLoadThreadFunc(void *arglist) {
while (true) {
s_mutex.lock();
while (s_filesToLoad.size() == 0) {
s_mutex.unlock();
Sleep(10);
s_mutex.lock();
}
loadObj *obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
s_mutex.unlock();
obj->loadFileAndMemcpy();
}
}
Now, examining your specific problem, I can see nothing wrong with the code you've provided. The main thread and file loader thread should quite happily run side-by-side if that mutex is the only contention between them.
I say that because there may be other points of contention, such as in the standard library, that your sample code doesn't show.
I'd write that loop this way, less locking unlock which could get messed up :P :
void fileLoadThreadFunc(void *arglist)
{
while(true)
{
loadObj *obj = NULL;
// protect all access to the vector
s_mutex.lock();
if(s_filesToLoad.size() != 0)
{
obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
}
s_mutex.unlock();
if( obj != NULL )
obj->loadFileAndMemcpy();
else
Sleep(10);
}
}
MSDN on Synchronization
if you can consider open source options, Java has a blocking queue [link] as does Python [link]. This would reduce your code to (queue here is bound to load_fcn, i.e. using a closure)
def load_fcn():
while True:
queue.get().loadFileAndMemcpy()
threading.Thread(target=load_fcn).start()
Even though you're maybe not supposed to use them, python 3.0 threads have a _stop() function and python2.0 threads have a _Thread__stop function. You could also write a "None" value to the queue and check in load_fcn().
Also, search stackoverflow for "[python] gui" and "[subjective] [gui] [java]" if you wish.
Based on the information present at this point, my guess would be that something in handler for the file loading is interacting with your main loop. I do not know the libraries involved, but based on your description the file handler does something along the following lines:
Load raw binary data for a 20k file
Interpret the 20k as a PNG file
Load into a structure representing a 2048x2048 pixel image
The following possibilities come to mind regarding the libraries you use to achieve these steps:
Could it be that the memory allocation for the uncompressed image data is holding a lock that the main thread needs for any drawing / interactive operations it performs?
Could it be that a call that is responsible for translating the PNG data into pixels actually holds a low-level game library lock that adversely interacts with your main thread?
The best way to get some more information would be to try and model the activity of your file loader handler without using the current code in it... write a routine yourself that allocates the right size of memory block and performs some processing that turns 20k of source data into a structure the size of the target block... then add further logic to it one bit at a time until you narrow down when performance crashes to isolate the culprit.
I think that your problem lies with access to the FilesToLoad object.
As I see it this object is locked by your thread when the it is actually processing it (which is every 10ms according to your code) and by your main thread as it is trying to update the list. This probably means that your main thread is waiting for a while to access it and/or the OS sorts out any race situations that may occur.
I would would suggest that you either start up a worker thread just to load a file when you as you want it, setting a semaphore (or even a bool value) to show when it has completed or use _beginthreadex and create a suspended thread for each file and then synchronise them so that as each one completes the next in line is resumed.
If you want a thread to run permenently in the background erasing and loading files then you could always have it process it's own message queue and use windows messaging to pass data back and forth. This saves a lot of heartache regarding thread locking and race condition avoidance.