I have a C++ function called write() that is supposed to write a large number of bytes on the users HDD. This woud normally take more than 30 seconds, so I want to give the user the ability to Abort operation from a GUI (created with Qt).
So I have the button generate_button. When this button is clicked(), I want to have something to end the function, wherever its progress is.
I thought about threads, but I am not sure. Can you advice me please?
I would probably use a thread. It should be quite simple to check a variable to see if the operation has been canceled.
Use a mutex to lock access to your cancel variable. That will make sure it is read and written in a proper way for multiple threads. Another option is if you are using C++11 use an atomic variable.
Break your large write into blocks of smaller size. 8 to 64 kilobytes should work. After writing each block check your cancel variable and if set, exit the thread.
Place the code that actually does the writing in a worker thread. Have a shared variable (one that is either atomic, or protected by a mutex). Have the worker thread check its value each iteration. If the user presses the "Abort" button, set the value for the variable.
You should use threads if this is a long running operation.
Since you are using C++11, std::atomic<bool> would probably serve you well.
Threaded guarantees that you will have a responsive GUI. But there is a learning curve to using a thread in this manner.
A threadless way to do this is to have in your routine that writes to the harddrive in the GUI thread, but gives time to the GUI thread to stay responsive.
QObject::connect(my_cancel_button, SIGNAL(clicked()), file_writer, SLOT(setCanceled()));
// open file for writing
QFile file("filename.txt");
file.open(//... );//
while(still_have_data_to_write && !canceled)
{
write( <1 MB of data> ); // or some other denomination of data
qApp->processEvents();// allows the gui to respond to events such as clicks on buttons
// update a progress bar... using a counter as a ratio of the total file size
emit updateProgressBar(count++);
}
if( canceled )
{
file.close();
// delete the partial file using QDir
}
Hope that helps.
Related
Back to stackoverflow with another question after hours of trying on my own haha.
Thank you all for reading this and helping in advance.
Please note the console program has following functionalities:
connect to a frame grabber
apply some configs
store the incoming data (640 * 480 16-bit grayscale imgs) in a stream of buffers inside a while loop
Exits the while loop upon a key press.
disconnect from device
And I'm only adding the displaying the images functionality on the MFC GUI app. In short,
i) Converting a console app to an MFC app (dialog based)
ii) decided to use thread for displaying images, but DK how to properly exit from thread when there are certain tasks to be done (such as call disconnectFromDevice(); freeBuffers();, etc) before exiting the thread.
iii) have tried making the while loop condition false but didn't work
( I actually want this to be a callback function that's called repeatedly but IDK how to implement it inside a thread)
iv) forcing AfxEndThread didn't work and it's not even the way it should be done (I think).
So my question is,
1. Are you supposed to use a while loop to excuete a certain job that should repeatedly be done? If not, do you have to implement a callback inside a thread? Or use Windows message loop? Why and how? Please provide a hello-world-like sample code example
(for example, you are printing "hello world" repeatedly inside a thread with a condtion in an MFC GUI app. How do you update or check the condition to end the thread if you can't just AfxEndThread() inside the threadproc)
2. If it's ok with a while, how do you exit from the while loop, in other words how do you properly update the exit condition outside the thread the while loop's in?
Please refer to the source code in the provided link
ctrl+F OnBnClickedConnectButton, AcquireImages and OnBnClickedDisconnectButton
https://github.com/MetaCortex728/img_processing/blob/main/IR140Dlg.cpp
Worker threads do not have message-queues, the (typically one and only) UI one does. The message-queue for a thread is created by the first call of the GetMessage() function. Why use messages to control processing in a worker thread? You would have to establish a special protocol for this, defining custom messages and posting them to the queue.
Worker threads can be implemented as a loop. The loop can be terminated based on various conditions, like failures to retrieve any data or request from the user. You can simply exit the thread proc to terminate the thread's execution. If the thread doesn't respond it may have stuck (unless it performs a really lengthy operation) and the UI thread must provide some mechanism to kill it. That is first request termination and if it doesn't respond within some set time then kill it.
The condition mechanism to terminate should best be some synchronization object (I would recommend a manual-reset event), interlocked variable or a simple boolean which you should access and set using a critical section.
Some considerations:
You pass a parameter block to the thread. Make sure that it remains alive throughout the thread's lifetime. For example, it should NOT be a local variable in a function that exits before the thread's termination.
The loop must be "efficient", ie do not loop infinitely if data are not available. Consider using blocking functions with timeouts, if available.
Resource management (eg connecting/disconnecting, allocating/releasing etc) should best be performed by the same thread.
An alternative implementation can be APCs. Then the thread's proc function is a while(!bTerminate) { SleepEx(INFINITE, TRUE); } loop, and other threads issue requests using a the QueueUserAPC() function.
The AfxEndThread(0) call in OnBnClickedDisconnectButton() is wrong, it terminates the current thread, which in this case is the main (UI) thread. Check the documentation.
A sidenote, my suggestion about the project type is not a dialog-based application but instead a normal MFC application without a document class (uncheck the Document/View architecture support option), as it offers features like menus, toolbars and the like, and most importantly the ON_UPDATE_COMMAND_UI handlers.
My application needs to read files from a nfs mount point. I'm using QFile::read(BUFFER_SIZE); which works as it is supposed to. The problem is, when the nfs unmounts (patchy network issues), this QFile::read() never timesout resulting in application hang. Any sensible solutions please?
Thanks.
You can use QFile::bytesAvailable () inherited from QIODevice which returns the number of bytes that are available for reading. Wait in a while loop until the bytes are available or some seconds elapsed :
QFile file("file.dat");
file.open(QIODevice::ReadOnly);
QElapsedTimer timer;
timer.start();
while(file.bytesAvailable()<BUFFER_SIZE || timer.nsecsElapsed()<3e9)
qApp->processEvents();
QByteArray array
if(file.waitForReadyRead())
{
array = file.read(BUFFER_SIZE);
}
Here it will wait 3 seconds for the data to be ready. The elapsed time is measured by QElapsedTimer. If BUFFER_SIZE bytes are not available after the timeout then it will continue with out hanging.
I think this is due to how NFS works. The process is stuck in the OS system call, in uninterruptible sleep. So, I think you have two reliable options:
Move file reading to another thread. It and its event loop may get stuck, just code your main thread so that it won't matter for the whole application. There are a few ways to go about using threads for this, but if you are not reading arbitrary number of different files at the same time, it'd probably be best to launch a dedicated thread for reading the single file, and then let the thread finish once everything is read. For this, you might even just subclass QThread, and override run() method with your own reading loop (you can still emit signals for progress if you want, you just can't receive them queued without Qt's event loop).
Move file reading to another process. In other words, write a small tool program or script which does the actual reading, and sends the contents to your main process. That way you should be able to kill the reading process without killing the whole application, should that become necessary. Again you have to decide if you want a separate single-use process for each reading (so you can give file name as command line paramater, for example), or if you launch one child process and implement communication with it so you can request files (or parts of files). Qt's QProcess makes this approach easy, too.
You can use QIODevice::readChannelFinished() signal and connect it to a slot that close the file reader
I have a function that process frames of video frame by frame and it's quite long. So I decided to put something like Pause and Resume button but I don't know how to end a running function using button. I guessed that it have to do something with thread but I don't know how to do it. Can you give me some advice?
(Note: I'm naming your function ProcessFramesOfVideo() just to make it easier to refer to below)
If the ProcessFramesOfVideo() function is running in your main thread, then most likely your GUI gets frozen up while the function runs, because the GUI event loop won't run until ProcessFramesOfVideo returns.
If that's the case, then the quick-and-dirty way to un-freeze your GUI while ProcessFramesOfVideo() is running would be to call qApp->processEvents() periodically (e.g. once every 50mS or something) from within your loop inside ProcessFramesOfVideo(). That would allow Qt to handle incoming GUI events (like mouse clicks) in a reasonably timely manner while the video is being processed, so that e.g. if the user resizes the window while video is being processed, the widgets resize as expected. Once you have that working, then it's just a matter of putting your Pause and Resume buttons in the GUI as usual, and maybe have them set a "please_stop_processing" boolean flag that also gets checked periodically by ProcessFramesOfVideo()'s loop; if the flag is true, ProcessFramesOfVideo() could return early, or etc.
If you are looking for a more elegant solution than that, the other option would be to spawn a QThread and run the ProcessFramesOfVideo() method in that separate thread. That would allow ProcessFramesOfVideo() to run truly in parallel with the Qt GUI event loop (especially on a multicore CPU), so that GUI responsiveness would not be reduced at all. ProcessFramesOfVideo() could still use the old periodic-check-of-a-boolean technique to know if it should pause or resume its processing; but since you'd have multiple threads involved you'd want to the GUI thread lock a mutex before writing to the boolean, and unlock the mutex afterwards, and you'd want to have the ProcessFramesOfVideo() function lock that same mutex before reading the boolean, and unlock the mutex afterwards. That would ensure that the CPUs' caches and/or compiler optimizations do not interfere with the threads "seeing" each others changes to the boolean flag.
Got really stuck, need some advise or real examples.
1) I have boost::thread vector producer thread (data arrives fast ~ 100 samples per second)
2) I want QMathGL to paint data as it arrives
3) I don't want my Qt gui freeze
I tried to move QMathGL::update() to separate thread - Qt argues that QPixmap not allowed in separate thread.
What should i try, Without modifying QMathGL?
Only thing comes in mind to repaint on timer (fps?), but i don't like this solution, please tell me if i am wrong.
I would strongly advise to go with a timer. Repaint operations are costly and I would assume that no user could realistically process more then 10 printed vectors a second. So I can't see a real benefit for the end user, apart from maybe that the display is updated more "smoothly" and entry for entry. But you could achieve these effects far easier with animations ;)
When repainting with every data change, you get the annoying behaviour you describe. Working around that is (imho) not worth the trouble.
I´ve also come along a similar problem sometimes.
The usual resolution i used is to buffer the data and repainting on a timer. This goes along the line of this (Pseudo Code) :
void Widget::OnNewData(void *dataSample)
{
this->threadSafebuffer->appendData(dataSample);
}
void Widget::OnTimeout()
{
DataBuffer renderBatch = this->threadSafebuffer->interlockedExchange();
/* Do UI updates according to renderBatch */
}
This assumes that OnNewData is called on a background thread. The OnTimeout is called from a QTimer on the UI-EventLoop. To prevent contention it justs does an interlocked exchange of the current buffer pointer with a second buffer. So no heavy synchronization (e.g. Mutext/Semaphore) is needed.
This will only work if the amount of work to do for rendering a renderBatch is less than the timeout.
My program does file loading and memcpy'ing in the background while the screen is meant to be updated interactively. The idea is to have async loading of files the program will soon need so that they are ready to be used when the main thread needs them. However, the loads/copies don't seem to happen in parallel with the main thread. The main thread pauses during the loading and will often wait for all loads (can be up to 8 at once) to finish before the next iteration of the main thread's main loop.
I'm using Win32, so I'm using _beginthread for creating the file-loading/copying thread.
The worker thread function:
void fileLoadThreadFunc(void *arglist)
{
while(true)
{
// s_mutex keeps the list from being updated by the main thread
s_mutex.lock(); // uses WaitForSingleObject INFINITE
// s_filesToLoad is a list added to from the main thread
while (s_filesToLoad.size() == 0)
{
s_mutex.unlock();
Sleep(10);
s_mutex.lock();
}
loadObj *obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
s_mutex.unlock();
obj->loadFileAndMemcpy();
}
}
main thread startup:
_beginThread(fileLoadThreadFunc, 0, NULL);
code in a class that the main thread uses to "kick" the thread for loading a file:
// I used the commented code to see if main thread was ever blocking
// but the PRINT never printed, so it looks like it never was waiting on the worker
//while(!s_mutex.lock(false))
//{
// PRINT(L"blocked! ");
//}
s_mutex.lock();
s_filesToLoad.push_back(this);
s_mutex.unlock();
Some more notes based on comments:
The loadFileAndMemcpy() function in the worker thread loads via Win32 ReadFile function - does this cause the main thread to block?
I reduced the worker thread priority to either THREAD_PRIORITY_BELOW_NORMAL and THREAD_PRIORITY_LOWEST, and that helps a bit, but when I move the mouse around to see how slowly it moves while the worker thread is working, the mouse "jumps" a bit (without lowering the priority, it was MUCH worse).
I am running on a Core 2 Duo, so I wouldn't expect to see any mouse lag at all.
Mutex code doesn't seem to be an issue since the "blocked!" never printed in my test code above.
I bumped the sleep up to 100ms, but even 1000ms doesn't seem to help as far as the mouse lag goes.
Data being loaded is tiny - 20k .png images (but they are 2048x2048).. they are small size since this is just test data, one single color in the image, so real data will be much larger.
You will have to show the code for the main thread to indicate how it is notified that it a file is loaded. Most likely the blocking issue is there. This is really a good case for using asynchronous I/O instead of threads if you can work it into your main loop. If nothing else you really need to use conditions or events. One to trigger the file reader thread that there is work to do, and another to signal the main thread a file has been loaded.
Edit: Alright, so this is a game, and you're polling to see if the file is done loading as part of the rendering loop. Here's what I would try: use ReadFileEx to initiate an overlapped read. This won't block. Then in your main loop you can check if the read is done by using one of the Wait functions with a zero timeout. This won't block either.
Not sure on your specific problem but you really should mutex-protect the size call as well.
void fileLoadThreadFunc(void *arglist) {
while (true) {
s_mutex.lock();
while (s_filesToLoad.size() == 0) {
s_mutex.unlock();
Sleep(10);
s_mutex.lock();
}
loadObj *obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
s_mutex.unlock();
obj->loadFileAndMemcpy();
}
}
Now, examining your specific problem, I can see nothing wrong with the code you've provided. The main thread and file loader thread should quite happily run side-by-side if that mutex is the only contention between them.
I say that because there may be other points of contention, such as in the standard library, that your sample code doesn't show.
I'd write that loop this way, less locking unlock which could get messed up :P :
void fileLoadThreadFunc(void *arglist)
{
while(true)
{
loadObj *obj = NULL;
// protect all access to the vector
s_mutex.lock();
if(s_filesToLoad.size() != 0)
{
obj = s_filesToLoad[0];
s_filesToLoad.erase(s_filesToLoad.begin());
}
s_mutex.unlock();
if( obj != NULL )
obj->loadFileAndMemcpy();
else
Sleep(10);
}
}
MSDN on Synchronization
if you can consider open source options, Java has a blocking queue [link] as does Python [link]. This would reduce your code to (queue here is bound to load_fcn, i.e. using a closure)
def load_fcn():
while True:
queue.get().loadFileAndMemcpy()
threading.Thread(target=load_fcn).start()
Even though you're maybe not supposed to use them, python 3.0 threads have a _stop() function and python2.0 threads have a _Thread__stop function. You could also write a "None" value to the queue and check in load_fcn().
Also, search stackoverflow for "[python] gui" and "[subjective] [gui] [java]" if you wish.
Based on the information present at this point, my guess would be that something in handler for the file loading is interacting with your main loop. I do not know the libraries involved, but based on your description the file handler does something along the following lines:
Load raw binary data for a 20k file
Interpret the 20k as a PNG file
Load into a structure representing a 2048x2048 pixel image
The following possibilities come to mind regarding the libraries you use to achieve these steps:
Could it be that the memory allocation for the uncompressed image data is holding a lock that the main thread needs for any drawing / interactive operations it performs?
Could it be that a call that is responsible for translating the PNG data into pixels actually holds a low-level game library lock that adversely interacts with your main thread?
The best way to get some more information would be to try and model the activity of your file loader handler without using the current code in it... write a routine yourself that allocates the right size of memory block and performs some processing that turns 20k of source data into a structure the size of the target block... then add further logic to it one bit at a time until you narrow down when performance crashes to isolate the culprit.
I think that your problem lies with access to the FilesToLoad object.
As I see it this object is locked by your thread when the it is actually processing it (which is every 10ms according to your code) and by your main thread as it is trying to update the list. This probably means that your main thread is waiting for a while to access it and/or the OS sorts out any race situations that may occur.
I would would suggest that you either start up a worker thread just to load a file when you as you want it, setting a semaphore (or even a bool value) to show when it has completed or use _beginthreadex and create a suspended thread for each file and then synchronise them so that as each one completes the next in line is resumed.
If you want a thread to run permenently in the background erasing and loading files then you could always have it process it's own message queue and use windows messaging to pass data back and forth. This saves a lot of heartache regarding thread locking and race condition avoidance.