How to solve the problem that multithreaded drawing is not smooth? - c++

I wrote a data acquisition program with Qt. I collect data using the child threads of the dual cache region written by QSemphore.
void QThreadShow::run() {
m_stop=false; // when start thread,m_stop=false
int n=fullBufs.available();
if (n>0)
fullBufs.acquire(n);
while (!m_stop) {
fullBufs.acquire(); // wait fo full buffer
QVector<double> dataPackage(BufferSize);
double seq=bufNo;
if (curBuf==1)
for (int i=0;i<BufferSize;i++){
dataPackage[i]=buffer2[i]; // copy data from full buffer
}
else
for (int i=0;i<BufferSize;i++){
dataPackage[i]=buffer1[i];
}
for (int k=0;k<BufferSize;k++) {
vectorQpointFbufferData[k]=QPointF(x,dataPackage[k]);
}
emptyBufs.release(); // release a buffer
QVariant variantBufferData;
variantBufferData.setValue(vectorQpointFbufferData);
emit newValue(variantBufferData,seq); // send data to main thread
}
quit();
}
When a cache of sub-threads has collected 500 data, the data is input into a QVector and sent to the main thread and is directly assigned to a lineseries in qchartview every 20ms for drawing. I use QtChart to chart the data.
void MainWindow::onthreadB_newValue(QVariant bufferData, double bufNo) {
// Analysis of QVariant data
CH1.hardSoftDataPointPackage = bufferData.value<QVector<QPointF>>();
if (ui->CH1_Source->currentIndex()==0) {
for (int p = 0;p<CH1.hardSoftDataPointPackage.size();p++) {
series_CH3->append(CH1.hardSoftDataPointPackage[p]);
}
}
}
There is a timer in the main thread.The interval is 20ms and there is a double time (time = time +1), which controls the X-axis.
void MainWindow::drawAxis(double time) {
// dynamic draw x axis
if (time<100) {
axisX->setRange(0, TimeBase/(1000/FrameRate) * 10);
// FrameRate=50
} else {
axisX->setRange(time-TimeBase/(1000/FrameRate) * 10, time);
}
}
But when I run my program, there is a problem that every time the subthread sends data to the main thread, the main thread gets stuck for a few seconds and the plot also gets stuck for a few seconds. I added a curve in the main thread getting data from the main thread, and found that both two curves will be stuck at the same time. I don't know how to solve this problem.
Besides, I want the main thread to draw the data from the child thread evenly within 20ms, instead of drawing all the points at once.

Your main thread stucks because you copy (add to series) a lot of data at one time. Instead this you can collect all your data inside your thread instance without emitting a signal. And from main thread just take little pieces of collected data every 20 ms.
Something like this:
while(!m_stop)
{
...
//QVariant variantBufferData;
//variantBufferData.setValue(vectorQpointFbufferData);
//emit newValue(variantBufferData,seq);//send data to main thread
//instead this just store in internal buffer
m_mutex.lock();
m_internalBuffer.append(vectorQpointFbufferData);
m_mutex.unlock();
}
Read method
QVector<QPointF> QThreadShow::takeDataPiece()
{
int n = 4;
QVector<QPointF> piece;
piece.reserve(n);
m_mutex.lock();
for (int i = 0; i < n; i++)
{
QPointF point = m_internalBuffer.takeFirst();
piece.append(point);
}
m_mutex.unlock();
return piece;
}
And in Main thread read in timeout slot
void MainWindow::OnDrawTimer()
{
QVector<QPointF> piece = m_childThread.takeDataPiece();
//add to series
...
//drawAxis
...
}

Related

detached std::thread on esp32 arduino sometimes blocks, sometimes doesn't

I have some code running on ESP32 microcontroller with arduino core,
In the setup() function I wish to have some code threadPressureCalib run independently in its own thread, so I do the following:
std::unique_ptr<std::thread> sensorCalib;
void setup()
{
sensorCalib.reset(new std::thread(threadPressureCalib));
std::thread* pc = sensorCalib.get();
pc->detach();
}
void loop()
{
...
}
Then, I define threadPressureCalib() as follows:
void threadPressureCalib()
{
float pressure=0;
int count;
for(timestarted = millis();(millis()-timestarted) < 10000;)
{ // THIS ONE BLOCKS SETUP() AND LOOP() CODE EXECUTION
Serial.println("Doing things");
}
Serial.println("Doing other things");
for (count=1; count<= 5;count++)
{ //THIS ONE DOES NOT BLOCK SETUP() and LOOP()
float temp;
while(!timer2.Delay(2000)); //Not sure if this is blocking anything
do{
temp = adc_pressure();
}while(temp>104.0 || temp<70.0); //Catch errors
pressure += temp;
}
changeSetting(pressure/5.0);
return;
}
Problem: During the first for loop, the setup() function's execution is stopped (as well as loop())
During the second for loop, nothing is stopped and the rest of the code runs in parallel (as expected)
Why is it that the first half of this code blocks, and then the second half does not?
Sorry if the question is vague or improperly asked, my first q here.
Explanation of timer2 per request in comments:
timer2 is a custom timer class, timer2.Delay(TIMEOUT) stores timestamp the first time it's called and returns false on every subsequent call until the current time = TIMEOUT, then it returns true and resets itself
NonBlockDelay timer2;
//time delay function (time in seconds to delay)
// Set iTimeout to current millis plus milliseconds to wait for
/**
* Called with milliseconds to delay.
* Return true if timer expired
*
*/
//Borrowed from someone on StackOverflow...
bool NonBlockDelay::Delay (unsigned long t)
{
if(TimingActive)
{
if((millis() >iTimeout)){
TimingActive = 0;
return(1);
}
return(0);
}
iTimeout = millis() + t;
TimingActive = 1;
return(0);
};
// returns true if timer expired
bool NonBlockDelay::Timeout (void)
{
if(TimingActive){
if((millis() >iTimeout)){
TimingActive = 0;
iTimeout = 0;
return(1);
}
}
return(false);
}
// Returns the current timeout value in milliseconds
unsigned long NonBlockDelay::Time(void)
{
return iTimeout;
}
There is not enough information here to tell you the answer but it seems that you have no idea what you are doing.
std::unique_ptr<std::thread> sensorCalib;
void setup(){
sensorCalib.reset(new std::thread(threadPressureCalib));
std::thread* pc = sensorCalib.get();
pc->detach();
}
So here you store a new thread that executes threadPressureCalib then immediately detach it. Once the thread is detached the instance std::thread no longer manages it. So what's the point of even having std::unique_ptr<std::thread> sensorCalib; in the first place if it literally does nothing? Do you realize that normally you need to join the thread if you wish to wait till it's completion? Could it be that you just start a bunch of instances of these threadPressureCalib - as you probably don't verify that they finished execution - and they interfere with each other?

Taking screenshots of widgets on a separate thread QT C++

I'm trying to periodically take screenshots of a list of widgets. I've connected a slot to a timer and once the timer ends the slot below is executed.
void SlotUpdateScreenshots()
{
QtConcurrent::run([=]() {
std::unique_lock<std::mutex> lock(m_Mutex);
QList<Workspace*> workspaces = GetAllWorkspaces();
for (int i = 0; i < workspaces.size(); i++) {
m_WorkspaceScreenshots[workspaces.at(i)].clear();
std::map<WorkspacePoint, Layout*>layoutMap = workspaces.at(i)->GetlayoutMap();
//Declare a list out layouts
QList<Layout*> LayoutList;
for (auto layout : layoutMap)
{
LayoutList.append(layout.second);
}
for (int j = 0; j < LayoutList.size(); j++)
{
QPixmap pix = LayoutList.at(j)->grab();
m_WorkspaceScreenshots[workspaces.at(i)].append(pix);
}
}
});
}
This slot goes through the widgets in question and takes screenshots of them by using the "grab" function. This works perfectly without the concurrency but it does clog up the main thread and causes the UI to freeze if there are lots of widgets to take screenshots of, which is why I want to move this to a thread.
However once I attempt to run this slot on a thread the grab function causes a crash as the widgets are created and owned by another thread;
ASSERT failure in QCoreApplication::sendEvent: "Cannot send events to objects owned by a different thread. Current thread 0x0x23ce917b2a0. Receiver 'qt_scrollarea_hcontainer' (of type 'QWidget') was created in thread 0x0x23ce408b650", file kernel\qcoreapplication.cpp, line 577
How can I take screenshots of these widgets from the thread I'm currently in without causing a crash?
I'd appreciate any input.

Caffe feature extraction blocks other executing threads (Qt / C++)

Background
I am developing a Qt application with three threads: main, thread1, and thread2.
main creates, starts, and displays the results of thread1 and
thread2, while also iteratively feeding them input.
thread1 performs an intensive computation (~1s) once every n
inputs, ignoring all other inputs. The majority of this time is spent on feature extraction
using the Caffe framework.
thread2 performs fast computations (~20ms) on every input. But
every n+1 inputs depend on the output of feeding input n to thread1.
During thread execution, thread1 appears to block thread2 when extracting features using the Caffe network. However, thread1 does not block thread2 during other processing steps (e.g. network input preprocessing).
At first, I thought this was caused by an unmet dependency: i.e. thread1 "blocks" thread2 because input (for example) 2n + 1 is ready to be processed by thread2 but input 2n has not been fully processed by thread1.
However, from analysing the execution flow, I noticed this "blocking" behaviour was occurring whilst dependencies were met: i.e. let n = 10, thread2 would pause execution at input 15 while thread1 was extracting Caffe features from input 20.
Question
How do I prevent thread2 from blocking thread1 during Caffe feature extraction?
Code
Below is a stripped-down version of my code, which shows the key components and logic of my program.
I have highlighted the problem in comments // !!! PROBLEM: ..., which can be found in thread1worker.cpp and featureengine.cpp.
main.cpp:
int main(int argc, char* argv[])
{
QApplication app(argc, argv);
qRegisterMetaType<Mat>("Mat");
Camera camera; // grabs camera frame every 30ms, emitting newFrame(frame)
/* thread1 */
QThread* thread1 = new QThread();
Thread1Worker* thread1_worker = new Thread1Worker();
thread1_worker->moveToThread(thread1);
QThread::connect(&camera, SIGNAL(newFrame(Mat)),
thread1_worker, SLOT(doWork(Mat)));
QThread::connect(thread1, SIGNAL(finished()),
thread1_worker, SLOT(deleteLater()));
QThread::connect(thread1, SIGNAL(finished()),
thread1, SLOT(deleteLater()));
/* thread2 */
ImageQueue* thread2_images = new ImageQueue();
QThread::connect(camera, SIGNAL(newFrame(Mat)),
thread2_images, SLOT(add(Mat)));
QThread* thread2 = new QThread();
Thread2Worker* thread2_worker = new Thread2Worker(thread2_images);
thread2_worker->moveToThread(thread2);
QThread::connect(thread2_worker, SIGNAL(workFinished(OutputType)),
thread2_worker, SLOT(addThread1Result(OutputType)));
QThread::connect(thread2, SIGNAL(finished()),
thread2_worker, SLOT(deleteLater()));
QThread::connect(thread2, SIGNAL(finished()),
thread2, SLOT(deleteLater()));
/* start threads */
thread1->start();
thread2->start();
camera->start();
return app.exec();
}
thread1worker.cpp:
Thread1Worker::Thread1Worker()
{
thread1_interval = 10; // this is "n"
is_initialized = false;
}
void Thread1Worker::doWork(Mat frame)
{
if (!is_initialized)
initialize();
// process only every nth frame
if (!isThread1Frame())
return;
// ... break frame up into multiple image patches
// !!! PROBLEM: this call blocks thread2
vector<vector<float> > features = feature_engine->extractFeatures(patches);
// ... use features to compute output
frame_count++;
emit workFinished(output);
}
void Thread1Worker::initialize()
{
InitGoogleLogging("caffe-demo");
feature_engine = new FeatureEngine();
is_initialized = true;
}
bool Thread1Worker::isThread1Frame()
{
return frame_count % thread1_interval == 0;
}
thread2worker.cpp:
void Thread2Worker::addThread1Result(OutputType output)
{
if (!is_initialized)
initialize();
thread1_output_queue.push(output);
thread1_count++;
processFrames();
}
void Thread2Worker::processFrames()
{
size_t num_process = (thread1_count * thread1_interval) - process_count;
size_t num_queue = thread2_images->size();
for (size_t i = 0; i < num_process && i < num_queue; i++)
{
Mat frame = thread2_images->get();
if (isThread1Frame()
{
curr_result = thread1_output_queue.front();
thread1_output_queue.pop();
}
else
{
curr_result = propagator->propagate(prev_result);
}
// update
prev_result = curr_result;
emit resultReady(curr_result);
}
}
void Thread2Worker::initialize()
{
propagator = new Propagator();
is_initialized = true;
}
bool Thread2Worker::isThread1Frame()
{
return process_count % thread1_interval == 0;
}
featureengine.cpp:
vector<vector<float> > FeatureEngine::extractFeatures(const vector<Mat>& images)
{
// setup Caffe network for feature extraction:
Blob<float>* input_layer = net->input_blobs()[0];
int num_images = images.size();
int height = input_geometry.height;
int width = input_geometry.width;
input_layer->Reshape(num_images, num_channels, height, width);
net->Reshape();
vector<Mat> input_channels;
wrapInputLayer(input_channels);
preprocess(images, &input_channels);
// !!! PROBLEM: this ~1s computation blocks thread2
// details: https://github.com/BVLC/caffe/blob/master/src/caffe/net.cpp#L594
net->ForwardPrefilled();
// copy Caffe network output to features vector
vector<vector<float> > features;
// ...
return features;
}

Qt timers cannot be stopped from another thread

Hy,
I'm writing my first Qt program and getting now in troubles with:
QObject::killTimer: timers cannot be stopped from another thread
QObject::startTimer: timers cannot be started from another thread
My program will communicate to a CANOpen bus for that I'm using the Canfestival Stack. The Canfestival will work with callback methods. To detects timeout in communication I setup a timer function (somehow like a watchdog). My timer package consist out of a "tmr" module, a "TimerForFWUpgrade" module and a "SingleTimer" module. The "tmr" module was originally C programmed so the static "TimerForFWUpgrade" methods will interface it. The "tmr" module will be part of a C programed Firmware update package.
The timer will work as follows. Before a message is sent I will call TMR_Set method. An then in my idle program loop with TMR_IsElapsed we check for a timer underflow. If TMR_IsElapsed I will do the errorhandling. As you see the TMR_Set method will be called continuously and restart the QTimer again and again.
The above noted errors are appearing if I start my program. Can you tell me if my concept could work? Why does this errors appear? Do I have to use additional threads (QThread) to the main thread?
Thank you
Matt
Run and Idle loop:
void run
{
// start communicate with callbacks where TMR_Set is set continously
...
while(TMR_IsElapsed(TMR_NBR_CFU) != 1);
// if TMR_IsElapsed check for errorhandling
....
}
Module tmr (interface to C program):
extern "C"
{
void TMR_Set(UINT8 tmrnbr, UINT32 time)
{
TimerForFWUpgrade::set(tmrnbr, time);
}
INT8 TMR_IsElapsed(UINT8 tmrnbr)
{
return TimerForFWUpgrade::isElapsed(tmrnbr);
}
}
Module TimerForFWUpgrade:
SingleTimer* TimerForFWUpgrade::singleTimer[NR_OF_TIMERS];
TimerForFWUpgrade::TimerForFWUpgrade(QObject* parent)
{
for(unsigned char i = 0; i < NR_OF_TIMERS; i++)
{
singleTimer[i] = new SingleTimer(parent);
}
}
//static
void TimerForFWUpgrade::set(unsigned char tmrnbr, unsigned int time)
{
if(tmrnbr < NR_OF_TIMERS)
{
time *= TimerForFWUpgrade::timeBase;
singleTimer[tmrnbr]->set(time);
}
}
//static
char TimerForFWUpgrade::isElapsed(unsigned char tmrnbr)
{
if(true == singleTimer[tmrnbr]->isElapsed())
{
return 1;
}
else
{
return 0;
}
}
Module SingleTimer:
SingleTimer::SingleTimer(QObject* parent) : QObject(parent),
pTime(new QTimer(this)),
myElapsed(true)
{
connect(pTime, SIGNAL(timeout()), this, SLOT(slot_setElapsed()));
pTime->setTimerType(Qt::PreciseTimer);
pTime->setSingleShot(true);
}
void SingleTimer::set(unsigned int time)
{
myElapsed = false;
pTime->start(time);
}
bool SingleTimer::isElapsed()
{
QCoreApplication::processEvents();
return myElapsed;
}
void SingleTimer::slot_setElapsed()
{
myElapsed = true;
}
Use QTimer for this purpose and make use of SIGNALS and SLOT for the purpose of starting and stopping the timer/s from different threads. You can emit the signal from any thread and catch it in the thread which created the timer to act on it.
Since you say you are new to Qt, I suggest you go through some tutorials before proceeding so that you will know what Qt has to offer and don't end up trying to reinvent the wheel. :)
VoidRealms is a good starting point.
You have this problem because the timers in the static array is created in Thread X, but started and stopped in Thread Y. This is not allowed, because Qt rely on thread affinity to timeout timers.
You can either create, start stop in the same thread or use signal and slots to trigger start and stop operations for timers. The signal and slot solution is a bit problematic Because you have n QTimer objects (Hint: how do you start the timer at position i?)
What you can do instead is create and initialize the timer at position tmrnbr in
TimerForFWUpgrade::set(unsigned char tmrnbr, unsigned int time)
{
singleTimer[tmrnbr] = new SingleTimer(0);
singleTimer[tmrnbr]->set(time);
}
which is executed by the same thread.
Futhermore, you don't need a SingleTimer class. You are using Qt5, and you already have all you need at your disposal:
SingleTimer::isElapsed is really QTimer::remainingTime() == 0;
SingleTimer::set is really QTimer::setSingleShot(true); QTimer::start(time);
SingleTimer::slot_setElapsed becomes useless
ThusSingleTimer::SingleTimer becomes useless and you dont need a SingleTimer class anymore
I got the errors away after changing my timer concept. I'dont use anymore my SingleTimer module. Before the QTimer I won't let timeout and maybe because of that I run into problems. Now I have a cyclic QTimer that times out every 100ms in slot function I will then count the events. Below my working code:
TimerForFWUpgrade::TimerForFWUpgrade(QObject* parent) : QObject(parent),
pTime(new QTimer(this))
{
connect(pTime, SIGNAL(timeout()), this, SLOT(slot_handleTimer()));
pTime->setTimerType(Qt::PreciseTimer);
pTime->start(100);
}
void TimerForFWUpgrade::set(unsigned char tmrnbr, unsigned int time)
{
if(tmrnbr < NR_OF_TIMERS)
{
if(timeBase != 0)
{
myTimeout[tmrnbr] = time / timeBase;
}
else
{
myTimeout[tmrnbr] = 0;
}
myTimer[tmrnbr] = 0;
myElapsed[tmrnbr] = false;
myActive[tmrnbr] = true;
}
}
char TimerForFWUpgrade::isElapsed(unsigned char tmrnbr)
{
QCoreApplication::processEvents();
if(tmrnbr < NR_OF_TIMERS)
{
if(true == myElapsed[tmrnbr])
{
return 1;
}
else
{
return 0;
}
}
else
{
return 0; // NOK
}
}
void TimerForFWUpgrade::slot_handleTimer()
{
for(UINT8 i = 0; i < NR_OF_TIMERS; i++)
{
if(myActive[i] == true)
{
myTimer[i]++;
if(myTimeout[i] < myTimer[i])
{
myTimer[i] = 0;
myElapsed[i] = true;
myActive[i] = false;
}
}
}
}

2 threads left hanging waiting on QWaitCondition in spite of wakeAll calls

I have threaded iterative generation of some geometries. I use VTK for rendering. After each iteration I would like to display (render) the current progress. My approach works as expected until the last 2 threads are left hanging waiting for QWaitCondition. They are blocked, even though their status in QWaitCondition's queue is wokenUp (inspected through debugger). I suspect that number of 2 threads is somehow connected with my processor's 4 cores.
Simplified code is below. What am I doing wrong and how to fix it?
class Logic
{
QMutex threadLock, renderLock;
//SOLUTION: renderLock should be per thread, not global like this!
QWaitCondition wc;
bool render;
...
}
Logic::Logic()
{
...
renderLock.lock(); //needs to be locked for QWaitCondition
}
void Logic::timerProc()
{
static int count=0;
if (render||count>10) //render wanted or not rendered in a while
{
threadLock.lock();
vtkRenderWindow->Render();
render=false;
count=0;
wc.wakeAll();
threadLock.unlock();
}
else
count++;
}
double Logic::makeMesh(int meshIndex)
{
while (notFinished)
{
...(calculate g)
threadLock.lock(); //lock scene
mesh[meshIndex]->setGeometry(g);
render=true;
threadLock.unlock();
wc.wait(&renderLock); //wait until rendered
}
return g.size;
}
void Logic::makeAllMeshes()
{
vector<QFuture<double>> r;
for (int i=0; i<meshes.size(); i++)
{
QFuture<double> future = QtConcurrent::run<double>(this, &Logic::makeMesh, i);
r.push_back(future);
}
while(any r is not finished)
QApplication::processEvents(); //give timer a chance
}
There is at least one defect in your code. count and render belong to the critical section, which means they need to be protected from concurrent access.
Assume there are more threads waiting on wc.wait(&renderLock);. Someone somewhere execute wc.wakeAll();. ALL the threads are woken up. Assume at least one thread sees notFinished as true (if any of your code make sense, this must be possible) and go back to execute :
threadLock.lock(); //lock scene
mesh[meshIndex]->setGeometry(g);
render=true;
threadLock.unlock();
wc.wait(&renderLock) <----OOPS...
The second time the thread comes back, he doesn't have the lock renderLock. So Kamil Klimek is right: you call wait on a mutex you don't hold.
You should remove the lock in constructor, and lock before the calling the condition. Wherever you lock renderlock, the thread should not hold threadlock.
The catch was that I needed one QMutex per thread, and not just one global QMutex. The corrected code is below. Thanks for help UmNyobe!
class Logic
{
QMutex threadLock;
QWaitCondition wc;
bool render;
...
}
//nothing in constructor related to threading
void Logic::timerProc()
{
//count was a debugging workaround and is not needed
if (render)
{
threadLock.lock();
vtkRenderWindow->Render();
render=false;
wc.wakeAll();
threadLock.unlock();
}
}
double Logic::makeMesh(int meshIndex)
{
QMutex renderLock; //fix
renderLock.lock(); //fix
while (notFinished)
{
...(calculate g)
threadLock.lock(); //lock scene
mesh[meshIndex]->setGeometry(g);
render=true;
threadLock.unlock();
wc.wait(&renderLock); //wait until rendered
}
return g.size;
}
void Logic::makeAllMeshes()
{
vector<QFuture<double>> r;
for (int i=0; i<meshes.size(); i++)
{
QFuture<double> future = QtConcurrent::run<double>(this, &Logic::makeMesh, i);
r.push_back(future);
}
while(any r is not finished)
QApplication::processEvents(); //give timer a chance
}