I am trying to mess around with multi-threading and OpenCV so I wrote the following program.
void performSearch(Mat& original, Mat& grey, Mat& result)
{
for(size_t index = 0; index < container.size(); ++index)
{
// do stuff
imshow("Results", result);
waitKey();
}
}
int main()
{
thread(performSearch, ref(image), ref(image_grey), ref(answer)).join();
thread(performSearch, ref(image2), ref(image2_grey), ref(answer2)).join();
}
This program will throw an NSInternalInconsistencyException exception because waitKey() needs to be run on the main thread. My question is, how do I attach it to the main thread?
I have an iOS background and with grand central dispatch, I'd simply have to do something along the lines of:
DispatchQueue.main.async // this gets me the main queue asynchronously
{
// update UI
}
Is it possible to achieve this in C++? If yes, how?
Heads Up:
I have looked at this question and they're using QT for their GUI. I'm not.
Related
As the title, i'm trying to do the following things:
First, a videoCapture object that read frame from external source (like video, webcam).
Second, for each frame captured, the processing thread(s) processes it and imshow to the screen.
I've tried this:
#define FRAME_FPS 10
void frameCaptureThread(cv::VideoCapture* cap, cv::Mat* ref){
while (cap->read(*ref)) {
cv::waitKey(1000/FRAME_FPS);
}
}
void frameProcessThread(cv::Mat* ref){
while(true){
if(!ref->empty()){
/* Processing ... */
cv::imshow("frameWindow", *ref);
cv::waitKey(1000/FRAME_FPS);
}
}
}
int main() {
cv::VideoCapture videoCapture("videoPath");
cv::Mat sharedMat;
std::thread t1(frameCaptureThread, &videoCapture, &sharedMat);
std::thread t2(frameProcessThread, &sharedMat);
t1.join();
t2.join();
return 0;
}
This looks wrong right, because i think the frameProcessingThread is put in a while(True) loop thats run forerver and beside that, it also has to check if the *ref is empty or not.
Is there a way to make the Processing Thread only do the process if:
the new frame is coming AND it just finished it recent task.
And if i want to create another thread to do other processing task, what should i do ?
I've look around for some async concepts but have no idea what is the optimal way for this situation.
After a few days of battling this issue, I've decided to reach out for some help.
Overview: I am using std::thread to collect information from system peripherals. This information is going to be displayed on a wxFrame using events (values continuously updated, ~10ms). The issue at the stage of wxFrame handling events (I haven't gotten to the display-of-information portion of it yet). I've read through the event documentation and I think my implementation is on the right path. I'm aware of threading samples with wxThread and wxThreadHelper, but my preference is to use std::thread and it seems like it should work.
Issue: The wxThreadEvent is generated in std::thread and sent using wxQueueEvent with the wxFrame as the target. The std::thread is started/managed in wxApp. The second event that is generated and queued by std::thread causes seg fault. Almost like the first event isn't being freed properly. Here some skimmed down set of code that I'm working with.
app.cpp:
wxIMPLEMENT_APP(ControlStationApp);
void communication_th(Comms * comms){
int i = 0;
while(i++ < 10){
comms->test_thread();
this_thread::sleep_for(std::chrono::milliseconds(3000));
}
}
ControlStationApp::~ControlStationApp(){
this->myThread.join();
}
bool ControlStationApp::OnInit(){
if ( !wxApp::OnInit() ) return false;
// Create the main frame window
this->frame = new wxMainFrame;
this->frame->Show(true);
//Start peripheral thread
Comms comms(this->frame);
this->myThread = std::thread(communication_th, &comms);
return true;
}
wxFrame.cpp
wxDEFINE_EVENT(wxEVT_COMMS_UPDATE, wxThreadEvent);
wxMainFrame::wxMainFrame() : wxFrame(NULL, wxID_ANY, "wxWidgets Layout Demo") {
Bind(wxEVT_COMMS_UPDATE,&wxMainFrame::onCommsUpdate_t,this);
...
}
void wxMainFrame::onCommsUpdate_t(wxThreadEvent & e){
std::cout << "threading eevnt!\n";
e.Skip();
}
void wxMainFrame::OnClose(wxCloseEvent& evt){
Destroy();
}
std::thread .cpp
Comms::Comms(wxMainFrame * f){
this->mainFrame = f;
}
Comms::~Comms(){
}
void Comms::test_thread(){
std::cout << "Test_thread()\n";
wxQueueEvent(this->mainFrame, new wxThreadEvent(wxEVT_COMMS_UPDATE);
}
This results in the following output:
Test_thread()
threading eevnt!
Test_thread()
Segmentation fault (core dumped)
Any input is greatly appreciated. Thanks!
I wrote a data acquisition program with Qt. I collect data using the child threads of the dual cache region written by QSemphore.
void QThreadShow::run() {
m_stop=false; // when start thread,m_stop=false
int n=fullBufs.available();
if (n>0)
fullBufs.acquire(n);
while (!m_stop) {
fullBufs.acquire(); // wait fo full buffer
QVector<double> dataPackage(BufferSize);
double seq=bufNo;
if (curBuf==1)
for (int i=0;i<BufferSize;i++){
dataPackage[i]=buffer2[i]; // copy data from full buffer
}
else
for (int i=0;i<BufferSize;i++){
dataPackage[i]=buffer1[i];
}
for (int k=0;k<BufferSize;k++) {
vectorQpointFbufferData[k]=QPointF(x,dataPackage[k]);
}
emptyBufs.release(); // release a buffer
QVariant variantBufferData;
variantBufferData.setValue(vectorQpointFbufferData);
emit newValue(variantBufferData,seq); // send data to main thread
}
quit();
}
When a cache of sub-threads has collected 500 data, the data is input into a QVector and sent to the main thread and is directly assigned to a lineseries in qchartview every 20ms for drawing. I use QtChart to chart the data.
void MainWindow::onthreadB_newValue(QVariant bufferData, double bufNo) {
// Analysis of QVariant data
CH1.hardSoftDataPointPackage = bufferData.value<QVector<QPointF>>();
if (ui->CH1_Source->currentIndex()==0) {
for (int p = 0;p<CH1.hardSoftDataPointPackage.size();p++) {
series_CH3->append(CH1.hardSoftDataPointPackage[p]);
}
}
}
There is a timer in the main thread.The interval is 20ms and there is a double time (time = time +1), which controls the X-axis.
void MainWindow::drawAxis(double time) {
// dynamic draw x axis
if (time<100) {
axisX->setRange(0, TimeBase/(1000/FrameRate) * 10);
// FrameRate=50
} else {
axisX->setRange(time-TimeBase/(1000/FrameRate) * 10, time);
}
}
But when I run my program, there is a problem that every time the subthread sends data to the main thread, the main thread gets stuck for a few seconds and the plot also gets stuck for a few seconds. I added a curve in the main thread getting data from the main thread, and found that both two curves will be stuck at the same time. I don't know how to solve this problem.
Besides, I want the main thread to draw the data from the child thread evenly within 20ms, instead of drawing all the points at once.
Your main thread stucks because you copy (add to series) a lot of data at one time. Instead this you can collect all your data inside your thread instance without emitting a signal. And from main thread just take little pieces of collected data every 20 ms.
Something like this:
while(!m_stop)
{
...
//QVariant variantBufferData;
//variantBufferData.setValue(vectorQpointFbufferData);
//emit newValue(variantBufferData,seq);//send data to main thread
//instead this just store in internal buffer
m_mutex.lock();
m_internalBuffer.append(vectorQpointFbufferData);
m_mutex.unlock();
}
Read method
QVector<QPointF> QThreadShow::takeDataPiece()
{
int n = 4;
QVector<QPointF> piece;
piece.reserve(n);
m_mutex.lock();
for (int i = 0; i < n; i++)
{
QPointF point = m_internalBuffer.takeFirst();
piece.append(point);
}
m_mutex.unlock();
return piece;
}
And in Main thread read in timeout slot
void MainWindow::OnDrawTimer()
{
QVector<QPointF> piece = m_childThread.takeDataPiece();
//add to series
...
//drawAxis
...
}
I'm new to SFML, been trying to have a multi-threading game system (all of the game logic on the main thread, and the rendering in a dedicated thread using sf::Thread; mainly for practicing with threads) as explained in this page ("Drawing with threads" section):
Unfortunately my program has a long processing time during it's update() and makes the rendering process completely out of control, showing some frames painted and some others completely empty. If it isn't obvious my rendering thread is trying to paint something that isn't even calculated, leaving this epileptic effect.
What I'm looking for is to allow the thread to render only when the main logic has been calculated. Here's what I got so far:
void renderThread()
{
while (window->isOpen())
{
//some other gl stuff
//window clear
//window draw
//window display
}
}
void update()
{
while (window->isOpen() && isRunning)
{
while (window->pollEvent(event))
{
if (event.type == sf::Event::Closed || sf::Keyboard::isKeyPressed(sf::Keyboard::Escape))
{
isRunning = false;
}
else if (m_event.type == sf::Event::Resized)
{
glViewport(0, 0, m_event.size.width, m_event.size.height);
}
}
// really resource intensive process here
time = m_clock.getElapsedTime();
clock.restart().asSeconds();
}
}
Thanks in advance.
I guess the errors happen because you manipulate elements that are getting rendered at the same time in parallel. You need to look into mutexes.
Mutexes lock the element you want to manipulate (or draw in the the other thread) for as long as the manipulation takes and frees it afterwards.
While the element is locked it can not be accessed by another thread.
Example in pseudo-code:
updateThread(){
renderMutex.lock();
globalEntity.manipulate();
renderMutex.unlock();
}
renderThread(){
renderMutex.lock();
window.draw(globalEntity);
renderMutex.unlock();
}
I want to render via openGL 2camera in the same time. I want to refresh each frame as soon as possible. To update those frame I am using a thread in an infinite loop.
My actual problem is that my program crash if I put the resolution of those camera to high.
With 160:120 there's no problem. But if i put the max resolution (1920:1080) there's just 5 or 6 update of the image before crashing.
note: the number of update is not always the same before crashing
I suppose that If the resolution is low enough, frame_L and frame_R are changed quick enough that there's no colision beetween the main loop and the thread.
So I suppose that my mutex isnt doing what it should. How should I do?
(I'm not an expert in thread and variable safety)
My code:
#include <boost/thread/mutex.hpp>
#include <boost/thread/thread.hpp>
#include <opencv2/opencv.hpp>
boost::mutex m; //for an other thread
boost::mutex n;
cv::VideoCapture capture_L(0);
cv::VideoCapture capture_R(1);
cv::Mat frame_L;
cv::Mat frame_R;
void MyThreadFunction()
{
while (1)
{
{
boost::mutex::scoped_lock lk(n);
if (capture_L.grab()){
capture_L.retrieve(frame_L);
cv::transpose(frame_L, frame_L);
}
if (capture_R.grab()){
capture_R.retrieve(frame_R);
cv::transpose(frame_R, frame_R);
}
}
}
}
int main()
{
capture_L.set(CV_CAP_PROP_FRAME_WIDTH, 160);
capture_L.set(CV_CAP_PROP_FRAME_HEIGHT, 120);
capture_R.set(CV_CAP_PROP_FRAME_WIDTH, 160);
capture_R.set(CV_CAP_PROP_FRAME_HEIGHT, 120);
boost::thread thrd(&MyThreadFunction);
while(1)
{
[ use frame_L and frame_R ]
}
}
This is the code that I use for my threaded camera grab. Its part of a camera Object (eg; Each camera has it's own object and own thread for the capturing.
void Camera::setCapture(cv::VideoCapture cap)
{
pthread_mutex_lock(&latestFrameMutex);
videoCapture = cap;
videoCapture.read(latestFrame);
pthread_mutex_unlock(&latestFrameMutex);
int iret = pthread_create(&cameraGrabThread,NULL,&Camera::exec,this);
}
void *Camera::exec(void* thr)
{
reinterpret_cast<Camera *> (thr)->grabFrame();
}
This ensures that a new thread is started when a capture is set for that camera. The following code is run by the exec part of the thread to actually grab the frame.
void *Camera::grabFrame()
{
while(videoCapture.isOpened())
{
pthread_mutex_lock(&latestFrameMutex);
if(!videoCapture.read(latestFrame))
std::cout << "Unable to read frame" << std::endl;
pthread_mutex_unlock(&latestFrameMutex);
usleep(8000); // Sleep 8ms
}
}
And finally, the camera needs to be able to return the latest frame;
cv::Mat Camera::getLatestFrame()
{
while (getMilisecSinceLastCapture() < 35) //Enforce min time of 35 ms between frame requests.
{
usleep(5000);
}
pthread_mutex_lock(&latestFrameMutex);
cv::Mat result = latestFrame.clone();
pthread_mutex_unlock(&latestFrameMutex);
return result.clone();
}
Changing the size can be done by using
void Camera::setImageSize(const cv::Size& size)
{
pthread_mutex_lock(&latestFrameMutex);
videoCapture.set(CV_CAP_PROP_FRAME_HEIGHT, size.height);
videoCapture.set(CV_CAP_PROP_FRAME_WIDTH, size.width);
pthread_mutex_unlock(&latestFrameMutex);
}