I try to write a producer-consumer demo with c++11 but a tricky problem happend.Here is the Code
#include <iostream>
#include <thread>
#include <condition_variable>
#include <windows.h>
using namespace std;
std::condition_variable pcv,ccv;
std::mutex m,m1;
const int N=10;
int buf[N];
int count=0;
void producer(){
Sleep(100);
while(true){
std::unique_lock<std::mutex> pulk(m);
while(count==N)
pcv.wait(pulk);
buf[count++]=1;
cout<<"produce data on the buff: "<<count<<endl;
while(count==1) //if I remove this no problem
ccv.notify_one();
pulk.unlock();
}
}
void consumer(){
while(true){
std::unique_lock<std::mutex> culk(m);
while(count==0)
ccv.wait(culk);
buf[--count]=0;
cout<<"consume data on the buff: "<<count<<endl;
while(count==N-1) //if I remove no problem
pcv.notify_one();
culk.unlock();
}
}
int main(int argc,char **argv){
std::thread pro(producer);
std::thread con(consumer);
pro.join();
con.join();
return 0;
the program will run the next line forever
while(count==1) //if the buffer empty?
ccv.notify_one()
I try to use GDB find this reason but no result
Here is the GDB output
The lines while(count==1) //if I remove this no problem and while(count==N-1) //if I remove no problem make your synchronization fragile. You consider only two states of ten (N) possible.
Related
Im trying to make multithreaded proxy checker in c++, when I start the threads and lock it all threads wait till the request is finished. I tried to remove the locks but that doesn't help either. Im using the cpr library to make the requests, the documentation can be found here: https://whoshuu.github.io/cpr/advanced-usage.html.
Reproduceable example:
#include <stdio.h>
#include <pthread.h>
#include <iostream>
#include <queue>
#include <mutex>
#include <cpr/cpr.h>
#include <fmt/format.h>
#define NUMT 10
using namespace std;
using namespace fmt;
std::mutex mut;
std::queue<std::string> q;
void* Checker(void* arg) {
while (!q.empty()) {
mut.lock();
//get a webhook at https://webhook.site
string protocol = "socks4";
string proxyformatted = format("{0}://{1}", protocol, q.front());
auto r = cpr::Get(cpr::Url{ "<webhook url>" },
cpr::Proxies{ {"http", proxyformatted}, {"https", proxyformatted} });
q.pop();
mut.unlock();
}
return NULL;
}
int main(int argc, char** argv) {
q.push("138.201.134.206:5678");
q.push("185.113.7.87:5678");
q.push("5.9.16.126:5678");
q.push("88.146.196.181:4153");
pthread_t tid[NUMT]; int i;
int thread_args[NUMT];
for (i = 0; i < NUMT; i++) {
thread_args[i] = i;
pthread_create(&tid[i], NULL, Checker, (void*) &thread_args);
}
for (i = 0; i < NUMT; i++) {
pthread_join(tid[i], NULL);
fprintf(stderr, "Thread %d terminated\n", i);
}
return 0;
}
Thanks in advance.
I suggest to implement a wrapper class for your queue that will hide the mutex.
That class can provide push(std::string s) and bool pop(std::string& s) that returns true and populate s if the queue wasn't empty or false othervise. Then your worker threads can simply loop
std::string s;
while(q.pop(s)) {
...
}
In my main program I am copying a string buffer into a boost ring buffer and then trying to consume that data in a created thread and writing to a file. In the main thread also I am writing the same data to a file but both input and output file is not matching.
I feel I am doing something incredibly stupid. Please help. Also, if there are any suggesting to improve the code that would really be appreciated.
#include <iostream>
#include <vector>
#include <boost/circular_buffer.hpp>
#include <numeric>
#include <assert.h>
#include <thread>
#include <mutex>
#include <chrono>
#include <time.h>
#include <cstdint>
#include <fstream>
#include <string>
using std::cin;
using std::cout;
using std::endl;
using std::fstream;
using std::string;
#define SOME_FIXED_HARDCODED_NUMBER 40980
class MyClass {
public:
std::vector<int8_t> vec;
public:
MyClass(std::vector<int8_t> v){ vec = v; }
};
boost::circular_buffer<MyClass> cb(300);
int waiting = 1;
std::mutex my_mutex;
FILE *out_file;
FILE *in_file;
void foo()
{
while (waiting) {
std::unique_lock<std::mutex> lock(my_mutex);
if (!cb.size() || waiting == 0) {
lock.unlock();
continue;
}
if (!waiting)
break;
MyClass local_buf = cb.front();
cb.pop_front();
fwrite(local_buf.vec.data(), 1, local_buf.vec.size(), out_file);
}
}
int main(int argc, char* argv[])
{
out_file = fopen("output_data.raw", "w");
in_file = fopen("input_data.raw", "w");
std::thread th1(foo);
char *buf = {"abc"};
int counter = 0;
std::vector<int8_t> mem;
mem.insert(mem.end(), buf, buf + strlen(buf));
while (counter < SOME_FIXED_HARDCODED_NUMBER)
{
{
std::unique_lock<std::mutex> lock(my_mutex);
/* if the circular buffer is full then wait for consumer to pull the data */
while (cb.full()) {
lock.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
std::unique_lock<std::mutex> lock(my_mutex);
}
cb.push_front(MyClass(mem));
fwrite(mem.data(), 1, mem.size(), in_file);
}
counter++;
}
waiting = 0;
th1.join();
fclose(out_file);
fclose(in_file);
return 0;
}
while (cb.full()) {
lock.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
>>> std::unique_lock<std::mutex> lock(my_mutex);
}
The marked unique_lock doesn't do anything as it will go out of scope immediately and unlock the mutex. Hence once you leave the loop the mutex is not locked and you have a racecondition. Instead, you should use lock.lock() to relock the mutex.
There is a few more bugs. You are not waiting for your foo thread to actually drain the buffer. It will stop as soon as the waiting flag is set by the main thread. Also, waiting should be an atomic.
My code acquires images and processes them. Performance is critical for my code, so I've tried my hand at multi-threading. Currently, I've only made the acquiring part a separate thread. I'm implementing a simple FIFO buffer using std::queue that stores the acquired images. The acquisition function AcquireImages writes raw image data to this buffer indefinitely until user interruption. Processing function, ProcessImages reads the buffer and processes the image data (currently in the main thread but I'm planning to make this a separate thread as well once I've ironed out issues). Here's my code (modified to form an MCV example):
#include <iostream>
#include <vector>
#include <queue>
#include <atomic>
#include <thread>
#define NUM_CAMERAS 2
void AcquireImages(std::queue<unsigned char*> &rawImageQueue, std::atomic<bool> &quit)
{
unsigned char* rawImage{};
while (!quit)
{
for (int camera = 0; camera < NUM_CAMERAS; camera++)
{
switch (camera)
{
case 0:
rawImage = (unsigned char*)"Cam0Image";
break;
case 1:
rawImage = (unsigned char*)"Cam1Image";
break;
default:
break;
}
rawImageQueue.push(std::move(rawImage));
}
}
}
int ProcessImages(const std::vector<unsigned char*> &rawImageVec, const int count)
{
// Do something to the raw image vector
if (count > 10)
{
return 1;
}
else
{
return 0;
} // In my application, this function only returns non-zero upon user interception.
}
int main()
{
// Preparation
std::vector<unsigned char*> rawImageVec;
rawImageVec.reserve(NUM_CAMERAS);
std::queue<unsigned char*> rawImageQueue;
int count{};
const unsigned int nThreads = 1; // this might grow later
std::atomic<bool> loopFlags[nThreads];
std::thread threads[nThreads];
// Start threads
for (int i = 0; i < nThreads; i++) {
loopFlags[i] = false;
threads[i] = std::thread(AcquireImages, rawImageQueue, ref(loopFlags[i]));
}
// Process images
while (true)
{
// Process the images
for (int cam{}; cam < NUM_CAMERAS; ++cam)
{
rawImageVec.push_back(rawImageQueue.front());
rawImageQueue.pop();
}
int processResult = ProcessImages(move(rawImageVec), count);
if (processResult)
{
std::cout << "Leaving while loop.\n"; // In my application this is triggered by the user
break;
}
rawImageVec.clear();
++count;
}
// Shutdown other threads
for (auto & flag : loopFlags) {
flag = true;
}
// Wait for threads to actually finish.
for (auto& thread : threads) {
thread.join();
}
return 0;
}
Some of you may have already noticed my blunder. What I know is that this program throws an exception atrawImageVec.push_back(rawImageQueue.front());.
The output after throwing the exception reads as follows:
Debug Assertion Failed!
Program: C:\WINDOWS\SYSTEM32\MSVCP140D.dll
File: c:\program files (x86)\microsoft visual studio 14.0\vc\include\deque
Line: 329
Expression: deque iterator not dereferencable
I understand the cause of the issue is probably that I'm reading something that is shared with another thread (Am I correct?). How do I resolve this?
I followed Praetorian's advice in the comments, after checking to see if rawImageQueue is empty, I see that it's always empty. I'm not sure what's causing this.
Here is a generalized example of producer/consumer on a shared queue. The idea is that if you're writing and reading from a data structure, you need some kind of protection around accesses.
For this, the below example uses condition variables and a mutex.
#include <thread>
#include <iostream>
#include <chrono>
#include <queue>
#include <mutex>
#include <vector>
#include <condition_variable>
using namespace std::chrono_literals;
using std::vector;
using std::thread;
using std::unique_lock;
using std::mutex;
using std::condition_variable;
using std::queue;
class WorkQueue
{
condition_variable work_available;
mutex work_mutex;
queue<int> work;
public:
void push_work(int item)
{
unique_lock<mutex> lock(work_mutex);
bool was_empty = work.empty();
work.push(item);
lock.unlock();
if (was_empty)
{
work_available.notify_one();
}
}
int wait_and_pop()
{
unique_lock<mutex> lock(work_mutex);
while (work.empty())
{
work_available.wait(lock);
}
int tmp = work.front();
work.pop();
return tmp;
}
};
int main() {
WorkQueue work_queue;
auto producer = [&]() {
while (true) {
work_queue.push_work(10);
std::this_thread::sleep_for(2ms);
}
};
vector<thread> producers;
producers.push_back(std::thread(producer));
producers.push_back(std::thread(producer));
producers.push_back(std::thread(producer));
producers.push_back(std::thread(producer));
std::thread consumer([&]() {
while (true)
{
int work_to_do = work_queue.wait_and_pop();
std::cout << "Got some work: " << work_to_do << std::endl;
}
});
std::for_each(producers.begin(), producers.end(), [](thread &p) {
p.join();
});
consumer.join();
}
Your case is relatively simple as seems you have just one producer and one consumer. Also image processing sounds quite slow (slow enough to not worry about threads contention) and you're switching from single-threaded version so probably no need to bother with highly efficient lock-free implementations.
I'd recommend to study this pseudo code: https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem#Using_monitors, then to learn about condition variables if you need: http://en.cppreference.com/w/cpp/thread/condition_variable.
Here is my source code:
#include "stdio.h"
#include <stdlib.h>
#include <string.h>
#include "thread"
#include "mutex"
int count0=0 ,count1 =0;
std::mutex main_thread;
std::unique_lock<std::mutex> lck(main_thread, std::defer_lock);
void Function00(long millisecond){
while (true){
lck.lock();
count1++;
printf("count0:%d count1:%d \n",count0,count1);
lck.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(millisecond));
}
}
void Function01(){
std::thread th(Function00, 1000);//count per 1 s
do{
lck.lock();
count0++;
lck.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(500));//count per 0.5 s
} while (1);
}
int main(int argc, char *argv[])
{
Function01();
return 0;
}
I then use the command to build my .o file:
g++ -std=c++11 -pthread testa.cpp -o a.o
However, it shows the error:
terminate called after throwing an instance of 'std::system_error'
what(): Resource deadlock avoided
Aborted
I feel confuse and no idea to solve it, so i try in Microsoft VS2013, and it runs with no error... I feel confuse to it. Is it the problem in linux? What should i modify to prevent the deadlock?
unique_lock cannot be locked twice, if you want to lock a mutex on two threads so that one of them will block, you need to use two unique_locks.
void Function00(long millisecond){
while (true){
{
std::unique_lock<std::mutex> lck(main_thread);
count1++;
printf("count0:%d count1:%d \n",count0,count1);
}
std::this_thread::sleep_for(std::chrono::milliseconds(millisecond));
}
}
void Function01(){
std::thread th(Function00, 1000);//count per 1 s
do{
{
std::unique_lock<std::mutex> lck(main_thread);
count0++;
}
std::this_thread::sleep_for(std::chrono::milliseconds(500));//count per 0.5 s
} while (1);
}
I've been doing pretty basic stuff with std::thread without any particular reason, simply in order to learn it. I thought that the simple example I created, where few threads are operating on the same data, locking each other before doing so, worked just fine, until I realized that every time I run it the returned value is different, while very close to each other, I am pretty sure they should equal each other. Some of the values I have received:
21.692524
21.699258
21.678871
21.705947
21.685744
Am I doing something wrong or maybe there is underlying reason for that behaviour?
#include <string>
#include <iostream>
#include <thread>
#include <math.h>
#include <time.h>
#include <windows.h>
#include <mutex>
using namespace std;
mutex mtx;
mutex mtx2;
int currentValue = 1;
double suma = 0;
int assignPart() {
mtx.lock();
int localValue = currentValue;
currentValue+=10000000;
mtx.unlock();
return localValue;
}
void calculatePart()
{
int value;
double sumaLokalna = 0;
while(currentValue<1500000000){
value = assignPart();
for(double i=value;i<(value+10000000);i++){
sumaLokalna = sumaLokalna + (1/(i));
}
mtx2.lock();
suma+=sumaLokalna;
mtx2.unlock();
sumaLokalna = 0;
}
}
int main()
{
clock_t startTime = clock();
// Constructs the new thread and runs it. Does not block execution.
thread watek(calculatePart);
thread watek2(calculatePart);
thread watek3(calculatePart);
thread watek4(calculatePart);
while(currentValue<1500000000){
Sleep(100);
printf("%-12d %-12lf \n",currentValue, suma);
}
watek.join();
watek2.join();
watek3.join();
watek4.join();
cout << double( clock() - startTime ) / (double)CLOCKS_PER_SEC<< " seconds." << endl;
//Makes the main thread wait for the new thread to finish execution, therefore blocks its own execution.
}
Your loop
while(currentValue<1500000000){
Sleep(100);
printf("%-12d %-12lf \n",currentValue, suma);
}
is printing intermediate results, but you're not printing the final result.
To print the final result, add the line
printf("%-12d %-12lf \n",currentValue, suma);
after joining the threads.