condition_variable is not working if use it inside structure - c++

condition_variable is not working if use it inside a structure. If I have it as a global variable all works fine. But I need a condition_variable for each packet as I don't know when I will receive an answer and I need to wait for it for each packet. What am I doing wrong?
This is console output:
Wait: 416
StopWait: 423
From it I can see that I receive data and unlock thread after I lock it.
Structures
struct Waiting {
bool IsWaiting = false;
mutable std::condition_variable cv;
mutable std::mutex m;
clock_t localCLock = 0;
void Wait() const {
const double ms = Utils::MillisecondsSpent(localCLock);
std::cout << "Wait: " << ms << std::endl;
std::unique_lock<std::mutex> lock(m);
cv.wait(lock, [this] { return IsWaiting; });
}
void StopWait() {
const double ms = Utils::MillisecondsSpent(localCLock);
std::cout << "StopWait: " << ms << std::endl;
std::unique_lock<std::mutex> lock(m);
IsWaiting = true;
cv.notify_all();
}
};
struct Packet
{
Packet() : id(0), waiting(new Waiting) {}
int id;
Waiting* waiting;
};
class Map
{
static Map* instance;
Map();
~Map();
Map(const Map&) = delete;
public:
static Map* Instance() {
if (!instance) instance = new Map;
return instance;
}
std::map<int, Packet> packets;
};
Threads
//Send Thread - called first
while(true){
Packet packet;
packet.id = 1;
//some send packet logic here
...
///
Map::Instance()->packets.insert(std::pair<int, Packet>(p.id, p));
Map::Instance()->packets[id].waiting->Wait(); // thread now locked and never unlocks
const Packet received = Map::Instance()->packets[id];
Map::Instance()->packets.erase(id);
}
//Receive Thread - called second
while(true){
//some receive packet logic here
...
///
const Packet packet = ... // receive a packet data;
Map::Instance()->packets[packet.id] = packet;
Map::Instance()->packets[packet.id].answered = true;
Map::Instance()->packets[packet.id].waiting->StopWait(); // i unlock Send Thread, but it won't work
}

Synchronization issues and memory leaks aside, every time you assign a Packet you are copying it by value, and a new Waiting is allocated. There are many different dangling Waiting objects floating around in memory, and there's no reason that calling StopWait on one will trigger the condition_variable on another.
See the code comments I've added.
while(true){
// *** PACKET A ***
Packet packet;
packet.id = 1;
//*** PACKET B ***
Map::Instance()->packets.insert(std::pair<int, Packet>(p.id, p));
Map::Instance()->packets[id].waiting->Wait();
}
while(true){
// *** PACKET C ***
const Packet packet = ...
//You are overwriting PACKET B with a copy of PACKET C which is PACKET D.
//Don't you mean to find a packet which has the same id as the received packet instead of overwriting it?
Map::Instance()->packets[packet.id] = packet;
Map::Instance()->packets[packet.id].answered = true;
// There's no reason calling StopWait on PACKET D's Waiting object will release PACKET B.
Map::Instance()->packets[packet.id].waiting->StopWait();
}

Related

How to properly wait for condition variable in C++?

In trying to create an asynchronous I/O file reader in C++ under Linux. The example I have has two buffers. The first read blocks. Then, for each time around the main loop, I asynchronously launch the IO and call process() which runs the simulated processing of the current block. When processing is done, we wait for the condition variable. The idea is that the asynchronous handler should notify the condition variable.
Unfortunately the notify seems to happen before wait, and it seems like this is not the way the condition variable wait() function works. How should I rewrite the code so that the loop waits until the asynchronous io has completed?
#include <aio.h>
#include <fcntl.h>
#include <signal.h>
#include <unistd.h>
#include <condition_variable>
#include <cstring>
#include <iostream>
#include <thread>
using namespace std;
using namespace std::chrono_literals;
constexpr uint32_t blockSize = 512;
mutex readMutex;
condition_variable cv;
int fh;
int bytesRead;
void process(char* buf, uint32_t bytesRead) {
cout << "processing..." << endl;
usleep(100000);
}
void aio_completion_handler(sigval_t sigval) {
struct aiocb* req = (struct aiocb*)sigval.sival_ptr;
// check whether asynch operation is complete
if (aio_error(req) == 0) {
int ret = aio_return(req);
bytesRead = req->aio_nbytes;
cout << "ret == " << ret << endl;
cout << (char*)req->aio_buf << endl;
}
{
unique_lock<mutex> readLock(readMutex);
cv.notify_one();
}
}
void thready() {
char* buf1 = new char[blockSize];
char* buf2 = new char[blockSize];
aiocb cb;
char* processbuf = buf1;
char* readbuf = buf2;
fh = open("smallfile.dat", O_RDONLY);
if (fh < 0) {
throw std::runtime_error("cannot open file!");
}
memset(&cb, 0, sizeof(aiocb));
cb.aio_fildes = fh;
cb.aio_nbytes = blockSize;
cb.aio_offset = 0;
// Fill in callback information
/*
Using SIGEV_THREAD to request a thread callback function as a notification
method
*/
cb.aio_sigevent.sigev_notify_attributes = nullptr;
cb.aio_sigevent.sigev_notify = SIGEV_THREAD;
cb.aio_sigevent.sigev_notify_function = aio_completion_handler;
/*
The context to be transmitted is loaded into the handler (in this case, a
reference to the aiocb request itself). In this handler, we simply refer to
the arrived sigval pointer and use the AIO function to verify that the request
has been completed.
*/
cb.aio_sigevent.sigev_value.sival_ptr = &cb;
int currentBytesRead = read(fh, buf1, blockSize); // read the 1st block
while (true) {
cb.aio_buf = readbuf;
aio_read(&cb); // each next block is read asynchronously
process(processbuf, currentBytesRead); // process while waiting
{
unique_lock<mutex> readLock(readMutex);
cv.wait(readLock);
}
currentBytesRead = bytesRead; // make local copy of global modified by the asynch code
if (currentBytesRead < blockSize) {
break; // last time, get out
}
cout << "back from wait" << endl;
swap(processbuf, readbuf); // switch to other buffer for next time
currentBytesRead = bytesRead; // create local copy
}
delete[] buf1;
delete[] buf2;
}
int main() {
try {
thready();
} catch (std::exception& e) {
cerr << e.what() << '\n';
}
return 0;
}
A condition varible should generally be used for
waiting until it is possible that the predicate (for example a shared variable) has changed, and
notifying waiting threads that the predicate may have changed, so that waiting threads should check the predicate again.
However, you seem to be attempting to use the state of the condition variable itself as the predicate. This is not how condition variables are supposed to be used and may lead to race conditions such as those described in your question. Another reason to always check the predicate is that spurious wakeups are possible with condition variables.
In your case, it would probably be appropriate to create a shared variable
bool operation_completed = false;
and use that variable as the predicate for the condition variable. Access to that variable should always be controlled by the mutex.
You can then change the lines
{
unique_lock<mutex> readLock(readMutex);
cv.notify_one();
}
to
{
unique_lock<mutex> readLock(readMutex);
operation_completed = true;
cv.notify_one();
}
and change the lines
{
unique_lock<mutex> readLock(readMutex);
cv.wait(readLock);
}
to:
{
unique_lock<mutex> readLock(readMutex);
while ( !operation_completed )
cv.wait(readLock);
}
Instead of
while ( !operation_completed )
cv.wait(readLock);
you can also write
cv.wait( readLock, []{ return operation_completed; } );
which is equivalent. See the documentation of std::condition_varible::wait for further information.
Of course, operation_completed should also be set back to false when appropriate, while the mutex is locked.

Why a waiting thread does not wake up after calling notify?

I am trying to simulate a sensor that outputs data at a certain frame rate while another is waiting to have a data ready and when it is ready it copies it locally and processes it.
Sensor sensor(1,1000);
Monitor monitor;
// Function that continuously reads data from sensor
void runSensor()
{
// Initial delay
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
for(int i = 0; i < SIZE_LOOP; i++)
{
monitor.captureData<Sensor>(sensor, &Sensor::captureData);
}
}
// Function that waits until sensor data is ready
void waitSensor()
{
monitor.saveData<Sensor>(sensor, &Sensor::saveData);
}
// Main function
int main()
{
// Threads that reads at some frame rate data from sensor
std::thread threadRunSensor(runSensor);
// Processing loop
for(int i = 0; i < SIZE_LOOP; i++)
{
// Wait until data from sensor is ready
std::thread threadWaitSensor(waitSensor);
// Wait until data is copied
threadWaitSensor.join();
// Process synchronized data while sensor are throwing new data
std::cout << "Init processing (" << sensor.getData() << /*"," << sensor2.getData() << */")"<< std::endl;
// Sleep to simulate processing load
std::this_thread::sleep_for(std::chrono::milliseconds(10000 + (rand() % 1000)));
//std::this_thread::sleep_for(std::chrono::milliseconds(500));
std::cout << "End processing" << std::endl;
}
return 0;
}
This is the sensor class. It has two methods. One that generates the data and other that copies the data locally.
class Sensor
{
private:
int counter;
int id;
int frameRate;
int dataCaptured;
int dataSaved;
public:
Sensor(int f_id, int f_frameRate)
{
id = f_id;
counter = 0;
frameRate = f_frameRate;
};
~Sensor(){};
void captureData()
{
dataCaptured = counter;
counter ++;
std::cout << "Sensor" << id << " (" << dataCaptured << ")"<< std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(frameRate + (rand() % 500)));
};
void saveData()
{
dataSaved = dataCaptured;
std::cout << "Copying sensor" << id << " (" << dataSaved << ")"<< std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(1 + (rand() % 5)));
}
int getData()
{
return dataSaved;
}
};
Then there is a class Monitor that ensures these operations are protected to concurrent accesses.
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <chrono>
#include <cstdlib>
#define SIZE_LOOP 1000
class Monitor
{
private:
std::mutex m_mutex;
std::condition_variable m_condVar;
bool m_isReady;
public:
Monitor()
{
init();
};
~Monitor()
{
};
void init()
{
m_isReady = false;
};
template<class T>
void captureData(T& objectCaptured, void (T::* f_captureFunction_p)())
{
// Lock read
std::unique_lock<std::mutex> lock = std::unique_lock<std::mutex>(m_mutex);
(objectCaptured.*f_captureFunction_p)();
m_isReady = true;
m_condVar.notify_one();
lock.unlock();
};
template<class T>
void saveData(T& objectSaved, void(T::*f_saveFunction_p)())
{
std::unique_lock<std::mutex> lock = std::unique_lock<std::mutex>(m_mutex);
while(!m_isReady)
{
m_condVar.wait(lock);
}
(objectSaved.*f_saveFunction_p)();
m_isReady = false;
lock.unlock();
};
};
Can anyone tell me why the waiting thread does not wakeup if the sensor is notifyng every frame rate?
The idea is having two threads with this workflow:
ThreadCapture captures a data consinuously notifying to ThreadProcessing when the data capture is done.
ThreadCapture must waits to capture a new data only if the current captured data is being copied on ThreadProcessing.
ThreadProcessing waits to a new captured data, makes a local copy, notifies to ThreadCapture that the copy is done and process the data.
The local copy is made on ThreadProcessing to allow ThreadCapture can capture new data while ThreadProcessing is processing.
Finally I found the solution adding a waiting step after the capture to give time to save data
template<class T>
void captureData(T& objectCaptured, void (T::* f_captureFunction_p)())
{
std::unique_lock<std::mutex> lockReady = std::unique_lock<std::mutex>(m_mutexReady, std::defer_lock);
std::unique_lock<std::mutex> lockProcess = std::unique_lock<std::mutex>(m_mutexProcess, std::defer_lock);
// Lock, capture, set data ready flag, unlock and notify
lockReady.lock();
(objectCaptured.*f_captureFunction_p)();
m_isReady = true;
lockReady.unlock();
m_conditionVariable.notify_one();
// Wait while data is ready and it is not being processed
lockReady.lock();
lockProcess.lock();
while(m_isReady && !m_isProcessing)
{
lockProcess.unlock();
m_conditionVariable.wait(lockReady);
lockProcess.lock();
}
lockProcess.unlock();
lockReady.unlock();
};
template<class T>
void saveData(T& objectSaved, void(T::*f_saveFunction_p)())
{
std::unique_lock<std::mutex> lockReady(m_mutexReady, std::defer_lock);
std::unique_lock<std::mutex> lockProcess(m_mutexProcess, std::defer_lock);
// Reset processing
lockProcess.lock();
m_isProcessing = false;
lockProcess.unlock();
// Wait until data is ready
lockReady.lock();
while(!m_isReady)
{
m_conditionVariable.wait(lockReady);
}
// Make a copy of the data, reset ready flag, unlock and notify
(objectSaved.*f_saveFunction_p)();
m_isReady = false;
lockReady.unlock();
m_conditionVariable.notify_one();
// Set processing
lockProcess.lock();
m_isProcessing = true;
lockProcess.unlock();
};
};

Synchronize two sensors with different frame rate

I want to synchronize the output of two sensors that works at different frame rate (~80ms vs ~40ms) in C++ using threads. The idea is like the producer-consumer problem but with 2 producers and 1 consumer, and without a buffer because only the last new products matters.
These are the points that shoud cover the problem:
Each sensor reading will be managed by a thread separately.
There will be a main thread that must take always the last new two data read from the sensors and process it.
The reading of one sensor should not block the reading of the other. I mean, the threads reading should not have the same mutex.
The main/process thread should not block the reading threads while it is working. I propose lock the data, make a local copy (it is faster than process directly), unlock and process the copy.
If there is no new data, the main thread should wait for it.
This is a time diagram of the requested functionality.
And this is the pseudocode:
void getSensor1(Data& data)
{
while (true)
{
mutex1.lock();
//Read data from sensor 1
mutex1.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(80 + (rand() % 5)));
}
}
void getSensor2(Data& data)
{
while (true)
{
mutex2.lock();
//Read data from sensor 2
mutex2.unlock();
std::this_thread::sleep_for(std::chrono::milliseconds(40 + (rand() % 5)));
}
}
int main()
{
Data sensor1;
Data sensor2;
std::thread threadGetScan(getSensor1, std::ref(sensor1));
std::thread threadGetFrame(getSensor2, std::ref(sensor2));
while(true)
{
// Wait for new data, lock, copy, unlock and process it
std::this_thread::sleep_for(std::chrono::milliseconds(100 + (rand() % 25)))
}
return 0;
}
Thanks in advance.
Since each sensor is only read from one thread, then mutex around the sensor access serves no purpose. You can get rid of that. Where you need thread safety is the means by which the thread which has read from a sensor passes data to the thread which is consuming it.
Have the thread reading from the sensor use only local variables, or variables only accessed by that thread, for its work of reading the sensor. Once it has the data completely, then put that data (or better yet, a pointer to the data) into a shared queue that the consuming thread will get it from.
Since you need to save only the latest data, your queue can have a max size of 1. Which can just be a pointer.
Access to this shared data structure should be protected with a mutex. But since it is just a single pointer, you can use std::atomic.
The reading thread could look like this:
void getData(std::atomic<Data*>& dataptr) {
while (true) {
Data* mydata = new Data; // local variable!
// stuff to put data into mydata
std::this_thread::sleep_for(80ms);
// Important! this line is only once that uses dataptr. It is atomic.
Data* olddata = std::atomic_exchange(&dataptr, mydata);
// In case the old data was never consumed, don't leak it.
if (olddata) delete olddata;
}
}
And the main thread could look like this:
void main_thread(void) {
std::atomic<Data*> sensorData1;
std::atomic<Data*> sensorData2;
std::thread sensorThread1(getData, std::ref(sensorData1));
std::thread sensorThread2(getData, std::ref(sensorData2));
while (true) {
std::this_thread::sleep_for(100ms);
Data* data1 = std::atomic_exchange(&sensorData1, (Data*)nullptr);
Data* data2 = std::atomic_exchange(&sensorData2, (Data*)nullptr);
// Use data1 and data2
delete data1;
delete data2;
}
}
After some researching work, I have found a solution that does what I wanted using mutexes and condition variables. I let you below the code I propose. Improvements and other suitable solutions are still accepted.
#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <chrono>
#include <cstdlib>
#define SIZE_LOOP 1000
// Struct where the data sensors is synchronized
struct Data
{
int data1; // Data of sensor 1
int data2; // Data of sensor 2
};
std::mutex mtx1; // Mutex to access sensor1 shared data
std::mutex mtx2; // Mutex to access sensor2 shared data
std::condition_variable cv1; // Condition variable to wait for sensor1 data availability
std::condition_variable cv2; // Condition variable to wait for sensor2 data availability
bool ready1; // Flag to indicate sensor1 data is available
bool ready2; // Flag to indicate sensor2 is available
// Function that continuously reads data from sensor 1
void getSensor1(int& data1)
{
// Initialize flag to data not ready
ready1 = false;
// Initial delay
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
// Reading loop (i represents an incoming new data)
for(int i = 0; i < SIZE_LOOP; i++)
{
// Lock data access
std::unique_lock<std::mutex> lck1(mtx1);
// Read data
data1 = i;
std::cout << "Sensor1 (" << data1 << ")"<< std::endl;
// Set data to ready
ready1 = true;
// Notify if processing thread is waiting
cv1.notify_one();
// Unlock data access
lck1.unlock();
// Sleep to simulate frame rate
std::this_thread::sleep_for(std::chrono::milliseconds(2000 + (rand() % 500)));
}
}
// Function that continuously reads data from sensor 2
void getSensor2(int& data2)
{
// Initialize flag to data not ready
ready2 = false;
// Initial delay
std::this_thread::sleep_for(std::chrono::milliseconds(3000));
// Reading loop (i represents an incoming new data)
for(int i = 0; i < SIZE_LOOP; i++)
{
// Lock data access
std::unique_lock<std::mutex> lck2(mtx2);
// Read data
data2 = i;
std::cout << "Sensor2 (" << data2 << ")"<< std::endl;
// Set data to ready
ready2 = true;
// Notify if processing thread is waiting
cv2.notify_one();
// Unlock data access
lck2.unlock();
// Sleep to simulate frame rate
std::this_thread::sleep_for(std::chrono::milliseconds(1000 + (rand() % 500)));
}
}
// Function that waits until sensor 1 data is ready
void waitSensor1(const int& dataRead1, int& dataProc1)
{
// Lock data access
std::unique_lock<std::mutex> lck1(mtx1);
// Wait for new data
while(!ready1)
{
//std::cout << "Waiting sensor1" << std::endl;
cv1.wait(lck1);
}
//std::cout << "No Waiting sensor1" << std::endl;
// Make a local copy of the data (allows uncoupling read and processing tasks what means them can be done parallely)
dataProc1 = dataRead1;
std::cout << "Copying sensor1 (" << dataProc1 << ")"<< std::endl;
// Sleep to simulate copying load
std::this_thread::sleep_for(std::chrono::milliseconds(200));
// Set data flag to not ready
ready1 = false;
// Unlock data access
lck1.unlock();
}
// Function that waits until sensor 2 data is ready
void waitSensor2(const int& dataRead2, int& dataProc2)
{
// Lock data access
std::unique_lock<std::mutex> lck2(mtx2);
// Wait for new data
while(!ready2)
{
//std::cout << "Waiting sensor2" << std::endl;
cv2.wait(lck2);
}
//std::cout << "No Waiting sensor2" << std::endl;
// Make a local copy of the data (allows uncoupling read and processing tasks what means them can be done parallely)
dataProc2 = dataRead2;
std::cout << "Copying sensor2 (" << dataProc2 << ")"<< std::endl;
// Sleep to simulate copying load
std::this_thread::sleep_for(std::chrono::milliseconds(400));
// Set data flag to not ready
ready2 = false;
// Unlock data access
lck2.unlock();
}
// Main function
int main()
{
Data dataRead; // Data read
Data dataProc; // Data to process
// Threads that reads at some frame rate data from sensor 1 and 2
std::thread threadGetSensor1(getSensor1, std::ref(dataRead.data1));
std::thread threadGetSensor2(getSensor2, std::ref(dataRead.data2));
// Processing loop
for(int i = 0; i < SIZE_LOOP; i++)
{
// Wait until data from sensor 1 and 2 is ready
std::thread threadWaitSensor1(waitSensor1, std::ref(dataRead.data1), std::ref(dataProc.data1));
std::thread threadWaitSensor2(waitSensor2, std::ref(dataRead.data2), std::ref(dataProc.data2));
// Shyncronize data/threads
threadWaitSensor1.join();
threadWaitSensor2.join();
// Process synchronized data while sensors are throwing new data
std::cout << "Init processing (" << dataProc.data1 << "," << dataProc.data2 << ")"<< std::endl;
// Sleep to simulate processing load
std::this_thread::sleep_for(std::chrono::milliseconds(10000 + (rand() % 1000)));
std::cout << "End processing" << std::endl;
}
return 0;
}

Filling and saving shared buffer between threads

I'm working with an API that retrieves I/Q data. Calling the function bbGetIQ(m_handle, &pkt);fills a buffer. This is a thread looping while the user hasn't input "stop". Pkt is a structure and the buffer used is pkt.iqData = &m_buffer[0]; which is a vector of float. The size of the vector is 5000 and each time we're looping the buffer is filled with 5000 values.
I want to save the data from the buffer into a file, and I was doing it right after a call to bbgetIQ but doing like so is a time consuming task, data wasn't retrieved fast enough resulting in the API dropping data so it can continue filling its buffer.
Here's what my code looked like :
void Acquisition::recordIQ(){
int cpt = 0;
ofstream myfile;
while(1){
while (keep_running)
{
cpt++;
if(cpt < 2)
myfile.open ("/media/ssd/IQ_Data.txt");
bbGetIQ(m_handle, &pkt); //Retrieve I/Q data
//Writing content of buffer into the file.
for(int i=0; i<m_buffer.size(); i++)
myfile << m_buffer[i] << endl;
}
cpt = 0;
myfile.close();
}
}
Then i tried to only write into the file when we leave the loop :
void Acquisition::recordIQ(){
int cpt = 0;
ofstream myfile;
int next=0;
vector<float> data;
while(1){
while ( keep_running)
{
if(keep_running == false){
myfile.open ("/media/ssd/IQ_Data.txt");
for(int i=0; i<data.size(); i++)
myfile << data[i] << endl;
myfile.close();
break;
}
cpt++;
data.resize(next + m_buffer.size());
bbGetIQ(m_handle, &pkt); //retrieve data
std::copy(m_buffer.begin(), m_buffer.end(), data.begin() + next); //copy content of the buffer into final vector
next += m_buffer.size(); //next index
}
cpt = 0;
}
}
I am no longer getting data loss from the API, but the issue is that i'm limited by the size of data vector. For example, I can't let it retrieve data all night.
My idea is to make 2 threads. One will retrieve data and the other will write the data into a file. The 2 threads will share a circular buffer where the first thread will fill the buffer and the second thread will read the buffer and write the content to a file. As it is a shared buffer, i guess i should use mutexes.
I'm new to multi-threading and mutex, so would this be a good idea? I don't really know where to start and how the consumer thread can read the buffer while the producer will fill it. Will locking the buffer while reading cause data drop by the API ? (because it won't be able to write it into the circular buffer).
EDIT : As i want my record thread to run in background so i can do other stuff while it's recording, i detached it and the user can launch a record by setting the condition keep_running to true.
thread t1(&Acquisition::recordIQ, &acq);
t1.detach();
You need to use something like this (https://en.cppreference.com/w/cpp/thread/condition_variable):
globals:
std::mutex m;
std::condition_variable cv;
std::vector<std::vector<float>> datas;
bool keep_running = true, start_running = false;
writing thread:
void writing_thread()
{
myfile.open ("/media/ssd/IQ_Data.txt");
while(1) {
// Wait until main() sends data
std::unique_lock<std::mutex> lk(m);
cv.wait(lk, []{return keep_running && !datas.empty();});
if (!keep_running) break;
auto d = std::move(datas);
lk.unlock();
for(auto &entry : d) {
for(auto &e : entry)
myfile << e << endl;
}
}
}
sending thread:
void sending_thread() {
while(1) {
{
std::unique_lock<std::mutex> lk(m);
cv.wait(lk, []{return keep_running && start_running;});
if (!keep_running) break;
}
bbGetIQ(m_handle, &pkt); //retrieve data
std::vector<float> d = m_buffer;
{
std::lock_guard<std::mutex> lk(m);
if (!keep_running) break;
datas.push_back(std::move(d));
}
cv.notify_one();
}
}
void start() {
{
std::unique_lock<std::mutex> lk(m);
start_running = true;
}
cv.notify_all();
}
void stop() {
{
std::unique_lock<std::mutex> lk(m);
start_running = false;
}
cv.notify_all();
}
void terminate() {
{
std::unique_lock<std::mutex> lk(m);
keep_running = false;
}
cv.notify_all();
thread1.join();
thread2.join();
}
In short:
Sending thread receives data from whatever it comes, locks mutex mt and moves data to datas storage. Then it uses cv condition variable to notify waiting threads, that there's something to do. Writing thread waits for condition variable to be signaled, then locks mutex mt, moves data from datas global variable to local, then releases mutex and proceed to write just received data to file. Key is to keep mutexed locked for least time possible.
EDIT:
to terminate whole thing you need to set keep_running to false. Then call cv.notify_all(). Then join threads involved. Order is important. You need to join threads, because writing thread might be still in process of writing data.
EDIT2:
added delayed start. Now create two threads, in one run sending_thread, in other writing_thread. Call start() to enable processing and stop() to stop it.

Boost asio, single TCP server, many clients

I am creating a TCP server that will use boost asio which will accept connections from many clients, receive data, and send confirmations. The thing is that I want to be able to accept all the clients but I want to work only with one at a time. I want all the other transactions to be kept in a queue.
Example:
Client1 connects
Client2 connects
Client1 sends data and asks for reply
Client2 sends data and asks for reply
Client2's request is put into queue
Client1's data is read, server replies, end of transaction
Client2's request is taken from the queue, server reads data, replies end of transaction.
So this is something between asynchronous server and blocking server. I want to do just 1 thing at once but at the same time I want to be able to store all client sockets and their demands in the queue.
I was able to create server-client communication with all the functionality that I need but only on single thread. Once client disconnects server is terminated as well. I don't really know how to start implementing what I have mentioned above. Should I open new thread each time connection is accepted? Should I use async_accept or blocking accept?
I have read boost::asio chat example, where many clients connect so single server, but there is no queuing mechanism that I need here.
I am aware that this post might be a bit confusing but TCP servers are new to me so I am not familiar enough with the terminology. There is also no source code to post because I am asking only for help with concept of this project.
Just keep accepting.
You show no code, but it typically looks like
void do_accept() {
acceptor_.async_accept(socket_, [this](boost::system::error_code ec) {
std::cout << "async_accept -> " << ec.message() << "\n";
if (!ec) {
std::make_shared<Connection>(std::move(socket_))->start();
do_accept(); // THIS LINE
}
});
}
If you don't include the line marked // THIS LINE you will indeed not accept more than 1 connection.
If this doesn't help, please include some code we can work from.
For Fun, A Demo
This uses just standard library features for the non-network part.
Network Listener
The network part is as outlined before:
#include <boost/asio.hpp>
#include <boost/asio/high_resolution_timer.hpp>
#include <istream>
using namespace std::chrono_literals;
using Clock = std::chrono::high_resolution_clock;
namespace Shared {
using PostRequest = std::function<void(std::istream& is)>;
}
namespace Network {
namespace ba = boost::asio;
using ba::ip::tcp;
using error_code = boost::system::error_code;
using Shared::PostRequest;
struct Connection : std::enable_shared_from_this<Connection> {
Connection(tcp::socket&& s, PostRequest poster) : _s(std::move(s)), _poster(poster) {}
void process() {
auto self = shared_from_this();
ba::async_read(_s, _request, [this,self](error_code ec, size_t) {
if (!ec || ec == ba::error::eof) {
std::istream reader(&_request);
_poster(reader);
}
});
}
private:
tcp::socket _s;
ba::streambuf _request;
PostRequest _poster;
};
struct Server {
Server(unsigned port, PostRequest poster) : _port(port), _poster(poster) {}
void run_for(Clock::duration d = 30s) {
_stop.expires_from_now(d);
_stop.async_wait([this](error_code ec) { if (!ec) _svc.post([this] { _a.close(); }); });
_a.listen();
do_accept();
_svc.run();
}
private:
void do_accept() {
_a.async_accept(_s, [this](error_code ec) {
if (!ec) {
std::make_shared<Connection>(std::move(_s), _poster)->process();
do_accept();
}
});
}
unsigned short _port;
PostRequest _poster;
ba::io_service _svc;
ba::high_resolution_timer _stop { _svc };
tcp::acceptor _a { _svc, tcp::endpoint {{}, _port } };
tcp::socket _s { _svc };
};
}
The only "connection" to the work service part is the PostRequest handler that is passed to the server at construction:
Network::Server server(6767, handler);
I've also opted for async operations, so we can have a timer to stop the service, even though we do not use any threads:
server.run_for(3s); // this blocks
The Work Part
This is completely separate, and will use threads. First, let's define a Request, and a thread-safe Queue:
namespace Service {
struct Request {
std::vector<char> data; // or whatever you read from the sockets...
};
Request parse_request(std::istream& is) {
Request result;
result.data.assign(std::istream_iterator<char>(is), {});
return result;
}
struct Queue {
Queue(size_t max = 50) : _max(max) {}
void enqueue(Request req) {
std::unique_lock<std::mutex> lk(mx);
cv.wait(lk, [this] { return _queue.size() < _max; });
_queue.push_back(std::move(req));
cv.notify_one();
}
Request dequeue(Clock::time_point deadline) {
Request req;
{
std::unique_lock<std::mutex> lk(mx);
_peak = std::max(_peak, _queue.size());
if (cv.wait_until(lk, deadline, [this] { return _queue.size() > 0; })) {
req = std::move(_queue.front());
_queue.pop_front();
cv.notify_one();
} else {
throw std::range_error("dequeue deadline");
}
}
return req;
}
size_t peak_depth() const {
std::lock_guard<std::mutex> lk(mx);
return _peak;
}
private:
mutable std::mutex mx;
mutable std::condition_variable cv;
size_t _max = 50;
size_t _peak = 0;
std::deque<Request> _queue;
};
This is nothing special, and doesn't actually use threads yet. Let's make a worker function that accepts a reference to a queue (more than 1 worker can be started if so desired):
void worker(std::string name, Queue& queue, Clock::duration d = 30s) {
auto const deadline = Clock::now() + d;
while(true) try {
auto r = queue.dequeue(deadline);
(std::cout << "Worker " << name << " handling request '").write(r.data.data(), r.data.size()) << "'\n";
}
catch(std::exception const& e) {
std::cout << "Worker " << name << " got " << e.what() << "\n";
break;
}
}
}
The main Driver
Here's where the Queue gets instantiated and both the network server as well as some worker threads are started:
int main() {
Service::Queue queue;
auto handler = [&](std::istream& is) {
queue.enqueue(Service::parse_request(is));
};
Network::Server server(6767, handler);
std::vector<std::thread> pool;
pool.emplace_back([&queue] { Service::worker("one", queue, 6s); });
pool.emplace_back([&queue] { Service::worker("two", queue, 6s); });
server.run_for(3s); // this blocks
for (auto& thread : pool)
if (thread.joinable())
thread.join();
std::cout << "Maximum queue depth was " << queue.peak_depth() << "\n";
}
Live Demo
See It Live On Coliru
With a test load looking like this:
for a in "hello world" "the quick" "brown fox" "jumped over" "the pangram" "bye world"
do
netcat 127.0.0.1 6767 <<< "$a" || echo "not sent: '$a'"&
done
wait
It prints something like:
Worker one handling request 'brownfox'
Worker one handling request 'thepangram'
Worker one handling request 'jumpedover'
Worker two handling request 'Worker helloworldone handling request 'byeworld'
Worker one handling request 'thequick'
'
Worker one got dequeue deadline
Worker two got dequeue deadline
Maximum queue depth was 6
The includes you need. Some maybe are unnecessary:
boost/asio.hpp, boost/thread.hpp, boost/asio/io_service.hpp
boost/asio/spawn.hpp, boost/asio/write.hpp, boost/asio/buffer.hpp
boost/asio/ip/tcp.hpp, iostream, stdlib.h, array, string
vector, string.h, stdio.h, process.h, iterator
using namespace boost::asio;
using namespace boost::asio::ip;
io_service ioservice;
tcp::endpoint sim_endpoint{ tcp::v4(), 4066 }; //{which connectiontype, portnumber}
tcp::acceptor sim_acceptor{ ioservice, sim_endpoint };
std::vector<tcp::socket> sim_sockets;
static int iErgebnis;
int iSocket = 0;
void do_write(int a) //int a is the postion of the socket in the vector
{
int iWSchleife = 1; //to stay connected with putty or something
static char chData[32000];
std::string sBuf = "Received!\r\n";
while (iWSchleife > 0)
{
boost::system::error_code error;
memset(chData, 0, sizeof(chData)); //clear the char
iErgebnis = sim_sockets[a].read_some(boost::asio::buffer(chData), error); //recv data from client
iWSchleife = iErgebnis; //if iErgebnis is bigger then 0 it will stay in the loop. iErgebniss is always >0 when data is received
if (iErgebnis > 0) {
printf("%d data received from client : \n%s\n\n", iErgebnis, chData);
write(sim_sockets[a], boost::asio::buffer(sBuf), error); //send data to client
}
else {
boost::system::error_code ec;
sim_sockets[a].shutdown(boost::asio::ip::tcp::socket::shutdown_send, ec); //close the socket when no data
if (ec)
{
printf("studown error"); // An error occurred.
}
}
}
}
void do_accept(yield_context yield)
{
while (1) //endless loop to accept limitless clients
{
sim_sockets.emplace_back(ioservice); //look to the link below for more info
sim_acceptor.async_accept(sim_sockets.back(), yield); //waits here to accept an client
boost::thread dosome(do_write, iSocket); //when accepted, starts the thread do_write and passes the parameter iSocket
iSocket++; //to know the position of the socket in the vector
}
}
int main()
{
sim_acceptor.listen();
spawn(ioservice, do_accept); //here you can learn more about Coroutines https://theboostcpplibraries.com/boost.coroutine
ioservice.run(); //from here you jump to do:accept
getchar();
}