Task Synchronization for Command/Response server (C++/ESP32/FreeRTOS) - c++

I want to synchronize two tasks in a command/response communication server. One task sends data to a serial port and another task receives data on the serial port. Received data should either be returned to the sender task or do something else with it.
I unsuccessfully tried using volatile bool flags but have now found that won't work with C++ (See When to use volatile with multi threading?)
So trying to use semaphores to do it but can't quite figure out how. Some (bad) psuedo-code using volatile bool is below. How/where to modify for semaphore give/take?
Actual code/platform is C++ 11 running on ESP32 (ESP-IDF). Resources are very limited so no C++ std:: libraries.
volatile bool responsePending = false;
volatile bool cmdAccepted = false;
char sharedBuffer[100];
// SENDER //
void Task1()
{
char localBuffer[100];
while (1)
{
responsePending = true;
cmdAccepted = false;
sendMessage();
while (responsePending)
sleep();
strcpy(localBuffer, sharedBuffer);
cmdAccepted = true; // signal Task2
}
}
// RECEIVER //
void Task2()
{
char localBuf[100];
int fd = open();
while (1)
{
if (select())
{
read(fd, localBuf);
if (responsePending)
{
strcpy(sharedBuffer, localBuf);
responsePending = false; // signal Task1
while (!cmdAccepted)
sleep();
}
else
{
// Do something else with the received data
}
}
}
}

Create a queue which holds a struct. One tasks waits for the serial, if it got data it will put the message to the struct and the struct to the queue.
Other task waits for the queue, if there are items in the queue it will process the struct.
Example:
struct queueData{
char messageBuffer[100];
};
QueueHandle_t queueHandle;
void taskOne(){
while(){
// Task one checks if it got serial data.
if( gotSerialMsg() ){
// create a struct
queueData data;
// copy the data to the struct
strcpy( getSerialMSG(), data.messageBuffer );
// send struct to queue ( waits indefinietly )
xQueueSend(queueHandle, &data, portMAX_DELAY);
}
vTaskDelay(1); // Must feed other tasks
}
}
void taskTwo(){
while(){
// Check if a structs has an item
if( uxQueueMessagesWaiting(queueHandle) > 0 ){
// create a holding struct
queueData data;
// Receive the whole struct
if (xQueueReceive(queueHandle, &data, 0) == pdTRUE) {
// Struct holds message like: data.messageBuffer
}
}
vTaskDelay(1); // Must feed other tasks
}
}
The good thing in passing structs to queues is that you can always put more data into it. booleans or ints or any other thing.

Related

Updating variables and killing threads during thread execution

I am working on a raspberry pi project and want to have a parallel thread in my application that blinks an LED with different frequencies for different states of the application. I think this should be done by threads...
Basically I have an enum and couple of structs to define the states, the GPIO pins and the appropriate action when a status is required. Each status will blink a LED for a given number of times with predefined ON and OFF times.
Then there is a thread that keeps track of a global variable for current state.
My problem is, the function that performs the GPIO state change and On/OFF times using "sleep", sometimes need to be abruptly stopped and a new state be initiated.
How to approach this problem? Below is my very simplified version of code.
enum class Status { // Possible states of the application
Ok = 0,
ErrorNetwork,
NetworkOk
/* and more status! */
};
struct StatusPin { // Define a pin
std::string name;
const int number; // Pin number
};
// Define a pin and its action
struct StatusAction {
std::string name;
const StatusPin pin;
const unsigned int cycles;
const unsigned int millisOn;
const unsigned int millisOff;
};
// Create one pin
const StatusPin G_StatusLed = {
"Status LED",
2
};
// Map for status states
std::map<const Status, StatusAction> G_StatusMap = {
// One short blink
{Status::Ok,
{"Running", G_StatusLed, 1, 250, 0}},
// Blink 3 times with 500ms delay
{Status::ErrorNetwork,
{ "Network Error", G_StatusLed, 3, 500, 500}},
// Blink 6 times, 1 second high, 500ms low
{Status::NetworkOk,
{ "Network OK", G_StatusLed, 6, 1000, 500}}
/* and more states! */
}
const void performStatusAction(const StatusAction &action) {
for (int i = 0; i < action.cycles; i++) {
pi::digitalWrite(action.pin.number, 1); // Set high
// Sleep for high duration
std::this_thread::sleep_for(
std::chrono::milliseconds(action.millisOn));
pi::digitalWrite(action.pin.number, 0); // Set LOW
// Sleep for low duration
std::this_thread::sleep_for(
std::chrono::milliseconds(action.millisOff));
}
}
//------ GLOBAL VARIABLES ------\\
std::mutex mutex;
App::StatusAction* G_CurrentAction = nullptr; // Variable for status action
std::atomic<bool> G_AppRunning {true};
std::atomic<bool> G_StatusWorkerBusy {false};
//------- THREAD FUNCTION -------\\
void statusWorker() {
while(G_AppRunning) {
mutex.lock();
G_StatusWorkerBusy = true;
if(G_CurrentAction && !G_CurrentAction->name.empty()) {
performStatusAction(*G_CurrentAction);
// If OneShot, set point to null so that it does not repeat
if(G_CurrentAction->type == App::StatusActionType::OneShot) {
G_CurrentAction = nullptr;
} else {
spdlog::warn("StatusWorker NULLPTR");
}
}
G_StatusWorkerBusy = false;
mutex.unlock();
std::this_thread::sleep_for(1ms);
}
}
int main() {
init(); // Setup GPIO mode, etc...
std::thread threadStatus(statusWorker); // Start status thead
// Update thread variable and set
G_CurrentAction= &G_StatusMap.at(Status::Ok);
bool isAppDone = false;
// Main loop
while(!isAppDone) {
// do something long e.g. Check for network failed
// Set status to network error
G_CurrentAction= &G_StatusMap.at(Status::ErrorNetwork);
// Immidieatly after 1 second network is back, set new status
// Pervious status should stop and new status should be set
// How to cancel loop inside the statusWorker performStatusAction() ?
G_CurrentAction= &G_StatusMap.at(Status::NetworkOk);
/*
Do good stuff here and update status thread if necessary
*/
// At some point exit the loop
isAppDone = true;
}
// Clear up stuff
threadStatus.join();
return 0;
}
If I understand correctly, we want the LED status thread to notice anytime the app's gross state changes, so the thread can change its flashing/occulting pattern accordingly.
To do this, we can have a std::condition_variable representing the predicate "the state changed." Then, inside the status thread's loop, anytime we're idle we wait_for a signal on that condition variable (instead of sleep_for some time). If state has changed, we'll start over with a new pattern of flashing.
It'd look something like this:
std::condition_variable status_cv;
std::mutex m;
....
// inside status thread
while (G_AppRunning) {
std::unique_lock<std::mutex> locked(m);
// what is app state when we enter this pass?
Status status = G_CurrentStatus;
ActionStatus action = G_StatusMap.at(status);
for (int i = 0; i < action.cycles; i++) {
LED_on();
if (status_cv.wait_for(locked, action.millisOn, [] { return G_CurrentStatus != status; })) {
break; // start anew with a status pattern
}
LED_off();
if (status_cv.wait_for(locked, action.millisOff, [] { return G_CurrentStatus != status; })) {
break; // new state
}
}
}

Passing a function to wxWidgets thread-pool with inter-thread communication

I am a hobby programmer learning C++ and multi-threading, and getting started on my first thread-pool attempt.
I use Code::Blocks 20.3, wxWidgets 3.1.4, and MinGW 17.1 on a Windows 10 Pro computer.
I have tried several thread-pool examples, but all blocked the GUI.
I found an example shown in https://wiki.wxwidgets.org/Inter-Thread_and_Inter-Process_communication
that uses detached threads in a pool. This should not block the GUI.
I have "restructured" the 1 file example to work in a test project (gui, app, main, thread-pool modules).
I placed the classes in their own file, and moved the "main" part to the Main.cpp in my test project and replaced the gui code with a separate class file.
The standard example works as expected.
In the example, strings are passed to the thread-pool and other strings back to the main thread.
I have been searching for main thread AddToQueue() to pass any function like e.g. aTask() (void, or returning something to the main thread) that executes in the thread-pool. My search was not successful :-(.
=== Simple Task to be executed in a thread ===
std::vector<wxString> wxThreadCom2Frame::aTask(wxString wsSomeString, int x)
{
std::vector<wxString> vTest{};
for(int i = 0; i < x; i++)
{
wxString wsTest{};
wsTest << wsSomeString << " [" << i << "]";
vTest.push_back(wsTest);
}
return vTest;
}
=== Or as alternative, pass the vector by reference
aTask(_T("Just some text"), 5, &vTest); // to be queued with AddJob
===
void wxThreadCom2Frame::aTask(wxString wsSomeString, int x, std::vector<wxString> *vTest)
{
for(int i = 0; i < x; i++)
{
wxString wsTest{};
wsTest << wsSomeString << " [" << i << "]\n";
vTest->push_back(wsTest);
}
}
===
I hope someone can help me understand how to do this.
This is the first step to what I actually like to achieve.
An 'extraction' function returns a structure of 20 tags from a music file (mp3, flac, etc).
The main 'collecting' function will call the 'extraction' function for each file (up to 7000) in a list and place it in the queue of the thread-pool.
The 'collecting' function returns a vector of structures to the main thread for further processing.
Regards, Ruud.
=== ThreadCom.cpp ===
/////////////////////////////////////////////////////////////////////////////
// https://wiki.wxwidgets.org/Inter-Thread_and_Inter-Process_communication //
/////////////////////////////////////////////////////////////////////////////
// Standard
#include <stdlib.h>
#include <assert.h>
#include <map>
#include <list>
// wxWidgets
#include <wx/frame.h>
#include <wx/thread.h>
#include <wx/menu.h>
#include <wx/app.h>
class tJOB
{
public:
enum tCOMMANDS // list of commands that are currently implemented
{
eID_THREAD_EXIT=wxID_EXIT, // thread should exit or wants to exit
eID_THREAD_NULL=wxID_HIGHEST+1, // dummy command
eID_THREAD_STARTED, // worker thread has started OK
eID_THREAD_JOB = ID_THREAD_JOB, // process normal job
eID_THREAD_JOBERR = ID_THREAD_JOBERR // process erroneous job after which thread likes to exit
}; // enum tCOMMANDS
tJOB() : m_cmd(eID_THREAD_NULL) {}
tJOB(tCOMMANDS cmd, const wxString& arg) : m_cmd(cmd), m_Arg(arg) {}
tCOMMANDS m_cmd; wxString m_Arg;
}; // class tJOB
class QUEUE
{
public:
enum tPRIORITY { eHIGHEST, eHIGHER, eNORMAL, eBELOW_NORMAL, eLOW, eIDLE }; // priority classes
QUEUE(wxEvtHandler* pParent) : m_pParent(pParent) {}
void AddJob(const tJOB& job, const tPRIORITY& priority=eNORMAL) // push a job with given priority class onto the FIFO
{
wxMutexLocker lock(m_MutexQueue); // lock the queue
m_Jobs.insert(std::make_pair(priority, job)); // insert the prioritized entry into the multimap
m_QueueCount.Post(); // new job has arrived: increment semaphore counter
} // void AddJob(const tJOB& job, const tPRIORITY& priority=eNORMAL)
tJOB Pop()
{
tJOB element;
m_QueueCount.Wait(); // wait for semaphore (=queue count to become positive)
m_MutexQueue.Lock(); // lock queue
element=(m_Jobs.begin())->second; // get the first entry from queue (higher priority classes come first)
m_Jobs.erase(m_Jobs.begin()); // erase it
m_MutexQueue.Unlock(); // unlock queue
return element; // return job entry
} // tJOB Pop()
void Report(const tJOB::tCOMMANDS& cmd, const wxString& sArg=wxEmptyString, int iArg=0) // report back to parent
{
wxCommandEvent evt(wxEVT_THREAD, cmd); // create command event object
evt.SetString(sArg); // associate string with it
evt.SetInt(iArg);
m_pParent->AddPendingEvent(evt); // and add it to parent's event queue
} // void Report(const tJOB::tCOMMANDS& cmd, const wxString& arg=wxEmptyString)
size_t Stacksize() // helper function to return no of pending jobs
{
wxMutexLocker lock(m_MutexQueue); // lock queue until the size has been read
return m_Jobs.size();
}
private:
wxEvtHandler* m_pParent;
std::multimap<tPRIORITY, tJOB> m_Jobs; // multimap to reflect prioritization: values with lower keys come first, newer values with same key are appended
wxMutex m_MutexQueue; // protects queue access
wxSemaphore m_QueueCount; // semaphore count reflects number of queued jobs
};
class WorkerThread : public wxThread
{
public:
WorkerThread(QUEUE* pQueue, int id=0) : m_pQueue(pQueue), m_ID(id) { assert(pQueue); wxThread::Create(); }
private:
QUEUE* m_pQueue;
int m_ID;
virtual wxThread::ExitCode Entry()
{
Sleep(1000); // sleep a while to simulate some time-consuming init procedure
tJOB::tCOMMANDS iErr;
m_pQueue->Report(tJOB::eID_THREAD_STARTED, wxEmptyString, m_ID); // tell main thread that worker thread has successfully started
try { while(true) OnJob(); } // this is the main loop: process jobs until a job handler throws
catch(tJOB::tCOMMANDS& i) { m_pQueue->Report(iErr=i, wxEmptyString, m_ID); } // catch return value from error condition
return (wxThread::ExitCode)iErr; // and return exit code
} // virtual wxThread::ExitCode Entry()
virtual void OnJob()
{
tJOB job=m_pQueue->Pop(); // pop a job from the queue. this will block the worker thread if queue is empty
switch(job.m_cmd)
{
case tJOB::eID_THREAD_EXIT: // thread should exit
Sleep(1000); // wait a while
throw tJOB::eID_THREAD_EXIT; // confirm exit command
case tJOB::eID_THREAD_JOB: // process a standard job
Sleep(2000);
m_pQueue->Report(tJOB::eID_THREAD_JOB, wxString::Format(wxT("Job #%s done."), job.m_Arg.c_str()), m_ID); // report successful completion
break;
case tJOB::eID_THREAD_JOBERR: // process a job that terminates with an error
m_pQueue->Report(tJOB::eID_THREAD_JOB, wxString::Format(wxT("Job #%s erroneous."), job.m_Arg.c_str()), m_ID);
Sleep(1000);
throw tJOB::eID_THREAD_EXIT; // report exit of worker thread
break;
case tJOB::eID_THREAD_NULL: // dummy command
default:
break; // default
} // switch(job.m_cmd)
} // virtual void OnJob()
}; // class WorkerThread : public wxThread
=== partial wxThreadCom2Main.cpp ===
void wxThreadCom2Frame::AddToQueue( wxCommandEvent& event )
{
int iJob=rand();
m_pQueue->AddJob(tJOB((tJOB::tCOMMANDS)event.GetId(), wxString::Format(wxT("%u"), iJob)));
SetStatusText(wxString::Format(wxT("Job #%i started."), iJob)); // just set the status text
}
void wxThreadCom2Frame::OnThread(wxCommandEvent& event) // handler for thread notifications
{
switch(event.GetId())
{
case tJOB::eID_THREAD_JOB:
// Get the returned vector and do something with it
SetStatusText(wxString::Format(wxT("[%i]: %s"), event.GetInt(), event.GetString().c_str())); // progress display
break;
case tJOB::eID_THREAD_EXIT:
SetStatusText(wxString::Format(wxT("[%i]: Stopped."), event.GetInt()));
m_Threads.remove(event.GetInt()); // thread has exited: remove thread ID from list
if(m_Threads.empty()) { EnableControls(false); } // disable some menu items if no more threads
break;
case tJOB::eID_THREAD_STARTED:
SetStatusText(wxString::Format(wxT("[%i]: Ready."), event.GetInt()));
EnableControls(true); // at least one thread successfully started: enable controls
break;
default:
event.Skip();
}
}
void wxThreadCom2Frame::EnableControls(bool bEnable) // en/dis-able Stop, Add Job, Add JobErr
{
wxMenu* pMenu=GetMenuBar()->GetMenu(0);
static const int MENUIDS[]={/*ID_START_THREAD, */ID_THREAD_EXIT, ID_THREAD_JOB, ID_THREAD_JOBERR};
for(unsigned int i=0; i<WXSIZEOF(MENUIDS); pMenu->Enable(MENUIDS[i++], bEnable));
}
===

Should i pass unique OVERLAPPED structure for each WSASend call , in this case?

I have a list of sockets.(Opened connections)
I have n worker threads.
Thread loop:
while (1)
{
_this.result = GetQueuedCompletionStatus(a_server.server_iocp, &_this.numberOfBytesTransfered,
&_this.completionKey, (OVERLAPPED**)&_this.iocp_task, INFINITE);
...
}
I have this simple struct:
struct iocp_send_global :public iocp_task<IOCP_SEND_GLOBAL> {
OVERLLAPED ov; //overlapped struct at top
std::atomic_uint32_t ref;
bool decr_ref(){ return ref.fetch_sub(1, std::memory_order_acq_rel) == 1;}
//packet data here
}
...
This is the 'Broadcast' function:
iocp_send_global * packet = new iocp_send_global;
[set packet data here]
for(int i=0;i<connectionsCount;++i){
WSASend(connections[i],...,&packet,...); //posting same packet to all connections
}
I want to do this in the worker loop after GetQueuedCompletionStatus call returns with the overlapped result;
if (_this.iocp_task->type == IOCP_SEND_GLOBAL) {
auto* task = (iocp_send_global*)_this.iocp_task;
if (!task->decr_ref()) {
_this.iocp_task = nullptr;
//dont delete the task yet,
//all send post must finish first
//[all posts share the same buffer]
}
else {
//delete the task containing the send data after all send posts finished
delete _this.iocp_task;
_this.iocp_task = nullptr;
}
}
From what i read on Microsoft WSASend documentation each WSASend overlapped call sould have its own OVERLAPPED structure passed, but is that valid when i WSASend the same buffer?
Thank you!
You must pass a different OVERLAPPED buffer for each call since you'll be making multiple pending calls. This is clearly spelled out in the documentation for the OVERLAPPED structure.

How can I use appropriate the lock mutex function, for three threads in C++?

I have a question about threads but I think that is difficult to explain, so be patient.
I have two pthreads in a QT/C++ program and one signal, Signal fills a buffer, One thread copies the buffer and one to process the buffer's data.
fill buffer1 ----Copy buffer1 to buffer2----process the buffer's 2 data
Signal's function:
void MainWindow::TcpData()
{
if(socket->bytesAvailable()>(DATA_LEN)) {
QByteArray array = socket ->readAll();
if(pthread_mutex_trylock(&data_mutex)==0)
{
if((p+array.size())<(MAX_TCP_BUFFER_SIZE+100))
{
memcpy(BUFFER+p,array.data(),array.size());
p+=array.size();
}
else {
p=0;
memcpy(BUFFER,array.data(),array.size());
p+=array.size();
}
pthread_mutex_unlock(&data_mutex);
}
}
}
Thread 1:
void *MainWindow::copyTCPdata() {
pthread_mutex_lock(&data_mutex);
while(1) {
if(data_ready) {
pthread_cond_wait(&data_cond,&data_mutex);
continue;
}
/* Move the last part of the previous buffer, that was not processed,
* on the start of the new buffer. */
memcpy(data, data+DATA_LEN, (FULL_LEN-1)*4);
/* Read the new data. */
memcpy(data+(FULL_LEN-1)*4, BUFFER,DATA_LEN);
memcpy(BUFFER,BUFFER+DATA_LEN,p);
if(p>DATA_LEN) p=p-DATA_LEN;
data_ready = 1;
pthread_cond_signal(&data_cond);
pthread_mutex_unlock(&data_mutex);
} }
Thread 2:
void *MainWindow::processData {
while(1) {
if(!data_ready) {
pthread_cond_wait(&data_cond,&data_mutex);
continue;
}
data_ready = 0;
pthread_cond_signal(&data_cond);
pthread_mutex_unlock(&data_mutex);
detectSignal(data);
pthread_mutex_lock(&data_mutex);
}
}
I think am loosing data with this way, but the program is more stable, Can someone suggest me a better solution?

Sending data in second thread with Mongoose server

I'm trying to create a multithread server application using mongoose web server library.
I have main thread serving connections and sending requests to processors that are working in their own threads. Then processors place results into queue and queue observer must send results back to clients.
Sources are looking that way:
Here I prepare the data for processors and place it to queue.
typedef std::pair<struct mg_connection*, const char*> TransferData;
int server_app::event_handler(struct mg_connection *conn, enum mg_event ev)
{
Request req;
if (ev == MG_AUTH)
return MG_TRUE; // Authorize all requests
else if (ev == MG_REQUEST)
{
req = parse_request(conn);
task_queue->push(TransferData(conn,req.second));
mg_printf(conn, "%s", ""); // (1)
return MG_MORE; // (2)
}
else
return MG_FALSE; // Rest of the events are not processed
}
And here I'm trying to send the result back. This function is working in it's own thread.
void server_app::check_results()
{
while(true)
{
TransferData res;
if(!res_queue->pop(res))
{
boost::this_thread::sleep_for(boost::chrono::milliseconds(100));
continue;
}
mg_printf_data(res.first, "%s", res.second); // (3)
}
}
The problem is a client doesn't receive anything from the server.
If I run check_result function manualy in the event_handler after placing a task into the queue and then pass computed result back to event_handler, I'm able to send it to client using mg_printf_data (with returning MG_TRUE). Any other way - I'm not.
What exactly should I change in this sources to make it works?
Ok... It looks like I've solved it myself.
I'd been looking into mongoose.c code and an hour later I found the piece of code below:
static void write_terminating_chunk(struct connection *conn) {
mg_write(&conn->mg_conn, "0\r\n\r\n", 5);
}
static int call_request_handler(struct connection *conn) {
int result;
conn->mg_conn.content = conn->ns_conn->recv_iobuf.buf;
if ((result = call_user(conn, MG_REQUEST)) == MG_TRUE) {
if (conn->ns_conn->flags & MG_HEADERS_SENT) {
write_terminating_chunk(conn);
}
close_local_endpoint(conn);
}
return result;
}
So I've tried to do mg_write(&conn->mg_conn, "0\r\n\r\n", 5); after line (3) and now it's working.