I'm a bit stumped on an issue I'm having with threading and C++. I'm writing a DSP plugin for Windows Media Player, and I want to send the data I intercept to a separate thread where I'll send it out on the network. I'm using a simple producer-consumer queue like the one explained here
The program is crashing on the isFull() function which just compares two integers:
bool ThreadSafeQueue::isFull()
{
if (inCount == outCount) //CRASH!
return true;
else
return false;
}
The thread that's doing the dequeuing:
void WMPPlugin::NetworkThread (LPVOID pParam)
{
ThreadSafeQueue* dataQueue = (ThreadSafeQueue*)(pParam);
while (!networkThreadDone)
{
Sleep(2); /// so we don't hog the processor or make a race condition
if (!dataQueue->isFull())
short s = dataQueue->dequeue();
if (networkThreadDone) // variable set in another process so we know to exit
break;
}
}
The constructor of the class that's creating the consumer thread:
WMPPlugin::WMPPlugin()
{
// etc etc
dataQueue = new ThreadSafeQueue();
_beginthread(WMPPlugin::NetworkThread, 0, dataQueue);
}
inCount and outCount are just integers and they're only read here, not written. I was under the impression this meant they were thread safe. The part that writes them aren't included, but each variable is only written to by one thread, never by both. I've done my best to not include code that I don't feel is the issue, but I can include more if necessary. Thanks in advance for any help.
Most often, when a crash happens accessing a normal member variable, it means this is NULL or an invalid address.
Are you sure you aren't invoking it on a NULL instance?
Regarding this line:
ThreadSafeQueue* dataQueue = (ThreadSafeQueue*)(pParam);
How sure are you that pParam is always non-NULL?
How sure are you that pParam is always a ThreadSafeQueue object?
Are you possible deleting the ThreadSafeQueue objects on other threads?
Related
Overview
I need to save a CDocument in a background worker thread. There is a point in our MFC application which prompts the user to save before continuing. Normally, they are able to continue without saving, and there is no problem. However, occasionally, we need that document later in the process, so if the user clicks "No", we want to save a temp version of the file in the background without making the user wait for the save to continue.
Problem
When I launch AfxBeginThread(SaveDocumentThread, &threadInput) the &threadinput has been cleared from memory before the SaveDocumentThread starts.
Code
BOOL SPackagerDoc::OnSaveDocument( IN LPCTSTR lpszPathName)
{
ProcessDocumentThreadInput threadInput(this, lpszPathName);
// Temp Save Mode
if (m_bTempMode)
{
m_TempSaveThread = AfxBeginThread(SaveDocumentThread, &threadInput);
// This fixes the problem, but is considered unstable
// if (m_TempSaveThread->m_hThread)
// WaitForSingleObject(m_TempSaveThread->m_hThread, 500);
return TRUE;
}
// Normal save mode
SFileLoadingDialog loadingDialog(SFileLoadingDialog::SAVE, lpszPathName, SaveDocumentThread, &threadInput);
BOOL result = (BOOL)loadingDialog.DoModal();
return result;
}
StUInt32 SPackagerDoc::SaveDocumentThread(IN StVoid* pParam)
{
ProcessDocumentThreadInput* input = (ProcessDocumentThreadInput*)pParam;
ASSERT_NOT_NULL(input);
ASSERT_NOT_NULL(input->pPackager);
ASSERT_NOT_NULL(input->pszPathName);
CString path_name(input->pszPathName);
BOOL result = input->pPackager->SPackagerDocBase::OnSaveDocument(path_name);
return result;
}
If I uncommend WaitForSingleObject(..., 500); then the thread starts, all the information is present, and there are no errors. But if I remove those lines then in SaveDocumentThread input is NULL and all data is zeros or garbage.
Is there a way to ensure the SaveDocumentThread has started before moving on. IE, wait for thread to start, but not for a specified amount of time (500 ms). It may be that 500 ms is not a sufficient wait time on some other computers.
Is there an "official" way to do this?
This is the issue of the scope of variable.
Following comments specified the scope of local variable threadInput.
ProcessDocumentThreadInput threadInput(this, lpszPathName); // <=== threadInput created
if (m_bTempMode)
{
m_TempSaveThread = AfxBeginThread(SaveDocumentThread, &threadInput);
// This fixes the problem, but is considered unstable
// if (m_TempSaveThread->m_hThread)
// WaitForSingleObject(m_TempSaveThread->m_hThread, 500);
return TRUE; // <=== threadInput destructed
}
Your workaround WaitForSingleObject() delays the destruction of the variable threadInput and you see the result.
To overcome the scope of local variable.
Store it in a class member variable.
Store it as a (better be smart) pointer and (better not to) handle it's destruction.
Edit:
As #Jabberwocky stated, function OnSaveDocument() might be called more than twice since it's called by background thread.
I'll suggest to refactor the save() function out and let if and else to call them seperately.
As others have pointed out, the problem is the lifetime of threadInput ends before the thread begins.
You can dynamically allocate the instance of ProcessDocumentThreadInput and pass the pointer to that instance to the thread.
auto* threadInput = new ProcessDocumentThreadInput(this, lpszPathName);
...
AfxBeginThread(SaveDocumentThread, threadInput);
However, in this case, the responsibility to release the memory gets messy.
Since you put C++11 tag in your question, you might want to make use of std::shared_ptr or std::unique_ptr and pass it to the thread, which would land you in using std::thread instead of AfxBeginThread. (BTW, I have no experience using MFC.)
BOOL SPackagerDoc::OnSaveDocument( IN LPCTSTR lpszPathName)
{
...
std::thread t(SaveDocumentThread, std::make_unique<ProcessDocumentThreadInput>(this, lpszPathName));
...
}
...
StUInt32 SaveDocumentThread(std::unique_ptr<ProcessDocumentThreadInput>&& threadInput)
{
...
}
I have main() and thread in the same program.
there is a variable named "status", that can get several values
I need that when the variable changes, to notify the thread (the thread cnat wait for the status variable, it is already doing fluent task) .
is there an easy way to do so? similar to interrupts? how about signals?
the function inside the main:
int main()
{
char *status;
...
...
while (1)
{
switch (status)
{
case: status1 ...notify the thread
case: status2 ...notify the thread
case: status3 ...notify the thread
}
}
}
if someone could give me an example it will be great!
thanks!
Since you're already using the pthread library you can use conditional variables to tell the thread that there is data ready for processing. Take a look at this StackOverflow question for more information.
I understand that you do not want to wait indefinitely for this notification, however C++ only implements cooperative scheduling. You cannot just pause a thread, fiddle with its memory, and resume it.
Therefore, the first thing you have to understand is that the thread which has to process the signal/action you want to send must be willing to do so; which in other words means must explicitly check for the signal at some point.
There are multiple ways for a thread to check for a signal:
condition variable: they require waiting for the signal (which might be undesirable) but that wait can be bounded by a duration
action queue (aka channel): you create a queue of signals/actions and every so often the target thread checks for something to do; if there is nothing it just goes on doing whatever it has to do, if there is something you have to decide whether it should do everything or only process the N firsts. Beware of overflowing the queue.
just check the status variable directly every so often, it does not tell you how many times it changed (unless it keeps an history: but then we are back to the queue), but it allows you to amend your ways.
Given your requirements, I would think that the queue is probably the best idea among those three.
Might be this example helpful for you.
DWORD sampleThread( LPVOID argument );
int main()
{
bool defValue = false;
bool* status = &defValue;
CreateThread(NULL, 0, sampleThread, status, 0,NULL);
while(1)
{
//.............
defValue = true; //trigger thread
// ...
}
return 0;
}
DWORD sampleThread( LPVOID argument )
{
bool* syncPtr = reinterpret_cast<bool*>(argument);
while (1)
{
if (false == *syncPtr)
{
// do something
}
else (true = *syncPtr)
{
//do somthing else
}
}
}
I want to pass some data around threads but want to refrain from using global variables if I can manage it. The way I wrote my thread routine has the user passing in a separate function for each "phase" of a thread's life cycle: For instance this would be a typical usage of spawning a thread:
void init_thread(void *arg) {
graphics_init();
}
void process_msg_thread(message *msg, void *arg) {
if (msg->ID == MESSAGE_DRAW) {
graphics_draw();
}
}
void cleanup_thread(void *arg) {
graphics_cleanup();
}
int main () {
threadCreator factory;
factory.createThread(init_thread, 0, process_msg_thread, 0, cleanup_thread, 0);
// even indexed arguments are the args to be passed into their respective functions
// this is why each of those functions must have a fixed function signature is so they can be passed in this way to the factory
}
// Behind the scenes: in the newly spawned thread, the first argument given to
// createThread() is called, then a message pumping loop which will call the third
// argument is entered. Upon receiving a special exit message via another function
// of threadCreator, the fifth argument is called.
The most straightforward way to do it is using globals. I'd like to avoid doing that though because it is bad programming practice because it generates clutter.
A certain problem arises when I try to refine my example slightly:
void init_thread(void *arg) {
GLuint tex_handle[50]; // suppose I've got 50 textures to deal with.
graphics_init(&tex_handle); // fill up the array with them during graphics init which loads my textures
}
void process_msg_thread(message *msg, void *arg) {
if (msg->ID == MESSAGE_DRAW) { // this message indicates which texture my thread was told to draw
graphics_draw_this_texture(tex_handle[msg->texturehandleindex]); // send back the handle so it knows what to draw
}
}
void cleanup_thread(void *arg) {
graphics_cleanup();
}
I am greatly simplifying the interaction with the graphics system here but you get the point. In this example code tex_handle is an automatic variable, and all its values are lost when init_thread completes, so will not be available when process_msg_thread needs to reference it.
I can fix this by using globals but that means I can't have (for instance) two of these threads simultaneously since they would trample on each other's texture handle list since they use the same one.
I can use thread-local globals but is that a good idea?
I came up with one last idea. I can allocate storage on the heap in my parent thread, and send a pointer to in to the children to mess with. So I can just free it when parent thread leaves away since I intend for it to clean up its children threads before it exits anyway. So, something like this:
void init_thread(void *arg) {
GLuint *tex_handle = (GLuint*)arg; // my storage space passed as arg
graphics_init(tex_handle);
}
void process_msg_thread(message *msg, void *arg) {
GLuint *tex_handle = (GLuint*)arg; // same thing here
if (msg->ID == MESSAGE_DRAW) {
graphics_draw_this_texture(tex_handle[msg->texturehandleindex]);
}
}
int main () {
threadCreator factory;
GLuint *tex_handle = new GLuint[50];
factory.createThread(init_thread, tex_handle, process_msg_thread, tex_handle, cleanup_thread, 0);
// do stuff, wait etc
...
delete[] tex_handle;
}
This looks more or less safe because my values go on the heap, my main thread allocates it then lets children mess with it as they wish. The children can use the storage freely since the pointer was given to all the functions that need access.
So this got me thinking why not just have it be an automatic variable:
int main () {
threadCreator factory;
GLuint tex_handle[50];
factory.createThread(init_thread, &tex_handle, process_msg_thread, &tex_handle, cleanup_thread, 0);
// do stuff, wait etc
...
} // tex_handle automatically cleaned up at this point
This means children thread directly access parent's stack. I wonder if this is kosher.
I found this on the internets: http://software.intel.com/sites/products/documentation/hpc/inspectorxe/en-us/win/ug_docs/olh/common/Problem_Type__Potential_Privacy_Infringement.htm
it seems Intel Inspector XE detects this behavior. So maybe I shouldn't do it? Is it just simply a warning of potential privacy infringement as suggested by the the URL or are there other potential issues that may arise that I am not aware of?
P.S. After thinking through all this I realize that maybe this architecture of splitting a thread into a bunch of functions that get called independently wasn't such a great idea. My intention was to remove the complexity of requiring coding up a message handling loop for each thread that gets spawned. I had anticipated possible problems, and if I had a generalized thread implementation that always checked for messages (like my custom one that specifies the thread is to be terminated) then I could guarantee that some future user could not accidentally forget to check for that condition in each and every message loop of theirs.
The problem with my solution to that is that those individual functions are now separate and cannot communicate with each other. They may do so only via globals and thread local globals. I guess thread local globals may be my best option.
P.P.S. This got me thinking about RAII and how the concept of the thread at least as I have ended up representing it has a certain similarity with that of a resource. Maybe I could build an object that represents a thread more naturally than traditional ways... somehow. I think I will go sleep on it.
Put your thread functions into a class. Then they can communicate using instance variables. This requires your thread factory to be changed, but is the cleanest way to solve your problem.
Your idea of using automatic variables will work too as long as you can guarantee that the function whose stack frame contains the data will never return before your child threads exit. This is not really easy to achieve, even after main() returns child threads can still run.
I have a program that processes neural spike data that is broadcast in UDP packets on a local network.
My current program has two threads a UI thread and a worker thread. The worker thread simply listens for data packets, parses them and makes them available to the UI thread for display and processing. My current implementation works just fine. However for a variety of reasons I'm trying to re-write the program in C++ using an Object Oriented approach.
The current working program initialized the 2nd thread with:
pthread_t netThread;
net = NetCom::initUdpRx(host,port);
pthread_create(&netThread, NULL, getNetSpike, (void *)NULL);
Here is the getNetSpike function that is called by the new thread:
void *getNetSpike(void *ptr){
while(true)
{
spike_net_t s;
NetCom::rxSpike(net, &s);
spikeBuff[writeIdx] = s;
writeIdx = incrementIdx(writeIdx);
nSpikes+=1;
totalSpikesRead++;
}
}
Now in my new OO version of the program I setup the 2nd thread in much the same way:
void SpikePlot::initNetworkRxThread(){
pthread_t netThread;
net = NetCom::initUdpRx(host,port);
pthread_create(&netThread, NULL, networkThreadFunc, this);
}
However, because pthead_create takes a pointer to a void function and not a pointer to an object's member method I needed to create this simple function that wraps the SpikePlot.getNetworSpikePacket() method
void *networkThreadFunc(void *ptr){
SpikePlot *sp = reinterpret_cast<SpikePlot *>(ptr);
while(true)
{
sp->getNetworkSpikePacket();
}
}
Which then calls the getNetworkSpikePacket() method:
void SpikePlot::getNetworkSpikePacket(){
spike_net_t s;
NetCom::rxSpike(net, &s);
spikeBuff[writeIdx] = s; // <--- SegFault/BusError occurs on this line
writeIdx = incrementIdx(writeIdx);
nSpikes+=1;
totalSpikesRead++;
}
The code for the two implementations is nearly identical but the 2nd implementation (OO version) crashes with a SegFault or BusError after the first packet that is read. Using printf I've narrowed down which line is causing the error:
spikeBuff[writeIdx] = s;
and for the life of me I can't figure out why its causing my program to crash.
What am I doing wrong here?
Update:
I define spikeBuff as a private member of the class:
class SpikePlot{
private:
static int const MAX_SPIKE_BUFF_SIZE = 50;
spike_net_t spikeBuff[MAX_SPIKE_BUFF_SIZE];
....
}
Then in the SpikePlot constructor I call:
bzero(&spikeBuff, sizeof(spikeBuff));
and set:
writeIdx =0;
Update 2: Ok something really weird is going on with my index variables. To test their sanity I changed getNetworkSpikePacket to:
void TetrodePlot::getNetworkSpikePacket(){
printf("Before:writeIdx:%d nspikes:%d totSpike:%d\n", writeIdx, nSpikes, totalSpikesRead);
spike_net_t s;
NetCom::rxSpike(net, &s);
// spikeBuff[writeIdx] = s;
writeIdx++;// = incrementIdx(writeIdx);
// if (writeIdx>=MAX_SPIKE_BUFF_SIZE)
// writeIdx = 0;
nSpikes += 1;
totalSpikesRead += 1;
printf("After:writeIdx:%d nspikes:%d totSpike:%d\n\n", writeIdx, nSpikes, totalSpikesRead);
}
And I get the following output to the console:
Before:writeIdx:0 nspikes:0 totSpike:0
After:writeIdx:1 nspikes:32763 totSpike:2053729378
Before:writeIdx:1 nspikes:32763 totSpike:2053729378
After:writeIdx:1 nspikes:0 totSpike:1
Before:writeIdx:1 nspikes:0 totSpike:1
After:writeIdx:32768 nspikes:32768 totSpike:260289889
Before:writeIdx:32768 nspikes:32768 totSpike:260289889
After:writeIdx:32768 nspikes:32768 totSpike:260289890
This method is the only method where I update their values (besides the constructor where I set them to 0). All other uses of these variables are read only.
I'm going to go on a limb here and say all your problems are caused by the zeroing out of the spike_net_t array.
In C++ you must not zero out objects with non-[insert word for 'struct-like' here] members. i.e. if you have an object that contains a complex object (a std string, a vector, etc. etc.) you cannot zero it out, as this destroys the initialization of the object done in the constructor.
This may be wrong but....
You seemed to move the wait loop logic out of the method and into the static wrapper. With nothing holding the worker thread open, perhaps that thread terminates after the first time you wait for a UDP packet, so second time around, sp in the static method now points to an instance that has left scope and been destructed?
Can you try to assert(sp) in the wrapper before trying to call its getNetworkSpikePacket()?
It looks like your reinterpret_cast might be causing some problems. When you call pthread_create, you are passing in "this" which is a SpikePlot*, but inside networkThreadFunc, you are casting it to a TetrodePlot*.
Are SpikePlot and TetrodePlot related? This isn't called out in what you've posted.
If you are allocating the spikeBuff array anywhere then make sure you are allocating sufficient storage so writeIdx is not an out-of-bounds index.
I'd also check that initNetworkRxThread is being called on an allocated instance of spikePlot object (and not on just a declared pointer).
I have an object that is called from two different threads and after it was called by both it destroys itself by "delete this".
How do I implement this thread-safe? Thread-safe means that the object never destroys itself exactly one time (it must destroys itself after the second callback).
I created some example code:
class IThreadCallBack
{
virtual void CallBack(int) = 0;
};
class M: public IThreadCallBack
{
private:
bool t1_finished, t2_finished;
public:
M(): t1_finished(false), t2_finished(false)
{
startMyThread(this, 1);
startMyThread(this, 2);
}
void CallBack(int id)
{
if (id == 1)
{
t1_finished = true;
}
else
{
t2_finished = true;
}
if (t1_finished && t2_finished)
{
delete this;
}
}
};
int main(int argc, char **argv) {
M* MObj = new M();
while(true);
}
Obviously I can't use a Mutex as member of the object and lock the delete, because this would also delete the Mutex. On the other hand, if I set a "toBeDeleted"-flag inside a mutex-protected area, where the finised-flag is set, I feel unsure if there are situations possible where the object isnt deleted at all.
Note that the thread-implementation makes sure that the callback method is called exactly one time per thread in any case.
Edit / Update:
What if I change Callback(..) to:
void CallBack(int id)
{
mMutex.Obtain()
if (id == 1)
{
t1_finished = true;
}
else
{
t2_finished = true;
}
bool both_finished = (t1_finished && t2_finished);
mMutex.Release();
if (both_finished)
{
delete this;
}
}
Can this considered to be safe? (with mMutex being a member of the m class?)
I think it is, if I don't access any member after releasing the mutex?!
Use Boost's Smart Pointer. It handles this automatically; your object won't have to delete itself, and it is thread safe.
Edit:
From the code you've posted above, I can't really say, need more info. But you could do it like this: each thread has a shared_ptr object and when the callback is called, you call shared_ptr::reset(). The last reset will delete M. Each shared_ptr could be stored with thread local storeage in each thread. So in essence, each thread is responsible for its own shared_ptr.
Instead of using two separate flags, you could consider setting a counter to the number of threads that you're waiting on and then using interlocked decrement.
Then you can be 100% sure that when the thread counter reaches 0, you're done and should clean up.
For more info on interlocked decrement on Windows, on Linux, and on Mac.
I once implemented something like this that avoided the ickiness and confusion of delete this entirely, by operating in the following way:
Start a thread that is responsible for deleting these sorts of shared objects, which waits on a condition
When the shared object is no longer being used, instead of deleting itself, have it insert itself into a thread-safe queue and signal the condition that the deleter thread is waiting on
When the deleter thread wakes up, it deletes everything in the queue
If your program has an event loop, you can avoid the creation of a separate thread for this by creating an event type that means "delete unused shared objects" and have some persistent object respond to this event in the same way that the deleter thread would in the above example.
I can't imagine that this is possible, especially within the class itself. The problem is two fold:
1) There's no way to notify the outside world not to call the object so the outside world has to be responsible for setting the pointer to 0 after calling "CallBack" iff the pointer was deleted.
2) Once two threads enter this function you are, and forgive my french, absolutely fucked. Calling a function on a deleted object is UB, just imagine what deleting an object while someone is in it results in.
I've never seen "delete this" as anything but an abomination. Doesn't mean it isn't sometimes, on VERY rare conditions, necessary. Problem is that people do it way too much and don't think about the consequences of such a design.
I don't think "to be deleted" is going to work well. It might work for two threads, but what about three? You can't protect the part of code that calls delete because you're deleting the protection (as you state) and because of the UB you'll inevitably cause. So the first goes through, sets the flag and aborts....which of the rest is going to call delete on the way out?
The more robust implementation would be to implement reference counting. For each thread you start, increase a counter; for each callback call decrease the counter and if the counter has reached zero, delete the object. You can lock the counter access, or you could use the Interlocked class to protect the counter access, though in that case you need to be careful with potential race between the first thread finishing and the second starting.
Update: And of course, I completely ignored the fact that this is C++. :-) You should use InterlockExchange to update the counter instead of the C# Interlocked class.