I'm trying to fix this problem which seems like I am accessing at an out of range index, but VS fails to stop where the error occurred leaving me confused about what's causing this.
The Error:
Debug Assertion Failed! Program: .... File: c:\program files\microsoft visual studio 10.0\vc\include\vector Line: 1440 Expression: String subscript out of range
What the program does:
There are two threads:
Thread 1:
The first thread looks (amongst other things) for changes in the current window using GetForegroundWindow(), the check happens not on a loop but when a WH_MOUSE_LL event is triggered. The data is split into structs of fixed size so that it can be sent to a server over tcp. The first thread and records the data (Window Title) into an std::list in the current struct.
if(change_in_window)
{
GetWindowTextW(hActWin,wTitle,256);
std::wstring title(wTitle);
current_struct->titles.push_back(title);
}
Thread 2:
The second thread is called looks for structs not send yet, and it puts their content into char buffers so that they can be sent over tcp. While I do not know exactly where the error is, looking from the type of error it was to do either with a string or a list, and this is the only code from my whole application using lists/strings (rest are conventional arrays). Also commenting the if block as mentioned in the code comments stops the error from happening.
BOOL SendStruct(DATABLOCK data_block,bool sycn)
{
[..]
int _size = 0;
// Important note, when this if block is commented the error ceases to exist, so it has something to do with the following block
if(!data_block.titles.empty()) //check if std::list is empty
{
for (std::list<std::wstring>::iterator itr = data_block.titles.begin(); itr != data_block.titles.end() ; itr++) {
_size += (((*itr).size()+1) * 2);
} //calculate size required. Note the +1 is for an extra character between every title
wchar_t* wnd_wbuffer = new wchar_t[_size/2](); //allocate space
int _last = 0;
//loop through every string and every char of a string and write them down
for (std::list<std::wstring>::iterator itr = data_block.titles.begin(); itr != data_block.titles.end(); itr++)
{
for(unsigned int i = 0; i <= (itr->size()-1); i++)
{
wnd_wbuffer[i+_last] = (*itr)[i] ;
}
wnd_wbuffer[_last+itr->size()] = 0x00A6; // separator
_last += itr->size()+1;
}
unsigned char* wnd_buffer = new unsigned char[_size];
wnd_buffer = (unsigned char*)wnd_wbuffer;
h_io->header_w_size = _size;
h_io->header_io_wnd = 1;
Connect(mode,*header,conn,buffer_in_bytes,wnd_buffer,_size);
delete wnd_wbuffer;
}
else
[..]
return true;
}
My attempt at thread synchronization:
There is a pointer to the first data_block created (db_main)
pointer to the current data_block (db_cur)
//datablock format
typedef struct _DATABLOCK
{
[..]
int logs[512];
std::list<std::wstring> titles;
bool bPrsd; // has this datablock been sent true/false
bool bFull; // is logs[512] full true/false
[..]
struct _DATABLOCK *next;
} DATABLOCK;
//This is what thread 1 does when it needs to register a mouse press and it is called like this:
if(change_in_window)
{
GetWindowTextW(hActWin,wTitle,256);
std::wstring title(wTitle);
current_struct->titles.push_back(title);
}
RegisterMousePress(args);
[..]
//pseudo-code to simplify things , although original function does the exact same thing.
RegisterMousePress()
{
if(it_is_full)
{
db_cur->bFull= true;
if(does db_main exist)
{
db_main = new DATABLOCK;
db_main = db_cur;
db_main->next = NULL;
}
else
{
db_cur->next = new DATABLOCK;
db_cur = db_cur->next;
db_cur->next = NULL;
}
SetEvent(eProcessed); //tell thread 2 there is at least one datablock ready
}
else
{
write_to_it();
}
}
//this is actual code and entry point of thread 2 and my attempy at synchronization
DWORD WINAPI InitQueueThread(void* Param)
{
DWORD rc;
DATABLOCK* k;
SockWClient writer;
k = db_main;
while(true)
{
rc=WaitForSingleObject(eProcessed,INFINITE);
if (rc== WAIT_OBJECT_0)
{
do
{
if(k->bPrsd)
{
continue;
}
else
{
if(!k)
{break;}
k->bPrsd = TRUE;
#ifdef DEBUG_NET
SendStruct(...);
#endif
}
if(k->next == NULL || k->next->bPrsd ==TRUE || !(k->next->bFull))
{
ResetEvent(eProcessed);
break;
}
} while (k = k->next); // next element after each loop
}
}
return 1;
}
Details:
Now something makes me believe that the error is not in there, because the substring error is very rare. I have been only able to reproduce it with 100% chance when pressing Mouse_Down+Wnd+Tab to scroll through windows and keeping it pressed for some time (while it certainly happened on other cases as well). I avoid posting the whole code because it's a bit large and confusion is unavoidable. If the error is not here I will edit the post and add more code.
Thanks in advance
There does not appear to be any thread synchronization here. If one thread reads from the structure while the other writes, it might be read during initialization, with a non-empty list containing an empty string (or something invalid, in between).
If there isn't a mutex or semaphore outside the posted function, that is likely the problem.
All the size calculations appear to be valid for Windows, although I didn't attempt to run it… and <= … -1 instead of < in i <= (itr->size()-1) and 2 instead of sizeof (wchar_t) in new wchar_t[_size/2](); are a bit odd.
The problem with your code is that while thread 2 correctly waits for the data and thread 1 correctly notifies about them, thread 2 doesn't prevent thread 1 from doing anything with them under its hands while it still process the data. The typical device used to solve such problem is the monitor pattern.
It consist of one mutex (used to protect the data, held anytime you access them) and a condition variable (=Event in Windows terms), which will convey the information about new data to the consumer.
The producer would normally obtain the mutex, produce the data, release the mutex, then fire the event.
The consumer is more tricky - it has to obtain the mutex, check if new data hasn't become available, then wait for the Event using the SignalObjectAndWait function that temporarily releases the mutex, then process newly acquired data, then release the mutex.
Related
Suppose I have some global:
std::atomic_int next_free_block;
and a number of threads each with access to a
std::atomic_int child_offset;
that may be shared between threads. I would like to allocate free blocks to child offsets in a contiguous manner, that is, I want to perform the following operation atomically:
if (child_offset != 0) child_offset = next_free_block++;
Obviously the above implementation does not work as multiple threads may enter the body of the if statement and then try to assign different blocks to child_offset.
I have also considered the following:
int expected = child_offset;
do {
if (expected == 0) break;
int updated = next_free_block++;
} while (!child_offset.compare_exchange_weak(&expected, updated);
But this also doesn't work because if the CAS fails, the side effect of incrementing next_free_block remains even if nothing is assigned to child_offset. This leaves gaps in the allocation of free blocks.
I am aware that I could do this with a mutex (or some kind of spin lock) around each child_offset and potentially DCLP, but I would like to know if this is possible to implement efficiently with atomic operations.
The use case for this is as follows: I have a large tree that I'm building in parallel. The tree is an array of the following:
struct tree_page {
atomic<uint32_t> allocated;
uint32_t child_offset[8];
uint32_t nodes[1015];
};
The tree is built level by level: first the nodes at depth 0 are created, then at depth 1, etc. A separate thread is dispatched for each non-leaf node at the previous step. If no more space is left in a page, a new page is allocated from the global next_free_page which points to the first unused page in the array of struct tree_page and is assigned to an element of child_ptr. A bit field is then set in the node word that indicates which element of the child_ptr array should be used to find the node's children.
The code I am trying to write looks like this:
int expected = allocated.load(relaxed), updated;
do {
updated = expected + num_children;
if (updated > NODES_PER_PAGE) {
expected = -1; break;
}
} while (!allocated.compare_exchange_weak(&expected, updated));
if (expected != -1) {
// successfully allocated in the same page
} else {
for (int i = 0; i < 8; ++i) {
// this is the operation I would like to be atomic
if (child_offset[i] == 0)
child_offset[i] = next_free_block++;
int offset = try_allocating_at_page(pages[child_offset[i]]);
if (offset != -1) {
// successfully allocated at child_offset i
// ...
break;
}
}
}
As far as I understood from you description you array of child_offset is filled with 0 initially and then filled with some concrete values concurrently by different threads.
In this case you can atomically "tag" value first and if you are successful assign valid value. Something like this:
constexpr int INVALID_VALUE = -1;
for (int i = 0; i < 8; ++i) {
int expected = 0;
// this is the operation I would like to be atomic
if (child_offset[i].compare_exchange_weak(expected, INVALID_VALUE)) {
child_offset[i] = next_free_block++;
}
// Not sure if this is needed in your environment, but just in case
if (child_offset[i] == INVALID_VALUE) continue;
...
}
This doesn't guarantee that all values in child_offset array will be in ascending order. But if you need that why not fill it without multithreading involved?
I encounter deadlocks while executing the code snippet below as a thread.
void thread_lifecycle(
Queue<std::tuple<int64_t, int64_t, uint8_t>, QUEUE_SIZE>& query,
Queue<std::string, QUEUE_SIZE>& output_queue,
std::vector<Object>& pgs,
bool* pgs_executed, // Initialized to array false-values
std::mutex& pgs_executed_mutex,
std::atomic<uint32_t>& atomic_pgs_finished
){
bool iter_bool = false;
std::tuple<int64_t, int64_t, uint8_t> next_query;
std::string output = "";
int64_t lower, upper;
while(true) {
// Get next query
next_query = query.pop_front();
// Stop Condition reached terminate thread
if (std::get<2>(next_query) == uint8_t(-1)) break;
//Set query params
lower = std::get<0>(next_query);
upper = std::get<1>(next_query);
// Scan bool array
for (uint32_t i = 0; i < pgs.size(); i++){
// first lock for reading
pgs_executed_mutex.lock();
if (pgs_executed[i] == iter_bool) {
pgs_executed[i] = !pgs_executed[i];
// Unlock and execute the query
pgs_executed_mutex.unlock();
output = pgs.at(i).get_result(lower, upper);
// If query yielded a result, then add it to the output
if (output.length() != 0) {
output_queue.push_back(output);
}
// Inform main thread in case of last result
if (++atomic_pgs_finished >= pgs.size()) {
output_queue.push_back("LAST_RESULT_IDENTIFIER");
atomic_pgs_finished.exchange(0);
}
} else {
pgs_executed_mutex.unlock();
continue;
}
}
//finally flip for next query
iter_bool = !iter_bool;
}
}
Explained:
I have a vector of objects containing information which can be queried (similar to as a table in a database). Each thread can access the objects and all of them iterate the vector ONCE to query the objects which have not been queried and return results, if any.
In the next query it goes through the vector again, and so on... I use the bool* array to denote the entries which are currently queried, so that the processes can synchronize and determine which query should be executed next.
If all have been executed, the last thread having possibly the last results will also return an identifier for the main thread in order to inform that all objects have been queried.
My Question:
Regarding the bool* as well as atomic-pgs_finished, can there be a scenario, in-which a deadlock can occur. As far as i can think, i cannot see a deadlock in this snippet. However, executing this and running this for a while results into a deadlock.
I am seriously considering that a bit (byte?) has randomly flipped causing this deadlock (on ECC-RAM), so that 1 or more objects actually were not executed. Is this even possible?
Maybe another implementation could help?
Edit, Implementation of the Queue:
template<class T, size_t MaxQueueSize>
class Queue
{
std::condition_variable consumer_, producer_;
std::mutex mutex_;
using unique_lock = std::unique_lock<std::mutex>;
std::queue<T> queue_;
public:
template<class U>
void push_back(U&& item) {
unique_lock lock(mutex_);
while(MaxQueueSize == queue_.size())
producer_.wait(lock);
queue_.push(std::forward<U>(item));
consumer_.notify_one();
}
T pop_front() {
unique_lock lock(mutex_);
while(queue_.empty())
consumer_.wait(lock);
auto full = MaxQueueSize == queue_.size();
auto item = queue_.front();
queue_.pop();
if(full)
producer_.notify_all();
return item;
}
};
Thanks to #Ulrich Eckhardt (,
#PaulMcKenzie and all the other comments, thank you for the brainstorming!). I probably have found the cause of the deadlock. I tried to reduce this example even more and thought on removing atomic_pgs_finished, a variable indicating whether all pgs have been queried. Interestingly: ++atomic_pgs_finished >= pgs.size() returns not only once but multiple times true, so that multiple threads are in this specific if-clause.
I simply fixed it by using another mutex around this if-clause. Maybe someone can explain why ++atomic_pgs_finished >= pgs.size() is not atomic and causes true for multiple threads.
Below i have updated the code (mostly the same as in the question) with comments, so that it might be more understandable.
void thread_lifecycle(
Queue<std::tuple<int64_t, int64_t, uint8_t>, QUEUE_SIZE>& query, // The input queue containing queries, in my case triples
Queue<std::string, QUEUE_SIZE>& output_queue, // The Output Queue of results
std::vector<Object>& pgs, // Objects which should be queried
bool* pgs_executed, // Initialized to an array of false-values
std::mutex& pgs_executed_mutex, // a mutex, protecting pgs_executed
std::atomic<uint32_t>& atomic_pgs_finished // atomic counter to count how many have been executed (to send a end signal)
){
// Initialize variables
std::tuple<int64_t, int64_t, uint8_t> next_query;
std::string output = "";
int64_t lower, upper;
// Set the first iteration to false for the very first query
// This flips on the second iteration to reuse pgs_executed with true values and so on...
bool iter_bool = false;
// Execute as long as valid queries are received
while(true) {
// Get next query
next_query = query.pop_front();
// Stop Condition reached terminate thread
if (std::get<2>(next_query) == uint8_t(-1)) break;
// "Parse query" to query the objects in pgs
lower = std::get<0>(next_query);
upper = std::get<1>(next_query);
// Now iterate through the pgs and pgs_executed (once)
for (uint32_t i = 0; i < pgs.size(); i++){
// Lock to read and write into pgs_executed
pgs_executed_mutex.lock();
if (pgs_executed[i] == iter_bool) {
pgs_executed[i] = !pgs_executed[i];
// Unlock since we now execute the query on the object (which was not queried before)
pgs_executed_mutex.unlock();
// Query Execution
output = pgs.at(i).get_result(lower, upper);
// If the query yielded a result, then add it to the output for the main thread to read
if (output.length() != 0) {
output_queue.push_back(output);
}
// HERE THE ROOT CAUSE OF THE DEADLOCK HAPPENS
// Here i would like to inform the main thread that we exexuted the query on
// every object in pgs, so that it should no longer wait for other results
if (++atomic_pgs_finished >= pgs.size()) {
// In this if-clause multiple threads are present at once!
// This is not intended and causes a deadlock, push_back-ing
// multiple times "LAST_RESULT_IDENTIFIER" in-which the main-thread
// assumed that a query has finished. The main thread then simply added the next query, while the
// previous one was not finished causing threads to race each other on two queries simultaneously
// and not having the same iter_bool!
output_queue.push_back("LAST_RESULT_IDENTIFIER");
atomic_pgs_finished.exchange(0);
}
// END: HERE THE ROOT CAUSE OF THE DEADLOCK HAPPENS
} else {
// This case happens when the next element in the list was already executed (by another process),
// simply unlock pgs_executed and continue with the next element in pgs
pgs_executed_mutex.unlock();
continue; // This is uneccessary and could be removed
}
}
//finally flip for the next query in order to reuse bool* (which now has trues if a second query is incoming)
iter_bool = !iter_bool;
}
}
I am writing an EventManager class for a game I am making for school. It utilizes a double queue system for events and holds an unordered_map consisting of events types and vectors of ID's registered to respond to the particular events. The problem I am having is that certain events (like when you click the play button for the main menu)may cause an actor to unregister events. While this is intentional it causes problems with the iterators which is not intentional. In the example where it switches from the main menu to the game it destroys all the actors(buttons and such) that are part of the main menu. These, in turn, unregister themselves with the event manager which deletes their id from the corresponding vector stored in the unordered_map and invalidates the iterator throwing an exception at the beginning of the loop. As such no event can cause an actor to unregister anything. This is undesirable because some trigger objects in the game may be one time triggers at which point they need to unregister themselves from receiving more events. Any help or ideas would be greatly appreciated.
void EventManager::Flush()
{
Event* current;
while (m_eventList[m_current].size() > 0)
{
current = m_eventList[m_current].front();
m_eventList[m_current].pop_front();
unordered_map<string, vector<IDTYPE>>::iterator it = m_registeredEvents.find(current->GetType());
if (it == m_registeredEvents.end())
continue;
vector<IDTYPE>* toProcessAct = &(it->second);
vector<IDTYPE>::iterator actIt = toProcessAct->begin();
while (actIt != toProcessAct->end()) //this becomes invalid
{
Actor* temp = ACTORS->GetActor(*actIt);
if (temp == NULL)
actIt = toProcessAct->erase(actIt);
else
{
actIt++;
temp->Process(current); //Because this may unregister events
}
}
delete current; current = 0;
}
Swap();
}
void EventManager::UnregisterEvent(string Event, IDTYPE actor)
{
unordered_map<string, vector<IDTYPE>>::iterator it = m_registeredEvents.find(Event);
//Found
if (it != m_registeredEvents.end())
{
//Find if actor is already registered
vector<IDTYPE>::iterator actIt = it->second.begin();
while (actIt != it->second.end())
{
if (actor == *actIt)
{
it->second.erase(actIt);
return;
}
actIt++;
}
}
}
This is a good way to shoot yourself in the foot
Event* current; // not the issue here but you should always initialize your variables.
while (m_eventList[m_current].size() > 0) {
current = m_eventList[m_current].front(); // current points the the first element
m_eventList[m_current].pop_front(); // now the first element gets destroyed
unordered_map<string, vector<IDTYPE>>::iterator it = m_registeredEvents.find(current->GetType()); // its pure coincident if this works.
pop_front() destroys the front element making the pointer invalid and you enter undefined behaviour. Make a copy of the record instead, then you also don't have problems with destroying it.
Further all iterators to elements after the a delete are invalid, so if your code saves iterators as a state you will get hurt, just presume all deletes makes all iterators to that or related containers invalid to be safe.
If you are running a multi-threaded program you have the further problem that other threads could change the content, use at least a mutex to protect it.
OK, now for what you actually asked for:
while (actIt != toProcessAct->end()) { // end() should update but actIt doesn't.
Actor* temp = ACTORS->GetActor(*actIt);
if (temp == NULL)
actIt = toProcessAct->erase(actIt);
else {
actIt = temp->Process(actIt, current); // make Process return the new valid It
}
}
delete current; // would fail as there is a continue higher up!!!
current = 0; // if using C++11 use nullptr instead else NULL
Make Process return the new valid Iterator or toProcessAct->end() if no more are valid after actIt.
One solution can be to iterate over a copy of event handlers vector, that is instead of
vector<IDTYPE>* toProcessAct = &(it->second);
write
vector<IDTYPE> toProcessAct = it->second;
(and make corresponding changes for the code to compile).
Also probably it will be faster if you make a copy of the whole m_registeredEvents in the start of Flush(), so you won't have to copy vectors several times.
Another solution would be to make unregistration deferred, that is UnregisterEvent() instead of executing instantly would enqueue the unregistration request into separate queue and execute it later, for example at the end of Flush().
I have a thread which reads multicast data and updates certain data structures
and another thread which is handled by chai 3d library
when I just run my library code it works fine.
when I run y thread also
I get access violation in one of the routine inside the chai3d code.
my thread code
unsigned int __stdcall ThreadFunc(void* data)
{
char *timeOld;
int ID;
while(1)
{
char *position = _com_util::ConvertBSTRToString(cpi->getData());
ID = cpi->getMulticastDataID();
char* timeNew = _com_util::ConvertBSTRToString(cpi->getTime());
if(timeFirst == true)
{
timeOld = new char[strlen(timeNew) + 1];
strcpy(timeOld,timeNew);
timeFirst = false;
}
if((strcmp(timeNew,timeOld) != 0) && (AddItselToList == true) && ( ID != 99))
{handlePacket(position,ID);
strcpy(timeOld,timeNew);}
delete[] position;
delete[] timeNew;
}
delete[] timeOld;
}
cpi is a pointer to com c# object where getdata, gettime return strings and getmulticastid returns int.
is there something worn with my thread code?
_beginthreadex(NULL,0,ThreadFunc,NULL,0,NULL);
delete[] position and timeNew before the closing brace of the while loop. delete[] timeOld before the closing brace of ThreadFunc. Also, are you sure timeOld is long enough to copy timeNew into it? This may explain your access violation.
I am working on multiple producer and Single consumer problem.I wanted to pass Thread like 1,2,3 in the thread function so that individual thread can be named based on these number.
But the program is crashing after count 7 while creating thread.I think problem is due to
variable nThreadNo;
if i limit the count less than 7 it works fine.but if i make count more than this it crashes.
void CEvent1Dlg::CreateProducerThreads()
{
try
{
nThreadNo = new int20];
memset(nThreadNo,0,20);
if (nThreadNo ==NULL) return;
}catch(...)
{
MessageBox(_T("Memory allocation Failed"),_T("Thread"),1);
return ;
}
int i = 0;
for ( i = 0;i<20;i++)
{
//nThreadNo = i+1;
nThreadNo[i] = i+1;
hWndProducer[i] = CreateThread(NULL,0,(LPTHREAD_START_ROUTINE)ProducerThrdFunc,(void*)(nThreadNo+i),0,&dwProducerThreadID[i]);
if (hWndProducer[i] == NULL)
{
//ErrorHandler(TEXT("CreateThread"));
ExitProcess(3);
}
}
//WaitForMultipleObjects(20,hWndProducer,TRUE,INFINITE);
}
DWORD WINAPI ProducerThrdFunc ( LPVOID n )
{
int *nThreadNo = (int*)n;
char chThreadNo[33];
memset(chThreadNo,0,33);
while(1)
{
itoa(*nThreadNo,chThreadNo,10);
char* pMsg1 = new char[100];
char* pMsg2 = new char[100];
memset(pMsg1,0,100);
memset(pMsg2,0,100);
strcpy(pMsg1,"Producer ");
strcat(pMsg1," Thread No:");
strcat(pMsg1,chThreadNo);
if (stThreadInfoProd.pEventQueue->AddTail(pMsg1)==TRUE)
{
strcpy(pMsg2,"Producer ");
strcat(pMsg2," Thread No:");
strcat(pMsg2,chThreadNo);
strcat(pMsg2," Added the Msg");
}
else
{
strcpy(pMsg2,"Producer ");
strcat(pMsg2," Thread No:");
strcat(pMsg2,chThreadNo);
strcat(pMsg2,"failed to Add the Msg");
}
PostMessage(stThreadInfoProd.hWndHandle,UWM_ONUPDATEPRODUCERLIST,(WPARAM)pMsg2,0);
strcat(pMsg1," Adding Msg:");
//PostMessage(stThreadInfoProd.hWndHandle,UWM_ONUPDATEPRODUCERLIST,(WPARAM)pMsg2,0);
Sleep(3000);
}
return 0;
}
You are zeroing out the first 20 bytes of nThreadNo, not the first 20 * sizeof(int) bytes as you should be doing.
There are other arrays you are indexing into in this code: hWndProducer, dwProducerThreadID. Are there enough elements in those as well?
The CreateThread call is passing the integer value, but the thread function itself is treating it as a pointer to integer. Rather than this:
int *nThreadNo = (int*)n;
It should probably be:
int nThreadNo = (int)n;
Edit: I looked more closely at the call and I do see that it is passing an integer pointer. However, that value is stack data, which may not exist by the time the thread tries to read it. So it should probably just pass the integer value: (void*)(nThreadNo[i])
This line
if (nThreadNo ==NULL) return;
is worthless.
The new operator in modern C++ doesn't return NULL on failure, it throws a std::badalloc exception, but even if you were using an allocator that returns NULL to indicate failure, it's too late to detect it, you're already passed the NULL pointer to memcpy.