I'm trying to implement some sort of waiting on many CONDITION_VARIABLE.
The answers here imply that WaitForMultipleObjects and such are valid options when dealing with Windows API (and many more places over the internet), but it appears that it is not the case.
first of all, nowhere in the MSDN documentation it is written that a Windows Condition variable is a valid argument for WaitFor... functions.
Second of all, it appears that WaitFor... only accepts HANDLE type as argument, which is basically a kernel object. but PCONDITION_VARIABLE is not really a HANDLE.
finally, trying to use a condition variable (both as a PCONDITION_VARIABLE and the undocumented CONDITION_VARIABLE::Ptr) makes the functions return error code 6 (invalid handle)
for example:
CONDITION_VARIABLE cv;
InitializeConditionVariable(&cv);
auto res = WaitForSingleObject(cv.Ptr, INFINITE); //returns immediately
if (res != WAIT_OBJECT_0) {
auto ec = GetLastError();
std::cout << ec << "\n";
}
so, can you really wait on a condition variable or it's just an urban legend?
I don't think so and it doesn't make any sense.
First of all, the WaitForXxx functions operate (mostly) on dispatcher objects - a subset of kernel objects including timers, events, mutexes, sempahores, threads and process (and a few internal object types like KAGTEs and KQUEUEs, but not access tokens or file mapping objects) that have a DISPATCHER_HEADER. It certainly won't work on user mode constructs that the kernel is unaware of.
Second, note that when you sleep ("wait") on a condition variable you have to specify whether this is critical section-based condition variable or a SRWL-based condition variable by using the correct function - either SleepConditionVariableCS or SleepConditionVariableSRW. So again, Windows (not only the kernel) has no idea what kind of condition variable you're passing it, but it needs this information to operate correctly. Since you don't provide this information to WaitForXxx it follows that they cannot be used with condition variables.
The simple answer to your question is no. You cannot use the WaitForXxx functions with the condition variables provided by the Windows synchronization APIs. From the linked documentation:
Condition variables are synchronization primitives that enable threads to wait until a particular condition occurs. Condition variables are user-mode objects that cannot be shared across processes.
The WaitForXxx functions accept parameters of the generic HANDLE type, which represents a handle to a kernel object. Condition variables are user-mode objects, not kernel objects, so you cannot use them with these functions, since they work only with kernel objects.
Moreover, the documentation for these functions is pretty explicit about which types of objects they can wait on, and condition variables are not on that list. For instance, WaitForMultipleObjects says:
The WaitForMultipleObjects function can specify handles of any of the following object types in the lpHandles array:
Change notification
Console input
Event
Memory resource notification
Mutex
Process
Semaphore
Thread
Waitable timer
They all have the same list, so no confusion there.
Technically speaking (and we're diving into undocumented implementation details here, so you shouldn't rely on this as gospel), the Win32 WaitForSingleObject and WaitForMultipleObjects functions are built upon the KeWaitForSingleObject and KeWaitForMultipleObjects functions provided by the kernel subsystem. You can divide the objects supported by the kernel into three basic categories: dispatcher objects, I/O objects/data structures, and everything else. The first category, dispatcher objects, are the lowest level objects and they are all represented using the same DISPATCHER_HEADER data structure in their bodies. Dispatcher objects are the only types of objects that are "waitable". It is this DISPATCHER_HEADER structure that makes an object waitable, by definition. If the object is represented using this data structure, then it can be passed to the kernel synchronization functions. Thus, the same rules would apply to the Win32 functions.
This entire question seems to be based around a single statement that Managu makes in his answer: "Windows has WaitForMultipleObjects as aJ posted, which could be a solution if you're willing to restrict your code to Windows synchronization primitives." Perhaps he doesn't consider condition variables (as they are implemented by Windows) to be synchronization primitives, or perhaps he is just wrong. aJ's answer, to which he refers, is pretty clear about stating that WaitForMultipleObjects is used "to wait for multiple kernel objects," and we have already established that condition variables are not kernel objects. Either way, I don't see any evidence for an "urban legend" that you can do this.
Obviously you cannot use the WaitForXxx family of functions with boost::condition_variable, or std::condition_variable, or anything else. I'm sure you already knew that, but your question has confused some people because it links to a question that refers to the Boost implementation.
It is not especially clear to me why you would need to wait on multiple condition variables simultaneously. I guess you could write your own implementation of condition variables, based on the classic Win32 synchronization primitives, such as mutexes, which you can then wait on with WaitForMultipleObjects. You can probably find examples of such code online, since condition variables did not become part of the operating system until Vista. For example, this article discusses strategies for implementing condition variables in Windows as they are defined by the POSIX Pthreads specification. You could also look into using Event Objects.
Related
I am writing a piece C++ code for Windows that needs to query the timings for the process and each of its individual threads.
To do the necessary system calls, I need the handles for the process and each of its threads. I am using the getCurrentProcess and getCurrentThread functions which both return a pseudo handle. Upon further inspection, I noticed that the pseudo handles for all threads share the same value.
After a brief search on the internet, I found the following article that reports the same values for the process and thread pseudo handles as I got: https://weseetips.wordpress.com/2008/03/26/getcurrentthread-returns-pseudo-handle-not-the-real-handle/
My question: is it safe and/or defined to call getCurrentThread once from one thread and use the returned pseudo handle in all other threads to let them refer to themselves?
Using the current implementation, this works as expected. I am just wondering if this behavior is guaranteed. In other words, will it work on any Windows platform that offers the getCurrentThread function; and would changing the behavior be considered a breaking change?
The documentation for the getCurrentThread function states (emphasis mine):
A pseudo handle is a special constant that is interpreted as the
current thread handle. The calling thread can use this handle to
specify itself whenever a thread handle is required. [...]
Which makes me believe this special pseudo handle is just an alias for "the current thread" and can therefore be shared among all threads to let them refer to themselves. On the other hand, the documentation also says that the return value can be used by the calling thread, hence my confusion!
yes, this is safe - this pseudo handles is well known values and documented in wdm.h (from windows wdk)
#define NtCurrentProcess() ( (HANDLE)(LONG_PTR) -1 )
#define ZwCurrentProcess() NtCurrentProcess()
#define NtCurrentThread() ( (HANDLE)(LONG_PTR) -2 )
#define ZwCurrentThread() NtCurrentThread()
#define NtCurrentSession() ( (HANDLE)(LONG_PTR) -3 )
#define ZwCurrentSession() NtCurrentSession()
and can therefore be shared among all threads to let them refer to
themselves.
of course no, it can not be "shared among all threads" - only current process/thread can use it for refer to themselves.
when kernel mode api got thread/process handle as input parameter - he need convert handle to object pointer (ETHREAD or EPROCESS). for this first check for this constant values - if yes - used pointer to current thread/process object. otherwise handle is index in process handle table
Yep. From that same page:
The function cannot be used by one thread to create a handle that can be used by other threads to refer to the first thread. The handle is always interpreted as referring to the thread that is using it. A thread can create a "real" handle to itself that can be used by other threads, or inherited by other processes, by specifying the pseudo handle as the source handle in a call to the DuplicateHandle function.
I have in a Server object multiple thread who are doing the same task. Those threads are init with a Server::* routine.
In this routine there is a infinite loop with some treatments.
I was wondering if it was thread safe to use the same method for multiple threads ? No wonder for the fields of the class, If I want to read or write it I will use a mutex. But what about the routine itself ?
Since a function is an address, those thread will be running in the same memory zone ?
Do I need to create a method with same code for every thread ?
Ps: I use std::mutex(&Server::Task, this)
There is no problem with two threads running the same function at the same time (whether it's a member function or not).
In terms of instructions, it's similar to if you had two threads reading the same field at the same time - that's fine, they both get the same value. It's when you have one writing and one reading, or two writing, that you can start to have race conditions.
In C++ every thread is allocated its own call stack. This means that all local variables which exist only in the scope of a given thread's call stack belong to that thread alone. However, in the case of shared data or resources, such as a global data structure or a database, it is possible for different threads to access these at the same time. One solution to this synchronization problem is to use std::mutex, which you are already doing.
While the function itself might be the same address in memory in terms of its place in the table you aren't writing to it from multiple locations, the function itself is immutable and local variables scoped inside that function will be stacked per thread.
If your writes are protected and the fetches don't pull stale data you're as safe as you could possibly need on most architectures and implementations out there.
Behind the scenes, int Server::Task(std::string arg) is very similar to int Server__Task(Server* this, std::string arg). Just like multiple threads can execute the same function, multiple threads can also execute the same member function - even with the same arguments.
A mutex ensures that no conflicting changes are made, and that each thread sees every prior change. But since code does not chance, you don't need a mutex for it, just like you don't need a mutex for string literals.
I am currently running into some design problems regarding concurrent programming in C++
and I was wondering if you could help me out:
Assume that some function func operates on some object obj. It is necessary during these operations to hold a lock (which might be a member variable of obj). Now assume that
func calls a subfunction func_2 while it holds the lock. Now func_2 operates on an object which is already locked. However, what if I also want to call func_2 from somewhere else without holding the lock? Should func_2 lock obj or should it not? I see 3 possibilites:
I could pass a bool to func_2 indicating whether ot not locking is required.
This seems to introduce a lot of boilerplate code though.
I could use a recursive lock and just always lock obj in func_2. Recursive locks
seem to
be problematic though, see here.
I could assume that every caller of func_2 holds the lock already. I would have
to document this and perhaps enforce this (at least in debugging mode). Is
it reasonable to have functions make assumptions regarding which locks are / are not
held by the calling thread? More generally, how do I decide from a design perspective
whether a function should lock Obj and which should assume that it is already locked?
(Obviously if a function assumes that certain locks are held then it can only call
functions which make at least equally strong assumptions but apart from that?)
My question is the following: Which one of these approaches is used in practice and why?
Thanks in advance
hfhc2
1. Passing an indicator whether to lock or not:
You give the the lock choice to the caller. This is error prone:
the caller might not do the right choice
the caller needs to know implementation details about your object, thus breaking the principle of encapsulation
the caller needs access to the mutex
If you have several objects, you eventually facilitate conditions for deadlocks
2. recursive lock:
You already highlighted the issue.
3. Pass locking responsibility to caller:
Among the different alternatives that you propose, this seems the most consistent. On contrary of 1, ou don't give the choice, but you pass complete responsibility for locking. It's part of the contract for using func_2.
You could even assert if a lock is set on the object, to prevent mistakes (although teh check wold be limited because you would not necessarily be in position to verivy who owns the lock).
4.Reconsider your design:
If you need to ensure in func_2 that the object is locked, it means that you have a critical section therein that you must protect. There are chances that both functions need to lock because they perform some lower level operations on obj and need to prevent data races on an instable state of the object.
I'd strongly advidse to look if it would be feasible to extract these lower-level routines from both func and func_2, and encapsulated them in simpler primitive functions on obj. This approach could also contribute to locking for shorter sequences thus increasing opportunity for real concurrency.
Ok, just as another follow-up. I recently read the API documentation of glib, in particular the section about message-passing queues. I found that most functions operating on these queues come in two variants, named function and function_unlocked. The idea is that if a programmer wants to execute a single operation, like popping from the queue this can be done using g_async_queue_pop(). The function automatically takes care of the locking/unlocking of the queue. However, if the programmer wants to for instance pop two elements without interruption, the following sequence may be used:
GAsyncQueue *queue = g_async_queue_new();
// ...
g_async_queue_lock(queue);
g_async_queue_pop_unlocked(queue);
g_async_queue_pop_unlocked(queue);
g_async_queue_unlock(queue);
This resembles my third approach. It is also the case that assumptions regarding the state of certain locks are made, they are required by the API and required to be documented.
What is the difference between event objects and condition variables?
I am asking in context of WIN32 API.
Event objects are kernel-level objects. They can be shared across process boundaries, and are supported on all Windows OS versions. They can be used as their own standalone locks to shared resources, if desired. Since they are kernel objects, the OS has limitations on the number of available events that can be allocated at a time.
Condition Variables are user-level objects. They cannot be shared across process boundaries, and are only supported on Vista/2008 and later. They do not act as their own locks, but require a separate lock to be associated with them, such as a critical section. Since they are user- objects, the number of available variables is limited by available memory. When a Conditional Variable is put to sleep, it automatically releases the specified lock object so another thread can acquire it. When the Conditional Variable wakes up, it automatically re-acquires the specified lock object again.
In terms of functionality, think of a Conditional Variable as a logical combination of two objects working together - a keyed event and a lock object. When the Condition Variable is put to sleep, it resets the event, releases the lock, waits for the event to be signaled, and then re-acquires the lock. For instance, if you use a critical section as the lock object, SleepConditionalVariableCS() is similar to a sequence of calls to ResetEvent(), LeaveCriticalSection(), WaitForSingleObject(), and EnterCriticalSection(). Whereas if you use a SRWL as the lock, SleepConditionVariableSRW() is similar to a sequence of calls to ResetEvent(), ReleaseSRWLock...(), WaitForSingleObject(), and AcquireSRWLock...().
They are very similar, but event objects work across process boundaries, whereas condition variables do not. From the MSDN documentation on condition variables:
Condition variables are user-mode
objects that cannot be shared across
processes.
From the MSDN documentation on event objects:
Threads in other processes can open a
handle to an existing event object by
specifying its name in a call to the
OpenEvent function.
The most significant difference is the Event object is a kernel object and can be shared across processes as long as it is alive when processes/threads are trying to acquire, on the contrary, Condition variable is a user mode object which is light(only has same size as a pointer and has nothing additional to be released after using it) and has better performance.
Typically, condition variable is often used along with locks, since we need to keep data synchronized properly. When considering Condition Variable, we treat it like keyed events which was improved since Vista.
Joe duffy has a blog post http://joeduffyblog.com/2006/11/28/windows-keyed-events-critical-sections-and-new-vista-synchronization-features/ that explained more detailed information.
Starting with pthreads, I cannot understand what is the business with pthread_key_t and pthread_once_t?
Would someone explain in simple terms with examples, if possible?
thanks
pthread_key_t is for creating thread thread-local storage: each thread gets its own copy of a data variable, instead of all threads sharing a global (or function-static, class-static) variable. The TLS is indexed by a key. See pthread_getspecific et al for more details.
pthread_once_t is a control for executing a function only once with pthread_once. Suppose you have to call an initialization routine, but you must only call that routine once. Furthermore, the point at which you must call it is after you've already started up multiple threads. One way to do this would be to use pthread_once(), which guarantees that your routine will only be called once, no matter how many threads try to call it at once, so long as you use the same control variable in each call. It's often easier to use pthread_once() than it is to use other alternatives.
No, it can't be explained in layman terms. Laymen cannot successfully program with pthreads in C++. It takes a specialist known as a "computer programmer" :-)
pthread_once_t is a little bit of storage which pthread_once must access in order to ensure that it does what it says on the tin. Each once control will allow an init routine to be called once, and once only, no matter how many times it is called from how many threads, possibly concurrently. Normally you use a different once control for each object you're planning to initialise on demand in a thread-safe way. You can think of it in effect as an integer which is accessed atomically as a flag whether a thread has been selected to do the init. But since pthread_once is blocking, I guess there's allowed to be a bit more to it than that if the implementation can cram in a synchronisation primitive too (the only time I ever implemented pthread_once, I couldn't, so the once control took any of 3 states (start, initialising, finished). But then I couldn't change the kernel. Unusual situation).
pthread_key_t is like an index for accessing thread-local storage. You can think of each thread as having a map from keys to values. When you add a new entry to TLS, pthread_key_create chooses a key for it and writes that key into the location you specify. You then use that key from any thread, whenever you want to set or retrieve the value of that TLS item for the current thread. The reason TLS gives you a key instead of letting you choose one, is so that unrelated libraries can use TLS, without having to co-operate to avoid both using the same value and trashing each others' TLS data. The pthread library might for example keep a global counter, and assign key 0 for the first time pthread_key_create is called, 1 for the second, and so on.
Wow, the other answers here are way too verbose.
pthread_once_t stores state for pthread_once(). Calling pthread_once(&s, fn) calls fn and sets the value pointed to by s to record the fact it has been executed. All subsequent calls to pthread_once() are noops. The name should become obvious now.
pthread_once_t should be initialized to PTHREAD_ONCE_INIT.