Only the owner of mutex can unlock it. But how mutex distinguish thread that locked it?
Does threads have any distinctive features in Linux?
You can look at the implementation source code for details (the pthread implementation from the GNU libc Git repository can be browsed here), but they have different IDs that are used internally. You can see this at the application level using pthread_self(). It returns a pthread_t value that is unique on a per-thread basis within a given process. There is no guarantee of uniqueness when you compare pthread_t values from different processes.
The actual type that pthread_t corresponds to is implementation-defined, however; it could be an arithmetic (e.g. integral) type, or it could be a structure. Therefore, you can't really do much with them in a portable way, other than compare them for equality using pthread_equal().
They are differentiated using the thread id;
Related
Is there a sane way to map a pthread_t value (as returned from pthread_create() or std::thread::native_hanle() ) to pid(tid) in Linux? Before someone gets duplicate-happy, this is not about finding thread's own pid (which can be done with gettid()).
The insane way would be to somehow compel a thread to call gettid() and pass along the result, but that's way too much trouble.
One of the possible applications I have in mind is to reconcile threads created within program (where pthread_t is available) with output provided by ps -T.
One (convoluted, non-portable, Linux-specific, lightly destructive) method of mapping pthread_t to tid without looking into struct pthread is as follows:
Use pthread_setname_np to set a thread name to something unique.
Iterate over subdirectories of /proc/self/task and read a line from a file named comm in each of those.
If the line equals to the unique string just used, extract tid from the last component of the subdirectory name. This is your answer.
The thread name is not used by the OS for anything, so it should be safe to change it. Nevertheless you probably want to set it back to the value it had originally (use pthread_getname_np to obtain it).
A somewhat hacky way of doing this on Linux would be with the process_vm_readv system call. If you know the pthread_t, use it to copy the memory over from the remote process to a local buffer, then cast the pointer to a struct pthread and retrieve the value of the tid field.
Pthreads are POSIX Threads.
In pthread_t is a typedef to some type of long depending on your architecture.
It is actually a pointer typecasted to an internal struct pthread as mentioned here above.
It is on purpose that the struct pthread is not returned.
Threading is highly dependent on the underlying OS. Threads are not implemented equally on all Unix flavored OS'es.
I don't believe even that gettid is a POSIX function, I believe it is Linux specific.
You can have a look at the glibc/nptl source code for linux specific implementation of struct pthread
See https://sourceware.org/git/?p=glibc.git;a=blob;f=nptl/descr.h;h=fdeb397eab94730a5dab3181abcdae815ed6914e;hb=48a8f8328122ab8d06b7333cb87be46feeaf7cca
But I believe what you are looking for is getpid. It is the process id and not thread id
I'm trying to implement some sort of waiting on many CONDITION_VARIABLE.
The answers here imply that WaitForMultipleObjects and such are valid options when dealing with Windows API (and many more places over the internet), but it appears that it is not the case.
first of all, nowhere in the MSDN documentation it is written that a Windows Condition variable is a valid argument for WaitFor... functions.
Second of all, it appears that WaitFor... only accepts HANDLE type as argument, which is basically a kernel object. but PCONDITION_VARIABLE is not really a HANDLE.
finally, trying to use a condition variable (both as a PCONDITION_VARIABLE and the undocumented CONDITION_VARIABLE::Ptr) makes the functions return error code 6 (invalid handle)
for example:
CONDITION_VARIABLE cv;
InitializeConditionVariable(&cv);
auto res = WaitForSingleObject(cv.Ptr, INFINITE); //returns immediately
if (res != WAIT_OBJECT_0) {
auto ec = GetLastError();
std::cout << ec << "\n";
}
so, can you really wait on a condition variable or it's just an urban legend?
I don't think so and it doesn't make any sense.
First of all, the WaitForXxx functions operate (mostly) on dispatcher objects - a subset of kernel objects including timers, events, mutexes, sempahores, threads and process (and a few internal object types like KAGTEs and KQUEUEs, but not access tokens or file mapping objects) that have a DISPATCHER_HEADER. It certainly won't work on user mode constructs that the kernel is unaware of.
Second, note that when you sleep ("wait") on a condition variable you have to specify whether this is critical section-based condition variable or a SRWL-based condition variable by using the correct function - either SleepConditionVariableCS or SleepConditionVariableSRW. So again, Windows (not only the kernel) has no idea what kind of condition variable you're passing it, but it needs this information to operate correctly. Since you don't provide this information to WaitForXxx it follows that they cannot be used with condition variables.
The simple answer to your question is no. You cannot use the WaitForXxx functions with the condition variables provided by the Windows synchronization APIs. From the linked documentation:
Condition variables are synchronization primitives that enable threads to wait until a particular condition occurs. Condition variables are user-mode objects that cannot be shared across processes.
The WaitForXxx functions accept parameters of the generic HANDLE type, which represents a handle to a kernel object. Condition variables are user-mode objects, not kernel objects, so you cannot use them with these functions, since they work only with kernel objects.
Moreover, the documentation for these functions is pretty explicit about which types of objects they can wait on, and condition variables are not on that list. For instance, WaitForMultipleObjects says:
The WaitForMultipleObjects function can specify handles of any of the following object types in the lpHandles array:
Change notification
Console input
Event
Memory resource notification
Mutex
Process
Semaphore
Thread
Waitable timer
They all have the same list, so no confusion there.
Technically speaking (and we're diving into undocumented implementation details here, so you shouldn't rely on this as gospel), the Win32 WaitForSingleObject and WaitForMultipleObjects functions are built upon the KeWaitForSingleObject and KeWaitForMultipleObjects functions provided by the kernel subsystem. You can divide the objects supported by the kernel into three basic categories: dispatcher objects, I/O objects/data structures, and everything else. The first category, dispatcher objects, are the lowest level objects and they are all represented using the same DISPATCHER_HEADER data structure in their bodies. Dispatcher objects are the only types of objects that are "waitable". It is this DISPATCHER_HEADER structure that makes an object waitable, by definition. If the object is represented using this data structure, then it can be passed to the kernel synchronization functions. Thus, the same rules would apply to the Win32 functions.
This entire question seems to be based around a single statement that Managu makes in his answer: "Windows has WaitForMultipleObjects as aJ posted, which could be a solution if you're willing to restrict your code to Windows synchronization primitives." Perhaps he doesn't consider condition variables (as they are implemented by Windows) to be synchronization primitives, or perhaps he is just wrong. aJ's answer, to which he refers, is pretty clear about stating that WaitForMultipleObjects is used "to wait for multiple kernel objects," and we have already established that condition variables are not kernel objects. Either way, I don't see any evidence for an "urban legend" that you can do this.
Obviously you cannot use the WaitForXxx family of functions with boost::condition_variable, or std::condition_variable, or anything else. I'm sure you already knew that, but your question has confused some people because it links to a question that refers to the Boost implementation.
It is not especially clear to me why you would need to wait on multiple condition variables simultaneously. I guess you could write your own implementation of condition variables, based on the classic Win32 synchronization primitives, such as mutexes, which you can then wait on with WaitForMultipleObjects. You can probably find examples of such code online, since condition variables did not become part of the operating system until Vista. For example, this article discusses strategies for implementing condition variables in Windows as they are defined by the POSIX Pthreads specification. You could also look into using Event Objects.
std::thread::get_id() gives you an implementation defined value which uniquely identifies a given thread, but the interesting thing for me is that there is a dedicated type for this, thread::id, is this type used anywhere in the standard library ?
A thread::id is used somewhere or in any interface that you know of ? AFAIK this type is used nowhere hence it looks like it's quite useless at the moment .
The purpose of such user defined types is to make things easy for implementors.
In many cases, you will be implementing C++ threads on top of existing code bases, OS systems, or the like. These may have different types of thread identification.
With that type, the C++ std implementation is more likely to be able to expose the underlying thread identification value directly, or with minimal modification.
Knowing what thread you are on is quite useful in many situations from the client side, and implementing it without an id from the system is complex.
std::thread::id can be sorted, compared (in a totally ordered way, with sensible equality) and std::hashed, all of which are useful with the std library. They can be copied (trivially) and constructed with no arguments (for the id that represents no thread). They can be turned into a string via ostream <<, with the only guarantee that the resulting string is never the same except for two == ids.
Any operations on them beyond that is undefined. But, an implementation could make thread_id be basically a pointer, or an unsigned integer index in an array, or one of many different underlying implementations. An implementation who accesses such information is doing something completely implementation dependent, however.
Is thread::id used anywhere in the standard C++ library?
No, thread::id is not used in the interface of the standard C++ library.
It might be used in the implementation of one of the recursive mutexes, but that would be an implementation detail. I do not know if any implementation currently uses it.
AFAIK this type is used nowhere hence it looks like it's quite useless
at the moment.
Here are a few of the other types in the std::library that are "useless" by this definition:
list
set
multiset
map
multimap
unordered_set
unordered_multimap
array
atomic
bitset
complex
condition_variable
condition_variable_any
forward_list
fstream
reverse_iterator
move_iterator
mutex
queue
stack
regex
thread
This is not an exhaustive list.
Somewhat reluctant update
Me:
Is your question: What is a motivating use case for thread::id?
user2485710:
yes, that sounds right, with, maybe, a special focus on the standard library.
thread::id is sometimes used to map a thread to attributes or vice-versa. For example one might implement "named threads" by associating a std::string with a std::thread::id in a std::map. When a thread of execution logged, or throws an exception, or has some notable event, one could look up the name of the thread to create a message for the log, error message, etc, in order to give better context. For example threads might have suggestive names such as: "database server" or "table updater".
thread::id is more convenient to use than thread for this application as thread is usually needed elsewhere to control joining.
Another use for thread::id is to detect if the current thread executing a function is the same thread as the last thread that executed that same function. I've seen this technique used in the implementation of recursive_mutex::lock(). For example, if the mutex is locked and this_thread::get_id() == stored_id, then increment lock count.
As far as "focus on the standard C++ library" is concerned, I really don't know what that means. If it means: "Used in the interface", then this question has already been answered earlier in this answer, and in other answers:
No, thread::id is not used in the interface of the standard C++ library.
There are many, many types in the std::lib that are not part of the API of other parts of the std::lib. thread::id was not standardized because it was needed in the API of other parts of the library. It was standardized because std::thread was being standardized, and thread::id is a natural part of the std::thread library, and because it is useful for the use cases such as those mentioned above.
The key difference between thread and thread::id is that thread maintains unique ownership of a thread of execution. Thus thread is a move-only type. This is very analogous to unique_ptr. Only one std::thread can be used to join(). In contrast, a thread::id is just a "name" for a thread. Names are copyable and comparable. They aren't used for ownership, only for identification.
This separation of concerns (privilege-to-join vs identification) is made more obvious in a language which supports both move-only and copyable types.
AFAIK this type is used nowhere hence it looks like it's quite useless at the moment .
A thread::id implements relational operators and hash support. This allows you, the user, to have them as keys in associative and unordered containers.
Simple edition: In a C++ program I'm using two different threads to work with some integer variable. but I'm sure one is always writing some value into it and the other one is only reading That. do I still need to use mutex lock when reading/writing data?
Now details : The main idea is that first thread generates some information and saves them into an array, and the second thread reads data from that array and process them. this array represents a queue. meaning I have two index values pointing to the first and last item in queue. Now I'm wondering if I have to lock these two index values whenever I'm reading or writing values or is it ok to check them without locking? note that generator thread is the only thread changing index of queue_back, and processor thread has exclusive permission to change queue_front.
if makes any change I'm developing for a linux based system and the code is compiled using gcc.
PS: In some codes which use threading, I've seen keyword volatile around variables shared between different threads, do I need to use that too?
No the read and write are not atomic, You will need to synchronize it using some synchronization mechanism.
Also, You MUST mark the shared integer as volatile, otherwise the optimizer might think the variable is never updated in one of your threads.
gcc allows you to do atomic operations on int‘s, long‘s and long long‘s (and their unsigned counterparts).
Look up for the functions:
type __sync_fetch_and_add (type *ptr, type value);
type __sync_fetch_and_sub (type *ptr, type value);
type __sync_fetch_and_or (type *ptr, type value);
type __sync_fetch_and_and (type *ptr, type value);
type __sync_fetch_and_xor (type *ptr, type value);
type __sync_fetch_and_nand (type *ptr, type value);
Do I still need to use mutex lock when reading/writing data?
Yes, you will need a lock. You may be interested in a more specific implementation called a read/write lock.
You can also use atomics and/or memory barriers. Using these will require a better understanding of your target architectures. Reproducing multithreading bugs can be very difficult, and these alternatives should be considered an optimization which may not be portable.
I've seen keyword volatile around variables shared between different threads, do I need to use that too?
Yikes. No! That is not a safe or portable solution to multithreaded reads and writes in C++. Use atomics, locks, copying, immutable and pure implementations (etc...) instead.
Interpretation of volatile can vary by platform and/or compiler, and it's not specified to operate any particular way in C or C++ for the purpose of multithreaded reads and writes (there's an old false legend that it can be used reliably as an atomic read/write). I once tested the effectiveness of volatile in a multithreaded C++ program for fun (on an intel-mac with apple's gcc). I won't provide the results because it worked well enough that some people might consider using it, although they should not because 'almost' isn't good enough.
And to qualify the use of volatile: It exists in my (large, strictly written, multithreading aware) codebase for the sole purpose of interfacing with platform dependent atomics APIs. And being entirely honest: there are a few other uses from earlier days, but they can and should be removed.
Yes, you need to synchronize access to the variable, either with a mutex, a critical section, interlocked access, etc to make sure that the reading thread does not read incomplete bytes while the writing thread is still saving them. This is especially important on multi-core/CPU systems, where the two threads can truely access the variable in parallel.
Reads and writes of properly aligned data no larger than a machine word (usually whatever int resolves to) are atomic on most major architectures. That does not mean every architecture.
This means that no, you cannot just read head and tail and expect that data is consistent. However, if for example sizeof(int) happens to be 4 and sizeof(short) happens to be 2, and if you don't care about "not mainstream" platforms, you can do some union trickery and get away without atomic operations or a mutex.
If you want your code to be portable, there is no way around proper locking or atomic compare/exchange.
About volatile, this does insert a memory barrier for Microsoft Visual C++ (as a compiler-specific sophistry) but the standard does not guarantee anything special other than that the compiler won't optimize the variable. Insofar, making something volatile does not really help much, and it guarantees thread safety in no way.
Yes, you need to protect the queue indexes in both the generator and the reader threads by using some kind of synchronization mechanism, like a mutex.
HTH
If you're really just sharing one single integer, then std::atomic<int> sounds like the right type to use, from the <atomic> header. (There should also be Boost or TR1 versions if you have an old compiler.) This ensures atomic reads and writes. There's no need for a volatile qualifier as far as I understand.
Starting with pthreads, I cannot understand what is the business with pthread_key_t and pthread_once_t?
Would someone explain in simple terms with examples, if possible?
thanks
pthread_key_t is for creating thread thread-local storage: each thread gets its own copy of a data variable, instead of all threads sharing a global (or function-static, class-static) variable. The TLS is indexed by a key. See pthread_getspecific et al for more details.
pthread_once_t is a control for executing a function only once with pthread_once. Suppose you have to call an initialization routine, but you must only call that routine once. Furthermore, the point at which you must call it is after you've already started up multiple threads. One way to do this would be to use pthread_once(), which guarantees that your routine will only be called once, no matter how many threads try to call it at once, so long as you use the same control variable in each call. It's often easier to use pthread_once() than it is to use other alternatives.
No, it can't be explained in layman terms. Laymen cannot successfully program with pthreads in C++. It takes a specialist known as a "computer programmer" :-)
pthread_once_t is a little bit of storage which pthread_once must access in order to ensure that it does what it says on the tin. Each once control will allow an init routine to be called once, and once only, no matter how many times it is called from how many threads, possibly concurrently. Normally you use a different once control for each object you're planning to initialise on demand in a thread-safe way. You can think of it in effect as an integer which is accessed atomically as a flag whether a thread has been selected to do the init. But since pthread_once is blocking, I guess there's allowed to be a bit more to it than that if the implementation can cram in a synchronisation primitive too (the only time I ever implemented pthread_once, I couldn't, so the once control took any of 3 states (start, initialising, finished). But then I couldn't change the kernel. Unusual situation).
pthread_key_t is like an index for accessing thread-local storage. You can think of each thread as having a map from keys to values. When you add a new entry to TLS, pthread_key_create chooses a key for it and writes that key into the location you specify. You then use that key from any thread, whenever you want to set or retrieve the value of that TLS item for the current thread. The reason TLS gives you a key instead of letting you choose one, is so that unrelated libraries can use TLS, without having to co-operate to avoid both using the same value and trashing each others' TLS data. The pthread library might for example keep a global counter, and assign key 0 for the first time pthread_key_create is called, 1 for the second, and so on.
Wow, the other answers here are way too verbose.
pthread_once_t stores state for pthread_once(). Calling pthread_once(&s, fn) calls fn and sets the value pointed to by s to record the fact it has been executed. All subsequent calls to pthread_once() are noops. The name should become obvious now.
pthread_once_t should be initialized to PTHREAD_ONCE_INIT.