I have tutorials on my university about cuncurrent programming.
My task is to write a program based on semaphores in which a symmetric
difference of two sets of numbers will be computed. I can't see where
the concurrent programming is necessary. I understand that CP is about
using same resources of data, but in which phase I should use two
threads and semaphore. Semaphores are used to hold some process which
may interfere the data, that are used by other process. I just don't
see it.
Any ideas?
The input sets are read-only, so your multiple threads will not step on each-others' toes (except maybe for memory bus contention), but you will need to synchronize your threads as they write into the result set.
Related
Let a thread declare a lock (v. gr. std::unique_lock<std::mutex>) to try to own a certain mutex. If contention, that thread will be added to a list of threads waiting for that mutex to be unlocked.
How long can that list get?
Considering that the actual length might differ between implementations, I'm just interested in knowing what is the order of magnitude. And if it is limited by stack size or memory?
While learning about mutexes and conditional variables, I figured that a particular implementation of a processing buffer could just generate a pool of threads with one lock per thread of the same mutex.
I understand that there is no actual limitation to the number of threads running simultaneously other than physical memory. So, is there, perhaps limitation in terms of the queue of threads waiting to acquire a single mutex?
I'm just curious. I'm not thinking of implementing this design... or maybe that is precisely why I should play with it and see its limitations, its "hackyness" and observe probably unforeseen problems with it. Is this a common design option?
Thank you,
I think there is a partial answer here for Linux machines. It depends on the limits applied to every process. Call ulimit in a terminal to see them.
I am trying to find out if there is an inherent speed difference between inter-thread and inter-process communication.
I know that when using threads the threads share the same memory, can use the same global variables, and the such while processes have to use other tricks, which basically means queues.
But take the following case:
An application is comprised of several completely separate .exe files. When all are run they form a producer/consumer (or publisher/subscriber) architecture, with some processes producing some values and other processes reading and using those values and maybe producing some other values.
This communication is done with conventional ways of IPC.
My question is: if I were to move the code around so that it's one process with multiple threads (assuming no conflicts with variable names and the such), but keep the communication methods the same, queues with all the locks and semaphores behind them, will the thread-based application be faster than the process-based one?
The startup costs of processes vs. threads are not important because the application is meant to run for a long time (hours) so a few milliseconds will not be important.
Google has yielded no conclusive answers to this.
To clarify some aspects of the question:
The factor I want to maximize is throughput.
Some external factor (an arduino sensor for example) produces an input for one of the nodes and the entire network takes some time while all the nodes consume and produce values. Then a new input can be processed. I would like to be able to process more inputs per minute/second.
The data being passed back and forth are mostly numbers or small arrays of numbers.
The entire network can have lets say between 5 and 25 nodes.
As for platform (if it is relevant) I would like answers for both Linux and Windows.
The specific use-case is too large to be described here so consider the use-case provided above. This is as much, if not more, an educational question for my own knowledge as it is a question about a specific problem.
Please ask for any other relevant information that I have not included here.
If I were to move the code around so that it's one process with multiple threads (assuming no conflicts with variable names and the such), but keep the communication methods the same, queues with all the locks and semaphores behind them, will the thread-based application be faster than the process-based one?
That is not possible. The multiple thread version will take advantage of the shared memory space and the multi-process version cannot do so. For example, you can take an ordinary object in one thread and access it through a pointer in another thread, and all the referenced sub-objects will "just work". Anything not modified can be accessed just as easily in one thread as another with no special effort.
This simply won't work across processes at all since they don't share an address space.
I have a general question about parallel programming in C and C++ and would appreciate it if you could answer it. As far as I know, we can declare a variable in at least one level higher (parent thread) to share it among children threads. So, I was wondering if there is any other way to share a variable among threads with the same parent thread? Is this API dependant or not?
For Posix threads, read some pthread tutorial.
For C++11, read the documentation of its thread library
All threads of the same process share the same address space in virtual memory. As commented by Marco A. consider also thread_local variables.
Notice that you share data or memory (not variables, which exist only in the source code)
In practice, you'll better protect with a mutex the shared data (for synchronization) to avoid data races.
In the simple case, the mutex and the shared data are in some global variables.
You could also use atomic operations.
BTW, you could also develop a parallel application using some message passing paradigm, e.g. using MPI (or simply using some RPC or other messages, e.g. JSON on sockets). You might consider for regular numerical applications to use the GPGPU e.g. using OpenCL. And of course you might mix all the approaches (using OpenCL, with several threads, and having your parallel software running in several such processes communicating with MPI).
Debugging a heavily parallel software can become a nightmare. Performance may depend upon the hardware system and may require tricky tuning. scalability and synchronization may becoming a growing concern.map-reduce is often a useful model.
In C++ and C any memory location (identified by a variable) can be shared among threads. The memory space is the same across all threads. There is no parent/child thread relationship with memory.
The challenge is to control or synchronize access to the memory location among the threads.
That is implementation dependent.
Any global variable is sharable among threads, since threads are light weight processes sharing the same address space. For synchronization, you need to ensure mutual exclusion while updating/accessing those global variables through semaphores or wait notify blocks.
If I have
1. mainThread: write data A,
2. Thread_1: read A and write it to into a Buffer;
3. Thread_2: read from the Buffer.
how to synchronize these three threads safely, with not much performance loss? Is there any existing solution to use? I use C/C++ on linux.
IMPORTANT: the goal is to know the synchronization mechanism or algorithms for this particular case, not how mutex or semaphore works.
First, I'd consider the possibility of building this as three separate processes, using pipes to connect them. A pipe is (in essence) a small buffer with locking handled automatically by the kernel. If you do end up using threads for this, most of your time/effort will be spent on creating nearly an exact duplicate of the pipes that are already built into the kernel.
Second, if you decide to build this all on your own anyway, I'd give serious consideration to following a similar model anyway. You don't need to be slavish about it, but I'd still think primarily in terms of a data structure to which one thread writes data, and from which another reads the data. By strong preference, all the necessary thread locking necessary would be built into that data structure, so most of the code in the thread is quite simple, reading, processing, and writing data. The main difference from using normal Unix pipes would be that in this case you can maintain the data in a more convenient format, instead of all the reading and writing being in text.
As such, what I think you're looking for is basically a thread-safe queue. With that, nearly everything else involved becomes borders on trivial (at least the threading part of it does -- the processing involved may not be, but at least building it with multiple threads isn't adding much to the complexity).
It's hard to say how much experience with C/C++ threads you have. I hate to just point to a link but have you read up on pthreads?
https://computing.llnl.gov/tutorials/pthreads/
And for a shorter example with code and simple mutex'es (lock object you need to sync data):
http://students.cs.byu.edu/~cs460ta/cs460/labs/pthreads.html
I would suggest Boost.Thread for this purpose. This is quite good framework with mutexes and semaphores, and it is multiplatform. Here you can find very good tutorial about this.
How exactly synchronize these threads is another problem and needs more information about your problem.
Edit The simplest solution would be to put two mutexes -- one on A and second on Buffer. You don't have to worry about deadlocks in this particular case. Just:
Enter mutex_A from MainThread; Thread1 waits for mutex to be released.
Leave mutex from MainThread; Thread1 enters mutex_A and mutex_Buffer, starts reading from A and writes it to Buffer.
Thread1 releases both mutexes. ThreadMain can enter mutex_A and write data, and Thread2 can enter mutex_Buffer safely read data from Buffer.
This is obviously the simplest solution, and probably can be improved, but without more knowledge about the problem, this is the best I can come up with.
What is the common theory behind thread communication? I have some primitive idea about how it should work but something doesn't settle well with me. Is there a way of doing it with interrupts?
Really, it's just the same as any concurrency problem: you've got multiple threads of control, and it's indeterminate which statements on which threads get executed when. That means there are a large number of POTENTIAL execution paths through the program, and your program must be correct under all of them.
In general the place where trouble can occur is when state is shared among the threads (aka "lightweight processes" in the old days.) That happens when there are shared memory areas,
To ensure correctness, what you need to do is ensure that these data areas get updated in a way that can't cause errors. To do this, you need to identify "critical sections" of the program, where sequential operation must be guaranteed. Those can be as little as a single instruction or line of code; if the language and architecture ensure that these are atomic, that is, can't be interrupted, then you're golden.
Otherwise, you idnetify that section, and put some kind of guards onto it. The classic way is to use a semaphore, which is an atomic statement that only allows one thread of control past at a time. These were invented by Edsgar Dijkstra, and so have names that come from the Dutch, P and V. When you come to a P, only one thread can proceed; all other threads are queued and waiting until the executing thread comes to the associated V operation.
Because these primitives are a little primitive, and because the Dutch names aren't very intuitive, there have been some ther larger-scale approaches developed.
Per Brinch-Hansen invented the monitor, which is basically just a data structure that has operations which are guaranteed atomic; they can be implemented with semaphores. Monitors are pretty much what Java synchronized statements are based on; they make an object or code block have that particular behavir -- that is, only one thread can be "in" them at a time -- with simpler syntax.
There are other modeals possible. Haskell and Erlang solve the problem by being functional languages that never allow a variable to be modified once it's created; this means they naturally don't need to wory about synchronization. Some new languages, like Clojure, instead have a structure called "transactional memory", which basically means that when there is an assignment, you're guaranteed the assignment is atomic and reversible.
So that's it in a nutshell. To really learn about it, the best places to look at Operating Systems texts, like, eg, Andy Tannenbaum's text.
The two most common mechanisms for thread communication are shared state and message passing.
THe most common way for threads to communicate is via some shared data structure, typically a queue. Some threads put information into the queue while others take it out. The queue must be protected by operating system facilities such as mutexes and semaphores. Interrupts have nothing to do with it.
If you're really interested in a theory of thread communications, you may want to look into formalisms like the pi Calculus.
To communicate between threads, you'll need to use whatever mechanism is supplied by your operating system and/or runtime. Interrupts would be unusually low level, although they might be used implicitly if your threads communicate using sockets or named pipes.
A common pattern would be to implement shared state using a shared memory block, relying on an os-supplied synchronization primitive such as a mutex to spare you from busy-waiting when your read from the block. Remember that if you have threads at all, then you must have some kind of scheduler already (whether it's native from the OS or emulated in your language runtime). So this scheduler can provide synchronization objects and a "sleep" function without necessarily having to rely on hardware support.
Sockets, pipes, and shared memory work between processes too. Sometimes a runtime will give you a lighter-weight way of doing synchronization for threads within the same process. Shared memory is cheaper within a single process. And sometimes your runtime will also give you an atomic message-passing mechanism.