I have a master-slave model in my MPI program. I want to make slaves wait for each other before going to the next iteration.
if (rank == 0) {
// master process
} else {
// slave process
for (int i = 0; i < 10; i++) {
// do stuff
// wait for all slaves to end iteration i
}
}
Basically, I don't want any processor to go into next iteration without all other slaves complete their current iteration. How can I do this? With MPI_Barrier?
You can create a communicator comprising with all slave processes and use it on a MPI_Barrier().
Fro creating this communicator, the simplest / safest is to use MPI_Comm_split() this way:
MPI_Comm slaves;
MPI_Comm_split( MPI_COMM_WORLD, ( rank == 0 ), rank, &slaves );
This will actually globally create 2 communicators: one comprising only the master process and one comprising all processes but the master.
For actual use, you can do it this way:
if (rank == 0) {
// master process
} else {
// slave process
for (int i = 0; i < 10; i++) {
// do stuff
// wait for all slaves to end iteration i
MPI_Barrier( slaves );
}
}
Related
I have a problem with my code:
#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
#include <windows.h>
#include <string.h>
#include <math.h>
HANDLE event;
HANDLE mutex;
int runner = 0;
DWORD WINAPI thread_fun(LPVOID lpParam) {
int* data = (int*)lpParam;
for (int j = 0; j < 4; j++) { //this loop necessary in order to reproduce the issue
if ((data[2] + 1) == data[0]) { // if it is last thread
while (1) {
WaitForSingleObject(mutex, INFINITE);
if (runner == data[0] - 1) { // if all other thread reach event break
ReleaseMutex(mutex);
break;
}
printf("Run:%d\n", runner);
ReleaseMutex(mutex);
Sleep(10);
}
printf("Check Done:<<%d>>\n", data[2]);
runner = 0;
PulseEvent(event); // let all other threads continue
}
else { // if it is not last thread
WaitForSingleObject(mutex, INFINITE);
runner++;
ReleaseMutex(mutex);
printf("Wait:<<%d>>\n", data[2]);
WaitForSingleObject(event, INFINITE); // wait till all other threads reach this stage
printf("Exit:<<%d>>\n", data[2]);
}
}
return 0;
}
int main()
{
event = CreateEvent(NULL, TRUE, FALSE, NULL);
mutex = CreateMutex(NULL, FALSE, NULL);
SetEvent(event);
int data[3] = {2,8}; //0 amount of threads //1 amount of numbers
HANDLE t[10000];
int ThreadData[1000][3];
for (int i = 0; i < data[0]; i++) {
memcpy(ThreadData[i], data, sizeof(int) * 2); // copy amount of threads and amount of numbers to the threads data
ThreadData[i][2] = i; // creat threads id
LPVOID ThreadsData = (LPVOID)(&ThreadData[i]);
t[i] = CreateThread(0, 0, thread_fun, ThreadsData, 0, NULL);
if (t[i] == NULL)return 0;
}
while (1) {
DWORD res = WaitForMultipleObjects(data[0], t, true, 1000);
if (res != WAIT_TIMEOUT) break;
}
for (int i = 0; i < data[0]; i++)CloseHandle(t[i]); // close all threads
CloseHandle(event); // close event
CloseHandle(mutex); //close mutex
printf("Done");
}
The main idea is to wait until all threads except one reach the event and wait there, meanwhile the last thread must release them from waiting.
But the code doesn't work reliably. 1 in 10 times, it ends correctly, and 9 times just gets stuck in while(1). In different tries, printf in while (printf("Run:%d\n", runner);) prints different numbers of runners (0 and 3).
What can be the problem?
As we found out in the comments section, the problem was that although the event was created in the initial state of being non-signalled
event = CreateEvent(NULL, TRUE, FALSE, NULL);
it was being set to the signalled state immediately afterwards:
SetEvent(event);
Due to this, at least on the first iteration of the loop, when j == 0, the first worker thread wouldn't wait for the second worker thread, which caused a race condition.
Also, the following issues with your code are worth mentioning (although these issues were not the reason for your problem):
According to the Microsoft documentation on PulseEvent, that function should not be used, as it can be unreliable and is mainly provided for backward-compatibility. According to the documentation, you should use condition variables instead.
In your function thread_fun, the last thread is locking and releasing the mutex in a loop. This can be bad, because mutexes are not guaranteed to be fair and it is possible that this will cause other threads to never be able to acquire the mutex. Although this possibility is mitigated by you calling Sleep(10); once in every loop iteration, it is still not the ideal solution. A better solution would be to use a condition variable, so that the thread only checks for changes of the variable runner when another thread actually signals a possible change. Such a solution would also be better for performance reasons.
I am looking for a good way to create 2 equally child processes, which will complete 1 task - to sort separately, one from the other, an array with numbers data with different algorithms for example Merge-Sort, and QuickSort.
The first process, which completed the task, should terminate another one (and print result)
Also which will be the best way to transfer data to processes (Pipes?)
Also is there something I didn't take into account?
I should use system functions, Posix and Linux based code
int main() {
pid_t child_a, child_b;
int unsortedData[MAX] = getNumericDataFromFile(/*filename*/);
// Open two pipes for communication
// The descriptors will be available to both
// parent and child.
int in_fd[2];
int out_fd[2];
if (pipe(in_fd) == -1) { // For child's stdin
fprintf(stderr, "Pipe In Failed");
return 1;
}
if (pipe(out_fd) == -1) { // For child's stdout
fprintf(stderr, "Pipe Out Failed");
return 1;
}
if ((child_a = fork()) < 0) { // create first child process
fprintf(stderr, "Fork of first child process Failed");
return 1;
}
if (child_a == 0) {
/* Child A code, execute first algorithm */
int sortedData[MAX] = firstChildProcess(/*unsortedData*/);
/* terminate other process properly if its firs ? */
} else {
if ((child_b = fork()) < 0) { // create second child process
fprintf(stderr, "Fork of second child process Failed");
return 1;
}
if (child_b == 0) {
/* Child B code, execute second algorithm */
int sortedData[MAX] = ChildProcess(/*unsortedData*/)
/* terminate other process properly if its first? */
} else {
parentCode(..);/* Parent Code */
}
}
return 0;
}
Input : 6, 5, 4, 3, 2, 1
Output: Second algorithm finished first sorting task with result:
1, 2, 3, 4, 5, 6
Thank you in advance
I have a loop which calls pthread_join but the order of the loop does not match the order of thread's termination.
how can i monitor thread completion then call join?
for ( int th=0; th<sections; th++ )
{
cout<<"start joining "<<th<<endl<<flush;
result_code = pthread_join( threads[th] , (void**)&status);
cout<<th<<" join error "<<strerror(result_code)<<endl<<flush;
cout<<"Join status is "<<status<<endl<<flush;
}
This is my solution, which seems to maximize multi-threading throughput by serving the first
done thread . This solution does not depend on pthread_join loop order.
// loop & wait for the first done thread
std::bitset<Nsections> ready;
std::bitset<Nsections> done;
ready.reset();
for (unsigned b=0; b<sections; b++) ready.flip(b);
done = ready;
unsigned currIdx = 1;
int th = 0;
int th_= 0;
int stat;
while ( done.any() )
{
// main loops waiting for 1st thread to complete.
// completion is checked by global vector
// vStatus (singlton write protected)
// and not by pthread_exit returned value,
// in ordder to maximize throughput by
// post processig the first
// finished thread.
if ( (obj.vStatus).empty() ) { Sleep (5); continue; }
while ( ready.any() )
{
if ( sections == 1 ) break;
if ( !(obj.vStatus).empty() )
{
if ( currIdx <= (obj.vStatus).size() )
{
th_ = currIdx-1;
std::string s =
ready.to_string<char,std::string::traits_type,std::string::allocator_type>();
cout<<"checking "<<th_<<"\t"<<s<<"\t"
<<(ready.test(th_)?"T":"F")<<"\t"<<(obj.vStatus)[th_].retVal <<endl;
if ((obj.vStatus)[th_].retVal < 1)
{
if (ready.test(th_))
{
th=th_;
ready.reset(th);
goto retry;
}
}
}
}
Sleep (2);
} // while ready
retry:
cout<<"start joining "<<th<<endl<<flush;
result_code = pthread_join( threads[th] , (void**)&status);
switch (result_code)
{
case EDEADLK: goto retry; break;
case EINVAL:
case ESRCH:
case 0:
currIdx++;
stat = status->retVal;
free (status);
done.reset(th);
std::string s =
done.to_string<char,std::string::traits_type,std::string::allocator_type>();
cout<<"joined thread "<<th<<"\t"<<s<<"\t"
<<(done.test(th)?"T":"F")<<"\t"<<stat <<endl;
while (true)
{
auto ret=pthread_cancel ( threads[th] ) ;
if (ret == ESRCH) { netTH--; break; }
Sleep (20);
}
break;
}
How can I monitor thread completion then call join ?
By letting join detect the completion. (i.e. do nothing special)
I have a loop which calls pthread_join but the order of the loop does not match the order of thread's termination.
The order of the loop does not matter.
a) thread[main] calling thread[1].'join' will simply be suspended until thread[1] exits. After that, thread[main] will be allowed to continue with the rest of the loop.
b) When thread[2] terminates before thread[1], thread[main] calling thread[2].join simply returns immediately. Again, thread[main] continues.
c) The effort to ensure thread[1] terminates prior to thread[2] (to match the loop sequence) is a surprisingly time consuming effort, with no benefit.
Update in progress ... looking for code I thought I have already submitted.
I'm trying to make a program which make 9 child process, so I use fork 9 times only if we are the father, like this:
for (int i = 0; i < 9; i++) { // Creo 9 hijos.
if (child_pid > 0) {
child_pid = fork();
childs[i] = child_pid;
}
if (child_pid < 0)
printf("Error...\n");
}
Now, I have to print on each children what children he is, starting from 0, so I was thinking about this:
printf("This is child #%d\n", getpid() - getppid());
But I'm not sure, Does this always work?, What if while the parent is creating childrens the operating system creates another process?, the number of children will be discontinued?.
And finally, if the answer is yes, how can I make that the #n children knows that he is the children number n?.
You can use the i variable to tell which child you are in, but the logic of your loop is incorrect. It should go like this:
for (int i = 0; i < 9; ++i) {
child_pid = fork();
if (child_pid == 0) {
// We are the child. The value of the i variable will tell us which one.
// If i == 0 we are the first child, i == 1 and we are the second, and so on.
printf("We are child #%d\n", i);
exit(EXIT_SUCCESS);
}
if (child_pid < 0) {
// Forking failed.
perror("fork()");
exit(EXIT_FAILURE);
}
// Otherwise we are the parent and forking was successful; continue the loop.
}
The operating system is not required to assign process IDs in sequential order. If another process is using the next one, it would be skipped over in a sequential assignment method, but the OS could really assign a random number as the pid as long as it is not in use.
I have a serial C++ program that I wish to parallelize. I know the basics of MPI, MPI_Send, MPI_Recv, etc. Basically, I have a data generation algorithm that runs significantly faster than the data processing algorithm. Currently they run in series, but I was thinking that running the data generation in the root process, having the data processing done on the slave processes, and sending a message from the root to a slave containing the data to be processed. This way, each slave processes a data set and then waits for its next data set.
The problem is that, once the root process is done generating data, the program hangs because the slaves are waiting for more.
This is an example of the problem:
#include "mpi.h"
#include <cassert>
#include <cstdio>
class Generator {
public:
Generator(int min, int max) : value(min - 1), max(max) {}
bool NextValue() {
++value;
return value < max;
}
int Value() { return value; }
private:
int value, max;
Generator() {}
Generator(const Generator &other) {}
Generator &operator=(const Generator &other) { return *this; }
};
long fibonnaci(int n) {
assert(n > 0);
if (n == 1 || n == 2) return 1;
return fibonnaci(n-1) + fibonnaci(n-2);
}
int main(int argc, char **argv) {
MPI_Init(&argc, &argv);
int rank, num_procs;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
if (rank == 0) {
Generator generator(1, 2 * num_procs);
int proc = 1;
while (generator.NextValue()) {
int value = generator.Value();
MPI_Send(&value, 1, MPI_INT, proc, 73, MPI_COMM_WORLD);
printf("** Sent %d to process %d.\n", value, proc);
proc = proc % (num_procs - 1) + 1;
}
} else {
while (true) {
int value;
MPI_Status status;
MPI_Recv(&value, 1, MPI_INT, 0, 73, MPI_COMM_WORLD, &status);
printf("** Received %d from process %d.\n", value, status.MPI_SOURCE);
printf("Process %d computed %d.\n", rank, fibonnaci(2 * (value + 10)));
}
}
MPI_Finalize();
return 0;
}
Obviously not everything above is "good practice", but it is sufficient to get the point across.
If I remove the while(true) from the slave processes, then the program exits when each of the slaves have exited. I would like the program to exit only after the root process has done its job AND all of the slaves have processed everything that has been sent.
If I knew how many data sets would be generated, I could have that many process running and everything would exit nicely, but that isn't the case here.
Any suggestions? Is there anything in the API that will do this? Could this be solved better with a better topology? Would MPI_Isend or MPI_IRecv do this better? I am fairly new to MPI so bear with me.
Thanks
The usual practice is to send to all worker processes an empty message with a special tag that signals them to exit the infinite processing loop. Let's say this tag is 42. You would do something like that in the worker loop:
while (true) {
int value;
MPI_Status status;
MPI_Recv(&value, 1, MPI_INT, 0, MPI_ANY_TAG, MPI_COMM_WORLD, &status);
if (status.MPI_TAG == 42) {
printf("Process %d exiting work loop.\n", rank);
break;
}
printf("** Received %d from process %d.\n", value, status.MPI_SOURCE);
printf("Process %d computed %d.\n", rank, fibonnaci(2 * (value + 10)));
}
The manager process would do something like this after the generator loop:
for (int i = 1; i < num_procs; i++)
MPI_Send(&i, 0, MPI_INT, i, 42, MPI_COMM_WORLD);
Regarding your next question. Using MPI_Isend() in the master process would deserialise the execution and increase the performance. The truth however is that you are sending very small messages and those are typically internally buffered (WARNING - implementation dependent!) so your MPI_Send() is actually non-blocking and you already have non-serial execution. MPI_Isend() returns an MPI_Request handle that you need to take care of later. You could either wait for it to finish with MPI_Wait() or MPI_Waitall() but you could also just call MPI_Request_free() on it and it will be automatically freed when the operation is over. This is usually done when you'd like to send many messages asynchronously and would not care on when the sends will be completed, but it's a bad practice nevertheless since having a large number of outstanding requests can consume lots of precious memory. As for the worker processes - they need the data in order to proceed with the computation so using MPI_Irecv() is not necessary.
Welcome to the wonderful world of MPI programming!