Related
Let's say that you separated your domain according to the number of ranks (with mpi_cart_create) then you have the indices of each domain. Then I want to use gatherv to put back a global array in order to write the results. This code seems to work for 2 or 4 processes but doesn't for 8. I feel like I am missing something. There should be no error message.
Edit : each rank contains its local array. Its elements are equal to the rank of the process (from 0 to n_procs). What I am trying to archieve is to put back the global array with gatherv. This is for a finite volume code. My aim is just to reproduce the domain decomposition and what I desire with gatherv. Here, there should be no -10 in the print and the block should be clearly dissociable with the decomposition done by mpi.
The code works as follow : I separate my domain, compute the delimitation/indices associated to each domain, definite my new type that will be received in the global array, then calculate the displacement and use gatherv.
program scatter
use mpi
implicit none
integer,parameter :: nrows = 8,ncols = 8
integer, dimension(8,8) :: global
integer, dimension(2) :: dims_orig,dims,coo,sizes,subsizes,coo_init
logical,dimension(2) :: periods
integer, dimension(:,:),allocatable :: local
integer, dimension(:),allocatable :: displ,counts
integer :: info,code,rank,n_procs,comm2d
integer :: ndi,ndj,nni,nnj,nni_last,nnj_last
integer :: i1,i2,j1,j2,ii1,ii2,jj1,jj2,row,col
integer :: type0,type_subarray,sizeofint,i,j
integer :: imax,jmax,loc_size
integer(kind=mpi_address_kind) :: start,extend
logical :: reorder
! dimensions of the global array
imax = nrows
jmax = ncols
! default value of the global array
global = -10
call mpi_init(code)
call mpi_comm_rank(mpi_comm_world,rank,code)
call mpi_comm_size(mpi_comm_world,n_procs,code)
dims_orig = 0
CALL MPI_DIMS_CREATE(n_procs ,2,dims_orig,code)
! number of domain in each direction
ndi = dims_orig(1)
ndj = dims_orig(2)
! nni/nnj = sizes of each domain
nni = nrows/ndi
nnj = ncols/ndj
nni_last = nrows-(ndi-1)*nrows/ndi
nnj_last = ncols-(ndj-1)*ncols/ndj
dims(1) = ndi
dims(2) = ndj
periods = .false.
reorder = .true.
call mpi_cart_create(MPI_COMM_WORLD,2,dims,periods,reorder,comm2d,code)
call mpi_comm_rank(comm2d,rank,code)
call mpi_cart_get(comm2d,2,dims,periods,coo,code)
if(coo(1)==ndi-1) then
nni = nni_last
i2 = imax
i1 = i2-nni+1
else
i1 = rank/ndj*nni+1
i2 = i1+nni-1
endif
if(coo(2)==ndj-1) then
nnj = nnj_last
j2 = jmax
j1 = j2-nnj+1
else
j1 = MOD(rank,ndj)*nnj+1
j2 = j1+nnj-1
endif
print*,rank,"|",i1,i2,j1,j2
call mpi_barrier(mpi_comm_world,code)
! create new types
sizes = [imax, jmax]
subsizes = [i2-i1+1,j2-j1+1]
coo_init = [i1-1 ,j1-1 ]
call mpi_type_create_subarray(2,sizes,subsizes,coo_init,&
mpi_order_fortran,MPI_integer,type0,code)
call mpi_type_size(MPI_INTEGER,sizeofint,code)
start = 0
extend = sizeofint*nnj
! call mpi_type_get_extent(type0,start,extend,code)
call MPI_TYPE_CREATE_RESIZED(TYPE0,start,extend,TYPE_SUBARRAY,info)
!type_subarray = type0
call MPI_TYPE_COMMIT(TYPE_SUBARRAY,code)
allocate(displ(ndi*ndj))
! forall(col=1:ndj,row=1:ndi)
! displ(1+(row-1)+(col-1)*ndi) = (row-1)+(col-1)*imax
! ! displ(1+(row-1)*ndi+(col-1)) = (row-1)*ndi+(col-1)
! endforall
! computing the displacement
do i =1,ndi
do j = 1,ndj
displ(1+(i-1)+(j-1)*ndj) = (i-1) + (j-1)*imax
enddo
enddo
allocate(local(i1:i2,j1:j2))
allocate(counts(n_procs))
counts = 1
local = rank
loc_size = (i2-i1+1)*(j2-j1+1)!(i2-i1+1)*(j2-j1+1)
call mpi_gatherv(local,loc_size,MPI_integer,&
global,counts,displ,type_subarray,0,&
MPI_COMM_WORLD,code)
call mpi_barrier(mpi_comm_world,code)
if(rank==0) then
do i = 1,nrows
print*,global(i,:)
enddo
endif
call mpi_finalize(code)
endprogram scatter
I have a fortran code that computes the solution vector using the thomas algorithm subroutine.
I want the solution vector to run in a loop for a certain number of time.
How do i call this subroutine in the loop?
my subroutine is the thomas algorithm subroutine.
It returns the solution vector u but I want it to use the vectors NN times in a loop. So the old u becomes the new u to use in the subroutine.
How do I do this?
Below is the what i tried
program thomasalg2
implicit double precision(A-H,O-Z)
real*8, dimension(9,1) :: a,b,c,r,u,uold!the dimension is subject to change depending on the size of the new matrix
!real*8, dimension(9,50) :: W
real*8 :: pi
real*8 :: h,k,lm,l,T
integer :: i,j,al,NN,n
l = 1!right endpoint on the X-axis
n = 9 !number of rols/cols of the coefficient matrix with boundaries included
T = 0.5 !maximum number of the time variable
NN = 50!number of time steps
np = n
h = l/n
k = T/NN
al = 1.0D0 !alpha
pi = dacos(-1.0D0)
lm = (al**2)*(k/(h**2)) !lambda
do i = 1,n
r(i,1) = sin(pi*i*h) !this is W_0
end do
a(1,1) = 0.0D0
do i = 2,n
a(i,1) = -lm
end do
do i = 1,n
b(i,1) = 1 + (2*lm)
end do
c(9,1) = 0.0D0
do i = 1,n-1
c(i,1) = -lm
end do
!the 3 diagonals are stored in the 1st, 2nd, 3rd & 4th files respectively.
open(10, file = 'thom1.txt')
open(11, file = 'thom2.txt')
open(12, file = 'thom3.txt')
open(13, file = 'thom4.txt')
write(10,*)
do i = 1,n
write(10,*) a(i,1)
end do
write(11,*)
do i = 1,n
write(11,*) b(i,1)
end do
write(12,*)
do i = 1,n
write(12,*) c(i,1)
end do
write(13,*)
do i = 1,n
write(13,*) r(i,1)
end do
open(14, file = 'tridag2.txt')
write(14,*)
n = 9
do i = 1,n
write(14,*) a(i,1),b(i,1),c(i,1),r(i,1) !write the given vectors in the file in the form of a column vector
end do
call tridag(a,b,c,r,u,n)
!solve the given system and return the solution vector u
do i = 1,NN
call tridag(a,b,c,r,u,n)
!write(15,*) u
r = u
end do
open(15, file = 'tridag2u.txt')
write(15,*)
!write the solution vector in the form of a column vector
do i = 1,n
write(15,*) u(i,1)
end do
!print *, "Your data has been written in 'tridag2.txt'"
end program thomasalg2
subroutine tridag(a,b,c,r,u,n)
implicit double precision (A-H, O-Z)
integer n, NMAX
real*8 a(n), b(n), c(n), r(n), u(n)
parameter (NMAX = 500)
integer j
real*8 bet, gam(NMAX)
if(b(1).eq.0.) stop "tridag: rewrite equations"
bet = b(1)
u(1)=r(1)/bet
do j = 2,n
gam(j) = c(j-1)/bet
bet = b(j)-a(j)*gam(j)
if (bet.eq.0.) stop "tridag failed"
u(j) = (r(j)-a(j)*u(j-1))/bet
end do
do j = n-1,1,-1
u(j) = u(j)-gam(j+1)*u(j+1)
end do
!print *, "The solution is", u
return
end subroutine
I want to run a Fortran program which calls a subroutine that I want to parallelize with MPI. I know this sounds complicated, but I want to be able to specify the number of processes for each call. What I would want to use is a structure like this:
program my_program
implicit none
!Define variables
nprocs = !formula for calculating number of processes.
call my_subroutine(output,nprocs,other input vars)
end my_program
I want to run my_subroutine with the same effect as this:
mpirun -n nprocs my_subroutine.o
where my_subroutine has been compiled with 'other input vars.'
Is this possible?
Here is a simple example. I try compiling as follows:
$ mpif90 -o my_program WAVE_2D_FP_TUNER_mpi.f90 randgen.f SIMPLE_ROUTINE.f90
I try to run it like this:
$ mpirun -np (1 or 2) my_program
PROGRAM WAVE_2D_FP_TUNER_mpi
USE MPI
IMPLICIT NONE
REAL(KIND=8) :: T,PARAM(1:3),Z,ZBQLU01
REAL(KIND=8) :: ERRORS,COSTS,CMAX,CMAX_V(1:1000),THRESHOLD,Z_MIN,Z_MAX
REAL(KIND=8) :: U,S,R(1:6),MATRIX(1:15)
INTEGER :: EN,INC,I,J,M,P
INTEGER :: NPROCS,IERR
!0.8,-0.4,0.4,10,4,4,7 -- [0.003,0.534]
!0.8,-0.2,0.2,10,4,4,7 -- [0.190,0.588]
CALL MPI_INIT(IERR)
CALL MPI_COMM_SIZE(MPI_COMM_WORLD,NPROCS,IERR)
THRESHOLD = 0.D0
EN = 81
INC = 1
Z_MIN = -2.D-1; Z_MAX = 2.D-1
T = 1.D0
PARAM(1) = 10.D0; PARAM(2) = 4.D0; PARAM(3) = 4.D0
CMAX = 7.D0 !Max that wave speed could possibly be.
CALL ZBQLINI(0.D0)
OPEN(UNIT = 1, FILE = "TUNER_F.txt")
WRITE(1,*) 'Grid Size: '
WRITE(1,*) T/(EN-1)
DO P = 1,15
S = 0
Z = Z_MIN + (1.d0/(15-1))*dble((P-1))*(Z_MAX - Z_MIN)
WRITE(1,*) 'Z: ',Z
DO I = 1,1000
DO J = 1,6
R(J) = ZBQLU01(0.D0)
END DO
!CALL PDE_WAVE_F_mpi(T,PARAM,R,Z,CMAX,EN,INC,NPROCS,U)
CALL SIMPLE_ROUTINE(T,PARAM,R,Z,CMAX,EN,INC,NPROCS,U)
IF (U<=threshold) THEN
S = S + 1.D0
ELSE
S = S + 0.D0
END IF
END DO
MATRIX(P) = (1.D0/1000)*S
END DO
DO I = 1,15
WRITE(1,*) MATRIX(I)
END DO
PRINT *,MINVAL(MATRIX)
PRINT *,MAXVAL(MATRIX)
CLOSE(1)
CALL MPI_FINALIZE(IERR)
END PROGRAM WAVE_2D_FP_TUNER_mpi
Here is the subroutine that I wish to parallelize with mpi.
SUBROUTINE SIMPLE_ROUTINE(T,PARAM,R,Z,CMAX,EN,INC,NPROCS,U)
! Outputs scalar U = T*Z*CMAX*INC*SUM(PARAM)*SUM(R)*SUM(Y)
USE MPI
IMPLICIT NONE
REAL(KIND=8), INTENT(IN) :: T,PARAM(1:3),R(1:6),Z,CMAX
INTEGER, INTENT(IN) :: EN,INC
INTEGER, INTENT(IN) :: NPROCS
REAL(KIND=8), INTENT(OUT) :: U
REAL(KIND=8) :: H,LOCAL_SUM,SUM_OF_X
REAL(KIND=8), DIMENSION(:), ALLOCATABLE :: X
INTEGER :: PX,PX_MAX,NXL,REMX,IX_OFF,P_LEFT,P_RIGHT
INTEGER :: J
INTEGER :: IERR,MYID
! Broadcast nprocs handle to all processes in MPRI_COMM_WORLD
CALL MPI_BCAST(&NPROCS, NPROCS, MPI_INT, 0, MPI_COMM_WORLD,IERR)
! Create subcommunicator SUBCOMM (Do not know how to define WORLD_GROUP?)
CALL MPI_COMM_SPLIT(MPI_COMM_WORLD,WORLD_GROUP,SUBCOMM,IERR)
! Assign IDs to processes in SUBCOMM
CALL MPI_COMM_RANK(SUBCOMM,MYID,IERR)
! Give NPROCS - 1 to SUBCOMM
CALL MPI_COMM_SIZE(SUBCOMM,NPROCS-1,IERR)
H = 2.D0/(EN-1)
! LABEL THE PROCESSES FROM 1 TO PX_MAX.
PX = MYID + 1
PX_MAX = NPROCS
! SPLIT UP THE GRID IN THE X-DIRECTION.
NXL = EN/PX_MAX !nxl = 10/3 = 3
REMX = EN-NXL*PX_MAX !remx = 10-3*3 = 1
IF (PX .LE. REMX) THEN !for px = 1,nxl = 3
NXL = NXL+1 !nxl = 4
IX_OFF = (PX-1)*NXL !ix_off = 0
ELSE
IX_OFF = REMX*(NXL+1)+(PX-(REMX+1))*NXL !for px = 2 and px = 3, ix_off = 1*(3+1)+(2-(1+1))*3 = 4, ix_off = 1*(3+1)+(3-(1+1))*3 = 7
END IF
! ALLOCATE MEMORY FOR VARIOUS ARRAYS.
ALLOCATE(X(0:NXL+1))
X(:) = (/(-1.D0+DBLE(J-1+IX_OFF)*H, J=1,EN)/)
LOCAL_SUM = SUM(X(1:NXL))
CALL MPI_REDUCE(LOCAL_SUM,SUM_OF_X,1,&
MPI_DOUBLE_PRECISION,MPI_SUM,&
0,MPI_COMM_WORLD,IERR)
U = T*Z*CMAX*INC*SUM(PARAM)*SUM(R)*SUM_OF_X
DEALLOCATE(X)
CALL MPI_COMM_FREE(SUBCOMM,IERR)
CALL MPI_BARRIER(MPI_COMM_WORLD,IERR)
END SUBROUTINE SIMPLE_ROUTINE
Ultimately, I want to be able to change the number of processors used in the subroutine, where I want nprocs to be calculated from the value of EN.
A simple approach is to start the MPI app with the maximum number of processes.
Then my_subroutine will first MPI_Bcast(&nprocs, ...) and MPI_COMM_SPLIT(MPI_COMM_WORLD, ..., &subcomm) in order to create a sub communicator subcomm with nprocs
(you can use MPI_UNDEFINED so the "other" communicator will be MPI_COMM_NULL.
Then the MPI tasks that are part of subcomm will perform the computation.
Finally, MPI_Comm_free(&subcomm) and MPI_Barrier(MPI_COMM_WORLD)
From a performance point of view, note sub-communicator creation can be expensive, but hopefully not significant compared to the computation time.
If not, you'd rather revamp your algorithm so it can have nprocs tasks do the job, and the other ones waiting.
An other approach would be to start your app with one MPI task, MPI_Comm_spawn() nprocs-1 tasks, merge the inter-communicator, perform the computation, and terminates the spawned tasks.
The overhead of task creation is way more important, and this might not be fully supported by your resource manager, so I would not advise this option.
This is a small portion of the data I am trying to read:
01/06/2009,Tom Sanders,,264,220,73,260
01/08/2009,Adam Apple,158,,260,,208
01/13/2009,Lori Freeman,230,288,218,282,234
01/15/2009,Diane Greenberg,170,,250,321,197
01/20/2009,Adam Apple,257,,263,256,190
01/21/2009,Diane Greenberg,201,,160,195,142
01/27/2009,Tom Sanders,267,,143,140,206
01/29/2009,Tina Workman,153,,124,155,140
02/03/2009,Tina Workman,233,,115,,163
02/03/2009,Adam Apple,266,130,310,,310
the numbers between each comma are from a different location
Where two commas would represent missing data and a trailing comma would mean the fifth data point is missing
My goal is to organize the data into a table after calculating the average of each site and person, hence my two dim arrays
I want my output to look something like the following:
(obviously neater formatting but a table nonetheless)
Average Observed TDS (mg/l)
Name Site 1 Site 2 Site 3 Site 4 Site 5
------------------------------------------------------
Tom Sanders 251.0 172.5 251.7 160.0 229.0
Adam Apple 227.0 130.0 277.7 256.0 236.0
Lori Freeman 194.0 288.0 216.7 279.0 202.7
Diane Greenberg 185.5 190.0 205.0 258.0 169.5
Tina Workman 193.0 140.0 119.5 155.0 163.0
This is my program so far:
program name_finder
implicit none
integer, parameter :: wp = selected_real_kind(15)
real(wp) :: m, tds
real(wp), dimension(20,5) :: avg_site, site_sum
integer, dimension(20) :: nobs
integer, dimension(5) :: x
integer :: ierror, i, nemp, cp, non, ni, n
character(len=40), dimension(20) :: names
character(len=200) :: line, aname
character(len=20) :: output, filename
character(len=3), parameter :: a = "(A)"
do
write(*,*) "Enter file to open."
read(*,*) filename
open(unit=10,file = filename, status = "old", iostat = ierror)
if (ierror==0) exit
end do
write(*,*) "File, ",trim(filename)," has been opened."
non = 0
outer: do
read(10,a, iostat = ierror) line
if (ierror/=0) exit
cp = index(line(12:),",") + 11
aname = line(12:cp-1)
n=0
middle: do
read(line,'(Tcp,f4.2)') tds
write(*,*) "tds=", tds
n=n+1
if (n>10) exit
i = 1
inner: do
if (i > non) then
non = non +1
names(non) = trim(aname)
!ni = non
exit
end if
if (aname == names(i)) then
!ni = i
!cycle outer
exit inner
end if
i = i + 1
end do inner
end do middle
end do outer
write(*,*)
write(*,*) "Names:"
do i = 1,non
write(*,*) i, names(i)
end do
close(10)
close(20)
STOP
end program name_finder
TLDR; I am having trouble reading the data from the file shown at the top of each site after the names.
Suggestions? Thanks!
I hope the following is helpful. I have omitted any easily assumed declarations or any further data manipulation or writing to another file. The code is used just to read the data line by line.
character(150) :: word
read(fileunit, '(A)') word ! read the entire line
comma_ind = index(word,',') ! find the position of first comma
! Find the position of next comma
data_begin = index(word(comma_ind+1:),',')
! Save the name
thename = word(comma_ind+1:comma_ind+data_begin-1)
! Define next starting point
data_begin = comma_ind+data_begin
! Read the rest of the data
outer: do
if (word(data_begin+1:data_begin+1) == ',') then
! decide what to do when missing an entry
data_begin = data_begin + 1
cycle outer
else if (word(data_begin+1:data_begin+1) == ' ') then
! Missing last entry
exit outer
else
! Use it to find the length of current entry
st_ind = index(word(data_begin+1:),',')
if (st_ind == 0) then
! You reached the last entry, read it and exit
read(word(data_begin+1:), *) realData
exit outer
else
! Read current entry
read(word(data_begin+1: data_begin+st_ind-1),*) realData
end if
! Update starting point
data_begin = data_begin + st_ind
end if
end do outer
There could be a more elegant way to do it but I cannot think of any at the moment.
I'm trying to read a four column data file. However, because I'm having so much trouble, I'm just trying to read a single column of integers. Here is my code:
program RFF_Simple
implicit none
! Variables
character(len = 100):: line_in
character(len = :), allocatable :: filename
integer, dimension(:), allocatable :: weight,numbers
real, dimension(:), allocatable :: fm, fc
integer :: iostat_1, iostat_2
integer :: lun, length, index
! Body of RFF_Simple
! filename = 'data.txt'
filename = 'data_test.txt'
iostat_1 = 0
iostat_2 = 0
length = 0
open(newunit = lun, file = filename, status = 'old', iostat = iostat_1)
!Count how many lines are in the file (length)
if (iostat_1 == 0) then
do while(iostat_2 == 0)
read(lun, '(a)', iostat = iostat_2) line_in
if (iostat_2 == 0) then
length= length + 1
endif
enddo
endif
rewind(lun)
allocate(numbers(length)) !Allocate arrays to have same length as number of lines
iostat_1 = 0 !Reset
iostat_2 = 0
index = 1 !This whole thing is confusing so I don't know whether starting from 1 or 0 is better....
if (iostat_1 == 0) then
do while(iostat_2 == 0)
if(iostat_2 == 0) then
read(lun,*, iostat = iostat_2) numbers(index) !This crashes the program (Severe 408)
index = index + 1
endif
enddo
endif
write(*,*) 'Press Enter to Exit'
read(*,*)
end program RFF_Simple
The code compiles no problem, but running it yields the following: http://imgur.com/a/6ciJS
Yes I that is a print screen.
I don't even know where to start with this one.
The problem here is that you increment index after each successful read. After the last successful read we have index=length. You then add 1 to index and then attempt to read numbers(length+1) which results in a bounds violation. Rather than looping with a do while you can just use a regular do loop since we know the number of lines to read.
do index = 1, length
read(lun,*) numbers(index)
enddo
You could also test whether index is greater than length and bail out of the loop.