Related
How do I write out data from an unformatted file in the terminal, in Fortran?
Is it possible to do something similar to:
write(*,*) file_with_data
The reason is that I'm using a program where I try to run this code in a do loop:
read(lun) Vl0
But I get an error message in the first iteration. The error message is
"forrtl: severe (67): input statement requires too much data, unit 99" (lun=99)
So I'm trying to understand why there is a mismatch between the amount of data in the lun-file and the allocated array Vlm. Because I don't really understand why there is a mismatch. I thought it would be a good idea to print the data from the lun file and the pre-allocated array for Vlm to see if they match in size.
For further details of the code see below:
The error message appears when the program uses a specific subroutine. Here is a part of the subroutine. I have only selected the lines that I find relevant (The entire subroutine is much longer).
double complex, allocatable :: Vlm(:)
allocate (Vlm((grid%Lmax+1)**2))
open(lun, file=vgrdfile,iostat=ios,form='unformatted')
read(lun) ! Skip the header
do i = 1, grid%nrp
read(lun) Vlm !Error appears here
end
When I run the program that uses this subroutine I get an error message "forrtl: severe (67): input statement requires too much data, unit 99" (lun=99). In the test I'm running, grid%Lmax=1. So Vlm is really allocated as allocate (Vlm(4)). So for every iteration it should read a line in lun and store that to Vlm. So in the end Vlm should be a double complex array with 4 columns and grid%nrp rows.
The vgrdfile is named chiral-RNN1000-Vlm-L01-R0400-NR00004 and is generated by the following fortran code called potV_chiral.f95 :
program h_pot
integer(kind=4) :: lmax, nr, lmax_pot
real(kind=8) :: rmax
integer(kind=1) :: spherical, linear, even
integer :: i
real(kind=8) :: pi, r
double complex, allocatable :: Vc(:)
real, allocatable :: Vr(:)
real, allocatable :: Vi(:)
nr = 4
lmax = 1
lmax_pot = 1
rmax = 400d0
spherical = 0
linear = 0
even = 0
pi = acos(-1d0)
allocate (Vc( (lmax+1)**2 ))
allocate (Vr( (lmax+1)**2 ))
allocate (Vi( (lmax+1)**2 ))
open(1,file="chiral-RNN1000-Vlm-L01-R0400-NR00004",form="unformatted")
open(2,file="test.txt",form="formatted")
write(1) lmax,nr,rmax,spherical,linear,even !Creating header
do i = 1, nr
do l=1, (lmax_pot+1)**2
read(*,*) Vr(l), Vi(l)
Vc(l)=dcmplx(Vr(l),Vi(l))
end do
write(1) Vc
write(2,*) Vc
end do
close(1)
end program h_pot
The unformatted vgrdfile is created writing "f95 potV_chiral.f95" and "./a.out <complex_sample "
Where the complex_sample file consists of two columns:
1.0 1.0
1.0 2.0
1.0 1.0
1.0 0.0
1.0 2.0
1.0 1.0
2.0 0.0
2.0 1.0
0.0 1.0
1.0 1.0
3.0 1.0
2.0 1.0
2.0 1.0
5.0 1.0
5.0 1.0
-1.0 1.0
The test.txt file looks like this:
( 1.00000000 , 1.00000000 ) ( 1.00000000 , 2.00000000 ) ( 1.00000000 , 1.00000000 ) ( 1.00000000 , 0.00000000 )
( 1.00000000 , 2.00000000 ) ( 1.00000000 , 1.00000000 ) ( 2.00000000 , 0.00000000 ) ( 2.00000000 , 1.00000000 )
( 0.00000000 , 1.00000000 ) ( 1.00000000 , 1.00000000 ) ( 3.00000000 , 1.00000000 ) ( 2.00000000 , 1.00000000 )
( 2.00000000 , 1.00000000 ) ( 5.00000000 , 1.00000000 ) ( 5.00000000 , 1.00000000 ) ( -1.00000000 , 1.00000000 )
Note that I have very little experience with Fortran and working with binary files.
Edit: As #IanBush suggested. It might be because there is insufficient data. But personally I suspect that the reason is that I store the data in the unformatted file wrong, and not so much because of too little or too much data.
The reason for this is that In the code I can run a similar case, where I instead make a double-precision array (with a different dimension) and create the unformatted file by reading a single data column. This case works perfectly.
double precision, allocatable :: Vl0(:)
! File header
integer(kind=1) :: head_sym(3)
integer(kind=4) :: head_lmax, head_nr
real(kind=8) :: head_rmax
lun = 99
open(lun, file=vgrdfile,iostat=ios,form='unformatted')
read(lun) head_lmax,head_nr,head_rmax,head_sym
close(lun)
open(lun, file=vgrdfile,iostat=ios,form='unformatted')
read(lun) ! Skip the header
allocate (Vl0(head_lmax+1))
do i = 1, grid%nrp
read(lun) Vl0 !works
end
Where the program I use to create the vgridfile is
program pot1
integer(kind=4) :: lmax, nr
real(kind=8) :: rmax
integer(kind=1) :: spherical, linear, even
integer :: i
real(kind=8) :: pi, r
double precision, allocatable :: vl0(:)
pi = acos(-1d0)
lmax=2
nr=4
rmax=160
spherical=0
linear=1
even=1
open(1,file="hyd_lineven-RNN1000-Vlm-L02-R0160-NR01024",form="unformatted")
open(2,file="output_lineven",form="formatted")
write(1) lmax,nr,rmax,spherical,linear,even
! Linear and even potential saved as
! V00(r1) V20(r1) V40(r1) V60(r1) ...
! V00(r2) V20(r2) V40(r2) V60(r2) ...
! ...
! V00(rn) V20(rn) V40(rn) V60(rn) ...
allocate ( vl0(lmax+1) )
do i=1,nr
read(*,*) vl0
write(*,*) vl0
write(1) vl0
write(2,*) vl0
end do
close(1)
end program pot1
Where the output_even.txt looks like this
0.0000000000000000 0.0000000000000000 0.0000000000000000
-22.687409291590601 0.0000000000000000 0.0000000000000000
-11.343704645795301 0.0000000000000000 0.0000000000000000
-7.5624697638635299 0.0000000000000000 0.0000000000000000
A small example serial code, which has the same structure as my code, is shown below.
PROGRAM MAIN
IMPLICIT NONE
INTEGER :: i, j
DOUBLE PRECISION :: en,ei,es
DOUBLE PRECISION :: ki(1000,2000), et(200),kn(2000)
OPEN(UNIT=3, FILE='output.dat', STATUS='UNKNOWN')
DO i = 1, 1000, 1
DO j = 1, 2000, 1
ki(i,j) = DBLE(i) + DBLE(j)
END DO
END DO
DO i = 1, 200, 1
en = 2.0d0/DBLE(200)*(i-1)-1.0d0
et(i) = en
es = 0.0d0
DO j = 1, 1000, 1
kn=ki(j,:)
CALL CAL(en,kn,ei)
es = es + ei
END DO
WRITE (UNIT=3, FMT=*) et(i), es
END DO
CLOSE(UNIT=3)
STOP
END PROGRAM MAIN
SUBROUTINE CAL (en,kn,ei)
IMPLICIT NONE
INTEGER :: i
DOUBLE PRECISION :: en, ei, gf,p
DOUBLE PRECISION :: kn(2000)
p = 3.14d0
ei = 0.0d0
DO i = 1, 2000, 1
gf = 1.0d0 / (en - kn(i) * p)
ei = ei + gf
END DO
RETURN
END SUBROUTINE CAL
I am running my code on the cluster, which has 32 CPUs on one node, and there are totally 250 GB memory shared by 32 CPUs on one node. I can use 32 nodes maximumly.
Every time when the inner Loop is done, there is one data to be collected. After all outer Loops are done, there are totally 200 data to be collected. If only the inner Loop is executed by one CPU, it would take more than 3 days (more than 72 hours).
I want to do the parallelization for both inner Loop and outer Loop respectively? Would anyone please suggest how to parallelize this code?
Can I use MPI technique for both inner Loop and outer Loop respectively? If so, how to differentiate different CPUs that execute different Loops (inner Loop and outer Loop)?
On the other hand, I saw someone mention the parallelization with hybrid MPI and OpenMP method. Can I use MPI technique for the outer Loop and OpenMP technique for the inner Loop? If so, how to collect one data to the CPU after every inner Loop is done each time and collect 200 data in total to CPU after all outer Loops are done. How to differentiate different CPUs that execute inner Loop and outer Loop respectively?
Alternatively, would anyone provide any other suggestion on parallelizing the code and enhance the efficiency? Thank you very much in advance.
As mentioned in the comments, a good answer will require more detailed question. However, at a first sight it seems that parallelizing the internal loop
DO j = 1, 1000, 1
kn=ki(j,:)
CALL CAL(en,kn,ei)
es = es + ei
END DO
should be enough to solve your problem, or at least it will be a good starter. First of all I guess that there is an error on the loop
DO i = 1, 1000, 1
DO j = 1, 2000, 1
ki(j,k) = DBLE(j) + DBLE(k)
END DO
END Do
since the k is set to 0 and and there is no cell with address corresponding to 0 (see your variable declaration). Also ki is declared ki(1000,2000) array while ki(j,i) is (2000,1000) array. Beside these error, I guess that ki should be calculated as
ki(i,j) = DBLE(j) + DBLE(i)
if true, I suggest you the following solution
PROGRAM MAIN
IMPLICIT NONE
INTEGER :: i, j, k,icr,icr0,icr1
DOUBLE PRECISION :: en,ei,es,timerRate
DOUBLE PRECISION :: ki(1000,2000), et(200),kn(2000)
INTEGER,PARAMETER:: nthreads=1
call system_clock(count_rate=icr)
timerRate=real(icr)
call system_clock(icr0)
call omp_set_num_threads(nthreads)
OPEN(UNIT=3, FILE='output.dat', STATUS='UNKNOWN')
DO i = 1, 1000, 1
DO j = 1, 2000, 1
ki(i,j) = DBLE(j) + DBLE(i)
END DO
END DO
DO i = 1, 200, 1
en = 2.0d0/DBLE(200)*(i-1)-1.0d0
et(i) = en
es = 0.0d0
!$OMP PARALLEL DO private(j,kn,ei) firstpribate(en) shared(ki) reduction(+:es)
DO j = 1, 1000, 1
kn=ki(j,:)
CALL CAL(en,kn,ei)
es = es + ei
END DO
!$OMP END PARALLEL DO
WRITE (UNIT=3, FMT=*) et(i), es
END DO
CLOSE(UNIT=3)
call system_clock(icr1)
write (*,*) (icr1-icr0)/timerRate ! return computing time
STOP
END PROGRAM MAIN
SUBROUTINE CAL (en,kn,ei)
IMPLICIT NONE
INTEGER :: i
DOUBLE PRECISION :: en, ei, gf,p
DOUBLE PRECISION :: kn(2000)
p = 3.14d0
ei = 0.0d0
DO i = 1, 2000, 1
gf = 1.0d0 / (en - kn(i) * p)
ei = ei + gf
END DO
RETURN
END SUBROUTINE CAL
I add some variables to check the computing time ;-).
This solution is computed in 5.14 s, for nthreads=1, and in 2.75 s, for nthreads=2. It does not divide the computing time by 2, but it seems to be a good deal for a first shot. Unfortunately, on this machine I have a core i3 proc. So I can't do better than nthreads=2. However, I wonder, how the code will behave with nthreads=16 ???
Please let me know
I hope that this helps you.
Finally, I warn about the choice of variables status (private, firstprivate and shared) that might be consider carefully in the real code.
I posted a similar question few weeks ago (iso_c_binding calling C routine with pointers from Fortran with arrays) and I found a solution to my problem.
Now I modified few things and I am having some problems again.
In the following a simplified version of my problem.
I have a main program in fortran:
program main_dummy
! compile: gcc -c dummy_trace.c
! f95 raytracing.f90 main_dummy.f90 dummy_trace.o -o main
use, intrinsic :: ISO_C_BINDING
use raytracing
implicit none
!real(kind=8) :: x_in(4), x_fin(4)
real(C_DOUBLE), dimension(0:3) :: x_in, x_fin
real(C_DOUBLE) :: spin
integer :: rt_ok
x_in = (/1,2,3,4/)
x_fin = (/0,0,0,0/)
spin = 0.7
write(*,*)'x_in, x_fin before = ', x_in, x_fin
rt_ok = photon_trace(spin,x_in,x_fin)
write(*,*)'return rt = ', rt_ok
write(*,*)'x_in, x_fin after = ', x_in, x_fin
end program main_dummy
Which use a subroutine containing the interface for the C routine:
module raytracing
Interface
integer (C_INT) function photon_trace(BHsp, x_init, x_final) &
bind(C, name='photon_trace')
use , intrinsic :: ISO_C_BINDING
implicit none
real(C_DOUBLE) :: BHsp, x_init(4), x_final(4)
end function photon_trace
end interface
end module raytracing
and this is the C routine:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#undef I
int photon_trace(double BHsp, double x_init[4], double x_final[4])
//***************************************************
{
printf("BHsp %f\n", BHsp);
double r,m,t,phi;
t = x_init[0];
r = x_init[1];
m = x_init[2];
phi = x_init[3];
printf("t0 %f\n", t);
printf("r0 %f\n", r);
printf("m0 %f\n", t);
printf("phi0 %f\n", r);
t=t+1.0;
r=r+1.0;
m=m+1.0;
phi=phi+1.0;
printf("t1 %f\n", t);
printf("r1 %f\n", r);
printf("m1 %f\n", t);
printf("phi1 %f\n", r);
x_final[0] = t;
x_final[1] = r;
x_final[2] = m;
x_final[3] = phi;
return 0;
}
If I compile and run the program, this is what I get:
x_in, x_fin before = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
BHsp 0.000000
t0 0.700000
r0 0.000000
m0 0.700000
phi0 0.000000
t1 1.700000
r1 1.000000
m1 1.700000
phi1 1.000000
return rt = 0
x_in, x_fin after = 1.6999999880790710 1.0000000000000000 1.0000000000000000 1.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
Notice that before putting the variable "spin" everything was working. It could read the input array, make the opration, and give the right output.
Now that I added ths variable, there are some problems for the C routine in reading what I'm passing and I cannot understand what's wrong.
Any suggestion ?
(Consider that in the real case I'm going to pass several variables as well as 2 input and 2 output arrays with dimension 4).
Many thanks in advance!!
Change your interface to:
module raytracing
Interface
integer (C_INT) function photon_trace(BHsp, x_init, x_final) &
bind(C, name='photon_trace')
use , intrinsic :: ISO_C_BINDING
implicit none
real(C_DOUBLE) :: x_init(4), x_final(4)
real(c_double), value :: BHsp
end function photon_trace
end interface
end module raytracing
Your C function takes a double rather than double* so you need to pass the scalar with the value attribute so that the Fortran knows to pass by value rather than its default pass by reference.
With this small change (and some minor changes to your C to actually print the values of m and phi) this is the output of your example code:
% ./main
x_in, x_fin before = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
BHsp 0.700000
t0 1.000000
r0 2.000000
m0 3.000000
phi0 4.000000
t1 2.000000
r1 3.000000
m1 4.000000
phi1 5.000000
return rt = 0
x_in, x_fin after = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 5.0000000000000000
I posted a similar question few weeks ago (iso_c_binding calling C routine with pointers from Fortran with arrays) and I found a solution to my problem.
Now I modified few things and I am having some problems again.
In the following a simplified version of my problem.
I have a main program in fortran:
program main_dummy
! compile: gcc -c dummy_trace.c
! f95 raytracing.f90 main_dummy.f90 dummy_trace.o -o main
use, intrinsic :: ISO_C_BINDING
use raytracing
implicit none
!real(kind=8) :: x_in(4), x_fin(4)
real(C_DOUBLE), dimension(0:3) :: x_in, x_fin
real(C_DOUBLE) :: spin
integer :: rt_ok
x_in = (/1,2,3,4/)
x_fin = (/0,0,0,0/)
spin = 0.7
write(*,*)'x_in, x_fin before = ', x_in, x_fin
rt_ok = photon_trace(spin,x_in,x_fin)
write(*,*)'return rt = ', rt_ok
write(*,*)'x_in, x_fin after = ', x_in, x_fin
end program main_dummy
Which use a subroutine containing the interface for the C routine:
module raytracing
Interface
integer (C_INT) function photon_trace(BHsp, x_init, x_final) &
bind(C, name='photon_trace')
use , intrinsic :: ISO_C_BINDING
implicit none
real(C_DOUBLE) :: BHsp, x_init(4), x_final(4)
end function photon_trace
end interface
end module raytracing
and this is the C routine:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#undef I
int photon_trace(double BHsp, double x_init[4], double x_final[4])
//***************************************************
{
printf("BHsp %f\n", BHsp);
double r,m,t,phi;
t = x_init[0];
r = x_init[1];
m = x_init[2];
phi = x_init[3];
printf("t0 %f\n", t);
printf("r0 %f\n", r);
printf("m0 %f\n", t);
printf("phi0 %f\n", r);
t=t+1.0;
r=r+1.0;
m=m+1.0;
phi=phi+1.0;
printf("t1 %f\n", t);
printf("r1 %f\n", r);
printf("m1 %f\n", t);
printf("phi1 %f\n", r);
x_final[0] = t;
x_final[1] = r;
x_final[2] = m;
x_final[3] = phi;
return 0;
}
If I compile and run the program, this is what I get:
x_in, x_fin before = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
BHsp 0.000000
t0 0.700000
r0 0.000000
m0 0.700000
phi0 0.000000
t1 1.700000
r1 1.000000
m1 1.700000
phi1 1.000000
return rt = 0
x_in, x_fin after = 1.6999999880790710 1.0000000000000000 1.0000000000000000 1.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
Notice that before putting the variable "spin" everything was working. It could read the input array, make the opration, and give the right output.
Now that I added ths variable, there are some problems for the C routine in reading what I'm passing and I cannot understand what's wrong.
Any suggestion ?
(Consider that in the real case I'm going to pass several variables as well as 2 input and 2 output arrays with dimension 4).
Many thanks in advance!!
Change your interface to:
module raytracing
Interface
integer (C_INT) function photon_trace(BHsp, x_init, x_final) &
bind(C, name='photon_trace')
use , intrinsic :: ISO_C_BINDING
implicit none
real(C_DOUBLE) :: x_init(4), x_final(4)
real(c_double), value :: BHsp
end function photon_trace
end interface
end module raytracing
Your C function takes a double rather than double* so you need to pass the scalar with the value attribute so that the Fortran knows to pass by value rather than its default pass by reference.
With this small change (and some minor changes to your C to actually print the values of m and phi) this is the output of your example code:
% ./main
x_in, x_fin before = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000
BHsp 0.700000
t0 1.000000
r0 2.000000
m0 3.000000
phi0 4.000000
t1 2.000000
r1 3.000000
m1 4.000000
phi1 5.000000
return rt = 0
x_in, x_fin after = 1.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 2.0000000000000000 3.0000000000000000 4.0000000000000000 5.0000000000000000
I have an large txt-file which contains measurements an the longitude/latitude of different stations, like:
1 1 bwb02 52.38783333 13.40266667 LR 0.2
2 2 bwb04 52.53716667 13.22583333 P1 0
3 3 bwb05 52.55516667 13.202 SP 0
4 4 bwb07 52.48433333 13.4385 N1 0
I just want to read the 4,5 and 7 column and put them into a matrix, like:
do j = 1,n
read(12,200) latB,lonB,r
200 Format (12x,f11.9,1x,f11.9,4x,f5.3)
beo_data(j,1) = j
beo_data(j,2) = lonB
beo_data(j,3) = latB
beo_data(j,4) = r
end do
But my format isn't working and i get an error:
Fortran runtime error: Bad value during floating point read
The problem seems to be that the format of the longitude and latitude is changing.
Could you tell me how i can read and work with such a file?
Just read each line as a string using a descriptor and then you can read your data using a list-directed read (* instead of the format). Or you can use the list directed read directly, if there is no danger of reading from two lines at once.
integer :: tmp1,tmp2
character(5) :: tmp3
character(2) :: tmp4
real :: values(3)
open(12,file="data.txt")
do j = 1,4
read(12,*) tmp1, tmp2, tmp3, values(1), values(2), tmp4, values(3)
print *,values
end do
close(12)
end
prints
52.3878326 13.4026670 0.200000003
52.5371666 13.2258329 0.00000000
52.5551682 13.2019997 0.00000000
52.4843330 13.4385004 0.00000000
Using the fixed column format is useful only when the columns are really fixed.
You can also just use single character variable tmp instead of all those different ones.