Related
This question already has answers here:
Unexpected "padding" in a Fortran unformatted file
(4 answers)
Closed 2 years ago.
I am trying to use a code on both Windows with an Intel compiler and on Mac OS with gfortran 6.5.0 and can't get the gfortran version to read parts of an unformatted file written on Windows. I wrote a test code that has the problematic section and verified it works in writing and reading the file on the Intel compiler and on the gfortran compiler. However, if I comment out the write portion and just try to read the file written on the Windows side of the machine (Parallels) with the gfortran version, some of the data is not read properly. I'm including the test code below. The "cel(1), f_ei,f_en,f_egyro,f_wall,f_ex,f_wallBC =" data are not read correctly (the data before that are read in properly, however). Is there some incompatibility in unformatted files?
PROGRAM Testo
IMPLICIT NONE
INTEGER, PARAMETER :: ncelMAX=6800
REAL*8 :: dt
LOGICAL :: MagON,BFmesh
INTEGER :: nslice_Ver, nslice_verzr, nslice_Edg, nslice_edgzr
INTEGER :: nFluid, nChrge, nCel, nBCs, nEdg, nVer , nCelzr, nBCszr, nEdgzr, &
nVerzr, MinSliceCel, MaxSliceCel, MinSliceCelzr, MaxSliceCelzr
TYPE cell_obj
INTEGER*4 verNo(4),edgNo(4)
INTEGER*4 Bln !BField Mesh
REAL*8 f_ei,f_en,f_egyro,f_wall,f_ex, f_wallBC
END TYPE cell_obj
TYPE(cell_obj) cel(ncelMAX)
dt = 2.0E-8
MagON = .TRUE.
BFmesh = .TRUE.
nslice_Ver = 1
nslice_verzr = 2
nslice_Edg = 3
nslice_edgzr = 4
nFluid = 3
nChrge = 3
nCel = 3000
nBCs = 4000
nEdg = 5000
nVer = 6000
nCelzr = 7000
nBCszr = 8000
nEdgzr = 9000
nVerzr = 10000
MinSliceCel = 100
MaxSliceCel = 200
MinSliceCelzr = 300
MaxSliceCelzr = 400
cel(1)%verNo(1) = 1
cel(1)%verNo(2) = 2
cel(1)%verNo(3) = 3
cel(1)%verNo(4) = 4
cel(1)%edgNo(1) = 1
cel(1)%edgNo(2) = 2
cel(1)%edgNo(3) = 3
cel(1)%edgNo(4) = 4
cel(1)%Bln = 1000
cel(1)%f_ei = 2.0E6
cel(1)%f_en = 2.0E7
cel(1)%f_egyro = 2.0E8
cel(1)%f_wall = 3.0E6
cel(1)%f_ex = 3.0E7
cel(1)%f_wallBC = 3.0E8
OPEN(UNIT=10,FILE='restartData_TEST',FORM='UNFORMATTED')
REWIND(10)
WRITE(10) dt, MagON, nslice_Ver, nslice_Edg, nslice_verzr, nslice_edgzr, BFmesh
WRITE(10) nFluid, nChrge, nCel, nBCs, nEdg, nVer , nCelzr, nBCszr, nEdgzr, nVerzr, MinSliceCel, MaxSliceCel,MinSliceCelzr,MaxSliceCelzr
WRITE(10) cel
CLOSE(10)
OPEN(UNIT=10,FILE='restartData_TEST',FORM='UNFORMATTED')
REWIND(10)
READ(10) dt,MagON,nslice_Ver,nslice_Edg, nslice_verzr, nslice_edgzr, BFmesh
PRINT *, "First line = ", dt,MagON,nslice_Ver,nslice_Edg, nslice_verzr, nslice_edgzr, BFmesh
READ(10) nFluid, nChrge, nCel, nBCs, nEdg, nVer , nCelzr, nBCszr, nEdgzr, nVerzr, MinSliceCel, MaxSliceCel,MinSliceCelzr,MaxSliceCelzr
PRINT *, "Second line = ", nFluid, nChrge, nCel, nBCs, nEdg, nVer , nCelzr, nBCszr, nEdgzr, nVerzr, MinSliceCel, MaxSliceCel,MinSliceCelzr,MaxSliceCelzr
READ(10) cel
PRINT *, "cel(1), verNo(4),edgNo(4) = ",cel(1)%verNo(1),cel(1)%verNo(2),cel(1)%verNo(3),cel(1)%verNo(4),cel(1)%edgNo(1),cel(1)%edgNo(2),cel(1)%edgNo(3),cel(1)%edgNo(4)
PRINT *, "cel(1), Bln = ",cel(1)%Bln
PRINT *, "cel(1), f_ei,f_en,f_egyro,f_wall,f_ex,f_wallBC = ",cel(1)%f_ei,cel(1)%f_en,cel(1)%f_egyro,cel(1)%f_wall,cel(1)%f_ex,cel(1)%f_wallBC
PAUSE 'DONE'
END PROGRAM Testo
There is absolutely no guarantee of the portability of Fortran unformatted files - if you don't use exactly the same hardware and compiler combination (down to the version of the compiler) all bets are off. If you need portability use a formatted file, or possibly stream I/O, or probably best one of the portable data formats such as netcdf (https://www.unidata.ucar.edu/software/netcdf/) or hdf5 (https://www.hdfgroup.org/solutions/hdf5/)
Also please don't use the non standard real*8 and integer*4 - see Fortran 90 kind parameter for how to do it properly
I have 4 .mtx files that I am reading the values from. Two of them run perfectly when read from with no issues and produce the correct outputs into a .DAT file. However, the last 2 are extremely large files; it appears the code correctly reads from the files and runs, but I get no outputs and no errors when reading from these 2...not even the code timer prints the time. Any help is much appreciated! Here is the code:
program proj2matrixC40
implicit none
integer,parameter::dp=selected_real_kind(15,307)
! Set Global Variables
real(kind=dp), allocatable::Ax(:,:),A(:,:),Iglobal(:,:)
integer::At(1,3)
integer::nnz,w,n,k,ii,ff,kk
real(kind=dp)::t1,t2
call cpu_time(t1)
open(unit=78,file="e40r5000.mtx",status='old')
read(78,*) At
close(unit=78)
nnz = At(1,3)
n = At(1,1)
k = 40
kk = 35
allocate(Ax(nnz+1,3),A(nnz,3),Iglobal(k,k))
open(unit=61,file="e40r5000.mtx",status='old')
do w=1,nnz+1
read(61,*) Ax(w,:)
end do
open (unit = 53, file = "proj2matrixC40points.dat")
do ff=1,k
do ii=1,k
Iglobal(ii,ff) = (ii/ff)*(ff/ii)
end do
end do
A(1:nnz,:) = Ax(2:nnz+1,:)
call Arno(A)
call cpu_time(t2)
print '("Time elapsed = ",f10.8," seconds")', (t2 - t1)
contains
subroutine Arno(a)
real(kind=dp), intent(in)::a(:,:)
real(kind=dp),dimension(k,k)::H
real(kind=dp),dimension(k,k+1)::u,q,qconj
real(kind=dp),dimension(k,1)::x0
integer::j,f
call random_number(x0)
q(:,1) = x0(:,1)/norm2(x0(:,1))
do f=1,k
call spmat(a,q(:,f),u(:,f))
do j=1,f
qconj(j,:) = (q(:,j))
H(j,f) = dot_product(qconj(j,:),u(:,f))
u(:,f) = u(:,f) - H(j,f)*q(:,j)
end do
if (f.lt.k) then
H(f+1,f) = norm2(u(:,f))
if (H(f+1,f)==0) then
print *, "Matrix is reducible"
stop
end if
q(:,f+1) = u(:,f)/H(f+1,f)
end if
if (f==k) then
call qrit(H)
end if
end do
end subroutine
! QR Iteration with Shifts Subroutine
subroutine qrit(a)
real(kind=dp), intent(in)::a(:,:)
real(kind=dp)::sigmak
real(kind=dp),dimension(kk,k)::dia
real(kind=dp),dimension(k,k)::Qfinal,Rfinal,HH
real(kind=dp),dimension(k,k,kk)::H0,needQR
integer::v,z
HH = a
H0(:,:,1) = HH
do v=1,kk
sigmak = H0(k,k,v)
if (v-1==0) then
needQR(:,:,v) = HH - sigmak*Iglobal
else
needQR(:,:,v) = H0(:,:,v-1) - sigmak*Iglobal
end if
call givens2(needQR(:,:,v),Rfinal,Qfinal)
H0(:,:,v) = matmul(Rfinal,Qfinal) + sigmak*Iglobal
do z = 1,k
dia(v,z) = H0(z,z,v)
write(53,*) v," ", dia(v,z) ! Write values to .DAT file
end do
end do
end subroutine
! Sparse Matrix Vector Multiplication Subroutine
subroutine spmat(a,b,c)
real(kind=dp), intent(in)::a(:,:)
real(kind=dp), intent(in), dimension(k,1)::b
real(kind=dp), intent(out), dimension(k,1)::c
integer::m,rowi,columni
real(kind=dp), dimension(k,1)::x,y
x = b
y(:,1) = 0
do m = 1,nnz
rowi = a(m,1)
columni = a(m,2)
y(rowi,1) = y(rowi,1) + a(m,3)*x(columni,1)
end do
c(:,1) = y(:,1)
end subroutine
! QR Factorization Givens Rotations Subroutine
subroutine givens2(a,Rfinal,Qfinal)
real(kind=dp), intent(in)::a(:,:)
real(kind=dp), dimension(k,k,(k*k))::G,QQ
real(kind=dp), dimension(k,k), intent(out)::Rfinal,Qfinal
real(kind=dp), dimension(k,k)::I2,y,aa
real(kind=dp), dimension(1,k)::ek1,ek2
real(kind=dp)::c,s
integer::kt,m,nn,j,i,l,p
m = size(a,1)
nn = size(a,2)
aa = a
i = 1
do kt=1,nn-1
do j=m,kt+1,-1
if (aa(j,kt).eq.0) then
continue
else
ek1(1,:) = 0
ek2(1,:) = 0
do p=1,m
do l=1,m
I2(l,p) = (l/p)*(p/l)
end do
end do
c = aa(kt,kt)/sqrt(aa(kt,kt)**2 + aa(j,kt)**2)
s = aa(j,kt)/sqrt(aa(kt,kt)**2 + aa(j,kt)**2)
ek1(1,kt) = c
ek1(1,j) = s
ek2(1,kt) = -s
ek2(1,j) = c
I2(kt,:) = ek1(1,:)
I2(j,:) = ek2(1,:)
G(:,:,i) = I2
if (i.eq.1) then
QQ(:,:,i) = G(:,:,i)
else
QQ(:,:,i) = matmul(G(:,:,i),QQ(:,:,i-1))
end if
y = matmul(G(:,:,i),aa)
aa = y
if (kt.eq.nn-1) then
if (j.eq.kt+1) then
Qfinal = transpose(QQ(:,:,i))
Rfinal = aa
end if
end if
i = i + 1
end if
end do
end do
end subroutine
end program proj2matrixC40
A couple notes. The line which I put asterisks around (for this question) call mat_print('H',H) can't be deleted otherwise I get the wrong answers (this is strange...thoughts?). Also so your computer won't freeze opening the big files, their names are 'e40r5000.mtx' and 's3dkt3m2.mtx' (these are the two I have issues with). I am using gfortran version 8.1.0
Here is the link to the files
https://1drv.ms/f/s!AjG0dE43DVddaJfY62ABE8Yq3CI
When you need to add a call to a subroutine that shouldn't actually change anything in order to get things working, you probably have a memory corruption. This happens most often when you access arrays outside of their boundaries.
I have compiled it with some run time checks:
gfortran -o p2m -g -O0 -fbacktrace -fcheck=all -Wall proj2mat.f90
And it's already giving me some issues:
It's warning me about implicit type conversions. That shouldn't be too much of an issue if you trust your data.
In line 46 you have an array length mismatch (x0(:, 1) has length 40, q(:,1) is 41)
Similarly on line 108 (x=b) x is really large, but b is only 41 long.
I have stopped now, but I implore you to go through your code and clean it up. Use the compiler options above which will let you know when and where there is an array bound violation.
So I'm trying to come up with a clever way to make this program read a catalog and take anything falling within specific spatial "grid" boxes and average the data in that box together. I'll paste my horrid attempt below and hopefully you'll see what I'm trying to do. I can't get the program to work correctly (it gets stuck in a loop somewhere that I haven't debugged), and before I bang my head against it anymore I want to know if this looks like a logical set of operations for what I'm looking to do, or if there is a better way to accomplish this.
Edit: To clarify, the argument section is for the trimming parameters---"lmin lmax bmin bmax" set the overall frame, and "deg" sets the square-degree increments.
program redgrid
implicit none
! Variable declarations and settings:
integer :: ncrt, c, i, j, k, count, n, iarg, D, db, cn
real :: dsun, pma, pmd, epma, epmd, ra, dec, degbin
real :: V, Per, Amp, FeH, EBV, Dm, Fi, FeHav, EBVav
real :: lmin, lmax, bmin, bmax, l, b, deg, lbin, bbin
real :: bbinmax, bbinmin, lbinmax, lbinmin
character(len=60) :: infile, outfile, word, name
parameter(D=20000)
dimension :: EBV(D), FeH(D), lbinmax(D), bbinmax(D)
dimension :: bbinmin(D), lbinmin(D)
103 format(1x,i6,4x,f6.2,4x,f6.2,4x,f7.2,3x,f6.2,4x,f5.2,4x,f5.2,4x,f5.2,4x,f6.4)
3 continue
iarg=iargc()
if(iarg.lt.7) then
print*, 'Usage: redgrid infile outfile lmin lmax bmin bmax square_deg'
stop
endif
call getarg(1, infile)
call getarg(2, outfile)
call getarg(3, word)
read(word,*) lmin
call getarg(4, word)
read(word,*) lmax
call getarg(5, word)
read(word,*) bmin
call getarg(6, word)
read(word,*) bmax
call getarg(7, word)
read(word,*) deg
open(unit=1,file=infile,status='old',err=3)
open(unit=2,file=outfile,status='unknown')
write(2,*)"| l center | b center | [Fe/H] avg | E(B-V) avg | "
FeHav = 0.0
EBVav = 0.0
lbinmin(1) = lmin
bbinmin(1) = bmin
degbin = (bmax-bmin)/deg
db = NINT(degbin)
do j = 1, db
bbinmax(j) = bbinmin(j) + deg
lbinmax(j) = lbinmin(j)*cos(bbinmax(j))
print*, lbinmin(j), bbinmin(j), db
cn = 1
7 continue
read(1,*,err=7,end=8) ncrt, ra, dec, l, b,&
V, dsun, FeH(cn), EBV(cn)
if(b.ge.bbinmin(j).and.b.lt.bbinmax(j)) then
if(l.ge.lbinmin(j).and.l.lt.lbinmax(j)) then
FeHav = FeHav + FeH(cn)
EBVav = EBVav + EBV(cn)
cn = cn + 1
end if
end if
goto 7
8 continue
FeHav = FeHav/cn
EBVav = EBVav/cn
write(2,*) lbinmax(j), bbinmax(j), FeHav, EBVav
bbinmin(j+1) = bbinmin(j) + deg
lbinmin(j+1) = lbinmin(j) + deg
end do
close(1)
close(2)
end program redgrid
Below is a small section of the table I'm working with. "l" and "b" are the two coordinates I am working with---they are angular, hence the need to make the grid components "b" and "l*cos(b)." For each 0.5 x 0.5 degree section, I need to have averages of E(B-V) and [Fe/H] within that block. When I write the file all I need are four columns: the two coordinates where the box is located, and the two averages for that box.
| Ncrt | ra | dec | l | b | V | dkpc | [Fe/H] | E(B-V) |
7888 216.53 -43.85 -39.56 15.78 15.68 8.90 -1.19 0.1420
7889 217.49 -43.13 -38.61 16.18 16.15 10.67 -1.15 0.1750
7893 219.16 -43.26 -37.50 15.58 15.38 7.79 -1.40 0.1580
Right now, the program gets stuck somewhere in the loop cycle. I've pasted the terminal output that happens when I run it, along with the command line I'm running it with. Please let me know if I can help clarify. This is a pretty complex problem for a Fortran rookie such as myself---perhaps I'm missing some fundamental knowledge that would make it much easier. Anyways, thanks in advance.
./redgrid table2.above redtest.trim -40 0 15 30 0.5
-40.0000000 15.0000000 30 0.00000000 0.00000000
-39.5000000 15.5000000 30 -1.18592596 0.353437036
^it gets stuck after two lines.
I assume that the program does what you want it to do, but you are looking for a few things to tidy the code up.
Well first up, I'd fix up the indentation.
Secondly, I'd not use unit numbers below 10.
INTEGER, PARAMETER :: in_unit = 100
INTEGER, PARAMETER :: out_unit = 101
...
OPEN(unit=in_unit, file=infile, status='OLD")
...
READ(in_unit, *) ...
...
CLOSE(in_unit)
Thirdly, I'd not use GOTOs and labels. You can do that in a loop far easier:
INTEGER :: read_status
DO j = 1, db
...
read_loop : DO
READ(in_unit, *, IOSTAT=read_status) ...
IF (read_status == -1) THEN ! EOF
EXIT read_loop
ELSEIF (read_status /= 0) THEN
CYCLE read_loop
ENDIF
...
END DO read_loop
...
END DO
There are a few dangers in your code, and even in this one above: It can lead to infinite loops. For example, if the opening of infile fails (e.g. the file doesn't exist), it loops back to label 3, but nothing changes, so it will eventually again try to open the same file, and probably have the same error.
Same above: If READ repeatedly fails without advancing, and without the error being an EOF, then the read loop will not terminate.
You have to think about what you want your program to do when something like this happens, and code it in. (For example: Print an error message and STOP if it can't open the file.)
You have a very long FORMAT statement. You can leave it like that, though I'd probably try to shorten it a bit:
103 FORMAT(I7, 2F10.2, F11.2, 4F9.2, F10.4)
This should be the same line, as numbers are usually right-aligned. You can also use strings as a format, so you could also do something like this:
CHARACTER(LEN=*), PARAMETER :: data_out_form = &
'(I7, 2F10.2, F11.2, 4F9.2, F10.4)'
WRITE(*, data_out_form) var1, var2, var3, ...
and again, that's one less label.
I am new to Fortran but I am trying to adapt a Fortran code and I am having trouble doing something which I think is probably quite simple.
I want to adapt a Fortran file called original.f so that it makes an input file called input.inp and populates it with 4 integers calculated earlier in original.f so that input.inp looks like, for example:
&input
A = 1
B = 2
C = 3
D = 4
&end
I know how to write this format:
OPEN(UNIT=10,FILE='input.inp')
WRITE (10,00001) 1,2,3,4
...
...
...
00001 Format (/2x,'&input',
& /2x,'A = ',i4,
& /2x,'B = ',i4,
& /2x,'C = ',i4,
& /2x,'D = ',i4,
& /2x,'&end')
(or something like this that I can fiddle with when I get it working) but I am not sure how to create the input.inp file write this into it and then use this input file.
The input file needs to be used to run an executable called "exec". I would run this in bash as:
./exec < input.inp > output.out
Where output.out contains two arrays called eg(11) and ai(11,6,2) (with dimensions given) like:
eg(1)= 1
eg(2)= 2
...
...
...
eg(11)= 11
ai(1,1,1)= 111
ai(1,2,1)= 121
...
...
...
ai(11,6,2)=1162
Finally I need to read these inputs back into original.f so that they can be used further down in file. I have defined these arrays at the beginning of original.f as:
COMMON /DATA / eg(11),ai(11,6,2)
But I am not sure of the Fortran to read data line by linw from output.out to populate these arrays.
Any help for any of the stages in this process would be hugely appreciated.
Thank you very much
James
Since you have shown how you create the input file, I assume the question is how to read it. The code shows how "a" and "b" can be read from successive lines after skipping the first line. On Windows, if the resulting executable is a.exe, the commands a.exe < data.txt or type data.txt | a.exe will read from data.txt.
program xread
implicit none
character (len=10) :: words(3)
integer, parameter :: iu = 5 ! assuming unit 5 is standard input
integer :: a,b
read (iu,*) ! skip line with &input
read (iu,*) words ! read "a", "=", and "1" into 3 strings
read (words(3),*) a ! read integer from 3rd string
read (iu,*) words ! read "b", "=", and "1" into 3 strings
read (words(3),*) b ! read integer from 3rd string
print*,"a =",a," b =",b
end program xread
If I understand the expanded question correctly, you have to work with an output file, produced by some other code you did not write, with lines like eg(1) = ....
For the simplest case where you know the number of elements and their ordering beforehand, you can simply search each line for the equals sign from behind:
program readme
implicit none
character(100) :: buffer
integer :: i, j, k, pos, eg(11), ai(11,6,2)
do i = 1,11
read*, buffer
pos = index(buffer, '=', back = .true.)
read(buffer(pos+1:), *) eg(i)
enddo
! I have assumed an arbitrary ordering here
do k = 1,2
do i = 1,11
do j = 1,6
read*, buffer
pos = index(buffer, '=', back = .true.)
read(buffer(pos+1:), *) ai(i,j,k)
enddo
enddo
enddo
end program
Assuming here for simplicity that the data are provided to standard input.
Dear All, I am writing a code that writes the out put in multiple files named as 1.dat, 2.dat, ..... Here is my code but it gives some unusual output. May you tell me what is wrong in my code please? Basically I could not get the correct syntax to open multiple files, write on them and close before the next file is opened. Thank you. My Code:
implicit double precision (a-h,o-z),integer(i-n)
dimension b(3300,78805),bb(78805)
character*70,fn
character*80,fnw
nf = 3600 ! NUMBER OF FILES
nj = 360 ! Number of rows in file.
do j = 1, nj
bb(j) = 0.0
end do
c-------!Body program-----------------------------------------------
iout = 0 ! Output Files upto "ns" no.
DO i= 1,nf ! LOOP FOR THE NUMBER OF FILES
if(mod(i,180).eq.0.0) then
open(unit = iout, file = 'formatted')
x = 0.0
do j = 1, nj
bb(j) = sin(x)
write(iout,11) int(x),bb(j)
x = x + 1.0
end do
close(iout)
iout = iout + 1
end if
END DO
11 format(i0,'.dat')
END
So there are a few things not immediately clear about your code, but I think here the most relevant bits are that you want to specify the filename with file = in the open statement, not the formatting, and looping over units with iout is problematic because you'll eventually hit system-defined units for stdin and stdout. Also, with that format line it looks like you're getting ready to create the filename, but you never actually use it.
I'm not sure where you're; going with the mod test, etc, but below is a stripped down version of above which just creates the files ina loop:
program manyfiles
implicit none
character(len=70) :: fn
integer, parameter :: numfiles=40
integer, parameter :: outunit=44
integer :: filenum, j
do filenum=1,numfiles
! build filename -- i.dat
write(fn,fmt='(i0,a)') filenum, '.dat'
! open it with a fixed unit number
open(unit=outunit,file=fn, form='formatted')
! write something
write(outunit, *) filenum
! close it
close(outunit)
enddo
end program manyfiles
In my case, I want the file name have an prefix likedyn_
program manyfiles
implicit none
character(len=70) :: filename
integer, parameter :: numfiles=40
integer, parameter :: outunit=44
integer :: filenum, j
do filenum=1,numfiles
write(filename,'("dyn_",i0,".dat")') filenum
open(unit=outunit,file=filename, form='formatted')
write(outunit, *) filenum
close(outunit)
enddo
end program manyfiles