Fortran: reading file with unknown number of lines - fortran

I am trying to open an ASCII file with unknown number of lines (but fixed number of entries in each line - please see example file below). It goes to the end of the file, but fails after that.
Option 1: Using iostat statement: I get the same error when using either "Use, intrinsic :: iso_fortran_env, Only : iostat_end" or simply treating io as an integer.
Option 2: Using "end= .." option. This is what I have used earlier (F77).
Both approaches should work in principle, but neither does. Any help in fixing this will be greatly appreciated.
Thanks,
Pinaki.
Program:
!=======================================!
program read
!=======================================!
! Use, intrinsic :: iso_fortran_env, Only : iostat_end
implicit none
integer :: i,io,n
n=0
open(10,file='a.dat',status='old',action='read')
!=================================!
! option 1
!=================================!
do
read(10,*,iostat=io)
write(*,*)io
! if (io.eq.iostat_end) exit
if (io.ne.0) exit
n=n+1
write(*,*)'n=',n
enddo
!=================================!
!=================================!
! option 2
!=================================!
do 10 i=1,1000000
read(10,*,end=10)
n=n+1
write(*,*)n
10 continue
!================================!
close(10)
write(*,*)'n=',n
end program read
!======================!
Compiled using "gfortran --std=f2003 -o read.out read.f90"
Error message:
==============================================
dyld: lazy symbol binding failed: Symbol not found: ___emutls_get_address
Referenced from: /usr/local/opt/gcc/lib/gcc/11/libgfortran.5.dylib
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: ___emutls_get_address
Referenced from: /usr/local/opt/gcc/lib/gcc/11/libgfortran.5.dylib
Expected in: /usr/lib/libSystem.B.dylib
Program received signal SIGABRT: Process abort signal.
Backtrace for this error:
#0 0x105364f8e
#1 0x10536419d
#2 0x7fff6eed05fc zsh: abort ./read.out
=====================================================
File I am trying to read:
=====================================================
1 -1.1559859375 2.0399371094 0.1686166667 0.8242152778
2 -1.1618015625 2.1375250000 0.1765231481 0.8105046296
3 -1.1696710937 2.2325417969 0.1860513889 0.7936782407
1 -1.1730312500 2.3271382813 0.1975254630 0.7718773148
2 -1.1767945313 2.3942726563 0.2113162500 0.7446933333
3 -1.1694437500 2.4738000000 0.2281966667 0.7099266667
1 -1.1566140625 2.5312164063 0.2494466667 0.6636841667
2 -1.1293765625 2.5746707031 0.2747766667 0.6066154167
3 -1.0836390625 2.5938144531 0.3026616667 0.5403733333
1 -1.0380632812 2.5721433594 0.3302462121 0.4727159091
=======================================================================

I tested your code, the first option worked. Your second option would also work if you remove the label 10 from do 10 i=1,1000000 and add a end do right before 10 continue. You would also have to add a rewind(10) or close(10); open(10,file='a.dat',status='old',action='read') right before the second option's do loop. But more interestingly, this problem is very easy to solve in Fortran 2008 using the is_iostat_end() intrinsic function that checks for the end-of-file error code occurrence. Here is a modern implementation,
program read
implicit none
integer :: i, n, iostat, fileUnit
n = 0
open(newunit = fileUnit, file = 'a.dat', status = 'old', action = 'read')
do
read(fileUnit,*,iostat = iostat)
if (is_iostat_end(iostat)) exit
write(*,*) iostat
n = n + 1
write(*,*)'n = ', n
end do
close(fileUnit)
end program read
Compile and rune it with the following commands,
gfortran --std=f2008 -o read.out read.f90
./read.out
Even though the problem was trivial, you asked a very nice question by providing a full code, data files, a recipe to compile the code, and the errors encountered. It deserves an upvote.

Related

Debugging access violation error: writing to 2071E05A0 instead of 3071E05A0

Final edit:
Some users on the silverfrost forums directed me very helpfully, to a simplification of the code and a solution.
The issue can be replicated using the following code:
PROGRAM ML14ERROR
INTEGER :: origzn, destzn
INTEGER,PARAMETER :: MXZMA = 1713, LXTZN = 1714, MXAV = 182
INTEGER,PARAMETER :: JTMPREL = 1003, av = 1
REAL(KIND=2) :: RANDOM#
REAL,dimension (1:mxav,lxtzn,lxtzn,JTMPREL:JTMPREL):: znzndaav
DO origzn=1,lxtzn
DO destzn=1,lxtzn
znzndaav(av,origzn,destzn,JTMPREL) = RANDOM#()
END DO
END DO
DO origzn=1,mxzma
DO destzn=1,mxzma
! This is where the error occurs
znzndaav(av,origzn,lxtzn,JTMPREL)=
$ znzndaav(av,origzn,lxtzn,JTMPREL)+
$ znzndaav(av,origzn,destzn,JTMPREL)
ENDDO
ENDDO
WRITE(6,*)'No errors'
END PROGRAM
The issue only arises when MXAV>182, which suggests a memory issue. Indeed, multiplying out the dimensions: 183 * 1714 * 1714 * 4 yields >2GB, exceeding the stack size.
A solution would be to use the heap as follows (Fortan 95):
PROGRAM ML14ERROR
INTEGER :: origzn, destzn
INTEGER,PARAMETER :: MXZMA = 1713, LXTZN = 1714, MXAV = 191
INTEGER,PARAMETER :: JTMPREL = 1003, av = 1
REAL(KIND=2) :: RANDOM#
REAL,allocatable :: znzndaav(:,:,:,:)
ALLOCATE( znzndaav(1:mxav,lxtzn,lxtzn,JTMPREL:JTMPREL) )
DO origzn=1,lxtzn
DO destzn=1,lxtzn
znzndaav(av,origzn,destzn,JTMPREL) = RANDOM#()
END DO
END DO
DO origzn=1,mxzma
DO destzn=1,mxzma
! This is where the error occurs
znzndaav(av,origzn,lxtzn,JTMPREL)= &
& znzndaav(av,origzn,lxtzn,JTMPREL)+ &
& znzndaav(av,origzn,destzn,JTMPREL)
ENDDO
ENDDO
DEALLOCATE(znzndaav)
WRITE(6,*)'No errors'
END PROGRAM
Once we do this, we can allocate more than 2GB and the array works fine. The program this small section of code stems from is a few years old, and we've only just now run into the issue because a model we've built is many times larger than any before. As Fortran 77 doesn't allow ALLOCATABLE arrays, we must either reduce stack usage, or port the code - or seek another optimisation.
Edited to add:
I have now put together a git repo which contains reproducible code.
Overview
I have a program that works fine when compiled to 32-bit, but presents an access violation error when compiled and run in 64-bit.
I'm using the Silverfrost Fortran compiler, FTN95 v8.51, though this issue occurs using v8.40 and v8.50.
Sample code
! .\relocmon.inc
INTEGER JTMPREL
PARAMETER(JTMPREL=1003)
REAL znda(lxtzn,JTMPREL:JTMPREL)
REAL zndaav(1:mxav,lxtzn,JTMPREL:JTMPREL)
REAL,dimension (lxtzn,lxtzn,JTMPREL:JTMPREL) :: znznda
REAL mlrlsum(lxtzn,lxtzn)
REAL,dimension (1:mxav,lxtzn,lxtzn,JTMPREL:JTMPREL):: znzndaav
COMMON /DDMON/ znda, znznda, mlrlsum,znzndaav, zndaav
! EOF .\relocmon.inc
! .\relocmon.inc with values
INTEGER JTMPREL
PARAMETER(JTMPREL=1003)
REAL znda(1714,JTMPREL:JTMPREL)
REAL zndaav(1:191,1714,JTMPREL:JTMPREL)
REAL,dimension (1714,1714,JTMPREL:JTMPREL) :: znznda
REAL mlrlsum(1714,1714)
REAL,dimension (1:191,1714,1714,JTMPREL:JTMPREL):: znzndaav
COMMON /DDMON/ znda, znznda, mlrlsum,znzndaav, zndaav
! EOF .\relocmon.inc
! .\main.for
INCLUDE 'relocmon.inc'
REAL,save,dimension(lxtzn,lxtzn,mxav) :: ddfuncval
DO origzn=1,mxzma
IF( zonedef(origzn,JZUSE) )THEN
DO destzn=1,mxzma
IF (zonedef(destzn,JZUSE)) THEN
znznda(origzn,destzn,JTMPREL)=znda(destzn,JTMPREL)*
$ ddfuncval(origzn,destzn,av)
znznda(origzn,lxtzn,JTMPREL)=znznda(origzn,lxtzn,JTMPREL)
$ +znznda(origzn,destzn,JTMPREL)
znzndaav(av,origzn,destzn,JTMPREL)=zndaav(av,destzn,JTMPREL)*
$ ddfuncval(origzn,destzn,av)
! LINE 309 -- where error occurs
znzndaav(av,origzn,lxtzn,JTMPREL)=
$ znzndaav(av,origzn,lxtzn,JTMPREL)
$ +znzndaav(av,origzn,destzn,JTMPREL)
ENDIF
ENDDO
ENDIF
ENDDO
! EOF .\main.for
NB the function zonedef simply checks that a zone is valid for the calculation we want to undertake. This function returns a logical.
Debugging
As I mentioned initially, the 32-bit compiled version of this program works fine. When attempting to run the 64-bit version, the output of the first loop is this:
from sdbg64.exe:
Error: Access Violation reading address
0x00000002071E05A0
main.for: 309
write exception to file:
Access violation (c0000005) at address 43a1f4
Within file ml14.exe
in main in line 309, at address 2b84
RAX = 0000000000000001 RBX = 000000027fff704c RCX = 000000000285e6b8 RDX = 00000002802296cc
RBP = 0000000000400000 RSI = 000000029ba3ad6c RDI = 0000000307695374 RSP = 000000000285be70
R8 = 0000000307695374 R9 = 00000002ffff5040 R10 = 000000029ba3ad6c R11 = 000000030731f0dc
R12 = 000000027fff5584 R13 = 00000002802296cc R14 = 000000028169f3ec R15 = 0000000281660928
43a1f4) addss XMM11,[85b401b4++R14]
For the rest of this... please bear with me. I'm not a trained software engineer or fortran developer by any stretch, so I'm stabbing in the dark a little to troubleshoot.
The value for ZNZNDAAV(1,337,337,1003) is 2.241640, and this is being added to ZNZNDAAV(1,337,1714,1003). This tallies with register XMM11 as detailed in the exception output. This value is at address 000000029BA3BD60. The other value is at address 00000003071E05A0.
IIUC, in relocmon.inc we're setting COMMON /DDMON/ to contain the dimensioned array znzndaav, so if the software were working nominally, the address of the value in question would be within the /DDMON/ block. The address range for /DDMON/ is z'000000027FFF6040' - z'0000000307421150'. If my logic is correct, the violation occurs outside of this block.
It appears to me that the program is attempting to write to 00000002071E05A0 when it should be using 00000003071E05A0.
Can anyone help me determine why this would be the case? There appears to be something systematic about it - could it be mere coincidence?

How to fix "Segmentation fault" in fortran program

I wrote this program that reads daily gridded climate model data (6 variables) from a file and uses it in further calculations. When running the pgm for a relatively short period (e.g. 5 years) it works fine, but when I want to run it for the required 30 year period I get a "Segmentation fault".
System description: Lenovo Thinkpad with Core i7 vPro with Windows 10 Pro
Program run in Fedora (64-bit) inside Oracle VM VirtualBox
After commenting out everything and checking section-by-section I found that:
everything works fine for 30 years as long as it reads 4 variables only
as soon as the 5th or 6th variable is added, the problem creeps in
alternatively, I can run it with all 6 variables but then it only works for a shorter analysis period (e.g. 22 years)
So the problem might lie with:
the statement: recl=AX*AY*4 which I borrowed from another pgm, yet changing the 4 doesn't fix it
the system I'm running the pgm on
I have tried the "ulimit -s unlimited" command suggested elsewhere, but only get the response "cannot modify limit: Operation not permitted".
File = par_query.h
integer AX,AY,startyr,endyr,AT
character pperiod*9,GCM*4
parameter(AX=162,AY=162) ! dim of GCM array
parameter(startyr=1961,endyr=1990,AT=endyr-startyr+1,
& pperiod="1961_1990")
parameter(GCM='ukmo')
File = query.f
program query
!# A FORTRAN program that reads global climate model (GCM) data to
!# be used in further calculations
!# uses parameter file: par_query.h
!# compile as: gfortran -c -mcmodel=large query.f
!# gfortran query.o
!# then run: ./a.out
! Declarations ***************************************************
implicit none
include 'par_query.h' ! parameter file
integer :: i,j,k,m,n,nn,leapa,leapb,leapc,leapn,rec1,rec2,rec3,
& rec4,rec5,rec6
integer, dimension(12) :: mdays
real :: ydays,nyears
real, dimension(AX,AY,31,12,AT) :: tmax_d,tmin_d,rain_d,rhmax_d,
& rhmin_d,u10_d
character :: ipath*43,fname1*5,fname2*3,nname*14,yyear*4,mmonth*2,
& ext1*4
! Data statements and defining characters ************************
data mdays/31,28,31,30,31,30,31,31,30,31,30,31/ ! Days in month
ydays=365. ! Days in year
nyears=real(AT) ! Analysis period (in years)
ipath="/run/media/stephan/SS_Elements/CCAM_africa/" ! Path to
! input data directory
fname1="ccam_" ! Folder where data is located #1
fname2="_b/" ! Folder where data is located #2
nname="ccam_africa_b." ! Input filename (generic part)
ext1=".dat"
leapa=0
leapb=0
leapc=0
leapn=0
! Read daily data from GCM ***************************************
do n=startyr,endyr ! Start looping through years --------------
write(yyear,'(i4.4)')n
nn=n-startyr+1
! Test for leap years
leapa=mod(n,4)
leapb=mod(n,100)
leapc=mod(n,400)
if (leapa==0) then
if (leapb==0) then
if (leapc==0) then
leapn=1
else
leapn=0
endif
else
leapn=1
endif
else
leapn=0
endif
if (leapn==1) then
mdays(2)=29
ydays=366.
else
mdays(2)=28
ydays=365.
endif
do m=1,12 ! Start looping through months --------------------
write(mmonth,'(i2.2)')m
! Reading daily data from file
print*,"Reading data for ",n,mmonth
open(101,file=ipath//fname1//GCM//fname2//nname//GCM//"."//
& yyear//mmonth//ext1,access='direct',recl=AX*AY*4)
do k=1,mdays(m) ! Start looping through days --------------
rec1=(k-1)*6+1
rec2=(k-1)*6+2
rec3=(k-1)*6+3
rec4=(k-1)*6+4
rec5=(k-1)*6+5
rec6=(k-1)*6+6
read(101,rec=rec1)((tmax_d(i,j,k,m,nn),i=1,AX),j=1,AY)
read(101,rec=rec2)((tmin_d(i,j,k,m,nn),i=1,AX),j=1,AY)
read(101,rec=rec3)((rain_d(i,j,k,m,nn),i=1,AX),j=1,AY)
read(101,rec=rec4)((rhmax_d(i,j,k,m,nn),i=1,AX),j=1,AY)
read(101,rec=rec5)((rhmin_d(i,j,k,m,nn),i=1,AX),j=1,AY)
read(101,rec=rec6)((u10_d(i,j,k,m,nn),i=1,AX),j=1,AY)
enddo ! k-loop (days) ends --------------------------------
close(101)
enddo ! m-loop (months) ends --------------------------------
enddo ! n-loop (years) ends -----------------------------------
end program query

Read large file in Fortran

The following code succeeded in reading txt files of 30x30 (7Kb) and 100x100 (69Kb) matrices of 1 and 0 values. Here, there is an example of a 3x3 matrix. The txt file "MyTxtFile.txt" reads as follows.
1 0 1
1 0 0
1 1 1
The code is presented bellow.
Integer,dimension(3,3):: ReadMatrix
Integer::row,col
open(12, file="MyTxtFile.txt")
do row=1,3
read(12,*) (ReadMatrix(row,col),col=1,3)
end do
print*, ReadMatrix (1,:) !Print first row to check
As an output, I obtain the following
1 0 1
However, when I tried with a 1000x1000 matrix (3186 Kb) an error appeared saying the following:
At line 8 of file C:\Users\...\...\... (unit=12, file='Matrix1000.txt')
Fortran runtime error: End of file
What is actually happening and what are my options?

Solve error: Fortran runtime error: Bad integer for item 0 in list input

I have recently changed my f90 editor to CodeBlocks for Mac OS X, and when I try to open a file located in the project folder to read the data, the next error message appears on screen when the code is run:
Fortran runtime error: Bad integer for item 0 in list input
I have introduced the same code I used to write in Windows 7 using the intel compiler for fortran and Visual Studio.
The code itself is:
subroutine read_input_data
use input_data
implicit none
integer i,j
open(UNIT=5, FILE='lifting_line_input_data.txt', STATUS='old', FORM='formatted', ACCESS='sequential')
read(5,*) C
read(5,*) U
read(5,*) alpha
read(5,*) rho
read(5,*) wake_length
read(5,*) wake_eps
read(5,*) n_chord
read(5,*) n_twist
if (n_chord .GE. n_twist ) then
i = n_chord
else
i = n_twist
end if
allocate(chord_twist(5,i))
do j = 1, i
read(5,*) chord_twist(:,j)
end do
close(5)
end subroutine read_input_data
Could you help me to solve this problem? Thank you very much.
PD. the data file is obtain from an Excel sheet saved as a .txt delimited by tabulations
! LIFTING-LINE WING
! Number of panels
6
! Free stream speed [m/s]
50
! Angle of attack [rad]
0.15
! Air density [kg/m^3]
1.225
! Wake length [m]
100
! Convergence parameter
0.01
! Number of data points given for the chord distribution
2
! Number of data points given for the twist distribution
2
! Y coord [m] ! X_LE [m] ! X_TE [m] ! Y coord [m] ! Twist [rad]
0 0 2 0 0
10 0 0.5 10 0.052359878
PD2. I have change the format of the .txt file to make it equal to the input files I had used in Visual Studio. Now the file is:
6 ! Number of panels
50 ! Free stream speed [m/s]
0.15 ! Angle of attack [rad]
1.225 ! Air density [kg/m^3]
100 ! Wake length [m]
0.01 ! Convergence parameter
2 ! Number of data points given for the chord distribution
2 ! Number of data points given for the twist distribution
0 0 2 0 0 ! Y coord [m] ! X_LE [m] ! X_TE [m] ! Y coord [m] !Twist [rad]
10 0 0.5 10 0.052359878
And now the error given at the terminal is that the file is not found. As I am a beginner in CodeBlocks, I will explain what I have done step by step because I do not find where I am wrong and I am starting to get desperate:
New Project -> Fortran application -> I indicate where I want to create the project file.
I remove the main.f95 file and I add the .f90 file with the code.
I write the code.
I save the .txt file in the same folder than all the files of the Project.
When I run the code it appears the error message of file not found.
The code is:
!************************************************
subroutine read_input_data
use input_data
implicit none
integer i,j
open(UNIT=10, FILE='lifting_line_wing_input.txt', STATUS='old', ACCESS='sequential')
read(10,*) C
read(10,*) U
read(10,*) alpha
read(10,*) rho
read(10,*) wake_length
read(10,*) wake_eps
read(10,*) n_chord
read(10,*) n_twist
if (n_chord .GE. n_twist ) then
i = n_chord
else
i = n_twist
end if
allocate(chord_twist(5,i))
do j = 1, i
read(10,*) chord_twist(:,j)
end do
close(10)
end subroutine read_input_data
!************************************************
Thank you very much for your time and help
This looks like your old system did something non-standard with exclamation marks on list-directed input.
Try reformatting your input data like
6 / number of panels
(the slash will terminate the READ).
i don't believe any fortran compiler ever automagically handled those comments.
If you want to read this file the way it is, one approach is to make each read handle the error, eg,
integer ios
ios = 1
do while(ios.ne.0)
read(unit,*,iostat=ios)c
end do
ios=1
do while(ios.ne.0)
read(unit,*,iostat=ios)u
end do
etc..
if its a one-off you could just edit the file and delete all the comments as well.

Fortran95 -- Reading from a formatted text file

I need to read some values from a table. These are the first five rows, to give you some idea of what it should look like:
1 + 3 98 96 1
2 + 337 2799 2463 1
3 + 2801 3733 933 1
4 + 3734 5020 1287 1
5 + 5234 5530 297 1
My interest is in the first four columns of each row. I need to read these into arrays. I used the following code:
program ----
implicit none
integer, parameter :: totbases = 4639675, totgenes = 4395
integer :: codtot, ks
integer, dimension(totgenes) :: ngene, lend, rend
character :: genome*4639675, sign*4
open(1,file='e_coli_g_info')
open(2,file='e_coli_g_str')
do ks = 1, totgenes
read(1,100) ngene(ks),sign(ks:ks),lend(ks), rend(ks)
end do
100 format(1x,i4,8x,a1, 2(5x,i7), 22x)
do ks = 1, 100
write(*,*) ngene(ks), sign(ks:ks),lend(ks), rend(ks)
end do
end program
The loop at the end of the program is to print the first hundred entries to test that they are being read correctly. The problem is that I am getting this garbage (the fourth row is the problem):
1 + 3 757934891
2 + 337 724249387
3 + 2801 757803819
4 + 3734 757803819
5 + 5234 757935405
Clearly, the fourth column is way off. In fact, I cannot find these values anywhere in the file that I am reading from. I am using the gfortran compiler for Ubuntu 12.04. I would greatly appreciate if somebody would point me in the right direction. I'm sure it's likely that I'm missing something very obvious because I'm new at Fortran.
Fortran formats are (traditionally, there's some newer stuff that I won't go into here) fixed format, that is, they are best suited for file formats with fixed columns. I.e. column N always starts at character position M, no ifs or buts. If your file format is more "free format"-like, that is, columns are separated by whitespace, it's often easier and more robust to read data using list formatting. That is, try to do your read loop as
do ks = 1, totgenes
read(1, *) ngene(ks), sign(ks:ks), lend(ks), rend(ks)
end do
Also, as a general advice, when opening your own files, start from unit 10 and go upwards from there. Fortran implementations typically use some of the low-numbered units for standard input, output, and error (a common choice is units 1, 5, and 6). You probably don't want to redirect those.
PS 2: I haven't tried your code, but it seems that you have a bounds overflow in the sign variable. It's declared of length 4, but then you assign to index ks which goes all the way up to totgenes. As you're using gfortran on Ubuntu 12.04 (that is, gfortran 4.6), when developing compile with options "-O1 -Wall -g -fcheck=all"