Memory allocation error when reading multiple files - fortran

Question

I'm trying to open 6 different files (at least) and then read the number of lines in each file, which should be about 20,000 lines each. I've read some posts on this forum about how to do that as I'm a newbie, and I've tried to implement it for my purposes.
I can do this individually without any problem, but when I try to read in all the files, I get an error message. I get either the "Killed: 9" error message or a malloc error:
malloc: *** mach_vm_map(size=63032829050880) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
What does this error mean regarding memory allocation? What am I doing wrong? How do I go about correcting this?
PROGRAM X
IMPLICIT NONE
INTEGER :: J,IO,NFILES,NLINES
CHARACTER (LEN=128) :: FILENAME
NFILES = 6
NLINES = 0
DO J = 0,NFILES-1
WRITE(FILENAME,'(A,I7.7,A)') 'data_',J*200,'.txt'
OPEN(1,FILE='FILENAME',FORM='FORMATTED')
DO
READ(1,*,IOSTAT=IO)
IF (IO/=0) EXIT
NLINES = NLINES + 1
END DO
WRITE(*,*) NLINES
CLOSE(1)
END DO
END PROGRAM X
I am using gfortran to compile.
UPDATE
I created 6 test files, data_0000000.txt, data_0000200.txt, ..., data_0001000.txt, each with less than 10 lines where there are less than 100 characters in each line. Unfortunately, I get the same error.

Answer 1

Obligatory disclaimer: If you just want to know the number of lines in a file, use wc -l <filename>. Don't reinvent the wheel if you don't have to.
I write this not necessarily because I think you didn't know that, but because someone else might come along, and think they need to write their own program to get the number of lines of files.
As for your question: I don't know why you get a malloc error. Maybe tell us which compiler and system you're using (including versions)? That said, there are three things that I noticed when reading your code:
You create a variable FILENAME, but then you don't use it. You're quoting it: FILE='FILENAME' which means that the open command looks for a file literally called FILENAME, not for a file with the name stored in the variable FILENAME. Remove the quotes:
OPEN(1, FILENAME=FILENAME, FORM='FORMATTED')
You use the unit number 1 -- that is dangerous. Different version of Fortran use specific unit numbers for specific uses. Use a handle far larger (at least 10, or more), or, even better, use the newunit descriptor in the open statement:
INTEGER :: u
OPEN(NEWUNIT=u, FILE=FILENAME, ACTION='READ', FORM='FORMATTED')
READ(u, *, IOSTAT=IO)
CLOSE(u)
You're not resetting the NLINES variable to 0 between files. The program will print a cumulative sum, not the number of lines for each file directly.

Answer 2

Adding to #chw21's response, if your sole purpose is to count the number of records (lines) in a file, here is a modular solution and a test program along with it (the counting is done inside subroutine getNumRecordInFile()):
module NumRecord_mod
implicit none
type :: Err_type
logical :: occurred = .false.
integer :: stat = -huge(0)
character(:), allocatable :: msg
end type Err_type
contains
! returns the number of lines in a file.
subroutine getNumRecordInFile(filePath,numRecord,Err)
implicit none
character(len=*), intent(in) :: filePath
integer, intent(out) :: numRecord
type(Err_type), intent(out) :: Err
character(len=8) :: record
integer :: fileUnit
logical :: fileExists, isOpen
integer :: iostat
character(*), parameter :: PROCEDURE_NAME = "#getNumRecordInFile()"
Err%occurred = .false.
Err%msg = ""
! Check if file exists
inquire( file=filePath, exist=fileExists, opened=isOpen, number=fileUnit, iostat=Err%stat )
if (Err%stat/=0) then
Err%occurred = .true.
Err%msg = PROCEDURE_NAME // ": Error occurred while inquiring the status of file='" // filePath // "'."
return
end if
if (.not.fileExists) then
Err%occurred = .true.
Err%msg = PROCEDURE_NAME // ": The input file='" // filePath // "' does not exist."
return
end if
if (isOpen) close(unit=fileUnit,iostat=Err%stat)
if (Err%stat>0) then
Err%occurred = .true.
Err%msg = PROCEDURE_NAME // ": Error occurred while attempting to close the open input file='" // filePath // "'."
return
end if
open(newunit=fileUnit,file=filePath,status="old",iostat=Err%stat)
if (Err%stat>0) then
Err%occurred = .true.
Err%msg = PROCEDURE_NAME // ": Error occurred while opening input file='" // filePath // "'."
return
end if
numRecord = 0
do
read(fileUnit,'(A)',iostat=iostat) record
if(iostat==0) then
numRecord = numRecord + 1
cycle
elseif(is_iostat_end(iostat)) then
exit
else
Err%occurred = .true.
Err%stat = iostat
Err%msg = PROCEDURE_NAME // ": Error occurred while reading input file='" // filePath // "'."
return
end if
end do
close(fileUnit,iostat=Err%stat)
if (Err%stat>0) then
Err%occurred = .true.
Err%msg = PROCEDURE_NAME // ": Error occurred while attempting to close the open input file='" // &
filePath // "' after counting the number of records in file."
return
end if
end subroutine getNumRecordInFile
end module NumRecord_mod
program test_numRecord
use NumRecord_mod
implicit none
type(Err_type) :: Err
integer :: numRecord
character(:), allocatable :: filePath
filePath = "main.f95"
call getNumRecordInFile(filePath=filePath,numRecord=numRecord,Err=Err)
if (Err%occurred) then
write(*,*) Err%msg
write(*,*) Err%stat
error stop
else
write(*,*) "Total number of records in file='" // filePath // "': ", numRecord
end if
end program test_numRecord
Now if you put this code in a file named "main.f95" and compile it under Fortran 2008 standard, then it should output the number of lines in your "main.f95" file, which should be something like the following:
$gfortran -std=f2008 *.f95 -o main
$main
Total number of records in file='main.f95': 98
For testing, you can simply copy paste the entire code in the online Fortran compiler here: https://www.tutorialspoint.com/compile_fortran_online.php
But keep in mind to change the compile option -std=f95 to -std=f2008 by going to Project -> Compile Options before executing the code.

Answer 3

Try this change (declaration and read line), there was no variable specified where the line contents shall go, here insert now dummy...
character(len=1000) :: dummy
...
READ(u, '(a)' , IOSTAT=IO) dummy
....