Reading more data files FORTRAN90 - fortran

I have a question about f90 as title says.
I have many .dat files (say 15 files each with 2 columns and 2000 rows) that i need to work with.
I need to save all the data in a unique matrix (that will have 30 columns and 2000 rows).
But when I use the instruction READ(21,*) MATRIX during execution it says
Fortran runtime error: End of file
I think it's because the dimension of the matrix is different from the dimension of the single data file.
Is there a way to make it work?
And is it possible to open all the data files maybe in a DO cycle w/o repeating the OPEN statement a large number of times?
Here is the code
program expected
REAL*8, ALLOCATABLE, DIMENSION ( :, :) :: dati
OPEN (unit=20, status='OLD', file='PRINT_07.CSV', form='FORMATTED')
OPEN (unit=21, status='OLD', file='PRINT_08.CSV', form='FORMATTED')
OPEN (unit=22, status='OLD', file='PRINT_09.CSV', form='FORMATTED')
OPEN (unit=23, status='OLD', file='PRINT_10.CSV', form='FORMATTED')
OPEN (unit=24, status='OLD', file='PRINT_11.CSV', form='FORMATTED')
OPEN (unit=25, status='OLD', file='PRINT_12.CSV', form='FORMATTED')
OPEN (unit=26, status='OLD', file='PRINT_13.CSV', form='FORMATTED')
OPEN (unit=27, status='OLD', file='PRINT_14.CSV', form='FORMATTED')
OPEN (unit=28, status='OLD', file='PRINT_15.CSV', form='FORMATTED')
OPEN (unit=29, status='OLD', file='PRINT_16.CSV', form='FORMATTED')
OPEN (unit=30, status='OLD', file='PRINT_17.CSV', form='FORMATTED')
OPEN (unit=31, status='OLD', file='PRINT_18.CSV', form='FORMATTED')
OPEN (unit=32, status='OLD', file='PRINT_19.CSV', form='FORMATTED')
OPEN (unit=33, status='OLD', file='PRINT_20.CSV', form='FORMATTED')
OPEN (unit=34, status='OLD', file='PRINT_21.CSV', form='FORMATTED')
ALLOCATE ( dati(30,2000) )
READ(20,*) dati
end program
In a simpler program i've solved the data importing in this way. But i still have the problem of opening multiple data files at once.
program mats
REAL, ALLOCATABLE, DIMENSION (:,:) :: mat
OPEN (UNIT=21, STATUS='OLD', file='mat1.dat', form='FORMATTED')
OPEN (UNIT=22, STATUS='OLD', file='mat2.dat', form='FORMATTED')
OPEN (UNIT=23, STATUS='UNKNOWN', file='mat.dat', form='FORMATTED')
DO i=1,2
OPEN (UNIT=20+i, STATUS='OLD', file= 'mat'i'.dat', form='FORMATTED')
END DO
ALLOCATE ( mat(5,2) )
mat=0
PRINT*, mat
READ(21,*) mat(1:3,1:2)
READ(22,*) mat(4:5,1:2)
PRINT*, mat
end program

Here's a sketch to give you a concept on how to deal with multiple files in a more elegant manner. It's not exactly a solution to your problem nor tested -- it shows some concepts.
character (len=80) :: filename
do i=7, 20
write (filename, '( "PRINT_", I2.2, ".CSV" )' ) i
open ( unit=55,file=filename, status="old", action="read", form="formattted" )
read (55, datai (i,:) )
close (unit=55)
end do

Related

Fortran ftell returning to wrong position

Here is a snippet of simple code that reads line from file, then returns to previous position and re-reads same line:
program main
implicit none
integer :: unit, pos, stat
character(128) :: buffer
! Open file as formatted stream
open( NEWUNIT=unit, FILE="data.txt", ACCESS="stream", FORM="formatted", STATUS="old", ACTION="read", IOSTAT=stat )
if ( stat /= 0 ) error stop
! Skip 2 lines
read (unit,*) buffer
read (unit,*) buffer
! Store position
pos = ftell(unit)
! Read & write next line
read (unit,*) buffer
write (*,*) "buffer=", trim(buffer)
! Return to previous position
call fseek(unit,pos,0)
! pos = ftell(unit) ! <-- ?!
! Read & write next line (should be same output)
read (unit,*) buffer
write (*,*) "buffer=", trim(buffer)
! Close file stream
close (UNIT=unit)
end program main
The "data.txt" is just a dummy file with 4 lines:
1
2
3
4
Now when I compile the snippet (gfortran 9.3.0) and run it, I get an answer:
buffer=3
buffer=4
which is wrong, as they should be same. More interestingly when I add an additional ftell (commented line in the snippet) after 'fseek' I get correct answer:
buffer=3
buffer=3
Any idea why it does that? or am I using ftell and fseek incorrectly?
gfortran's documentation for FTELL and FSEEK clearly states that these routines are provided for backwards compatibility with g77. As your code is using NEWUNIT, ERROR STOP, and STREAM access, you are not compiling old moldy code. You ought to use standard conforming methods as pointed out by #Vladimir.
A quick debugging session shows that FTELL and FSEEK are using a 0-based reference for the file position while the inquire method of modern Fortran is 1 based. There could be an off-by-one type bug in gfortran, but as FTELL and FSEEK are for backwards compatibility with g77 (an unmaintained 15+ year old compiler), someone would need to do some code spelunking to determine the intended behavior. I suspect none of the current, active, gfortran developers care enough to explore the problem. So, to fix your problem
program main
implicit none
integer pos, stat, unit
character(128) buffer
! Open file as formatted stream
open(NEWUNIT=unit, FILE="data.txt", ACCESS="stream", FORM="formatted", &
& STATUS="old", ACTION="read", IOSTAT=stat)
if (stat /= 0) stop
! Skip 2 lines
read (unit,*) buffer
read (unit,*) buffer
! Store position
inquire(unit, pos=pos)
! Read & write next line
read (unit,*) buffer
write (*,*) "buffer=", trim(buffer)
! Reread & write line (should be same output)
read (unit,*,pos=pos) buffer
write (*,*) "buffer=", trim(buffer)
! Close file stream
close (UNIT=unit)
end program main

Fortran code produces runtime error 'operation not supported' when attempting to open a text file

I am trying to run a piece of fortran code written in f95. I have compiled it using gfortran in Ubuntu.
In the code there is a command to read in a text file. When I run it, it gives me the following error:
Fortran runtime error: Cannot open file 'input_parameters.txt': Operation not supported
This is the code up until the point that we attempt to read the text file:
program LSmodel
implicit none !this is a fortran thing that means that all variables that start with i,j,k,l,m,n are integers.
real :: sec,ran,gasdev ! random generator variables
real :: x,y,z,u,v,w,ut,vt,wt,t,dt ! simulation variables
real :: wg ! seed parametes
real :: Um,sigma_u,sigma_v,sigma_w,uw ! wind statistics variables
real :: dvaru_dz,dvarv_dz,dvarw_dz,duw_dz ! wind statistics variables
real :: dissip_m,TL ! vector over the range of ustars
real :: zs,zg,zmax ! release height & boundaries
real :: Ainv,C0inv ! inverse parameters
real :: C0,A,b,au,av,aw,dt_on_TL ! LS model parameters
real :: dz_max,dt_max ! time step limit
real :: CT,beta ! Crossing Trajectories correction
real :: C_chi,chi,TKE,T_chi,omega ! DI parameters
real :: a_ln,b_ln,sigma_chi,dissip_s ! DI parameters
real :: rhop,rho,r,g,gt,Re,AIP,Cd,nu ! IP parameters
real :: up,vp,wp,upt,vpt,wpt,vr,dt_ip,alpha ! IP parameters
real :: keepseed, maxheight
integer :: seed ! random generator variables, keepseed decides whether to keep the same seed or not for comparison of simulation
integer :: pnum, traj_exit ! simulation parameters. traj_exit counts the number of particles that have exited from the topo f the wind flow.
integer :: i,j,jj,n,ii ! counting parameters
integer :: n_ip,IP=1 ! IP parameters
character(len=80) :: filename, wgchar, foldername
real, allocatable,dimension(:) :: z_vec,Um_vec,sigma_u_vec,sigma_v_vec,sigma_w_vec,uw_vec
real, allocatable,dimension(:) :: dvaru_dz_vec,dvarv_dz_vec,dvarw_dz_vec,duw_dz_vec,dissip_m_vec
! input
open (23,file='input_parameters.txt') !opening a file for the input parameters....
read (23, *) x,C0,wg,zs,zg,beta,dt_on_TL,y,sigma_chi,C_chi,r,rhop,alpha,rho,nu, keepseed, foldername
close(23)
I am running Ubuntu 18.04.2 LTS.
An update - I have found (I believe) the reason this code was not working, although I don't know why.
The folder was in a network drive, not on my local computer. Once I moved the folder onto my local computer, I stopped getting this error.

Reading large HDF using with Fortran

I have some HDF data that has been created with PyTables. This data is very large, an array of 3973850000 x 8 double precision values, but with PyTables compression this can easily be stored.
I want to access this data using Fortran. I do,
PROGRAM HDF_READ
USE HDF
IMPLICIT NONE
CHARACTER(LEN=100), PARAMETER :: filename = 'example.h5'
CHARACTER(LEN=100), PARAMETER :: dsetname = 'example_dset.h5'
INTEGER error
INTEGER(HID_T) :: file_id
INTEGER(HID_T) :: dset_id
INTEGER(HID_T) :: space_id
INTEGER(HSIZE_T), DIMENSION(2) :: data_dims, max_dims
DOUBLE PRECISION, DIMENSION(:,:), ALLOCATABLE :: dset_data
!Initialize Fortran interface
CALL h5open_f(error)
!Open an existing file
CALL h5open_f(filename, H5F_ACC_RDONLY_F, file_id,error)
END PROGRAM HDF_READ
!Open a dataset
CALL h5dopen_f(file_id, dsetname, dset_id, error)
!Get dataspace ID
CALL h5dget_space_f(dset_id, space_id, error)
!Get dataspace dims
CALL h5sget_simple_extent_dims_f(space_id, data_dims,max_dims, error)
!Create array to read into
ALLOCATE(dset_data(data_dims(1), data_dims(2)))
!Get the data
CALL h5dread_f(dset_id, H5T_NATIVE_DOUBLE, dset_data, data_dims,error)
However, this creates an obvious problem, in that the array cannot be allocated to such a large size with double precision floats as it becomes greater than the system memory.
What is the best method for accessing this data? My current thoughts are for some sort of chunking method? Or is there a way to store the array on disk? Does HDF have methods for dealing with large data like this - I have read around but can find nothing pertaining to my case.

How can I read data from a text file and save/write parts/variables to different text files?

I'm fairly new to Fortran so this might be a naive question. I would like to read a huge .txt file with a # of rows=540001. The data are from a sonic anemometer with measurements of velocity and temperature as u,v,w,T at five heights. Here are the first few lines of the file:
"2011-07-10 09:30:00",9838,1.132,2.30225,-0.5635,29.18585,0.30275,0.689,-0.01125,29.67004,0.2165,-0.25475,0.12725,29.8923,0.51425,3.0405,-0.58375,29.5242,-0.0085,3.6235,-0.65175,29.61972,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
"2011-07-10 09:30:00.05",9839,-0.21325,3.22775,-0.17,29.10953,0.33925,0.6867501,-0.0015,29.67874,0.1715,-0.196,0.1235,29.8923,0.035,2.6915,-0.3845,29.82806,-0.102,3.5505,-0.15825,29.61795,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
"2011-07-10 09:30:00.1",9840,0.403,3.1195,-0.37175,29.22574,0.06550001,0.6655,0.1275,29.76208,0.196,-0.2,0.1,29.901,0.16225,2.31525,-0.5975,29.69263,0.24175,3.11925,-0.3725,29.57977,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
etc...
I would like to save/write the matrices u(5,540001),v(5,540001),w(5,540001), and T(5,540001) so that I can do some calculations and plots. Here is what I am using:
PROGRAM READ_MAIZE
IMPLICIT NONE
REAL,DIMENSION(:,:),Allocatable :: u_r, v_r, w_r, T_r
CHARACTER(len=*) :: fname
fname='FINALDATA.txt'
open(unit=1,file=fname,status='old',action='read')
do i=1,540001
READ(1,*)timestamp(i),count(i),u_r(5,i),v_r(5,i),w_r(5,i), &
T_r(5,i),u_r(2,i),v_r(2,i),w_r(2,i),T_r(2,i), &
u_r(1,i),v_r(1,i),w_r(1,i),T_r(1,i), &
u_r(3,i),v_r(3,i),w_r(3,i),T_r(3,i), &
u_r(4,i),v_r(4,i),w_r(4,i),T_r(4,i),flags(1:20)
end do
close(1)
WRITE(U_maize,'(A,I7.7,A,I7.7,A)'), &
'.txt'
open(11,file=U_maize,status='unknown',action='write')
write(11,'(F20.14)')(u_r)
end
Never mind the order in u_r(5,i) followed by u_r(2,i)... (they just correspond to different heights that are out of order). This is not working.
There's quite a lot going on in your code which makes it hard to understand what you're trying to do in the first place. I have annotated your code below and turned it into something that compiles and produces output. Maybe it'll help.
PROGRAM READ_MAIZE
IMPLICIT NONE ! This means that every variable has to be declared
! explicitly. You don't do that. So I did it for you
REAL,DIMENSION(:,:),Allocatable :: u_r, v_r, w_r, T_r
integer, dimension(:), allocatable :: data_count ! using fortran keywords
! (such as "count") as variables is dangerous and should be avoided
character(len=len("2011-07-10 09:30:00.05")), allocatable :: timestamp(:)
CHARACTER(len=*), parameter :: fname='FINALDATA.txt'
character(len=len("U_XXXXXXX_XXXXXXX.txt")) :: U_maize
integer :: in_unit, out_unit ! Use integer variables for the unit.
! together with newunit keyword, this is
! safer
integer, parameter :: num_records = 3 ! you need to up this number to
! 540001 again
integer :: i
! If you have allocatable arrays, you need to allocate them before you
! can use them
allocate(u_r(5, num_records))
allocate(v_r(5, num_records))
allocate(w_r(5, num_records))
allocate(T_r(5, num_records))
allocate(data_count(num_records))
allocate(timestamp(num_records))
! the "newunit" keyword is a safe way to create a unique unit
! identifier. You should really use this.
open(newunit=in_unit,file=fname,status='old',action='read')
do i=1,num_records
READ(in_unit,*) timestamp(i), data_count(i), &
u_r(5,i),v_r(5,i),w_r(5,i),T_r(5,i), &
u_r(2,i),v_r(2,i),w_r(2,i),T_r(2,i), &
u_r(1,i),v_r(1,i),w_r(1,i),T_r(1,i), &
u_r(3,i),v_r(3,i),w_r(3,i),T_r(3,i), &
u_r(4,i),v_r(4,i),w_r(4,i),T_r(4,i) ! I left out the flags
! since I didn't know what
! that was.
end do
close(in_unit)
! I don't know how the file name should be constructed, except
! that it should end in a .txt, and the format. So I made something up.
write(U_maize, '(A, I7.7, A, I7.7, A)') 'U_', 35, '_', 6, '.txt'
open(newunit=out_unit,file=U_maize,status='unknown',action='write')
! To make it more readable, I tell write to write 5 numbers per row,
! Not sure whether this is what you want.
write(out_unit,'(5(X, F20.14))') u_r
close(out_unit) ! I know it isn't technically needed, but please always
! close files when finished, even if the program terminates anyway.
end program READ_MAIZE ! tell the compiler what you want to end here.

Error: Two main PROGRAMs at (1) and (2)

I'm using the Simply Fortran compiler and when I try to compile I get the error:
prog.f95:35.13:
1 Implicit None
prog.f95:53.65:
2 open (unit=1,file='in',status='OLD') ! opens file with parameters
Error: Two main PROGRAMs at (1) and (2)
I have included only the parts of the code in which the errors occur since the whole thing is quite long. This begins at the very beginning of the program. Let me know if I should include more.
Implicit None
Integer :: i,j,iter
real(8) :: Elow,Ehigh,chi,B_NS,Vbrprof,Neprof,taues
real(8) :: Xcyclave,a
character(8) systemdate
character(10) systemtime
character(5) timezone
integer dateandtime(8)
character(8) systemdate2
character(10) systemtime2
character(5) timezone2
integer dateandtime2(8)
character(len=40) :: infname,outfname,comm
include 'common.f95'
open (unit=1,file='in',status='OLD') ! opens file with parameters
read (1,1) ! comment line
read (1,1) outfname
read (1,*) Elow,Ehigh ! lower and higher energy
read (1,*) Eminf,Emind,Emaxf ! min and max energy for fedd
read (1,*) Rin, Rout ! inner and outer radii
read (1,*) profpar(1) ! for Ne
read (1,*) profpar(2) ! Te in keV
read (1,*) profpar(3) ! for absorption+emission
read (1,*) profpar(4) ! T_bb for neutron star in keV
read (1,*) profpar(5) ! for bulk velocity
read (1,*) profpar(6) ! other parameter for model
read (1,*) profpar(10) ! magnetic moment in 10^27 CGS
1 format (A10)
close (1)
The compiler is probably seeing an END xxx statement in the file common.f95. The file common.f95 is possibly not meant to be used as an INCLUDE file - it may be a program unit in its own right.