I am writing an application in Fortran that dumps a single file containing several arrays at each time step. At the end of the time-step, a new file is opened and data is written to it. Each of these files is created by calling:
! Create a new file
CALL h5fcreate_f(trim(filename), H5F_ACC_TRUNC_F, file_id, error)
and closed by calling the following subroutine close_hdf() at the end of each time step:
subroutine close_hdf()
! Close the file
CALL h5fclose_f(file_id, error)
! Close FORTRAN interface
CALL h5close_f(error)
end subroutine close_hdf
Yet, at the end of the time-step, the file is zero bytes and I have to either kill the executable or wait till all the time steps have finished
to be able to analyze the data. Why are the files not being flushed?
I have also tried to flush the data by calling
CALL h5fflush_f(file_id, H5F_SCOPE_GLOBAL , error)
inside close_hdf() but that did not help either. How do I flush and close/write these files at the end of the time step as soon as I am done with them?
Related
I have some code that writes data to some files, I want to loop over this code to continue opening and writing to the files with different parameters. However, whenever I try this, once the files are created (after the first iteration of the loop) an error message occurs:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
The code would look a little like this:
program main
! premable here
DO a=1,10
call something(a, b, c)
END DO
contains
SUBROUTINE something(a, b, c)
!premable + data manipulation
open(12, FILE = 'file.name', STATUS='UNKNOWN') ! the problem occurs here for
! the second iteration of the loop
! in the main
DO i=.... ! general loop for extracting values from array
write(12, '(4F16.12)') b(:, i)
END DO
close(12)
END SUBROUTINE something
end program main
I have tried using STATUS='OLD' rather than 'UNKNOWN' with empty files pre-created but this hasn't worked either (in fact with this the loop won't even complete the first iteration).
I am using GNU Fortran compiler with Windows and CODE::BLOCKS.
I have a program in Fortran that calculates a file, say, named
wvfunc3d.dat
which I want to visualize with Gnuplot in real time during the execution of my program. After the code that creates this file, I put in my program a string
jret=SYSTEM('gnuplot wf3d.plt')
the script file wf3d.plt has the only string and looks like:
splot 'wvfunc3d.dat' w l
All of this really draws a plot I want to see, but, as is well known, it immediately disappears. I know, there is an option to avoid the closing of the window,
jret=SYSTEM('gnuplot -persist wf3d.plt')
that lets my plot not to disappear, but then the execution of the Fortran program also freezes until I close the window with the graph.
So, I want the plot to persist until I have new data, to be automatically updated after a new call of a command in Fortran, but I also need my program to run calculations! Is there a way to solve this problem? I use Windows XP.
I think you may be able to use EXECUTE_COMMAND_LINE instead of system to achieve what you want. This allows you to include a wait option which when set to .false. allows the fortran code to keep running. You can then add a sleep and reread to your gnuplot script (e.g. sleep 1 and reread) as suggested in this post.
If this doesn't work, you could consider a multi-threaded strategy ( openMP or mpi in fortran). Personally, I usually just run gnuplot at the same time and trigger an update of the plotted data by pressing the a key. I use linux so cannot test it for windows but a minimal example which works for me is,
program gnuplot
implicit none
logical :: gnuplot_open = .false.
integer :: i,t,N,redraw
real(kind(0.d0)),dimension(:),allocatable :: x,y
real(kind(0.d0)),parameter:: pi=4.d0*atan(1.d0)
N = 1000
allocate(x(N),y(N))
redraw = 100
do t = 1,300000
do i=1,N
x(i) = 6.d0*i/N*pi
y(i) = sin(x(i)+t*0.2)
enddo
if (mod(t,redraw) .eq. 0) then
open(1,FILE='./tempout',status='replace')
do i=1,N
write(1,*) x(i),y(i)
enddo
close(1,status='keep')
endif
if (.not. gnuplot_open) then
call execute_command_line('gnuplot --persist plot_tempout', wait=.false.)
gnuplot_open = .true.
endif
enddo
end program gnuplot
and plot_tempout is,
plot 'tempout' u 1:2 w l
pause 0.1
reread
Ed, thank you very much for your thorough reply. I will try to work on it.
Before encountering this problem I was able to easily draw the plots of small enough files using a cycle directly in gnuplot. Something like this:
do for [i=1:100500] {plot 'littldat.dat' w l; pause 3}
that did it well. But when I tried to do this with large file, it was very often caught read by gnuplot in the moment it was not completed yet: I had either a full plot or a plot of a part of my data, and it was not good. Because of this I began to seek for the way to do it by the means of programming language.
Before you answered I finally found a very simply, though not very elegant solution: you write data to a temporary file and then, once it is completed, give it the final name to be read by gnuplot. So, gnuplot reads either old data, or new ones, but never an incomplete file. It results to be something like this in Fortran:
open(1,file='donnees_temp.dat')
write(1,*)x,y,z
close(1)
call rename ('donnees_temp.dat','donnees.dat')
and, in Gnuplot I used a cycle like the one above:
do for [i=1:100500] {splot 'donnees.dat' w l; pause 5}
so it works, and the program executes.
I would like to know if in Fortran it is possible to use just a single command (with options/specifiers) to do the following:
open a file if it exists and append some data
(this can be done with: open(unit=40,file='data.data',Access = 'append',Status='old') but if the file does not exist a runtime error is issued)
create the file if it does not exist and write some data.
I am currently using inquire to check whether the file exist or not but then I still have to use the open statement to append or write data.
As far as I am aware of, the only safe solution is to do the way you're already doing it, using different open statements for the different cases:
program proba
implicit none
logical :: exist
inquire(file="test.txt", exist=exist)
if (exist) then
open(12, file="test.txt", status="old", position="append", action="write")
else
open(12, file="test.txt", status="new", action="write")
end if
write(12, *) "SOME TEXT"
close(12)
end program proba
You may be interested in my Fortran interface library to libc file system calls (modFileSys), which could at least spare you the logical variable and the inquire statement by querying the file status directly:
if (file_exists("test.txt")) then
...
else
...
end if
but of course you can program a similar function easily yourself, and especially it won't save you from the two open statements...
open(61,file='data.txt',action='write',position='append')
write(61,*) 'hey'
close(61)
This will append to an existing file, otherwise create and write. Adding status='unknown' would be equivalent.
if you replace the status from 'old' to 'unknown' then you will not get the run time error if the file exists or now.
Thanks
In open statement add the attribute access as follows;
Open(unit=031,file='filename.dat',form='formatted',status='unknown',access='append')
The above statement will open the file without destroying old data and write command will append the new lines in the file.
The simplest solution for fortran 90.
I have a shell script from which I pass a binary file to a fortran program such that
Mth=$1
loop=1
it=1
while test $it -le 12
do
Mth=`expr $Mth + $loop`
file="DataFile"$Mth".bin"
./fort_exe ${Yr} ${nt} ${it}
# Increment loop
it=`expr $it + 1`
done
This script is used to pass 12 files within a do loop to the fortran program. In the fortran program, I read the binary file passed from the shell script and I am trying to write a 2nd file which would compile in a single file all the data that was read from the consecutive files e.g.
!Open binary file passed from shell script
open(1,file='Datafile'//TRIM{Mth)//.bin',action='read',form='unformatted',access='direct', &
recl=4*x*y, status='old')
! Open write file for t 1. The status is different in t 1 and t > 1 so I open it twice: I guess there is a more elegant way to do this...
open(2,file='Newfile.bin',action='write',form='unformatted', &
access='stream', position='append', status='replace')
irec = 0
do t = 1, nt
! Read input file
irec = irec + 1
read(1,rec=irec) val(:,:)
! write output file
irecW= irec + (imonth-1)*nt
if ( t .eq. 1) write(2,pos=irecW) val(:,:)
! Close file after t = 1, update the status to old and reopen.
if ( t .eq. 2) then
close (2)
open(2,file='Newfile.bin',action='write',form='unformatted', &
access='stream', position='append',status='old')
endif
if ( t .ge. 2) write(2,pos=irecW) val(:,:)
enddo
I can read the binary data from the first file no problem but when I try and read from another program the binary data from the file that I wrote in the first program such that
open(1,file='Newfile.bin',action='read',form='unformatted', &
access='stream', status='old')
irec=0
do t = 1, nt
! Read input file
irec = irec + 1
read(1,pos=irec) val(:,:)
write(*,*) val(:,:)
enddo
val(:,:) is nothing but a list of zeros. This is the first time I use access=stream which I believe is the only way I can use position='append'. I have tried compiling with gfortran and ifort but I do not get any error messages.
Does anyone have any idea why this is happening?
Firstly, I do not think you need to close and reopen your output file as you are doing. The status specifier is only relevant to the open statement in which it appears: replace will delete Newfile.bin if it exists at that time, before opening a new file with the same name. The status is implicitly changed to old, but this does not affect any operations done to the file.
However, since your Fortran code does not know you run it 12 times, you should have a way of making sure the file is only replaced the first time and opened as old afterwards; otherwise, Newfile.bin will only contain the information from the last file processed.
As for reading in the wrong values, this most likely occurs because of the difference between direct access (where you can choose a record length) and stream access (where you cannot). With stream access, data is stored as a sequence of "file storage units". Their size is in general compiler-dependent, but is available through the module iso_fortran_env as file_storage_size; it is usually 8 bits. This means that each entry will usually occupy multiple storage units, so you have to take care that a read or write with the pos = specifier does not access the wrong storage units.
Edit:
Some example code writing and reading with stream access:
program stream
use, intrinsic :: iso_fortran_env
implicit none
integer :: i, offset
real(real32), dimension(4,6) :: val, nval
open(unit=2, file='Newfile.bin', action='readwrite', form='unformatted', &
access='stream', status='replace')
do i = 1,2
call random_number(val)
write(2) val
enddo
! The file now contains two sequences of 24 reals, each element of which
! occupies the following number of storage units:
offset = storage_size(val) / file_storage_size
! Retrieve the second sequence and compare:
read(2, pos = 1 + offset*size(val)) nval
print*, all(nval == val)
close(2)
end program
The value true should be printed to the screen.
Note also that it's not strictly necessary to specify a pos while writing your data to the file, because the file will automatically be positioned beyond the last record read or written.
That said, direct or stream access is most beneficial if you need to access the data in a non-sequential manner. If you only need to combine input files into one, it could be easier to write the output file with sequential access, for which you can also specify recl and position = 'append'.
You can check for the existence of a file in standard Fortran, by using the inquire statement:
logical :: exist
inquire(file="test.dat", exist=exist)
if (exist) then
print *, "File test.dat exists"
else
print *, "File test.dat does not exist"
end if
Alternatively you can have a look at the modFileSys library which provides libc like file manipulation routines.
As for appending and streams: Appending files is also possible when you use "classical" record based fortran files, you do not have to use streams for that.
I wrote a subroutine for a Fortran program and I want to execute a command (delete file in the program directory and open a new one) the first, and only first time, the subroutine is called. I know that I can achieve this by creating some flag outside of the subroutine, in the main program, that I set to false at program startup and then set to true upon entering the subroutine. Then I could use this flag in an if statement to figure if the commands I want to execute on the initial call should be executed or not. But this requires me modifying the existing program and I didn't want to do that if I could avoid it. Is there some other way to do what I want to do?
An example might be:
subroutine test(a)
implicit none
integer, intent(inout) :: a
logical, save :: first_time=.true.
if(first_time) then
first_time=.false.
a = a + 12345
else
a = a - 67890
end if
end subroutine test
How about using some characteristic of the output file to determine whether or not to delete it? Time stamp, file lock, a particular file extension, etc.