I am trying to figure out how to discover how many global attributes a netcdf file has. By doing so, I am using the call:
status = nf90_Inquire_Variable(ncid, NF90_GLOBAL, nAtts=natts)
In order to test it I created a simple example. First, it creates a file with global attributes and then it tries to read it:
...
! create the file
call check( nf90_create("test.nc", NF90_NETCDF4, ncid) )
call check( nf90_put_att(ncid, NF90_GLOBAL, "date", "01/01/2021") )
call check( nf90_put_att(ncid, NF90_GLOBAL, "time", "00:00:00") )
call check( nf90_put_att(ncid, NF90_GLOBAL, "seconds", 1000) )
call check( nf90_close(ncid) )
! Read the file.
status = nf90_open("test.nc", NF90_NOWRITE, ncid)
call check(status)
! how many global attributes?
status = nf90_Inquire_Variable(ncid, NF90_GLOBAL, nAtts=natts)
call check(status)
! bye
status = nf90_close(ncid)
call check(status)
The netcdf file is properly created
netcdf simpletest {
// global attributes:
:date = "01/01/2021" ;
:time = "00:00:00" ;
:seconds = 1000 ;
}
But the following error shows up:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0x7f34c7156d01 in ???
#1 0x7f34c7155ed5 in ???
#2 0x7f34c6e2020f in ???
#3 0x7f34c749304e in nf_inq_var_
at /to/some/path/netcdf-fortran-4.5.1/fortran/nf_genvar.f90:181
#4 0x7f34c74ebedf in __netcdf_MOD_nf90_inquire_variable
at /to/some/path/netcdf-fortran-4.5.1/fortran/netcdf4_variables.f90:293
Violació de segment (s'ha bolcat la memòria)
GDB stops at:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f5e04e in nf_inq_var (ncid=1, varid=0, name=..., xtype=1077936128, ndims=1073741824, dimids=..., natts=3, _name=256)
at nf_genvar.f90:181
warning: Source file is more recent than executable.
181 dimids(1:ndims) = cdimids(ndims:1:-1)+1
So it seems to me I am doing something wrong. Am I using the original call in a way it is meant to?
If not, how could it be done? By reading the netcdf fortran documentation I am not able to find it.
I am using the following versions of:
gfortran 9.3
zlib 1.2.11
hdf5 1.10.5
netcdf-c 4.7.1
netcdf-fortran 4.5.1
Edit:
As suggested in one of the comments below, I did a couple of minimum reproducible examples with different choices as I am a bit confused regarding the options.
I also created a new fortran bindings to the nc_inq_natts call but renamed to nf90_inq_natts.
a fortran file which creates and read a netcdf file
a c file which creates a netcdf file with 1 global attribute
a fortran file which only read the file from point 2
All files are compiled in debug mode when running.
The following outputs came up:
Create a new file (C):
*** SUCCESS writing example file simple_xy.nc!
Only read netcdf file (fortran):
nf90_inquire_variable ...
NetCDF: Variable not found
nf90_inq_natts ...
NetCDF: Not a valid ID
Global attributes (inquire_variable(GLOBAL)): 1788528357 <-- not working
Global attributes (inq_natts): 260315136 <-- not working
Create and read file (fortran):
nf90_inquire_variable ...
nf90_inq_natts ...
NetCDF: Not a valid ID
Global attributes (inquire_variable(GLOBAL)): 3 <-- it works
Global attributes (inq_natts): 1256308480 <-- not working
The results:
I cannot reproduce the error I first mentioned. :-/
nc_inq_natts does not seems to work when mixed with Fortran (NetCDF: Not a valid ID)
nf90_inquire_variables seems to work when the netcdf file is created in the same file (?). Regarding documention it is not supposed to happen.
C example taken from github works as expected
From a thread on the netcdfgroup mail list found by googling:
A variable ID of NF90_GLOBAL is explicitly allowed with
nf90_inquire_attribute, but not with nf90_inquire_variable. The intended
functions for discovering global attributes are nf90_inquire and
nf90_inquire_attribute. With a bit of study, the F90 documentation on this
topic seems rather clear to me.
Attempting to use nf90_inquire_variable for global attributes is a
mis-application of that function. Furthermore, that function is probably
returning the appropriate error code in this case, as documented: "The
variable ID is invalid for the specified netCDF dataset."
i.e., change the function you call to nf90_inquire_attribute.
Related
What's the best way to check whether a given netCDF file exists in Fortran90? I have a Fortran program which simulates several systems and stores the data in a NETCDF file, and I'd like to be able to set it up so that I can run the code with already-existing netCDF files and just append the extra data.
It seems to work with the following:
status = nf90_open(filePath, NF90_WRITE, netcdfID)
if (status /= nf90_noerr) then
new_status = nf90_create(filePath, NF90_NETCDF4, netcdfID)
! setup
else
! check file has correct dimensions etc
end if
But I'd like to be able to be specific about the error, but the documentation isn't very specific about what the error code is when the file already exists.
I've also considered checking the status of the nf90_create(...) call, but I don't know how to specify cmode=NF90_NETCDF4 and cmode=NF90_NOCLOBBER at the same time.
To be honest, when I use NetCDF with Fortran and need to look at the documentation, I look at the C-documentation at https://www.unidata.ucar.edu/software/netcdf/docs/modules.html and 'translate' it to the Fortran API (for example replacing the nc_ with nf90_) because the C documentation is far more current than the Fortran one.
There seem to be these possible status results:
NF90_NOERR No error.
NF90_EPERM Attempting to create a netCDF file in a directory where you do not have permission to open files.
NF90_ENFILE Too many files open
NF90_ENOMEM Out of memory.
NF90_EHDFERR HDF5 error. (NetCDF-4 files only.)
NF90_EDIMMETA Error in netCDF-4 dimension metadata. (NetCDF-4 files only.)
None of these suggest that you were opening a file that doesn't exist. Maybe try it out?
But if you only want to know whether the file already exists, why not use Fortran's INQUIRE statement?
logical :: fileExists
...
inquire(file=filePath, exist=fileExists)
if (fileExists) then
call nf90_open(
else
call nf90_create(
end if
I am an MPI and Fortran 77 noob. I have a fortran 77 code FKRPRO.f which I wanted to parallelize using OpenMPI. The code requires a lot of parameters which are fed into it during run time from a separate file. Compilation and running is something like this
gfortran -o FKRPRO FKRPRO.f
./FKRPRO < Modelfile.txt
the equivalent lines in the code (not my code) are
PARAMETER(LIN=5)
INTEGER ERROR
LOGICAL PRNTA
PRNTA=.FALSE.
READ(LIN,'(L3)') PRNTA
READ(LIN,21) M1,M2
21 FORMAT(11I5)
and so on. Can someone please explain to me what READ(LIN,'(L3)') PRNTA means. The input in the input file Modelfile.txt is something like this
.F.
0 64
and so on..
I put the necessary MPI statements in the code.
INCLUDE 'MPIF.H'
...
CALL MPI_INIT(ERROR)
CALL MPI_COMM_SIZE(MPI_COMM_WORLD,NPROCS,ERROR)
CALL MPI_COMM_RANK(MPI_COMM_WORLD,PRANK,ERROR)
...
CALL MPI_TYPE_FREE(NEWMATRIX,ERROR)
CALL MPI_FINALIZE(ERROR)
All processes are not being able to read the input file. I have compiled and run the code like this
mpif77 -o bc3 FKRPROG5.f
mpirun -np 4 bc3 < Modelfile.txt
This is not working. I get the following errors. Only the first process or rank 0 can read the file.
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
mpirun has exited due to process rank 3 with PID 866 on
node Avinash-rMBP.local exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
50th line is READ(LIN,'(L3)') PRNTA.Someone kindly point out where I am going wrong :(
So, how can I make all processes read from this input file < Modelfile.txt ?? Thanks.
The statement
READ(LIN,'(L3)') PRNTA
causes the program to read, from the unit attached to the channel with id LIN, a 3-character sequence which represents a logical value and assigns the value read to the variable PRNTA. From the fragments you've shown us the program will read .F. and set PRNTA to .false..
LIN is set to the constant value 5, which usually means stdin. This use of 5 to denote stdin is not a de jure standard, it is more of a de facto standard.
The straightforward way to read a parameter file into an MPI program is to ensure that only one process reads the file and then sends out the values to the other processes which need them.
You seem to have written a program in which all processes try to read the same input file but, at run-time, the redirection you've used to pass Modelfile.txt is only working for one process (presumably the process with rank 0). The other processes are not getting an input file at all and are complaining, then bringing the program crashing down. The error message you show is typical of a Fortran program which doesn't find an input file at all when it tries to read.
Far better to write code along the lines:
call mpi_init ...
...
if (myrank==0) then
open(...) inputfile
read(...) parameters
close(...)
end if
...
call mpi_bcast(parameters from 0 to all)
...
In general, don't expect the run-time environment for MPI processes to be identical copies of the run-time environment for a sequential program. I think that you are seeing evidence that your run-time directs the input only to the first process created when your program runs. Since mpirun is not standardised (though mpiexec is) I don't think you can rely on this run-time behaviour being the same for all MPI implementations. For portability and compatibility you're better off handling I/O explicitly within your program than using o/s features such as redirection.
You could, rather than have process 0 read the parameters and distribute them to other processes, write your code such that each process reads the same file. If you do write your code this way take care to ensure that the processes aren't fighting over access to the I/O channels; having multiple processes trying to (nearly-)simultaneously read across a single input channel is a sure way to slow things down.
I would like to know if in Fortran it is possible to use just a single command (with options/specifiers) to do the following:
open a file if it exists and append some data
(this can be done with: open(unit=40,file='data.data',Access = 'append',Status='old') but if the file does not exist a runtime error is issued)
create the file if it does not exist and write some data.
I am currently using inquire to check whether the file exist or not but then I still have to use the open statement to append or write data.
As far as I am aware of, the only safe solution is to do the way you're already doing it, using different open statements for the different cases:
program proba
implicit none
logical :: exist
inquire(file="test.txt", exist=exist)
if (exist) then
open(12, file="test.txt", status="old", position="append", action="write")
else
open(12, file="test.txt", status="new", action="write")
end if
write(12, *) "SOME TEXT"
close(12)
end program proba
You may be interested in my Fortran interface library to libc file system calls (modFileSys), which could at least spare you the logical variable and the inquire statement by querying the file status directly:
if (file_exists("test.txt")) then
...
else
...
end if
but of course you can program a similar function easily yourself, and especially it won't save you from the two open statements...
open(61,file='data.txt',action='write',position='append')
write(61,*) 'hey'
close(61)
This will append to an existing file, otherwise create and write. Adding status='unknown' would be equivalent.
if you replace the status from 'old' to 'unknown' then you will not get the run time error if the file exists or now.
Thanks
In open statement add the attribute access as follows;
Open(unit=031,file='filename.dat',form='formatted',status='unknown',access='append')
The above statement will open the file without destroying old data and write command will append the new lines in the file.
The simplest solution for fortran 90.
I have problem with this code when I run inversion script.I have tried to debug with gdb:
Starting program: /home/milenko/fg77/f2/bin/inverse_2d
INVERSE: calculate slowness update using LSQR and regularization
open: No such file or directory
apparent state: unit 48 named stop.in
lately writing sequential formatted external IO
Program received signal SIGABRT, Aborted.
0x0012d422 in __kernel_vsyscall ()
It looks as if your program has tried to write to a file called stop.in on unit 48 and encountered a problem. The most probable cause of this is that that file does not exist when a WRITE statement is executed, which in turn suggests that the OPEN statement to 'attach' that file to unit 48 was not successful.
EDIT
Your code snippet does not show that you check the error code returned by the statement
open(48, file='stop.in', status='old')
I suggest you pop in the iostat argument and see what it tells you.
FURTHER EDIT
In the second sentence of my original reply replace 'does not exist' with the words 'does not exist in the location the program expects'.
I've got a protected object that presents functions and procedures in its interface.
In gdb, when I set a bp on the first line of one of those, I get odd results.
Here's a snippet from my gdb console:
(gdb)
(gdb) b database-access_manager.adb:20001
Breakpoint 3 at 0x1a10588: file y:/svs/central_switch/controller/database/
database-access_manager.ads, line 20001.
(gdb)
You can see that gdb is confused. I specified a bp at 20001 of the .adb file but gdb responded by saying it had set the bp at 20001 of the corresponding ads file - which doesn't have that many lines.
What gives?
That .ads file wouldn't happen to be defining or using a generic, would it?
I have yet to find a debugger that handles Ada generics very well. The compiler often creates a raft of semi-invisible code that confuses the heck out of debuggers. I suspect C++ templates have the same issue.
Another possibility is that you are looking at a source file that has been modified since your program was compiled.
Running on Windows with GNAT Pro 6.3.1 (I realise this isn't an ideal data point for you!) this worked fine.
I did notice that when I requested a bp on the subprogram specification, GDB effectively set two bps, one in the specification and one at the first statement: so, given
package body Protected_Object is
protected body PO is
procedure Put (V : Integer) is
begin
Value := V;
end Put;
function Get return Integer is
begin
return Value;
end Get;
end PO;
end Protected_Object;
the GDB console says (for Put)
gdb) break protected_object.adb:4
Breakpoint 1 at 0x401729: file protected_object.adb, line 6. (2 locations)
and at run time, sure enough there are 2 breaks:
Breakpoint 1, <protected_object__po__putP> (<_object>=..., v=42) at protected_object.adb:4
(gdb) cont
Breakpoint 1, protected_object.po.put (<_object>=..., v=42) at protected_object.adb:6
Version: GNU gdb (GDB) 7.0.1 for GNAT Pro 6.3.1 (20100112) [rev:158983]
Here's the update on my problem.
I made a protected type with access methods and used it in a small main and found that breakpoints in my example protected type worked fine.
Now I'm trying to understand why, within the context of my company's very large build, the breakpoints don't work.
I'm using the same gdb, GPS, & compiler switches in each case and it works for the small program but not in the large one.
I'll post my results when/if I have any.
Thanks to all the repliers.
Tom