Folks,
we collect large amounts of data and create error, status, info log files to let us know what's
going on. We use ofstreams to write to these files. After some period of time (days), we get a
file error (indicated by .good() call) on one of the ofstreams. In the affected log file, it
appears that the write of a single line begins but is interrupted by a write of the exact same
line. For example,
### Random Line of Text 1 ###
### Random Line of Text 2 ###
### Random Line of Text 3### Random Line of Text 3 ###
Each file/ofstream has a single thread that does the actual writing. We don't flush for performance
reasons and shouldn't have to.
Its always the same type of error.
It does only happen on one of three machines running the same code but we don't see any I/O errors
but maybe not looking in the right place.
Thanks for you time.
Related
In a lot of the task flow jobs I'm running, I constantly am getting the
FR_3085 ERROR: Row [1]: 2-th character is a null character, which is
not allowed in a text input file
error. These occur usually in data synchronization tasks but I sometimes see this in mapping configurations as well. How do I resolve this error?
This error occurs when you have NULL characters in your flat file.
One way for doing this is using the OS utilities for removing the NULL characters from your flat file automatically and this will depend on what OS you're using.
I am struggling to read from one input text file into a new text file using the Fortran compiler in Visual Studio 10. My input file is in gslib format (attached)
program main
! Unit numbers:
lisa = 4
lfke = 9
! Writing input data to file Lisa
open(unit=lisa,file='fake.dat',status='OLD',err=80)
open(unit=lfke,file='dlfke',status='NEW', err=80)
read(lisa,*,err=80)
read(lisa,*,err=80) line
write(9,*) line
close(unit=lisa)
close(unit=lfke)
80 stop 'ERROR in test file!'
end
This program is always going to produce the error message "ERROR in test file", whether or not there is an error. After executing line the close statements, it will next execute line 80 with the stop statement. You will probably find it easier, as your develop your program, to remove the err=80 tests in each IO statement. Then if there is an IO error, the program will automatically terminate and produce a specific message about the program. These err=branches are useful if you want to have your program handle the error, but with this implementation its hiding what the IO error was since all IO errors produce the same message.
I am an MPI and Fortran 77 noob. I have a fortran 77 code FKRPRO.f which I wanted to parallelize using OpenMPI. The code requires a lot of parameters which are fed into it during run time from a separate file. Compilation and running is something like this
gfortran -o FKRPRO FKRPRO.f
./FKRPRO < Modelfile.txt
the equivalent lines in the code (not my code) are
PARAMETER(LIN=5)
INTEGER ERROR
LOGICAL PRNTA
PRNTA=.FALSE.
READ(LIN,'(L3)') PRNTA
READ(LIN,21) M1,M2
21 FORMAT(11I5)
and so on. Can someone please explain to me what READ(LIN,'(L3)') PRNTA means. The input in the input file Modelfile.txt is something like this
.F.
0 64
and so on..
I put the necessary MPI statements in the code.
INCLUDE 'MPIF.H'
...
CALL MPI_INIT(ERROR)
CALL MPI_COMM_SIZE(MPI_COMM_WORLD,NPROCS,ERROR)
CALL MPI_COMM_RANK(MPI_COMM_WORLD,PRANK,ERROR)
...
CALL MPI_TYPE_FREE(NEWMATRIX,ERROR)
CALL MPI_FINALIZE(ERROR)
All processes are not being able to read the input file. I have compiled and run the code like this
mpif77 -o bc3 FKRPROG5.f
mpirun -np 4 bc3 < Modelfile.txt
This is not working. I get the following errors. Only the first process or rank 0 can read the file.
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
At line 50 of file FKRPROG5.f (unit = 5, file = 'stdin')
Fortran runtime error: End of file
mpirun has exited due to process rank 3 with PID 866 on
node Avinash-rMBP.local exiting improperly. There are two reasons this could occur:
1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
This may have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
50th line is READ(LIN,'(L3)') PRNTA.Someone kindly point out where I am going wrong :(
So, how can I make all processes read from this input file < Modelfile.txt ?? Thanks.
The statement
READ(LIN,'(L3)') PRNTA
causes the program to read, from the unit attached to the channel with id LIN, a 3-character sequence which represents a logical value and assigns the value read to the variable PRNTA. From the fragments you've shown us the program will read .F. and set PRNTA to .false..
LIN is set to the constant value 5, which usually means stdin. This use of 5 to denote stdin is not a de jure standard, it is more of a de facto standard.
The straightforward way to read a parameter file into an MPI program is to ensure that only one process reads the file and then sends out the values to the other processes which need them.
You seem to have written a program in which all processes try to read the same input file but, at run-time, the redirection you've used to pass Modelfile.txt is only working for one process (presumably the process with rank 0). The other processes are not getting an input file at all and are complaining, then bringing the program crashing down. The error message you show is typical of a Fortran program which doesn't find an input file at all when it tries to read.
Far better to write code along the lines:
call mpi_init ...
...
if (myrank==0) then
open(...) inputfile
read(...) parameters
close(...)
end if
...
call mpi_bcast(parameters from 0 to all)
...
In general, don't expect the run-time environment for MPI processes to be identical copies of the run-time environment for a sequential program. I think that you are seeing evidence that your run-time directs the input only to the first process created when your program runs. Since mpirun is not standardised (though mpiexec is) I don't think you can rely on this run-time behaviour being the same for all MPI implementations. For portability and compatibility you're better off handling I/O explicitly within your program than using o/s features such as redirection.
You could, rather than have process 0 read the parameters and distribute them to other processes, write your code such that each process reads the same file. If you do write your code this way take care to ensure that the processes aren't fighting over access to the I/O channels; having multiple processes trying to (nearly-)simultaneously read across a single input channel is a sure way to slow things down.
i want to create new file at each loop, i don't know how to do...
!file1, file2,....., file[n]
OPEN (1,FILE='file1.out',ACCESS='SEQUENTIAL',STATUS='UNKNOWN')
do ph=1,N6
do i=1,nx-1
A(i)=mu*U(i-1)
end do
do j=0,nx
U(j)=A(j)
end do
if (mod(ph,Ne)==0) then ! ?
WRITE(1,200) nt,U(i)
endif
200 format(5E12.4)
end do
Or maybe, i can write with a newline or column ?
I'm a beginner in fortran.
Thanks
Your existing code should output more than just last write statement to the file ... it is a sequential file, which means that the output is added to the file in sequential order. If you are only seeing one output, perhaps that is all that the IF statement is causing to be output?
If you still wish to output to multiple files, the easiest way to output to multiple files is to reuse the unit number and have the program create filenames. You need to close the file / unit and reopen it. This is much more easier than have multiple open statements and unit numbers, which would quickly become awkward as the number of files increased. Here is a code fragment that assumes fewer than 100 files:
do i=1, N
write (filename, '("myfile", I2.2, ".txt")' ) I
open (file=filename,unit=16,...)
calculations...
write (16,'(5E12.4)') nt,U(i)
close (16)
end do
Actually I have trouble naming the title of this post. Because I don't know how to summarize my meaning in a professional way. But I'll explain my question as below:
I'm running a program written by C++, command is:
./VariationHunter_SC
Then it'll let you type in many parameters:
Please enter the minimum paired-end insert size:
Please enter the maximum paired-end insert size:
Please enter the pre-processing mapping prune probability:
Please enter the name of the input file:
Please enter the minimum support for a cluster:
Obviously I need to type in such parameters one by one to run the program; But I have thousands of such jobs, and need to pre-assign such parameters in script, and submit script to computer.
So how can I do that?
thx
Edit
so how can I make parameter-list?
Just like below?:
140
160
0
mrfast.vh
1
Seems the program cannot recognize these numbers, and distribute numbers..
This depends on how the program actually reads the data that you type in - it's likely that its reading stdin, so you could use separate files with the parameters and pass them in via redirection: ./VariationHunter_SC < parameter-file
It's also possible that the program will accept parameters on the command line, but there's no way of really knowing that (or how) except by whatever documentation the program might come with (or by reading the source code, if it's available and there is no other accurate docs).
Simply use the piping character to pipe the contents of a file to your program
example, in a windows command shell:
echo "asdf" | pause
This will pass "asdf" to the pause program. As a result, pause will print a "Press any key to continue" message, then imediately continue because it will receive the "asdf" string as a response.
So, overall, write or use a program that outputs the contents of your file. Call it, then pipe its output to the program that needs the input.
The unix cat command is such a command that writes the contents of a file to output, or to the input of another executable if you are piping the output.