Fortran-- reading columns with specific time stamps - fortran

I have this data set and I would like to read it using Fortran and apply a few operations . I have presented the data set below. Here's a short description of the data set. The data set starts of with a time stamp and the subsequent lines (around 60 or 120)are the data points recorded at that time, which is again followed by a time stamp and data and so on so forth. Each file has around 30 observations with 15 different time stamps(i.e two adjacent observations have the same time stamps). I now want to read this file and perform operations such as addition or subtraction of columns on the data points with the same time stamps.
Here the '6:31: 5', '6:27: 9' are the time stamps followed by the data.
Please help as I am unable to figure out a way to do this.
Thank you very much.

Your question leaves a lot of ambiguity. The way I understand it is that you want to read a file where the contents of a line might be one of two different things: a value, or a time stamp.
And you don't know which one it is before you read it.
The best way, I think, is to first read the line into a string.
character(len=20) :: line
read(1001, '(A)') line
Then, based on the first character in that string, you can deduce which one it is.
if (line(1:1) == "'") then
! It's a time stamp
else
! It's a new value
end if
If it's a value, you can read it straight out of the line variable:
read(line, *) val
Reading the time stamp is more complicated, because I don't quite know where the colons are inside it. If the hours are all single digits, you could get away with something like this:
read(line, '(X, I1, X, I2, X, I2)') hour, minute, second
But because there are hours that can't be written with a single digit, I rather doubt that.
So I suggest that you create a subroutine that uses the INDEX function to locate the colons, and then slice the line to read each value separately.
hm = index(line, ':', back=.FALSE.) ! hour-minute separator
ms = index(line, ':', back=.TRUE.) ! minute-second separator
read(line(2:hm-1), *) hour
read(line(hm+1:ms-1), *) minute
read(line(ms+1:), *) second
All put together:
program timestamp
implicit none
integer, parameter :: runit = 1001
integer :: io_err
character(len=20) :: line
real :: val
integer :: hour, minute, second
open(unit=runit, file='data.txt', action='READ', status='OLD', iostat=io_err)
if (io_err /= 0) then
write(*, *) "Unable to open file"
stop 1
end if
do
read(runit, '(A)', iostat=io_err) line
if (io_err == -1) then
write(*, *) "End of file"
exit
end if
if (line(1:1) == "'") then
call read_timestamp(line, hour, minute, second)
write(*, '(A, 2(I2.2, ":"), I2.2)') "New timestamp: ", hour, minute, second
else
read(line, *) val
write(*, *) "Data: ", val
end if
end do
close(runit)
contains
subroutine read_timestamp(line, hour, minute, second)
implicit none
character(len=*), intent(in) :: line
integer, intent(out) :: hour, minute, second
integer :: hm, ms ! Indices of separators
hm = index(line, ':', back=.FALSE.)
ms = index(line, ':', back=.TRUE.)
read(line(2:hm-1), *) hour
read(line(hm+1:ms-1), *) minute
read(line(ms+1:), *) second
return
end subroutine read_timestamp
end program timestamp

Related

FORTRAN parsing file with varying line formate

I have only limited experience with FORTRAN and I need to parse files with a structure similar to this:
H s 13.010000 0.019685
1.962000 0.137977
0.444600 0.478148
s 0.122000 1.000000
p 0.727000 1.000000
***
He s 38.360000 0.023809
5.770000 0.154891
1.240000 0.469987
s 0.297600 1.000000
p 1.275000 1.000000
***
I need to search for the label (e.g. He) and then read the corresponding blocks into an array.
I know I can parse file by specifying the format each line is supposed to have, but here there are different formats possible.
In Python I would just split each line by the white spaces and deal with it depending on the number of columns. But how to approach this in FORTRAN?
You can read each line as a character string and then process it. If, as it seems, the format is fixed (element symbol in first two characters, orbital letter in sixth character, etc.), the following program could serve you as inspiration:
program elms
implicit none
integer, parameter :: MAX_LEN = 40
character(len=MAX_LEN) :: line_el, line
integer :: u
integer :: is
integer :: nlin
character(len=2) :: element = 'He'
integer, parameter :: MAX_LINES = 20
real, dimension(MAX_LINES) :: e, f
open(newunit=u, file='elms.dat', status='old', action='read')
main_loop: do
! Read line
read(u, '(a)', iostat=is) line_el
if (eof_iostat(is)) exit main_loop
! Check first two characters of the line vs. chemical element.
if (line_el(1:2) .eq. element) then
! This is the beginning of an element block
nlin = 0
line = line_el
do
if (line .ne. '') then
! Line is not empty or only spaces.
nlin = nlin + 1
if (line(6:6) .ne. ' ') then
! Line contains an orbital letter - process it.
end if
! Read the real values in the rest of the line
read(line(7:),*) e(nlin), f(nlin)
end if
! Read next line
read(u, '(a)', iostat=is) line
if (eof_iostat(is)) exit main_loop
if (line(1:2) .ne. ' ') then
! Finished processing element block.
exit main_loop
end if
end do
end if
end do main_loop
! Close file
close(u)
contains
logical function eof_iostat(istat)
! Returns true if the end of file has been reached
use, intrinsic :: iso_fortran_env, only: IOSTAT_END
implicit none
integer, intent(in) :: istat
select case (istat)
case (0) ! No error
eof_iostat = .false.
case (IOSTAT_END) ! End of file reached
eof_iostat = .true.
case default ! Error
STOP
end select
end function eof_iostat
end program
You will probably need to make the program a subroutine, make element an intent(in) dummy argument, process the orbital symbols, etc.
Note that, if possible, it would be easier to just read all the data from the file in one go, and then search for the relevant data in the memory (e.g., having an array with the chemical symbols).

Trouble reading reals from unknown length character string in Fortran

This is a small portion of the data I am trying to read:
01/06/2009,Tom Sanders,,264,220,73,260
01/08/2009,Adam Apple,158,,260,,208
01/13/2009,Lori Freeman,230,288,218,282,234
01/15/2009,Diane Greenberg,170,,250,321,197
01/20/2009,Adam Apple,257,,263,256,190
01/21/2009,Diane Greenberg,201,,160,195,142
01/27/2009,Tom Sanders,267,,143,140,206
01/29/2009,Tina Workman,153,,124,155,140
02/03/2009,Tina Workman,233,,115,,163
02/03/2009,Adam Apple,266,130,310,,310
the numbers between each comma are from a different location
Where two commas would represent missing data and a trailing comma would mean the fifth data point is missing
My goal is to organize the data into a table after calculating the average of each site and person, hence my two dim arrays
I want my output to look something like the following:
(obviously neater formatting but a table nonetheless)
Average Observed TDS (mg/l)
Name Site 1 Site 2 Site 3 Site 4 Site 5
------------------------------------------------------
Tom Sanders 251.0 172.5 251.7 160.0 229.0
Adam Apple 227.0 130.0 277.7 256.0 236.0
Lori Freeman 194.0 288.0 216.7 279.0 202.7
Diane Greenberg 185.5 190.0 205.0 258.0 169.5
Tina Workman 193.0 140.0 119.5 155.0 163.0
This is my program so far:
program name_finder
implicit none
integer, parameter :: wp = selected_real_kind(15)
real(wp) :: m, tds
real(wp), dimension(20,5) :: avg_site, site_sum
integer, dimension(20) :: nobs
integer, dimension(5) :: x
integer :: ierror, i, nemp, cp, non, ni, n
character(len=40), dimension(20) :: names
character(len=200) :: line, aname
character(len=20) :: output, filename
character(len=3), parameter :: a = "(A)"
do
write(*,*) "Enter file to open."
read(*,*) filename
open(unit=10,file = filename, status = "old", iostat = ierror)
if (ierror==0) exit
end do
write(*,*) "File, ",trim(filename)," has been opened."
non = 0
outer: do
read(10,a, iostat = ierror) line
if (ierror/=0) exit
cp = index(line(12:),",") + 11
aname = line(12:cp-1)
n=0
middle: do
read(line,'(Tcp,f4.2)') tds
write(*,*) "tds=", tds
n=n+1
if (n>10) exit
i = 1
inner: do
if (i > non) then
non = non +1
names(non) = trim(aname)
!ni = non
exit
end if
if (aname == names(i)) then
!ni = i
!cycle outer
exit inner
end if
i = i + 1
end do inner
end do middle
end do outer
write(*,*)
write(*,*) "Names:"
do i = 1,non
write(*,*) i, names(i)
end do
close(10)
close(20)
STOP
end program name_finder
TLDR; I am having trouble reading the data from the file shown at the top of each site after the names.
Suggestions? Thanks!
I hope the following is helpful. I have omitted any easily assumed declarations or any further data manipulation or writing to another file. The code is used just to read the data line by line.
character(150) :: word
read(fileunit, '(A)') word ! read the entire line
comma_ind = index(word,',') ! find the position of first comma
! Find the position of next comma
data_begin = index(word(comma_ind+1:),',')
! Save the name
thename = word(comma_ind+1:comma_ind+data_begin-1)
! Define next starting point
data_begin = comma_ind+data_begin
! Read the rest of the data
outer: do
if (word(data_begin+1:data_begin+1) == ',') then
! decide what to do when missing an entry
data_begin = data_begin + 1
cycle outer
else if (word(data_begin+1:data_begin+1) == ' ') then
! Missing last entry
exit outer
else
! Use it to find the length of current entry
st_ind = index(word(data_begin+1:),',')
if (st_ind == 0) then
! You reached the last entry, read it and exit
read(word(data_begin+1:), *) realData
exit outer
else
! Read current entry
read(word(data_begin+1: data_begin+st_ind-1),*) realData
end if
! Update starting point
data_begin = data_begin + st_ind
end if
end do outer
There could be a more elegant way to do it but I cannot think of any at the moment.

Call a subroutine for a list of points instead of a single point

I have a certain piece of code in fortran. The code takes 'pq' as an input from the user and is a single point. Instead of doing this I want to read a set of points 'pq' from a file points.txt and run it for those number of points instead of just one single user input. Is it possible? The code is as follows:
program prop
use module
implicit none
character(len=80) :: ErrorMsg
character(2) :: xy
real(8) :: Conc(20) = 0.d0
character(len=20) :: fn, fl
real(8) :: Mmolar, Tcritical, Pcritical, Tmininimum, Tmaximum, x, y
call Init_module()
write(*,*) 'Insert the gas name:'
read(*,*) fn
write(*,*) 'Insert the gas library:'
read(*,*) fl
write(*,*) 'Insert the copule pq:'
read(*,*) pq
write(*,*) 'Insert the value of ', pq(1:1)
read(*,*) x
write(*,*) 'Insert the value of ', pq(2:2)
read(*,*) y
write(*,*) 'Pres = ', Pres( pq, x, y, ErrorMsg)
write(*,*) 'Temp = ', Temperature( pq, x, y, ErrorMsg)
call ReleaseObjects()
end program prop
Instead of reading pq as a single point x,y from the user in the above code, I want to read a set of points from file.txt, for example 50 points and then run subroutines Pres and Temperature.
Each line of the file contains one point x,y and x and y in each line are separated by a few space characters.
The first few lines of file.txt are:
Ts
500
0.04781564 159.81587875
0.20396084 165.46398084
0.08159885 166.81382894
0.03879184 164.17497877
0.12585959 165.37000305
0.09895530 165.95997769
0.10389518 170.74235496
It must be noted that the length and the sign of the floating numbers can vary. The file.txt is originally written through python with the formatting for x, y being '%-12.8f %-12.8f\n'%. I have the following code to try and read the file but am not able to read from the 3rd line onwards:
real, allocatable :: x(:),y(:)
integer :: np
open(12,file=trim('file.txt'),status='old', &
access='sequential', form='formatted', action='read' )
read(12,*)pq
write(*,*)'pq:', pq
read(12,*)np
write(*,*)'number of points:',np
allocate (x(np))
allocate (y(np))
do i=1,np
read(12,*)x(i),y(i)
write(*,*)x(i),y(i)
enddo
Instead of using the READ statement with the asterisk (*) as the first argument asking for an user input, use a file identifier. You need to OPEN your file containing the set of points, assuming it is ASCII :
OPEN(UNIT=10,FILE=file.txt,ACTION='read',STATUS='old')
I think the arguments of this command are quite explanatory.
Then assuming your file contains multiple lines with x and y values, you can read each line of your file by doing :
READ(10,*) x,y
If you have multiple points to read, just use a DO if you know the number of points to read, a DO WHILE otherwise. To take your example with 50 points, something like this should work :
OPEN(UNIT=10,FILE=file.txt,ACTION='read',STATUS='old') ! Open file
DO i=1,50
READ(10,*) x,y
write(*,*) 'Pres = ', Pres( pq, x, y, ErrorMsg)
write(*,*) 'Temp = ', Temperature( pq, x, y, ErrorMsg)
END DO
CLOSE(10) ! Close file
EDIT
Your suggestion is almost correct. You forgot to declare pq as a character(len=2). You should not have been able to pass line 1 because of that.
As I said, there is a space separator that is naturally treated by a asterisk as a format. Anyway, if you want to exactly match the format, use the same with which you wrote your data. Reading your format Python, I assume you wrote two floats with a space separator, and indeed if you count the number of character of your digits :
0.04781564 159.81587875
^^^^^^^^^^^^|^^^^^^^^^^^^
1 12|1 12
|
space
which gives the following format in Fortran :
read(12,'(f12.8,1X,f12.8)') x(i),y(i)
X means a space separator in Fortran formats.
Then you can write you data onscreen with the same format to check :
write(*,'(f12.8,1X,f12.8)') x(i),y(i)
It gives :
pq:Ts
number of points: 500
0.04781564 159.81587219
0.20396084 165.46397400
0.08159885 166.81382751
0.03879184 164.17497253
0.12585959 165.37001038
0.09895530 165.95997620
0.10389518 170.74235535
You may have noticed that you lost precision on the last digits. It is because you have declared a simple real (4 bytes). Switch your real to 8 bytes with real(kind=8) or real*8 according to your compiler (be aware, not the right way to do it, not portable but sufficient in your case)
Do not forget to close your file when you are done dealing with it :
close(12)

Program to create spatial grid, average values that fall within grid, write to table

So I'm trying to come up with a clever way to make this program read a catalog and take anything falling within specific spatial "grid" boxes and average the data in that box together. I'll paste my horrid attempt below and hopefully you'll see what I'm trying to do. I can't get the program to work correctly (it gets stuck in a loop somewhere that I haven't debugged), and before I bang my head against it anymore I want to know if this looks like a logical set of operations for what I'm looking to do, or if there is a better way to accomplish this.
Edit: To clarify, the argument section is for the trimming parameters---"lmin lmax bmin bmax" set the overall frame, and "deg" sets the square-degree increments.
program redgrid
implicit none
! Variable declarations and settings:
integer :: ncrt, c, i, j, k, count, n, iarg, D, db, cn
real :: dsun, pma, pmd, epma, epmd, ra, dec, degbin
real :: V, Per, Amp, FeH, EBV, Dm, Fi, FeHav, EBVav
real :: lmin, lmax, bmin, bmax, l, b, deg, lbin, bbin
real :: bbinmax, bbinmin, lbinmax, lbinmin
character(len=60) :: infile, outfile, word, name
parameter(D=20000)
dimension :: EBV(D), FeH(D), lbinmax(D), bbinmax(D)
dimension :: bbinmin(D), lbinmin(D)
103 format(1x,i6,4x,f6.2,4x,f6.2,4x,f7.2,3x,f6.2,4x,f5.2,4x,f5.2,4x,f5.2,4x,f6.4)
3 continue
iarg=iargc()
if(iarg.lt.7) then
print*, 'Usage: redgrid infile outfile lmin lmax bmin bmax square_deg'
stop
endif
call getarg(1, infile)
call getarg(2, outfile)
call getarg(3, word)
read(word,*) lmin
call getarg(4, word)
read(word,*) lmax
call getarg(5, word)
read(word,*) bmin
call getarg(6, word)
read(word,*) bmax
call getarg(7, word)
read(word,*) deg
open(unit=1,file=infile,status='old',err=3)
open(unit=2,file=outfile,status='unknown')
write(2,*)"| l center | b center | [Fe/H] avg | E(B-V) avg | "
FeHav = 0.0
EBVav = 0.0
lbinmin(1) = lmin
bbinmin(1) = bmin
degbin = (bmax-bmin)/deg
db = NINT(degbin)
do j = 1, db
bbinmax(j) = bbinmin(j) + deg
lbinmax(j) = lbinmin(j)*cos(bbinmax(j))
print*, lbinmin(j), bbinmin(j), db
cn = 1
7 continue
read(1,*,err=7,end=8) ncrt, ra, dec, l, b,&
V, dsun, FeH(cn), EBV(cn)
if(b.ge.bbinmin(j).and.b.lt.bbinmax(j)) then
if(l.ge.lbinmin(j).and.l.lt.lbinmax(j)) then
FeHav = FeHav + FeH(cn)
EBVav = EBVav + EBV(cn)
cn = cn + 1
end if
end if
goto 7
8 continue
FeHav = FeHav/cn
EBVav = EBVav/cn
write(2,*) lbinmax(j), bbinmax(j), FeHav, EBVav
bbinmin(j+1) = bbinmin(j) + deg
lbinmin(j+1) = lbinmin(j) + deg
end do
close(1)
close(2)
end program redgrid
Below is a small section of the table I'm working with. "l" and "b" are the two coordinates I am working with---they are angular, hence the need to make the grid components "b" and "l*cos(b)." For each 0.5 x 0.5 degree section, I need to have averages of E(B-V) and [Fe/H] within that block. When I write the file all I need are four columns: the two coordinates where the box is located, and the two averages for that box.
| Ncrt | ra | dec | l | b | V | dkpc | [Fe/H] | E(B-V) |
7888 216.53 -43.85 -39.56 15.78 15.68 8.90 -1.19 0.1420
7889 217.49 -43.13 -38.61 16.18 16.15 10.67 -1.15 0.1750
7893 219.16 -43.26 -37.50 15.58 15.38 7.79 -1.40 0.1580
Right now, the program gets stuck somewhere in the loop cycle. I've pasted the terminal output that happens when I run it, along with the command line I'm running it with. Please let me know if I can help clarify. This is a pretty complex problem for a Fortran rookie such as myself---perhaps I'm missing some fundamental knowledge that would make it much easier. Anyways, thanks in advance.
./redgrid table2.above redtest.trim -40 0 15 30 0.5
-40.0000000 15.0000000 30 0.00000000 0.00000000
-39.5000000 15.5000000 30 -1.18592596 0.353437036
^it gets stuck after two lines.
I assume that the program does what you want it to do, but you are looking for a few things to tidy the code up.
Well first up, I'd fix up the indentation.
Secondly, I'd not use unit numbers below 10.
INTEGER, PARAMETER :: in_unit = 100
INTEGER, PARAMETER :: out_unit = 101
...
OPEN(unit=in_unit, file=infile, status='OLD")
...
READ(in_unit, *) ...
...
CLOSE(in_unit)
Thirdly, I'd not use GOTOs and labels. You can do that in a loop far easier:
INTEGER :: read_status
DO j = 1, db
...
read_loop : DO
READ(in_unit, *, IOSTAT=read_status) ...
IF (read_status == -1) THEN ! EOF
EXIT read_loop
ELSEIF (read_status /= 0) THEN
CYCLE read_loop
ENDIF
...
END DO read_loop
...
END DO
There are a few dangers in your code, and even in this one above: It can lead to infinite loops. For example, if the opening of infile fails (e.g. the file doesn't exist), it loops back to label 3, but nothing changes, so it will eventually again try to open the same file, and probably have the same error.
Same above: If READ repeatedly fails without advancing, and without the error being an EOF, then the read loop will not terminate.
You have to think about what you want your program to do when something like this happens, and code it in. (For example: Print an error message and STOP if it can't open the file.)
You have a very long FORMAT statement. You can leave it like that, though I'd probably try to shorten it a bit:
103 FORMAT(I7, 2F10.2, F11.2, 4F9.2, F10.4)
This should be the same line, as numbers are usually right-aligned. You can also use strings as a format, so you could also do something like this:
CHARACTER(LEN=*), PARAMETER :: data_out_form = &
'(I7, 2F10.2, F11.2, 4F9.2, F10.4)'
WRITE(*, data_out_form) var1, var2, var3, ...
and again, that's one less label.

Fortran: Add column per column to file

The code consists of a do-loop and creates arrays of data as long as running. I need these arrays added to a file as new columns.
The first column is fixed (wavelengths) and the second is generated within the first run:
OPEN (unit=11,file=filename // '.csv')
WRITE(11,'(i4,A1,f10.6)') (lambda(ii),tab,resv(ii), ii=1,nw)
CLOSE(11)
lambda are the wavelengths (4 digits), tab is declared as char(9) and resv are my data (floating). The array consists of nw=2000 items.
First time running the script gives me a nice output which I can load into MS Excel as .csv
However, the script is to return to the beginning of the loop, calculate new data and store the changed "resv" items into a new column.
But when I go like
WRITE(11,'(T17,i4,A1,f10.6)') (lambda(ii),tab,resv(ii), ii=1,nw)
the new data is indeed stored into column 17, but all the data before is being removed!
So how can I tell Fortran to "add" a new column?
Try a variant of this. Have a look at the code in addcolumn. The restriction is that your line can't be more than 1024 characters long. You can increase the size of the array to suit your needs.
module mod_helper
integer, parameter:: AMAX = 10
integer, dimension(AMAX):: coldata
integer:: revision
contains
subroutine init
revision = 0
do ii = 1, AMAX
coldata(ii) = ii
end do
end subroutine init
subroutine increment(howmuch)
integer, intent(in):: howmuch
coldata = coldata + howmuch
end subroutine
subroutine addcolumn(csvname)
character*(*), intent(in) csvname
integer(kind=2):: status
integer:: prevlen
character(1024):: prev, oldname, newname
integer, parameter:: oldfile = 20, newfile = 30
write(oldname, "(A,I3.3,'.csv')") csvname, revision
revision = revision + 1
write(newname, "(A,I3.3,'.csv')") csvname, revision
if (revision .gt. 1) then
! Open the old file
open(oldfile, file=oldname, access="sequential", status="old")
end if
open(newfile, file=newname, access="sequential", status="new", action="write")
prev = ' '
do ii = 1, AMAX
if (revision .gt. 1) then
! read previous contents as a string
read(oldfile, "(A)") prev
prevlen = len(trim(prev))
else
prevlen = 1
end if
! write previous and new contents
write(newfile, "(A, I4, ',')") prev(1:prevlen), coldata(ii)
end do
! delete the previous file
if (revision .gt. 1) close(oldfile, status='delete')
close(newfile)
end subroutine
end module
program main
use mod_helper
call init
call addcolumn('col')
call increment(1)
call addcolumn('col')
call increment(20)
call addcolumn('col')
end program