Differentiate entries in Block Data with a flag in Fortran77 - fortran

I am initializing a header struct in Fortran 77 in a BLOCK DATA . Whereas I have two versions of this *.f - file. One got an update with new functionality but the old version should be still supported.
The new version looks alike:
Block Data INIT_HS
Implicit None
C
Include 'hdr_str.inc'
C
Data (HDR_STRUC (I, NAVA), I = 1, 5)/
& CITY , AREA,
& LAT , LONG,
& ELEV,
& END/
C
Data (HDR_STRUC (I, NAVB), I = 1, 3)/
& CITY , CUST AREA,
& NAME , END/
C
Data HDR_STRING (CODE) /'CODE'/
Data HDR_STRING (ALT ) /'ALT'/
Data HDR_STRING (LAT) /'LAT'/
Data HDR_STRING (LON) /'LON'/
Data HDR_STRING (ACCL) /'ACCL'/
END
And in the old version some entries are not there:
Block Data INIT_HS
Implicit None
C
Include 'hdr_str.inc'
C
Data (HDR_STRUC (I, NAVA), I = 1, 3)/
& LAT , LONG,
& ELEV,
& END/
C
Data (HDR_STRUC (I, NAVB), I = 1, 2)/
& CITY , CUST AREA,
& END/
C
Data HDR_STRING (CODE) /'CODE'/
Data HDR_STRING (LAT) /'LAT'/
Data HDR_STRING (LON) /'LON'/
END
For the rest of the merged source code, I found a solution in handing over an argument when calling the executable:
DUMMY.exe VERSOLD
or:
DUMMY.exe VERSNEW
So I implemented a function setting a flag which version it is. For normal PROGRAM, SUBROUTINE etc. it is no issue but within this BLOCK DATA I can not just implement a CALL GETVERS ... IF (VERSOLD) THEN
I will get the following Error message from the compiler (gfortran -m32 -std=legacy -ffixed-line-length-none -c -g -fno-automatic -fno-second-underscore -finit-local-zero ) (..yes former g77..)
Error: Syntax error in DATA statement at (1)
Of course I could just implement a second BLOCK DATA with a different name.
I really wonder whether there is any other nice solution for this?

Related

How to read binary data written using MPI-IO in Fortran?

I want to read out some output data from a Fortran code to data files for postprocessing. The code given to me uses MPI to write to those data files. For the most basic case, when the number of processors should be 1 and also the Nx, Ny, Nz are all set to 1 and l is 3, the output of the written to the data file should be all 1. However, the output written to the data file is in some non-readable format, as below:
^#^#^#^#^#^#^#^#^#^#^#^#
The relevant portion of the code which writes to the data file is appended below.
do rst=1,1
fname='phi'
fname = trim('ufs')//":" // trim(fname)
write(buffer,"(i3.3)") 2*rst-1
fname = trim(adjustl(fname))//'.'//trim(adjustl(buffer))
print *, 'The output file to which all the data is written is ', fname
call MPI_FILE_OPEN(MPI_COMM_WORLD,fname,MPI_MODE_WRONLY+MPI_MODE_CREATE,MPI_INFO_NULL,ifile,ierr)
do l=1,numvar
disp = Nx_MOK*Ny_MOK*Nz_MOK*WP_MOK*(l-1)
call MPI_FILE_SET_VIEW(ifile,disp,MPI_REAL_SP,view,"native",MPI_INFO_NULL,ierr)
call MPI_FILE_WRITE_ALL(ifile,phi_xyz(1:Nxp,1:Nyp,1:Nzp,l,2*rst 1),Nxp*Nzp*Nyp, MPI_REAL_SP,status,ierr)
end do
call MPI_FILE_CLOSE(ifile,ierr)
call MPI_BARRIER(MPI_COMM_WORLD, ierr)
end do

BadDataError when editing a .dbf file using dbf package

I have recently produced several thousand shapefile outputs and accompanying .dbf files from an atmospheric model (HYSPLIT) on a unix system. The converter txt2dbf is used to convert shapefile attribute tables (text file) to a .dbf.
Unfortunately, something has gone wrong (probably a separator/field length error) because there are 2 problems with the output .dbf files, as follows:
Some fields of the dbf contain data that should not be there. This data has "spilled over" from neighbouring fields.
An additional field has been added that should not be there (it actually comes from a section of the first record of the text file, "1000 201").
This is an example of the first record in the output dbf (retrieved using dbview unix package):
Trajnum : 1001 2
Yyyymmdd : 0111231 2
Time : 300
Level : 0.
1000 201:
Here's what I expected:
Trajnum : 1000
Yyyymmdd : 20111231
Time : 2300
Level : 0.
Separately, I'm looking at how to prevent this from happening again, but ideally I'd like to be able to repair the existing .dbf files. Unfortunately the text files are removed for each model run, so "fixing" the .dbf files is the only option.
My approaches to the above problems are:
Extract the information from the fields that do exist to a new variable using dbf.add_fields and dbf.write (python package dbf), then delete the old incorrect fields using dbf.delete_fields.
Delete the unwanted additional field.
This is what I've tried:
with dbf.Table(db) as db:
db.add_fields("TRAJNUMc C(4)") #create new fields
db.add_fields("YYYYMMDDc C(8)")
db.add_fields("TIMEc C(4)")
for record in db: #extract data from fields
dbf.write(TRAJNUMc=int(str(record.Trajnum)[:4]))
dbf.write(YYYYMMDDc=int(str(record.Trajnum)[-1:] + str(record.Yyyymmdd)[:7]))
dbf.write(TIMEc=record.Yyyymmdd[-1:] + record.Time[:])
db.delete_fields('Trajnum') # delete the incorrect fields
db.delete_fields('Yyyymmdd')
db.delete_fields('Time')
db.delete_fields('1000 201') #delete the unwanted field
db.pack()
But this produces the following error:
dbf.ver_2.BadDataError: record data is not the correct length (should be 31, not 30)
Given the apparent problem that there has been with the txt2dbf conversion, I'm not surprised to find an error in the record data length. However, does this mean that the file is completely corrupted and that I can't extract the information that I need (frustrating because I can see that it exists)?
EDIT:
Rather than attempting to edit the 'bad' .dbf files, it seems a better approach to 1. extract the required data to a text from the bad files and then 2. write to a new dbf. (See Ethan Furman's comments/answer below).
EDIT:
An example of a faulty .dbf file that I need to fix/recover data from can be found here:
https://www.dropbox.com/s/9y92f7m88a8g5y4/p0001120110.dbf?dl=0
An example .txt file from which the faulty dbf files were created can be found here:
https://www.dropbox.com/s/d0f2c0zehsyy8ab/attTEST.txt?dl=0
To fix the data and recreate the original text file, this snippet should help:
import dbf
table = dbf.Table('/path/to/scramble/table.dbf')
with table:
fixed_data = []
for record in table:
# convert to str/bytes while skipping delete flag
data = record._data[1:].tostring()
trajnum = data[:4]
ymd = data[4:12]
time = data [12:16]
level = data[16:].strip()
fixed_data.extend([trajnum, ymd, time, level])
new_file = open('repaired_data.txt', 'w')
for line in fixed_data:
new_file.write(','.join(line) + '\n')
Assuming all your data files look like your sample (the big IF being the data has no embedded commas), then this rough code should help translate your text files into dbfs:
raw_data = open('some_text_file.txt').read().split('\n')
final_table = dbf.Table(
'dest_table.dbf',
'trajnum C(4); yyyymmdd C(8); time C(4); level C(9)',
)
with final_table:
for line in raw_data:
fields = line.split(',')
final_table.append(tuple(fields))
# table has been populated and closed
Of course, you could get fancier and use actual date, and number fields if you want to:
# dbf string becomes
'trajnum N; yyyymmdd D; time C(4), level N'
#appending data loop becomes
for line in raw_data:
trajnum, ymd, time, level = line.split(',')
trajnum = int(trajnum)
ymd = dbf.Date(ymd[:4], ymd[4:6], ymd[6:])
level = int(level)
final_table.append((trajnum, ymd, time, level))

converting pipe seperated raw data to xlsx using xlsxwriter module of python

I'm new to python and trying to write a small code to convert raw pipe seperated data to xlsx. Details as below :
raw data
a|b|c|d|
.
.
.
1000s of row
expected output
a b c d {each representing one cell in xlsx}
My code:
workbook=xlsxwriter.Workbook('ABC.xlsx')
worksheet=workbook.add_worksheet()
input_file=(raw_data.txt,'r')
row=0
col=0
for line in input_file:
line=line.split('|')
workseet.write(row,col,line)
row +=1
worsheet.close()
The output that i'm getting is :
a b c d (all in single cell)
Any pointers on what i'm missing here will be helpful.

PSS/E fortran array modules

I am developing a user-written control model for an inverter in PSS/E through Fortran language. I want to link this control model to a standard generator model.
I've already check that the control model works fine. I am entering 2 input commands: Real Power (P) and Reactive Power (Q). I have check through internal variables that the model takes well these values defined by me. And the outputs of this control model are "correct" both outpus signals are what are supposed to be.
The problem is that I'm not be able to link it to the standard generator model. As this model is contained in an external .dll file I use the USE [MODULE] statement at the beginning of the code and in MODE 3 I just write:
WIPCMD1(MC)=VAR(L+5)
WIQCMD1(MC)=VAR(L+4)
Where MC is the machine index where apply both models, WIPCMD and WIQCMD are the arrays that work as input of the generator model and VAR(L+5) and VAR(L+4) are the outputs from my user-written control model.
I know that the connection is wrong because when I change the 2 inputs of the control model, the 2 outputs of this control change too, but the outputs of the generator model are always de same (I think that are the initialized ones from the power flow).
Another possible cause is that I am compiling wrong the files in the PSS/E Environment Manager: I just put the my control model .flx file inside "User Model Fortran Source Files" and the .obj file with the same name of the .mod file that contains the GEPVMODULE in "User Model Object, Library Files". I use the resultant .dll file.
Any idea? Bellow is the whole code of the .flx control model.
Thank you very much.
Regards
SUBROUTINE INVCTRLELEC (MC,ISLOT)
C
USE GEPVMODULE
C
$INSERT COMON4.INS
C
INTEGER I,J,K,L,MC,ISLOT,BS
INTEGER IERR1,IERR3
REAL VINP1,VINP2,VINP3,VINP4,VINP5,VINP6,VINP7,VINP8,VINP9,VINP10
REAL VOUT1,VOUT2,VOUT3,VOUT4,VOUT5,VOUT6,VOUT7,VOUT8,VOUT9,VOUT10
C
C
I=STRTIN(4,ISLOT)
J=STRTIN(1,ISLOT)
K=STRTIN(2,ISLOT)
L=STRTIN(3,ISLOT)
C
CALL FLOW1(I,L+6,L+1,L+7)
C
IF (MODE.EQ.8)
. CON_DSCRPT(1)='Kqi'
. CON_DSCRPT(2)='Kvi'
. CON_DSCRPT(3)='Ihql'
. CON_DSCRPT(4)='Ipmax'
. CON_DSCRPT(5)='Iphl'
. CON_DSCRPT(6)='Vmaxcl'
. CON_DSCRPT(7)='Vmincl'
. CON_DSCRPT(8)='Vdc'
. CON_DSCRPT(9)='Porder'
. CON_DSCRPT(10)='Qorder'
. ICON_DSCRPT(1)='Bus origen'
. ICON_DSCRPT(2)='Bus destino'
. ICON_DSCRPT(3)='Identificador'
. RETURN
...FIN
C
IF (MODE.GT.4) GO TO 1000
C
GO TO (100,200,300,400), MODE
C
100 VOUT1=CON(J+9)-VAR(L+1)
VINP1=INT_MODE1(1.0,VOUT1,K,IERR1)
C
VOUT3=VOUT2-VAR(L+2)
VINP3=INT_MODE1(1.0,VOUT3,K+1,IERR3)
VAR(L+2)=ABS(VOLT(ICON(I)))
C
VAR(L+5)=WIPCMD1(MC)
VAR(L+4)=WIQCMD1(MC)
C
RETURN
C
200 VAR(L+2)=ABS(VOLT(ICON(I)))
C
VINP1=CON(J+9)-VAR(L+1)
VOUT1=INT_MODE2(1.0,VINP1,K)
VAR(L)=CON(J+9)
C
VINP2=VOUT1*CON(J)
VOUT2=MIN(CON(J+5),MAX(CON(J+6),VINP2))
C
VINP3=VOUT2-VAR(L+2)
VOUT3=INT_MODE2(1.0,VINP3,K+1)
VAR(L+11)=VOUT3*CON(J+1)
C
VINP5=VAR(L+5)
VOUT5=((CON(J+3))**(2)-(VAR(L+5))**(2))**(0.5)
C
VINP6=VOUT5
VOUT6=MIN(CON(J+2),VINP6)
VAR(L+8)=VOUT6
C
VINP7=VAR(L+2)
VOUT7=-0.5+0.01733*(VINP7*360-360)-0.0117*(CON(J+7)-560)+VAR(L+6)*1/4.37
VAR(L+9)=VOUT7
C
VINP4=VOUT3*CON(J+1)
VOUT4=MIN(VOUT6,MAX((MAX(VOUT7,-VOUT6)),VINP4))
VAR(L+4)=VOUT4
C
VINP8=VAR(L+2)
VOUT8=MAX(0.01,VAR(L+2))
C
VINP9=CON(J+8)
VAR(L+3)=CON(J+8)
VOUT9=VINP9/VOUT8
C
VINP10=VOUT9
VOUT10=MIN(MIN(CON(J+3),CON(J+4)),VINP10)
VAR(L+5)=VOUT10
C
RETURN
C
300 VAR(L+2)=ABS(VOLT(ICON(I)))
C
VINP1=CON(J+9)-VAR(L+1)
VOUT1=INT_MODE3(1.0,VINP1,K)
VAR(L)=CON(J+9)
C
VINP2=VOUT1*CON(J)
VOUT2=MIN(CON(J+5),MAX(CON(J+6),VINP2))
C
VINP3=VOUT2-VAR(L+2)
VOUT3=INT_MODE3(1.0,VINP3,K+1)
VAR(L+11)=VOUT3*CON(J+1)
C
VINP5=VAR(L+5)
VOUT5=((CON(J+3))**(2)-(VAR(L+5))**(2))**(0.5)
C
VINP6=VOUT5
VOUT6=MIN(CON(J+2),VINP6)
VAR(L+8)=VOUT6
C
VINP7=VAR(L+2)
VOUT7=-0.5+0.01733*(VINP7*360-360)-0.0117*(CON(J+7)-560)+VAR(L+6)*1/4.37
VAR(L+9)=VOUT7
C
VINP4=VOUT3*CON(J+1)
VOUT4=MIN(VOUT6,MAX((MAX(VOUT7,-VOUT6)),VINP4))
VAR(L+4)=VOUT4
C
VINP8=VAR(L+2)
VOUT8=MAX(0.01,VAR(L+2))
C
VINP9=CON(J+8)
VAR(L+3)=CON(J+8)
VOUT9=VINP9/VOUT8
C
VINP10=VOUT9
VOUT10=MIN(MIN(CON(J+3),CON(J+4)),VINP10)
VAR(L+5)=VOUT10
C
WIPCMD1(MC)=VAR(L+5)
WIQCMD1(MC)=VAR(L+4)
C
VAR(L+12)=MC
VAR(L+13)=ISLOT
C
RETURN
C
400 NINTEG=MAX(NINTEG,K+3)
RETURN
C
IF (MODE.EQ.6) GO TO 2000
C
900 RETURN
1000 RETURN
2000 RETURN
RETURN
END

Fortran95 -- Reading from a formatted text file

I need to read some values from a table. These are the first five rows, to give you some idea of what it should look like:
1 + 3 98 96 1
2 + 337 2799 2463 1
3 + 2801 3733 933 1
4 + 3734 5020 1287 1
5 + 5234 5530 297 1
My interest is in the first four columns of each row. I need to read these into arrays. I used the following code:
program ----
implicit none
integer, parameter :: totbases = 4639675, totgenes = 4395
integer :: codtot, ks
integer, dimension(totgenes) :: ngene, lend, rend
character :: genome*4639675, sign*4
open(1,file='e_coli_g_info')
open(2,file='e_coli_g_str')
do ks = 1, totgenes
read(1,100) ngene(ks),sign(ks:ks),lend(ks), rend(ks)
end do
100 format(1x,i4,8x,a1, 2(5x,i7), 22x)
do ks = 1, 100
write(*,*) ngene(ks), sign(ks:ks),lend(ks), rend(ks)
end do
end program
The loop at the end of the program is to print the first hundred entries to test that they are being read correctly. The problem is that I am getting this garbage (the fourth row is the problem):
1 + 3 757934891
2 + 337 724249387
3 + 2801 757803819
4 + 3734 757803819
5 + 5234 757935405
Clearly, the fourth column is way off. In fact, I cannot find these values anywhere in the file that I am reading from. I am using the gfortran compiler for Ubuntu 12.04. I would greatly appreciate if somebody would point me in the right direction. I'm sure it's likely that I'm missing something very obvious because I'm new at Fortran.
Fortran formats are (traditionally, there's some newer stuff that I won't go into here) fixed format, that is, they are best suited for file formats with fixed columns. I.e. column N always starts at character position M, no ifs or buts. If your file format is more "free format"-like, that is, columns are separated by whitespace, it's often easier and more robust to read data using list formatting. That is, try to do your read loop as
do ks = 1, totgenes
read(1, *) ngene(ks), sign(ks:ks), lend(ks), rend(ks)
end do
Also, as a general advice, when opening your own files, start from unit 10 and go upwards from there. Fortran implementations typically use some of the low-numbered units for standard input, output, and error (a common choice is units 1, 5, and 6). You probably don't want to redirect those.
PS 2: I haven't tried your code, but it seems that you have a bounds overflow in the sign variable. It's declared of length 4, but then you assign to index ks which goes all the way up to totgenes. As you're using gfortran on Ubuntu 12.04 (that is, gfortran 4.6), when developing compile with options "-O1 -Wall -g -fcheck=all"