This question already has answers here:
Fortran array automatically growing when adding a value
(1 answer)
How to add new element to dynamical array in Fortran 90
(2 answers)
Closed last year.
I'm new to Fortran so I'm really lost on how to do this. I have a data file with 5 groups of 5000 values. My code reads in the first 5000 values and puts them in an array called flux, does some stuff, then cycles back through to read the next set of 5000 values (while deleting the values in the original flux array and storing the new ones).
I need to be able to save each value so that I can plot them all later. I was hoping to to take the flux array values and append them to a different array, that way at the end of the loop I have one array with all 25,000 values in it. In Python this would be a one liner, but from what I'm reading online it seems this is not easy to do in Fortran.
Thanks in advance!
real, allocatable :: flux(:), s(:)
open(1,file='spec_1503-070_th45',status='old')
read(1,*) nf
read(1,*) (xlam(ij),ij = 1,nf)
read(1,'(25x,f10.1,12x,e10.3)')teff,grav
write(header,111) int(teff),alog10(grav)
do il=100,1000
read(1,*,end=99) bmag,az,ax,ay
read(1,*)(flux(ij),ij = 1,nf)
fmax=0.
do ij=1,nf
if(flux(ij).gt.fmax) fmax=flux(ij)
enddo
do ij=1,nf
flux(ij)=flux(ij)/fmax
enddo
s=[s,flux]
Appending to or resizing arrays in Fortran is as simple as declaring them as an allocatable array,
integer, allocatable :: vecA(:), vecB(:), vecC(:)
character(*), parameter :: csv = "(*(g0,:,', '))"
vecA = [1,2,3]
vecB = [4,5,6]
vecC = [vecA, vecB]
write(*,csv) "vecA", vecA
write(*,csv) "vecB", vecB
write(*,csv) "vecC", vecC
vecC = [vecC, vecC(size(vecC):1:-1)]
write(*,csv) "vecC", vecC
end
vecA, 1, 2, 3
vecB, 4, 5, 6
vecC, 1, 2, 3, 4, 5, 6
vecC, 1, 2, 3, 4, 5, 6, 6, 5, 4, 3, 2, 1
The left-hand side is automatically resized to the proper length. Notice, how vecC = [vecC,vecC] doubles the length of vecC. If you have performance-critical code, you probably would want to avoid such automatic reallocations. But that becomes relevant only when the code is to be called on the order of billions of times.
Related
Given:
a = [5, 2, 8, 3, 9]
b = [3, 5, 7, 6, 8]
c = [8, 5, 7, 4, 9].
What is needed:
d = [(9, 8), (8, 7), ..., (5, 5, 5), (5, 6, 5), (5, 6, 7), ..., (8, 7, 7), (9, 8, 9), ...].
Description:
(1) In the above example, there are three lists a, b, c having integer elements and the output is another list d of tuples.
(2) The tuples in d have elements belonging to (a and b and c) or (a and b) or (b and c) such that difference between elements within any tuple is not greater than 1.
(3) Problem: How to find the complete list d where we take any element from any input list and find the difference less than or equal to 1. Generalize to more than just three input list: a, b, c, d, e, ... and each one is having ~ 1000 elements. I also need to retrieve the indices relative to the input lists/ arrays that form the tuples.
(4) Clarification: (a) All such tuples which contain entries not differing by more than 1 are allowed.
(b) Tuples must have elements that are close to at least one other element by not more than 1.
(c) Entries within a tuple must belong to different input arrays/ lists.
Let me know if there are further clarifications needed!
You can use sorting to find results faster than a naive brute-force. That being said, this assumes the number of output tuple is reasonably small. Otherwise, there is no way to find a solution in a reasonable time (eg. several months). As #mosway pointed out in the comments, the number of combinations can be insanely huge since the complexity is O(N ** M) (ie. exponential) where N is the number of list and M is the length of the lists.
The idea is to use np.unique on all lists so to get many sorted arrays with unique items. Then, you can iterate over the first array, and for each number (in the first array), find the range of values in the second one fitting in [n-1;n+1] using a np.searchsorted. You can then iterate over the filtered values of the second array and recursively do that on other array.
Note that regarding which array is chosen first, the method can be significantly faster. Thus, a good heuristic could be to select an array containing values very distant from others. Computing a distance matrix with all the values of all array and selecting the one having the biggest average distance should help.
Note also that using Numba should significantly speed up the recursive calls.
I am kind of new in the fortran proramming.
Can anyone please help me out with the solution.
i am having a problem of generating integer random number
in the range [0,5] in fortran random number using
random_seed and rand
To support the answer by Alexander Vogt, I'll generalize.
The intrinsic random_number(u) returns a real number u (or an array of such) from the uniform distribution over the interval [0,1). [That is, it includes 0 but not 1.]
To have a discrete uniform distribution on the integers {n, n+1, ..., m-1, m} carve the continuous distribution up into m+1-n equal sized chunks, mapping each chunk to an integer. One way could be:
call random_number(u)
j = n + FLOOR((m+1-n)*u) ! We want to choose one from m-n+1 integers
As you can see, for the initial question for {0, 1, 2, 3, 4, 5} this reduces to
call random_number(u)
j = FLOOR(6*u) ! n=0 and m=5
and for the other case in your comment {-1, 0, 1}
call random_number(u)
j = -1 + FLOOR(3*u) ! n=-1 and m=1
Of course, other transformations will be required for sets of non-contiguous integers, and one should pay attention to numerical issues.
What about:
program rand_test
use,intrinsic :: ISO_Fortran_env
real(REAL32) :: r(5)
integer :: i(5)
! call init_random_seed() would go here
call random_number(r)
! Uniform distribution requires floor: Thanks to #francescalus
i = floor( r*6._REAL32 )
print *, i
end program
Suppose I have an array A in Fortran of dimension 10 with numbers.
However I'm only interested in a subset of those numbers (for example 3).
I store those number in a smaller array B
B(1) = A(1)
B(2) = A(5)
B(3) = A(6)
I can also define a mapping table to store index 1, 5, 6 for example
MAP(1) = 1
MAP(2) = 5
MAP(3) = 6
How can I create an inverse map INVMAP such that
INVMAP(1) = 1
INVMAP(5) = 2
INVMAP(6) = 3
with the constrain that INVMAP has dimension 3 (and not 10).
The point is that the array A is too big to be stored in memory and B
is obtained iteratively (A is never really allocated).
Considerations:
I do not care about the 7 discarded values but I care about the position of the one we keep.
Since MAP and INVMAP are storing positions, there will never be collision (its a one to one correspondence).
Maybe it could be possible with HASH or Fortran table but I'm not really sure how because I'm mapping numbers, not keys. Any idea ?
Thanks a lot,
Sam
Here's a very simple solution. No Fortran on this machine so not entirely sure that I have the syntax absolutely correct. Define a derived type like this:
type :: row
integer :: a_index
integer :: a_value ! I've assumed that your A array contains integers
! use another type if you want to
end type
then
type(row), dimension(100) :: b ! In practice you'll probably want b to be
! allocatable
and
b(1) = (1, a(1)) ! each row of b contains the value at an index into a and
! the index
b(2) = (5, a(5))
b(3) = (6, a(6))
Now your map function is simply, in pseudo-code, map(n) = b(n)%a_index
and your inverse map is, again in pseudo-code, invmap(n) = findloc(b%a_index, n).
Since the inverse map is a simple scan, it might become too time-consuming for your taste when b becomes large. Then I might introduce an auxiliary index array pointing into b at intervals, or I might go crazy and start a binary search of b%a_index.
I can also define a mapping table to store index 1, 5, 6 for example
MAP(1) = 1
MAP(2) = 5
MAP(3) = 6
I don't know if you know, but Fortran has a nice feature (one of my favorites) known as Vector Subscripts. You can pass an 'array of indices' as an index to an array, and get the elements corresponding to each index, like this:
integer :: A(10) = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
integer :: map(3) = [1, 5, 6]
print *, A(map)
! outputs 10 50 60
How can I create an inverse map INVMAP such that
INVMAP(1) = 1
INVMAP(5) = 2
INVMAP(6) = 3
Well, if your INVMAP will have a domain and an image of different sizes, it must be a function, not an array. I mean, INVMAP cannot be an array of length 3 and accept indices from 1 to 6, right? Arrays in Fortran (and in most languages) imply contiguous indices.
The intrinsic function FINDLOC can be pretty handy here (I am asuming your function is bijector).
function invmap(map, ids)
integer :: ids(:), map(:), invmap(size(ids))
invmap = [(findloc(map, i), i = 1, size(ids))]
end
You could use this function to relate each map value to its position on map
integer :: myinvmap = invmap(map, [6, 1, 5])
print *, myinvmap ! outputs 3 1 2
print *, invmap(map, [5]) ! outputs 2
The point is that the array A is too big to be stored in memory and B
is obtained iteratively (A is never really allocated).
Now, if you will never allocate the big array, then its values will also be accessed by some function (you can consider it a function actually). You have basically two options here:
Have two arrays, one with the values got from big_array_function, and one with the parameter you passed to big_array (the indices).
Have one array of pairs [index, value]. It is the answer that #HighPerformanceMark provided.
Alternatively... (and not tested...)
Integer, dimension(100) :: A
Logical, dimension(100) :: A_Mask
Integer, dimension( 3) :: B
Integer, dimension(. 3) :: A_pos
Integer, dimension(. 3) :: I, J
A_Mask = .false.
A_Mask(1) = .true.
A_Mask(1) = .true.
A_Mask(1) = .true.
B = PACK(A, MASK=A_Mask)
J = 0
Collect_Positions: Do I = 1, SIZE(A)
If(.not. A_Mask(I)) CYCLE Collect_Positions
J = J+1
A_Pos(J) = I
ENDDO Collect_Positions
...
And then if one want to UNPACK, then the mask has the position... so it is possible to not worry about the position of A in general sense (but may be needed in the OP's case)
I have an array with multiple dimensions (the goal is to allow for about 100) and each dimension has a size of about 2^10 and I only need to store in it about 1000 double precision coefficients. I don't need to do any operation with this array aside from reading and writing into it. The code is written in Fortran 90.
I assume that if I a library like one of the ones mentioned in this answer I would be able to store the do this, but would this be optimized for the simple reading and writing operations? Is there a library that would be most efficient for that purpose?
Edit: By "simple reading and writing operations" I mean the following. Suppose
REAL(8), DIMENSION(1000) :: coeff1
INTEGER, DIMENSION(1000,5) :: index
I want to define coeff2 to store the values in coeff1 and then read itat the indices in index, that is
DO i = 1,1000
index(i,:) = [something]
coeff1(i) = [another something]
coeff2(index(i,1),index(i,2),index(i,3),index(i,4),index(i,5)) = coeff1(i)
ENDDO
Then, for any i I would like to access the value of
coeff2(index(i,1),index(i,2),index(i,3),index(i,4),index(i,5))
as quickly as possible. Being able to do this fast is what I mean by "efficient".
Since the indices in [something] are at most 2^10 I am currently defining coeff2 as follows:
REAL(8), DIMENSION(2**10,2**10,2**10,2**10,2**10) :: coeff2
but this is too wasteful of memory specially since I need to increase the number of dimensions, now 5, to the order of 100 and most elements of this array are equal to 0. So, another measure of efficiency that is relevant to me is that the memory necessary to store coeff2 should not explode as I increase the number of dimensions.
Well, It's still not totally clear to me the nature of your data and the way you want to use it.
If what you need is indexed data, whose indices are not consecutive,
Sparse matrix can be an answer, and there are many solutions already implemented over the internet (as shown in the link you provided). But maybe it would be overkill for what I think you are trying to do. Maybe a simple datatype could serve your purpose, like this:
program indexed_values
implicit none
type :: indexed
integer :: index
real(8) :: value
end type
integer, parameter :: n_coeffs = 1000
integer, parameter :: n_indices = 5
integer :: i
real(8), dimension(n_coeffs) :: coeff1
integer, dimension(n_coeffs, n_indices) :: index
type(indexed), dimension(n_coeffs, n_indices) :: coeff2
type(indexed) :: var
do i = 1, n_coeffs
index(i, :) = [1, 2, 4, 16, 32] * i ! your calc here
coeff1(i) = real(i * 3, 8) ! more calc here
coeff2(i, :)%index = index(i, :)
coeff2(i, :)%value = coeff1(i)
end do
! that's how you fetch the indices and values by stored position
var = coeff2(500, 2)
print*, var%index, var%value ! outputs: 1000 1500.0
! that's how you fetch a value by its index
print*, fetch_by_index(coeff2(500, :), 1000) ! outputs: 1500.0
contains
real(8) function fetch_by_index(indexed_pairs, index)
type(indexed), dimension(:) :: indexed_pairs
integer, intent(in) :: index
integer :: i
do i=1, size(indexed_pairs)
if(index == indexed_pairs(i)%index) then
fetch_by_index = indexed_pairs(i)%value
return
end if
end do
stop "No value stored for this index"
end
end
The provided function for fetching values by its indices could be improved if your indices will be alwyas stored in ascending order (no need to traverse the whole list to fail). Moreover, if you will assing a constant result of coeff1 to all the indices at each row, you could do even better and just not having a coeff2 array at all, just have coeff1 for values and index for the indices, and correlate them by position.
I am trying to find the most efficient way to do slicing for a 3D numpy array. This is a subset of the data, just for test purposes :
in_arr =np.array([[[0,1,2,5],[2,3,2,6],[0,1,3,2]],[[1,2,3,4],[3,1,0,5],[2,4,0,1]]])
indx =[[3,1,2],[2,0,1]]
I need to get the value at the indx as stated. For example, indx[0][0] is 3, so I am looking for the 3rd elem of in_arr[0][0], in this case, 5.
I have the following code that will do what i need it to do, but the time complexeity is n^2, which I am not happy about.
list_in =[]
for x in range(len(indx)):
arr2 = []
for y in range(len(indx[x])):
arr2.append(in_arr[x][y][indx[x][y]])
#print in_arr[x][y][indx[x][y]]
list_in.append(arr2)
print list_in
I am looking for a very fast and efficient way to do the same task for a large dataset.
You can do this efficiently using broadcasted arrays of indices; for example:
i1 = np.arange(2)[:, np.newaxis]
i2 = np.arange(3)[np.newaxis, :]
i3 = np.array(indx)
in_arr[i1, i2, i3]
# array([[5, 3, 3],
# [3, 3, 4]])
What numpy does here is to effectively match the entries of the three index arrays, and extract the associated entries from in_arr: the reason for the [:, np.newaxis] and [np.newaxis, :] terms is that it reshapes the three arrays to be compatible via numpy's broadcasting rules.