This question already has answers here:
finding specific indices with pointer array
(3 answers)
Closed 2 years ago.
I have the following array
integer, dimension(4) :: my_array = (/160,0,230,0/)
I would like to find the indices of the elements that are non-zero and store them in another array or individual variables for future use.
I am not sure how to do this because I don't know a prior how many elements are non-zero. I was thinking of using a loop combined with count(my_array/=0) and maxloc.
Is some sort of loop the only way? I can't think of a good way to use WHERE or FINDLOC.
I have tried this
ii = COUNT(my_array.NE.0)
ALLOCATE(choices(ii))
choices = PACK(my_array,my_array.NE.0)
But only makes a new array without zero elements, so I lose the original indices.
I'm not 100% sure I understand what you want, but does this do it?
ian#eris:~/work/stack$ cat pack.f90
Program pack_index
Implicit None
Integer, Dimension( 1:4 ) :: my_array = [ 160, 0, 230, 0 ]
Integer, Dimension( : ), Allocatable :: choices
Integer, Dimension( : ), Allocatable :: indices
Integer :: i
indices = Merge( 0, [ ( i, i = 1, Size( my_array ) ) ], my_array == 0 )
choices = Pack( indices, indices /= 0 )
Write( *, * ) choices
End Program pack_index
ian#eris:~/work/stack$ gfortran-8 -std=f2008 -fcheck=all pack.f90
ian#eris:~/work/stack$ ./a.out
1 3
Related
I want to add elements to a 1d matrix mat, subject to a condition as in the test program below. In Fortran 2003 you can add an element
mat=[mat,i]
as mentioned in the related question Fortran array automatically growing when adding a value. Unfortunately, this is very slow for large matrices. So I tried to overcome this, by writing the matrix elements in an unformatted file and reading them afterwards. This turned out to be way faster than using mat=[mat,i]. For example for n=2000000_ilong the run time is 5.1078133666666661 minutes, whereas if you store the matrix elements in the file the run time drops to 3.5234166666666665E-003 minutes.
The problem is that for large matrix sizes the file storage.dat can be hundreds of GB...
Any ideas?
program test
implicit none
integer, parameter :: ndig=8
integer, parameter :: ilong=selected_int_kind(ndig)
integer (ilong), allocatable :: mat(:)
integer (ilong), parameter :: n=2000000_ilong
integer (ilong) :: i, cn
logical, parameter :: store=.false.
real(8) :: z, START_CLOCK, STOP_CLOCK
open(1, file='storage.dat',form='unformatted')
call cpu_time(START_CLOCK)
if(store) then
cn=0
do i=1,n
call random_number(z)
if (z<0.5d0) then
write(1) i
cn=cn+1
end if
end do
rewind(1); allocate(mat(cn)); mat=0
do i=1,cn
read(1) mat(i)
end do
else
allocate(mat(1)); mat=0
do i=1,n
call random_number(z)
if (z<0.5d0) then
mat=[mat,i]
end if
end do
end if
call cpu_time(STOP_CLOCK)
print *, 'run took:', (STOP_CLOCK - START_CLOCK)/60.0d0, 'minutes.'
end program test
If the data file has hundreds of gigabytes, than there can may be no solution available at all, because you need so much RAM memory anyway for your array. Maybe you made the mistake of storing the data as text and then the memory size will be somewhat lower, but still tens of GB.
What is often done, when you need to add elements one-by-one and you do not know the final size, is growing the array geometrically in steps. That means pre-allocate an array to size N. When the array is full, you allocate a new array of size 2*N. When the array is full again, you allocate it to 4*N. And so on. Either you are finished or you exhausted all your memory.
Of course, it is often best to know the size of the array beforehand, but in some algorithms you simply do not have the information.
Maybe you need a dynamic container such as C++'s std::vector, with a push_back() function.
The following is a simplified version. You probably ought to check the allocation to make sure that you don't run out of addressable memory.
Note the need for random_seed.
module container
use iso_fortran_env
implicit none
type array
integer(int64), allocatable :: A(:)
integer(int64) num
contains
procedure push_back
procedure print
end type array
interface array ! additional constructors
procedure array_constructor
end interface array
contains
!----------------------------------------------
function array_constructor() result( this ) ! performs initial allocation
type(array) this
allocate( this%A(1) )
this%num = 0
end function array_constructor
!----------------------------------------------
subroutine push_back( this, i )
class(array), intent(inout) :: this
integer(int64) i
integer(int64), allocatable :: temp(:)
if ( size(this%A) == this%num ) then ! Need to resize
allocate( temp( 2 * this%num ) ) ! <==== for example
temp(1:this%num ) = this%A
call move_alloc( temp, this%A )
! print *, "Resized to ", size( this%A ) ! debugging only!!!
end if
this%num = this%num + 1
this%A(this%num) = i
end subroutine push_back
!----------------------------------------------
subroutine print( this )
class(array), intent(in) :: this
write( *, "( *( i0, 1x ) )" ) ( this%A(1:this%num) )
end subroutine print
end module container
!=======================================================================
program test
use iso_fortran_env
use container
implicit none
type(array) mat
integer(int64) :: n = 2000000_int64
integer(int64) i
real(real64) z, START_CLOCK, STOP_CLOCK
mat = array() ! initial trivial allocation
call random_seed ! you probably need this
call cpu_time(START_CLOCK)
do i = 1, n
call random_number( z )
if ( z < 0.5_real64 ) call mat%push_back( i )
end do
call cpu_time(STOP_CLOCK)
print *, 'Run took ', ( STOP_CLOCK - START_CLOCK ) / 60.0_real64, ' minutes.'
! call mat%print ! debugging only!!!
end program test
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed last month.
Improve this question
I am looking for fortran routines that perform ranking. That is given a one dimensional array I would like a routine that returns the indices in that array of the (descending) sorted values. All I could fined are routines that perform sorting. Additionally it would be nice to have a routine that performs partial ordering. That is returning for a given number M the indices of the M biggest values - and others in any order. I found dlasrt but it only does sorting...
Well, for partial-sorting you just can just choose a sorting algorithm like selection sort and terminate after the first required number of elements rather than all of them. Make sure that you (partial-)sort a parallel array of indices rather than the array itself.
There's a partial_sort algorithm built into the standard library of C++, but not Fortran. Maybe you could check out how that is implemented.
module sorting
implicit none
contains
subroutine swap( a, b )
integer, intent(inout) :: a, b
integer temp
temp = a
a = b
b = temp
end subroutine swap
function best_n( A, nfirst ) result( indices )
integer, intent(in) :: A(:)
integer, intent(in) :: nfirst
integer, allocatable :: indices(:)
integer n
integer i, j, jmin
n = size( A )
indices = [ ( i, i = 1, n ) ]
do i = 1, nfirst
jmin = i
do j = i + 1, n
if ( A(indices(j)) > A(indices(jmin)) ) jmin = j
end do
if ( jmin /= i ) call swap( indices(i), indices(jmin) )
end do
end function best_n
end module sorting
!======================================================================
program main
use sorting
implicit none
integer, allocatable :: A(:), indices(:)
integer n
character(len=*), parameter :: fmt = "( a, *(i3,1x) )"
A = [ 1, -1, 2, -2, 3, -3 ]
n = 3
indices = best_n( A, n )
write( *, fmt ) "Original array: ", A
write( *, fmt ) "Best indices: ", indices(1:n)
write( *, fmt ) "Best values: ", A(indices(1:n))
end program main
Original array: 1 -1 2 -2 3 -3
Best indices: 5 3 1
Best values: 3 2 1
I have the following code and I would like to save the value of idx to use afterwards.
program use_value_allocatable
implicit none
integer :: i, ii
integer, dimension(3) :: array_save
character(1), dimension(3) :: array_char_ref = (/'a','b','c'/), array_char_1 = (/'c','a','b'/)
integer, allocatable :: idx(:)
array_save = 0
do i = 1, 3
idx = pack([(ii,ii=1,3)], array_char_ref == array_char_1(i) )
print*, 'i=', i, ', idx=', idx, ', array_save(i) =', array_save(i)
!!$ array_save(i) = idx
deallocate(idx)
end do
end program use_value_allocatable
Having array_save(i) = idx in the code leads to an error as follows:
Error: Incompatible ranks 0 and 1 in assignment at (1)
So, I can conclude that I cannot use the value of an allocatable variable (here idx). How I can circumvent this problem?
P.S.: in this example I assume that idx will always be an integer of dimension 1
As you say, "idx will always be an integer of dimension 1", so you just need
array_save(i) = idx(1)
i.e. you need to store the first (and only) element of idx in array_save(i), rather than the whole array.
A side note: you do not need the line deallocate(idx). idx will be implicitly re-allocated by the line idx = ..., and will be automatically deallocated when it drops out of scope.
Let's say we have several different contiguous arrays
real :: a1(2),a2(2),a3(2)
real :: b1(2,2), b2(2,2),b3(2,2)
real :: c1(3,3,3), c2(3,3,3),c3(3,3,3)
and a subroutine
subroutine mtpy(x,y,z)
real,contiguous, intent(in) :: x(:), y(:)
real, intent(out) :: z(size(x))
z=x*y
end subroutine mtpy
How to use mtpy in the following series of calls:
call mtpy(a1,a2,a3)
call mtpy(b1,b2,b3)
call mtpy(c1,c2,c3)
Obviously, this is going to cause the compiler errors as the shape of the actual and dummy arguments do not match. In cases like these, I used to declare several specific procedures that each handles a specific shape and then wrap all of them using an interface. However, this is quite tedious (Imagine having huge amount of simple elemental function and pure procedures that treat multidimensional arrays (with up to three dimensions) as single dimension arrays and then providing instances such as sub_1d, sub_2d, sub_3d, .. for each of them despite all of them actually doing the same job).
One partial solution is to, I suppose, use RESHAPE
call mtpy(reshape(b1,[4]),reshape(b2,[4]),bb)
but, can I be certain that the compiler (I am interested in gfortran and ifort mostly) won't start creating 1d temporaries to hold the reshaped b1 and b2 arrays?
Now, I am also aware that one can declare an array pointer such as
real, pointer, contiguous :: p1(:),p2(:),p3(:)
and make the following pointer assignment such as
p1(1:size(c1))=>c1
However, this approach has a drawback that I need to declare the original arrays as targets. Isn't this going to impact on the optimisations the compiler is going to be able to perform?
Yet another solution is, I suppose, to use assumed size arrays, but I noticed that Metcalf et al call their usage 'deprecated' and again, I am not sure about the impact on omptimisations.
So, is there a simple way of treating a multi-dimensional fortran array as a single dimension array (in a subroutine, or a fuction) which does not impose unnecessary assumptions (such as TARGET) on that array? If I can use RESHAPE without fear of creating temporaries (I am only dealing with contiguous arrays) I'd go for that. Any suggestions?
The future Fortran 2018 standard will provide assumed-rank arrays (that will allow to receive arrays of any rank) and a select rank construct that will easily allow to address situations like this with one assumed-rank array (see, e.g., The new features of Fortran 2018 starting on page 16), and with more difficulty with multiple assumed-rank arrays.
Assumed-size arrays, while not fashionable or recommended are valid, non-obsolescent features of the current standard (Fortran 2008), as well as of the next standard draft (Fortran 2018), so they can be used if needed. Since a lot of Fortran 77 code depends on this, and much of it is decades old, I would expect it to be significantly optimised in most compilers.
However, you don't need to use assumed-size arrays, you can use explicit-shape arrays (arrays with explicit dimensions), and as long as the array actual arguments have enough elements the code will be valid, since, according to paragraph 4 of section 12.5.2.11 of the 2008 standard,
An actual argument that represents an element sequence and corresponds to a dummy argument that is an array is sequence associated with the dummy argument if the dummy argument is an explicit-shape or assumed-size array. The rank and shape of the actual argument need not agree with the rank and shape of the dummy argument, but the number of elements in the dummy argument shall not exceed the number of elements in the element sequence of the actual argument. If the dummy argument is assumed-size, the number of elements in the dummy argument is exactly the number of elements in the element sequence.
So you could
call mtpy(a1,a2,a3,size(a3))
call mtpy(b1,b2,b3,size(b3))
call mtpy(c1,c2,c3,size(c3))
...
subroutine mtpy(x,y,z,n)
integer, intent(in) :: n
real, intent(in) :: x(n), y(n)
real, intent(out) :: z(n)
z=x*y
end subroutine mtpy
Because I'm also not sure whether reshape() makes a temporary array even for contiguous cases, I've tried printing the address of the original and passed arrays by c_loc(). Then, even for small 1-dimensional arrays, reshape() in both gfortran-8 and ifort-16 seems to create temporaries (because the address of the first element is different). So, it seems safer to assume that temporaries are created even for simple cases (for more info, please see comments by francescalus below.)
module test
use iso_c_binding, only: c_loc
implicit none
interface linear
module procedure linear_r2d, linear_r3d
endinterface
contains
subroutine calc_ver1( a ) !! assumed-shape dummy array
real, contiguous, target :: a(:)
print *, "addr = ", c_loc( a(1) )
print *, "vals = ", a
endsubroutine
subroutine calc_ver2( a, n ) !! explicit-shape dummy array
integer :: n
real, target :: a( n )
print *, "addr = ", c_loc( a(1) )
print *, "vals = ", a
endsubroutine
function linear_r2d( a ) result( ptr ) !! returns a 1-d pointer from 2-d array
real, contiguous, target :: a(:,:)
real, contiguous, pointer :: ptr(:)
ptr( 1 : size(a) ) => a
endfunction
function linear_r3d( a ) result( ptr ) !! returns a 1-d pointer from 3-d array
real, contiguous, target :: a(:,:,:)
real, contiguous, pointer :: ptr(:)
ptr( 1 : size(a) ) => a
endfunction
endmodule
program main
use test
implicit none
integer i
real, target :: a(2), b(2,2), c(2,2,2)
a = [1,2]
b = reshape( [( 2*i, i=1,4 )], [2,2] )
c = reshape( [( 3*i, i=1,8 )], [2,2,2] )
print *, "addr(a) = ", c_loc( a(1) )
print *, "addr(b) = ", c_loc( b(1,1) )
print *, "addr(c) = ", c_loc( c(1,1,1) )
print *, "[ use assumed-shape dummy ]"
call calc_ver1( a )
! call calc_ver1( b ) ! rank mismatch
! call calc_ver1( c ) ! rank mismatch
print *, "--- with reshape() ---"
call calc_ver1( reshape( b, [size(b)] ) )
call calc_ver1( reshape( c, [size(c)] ) )
print *, "--- with linear() ---"
call calc_ver1( linear( b ) )
call calc_ver1( linear( c ) )
print *
print *, "[ use explicit-shape dummy ]"
call calc_ver2( a, size(a) )
call calc_ver2( b, size(b) )
call calc_ver2( c, size(c) )
end
Result of ifort-16 on Linux:
addr(a) = 7040528
addr(b) = 7040544
addr(c) = 7040560
[ use assumed-shape dummy ]
addr = 7040528
vals = 1.000000 2.000000
--- with reshape() ---
addr = 140736361693536
vals = 2.000000 4.000000 6.000000 8.000000
addr = 140736361693560
vals = 3.000000 6.000000 9.000000 12.00000 15.00000 18.00000 21.00000 24.00000
--- with linear() ---
addr = 7040544
vals = 2.000000 4.000000 6.000000 8.000000
addr = 7040560
vals = 3.000000 6.000000 9.000000 12.00000 15.00000 18.00000 21.00000 24.00000
[ use explicit-shape dummy ]
addr = 7040528
vals = 1.000000 2.000000
addr = 7040544
vals = 2.000000 4.000000 6.000000 8.000000
addr = 7040560
vals = 3.000000 6.000000 9.000000 12.00000 15.00000 18.00000 21.00000 24.00000
Result of gfortran-8 on OSX10.11:
addr(a) = 140734555734776
addr(b) = 140734555734752
addr(c) = 140734555734720
[ use assumed-shape dummy ]
addr = 140734555734776
vals = 1.00000000 2.00000000
--- with reshape() ---
addr = 140734555734672
vals = 2.00000000 4.00000000 6.00000000 8.00000000
addr = 140734555733984
vals = 3.00000000 6.00000000 9.00000000 12.0000000 15.0000000 18.0000000 21.0000000 24.0000000
--- with linear() ---
addr = 140734555734752
vals = 2.00000000 4.00000000 6.00000000 8.00000000
addr = 140734555734720
vals = 3.00000000 6.00000000 9.00000000 12.0000000 15.0000000 18.0000000 21.0000000 24.0000000
[ use explicit-shape dummy ]
addr = 140734555734776
vals = 1.00000000 2.00000000
addr = 140734555734752
vals = 2.00000000 4.00000000 6.00000000 8.00000000
addr = 140734555734720
vals = 3.00000000 6.00000000 9.00000000 12.0000000 15.0000000 18.0000000 21.0000000 24.0000000
And I also think that explicit-shape dummy arrays are useful depending on cases, and the code in the Question seems precisely the case. (Because the actual argument is contiguous, there is no array temporary created.) If the size argument n in calc_ver2() is not desired, we could use a function that returns a 1-d array pointer (see linear() above), but I guess this may be overkill considering the simplicity of calc_ver2()... (By the way, I've attached target in various places in the code, which is simply because c_loc() requires it).
This is my code:
Program Arrays_0
Implicit none
Integer :: i , Read_number , Vig_Position , Vipg_Position , n_iter
Integer , parameter :: Br_gra = 12
Integer , parameter , dimension ( Br_gra ) :: Vig = [ ( i , i = 1 , Br_gra) ]
Integer , parameter , dimension ( Br_gra ) :: Vipg = [ 0 , 1 , 1 , 1 , 2 , 2 , 3 , 4 , 4 , 7 , 7 , 7 ]
Integer :: Result_of_calculation
Write(*,*)"Enter the number (From 1 to Br_gra):"
Read(*,*) Read_number
Vig_Position = Vig(Read_number)
Vipg_Position = Vipg(Vig_Position)
!K_str( Vig_Position_temp ) = Vig_Position_temp + 2.3
n_iter = 0
Result_of_calculation = Vig_Position
Do while( Vipg_Position .ne. Vipg(1) )
n_iter = n_iter + 1
Vig_Position = Vipg_Position
! K_str( Vig_Position_temp ) = Vig_Position_temp + 2.3
Result_of_calculation = Result_of_calculation + Vig_Position
Vipg_Position = Vipg(Vig_Position)
End Do
Write(*,'(a,1x,i0)')"The number of iteration is:",n_iter
Write(*,'(a,1x,i0)')"The result of calculation is:",Result_of_calculation
End Program Arrays_0
There is no problem with code if I want to make calculation for a n_iter and Result_of_calculation but I have a problem with declaration of K_str in way that can follow correctly specific use of this two variables (my intention for using this variables in calculation was showed in comments).
So question is how to declare, for example, in case that Read_number is 12?
In that case I have: K_str(12), K_str(7), K_str(3) and K_str(1).
What I can do is this:
Real, dimension (Br_gra):: K_str
But in this case a must import one more loop for all elements from Vig (12 calculation). I want to prevent that number of calculation and in this case, I want to that my code make just a 4 calculation.
How to do that?
So you want to get an array, which e.g. starts at index 1, ends at index 12, but does not contain all the indexes in between, just some of them?
That is not possible with Fortran arrays. Actually, it is not possible with arrays in any other language I know.
One can use the dictionary data structure for something like that, which is intrinsic in some languages, but not Fortran. There are external libraries for similar data-structures in Fortran. See http://fortranwiki.org/fortran/show/Hash+tables
You could also use a linked list with all usual drawbacks and advantages (no direct indexing etc.).
Unless your need is for some very large ranges of indexes, much much larger than your example, use a regular array that contains all indexes.
You can also use one array which will contain the data (indexed 1 to 4) and another array of the same size, which will contain the global position (1,3,7 and 12). Or a derived type with these two components. But it will not be the same usage as you propose.