So reserve is quite useful when you have a rough idea of your size requirements. Does anyone know of a similar method to pre-allocate arrays in MATLAB?
I'm not really interested in hacky (but effective) methods like the following:
x = zeros(1000,1);
for i = 1:10000
if i > numel(x)
x = [x;zeros(size(x))];
end
x(i) = rand;
end
x(i+1:end) = [];
The "hacky" way is the only way to do it. However, you do not need to check i <= numel(x). The array will be expanded automatically (but without array doubling):
x = zeros(1000,1);
for i = 1:10000
x(i) = rand;
end
x(i+1:end) = [];
EDIT: To keep it simple while still retaining the array doubling, you can write a class, or simply a few helper functions (below).
EDIT2: The usage of helper functions will slow things down compared to the manual hack. In MATLAB 2010 it is still much faster than naive growth. In MATLAB 2011 the naive approach is actually faster, suggesting that this version has smarter allocation. Perhaps it is fast enough so that no hack is needed at all. Thanks to Andrew Janke for pointing this out.
function listtest()
n = 10000;
l = new_list();
for i=1:n
l = list_append(l, i);
end
a = list_to_array(l);
end
function l = new_list()
l = [0 0];
end
function l = list_append(l, e)
if l(1)+1 == length(l)
l(length(l)*2) = 0;
end
l(1) = l(1)+1;
l(l(1)+1) = e;
end
function a = list_to_array(l)
a = l(2:1+l(1));
end
EDIT (from AndrewJanke)
Here's code to compare the speed of the implementations.
function manual_reserve_example(n)
x = zeros(1000,1);
for i = 1:n
if i > numel(x)
x = [x;zeros(size(x))];
end
x(i) = i;
end
x(i+1:end) = [];
end
function naive_growth(n)
x = 0;
for i = 1:n
x(i) = i;
end
end
function compare_them(n)
fprintf('Doing %d elements in Matlab R%s\n', n, version('-release'));
tic;
naive_growth(n);
fprintf('%30s %.6f sec\n', 'naive_growth', toc);
tic;
manual_reserve_example(n);
fprintf('%30s %.6f sec\n', 'manual_reserve', toc);
tic;
listtest(n);
fprintf('%30s %.6f sec\n', 'listtest', toc);
end
The cleanest solution to the example that you provided is to iterate backwards.
for i = 10000:-1:1
x(i) = rand;
end
This does not work in cases where the end size is actually unknown, but it has come in handy for me more often than I would have expected.
Otherwise I usually implement a "double on overflow" algorithm like you show in the original question.
The clean solution is to wrap a Matlab class around a respectible vector resize algorithm, and then use that class. I am not aware of any reason such a class could not be built, but I've never actually sat down and tried to implement one. (I'm curious if an example already exists on the file exchange somewhere.)
There is a way to preallocate memory for a structure in MATLAB 7.6 (R2008a) using the STRUCT and REPMAT commands.
EXAMPLE 1: A structure with two fields
s.field1
s.field2
s = struct('field1',cell(1),'field2',cell(1));
EXAMPLE 2: A structure with a field with a subfield
s.field1.subfield
s = struct('field1',struct('subfield',cell(1)));
EXAMPLE 3: An array of structures
v(1).field1
...
v(100).field1
s = struct('field1',cell(1));
v = repmat(s,100,1);
Related
I have recently started to parallelize a serial code I've been developing and was curious if anyone had input on how to properly apply OpenMP for these loops.
F_vol = 0.0_bp
G_vol = 0.0_bp
H_vol = 0.0_bp
RHS_vol = 0.0_bp
!$OMP PARALLEL DO PRIVATE(e,i1,j1,F,G,H)
DO e=1,NE
DO i1=1,Np
DO j1=1,NPTS
CALL Flux(Q(j1,e,:),F,G,H)
F_Vol(i1,e,:) = F_Vol(i1,e,:) + Stuff(j1)*F(:)
G_Vol(i1,e,:) = G_vol(i1,e,:) + Stuff(j1)*G(:)
H_Vol(i1,e,:) = H_vol(i1,e,:) + Stuff(j1)*H(:)
END DO
END DO
END DO
!$OMP END PARALLEL DO
As a note, the arrays, F, G, and H are of size 5 and are temporary arrays. Additionally, F_Vol,G_Vol,H_Vol is of dimension (NE,Np,5) The part I am unsure on is, how to properly parallelize the arrays I sum from j1=1,NPTS. Given that they are not dependent on each other but vary between i1,e, I think using PRIVATE() is required. as to avoid overwriting. Lastly, I am fixated on these loops as according to GProf, a good portion of my computational expense is in this area of code.
Is there a simple and quick way to multiply a column of a matrix with element of a vector. We can do this explicitly,
program test
integer :: x(3,3), y(3), z(3,3)
x = reshape([(i,i=1,9)],[3,3])
y = [1,2,3]
do i=1,3
z(:,i) = x(:,i) * y(i)
print *, z(:,i)
enddo
end program test
Is there a way to perform the do loop in one line. For example in Numpy python we can do this to do the job in one shot
z = np.einsum('ij,i->ij',x,y)
#or
z = x*y[:,None]
Try
z = x * spread(y,1,3)
and if that doesn't work (no Fortran on this computer so I haven't checked) fiddle around with spread until it does. In practice you'll probably want to replace the 3 by size(x,1) or suchlike.
I expect that this will cause the compiler to create temporary arrays. And I expect it will be easy to find situations where it underperforms the explicit looping scheme in the question. 'neat' one-liners often have a cost in both time and space. And often tried-and-trusted Fortran approach of explicit looping is the one to go with.
Why replace clear easy to read code with garbage?
program test
implicit none
integer i,j
integer :: x(3,3), y(3), z(3,3)
x = reshape([(i,i=1,9)],[3,3])
y = [1,2,3]
z = reshape ([((x(j,i)*y(i) ,j=1,3),i=1,3)], [3,3])
print *, z(1,:)
print *, z(2,:)
print *, z(3,:)
end program test
The fold manual gives an example:
input price = close;
input length = 9;
plot SMA = (fold n = 0 to length with s do s + getValue(price, n, length - 1)) / lenth;
This effectively calls a function iteratively like in a for loop body.
When I use this statement to call my own function as follows, then it breaks because the loop index variable is not recognized as a variable that can be passed to my function:
script getItem{
input index = 0;
plot output = index * index;
}
script test{
def total = fold index = 0 to 10 with accumulator = 0 do
accumulator + getItem(index);########## Error: No such variable: index
}
It is a known bug / limitation. Has been acknowledged without a time line for a fix. No workaround available.
Have you tried adding a small remainder to your defined variable within the fold and then pass that variable? You can strip the integer value and then use the remainder as your counter value. I've been playing around with somethin similar but it isn't working (yet). Here's an example:
script TailOverlap{
input i = 0;
def ii = (Round(i, 1) - i) * 1000;
... more stuff
plot result = result;
};
def _S = (
fold i = displace to period
with c = 0
do if
TailOverlap(i = _S) #send cur val of _S to script
then _S[1] + 1.0001 #increment variable and counter
else _S[1] + 0.0001 #increment the counter only
);
I'm going to continue playing around with this. If I get it to work I'll post the final solution. If you're able to get work this (or have discovered another solution) please do post it here so I know.
Thanks!
I'm very new to OCaml / functional programming, and I'm confused about the implementation of some things that are relatively simple other languages I know. I could use any and all help.
Chiefly: in a program I'm working on, I either increment or decrement a variable based on a certain parameter. Here's something representative of what I have:
let tot = ref 0 in
for i = 0 to s do
if test_num > 0 then
tot := !tot + other_num
else
tot := !tot - other_num
done;;
This is obviously not the way to go about it, because even if the else statement is never taken, the code acts as if it is, each and every time, presumably because it's closer to the bottom of the program? I know OCaml has pretty sophisticated pattern matching, but within this level of coed I need access to a handful of lists I've already created, and, as far as I understand, I can't access those lists from a top-level function without passing them all as parameters.
I know I'm going about this the wrong way, but I have no idea how to do this idiomatically.
Suggestions? Thanks.
edit
Here's a more concise example:
let ex_list = [1; -2; 3; -4] in
let max_mem = ref 0 in
let mem = ref 0 in
let () =
for i = 0 to 3 do
let transition = List.nth ex_list i in
if transition > 0 then (
mem := (!mem + 10);
) else
mem := (!mem - 1);
if (!mem > !max_mem) then (max_mem := !mem);
done;
print_int !max_mem; print_string "\n";
in !mem;
At the end, when I print max_mem, I get 19, though this value should be (0 + 10 - 1 + 10 - 1 = 18). Am I doing the math wrong, or does the problem come from somewhere else?
Your code looks fine to me. It doesn't make a lot of sense as actual code, but I think you're just trying to show a general layout. It's also written in imperative style, which I usually try to avoid if possible.
The if in OCaml acts just like it does in other languages, there's no special thing about being near the bottom of the program. (More precisely, it acts like the ? : ternary operator from C and related languages; i.e., it's an expression.)
Your code doesn't return a useful value; it always returns () (the quintessentially uninteresting value known as "unit").
If we replace your free variables (ones not defined in this bit of code) by constants, and change the code to return a value, we can run it:
# let s = 8 in
let other_num = 7 in
let test_num = 3 in
let tot = ref 0 in
let () =
for i = 0 to s do
if test_num > 0 then
tot := !tot + other_num
else
tot := !tot - other_num
done
in
!tot;;
- : int = 63
#
If you're trying to learn to write in a functional style (i.e., without mutable variables), you would write this loop as a recursive function and make tot a parameter:
# let s = 8 in
let other_num = 7 in
let test_num = 3 in
let rec loop n tot =
if n > s then
tot
else
let tot' =
if test_num > 0 then tot + other_num else tot - other_num
in
loop (n + 1) tot'
in
loop 0 0;;
- : int = 63
It would probably be easier to help if you gave a (edited to add: small :-) self-contained problem that you're trying to solve.
The other parts of your question aren't clear enough to give any advice on. One thing that I might point out is that it's completely idiomatic to use pattern matching when processing lists.
Also, there's nothing wrong with passing things as parameters. That's why the language is called "functional" -- your code consists of functions, which have parameters.
Update
I like to write let () = expr1 in expr2 instead of expr1; expr2. It's just a habit I got into, sorry if it's confusing. The essence is that you're evaluating the first expression just for its side effects (it has type unit), and then returning the value of the second expression.
If you don't have something after the for, the code will evaluate to (), as I said. Since the purpose of the code seems to be to compute the value of !tot, this is what I returned. At the very least, this lets you see the calculated value in the OCaml top level.
tot' is just another variable. If you calculate a new value straightforwardly from a variable named var, it's conventional to name the new value var'. It reads as "var prime".
Update 2
Your example code works OK, but it has the problem that it uses List.nth to traverse a list, which is a slow (quadratic) operation. In fact your code is naturally considered a fold. Here's how you might write it in a functional style:
# let ex_list = [1; -2; 3; -4] in
let process (tot, maxtot) transition =
let tot' = if transition > 0 then tot + 10 else tot - 1 in
(tot', max maxtot tot')
in
List.fold_left process (0, 0) ex_list;;
- : int * int = (18, 19)
#
In addition to Jeffrey's answer, let me second that this is not how you would usually write such code in Ocaml, since it is a very low-level imperative approach. A more functional version would look like this:
let high ex_list =
let deltas = List.map (fun x -> if x > 0 then 10 else -1) ex_list in
snd (List.fold_left (fun (n, hi) d -> (n+d, max (n+d) hi)) (0, 0) deltas)
let test = high [1; -2; 3; -4]
Hey, I wrote this (fortran) with the aim of finding the minimum spanning tree of a bunch of points (syscount of them). I know for a fact that this approach works, since i wrote it in javascript earlier today. js is slow though, and i wanted to see how much faster fortran would be!!
only problem is it's not working, i'm getting an annoying error;
prims.f95:72.43:
if((check == 1) .and. (path(nodesin(j))(k) < minpath)) then
1
Error: Expected a right parenthesis in expression at (1)
What the hell is that about?! the 43rd character on the line is the "h" of "path"
nodesin(1) = 1
do i = 1,syscount-1
pathstart = -1
pathend = -1
minpath = 2000
do j = 1,i
do k = 1, syscount
check = 1
do l = 1, i
if(nodesin(l) == k) then
check = 0
end if
end do
if((check == 1) .and. (path(nodesin(j))(k) < minpath)) then
minpath = path(nodesin(j))(k)
pathstart = nodesin(j)
pathend = k
end if
end do
end do
nodesin(i+1) = pathend
minpaths(i)(1) = pathstart
minpaths(i)(2) = pathend
end do
Also, i'm fairly new to fortran, so i have a few other questions;
can i use && instead of .and. ?
is there a versions of the for(object in list){} loop found in many other languages?
is there a verion of the php function in_array ? i.e. bool in_array(needle,haystack), and if there is, is there a better way of doing it than:
check = false
Asize = size(array)
do i = 1, Asize
if(array(i) == needle) then
check = true
end if
end do
then to using the check variable to see if it's there?
(I haven't posted anything on stackoverflow before. please don't get angry if i've broken loads of etiquette things!)
It looks like you have defined path and minpaths as two-dimensional arrays. Multi-dimensional arrays are accessed differently in Fortran when compared to C-like languages. In Fortran you separate the indices by commas within one set of parentheses.
I'm guessing by the use of these variables they are integer arrays. Here is how you access elements of those arrays (since you didn't share your variable declarations I am making up the shape of these arrays):
integer :: path(n1, n2)
integer :: minpaths(n3, 2)
your if statement should be:
if((check == 1) .and. (path(nodesin(j), k) < minpath)) then
your access to minpaths should be:
minpaths(i, 1) = pathstart
minpaths(i, 2) = pathend
Also, if you are not using IMPLICIT NONE I recommend you consider it. Not using it is dangerous, and you are using variable names that are close to each other (minpath and minpaths). You could save hours of hair pulling debugging by using IMPLICIT NONE.
While .EQ. can be replaced with ==, there is still only .AND.
For your code block to check whether a "variable is there", you can use "where" and have much shorter code!
In Fortran >= 90 statements and functions can operate on arrays so that explicit loops don't have to be used as frequently.
There is no for (object in list), but using the where statement can do something very similar.
Many of the intrinsic functions that act on arrays also take masks as optional arguments to selectively operate.
I suggest reading a book to learn about these features. I like the one by Metcalf, Reid and Cohen. In the meantime, the second Wikipedia article may help: http://en.wikipedia.org/wiki/Fortran_95_language_features