Remote variable declarations - chapel

Question

How do remote variable declarations work? I've tried augmenting an ordinary variable declaration with the on clause as described in section 26.2.1 of the Chapel language specification, but it doesn't seem to work. For example, this line of code:
on Locales[1] var x: [0..10] real;
fails to compile, with the error syntax error: near 'var'.

Answer 1

In short, the syntax is specified but it isn't currently implemented. Unfortunately the language spec doesn't currently point out that it's a future feature.
Thanks for pointing out the issue. This one is arguably better as a GitHub issue against the Chapel project, so I've created an issue to track the problem.
The typical workaround is to choose one of:
Use nested on statements to achieve the desired effect
Allocate a class instance in an on statement
Use a distributed array
Here I will describe each.
First, we need a slightly longer example. Suppose you were trying to write:
on Locales[1] var A: [0..10] real; // declare array stored on Locales[1]
A = 1; // on Locale 0, set every element of A to 1
writeln(A); // on Locale 0, print out the array
// print out the locale storing each element
for x in A {
write(x.locale.id, " ");
}
writeln();
An equivalent way to write that using nested on statements is this:
on Locales[1] {
var A: [0..10] real;
on Locales[0] {
A = 1;
writeln(A);
for x in A {
write(x.locale.id, " ");
}
writeln();
}
}
// result, when run on 2 locales:
// from printing array elements:
// 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
// from printing element locales:
// 1 1 1 1 1 1 1 1 1 1 1
Note that in the example, we know the assignment is to happen on Locale 0. If we didn't know what Locale we were running on, we could save it in to a variable before the first on (e.g. var fromLocale = here;) and use that variable in the second on.
In some cases, it might be more convenient to use the on statement to specify where a variable is initialized without changing where it is declared. Right now, this can be done with class instances. Note that these are not garbage collected - you'll need to either use Owned/Shared or ensure delete is called.
In the spirit of a simpler answer to the question, I'll show a version that calls delete.
class MyArrayWrapper {
var A: [0..10] real;
}
var myObject: MyArrayWrapper; // starts out as nil
on Locales[1] {
// set myObject to a new instance
// since we do that on Locales[1], it is allocated there
// and the contained array is stored there too.
myObject = new MyArrayWrapper();
}
myObject.A = 1;
writeln(myObject.A);
for x in myObject.A {
write(x.locale.id, " ");
}
writeln();
delete myObject;
// result, when run on 2 locales:
// from printing array elements:
// 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
// from printing element locales:
// 1 1 1 1 1 1 1 1 1 1 1
Now, how could we achieve the same using a distributed array?
First, consider this example, which distributes the 11 elements among whatever Locales the Chapel program is run with.
use BlockDist;
const MyDom = {0..10}; // this domain represents the index set 0..10
// declare a Block-distributed index set 0..10
// by default, this is distributed over all available Locales
const MyBlockDistributedDomain = MyDom dmapped Block(boundingBox=MyDom);
// declare an Block-distributed array
var BlockDistributedA: [MyBlockDistributedDomain] real;
BlockDistributedA = 1;
writeln(BlockDistributedA);
for x in BlockDistributedA {
write(x.locale.id, " ");
}
writeln();
// result, when run on 2 locales:
// from printing array elements:
// 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
// from printing element locales:
// 0 0 0 0 0 0 1 1 1 1 1
That distributed the array over the available locales, but that behavior is just a default for the Block distribution. We can specify the locales to use with an argument to the Block constructor as the following example shows:
use BlockDist;
const MyDom = {0..10};
// This time, specify the target locales for the Block distribution to use.
// Here we pass in an anonymous array storing just Locales[1], so that
// the resulting array only stores elements on Locales[1].
const MyDistributedDomain = MyDom dmapped Block(boundingBox=MyDom,
targetLocales=[Locales[1]]);
var DistributedA: [MyDistributedDomain] real;
DistributedA = 1;
writeln(DistributedA);
for x in DistributedA {
write(x.locale.id, " ");
}
writeln();
// result, when run on 2 locales:
// from printing array elements:
// 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
// from printing element locales:
// 1 1 1 1 1 1 1 1 1 1 1