I am new to Golang. Right now I am trying to figure out how to make an any-to-one channel in Golang, where the setup is as follows:
say I have two goroutines numgen1 and numgen2 executing concurrently and writing numbers to channels num1 resp. num2. I would like to add the numbers sent from numgen1 and numgen2 in a new process, addnum. I have tried something like this:
func addnum(num1, num2, sum chan int) {
done := make(chan bool)
go func() {
n1 := <- num1
done <- true
}()
n2 := <- num2
<- done
sum <- n1 + n2
}
but this seems sadly incorrect. Could someone please give me some ideas?
Thank you very much for your help.
Depending on your requirements, you may need to read both of the channels for every iteration (i.e. a sort-of 'zip' function). You can do this with a select, similarly to user860302's answer:
func main() {
c1 := make(chan int)
c2 := make(chan int)
out := make(chan int)
go func(in1, in2 <-chan int, out chan<- int) {
for {
sum := 0
select {
case sum = <-in1:
sum += <-in2
case sum = <-in2:
sum += <-in1
}
out <- sum
}
}(c1, c2, out)
}
This runs forever. My preferred way to terminate goroutines like this one is to close the input channels. In this case you would need to wait for both to close, then close(out) before terminating.
Tip: note the use of directional channels as goroutine formal parameters. The compiler catches more mistakes when you write it this way. Happiness!
The simplest answer would be
func addnum(num1, num2, sum chan int) {
n1 := <- num1
n2 := <- num2
sum <- n1 + n2
}
Since you need both num1 and num2 to do the calculation, it makes no sense to do it otherwise. After all, there are two possible execution orders:
num1 generates a number, followed by num2
num2 generates a number, followed by num1
In the first case, our channel reads correspond exactly to the execution order. In the second case, our first read will block until num1 has finally produced a number; the second read will complete near-instantaneous because the num2 channel already has a number.
If you want to know more about channels in Go, I'd suggest to have a look at http://godoc.org/github.com/thomas11/csp -- this is a collection of Hoare's CSP examples written in Go.
To answer the question "Reading from multiple channels simultaneously"
There is a way to listen to multiple channels simultaneously :
func main() {
c1 := make(chan string)
c2 := make(chan string)
...
go func() {
for {
select {
case msg1 := <- c1:
fmt.Println(msg1)
case msg2 := <- c2:
fmt.Println(msg2)
}
}
}()
In this example, I create a channel msg1 and msg2.
Then I create a go routine with an infinite loop. In this loop, I listen to msg1 AND msg2.
This system allow you to read to multiple channels simultaneously and to process the messages when the arrive.
In order to avoid leaks, I should probably add another channel to stop the goroutine.
Related
I was studying Go concurrency pattern.
One pattern I am not sure is： Daisy Chain https://talks.golang.org/2012/concurrency.slide#39
It's very hard for me to understand the control flow of the code.
Can someone explain to me ?
package main
import (
"fmt"
)
func f(left, right chan int) {
left <- 1 + <-right
}
func main() {
const n = 10000
leftmost := make(chan int)
right := leftmost //point B: what does these do ?
left := leftmost
for i := 0; i < n; i++ {
right = make(chan int)
go f(left, right)
left = right //point A
}
go func(c chan int) { c <- 1 }(right)
fmt.Println(<-leftmost)
}
Conclusion:
the flow of channel going from right to left. It is good practice to write
func f(left chan<- int, right <-chan int) rather than original function signature as above.
'chain reaction' does not start until c <- 1, when signal 1 is sent to right most channel,
reaction goes all the way to left most end. Print out 10001.
The reason is go channel block 'read' until received channel receive signal.
#Rick-777 shows how to use array like structure for easy understanding.
Since each go coroutine is just around 6k big. It's not a bad idea to make 10k channel.
I clean up some code around Point B, for channel initialization.
Here is the source code:
http://play.golang.org/p/1kFYPypr0l
VonC has already given a direct answer. Here are some further remarks.
A slightly tidied-up version is in the playground, the difference being that the channels passed as parameters have their direction specified explicitly, ie. <-chan and chan<-. It's good practice to do this because the compiler can catch more mistakes for you.
An alternative and equivalent program that has a daisy-chain of n goroutines can be written using an array of channels instead. This allocates the same total number of channels using fewer lines of code. See playground:
package main
import (
"fmt"
)
func f(left chan<- int, right <-chan int) {
left <- 1 + <-right
}
func main() {
const n = 10000
// first we construct an array of n+1 channels each being a 'chan int'
var channels [n+1]chan int
for i := range channels {
channels[i] = make(chan int)
}
// now we wire n goroutines in a chain
for i := 0; i < n; i++ {
go f(channels[i], channels[i+1])
}
// insert a value into the right-hand end
go func(c chan<- int) { c <- 1 }(channels[n])
// pick up the value emerging from the left-hand end
fmt.Println(<-channels[0])
}
I hope you can see now how the original program is equivalent to this program. There is one minor difference: the original program does not create any channel array, so uses just a little less memory.
It illustrates you can generate a large number of goroutines.
Here, each go f(left, right) blocks: left <- 1 + <-right blocks because it waits for right to get a value. See "do golang channels maintain order".
All channels created here are unbuffered channels.
All 10000 goroutines are created.
Point B: right and left are declared, using the short variable declaration.
right is initialized to leftmost, but it doesn't matter, because it will be reassigned to a new channel in the for loop (right = make(chan int)).
Another way to declare right would have been:
var right chan int
left is initialized with leftmost, the very first channel created.
Point A: But once that channel start waiting (left <- 1 + <-right), the for loop set left to right, and created a new right: that is how the daisy chain is build
left <- (new) right (now left) <- (new) right (now left) <- ...
Then, one value is sent to the last right channel created: {c <- 1 }(right)
And you wait for the first leftmost channel created to receive its value (incremented 10000 time).
Since receivers always block until there is data to receive, the main() function itself doesn't exit before leftmost finally receive its value.
If main() exited too soon, the daisy chain wouldn't have time to complete.
I found dry-run this program could be really helpful to understand it.
At first, after executing
leftmost := make(chan int)
right := leftmost
left := leftmost
leftmost, left, and right are all referring to the same chan int
[chan int]
|
left, leftmost, right
Let's run some iterations for the for-loop.
i = 0
When we just enter the for loop,
[chan int]
|
left, leftmost, right
after executing right = make(chan int) and go f(left, right).
[chan int] <-(+1)- [chan int]
| |
left, leftmost right
after executing left = right
[chan int] <-(+1)- [chan int]
| |
leftmost left, right
i = 1
When we just enter the for loop,
[chan int] <-(+1)- [chan int]
| |
leftmost left, right
after executing right = make(chan int) and go f(left, right).
[chan int] <-(+1)- [chan int] <-(+1)- [chan int]
| | |
leftmost left right
after executing left = right
[chan int] <-(+1)- [chan int] <-(+1)- [chan int]
| |
leftmost left, right
I feel like two loops are enough to see the pattern:
Every loop we create a new chan int and append it at the end of the "linked list of chan int".
So after n = 100000 loops, we created 100000 new chan int, and the number of chan int in the "linked list of chan int will be 100001.
100001 chan int means 100000 gaps between each pair of adjacent chan int, and each gap means one +1.
Before the for loop, because all chan int are acting as receivers and there is no pass-in value, so all chan int will just wait.
After the for loop, we execute go func(c chan int) { c <- 1 }(right), then the 1 is passed into the "linked list of chan int" and perform +1 on the value for 100000 times, so the final result to the leftmost will be 100001.
Things will be like when we pass 1 into the "linked list of chan int":
[chan int] <-(+1)- [chan int] <-(+1)- ...... <-(+1)- [chan int] <- 1
| |
leftmost left, right
I created a leetcode playground holding all the code. You could try it here (https://leetcode.com/playground/gAa59fh3).
I was following the Go Tour and I am a bit stuck when it comes to goroutines. I understand that they are very lightweight and that every time a goroutine blocks, another one will start but I can't get my head around how this example actually works:
package main
import (
"fmt"
"time"
)
func say(s string) {
for i := 0; i < 5; i++ {
time.Sleep(1000 * time.Millisecond)
fmt.Println(s)
}
}
func main() {
go say("world")
say("hello")
}
Playground
I understand that a goroutine is started for the say function with the argument "world", but as far as I understand that should print "world" five times and "hello" once. However I don't understand why the output is as it is:
hello
world
hello
world
hello
world
hello
world
hello
From my limited understanding of threads from other languages the output should have been something like this:
hello
world
world
world
world
world
or like this:
world
world
world
hello
world
world
Why does the second line execute five times as well? Does anything below a go statement classify as part of the go routine?
Also the next slide shows something I can't get my head round again:
package main
import "fmt"
func sum(a []int, c chan int) {
sum := 0
for _, v := range a {
sum += v
}
c <- sum // send sum to c
}
func main() {
a := []int{7, 2, 8, -9, 4, 0}
c := make(chan int)
go sum(a[:len(a)/2], c)
go sum(a[len(a)/2:], c)
x, y := <-c, <-c // receive from c
fmt.Println(x, y, x+y)
}
Playground
A goroutine is started for the second half of the slice and then another one for the first part of the slice, however the values x and y have been assigned two different values. The way I see it the sum function will send it's sum to channel c and then the next sum will send it's sum to the same channel c so how can the two variables be assigned two different values? Shouldn't channel c have one single sum value in there?
I appreciate that this is quite a long question but I wasn't able to find the answer to these questions.
Why does the second line execute 5 times as well?
The second line will print hello every second 5 times in the main() thread.
But concurrently the first line go say("world") will also print world every seconds five times in a separate goroutine.
The Sleep ensure that each routine yields, allowing the other to resume.
Hence the output:
hello
world
hello
world
hello
world
hello
world
hello
The way I see it the sum function will send it's sum to channel c and then the next sum will send it's sum to the same channel c so how can the two variables be assigned two different values?
Because each send will block on c until channel c is read.
Since there are two write to c, you need to read:
x, y := <-c, <-c // receive from c twice.
The Assignement section of Golang Spec allows for a tuple assignment if:
the number of operands on the left must equal the number of expressions on the right, each of which must be single-valued, and the nth expression on the right is assigned to the nth operand on the left.
For the first function you should see values in the style VonC presented. The reason hello prints 5 times as well is because the function say prints things 5 times. Just imagine the program without the goroutine. I think it doesn't guarantee that you will get hello and world perfectly interspersed but I may be wrong.
The reason the channel works is:
Golang let's you do multiple assignment as VonC mentions
Channels empty out, i.e. when you assign c to x it removes the first sum that was passed into the channel and when it assigns c to y it passes in the second value (again I think the order is not a guarantee as x could have the first half and y the second half sum or vice-versa.
If you imagine channels as a sort of a queue I think it makes more sense. The summing goroutines push values onto the queue and assignments pop the values sequentially.
When executing the following code I get what I expect when the first loop is done (sequence from 0 to 9). But when the second loop finishes, the result is not what I expected (I expected the same result as in the first loop, but it prints only '10's):
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func(j int) {
defer wg.Done()
fmt.Println(j)
}(i)
}
wg.Wait()
fmt.Println("done first")
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println(i)
}()
}
wg.Wait()
fmt.Println("done second")
}
Output:
0
1
2
3
4
5
6
7
8
9
done first
10
10
10
10
10
10
10
10
10
10
done second
Why doesn't the second loop print a sequence?
Because the first one gets a copy of the loop counter each time. Whereas the second gets the variable captured as part of a closure.
In the first, you're passing it in here in every iteration of the loop:
go func(j int) {
defer wg.Done()
fmt.Println(j)
}(i) // <------------ its passed in here as part of each loop iteration
The second one receives nothing.. so, the loop counter i is captured as part of a closure. By the time the first go routine executes, the for loop has finished. The loop finishing has set the i variable (that is now part of a closure) to 10. Go routine #1 executes and prints the value of i.. which is now already 10 and the rest follow suit.
TLDR: The problem here is that the loop is finishing before any go routines are scheduled to be run - its just that quick. Therefore, i == 10 when the go routines run.
I want to create a producer/consumer with manager program in Go. For example: I have a 5 producers, 5 consumers and manager. Producers have their own local arrays, they iterate over them and send the elements to the manager. Consumers have their own local arrays with info that elements consume; they send them to the manager too. The Manager has it own array, where it stores what and how many elements there are (for example - if the producer sends 1 1 2 3 1 2 0 elements, the manager array looks like 1 3 2 1 (one 0, three 1, two 2 and one 3 ) and it handles producers' and consumers' requests - placing an element into the array (produce) or removing it (consume).
Is it possible to make a program like this in Go? I already did this in JAVA + CSP with channels to send info and guards in the manager to determine which procedure should be done first when producer and consumer try to process the same element (for example, a producer wants to add 1 to the manager array and at the same time a consumer wants to consume 1).
Any examples or advice are welcome, because I don't find any info about what I want to do. If needed I can give my JAVA+CSP code.
UPDATE. How about synchronization (not to take from empty array)? For example - if consumer wants to consume element from manager array that does not exist yet (for example consumer wants to consume '3', but manager don't have any of thems) but producer has this element and it will be produced after few iterations - how can I make consumers to check manager array again and again until producers work is finished? Should I need to create structs (or classes) for consumers elements and mark that they are used or not, or Go has specific methods to do this?
Here's a full example, including channel cleanup. After all the consumers and producers are finished, the Manager prints out the result (for which I've used a map rather than a slice, since I think it makes the code a tad easier).
package main
import "fmt"
// Consume processes the numbers in ns, sending them on ch after they're
// processed. When the routine is finished, it signals that on done.
func Consume(done chan bool, ch chan int, ns []int) {
for i := range ns {
ch <- i
}
done <- true
}
// Produce "creates" the numbers in ns, sending them on ch after they're
// produced. When the routine is finished, it signals that on done.
func Produce(done chan bool, ch chan int, ns []int) {
for i := range ns {
ch <- i
}
done <- true
}
// Manage creates consumers and producers for the given int slices.
// It returns once all consumers and producers are finished.
func Manage(cons, pros [][]int) {
cch := make(chan int)
pch := make(chan int)
dch := make(chan bool)
n := len(cons) + len(pros)
data := make(map[int]int)
for _, c := range cons {
go Consume(dch, cch, c)
}
for _, p := range pros {
go Produce(dch, pch, p)
}
for n > 0 {
select {
case c := <-cch:
data[c] -= 1
case c := <-pch:
data[c] += 1
case <-dch:
n -= 1
}
}
close(cch)
close(pch)
close(dch)
fmt.Println(data)
}
func main() {
cons := [][]int{{1, 3, 5}, {0, 1, 5}}
pros := [][]int{{0, 1, 1}, {3, 5, 5, 7}}
Manage(cons, pros)
}
I did a very similar example to what you're trying to do, check this gist github gist
The way I implemented that is using a single channel for my process consumer and another for my 2x processes that produce items, the for block controls the push from producers to the consumer and whenever the producers arent pushing anything the loop will default. The objects moved through the channels are slices, processes would produce and consume the headlines in each slice sent through the channel. I believe you can adjust this code to fit your example.
I need some help and also some insight. This is a program in Ada-2005 which has 3 tasks. The output is 'z'. If the 3 tasks do not happen in the order of their placement in the program then output can vary from z = 2, z = 1 to z = 0 ( That is easy to see in the program, mutual exclusion is attempted to make sure output is z = 2).
WITH Ada.Text_IO; USE Ada.Text_IO;
WITH Ada.Integer_Text_IO; USE Ada.Integer_Text_IO;
WITH System; USE System;
procedure xyz is
x : Integer := 0;
y : Integer := 0;
z : Integer := 0;
task task1 is
pragma Priority(System.Default_Priority + 3);
end task1;
task task2 is
pragma Priority(System.Default_Priority + 2);
end task2;
task task3 is
pragma Priority(System.Default_Priority + 1);
end task3;
task body task1 is
begin
x := x + 1;
end task1;
task body task2 is
begin
y := x + y;
end task2;
task body task3 is
begin
z := x + y + z;
end task3;
begin
Put(" z = ");
Put(z);
end xyz;
I first tried this program
(a) without pragmas, the result : In 100 tries, occurence of 2: 86, occurence of 1: 10, occurence of 0: 4.
Then
(b) with pragmas, the result : In 100 tries, occurence of 2: 84, occurence of 1 : 14, occurence of 0: 2.
Which is unexpected as the 2 results are nearly identical. Which means pragmas or no pragmas the output has same behavior.
Those who are Ada concurrency Gurus please shed some light on this topic. Alternative solutions with semaphores (if possible) is also invited.
Further in my opinion for a critical process (that is what we do with Ada), with pragmas the result should be z = 2, 100% at all times, hence or otherwise this program should be termed as 85% critical !!!! (That should not be so with Ada)
A protected object to do the three operations might look something like this. But note, all this does is make sure that the three variables x, y and z are consistent with the order that the updates occurred in; it says nothing about the order.
protected P is
procedure Update_X;
procedure Update_Y;
procedure Update_Z;
function Get_Z return Integer;
private
X : Integer := 0;
Y : Integer := 0;
Z : Integer := 0;
end P;
protected body P is
procedure Update_X is
begin
X := X + 1;
end Update_X;
procedure Update_Y is
begin
Y := Y + X;
end Update_Y;
procedure Update_Z is
begin
Z := X + Y + Z;
end Update_Z;
function Get_Z return Integer is
begin
return Z;
end Get_Z;
end P;
On the other hand, to make sure that the three tasks "submit their results" in the proper order, you could rewrite P so that a call to say Update_Y will block until Update_X has been called: Get_Z now has to be an entry with an out parameter rather than a function.
protected P is
entry Update_X;
entry Update_Y;
entry Update_Z;
entry Get_Z (Result : out Integer);
private
X_Updated : Boolean := False;
Y_Updated : Boolean := False;
Z_Updated : Boolean := False;
X : Integer := 0;
Y : Integer := 0;
Z : Integer := 0;
end P;
protected body P is
entry Update_X when True is
begin
X := X + 1;
X_Updated := True;
end Update_X;
entry Update_Y when X_Updated is
begin
Y := Y + X;
Y_Updated := True;
end Update_Y;
entry Update_Z when Y_Updated is
begin
Z := X + Y + Z;
Z_Updated := True;
end Update_Z;
entry Get_Z (Result : out Integer) when Z_Updated is
begin
Result := Z;
end Get_Z;
end P;
The three tasks can now have any priority you like. But the task that calls Update_Z will block until the other two have reported.
Well, those pragmas just prioritize the tasks on your system, they don't guarantee any kind of mutual-exclusion on those variables.
There may be some systems where that is enough. However, most Ada implementations these days map Ada tasks to OS threads, and most consumer PC's these days have multiple processors and can spilt their threads among them. There's nothing stopping the OS from scheduling the next lower-priority thread on your second processor while the highest priority thread is running.
This kind of behavior in a program is called a "race condition".
If you want mutual-exlusion on those variables, you need to implement that. Either give control of the variables to one task and use rendezvous to modify them from other tasks, or look into putting them into protected objects. I'd suggest the latter, as rendezvous can be much more difficult to get right. However, if you want to order the calls in a specific way, a master controller task calling rendezvous on the other tasks might be the way to go.
here is a program that is a sequential variant of the above ! .... with just one task (even that can be avoided if I use a single procedure, as suggested by Marc C and T.E.D )
WITH Ada.Text_IO; USE Ada.Text_IO;
WITH Ada.Integer_Text_IO; USE
Ada.Integer_Text_IO;
WITH System; USE System;
procedure xyzsimple is
x : Integer := 0;
y : Integer := 0;
z : Integer := 0;
task type xyz;
T: xyz;
task body xyz is
begin
x:= x + 1;
y:= x + y;
z:= x + y + z;
end xyz;
begin Put(" z = ");
Put(z);
end xyzsimple;
This program always gvies output z = 2, but then it doesn't serve to illustrate what I was trying to do. This program is deterministic, not into the paradigm of concurrency ! Further, this program will never exhibit 'RACE CONDITION' that T.E.D had mentioned.