I'm new to Go and have a problem understanding the concurrency and channel.
package main
import "fmt"
func display(msg string, c chan bool){
fmt.Println("display first message:", msg)
c <- true
}
func sum(c chan bool){
sum := 0
for i:=0; i < 10000000000; i++ {
sum++
}
fmt.Println(sum)
c <- true
}
func main(){
c := make(chan bool)
go display("hello", c)
go sum(c)
<-c
}
The output of the program is:
display first message: hello
10000000000
But I thought it should be only one line:
display first message: hello
So in the main function, <-c is blocking it and waits for the other two go rountines to send data to the channel. Once the main function receives the data from c, it should proceed and exit.
display and sum run simultaneously and sum takes longer so display should send true to c and the program should exit before sum finishes...
I'm not sure I understand it clearly. Could someone help me with this? Thank you!
The exact output of your program is not defined and depends on the scheduler. The scheduler can choose freely between all goroutines that are currently not blocked. It tries to run those goroutines concurrently by switching the current goroutine in very short time intervals so that the user gets the feeling that everything happens simultanously. In addition to that, it can also execute more than one goroutine in parallel on different CPUs (if you happen to have a multicore system and increase runtime.GOMAXPROCS). One situation that might lead to your output is:
main creates two goroutines
the scheduler chooses to switch to one of the new goroutines immediately and chooses display
display prints out the message and is blocked by the channel send (c <- true) since there isn't a receiver yet.
the scheduler chooses to run sum next
the sum is computed and printed on the screen
the scheduler chooses to not resume the sum goroutine (it has already used a fair amount of time) and continues with display
display sends the value to the channel
the scheduler chooses to run main next
main quits and all goroutines are destroyed
But that is just one possible execution order. There are many others and some of them will lead to a different output. If you want to print just the first result and quit the program afterwards, you should probably use a result chan string and change your main function to print fmt.Println(<-result).
Related
I have a somewhat simple multithreaded application written using the C++ std::thread library for both Ubuntu 14.04 and Windows 8.1. The code is nearly completely identical except that I'm using the operating system respective libraries windows.h and unistd.h to use Sleep/sleep to pause execution for a time. They both actually begin to run and the Ubuntu version does keep running for a short time but then hangs. I am using the proper arguments to the sleep/Sleep functions since I know Windows Sleep takes milliseconds, while Unix sleep takes seconds.
I've run the code multiple times and on Ubuntu it never makes it past two minutes whereas I've run it on windows twice for 20 minutes and then multiple times for roughly five minutes each to see if I was just lucky. Is this just an incompatibility with the thread library or does sleep not do what I think it does, or something else? The infinite loops are there because this is a school project and is expected to run without deadlocks or crashing.
The gist is that this is a modified 4-way stop where cars who arrive first don't have to slow down and stop. We only had to let one car through the intersection at a time which takes 3 seconds to cross, hence Sleep(3000), and don't have to worry about turns. Three threads run the spawnCars function and there are four other threads that each monitor one of the four directions N, E, S, and W. I hope that it's understandable why I can't post the entire code in the chance some other student stumbles upon this. These two functions are the only place where code is different aside from the operating system dependent library inclusion at the top. Thanks.
edit: Since I've just gone and posted all the code for the project, if the problem does end up being a deadlock, may I request that you only say so, and not post an in depth solution? I'm new here so if that's against the spirit of SO then fire away and I'll try to figure it out without reading the details.
/* function clearIntersection
Makes a car go through the intersection. The sleep comes before the removal from the queue
because my understanding is that the wait condition simulates the go signal for drivers.
It wouldn't make sense for the sensors to tell a car to go if the intersection isn't yet
clear even if the lock here would prevent that.
*/
void clearIntersection(int direction)
{
lock->lock();
Sleep(3000);
dequeue(direction);
lock->unlock();
}
/* function atFront(int direction)
Checks whether the car waiting at the intersection from a particular direction
has permission to pass, meaning it is at the front of the list of ALL waiting cars.
This is the waiting condition.
*/
bool isAtFront(int direction)
{
lock->lock();
bool isAtFront = cardinalDirections[direction].front() == list->front();
lock->unlock();
return isAtFront;
}
void waitInLine()
{
unique_lock<mutex> conditionLock(*lock);
waitForTurn->wait(conditionLock);
conditionLock.unlock();
}
//function broadcast(): Let all waiting threads know they can check whether or not their car can go.
void broadcast()
{
waitForTurn->notify_all();
}
};
/* function monitorDirection(intersectionQueue,int,int)
Threads will run this function. There are four threads that run this function
in total, one for each of the cardinal directions. The threads check to see
if the car at the front of the intersectionQueue, which contains the arrival order
of cars regardless of direction, is the car at the front of the queue for the
direction the thread is assigned to monitor. If not, it waits on a condition
variable until it is the case. It then calls the function to clear the intersection.
Broadcast is then used on the condition variable so all drivers will check if they
are allowed to pass, which one will unless there are 0 waiting cars, waiting again if not the case.
*/
void monitorDirection(intersectionQueue *intersection, int direction, int id)
{
while (true) //Do forever to see if crashes can occur.
{
//Do nothing if there are no cars coming from this direction.
//Possibly add more condition_variables for each direction?
if (!intersection->empty(direction))
{
while (!intersection->isAtFront(direction))
intersection->waitInLine();
intersection->clearIntersection(direction);
cout << "A car has gone " << numberToDirection(direction) << endl;
//All cars at the intersection will check the signal to see if it's time to go so broadcast is used.
intersection->broadcast();
}
}
}
Your culprit is likely your while (!isAtFront(...)) loop. If another thread gets scheduled between the check and the subsequent call to waitInLine(), the state of your queues could change, causing all of your consumer threads to end up waiting. At that point there's no thread to signal your condition_variable, so they will wait forever.
I asked this in a previous question, but some people felt that my original question was not detailed enough ("Why would you ever want a timed condition wait??") so here is a more specific one.
I have a goroutine running, call it server. It is already started, will execute for some amount of time, and do its thing. Then, it will exit since it is done.
During its execution some large number of other goroutines start. Call them "client" threads if you like. They run step A, and step B. Then, they must wait for the "server" goroutine to finish for a specified amount of time, and exit with status if "server is not finished, and say run step C if it finishes.
(Please do not tell me how to restructure this workflow. It is hypothetical and a given. It cannot be changed.)
A normal, sane way to do this is to have the server thread signal a condition variable with a selectAll or Broadcast function, and have the other threads in a timed wait state monitoring the condition variable.
func (s *Server) Join(timeMillis int) error {
s.mux.Lock()
defer s.mux.Unlock()
while !s.isFinished {
err = s.cond.Wait(timeMillis)
if err != nil {
stepC()
}
}
return err
}
Where the server will enter a state where isFinished becomes true and broadcast signal the condition variable atomically with respect to the mutex. Except this is impoosible, since Go does not support timed condition waits. (But there is a Broadcast())
So, what is the "Go-centric" way to do this? I've reall all of the Go blogs and documentation, and this pattern or its equivalent, despite its obviousness, never comes up, nor any equivalent "reframing" of the basic problem - which is that IPC style channels are between one routine and one other routine. Yes, there is fan-in/fan-out, but remember these threads are constantly appearing and vanishing. This should be simple - and crucially /not leave thousands of "wait-state" goroutines hanging around waiting for the server to die when the other "branch" of the mux channel (the timer) has signalled/.
Note that some of the "client" above might be started before the server goroutine has started (this is when channel is usually created), some might appear during, and some might appear after... in all cases they should run stepC if and only if the server has run and exited after timeMillis milliseconds post entering the Join() function...
In general the channels facility seems sorely lacking when there's more than one consumer. "First build a registry of channels to which listeners are mapped" and "there's this really nifty recursive data structure which sends itself over a channel it holds as field" are so.not.ok as replacements to the nice, reliable, friendly, obvious: wait(forSomeTime)
I think what you want can be done by selecting on a single shared channel, and then having the server close it when it's done.
Say we create a global "Exit channel", that's shared across all goroutines. It can be created before the "server" goroutine is created. The important part is that the server goroutine never sends anything down the channel, but simply closes it.
Now the client goroutines, simply do this:
select {
case <- ch:
fmt.Println("Channel closed, server is done!")
case <-time.After(time.Second):
fmt.Println("Timed out. do recovery stuff")
}
and the server goroutine just does:
close(ch)
More complete example:
package main
import(
"fmt"
"time"
)
func waiter(ch chan struct{}) {
fmt.Println("Doing stuff")
fmt.Println("Waiting...")
select {
case <- ch:
fmt.Println("Channel closed")
case <-time.After(time.Second):
fmt.Println("Timed out. do recovery stuff")
}
}
func main(){
ch := make(chan struct{})
go waiter(ch)
go waiter(ch)
time.Sleep(100*time.Millisecond)
fmt.Println("Closing channel")
close(ch)
time.Sleep(time.Second)
}
This can be abstracted as the following utility API:
type TimedCondition struct {
ch chan struct{}
}
func NewTimedCondition()*TimedCondition {
return &TimedCondition {
ch: make(chan struct{}),
}
}
func (c *TimedCondition)Broadcast() {
close(c.ch)
}
func (c *TimedCondition)Wait(t time.Duration) error {
select {
// channel closed, meaning broadcast was called
case <- c.ch:
return nil
case <-time.After(t):
return errors.New("Time out")
}
}
All I want to do is to send one value to the channel and return from the main() function and exit the program right after I receive one value from the channel ch.
But this keeps running forever:
ch := make(chan int)
for {
go func() {
ch <- 1
}()
}
<-ch
How do I return with this for loop after I receive one value from the channel?
And why does this run forever?
Go playground link is here
http://play.golang.org/p/EvmT6Pw96Y
Since the for loop runs forever, your code runs forever. Receiving from ch after the for loop is never reached.
If you want your code to exit sometime, you have to end the for loop sometime, e.g. make it run only a certain number of iterations:
for i := 0; i < 5; i++ {
...
}
So if you only run 5 iterations (like in my example), after the for loop receiving from ch will be executed which may block (because ch is unbuffered) if other goroutines have not yet run, and at least one of the started goroutines will be executed which sends a value on the channel which may trigger the main goroutine to continue (because its blocking operating is no longer blocking).
Also note that the Go Memory Model only guarantees that certain events happen before other events, you have no guarantee how 2 concurrent goroutines are executed. You have no guarantee that the goroutines started in the for loop will start to execute before the main goroutine gets blocked at receiving from ch.
Usually when a goroutine gets blocked (e.g. blocking channel receive operation), the runtime may schedule another goroutine to run.
Everything works great apart from the mutex. After I lock and unlock, it won't do anything. Is there something obvious I'm missing?
On top of this, after unlocking, I want to run a function inside this function. I've tried just calling it as a regular function (timer()) and even (go timer()).
func shield(state *State){
for s := range state.ToggleShield { //run if data on channel
if s == true { //if data on channel is true
fmt.Println("Opening the shields This is uninteruptable. Please wait...")
state.VariableMutex.Lock()
state.Finished = false //disable other commands
state.VariableMutex.Unlock()
fmt.Println("Move!!")
ticker := time.Tick(time.Second)
for i := 10; i >= 0; i-- {
<-ticker
fmt.Printf("\rOn 10/%d", i)
}
}
}
}
The Go Programming Language Specification
Go statements
A "go" statement starts the execution of a function call as an
independent concurrent thread of control, or goroutine, within the
same address space.
The function value and parameters are evaluated as usual in the
calling goroutine, but unlike with a regular call, program execution
does not wait for the invoked function to complete. Instead, the
function begins executing independently in a new goroutine. When the
function terminates, its goroutine also terminates.
Your program does not appear to have proper mechanisms to wait until your goroutines complete: "program execution does not wait for the invoked function to complete." To demonstrate this, I inserted a crude wait mechanism at the end of your program main function:
// wait for a while to give goroutines a chance to complete
time.Sleep(5 * time.Second)
Program: https://play.golang.org/p/ODdEihip4m
Output:
Toggling Shield
Opening the shields This is uninteruptable. Please wait...
Move!!
On 10/10
On 10/9
On 10/8
On 10/7
On 10/6
Program exited.
How can other goroutines keep executing whilst invoking a syscall? (when using GOMAXPROCS=1)
As far as I'm aware of, when invoking a syscall the thread gives up control until the syscall returns.
How can Go achieve this concurrency without creating a system thread per blocking-on-syscall goroutine?
From the documentation:
Goroutines
They're called goroutines because the existing terms—threads,
coroutines, processes, and so on—convey inaccurate connotations. A
goroutine has a simple model: it is a function executing concurrently
with other goroutines in the same address space. It is lightweight,
costing little more than the allocation of stack space. And the stacks
start small, so they are cheap, and grow by allocating (and freeing)
heap storage as required.
Goroutines are multiplexed onto multiple OS threads so if one should
block, such as while waiting for I/O, others continue to run. Their
design hides many of the complexities of thread creation and
management.
If a goroutine is blocking, the runtime will start a new OS thread to handle the other goroutines until the blocking one stops blocking.
Reference : https://groups.google.com/forum/#!topic/golang-nuts/2IdA34yR8gQ
Ok, so here's what I've learned:
When you're doing raw syscalls, Go indeed creates a thread per blocking goroutine. For example, consider the following code:
package main
import (
"fmt"
"syscall"
)
func block(c chan bool) {
fmt.Println("block() enter")
buf := make([]byte, 1024)
_, _ = syscall.Read(0, buf) // block on doing an unbuffered read on STDIN
fmt.Println("block() exit")
c <- true // main() we're done
}
func main() {
c := make(chan bool)
for i := 0; i < 1000; i++ {
go block(c)
}
for i := 0; i < 1000; i++ {
_ = <-c
}
}
When running it, Ubuntu 12.04 reported 1004 threads for that process.
On the other hand, when utilizing Go's HTTP server and opening 1000 sockets to it, only 4 operating system threads were created:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
So, it's a mix between an IOLoop and a thread per blocking system call.
It can't. There's only 1 goroutine that can be running at a time when GOMAXPROCS=1, whether that one goroutine is doing a system call or something else.
However, most blocking system calls, such as socket I/O, waiting for a timer are not blocked on a system call when performed from Go. They're multiplexed by the Go runtime onto epoll, kqueue or similar facilities the OS provides for multiplexing I/O.
For other kinds of blocking system calls that cannot be multiplexed with something like epoll, Go does spawn a new OS thread, regardless of the GOMAXPROCS setting (albeit that was the state in Go 1.1, I'm not sure if the situation is changed)
You have to differentiate processor number and thread number: you can have more threads than physical processors, so a multi-thread process can still execute on a single core processor.
As the documentation you quoted explain, a goroutine isn't a thread: it's merely a function executed in a thread that is dedicated a chunk of stack space. If your process have more than one thread, this function can be executed by either thread. So a goroutine that is blocking for a reason or another (syscall, I/O, synchronization) can be let in its thread while other routines can be executed by another.
I’ve written an article explaining how they work with examples and diagrams. Please feel free to take a look at it here: https://osmh.dev/posts/goroutines-under-the-hood
From documentation of runtime:
The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit.
so when one OS thread is blocked for syscall, another thread can be started - and GOMAXPROCS = 1 is still satisfied. The two threads are NOT running simultaneously.
Hope this example of sum numbers helps you.
package main
import "fmt"
func put(number chan<- int, count int) {
i := 0
for ; i <= (5 * count); i++ {
number <- i
}
number <- -1
}
func subs(number chan<- int) {
i := 10
for ; i <= 19; i++ {
number <- i
}
}
func main() {
channel1 := make(chan int)
channel2 := make(chan int)
done := 0
sum := 0
//go subs(channel2)
go put(channel1, 1)
go put(channel1, 2)
go put(channel1, 3)
go put(channel1, 4)
go put(channel1, 5)
for done != 5 {
select {
case elem := <-channel1:
if elem < 0 {
done++
} else {
sum += elem
fmt.Println(sum)
}
case sub := <-channel2:
sum -= sub
fmt.Printf("atimta : %d\n", sub)
fmt.Println(sum)
}
}
close(channel1)
close(channel2)
}