I'm having something of a problem with goroutines. Why is it that this code executes in ~125ms (note sequential execution):
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
cmd.Run()
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
When this code takes about 20 seconds (concurrent execution using goroutines):
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
ch := make(chan bool)
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
go lessc(ch, cmd)
}
fmt.Println(n, " goroutines started.")
for i := 0; i < n; i++ {
_ = <-ch
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
func lessc(ch chan bool, c *exec.Cmd) {
c.Run()
ch <- true
}
Using go 1.0.3 on i7 720QM (4C/8T) 8GB RAM linux/x86-64
Also built and tested using 1.0.2 and got the same problem on the same machine.
Edit: Solved by #jnml below. If anyone cares about the new fixed concurrent code here it is:
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
ch := make(chan bool)
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
go lessc(ch)
}
fmt.Println(n, " goroutines started.")
for i := 0; i < n; i++ {
_ = <-ch
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
func lessc(ch chan bool) {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
cmd.Run()
ch <- true
}
IMO your program is not correct. It contains a race condition and thus can do literally anything. Any timing of it don't make sense.
You're creating one exec.Cmd and then concurrently (== data race) perform its Run method from several goroutines. exec.Cmd never mentions it could be reused for more than once Run - even if serially.
exec.Cmd has some state initialized by exec.Command and a different state after executing Run. IOW, after executing the Run method, the state is not anymore initialized and probably not fit for another Run
Related
I'm building the skeleton of a worker system using Go and I'm getting the "fatal error: all goroutines are asleep - deadlock!".
I am using two channels for coordination, one to create the jobs and the second to send the results. After creating the jobs I close the input channel.
My question is how I close the output channel so the program can exit correctly.
The code is:
package main
import (
"bufio"
"flag"
"fmt"
"log"
"math/rand"
"os"
"time"
)
type Work struct {
id int
ts time.Duration
}
const (
NumWorkers = 5000
NumJobs = 100000
)
func worker(in <-chan *Work, out chan<- *Work) {
for w := range in {
st := time.Now()
time.Sleep(time.Duration(rand.Int63n(int64(200 * time.Millisecond))))
w.ts = time.Since(st)
out <- w
}
}
func main() {
wait := flag.Bool("w", false, "wait for <enter> before starting")
flag.Parse()
if *wait {
fmt.Printf("I'm <%d>, press <enter> to continue", os.Getpid())
reader := bufio.NewReader(os.Stdin)
reader.ReadString('\n')
}
Run()
}
func Run() {
in, out := make(chan *Work, 100), make(chan *Work, 100)
for i := 0; i < NumWorkers; i++ {
go worker(in, out)
}
go createJobs(in)
receiveResults(out)
}
func createJobs(queue chan<- *Work) {
for i := 0; i < NumJobs; i++ {
work := &Work{i, 0}
queue <- work
}
close(queue)
}
func receiveResults(completed <-chan *Work) {
for w := range completed {
log.Printf("job %d completed in %s", w.id, w.ts)
}
}
Any help is appreciated :)
I missed the part about you knowing the cause of deadlock in original answer.
You mentioned WaitGroup, that is basically just a semaphore
You could use another "control" channel that workers sinal on when they are done
-
func worker(ctrl chan<- bool, in <-chan *Work, out chan<- *Work) {
for w := range in {
st := time.Now()
time.Sleep(time.Duration(rand.Int63n(int64(200 * time.Millisecond))))
w.ts = time.Since(st)
out <- w
}
ctrl <- true
}
func control(ctrl <-chan bool, numWorkers int, out chan<- *Work) {
for i=0; i<numWorkers; i++ {
<-ctrl
}
close(out)
}
original answer:
You do a range on completed:
for w := range completed {
log.Printf("job %d completed in %s", w.id, w.ts)
}
but that channel is never closed
I made a small program to benchmark go channel throughput, however it always deadlocks, I tried very hard but cannot understand why:
package main
import (
"fmt"
"runtime"
)
const CONCURRENCY = 32
const WORK_PER_WORKER = 100
const TOTAL_WORK = CONCURRENCY * WORK_PER_WORKER
func work() {
sum := 0
for i := 0; i < 10000000; i++ {
sum *= i
}
}
type WorkItem struct {
Done chan int
}
func main() {
runtime.GOMAXPROCS(CONCURRENCY)
var workQueue [CONCURRENCY]chan *WorkItem
// initialize workers
for i := 0; i < CONCURRENCY; i++ {
workQueue[i] = make(chan *WorkItem)
}
// start workers
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
anItem := <-workQueue[i]
work()
anItem.Done <- 1
}(i)
}
completed := make(chan bool, TOTAL_WORK)
for i := 0; i < TOTAL_WORK; i++ {
go func(i int) {
// send work to queues
workToDo := &WorkItem{Done: make(chan int)}
workQueue[i/WORK_PER_WORKER] <- workToDo // !! DEADLOCK
// wait until the work is done
<-workToDo.Done
completed <- true
}(i)
}
fmt.Println("Waiting")
for i := 0; i < TOTAL_WORK; i++ {
<-completed
}
}
Your code go func(i int) { anItem := <-workQueue[i]; ... } removes juste 1 item from workQueue[i] but you are trying to stuff WORK_PER_WORKER items into it. You will work on CONCURRENCY many items and after that all reading goroutines have terminated and you have your deadlock.
Looping in the worker goroutines "solves" your deadlock: http://play.golang.org/p/j2pavqnBDv
Just "solves" because these worker goroutines will never terminate. Maybe you can experiment with closeing your channels to notify the worker goroutines when nothing will be sent.
Because your worker only process one task and then exit. Thus, only first CONCURRENCY items proceed and then workQueue[i/WORK_PER_WORKER] <- workToDo blocks indifinitely. Thus, completed chan never receive enough values and main also blocks forever.
Your worker should do work in loops, like this:
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
for anItem := range workQueue[i] {
work()
anItem.Done <- 1
}
}(i)
}
func main() {
jobs := []Job{job1, job2, job3}
numOfJobs := len(jobs)
resultsChan := make(chan *Result, numOfJobs)
jobChan := make(chan *job, numOfJobs)
go consume(numOfJobs, jobChan, resultsChan)
for i := 0; i < numOfJobs; i++ {
jobChan <- jobs[i]
}
close(jobChan)
for i := 0; i < numOfJobs; i++ {
<-resultsChan
}
close(resultsChan)
}
func (b *Blockchain) consume(num int, jobChan chan *Job, resultsChan chan *Result) {
for i := 0; i < num; i++ {
go func() {
job := <-jobChan
resultsChan <- doJob(job)
}()
}
}
In the above example, jobs are pushed into the jobChan and goroutines will pull it off the jobChan and execute the jobs concurrently and push results into resultsChan. We will then pull results out of resultsChan.
Question 1:
In my code, there is no serialized/linearilized results. Although jobs go in the order of job1, job2, job3. The results might come out as job3, job1, job2, depending which one takes the longest.
I would still like to execute the jobs concurrently, however, I need to make sure that results come out of the resultsChan in the same order that it went in as jobs.
Question2:
I have approximately 300k jobs, this means the code will generate up to 300k goroutines. Is this efficient to have so many goroutines or would I be better off group the jobs together in a slice of 100 or so and have each goroutine go through 100 rather than 1.
Here's a way I've handled serialization (and also setting a limited number of workers). I set some worker objects with input and output fields and synchronization channels, then I go round-robin through them, picking up any work they've done and giving them a new job. Then I make one final pass through them to pick up any completed jobs that are left over. Note you might want the worker count to exceed your core count somewhat, so that you can keep all resources busy for a bit even when there's one unusually long job. Code is at http://play.golang.org/p/PM9y4ieMxw and below.
This is hairy (hairier than I remember it being before sitting down to write an example!)--would love to see what anyone else has, either just better implementations or a whole different way to accomplish your goal.
package main
import (
"fmt"
"math/rand"
"runtime"
"time"
)
type Worker struct {
in int
out int
inited bool
jobReady chan bool
done chan bool
}
func (w *Worker) work() {
time.Sleep(time.Duration(rand.Float32() * float32(time.Second)))
w.out = w.in + 1000
}
func (w *Worker) listen() {
for <-w.jobReady {
w.work()
w.done <- true
}
}
func doSerialJobs(in chan int, out chan int) {
concurrency := 23
workers := make([]Worker, concurrency)
i := 0
// feed in and get out items
for workItem := range in {
w := &workers[i%
concurrency]
if w.inited {
<-w.done
out <- w.out
} else {
w.jobReady = make(chan bool)
w.done = make(chan bool)
w.inited = true
go w.listen()
}
w.in = workItem
w.jobReady <- true
i++
}
// get out any job results left over after we ran out of input
for n := 0; n < concurrency; n++ {
w := &workers[i%concurrency]
if w.inited {
<-w.done
out <- w.out
}
close(w.jobReady)
i++
}
close(out)
}
func main() {
runtime.GOMAXPROCS(10)
in, out := make(chan int), make(chan int)
allFinished := make(chan bool)
go doSerialJobs(in, out)
go func() {
for result := range out {
fmt.Println(result)
}
allFinished <- true
}()
for i := 0; i < 100; i++ {
in <- i
}
close(in)
<-allFinished
}
Note that only in and out in this example carry actual data--all the other channels are just for synchronization.
I am taking a look at the Google Go language as I am building a realtime system, and I find the sharing of resources through channels a bit confusing. I'm trying for the sake of understanding, to let to different goroutines to increment and decrement a shared value the same number of times, ending up at 0. I do know my code is wrong, but I'm not really getting the hang of it. Anybody care to explain what's wrong here?
package main
import (
. "fmt"
. "runtime"
)
func increment(c chan int) {
for x := 0; x < 10; x++ {
a := <-c
a++
c <- a
}
}
func decrement(c chan int) {
for x := 0; x < 10; x++ {
a := <-c
a--
c <- a
}
}
func main() {
GOMAXPROCS(NumCPU())
c := make(chan int)
go increment(c)
go decrement(c)
Println(<-c)
}
I could use a mutex or a semaphore similar to what I would do using C or Python, although I want to take advantage of the channels in Go.
**UPDATE
Would adding a WaitGroup change the program flow? I added a WaitGroup, and it worked well. Although, I added the Done() function after the whole for loop, will then the whole increment run before decrement? I kind of want them to go 'in parallel' as far as it can, I know that only one routine can access I, but I want them to run independent of each other.
There are a few problems with your code:
Both goroutines try to read from the channel at the same time. This is a deadlock as there is nothing in the channel to read.
Println(<-c) reads one value from the channel, not a result. It might read a result if you waited for both goroutines to finish, but that requires adding a WaitGroup. a Waitgroup is like a semaphore allowing each goroutine to decrement a counter of pending goroutines, and allowing the caller to wait for them to finish some task.
Since sending blocks if there is no reader and reading is blocking if there's no sender, and you're a. waiting for both goroutines to finish first and b. doing one more read than writes (the Println read), you need a buffered channel, that has exactly one extra place in the buffer.
You need to push an initial value in the channel for the process to start.
I've changed your code a bit and this example now works (although notice that it's not realy increment->decrement->increment->.... but rathter increment->increment->...->decrement->decrement->.... until we're done.
package main
import (
. "fmt"
. "runtime"
"sync"
)
func increment(c chan int, wg *sync.WaitGroup) {
for x := 0; x < 10; x++ {
a := <-c
Println("increment read ", a)
a++
c <- a
}
Println("Incrment done!")
wg.Done()
}
func decrement(c chan int, wg *sync.WaitGroup) {
for x := 0; x < 10; x++ {
a := <-c
Println("Decrement read ", a)
a--
c <- a
}
Println("Dencrment done!")
wg.Done()
}
func main() {
GOMAXPROCS(NumCPU())
//we create a buffered channel with 1 extra space. This means
//you can send one extra value into it if there is no reader, allowing for the final result to be pushed to println
c := make(chan int, 1)
//we create a wait group so we can wait for both goroutines to finish before reading the result
wg := sync.WaitGroup{}
wg.Add(1) //mark one started
go increment(c, &wg)
wg.Add(1) //mark another one started. We can just do Add(2) BTW
go decrement(c, &wg)
//now we push the initial value to the channel, starting the dialog
c <- 0
//let's wait for them to finish...
wg.Wait()
//now we have the result in the channel's buffer
Println("Total: ", <-c )
}
Here is a complete example of the kind of shared state engine that I think you are talking about
Note use of WaitGroup as you suggested in your edit to synchronise the two channels.
PS don't use import . "fmt" it is considered to be bad practice.
package main
import (
"fmt"
"runtime"
"sync"
)
// Commands for the engine
const (
INC = iota
DEC
ANSWER
QUIT
)
// Engine which takes commands and acts on some shared state
func engine(c chan int, reply chan int) {
counter := 0
for {
switch <-c {
case INC:
counter++
case DEC:
counter--
case ANSWER:
reply <- counter
case QUIT:
reply <- counter
return
}
}
}
// Add n times then signal done via the waitgroup
func increment(n int, c chan int, wg *sync.WaitGroup) {
defer wg.Done()
for x := 0; x < n; x++ {
c <- INC
}
}
// Subtract n times then signal done
func decrement(n int, c chan int, wg *sync.WaitGroup) {
defer wg.Done()
for x := 0; x < n; x++ {
c <- DEC
}
}
func main() {
runtime.GOMAXPROCS(runtime.NumCPU())
// Start the engine
c := make(chan int)
reply := make(chan int)
go engine(c, reply)
// Do adding and subtracting and wait for them to finish
wg := new(sync.WaitGroup)
wg.Add(2)
go increment(101, c, wg)
go decrement(100, c, wg)
wg.Wait()
// Read the answer
c <- ANSWER
fmt.Printf("Total is %d\n", <-reply)
// Stop the engine
c <- QUIT
<-reply
fmt.Printf("All done\n")
}
What I would like to do is have a set of producer goroutines (of which some may or may not complete) and a consumer routine. The issue is with that caveat in parentheses - we don't know the total number that will return an answer.
So what I want to do is this:
package main
import (
"fmt"
"math/rand"
)
func producer(c chan int) {
// May or may not produce.
success := rand.Float32() > 0.5
if success {
c <- rand.Int()
}
}
func main() {
c := make(chan int, 10)
for i := 0; i < 10; i++ {
go producer(c, signal)
}
// If we include a close, then that's WRONG. Chan will be closed
// but a producer will try to write to it. Runtime error.
close(c)
// If we don't close, then that's WRONG. All goroutines will
// deadlock, since the range keyword will look for a close.
for num := range c {
fmt.Printf("Producer produced: %d\n", num)
}
fmt.Println("All done.")
}
So the issue is, if I close it's wrong, if I don't close - it's still wrong (see comments in code).
Now, the solution would be an out-of-band signal channel, that ALL producers write to:
package main
import (
"fmt"
"math/rand"
)
func producer(c chan int, signal chan bool) {
success := rand.Float32() > 0.5
if success {
c <- rand.Int()
}
signal <- true
}
func main() {
c := make(chan int, 10)
signal := make(chan bool, 10)
for i := 0; i < 10; i++ {
go producer(c, signal)
}
// This is basically a 'join'.
num_done := 0
for num_done < 10 {
<- signal
num_done++
}
close(c)
for num := range c {
fmt.Printf("Producer produced: %d\n", num)
}
fmt.Println("All done.")
}
And that totally does what I want! But to me it seems like a mouthful. My question is: Is there any idiom/trick that lets me do something similar in an easier way?
I had a look here: http://golang.org/doc/codewalk/sharemem/
And it seems like the complete chan (initialised at the start of main) is used in a range but never closed. I do not understand how.
If anyone has any insights, I would greatly appreciate it. Cheers!
Edit: fls0815 has the answer, and has also answered the question of how the close-less channel range works.
My code above modifed to work (done before fls0815 kindly supplied code):
package main
import (
"fmt"
"math/rand"
"sync"
)
var wg_prod sync.WaitGroup
var wg_cons sync.WaitGroup
func producer(c chan int) {
success := rand.Float32() > 0.5
if success {
c <- rand.Int()
}
wg_prod.Done()
}
func main() {
c := make(chan int, 10)
wg_prod.Add(10)
for i := 0; i < 10; i++ {
go producer(c)
}
wg_cons.Add(1)
go func() {
for num := range c {
fmt.Printf("Producer produced: %d\n", num)
}
wg_cons.Done()
} ()
wg_prod.Wait()
close(c)
wg_cons.Wait()
fmt.Println("All done.")
}
Only producers should close channels. You could achieve your goal by invoking consumer(s) which iterates (range) over the resulting channel once your producers were started. In your main thread you wait (see sync.WaitGroup) until your consumers/producers finished their work. After producers finished you close the resulting channel which will force your consumers to exit (range will exit when channels are closed and no buffered item is left).
Example code:
package main
import (
"log"
"sync"
"time"
"math/rand"
"runtime"
)
func consumer() {
defer consumer_wg.Done()
for item := range resultingChannel {
log.Println("Consumed:", item)
}
}
func producer() {
defer producer_wg.Done()
success := rand.Float32() > 0.5
if success {
resultingChannel <- rand.Int()
}
}
var resultingChannel = make(chan int)
var producer_wg sync.WaitGroup
var consumer_wg sync.WaitGroup
func main() {
rand.Seed(time.Now().Unix())
for c := 0; c < runtime.NumCPU(); c++ {
producer_wg.Add(1)
go producer()
}
for c := 0; c < runtime.NumCPU(); c++ {
consumer_wg.Add(1)
go consumer()
}
producer_wg.Wait()
close(resultingChannel)
consumer_wg.Wait()
}
The reason I put the close-statement into the main function is because we have more than one producer. Closing the channel in one producer in the example above would lead to the problem you already ran into (writing on closed channels; the reason is that there could one producer left who still produces data). Channels should only be closed when there is no producer left (therefore my suggestion on closing the channel only by the producer). This is how channels are constructed in Go. Here you'll find some more information on closing channels.
Related to the sharemem example: AFAICS this example runs endless by re-queuing the Resources again and again (from pending -> complete -> pending -> complete... and so on). This is what the iteration at the end of the main-func does. It receives the completed Resources and re-queues them using Resource.Sleep() to pending. When there is no completed Resource it waits and blocks for new Resources being completed. Therefore there is no need to close the channels because they are in use all the time.
There are always lots of ways to solve these problems. Here's a solution using the simple synchronous channels that are fundamental in Go. No buffered channels, no closing channels, no WaitGroups.
It's really not that far from your "mouthful" solution, and--sorry to disappoint--not that much smaller. It does put the consumer in it's own goroutine, so that the consumer can consume numbers as the producer produces them. It also makes the distinction that a production "try" can end in either success or failure. If production fails, the try is done immediately. If it succeeds, the try is not done until the number is consumed.
package main
import (
"fmt"
"math/rand"
)
func producer(c chan int, fail chan bool) {
if success := rand.Float32() > 0.5; success {
c <- rand.Int()
} else {
fail <- true
}
}
func consumer(c chan int, success chan bool) {
for {
num := <-c
fmt.Printf("Producer produced: %d\n", num)
success <- true
}
}
func main() {
const nTries = 10
c := make(chan int)
done := make(chan bool)
for i := 0; i < nTries; i++ {
go producer(c, done)
}
go consumer(c, done)
for i := 0; i < nTries; i++ {
<-done
}
fmt.Println("All done.")
}
I'm adding this because the extant answers don't make a couple things clear. First, the range loop in the codewalk example is just an infinite event loop, there to keep re-checking and updating the same url list forever.
Next, a channel, all by itself, already is the idiomatic consumer-producer queue in Go. The size of the async buffer backing the channel determines how much producers can produce before getting backpressure. Set N = 0 below to see lock-step producer consumer without anyone racing ahead or getting behind. As it is, N = 10 will let the producer produce up to 10 products before blocking.
Last, there are some nice idioms for writing communicating sequential processees in Go (e.g. functions that start go routines for you, and using the for/select pattern to communicate and accept control commands). I think of WaitGroups as clumsy, and would like to see idiomatic examples instead.
package main
import (
"fmt"
"time"
)
type control int
const (
sleep control = iota
die // receiver will close the control chan in response to die, to ack.
)
func (cmd control) String() string {
switch cmd {
case sleep: return "sleep"
case die: return "die"
}
return fmt.Sprintf("%d",cmd)
}
func ProduceTo(writechan chan<- int, ctrl chan control, done chan bool) {
var product int
go func() {
for {
select {
case writechan <- product:
fmt.Printf("Producer produced %v\n", product)
product++
case cmd:= <- ctrl:
fmt.Printf("Producer got control cmd: %v\n", cmd)
switch cmd {
case sleep:
fmt.Printf("Producer sleeping 2 sec.\n")
time.Sleep(2000 * time.Millisecond)
case die:
fmt.Printf("Producer dies.\n")
close(done)
return
}
}
}
}()
}
func ConsumeFrom(readchan <-chan int, ctrl chan control, done chan bool) {
go func() {
var product int
for {
select {
case product = <-readchan:
fmt.Printf("Consumer consumed %v\n", product)
case cmd:= <- ctrl:
fmt.Printf("Consumer got control cmd: %v\n", cmd)
switch cmd {
case sleep:
fmt.Printf("Consumer sleeping 2 sec.\n")
time.Sleep(2000 * time.Millisecond)
case die:
fmt.Printf("Consumer dies.\n")
close(done)
return
}
}
}
}()
}
func main() {
N := 10
q := make(chan int, N)
prodCtrl := make(chan control)
consCtrl := make(chan control)
prodDone := make(chan bool)
consDone := make(chan bool)
ProduceTo(q, prodCtrl, prodDone)
ConsumeFrom(q, consCtrl, consDone)
// wait for a moment, to let them produce and consume
timer := time.NewTimer(10 * time.Millisecond)
<-timer.C
// tell producer to pause
fmt.Printf("telling producer to pause\n")
prodCtrl <- sleep
// wait for a second
timer = time.NewTimer(1 * time.Second)
<-timer.C
// tell consumer to pause
fmt.Printf("telling consumer to pause\n")
consCtrl <- sleep
// tell them both to finish
prodCtrl <- die
consCtrl <- die
// wait for that to actually happen
<-prodDone
<-consDone
}
You can use simple unbuffered channels without wait groups if you use the generator pattern with a fanIn function.
In the generator pattern, each producer returns a channel and is responsible for closing it. A fanIn function then iterates over these channels and forwards the values returned on them down a single channel that it returns.
The problem of course, is that the fanIn function forwards the zero value of the channel type (int) when each channel is closed.
You can work around it by using the zero value of your channel type as a sentinel value and only using the results from the fanIn channel if they are not the zero value.
Here's an example:
package main
import (
"fmt"
"math/rand"
)
const offset = 1
func producer() chan int {
cout := make(chan int)
go func() {
defer close(cout)
// May or may not produce.
success := rand.Float32() > 0.5
if success {
cout <- rand.Int() + offset
}
}()
return cout
}
func fanIn(cin []chan int) chan int {
cout := make(chan int)
go func() {
defer close(cout)
for _, c := range cin {
cout <- <-c
}
}()
return cout
}
func main() {
chans := make([]chan int, 0)
for i := 0; i < 10; i++ {
chans = append(chans, producer())
}
for num := range fanIn(chans) {
if num > offset {
fmt.Printf("Producer produced: %d\n", num)
}
}
fmt.Println("All done.")
}
producer-consumer is such a common pattern that I write a library prosumer for convenience with dealing with chan communication carefully. Eg:
func main() {
maxLoop := 10
var wg sync.WaitGroup
wg.Add(maxLoop)
defer wg.Wait()
consumer := func(ls []interface{}) error {
fmt.Printf("get %+v \n", ls)
wg.Add(-len(ls))
return nil
}
conf := prosumer.DefaultConfig(prosumer.Consumer(consumer))
c := prosumer.NewCoordinator(conf)
c.Start()
defer c.Close(true)
for i := 0; i < maxLoop; i++ {
fmt.Printf("try put %v\n", i)
discarded, err := c.Put(i)
if err != nil {
fmt.Errorf("discarded elements %+v for err %v", discarded, err)
wg.Add(-len(discarded))
}
time.Sleep(time.Second)
}
}
close has a param called graceful, which means whether drain the underlying chan.