I'm trying to do comparisons on big numbers but can only get a the string value.
So how do you do a condition on a big.Int. Below is the closest to what I have tried.
package main
import (
"fmt"
"math/big"
)
func main() {
dirtyVal := "9446744073709551615"
dv := big.NewInt(0)
dv.SetString(dirtyVal, 10)
userVal := dv.String()
maxVal := "18446744073709551615"
mv := big.NewInt(0)
mv.SetString(maxVal, 10)
// maxValue := mv.String()
if userVal > maxVal {
fmt.Println("True")
} else {
fmt.Println("False")
}
}
You can use func (*Int) Cmp to compare two big.Int http://golang.org/pkg/math/big/#Int.Cmp
if dv.Cmp(mv)>0 {
fmt.Println("True")
} else {
fmt.Println("False")
}
Related
I'm looking for a function that returns different integer values by matching an input string for different expressions.
Here is how it could be, but syntax:
package main
func MatchType(data []byte) int {
%% machine scanner;
%% write data;
cs, p, pe, eof := 0, 0, len(data), len(data)
_ = eof
%%{
main := ( 0..120 '-first-' ) #{ return 1 } ;
main := ( 0..120 '-second-' ) #{ return 2 } ;
write init;
write exec;
}%%
return -1
}
Can anybody please advise?
%%{
action a1 { return 8; }
action a2 { return 10; }
main := ( 0..120 '-first-' $a1 | 0..120 '-second-' $a2 ) #{ return 0 } ;
write init;
write exec;
}%%
I'm building the skeleton of a worker system using Go and I'm getting the "fatal error: all goroutines are asleep - deadlock!".
I am using two channels for coordination, one to create the jobs and the second to send the results. After creating the jobs I close the input channel.
My question is how I close the output channel so the program can exit correctly.
The code is:
package main
import (
"bufio"
"flag"
"fmt"
"log"
"math/rand"
"os"
"time"
)
type Work struct {
id int
ts time.Duration
}
const (
NumWorkers = 5000
NumJobs = 100000
)
func worker(in <-chan *Work, out chan<- *Work) {
for w := range in {
st := time.Now()
time.Sleep(time.Duration(rand.Int63n(int64(200 * time.Millisecond))))
w.ts = time.Since(st)
out <- w
}
}
func main() {
wait := flag.Bool("w", false, "wait for <enter> before starting")
flag.Parse()
if *wait {
fmt.Printf("I'm <%d>, press <enter> to continue", os.Getpid())
reader := bufio.NewReader(os.Stdin)
reader.ReadString('\n')
}
Run()
}
func Run() {
in, out := make(chan *Work, 100), make(chan *Work, 100)
for i := 0; i < NumWorkers; i++ {
go worker(in, out)
}
go createJobs(in)
receiveResults(out)
}
func createJobs(queue chan<- *Work) {
for i := 0; i < NumJobs; i++ {
work := &Work{i, 0}
queue <- work
}
close(queue)
}
func receiveResults(completed <-chan *Work) {
for w := range completed {
log.Printf("job %d completed in %s", w.id, w.ts)
}
}
Any help is appreciated :)
I missed the part about you knowing the cause of deadlock in original answer.
You mentioned WaitGroup, that is basically just a semaphore
You could use another "control" channel that workers sinal on when they are done
-
func worker(ctrl chan<- bool, in <-chan *Work, out chan<- *Work) {
for w := range in {
st := time.Now()
time.Sleep(time.Duration(rand.Int63n(int64(200 * time.Millisecond))))
w.ts = time.Since(st)
out <- w
}
ctrl <- true
}
func control(ctrl <-chan bool, numWorkers int, out chan<- *Work) {
for i=0; i<numWorkers; i++ {
<-ctrl
}
close(out)
}
original answer:
You do a range on completed:
for w := range completed {
log.Printf("job %d completed in %s", w.id, w.ts)
}
but that channel is never closed
I have a test program that gives different results when executing more than one goroutine on more than one Cpu (Goroutines = Cpus). The "test" is about syncing goroutines using channels, and the program itself counts occurences of chars in strings. It produces consistent results on one Cpu / one goroutine.
See code example on playground (Note: Run on local machine to execute on multi core, and watch the resulting numbers vary): http://play.golang.org/p/PT5jeCKgBv .
Code summary: The program counts occurences of 4 different chars (A,T, G,C) in (DNA) strings.
Problem: Result (n occurences of chars) varies when executed on multiple Cpu's (goroutines). Why?
Description:
A goroutine spawns work (SpawnWork) as strings to Workers. Sets up
artificial string input data (hardcoded strings are copied n times).
Goroutine Workers (Worker) are created equalling the numbers of Cpu's.
Workers checks each char in string and counts A,T's and sends the
sum into a channel, and G,C counts to another channel.
SpawnWork closes workstring channel as to control Workers (which consumes strings using range, which quits when the input channel is closed by SpawnWork).
When Workers has consumed its ranges (of chars) it sends a quit signal on the quit channel (quit <- true). These "pulses" will occure Cpu number of times ( Cpu count = goroutines count).
Main (select) loop will quit when it has received Cpu-count number of quit
signals.
Main func prints a summary of occurences of Chars (A,T's, G,C's).
Simplified code:
1. "Worker" (goroutines) counting chars in lines:
func Worker(inCh chan *[]byte, resA chan<- *int, resB chan<- *int, quit chan bool) {
//for p_ch := range inCh {
for {
p_ch, ok := <-inCh // similar to range
if ok {
ch := *p_ch
for i := 0; i < len(ch); i++ {
if ch[i] == 'A' || ch[i] == 'T' { // Count A:s and T:s
at++
} else if ch[i] == 'G' || ch[i] == 'C' { // Count G:s and C:s
gc++
}
}
resA <- &at // Send line results on separate channels
resB <- &gc // Send line results on separate channels
} else {
quit <- true // Indicate that we're all done
break
}
}
}
2. Spawn work (strings) to workers:
func SpawnWork(inStr chan<- *[]byte, quit chan bool) {
// Artificial input data
StringData :=
"NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN\n" +
"NTGAGAAATATGCTTTCTACTTTTTTGTTTAATTTGAACTTGAAAACAAAACACACACAA\n" +
"... etc\n" +
// ...
for scanner.Scan() {
s := scanner.Bytes()
if len(s) == 0 || s[0] == '>' {
continue
} else {
i++
inStr <- &s
}
}
close(inStr) // Indicate (to Workers) that there's no more strings coming.
}
3. Main routine:
func main() {
// Count Cpus, and count down in final select clause
CpuCnt := runtime.NumCPU()
runtime.GOMAXPROCS(CpuCnt)
// Make channels
resChA := make(chan *int)
resChB := make(chan *int)
quit := make(chan bool)
inStr := make(chan *[]byte)
// Set up Workers ( n = Cpu )
for i := 0; i < CpuCnt; i++ {
go Worker(inStr, resChA, resChB, quit)
}
// Send lines to Workers
go SpawnWork(inStr, quit)
// Count the number of "A","T" & "G","C" per line
// (comes in here as ints per row, on separate channels (at and gt))
for {
select {
case tmp_at := <-resChA:
tmp_gc := <-resChB // Ch A and B go in pairs anyway
A += *tmp_at // sum of A's and T's
B += *tmp_gc // sum of G's and C's
case <-quit:
// Each goroutine sends "quit" signals when it's done. Since
// the number of goroutines equals the Cpu counter, we count
// down each time a goroutine tells us it's done (quit at 0):
CpuCnt--
if CpuCnt == 0 { // When all goroutines are done then we're done.
goto out
}
}
}
out:
// Print report to screen
}
Why does this code count consistently only when executed on a singel cpu/goroutine? That is, the channels doesn't seem to sync, or the main loop quits forcefully before all goroutines are done? Scratching head.
(Again: See/run the full code at the playground: http://play.golang.org/p/PT5jeCKgBv )
// Rolf Lampa
Here is a working version which consistently produces the same results no matter how many cpus are used.
Here is what I did
remove passing of *int - very racy to pass in a channel!
remove passing of *[]byte - pointless as slices are reference types anyway
copy the slice before putting it in the channel - the slice points to the same memory causing a race
fix initialisation of at and gc in Worker - they were in the wrong place - this was the major cause of the difference in results
use sync.WaitGroup for synchronisation and channel close()
I used the -race parameter of go build to find and fix the data races.
package main
import (
"bufio"
"fmt"
"runtime"
"strings"
"sync"
)
func Worker(inCh chan []byte, resA chan<- int, resB chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("Worker started...")
for ch := range inCh {
at := 0
gc := 0
for i := 0; i < len(ch); i++ {
if ch[i] == 'A' || ch[i] == 'T' {
at++
} else if ch[i] == 'G' || ch[i] == 'C' {
gc++
}
}
resA <- at
resB <- gc
}
}
func SpawnWork(inStr chan<- []byte) {
fmt.Println("Spawning work:")
// An artificial input source.
StringData :=
"NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN\n" +
"NTGAGAAATATGCTTTCTACTTTTTTGTTTAATTTGAACTTGAAAACAAAACACACACAA\n" +
"CTTCCCAATTGGATTAGACTATTAACATTTCAGAAAGGATGTAAGAAAGGACTAGAGAGA\n" +
"TATACTTAATGTTTTTAGTTTTTTAAACTTTACAAACTTAATACTGTCATTCTGTTGTTC\n" +
"AGTTAACATCCCTGAATCCTAAATTTCTTCAGATTCTAAAACAAAAAGTTCCAGATGATT\n" +
"TTATATTACACTATTTACTTAATGGTACTTAAATCCTCATTNNNNNNNNCAGTACGGTTG\n" +
"TTAAATANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN\n" +
"NNNNNNNCTTCAGAAATAAGTATACTGCAATCTGATTCCGGGAAATATTTAGGTTCATAA\n"
// Expand data n times
tmp := StringData
for n := 0; n < 1000; n++ {
StringData = StringData + tmp
}
scanner := bufio.NewScanner(strings.NewReader(StringData))
scanner.Split(bufio.ScanLines)
var i int
for scanner.Scan() {
s := scanner.Bytes()
if len(s) == 0 || s[0] == '>' {
continue
} else {
i++
s_copy := append([]byte(nil), s...)
inStr <- s_copy
}
}
close(inStr)
}
func main() {
CpuCnt := runtime.NumCPU() // Count down in select clause
CpuOut := CpuCnt // Save for print report
runtime.GOMAXPROCS(CpuCnt)
fmt.Printf("Processors: %d\n", CpuCnt)
resChA := make(chan int)
resChB := make(chan int)
inStr := make(chan []byte)
fmt.Println("Spawning workers:")
var wg sync.WaitGroup
for i := 0; i < CpuCnt; i++ {
wg.Add(1)
go Worker(inStr, resChA, resChB, &wg)
}
fmt.Println("Spawning work:")
go func() {
SpawnWork(inStr)
wg.Wait()
close(resChA)
close(resChB)
}()
A := 0
B := 0
LineCnt := 0
for tmp_at := range resChA {
tmp_gc := <-resChB // Theese go together anyway
A += tmp_at
B += tmp_gc
LineCnt++
}
if !(A+B > 0) {
fmt.Println("No A/B was found!")
} else {
ABFraction := float32(B) / float32(A+B)
fmt.Println("\n----------------------------")
fmt.Printf("Cpu's : %d\n", CpuOut)
fmt.Printf("Lines : %d\n", LineCnt)
fmt.Printf("A+B : %d\n", A+B)
fmt.Printf("A : %d\n", A)
fmt.Printf("B : %d\n", A)
fmt.Printf("AB frac: %v\n", ABFraction*100)
fmt.Println("----------------------------")
}
}
I'm having something of a problem with goroutines. Why is it that this code executes in ~125ms (note sequential execution):
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
cmd.Run()
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
When this code takes about 20 seconds (concurrent execution using goroutines):
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
ch := make(chan bool)
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
go lessc(ch, cmd)
}
fmt.Println(n, " goroutines started.")
for i := 0; i < n; i++ {
_ = <-ch
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
func lessc(ch chan bool, c *exec.Cmd) {
c.Run()
ch <- true
}
Using go 1.0.3 on i7 720QM (4C/8T) 8GB RAM linux/x86-64
Also built and tested using 1.0.2 and got the same problem on the same machine.
Edit: Solved by #jnml below. If anyone cares about the new fixed concurrent code here it is:
package main
import (
"os/exec"
"time"
"fmt"
)
func main() {
ch := make(chan bool)
n := 2000
start := time.Now()
for i := 0; i < n; i++ {
go lessc(ch)
}
fmt.Println(n, " goroutines started.")
for i := 0; i < n; i++ {
_ = <-ch
}
finish := time.Now()
fmt.Printf("Program took %v to run\n", finish.Sub(start))
}
func lessc(ch chan bool) {
cmd := exec.Command("lessc", "--yui-compress", "test.less")
cmd.Run()
ch <- true
}
IMO your program is not correct. It contains a race condition and thus can do literally anything. Any timing of it don't make sense.
You're creating one exec.Cmd and then concurrently (== data race) perform its Run method from several goroutines. exec.Cmd never mentions it could be reused for more than once Run - even if serially.
exec.Cmd has some state initialized by exec.Command and a different state after executing Run. IOW, after executing the Run method, the state is not anymore initialized and probably not fit for another Run
I'm doing some stream processing in Go and got stuck trying to figure out how to do this the "Go way" without locks.
This contrived example shows the problem I'm facing.
We get one thing at a time.
There is a goroutine which buffers them into a slice called things.
When things becomes full len(things) == 100 then it is processed somehow and reset
There are n number of concurrent goroutines that need to access things before it's full
Access to the "incomplete" things from other goroutines is not predictable.
Neither doSomethingWithPartial nor doSomethingWithComplete needs to mutate things
Code:
var m sync.Mutex
var count int64
things := make([]int64, 0, 100)
// slices of data are constantly being generated and used
go func() {
for {
m.Lock()
if len(things) == 100 {
// doSomethingWithComplete does not modify things
doSomethingWithComplete(things)
things = make([]int64, 0, 100)
}
things = append(things, count)
m.Unlock()
count++
}
}()
// doSomethingWithPartial needs to access the things before they're ready
for {
m.Lock()
// doSomethingWithPartial does not modify things
doSomethingWithPartial(things)
m.Unlock()
}
I know that slices are immutable so does that mean I can remove the mutex and expect it to still work (I assume no).
How can I refactor this to use channels instead of a mutex.
Edit: Here's the solution I came up with that does not use a mutex
package main
import (
"fmt"
"sync"
"time"
)
func Incrementor() chan int {
ch := make(chan int)
go func() {
count := 0
for {
ch <- count
count++
}
}()
return ch
}
type Foo struct {
things []int
requests chan chan []int
stream chan int
C chan []int
}
func NewFoo() *Foo {
foo := &Foo{
things: make([]int, 0, 100),
requests: make(chan chan []int),
stream: Incrementor(),
C: make(chan []int),
}
go foo.Launch()
return foo
}
func (f *Foo) Launch() {
for {
select {
case ch := <-f.requests:
ch <- f.things
case thing := <-f.stream:
if len(f.things) == 100 {
f.C <- f.things
f.things = make([]int, 0, 100)
}
f.things = append(f.things, thing)
}
}
}
func (f *Foo) Things() []int {
ch := make(chan []int)
f.requests <- ch
return <-ch
}
func main() {
foo := NewFoo()
var wg sync.WaitGroup
wg.Add(10)
for i := 0; i < 10; i++ {
go func(i int) {
time.Sleep(time.Millisecond * time.Duration(i) * 100)
things := foo.Things()
fmt.Println("got things:", len(things))
wg.Done()
}(i)
}
go func() {
for _ = range foo.C {
// do something with things
}
}()
wg.Wait()
}
It should be noted that the "Go way" is probably just to use a mutex for this. It's fun to work out how to do it with a channel but a mutex is probably simpler and easier to reason about for this particular problem.