Assembly Line in Golang using concurrency - concurrency

New to Go. I'm attempting to code an "assembly line" where multiple functions act like workers and pass some data structure to each other down the line, each doing something to the data structure.
type orderStruct struct {
orderNum,capacity int
orderCode uint64
box [9]int
}
func position0(in chan orderStruct){
order := <-in
if((order.orderCode<<63)>>63 == 1){
order.box[order.capacity] = 1
order.capacity += 1
}
fmt.Println(" filling box {", order.orderNum, order.orderCode, order.box, order.capacity, "} at position 0")
}
func startOrder(in chan orderStruct){
order := <-in
fmt.Printf("\nStart an empty box for customer order number %d , request number %d\n", order.orderNum, order.orderCode)
fmt.Println(" starting box {", order.orderNum, order.orderCode, order.box, order.capacity, "}")
d := make(chan orderStruct,1)
go position0(d)
d <- order
}
func main() {
var orders [10]orderStruct
numOrders := len(os.Args)-1
var x int
for i := 0; i < numOrders; i++{
x, _ = strconv.Atoi(os.Args[i+1])
orders[i].orderCode = uint64(x)
orders[i].orderNum = i+1
orders[i].capacity = 0
for j := 0; j < 9; j++{
orders[i].box[j] = 0
}
c := make(chan orderStruct)
go startOrder(c)
c <- orders[i]
}
}
So basically the issue I'm having is that the print statements in startOrder() execute fine, but when I try to pass the struct to position0(), nothing is printed. Am I misunderstanding how channels work?

Pipelines are a great place to start when learning to program concurrently in Go. Nick Craig-Wood's answer provides a working solution to this specific challenge.
There is a whole range of other ways to use concurrency in Go. Broadly, there are three categories divided according to what is being treated as concurrent:
Functional decomposition - Creating pipelines of several functions is a good way to get started - and is your question's topic. It's quite easy to think about and quite productive. However, if it progresses to truly parallel hardware, it's quite hard to balance the load well. Everything goes at the speed of the slowest pipeline stage.
Geometric decomposition - Dividing the data up into separate regions that can be processed independently (or without too much communication). Grid-based systems are popularly used in certain domains of scientific high-performance computing, such as weather-forecasting.
Farming - Identifying how the work to be done can be chopped into (a large number of) tasks and these tasks can be allocated to 'workers' one by one until all are completed. Often, the number of tasks far exceeds the number of workers. This category includes all the so-called 'embarrassingly parallel' problems (embarrassing because if you fail to get your high-performance system to give linear speed-up, you look a bit daft).
I could add a fourth category of hybrids of several of the above.
There is quite a lot of literature about this, including much from the days of Occam programming in the '80s and '90s. Go and Occam both use CSP message passing so the issues are similar. I would single out the helpful book Practical Parallel Processing: An introduction to problem solving in parallel (Chalmers and Tidmus 1996).

I've attempted to re-write what you've written to work properly. You can run it on the playground
The main differences are
only two go routines are started - these act as the two workers on the production line - one taking orders and the other filling boxes
use of sync.WaitGroup to find out when they end
use of for x := range channel
use of close(c) to signal end of channel
you could start multiple copies of each worker and the code would still work fine (repeat the wg.Add(1); go startOrders(c, wg) code)
Here is the code
package main
import (
"fmt"
"sync"
)
type orderStruct struct {
orderNum, capacity int
orderCode uint64
box [9]int
}
func position0s(in chan orderStruct, wg *sync.WaitGroup) {
defer wg.Done()
for order := range in {
if (order.orderCode<<63)>>63 == 1 {
order.box[order.capacity] = 1
order.capacity += 1
}
fmt.Println(" filling box {", order.orderNum, order.orderCode, order.box, order.capacity, "} at position 0")
}
}
func startOrders(in chan orderStruct, wg *sync.WaitGroup) {
defer wg.Done()
d := make(chan orderStruct)
wg.Add(1)
go position0s(d, wg)
for order := range in {
fmt.Printf("\nStart an empty box for customer order number %d , request number %d\n", order.orderNum, order.orderCode)
fmt.Println(" starting box {", order.orderNum, order.orderCode, order.box, order.capacity, "}")
d <- order
}
close(d)
}
func main() {
var orders [10]orderStruct
numOrders := 4
var x int = 10
wg := new(sync.WaitGroup)
c := make(chan orderStruct)
wg.Add(1)
go startOrders(c, wg)
for i := 0; i < numOrders; i++ {
orders[i].orderCode = uint64(x)
orders[i].orderNum = i + 1
orders[i].capacity = 0
for j := 0; j < 9; j++ {
orders[i].box[j] = 0
}
c <- orders[i]
}
close(c)
wg.Wait()
}

Related

Long cycle blocks application

I hve following cycle in my app
var maxIterations: Int = 0
func calculatePoint(cn: Complex) -> Int {
let threshold: Double = 2
var z: Complex = .init(re: 0, im: 0)
var z2: Complex = .init(re: 0, im: 0)
var iteration: Int = 0
repeat {
z2 = self.pow2ForComplex(cn: z)
z.re = z2.re + cn.re
z.im = z2.im + cn.im
iteration += 1
} while self.absForComplex(cn: z) <= threshold && iteration < self.maxIterations
return iteration
}
and rainbow wheel is showing during the cycle execution. How I can manage that app is still responding to UI actions?
Note I have NSProgressIndicator updated in different part of code which is not being updated (progress is not shown) while the cycle is running.
I have suspicion that it has something to do with dispatcing but I'm quite "green" with that. I do appreciate any help.
Thanks.
To dispatch something asynchronously, call async on the appropriate queue. For example, you might change this method to do the calculation on a global background queue, and then report the result back on the main queue. By the way, when you do that, you shift from returning the result immediately to using a completion handler closure which the asynchronous method will call when the calculation is done:
func calculatePoint(_ cn: Complex, completionHandler: #escaping (Int) -> Void) {
DispatchQueue.global(qos: .userInitiated).async {
// do your complicated calculation here which calculates `iteration`
DispatchQueue.main.async {
completionHandler(iteration)
}
}
}
And you'd call it like so:
// start NSProgressIndicator here
calculatePoint(point) { iterations in
// use iterations here, noting that this is called asynchronously (i.e. later)
// stop NSProgressIndicator here
}
// don't use iterations here, because the above closure is likely not yet done by the time we get here;
// we'll get here almost immediately, but the above completion handler is called when the asynchronous
// calculation is done.
Martin has surmised that you are calculating a Mandelbrot set. If so, dispatching the calculation of each point to a global queue is not a good idea (because these global queues dispatch their blocks to worker threads, but those worker threads are quite limited).
If you want to avoid using up all of these global queue worker threads, one simple choice is to take the async call out of your routine that calculates an individual point, and just dispatch the whole routine that iterates through all of the complex values to a background thread:
DispatchQueue.global(qos: .userInitiated).async {
for row in 0 ..< height {
for column in 0 ..< width {
let c = ...
let m = self.mandelbrotValue(c)
pixelBuffer[row * width + column] = self.color(for: m)
}
}
let outputCGImage = context.makeImage()!
DispatchQueue.main.async {
completionHandler(NSImage(cgImage: outputCGImage, size: NSSize(width: width, height: height)))
}
}
That's solves the "get it off the main thread" and the "don't use up the worker threads" problems, but now we've swung from using too many worker threads, to only using one worker thread, not fully utilizing the device. We really want to do as many calculations in parallel (while not exhausting the worker threads).
One approach, when doing a for loop for complex calculations, is to use dispatch_apply (now called concurrentPerform in Swift 3). This is like a for loop, but it does the each of the loops concurrently with respect to each other (but, at the end, waits for all of those concurrent loops to finish). To do this, replace the outer for loop with concurrentPerform:
DispatchQueue.global(qos: .userInitiated).async {
DispatchQueue.concurrentPerform(iterations: height) { row in
for column in 0 ..< width {
let c = ...
let m = self.mandelbrotValue(c)
pixelBuffer[row * width + column] = self.color(for: m)
}
}
let outputCGImage = context.makeImage()!
DispatchQueue.main.async {
completionHandler(NSImage(cgImage: outputCGImage, size: NSSize(width: width, height: height)))
}
}
The concurrentPerform (formerly known as dispatch_apply) will perform the various iterations of that loop concurrently, but it will automatically optimize the number of concurrent threads for the capabilities of your device. On my MacBook Pro, this made the calculation 4.8 times faster than the simple for loop. Note, I still dispatch the whole thing to a global queue (because concurrentPerform runs synchronously, and we never want to perform slow, synchronous calculations on the main thread), but concurrentPerform will run the calculations in parallel. It's a great way to enjoy concurrency in a for loop in such a way that you won't exhaust GCD worker threads.
By the way, you mentioned that you are updating a NSProgressIndicator. Ideally, you want to update it as every pixel is processed, but if you do that, the UI may get backlogged, unable to keep up with all of these updates. You'll end up slowing the final result to allow the UI to catch up to all of those progress indicator updates.
The solution is to decouple the UI update from the progress updates. You want the background calculations to inform you as each pixel is updated, but you want the progress indicator to be updated, each time effectively saying "ok, update the progress with however many pixels were calculated since the last time I checked". There are cumbersome manual techniques to do that, but GCD provides a really elegant solution, a dispatch source, or more specifically, a DispatchSourceUserDataAdd.
So define properties for the dispatch source and a counter to keep track of how many pixels have been processed thus far:
let source = DispatchSource.makeUserDataAddSource(queue: .main)
var pixelsProcessed: UInt = 0
And then set up an event handler for the dispatch source, which updates the progress indicator:
source.setEventHandler() { [unowned self] in
self.pixelsProcessed += self.source.data
self.progressIndicator.doubleValue = Double(self.pixelsProcessed) / Double(width * height)
}
source.resume()
And then, as you process the pixels, you can simply add to your source from the background thread:
DispatchQueue.concurrentPerform(iterations: height) { row in
for column in 0 ..< width {
let c = ...
let m = self.mandelbrotValue(for: c)
pixelBuffer[row * width + column] = self.color(for: m)
self.source.add(data: 1)
}
}
If you do this, it will update the UI with the greatest frequency possible, but it will never get backlogged with a queue of updates. The dispatch source will coalesce these add calls for you.

Goroutine execution time with different input data

I am experimenting with goroutine for parallelizing some computation. However, the execution time of goroutine confuse me. My experiment setup is simple.
runtime.GOMAXPROCS(3)
datalen := 1000000000
data21 := make([]float64, datalen)
data22 := make([]float64, datalen)
data23 := make([]float64, datalen)
t := time.Now()
res := make(chan interface{}, dlen)
go func() {
for i := 0; i < datalen; i++ {
data22[i] = math.Sqrt(13)
}
res <- true
}()
go func() {
for i := 0; i < datalen; i++ {
data22[i] = math.Sqrt(13)
}
res <- true
}()
go func() {
for i := 0; i < datalen; i++ {
data22[i] = math.Sqrt(13)
}
res <- true
}()
for i:=0; i<3; i++ {
<-res
}
fmt.Printf("The parallel for loop took %v to run.\n", time.Since(t))
Notice that I loaded the same data in 3 goroutines, the execution time for this program is
The parallel for loop took 7.436060182s to run.
However, if I let each goroutine handle different data as follows:
runtime.GOMAXPROCS(3)
datalen := 1000000000
data21 := make([]float64, datalen)
data22 := make([]float64, datalen)
data23 := make([]float64, datalen)
t := time.Now()
res := make(chan interface{}, dlen)
go func() {
for i := 0; i < datalen; i++ {
data21[i] = math.Sqrt(13)
}
res <- true
}()
go func() {
for i := 0; i < datalen; i++ {
data22[i] = math.Sqrt(13)
}
res <- true
}()
go func() {
for i := 0; i < datalen; i++ {
data23[i] = math.Sqrt(13)
}
res <- true
}()
for i:=0; i<3; i++ {
<-res
}
fmt.Printf("The parallel for loop took %v to run.\n", time.Since(t))
The execution time for this is almost 3 times more than previous and is almost equal/worse then sequential execution without goroutine
The parallel for loop took 20.744438468s to run.
I guess maybe I use the goroutine in a wrong way. So what should be the correct way to use multiple goroutines to handle different pieces of data;
Since your example program is not performing any substantial calculation, the bottleneck is going to be the speed at which data can be written to memory. With the settings in the example, we're talking about 22 GB of writes which is not insignificant.
Given the time difference in the run time of the two examples, one likely possibility is that it isn't actually writing as much to the RAM. Given that memory writes are cached by the CPU, the execution probably looks something like this:
the first goroutine writes out data to a cache line representing the start of the data22 array.
the second goroutine writes out data to a cache line representing the same location. The CPU running the first goroutine notices that the write invalidates its own cached write, so throws away its changes.
the third goroutine writes out data to a cache line representing the same location. The CPU running the second goroutine notices that the write invalidates its own cached write, so throws away its changes.
the cache line in the third CPU is evicted and the changes are written out to RAM.
This process continues as the goroutines progress through the data22 array. Since RAM is the bottleneck and we end up writing one third as much data in this scenario, it isn't that surprising that it runs approximately 3 times as fast as the second case.
You are using enormous amounts of memory. 1000000000 * 8 = 8GB in the first example and 3 * 1000000000 * 8 = 24GB in the second example. In the second example you are probably using lots of swap space. Disk I/O is very, very slow, even on an SSD.
Change datalen := 1000000000 to datalen := 100000000, a 10-fold decrease. What are your run times now? Average at least three runs of each example. How much memory does your computer have? Are you using an SSD?

What is the chance of getting the same id by using CreateUUID() [duplicate]

How safe is it to use UUID to uniquely identify something (I'm using it for files uploaded to the server)? As I understand it, it is based off random numbers. However, it seems to me that given enough time, it would eventually repeat it self, just by pure chance. Is there a better system or a pattern of some type to alleviate this issue?
Very safe:
the annual risk of a given person being hit by a meteorite is
estimated to be one chance in 17 billion, which means the
probability is about 0.00000000006 (6 × 10−11), equivalent to the odds
of creating a few tens of trillions of UUIDs in a year and having one
duplicate. In other words, only after generating 1 billion UUIDs every
second for the next 100 years, the probability of creating just one
duplicate would be about 50%.
Caveat:
However, these probabilities only hold when the UUIDs are generated
using sufficient entropy. Otherwise, the probability of duplicates
could be significantly higher, since the statistical dispersion might
be lower. Where unique identifiers are required for distributed
applications, so that UUIDs do not clash even when data from many
devices is merged, the randomness of the seeds and generators used on
every device must be reliable for the life of the application. Where
this is not feasible, RFC4122 recommends using a namespace variant
instead.
Source: The Random UUID probability of duplicates section of the Wikipedia article on Universally unique identifiers (link leads to a revision from December 2016 before editing reworked the section).
Also see the current section on the same subject on the same Universally unique identifier article, Collisions.
If by "given enough time" you mean 100 years and you're creating them at a rate of a billion a second, then yes, you have a 50% chance of having a collision after 100 years.
There is more than one type of UUID, so "how safe" depends on which type (which the UUID specifications call "version") you are using.
Version 1 is the time based plus MAC address UUID. The 128-bits contains 48-bits for the network card's MAC address (which is uniquely assigned by the manufacturer) and a 60-bit clock with a resolution of 100 nanoseconds. That clock wraps in 3603 A.D. so these UUIDs are safe at least until then (unless you need more than 10 million new UUIDs per second or someone clones your network card). I say "at least" because the clock starts at 15 October 1582, so you have about 400 years after the clock wraps before there is even a small possibility of duplications.
Version 4 is the random number UUID. There's six fixed bits and the rest of the UUID is 122-bits of randomness. See Wikipedia or other analysis that describe how very unlikely a duplicate is.
Version 3 is uses MD5 and Version 5 uses SHA-1 to create those 122-bits, instead of a random or pseudo-random number generator. So in terms of safety it is like Version 4 being a statistical issue (as long as you make sure what the digest algorithm is processing is always unique).
Version 2 is similar to Version 1, but with a smaller clock so it is going to wrap around much sooner. But since Version 2 UUIDs are for DCE, you shouldn't be using these.
So for all practical problems they are safe. If you are uncomfortable with leaving it up to probabilities (e.g. your are the type of person worried about the earth getting destroyed by a large asteroid in your lifetime), just make sure you use a Version 1 UUID and it is guaranteed to be unique (in your lifetime, unless you plan to live past 3603 A.D.).
So why doesn't everyone simply use Version 1 UUIDs? That is because Version 1 UUIDs reveal the MAC address of the machine it was generated on and they can be predictable -- two things which might have security implications for the application using those UUIDs.
The answer to this may depend largely on the UUID version.
Many UUID generators use a version 4 random number. However, many of these use Pseudo a Random Number Generator to generate them.
If a poorly seeded PRNG with a small period is used to generate the UUID I would say it's not very safe at all. Some random number generators also have poor variance. i.e. favouring certain numbers more often than others. This isn't going to work well.
Therefore, it's only as safe as the algorithms used to generate it.
On the flip side, if you know the answer to these questions then I think a version 4 uuid should be very safe to use. In fact I'm using it to identify blocks on a network block file system and so far have not had a clash.
In my case, the PRNG I'm using is a mersenne twister and I'm being careful with the way it's seeded which is from multiple sources including /dev/urandom. Mersenne twister has a period of 2^19937 − 1. It's going to be a very very long time before I see a repeat uuid.
So pick a good library or generate it yourself and make sure you use a decent PRNG algorithm.
For UUID4 I make it that there are approximately as many IDs as there are grains of sand in a cube-shaped box with sides 360,000km long. That's a box with sides ~2 1/2 times longer than Jupiter's diameter.
Working so someone can tell me if I've messed up units:
volume of grain of sand 0.00947mm^3 (Guardian)
UUID4 has 122 random bits -> 5.3e36 possible values (wikipedia)
volume of that many grains of sand = 5.0191e34 mm^3 or 5.0191e+25m^3
side length of cubic box with that volume = 3.69E8m or 369,000km
diameter of Jupiter: 139,820km (google)
I concur with the other answers. UUIDs are safe enough for nearly all practical purposes1, and certainly for yours.
But suppose (hypothetically) that they aren't.
Is there a better system or a pattern of some type to alleviate this issue?
Here are a couple of approaches:
Use a bigger UUID. For instance, instead of a 128 random bits, use 256 or 512 or ... Each bit you add to a type-4 style UUID will reduce the probability of a collision by a half, assuming that you have a reliable source of entropy2.
Build a centralized or distributed service that generates UUIDs and records each and every one it has ever issued. Each time it generates a new one, it checks that the UUID has never been issued before. Such a service would be technically straight-forward to implement (I think) if we assumed that the people running the service were absolutely trustworthy, incorruptible, etcetera. Unfortunately, they aren't ... especially when there is the possibility of governments' security organizations interfering. So, this approach is probably impractical, and may be3 impossible in the real world.
1 - If uniqueness of UUIDs determined whether nuclear missiles got launched at your country's capital city, a lot of your fellow citizens would not be convinced by "the probability is extremely low". Hence my "nearly all" qualification.
2 - And here's a philosophical question for you. Is anything ever truly random? How would we know if it wasn't? Is the universe as we know it a simulation? Is there a God who might conceivably "tweak" the laws of physics to alter an outcome?
3 - If anyone knows of any research papers on this problem, please comment.
Quoting from Wikipedia:
Thus, anyone can create a UUID and use
it to identify something with
reasonable confidence that the
identifier will never be
unintentionally used by anyone for
anything else
It goes on to explain in pretty good detail on how safe it actually is. So to answer your question: Yes, it's safe enough.
UUID schemes generally use not only a pseudo-random element, but also the current system time, and some sort of often-unique hardware ID if available, such as a network MAC address.
The whole point of using UUID is that you trust it to do a better job of providing a unique ID than you yourself would be able to do. This is the same rationale behind using a 3rd party cryptography library rather than rolling your own. Doing it yourself may be more fun, but it's typically less responsible to do so.
Been doing it for years. Never run into a problem.
I usually set up my DB's to have one table that contains all the keys and the modified dates and such. Haven't run into a problem of duplicate keys ever.
The only drawback that it has is when you are writing some queries to find some information quickly you are doing a lot of copying and pasting of the keys. You don't have the short easy to remember ids anymore.
Here's a testing snippet for you to test it's uniquenes.
inspired by #scalabl3's comment
Funny thing is, you could generate 2 in a row that were identical, of course at mind-boggling levels of coincidence, luck and divine intervention, yet despite the unfathomable odds, it's still possible! :D Yes, it won't happen. just saying for the amusement of thinking about that moment when you created a duplicate! Screenshot video! – scalabl3 Oct 20 '15 at 19:11
If you feel lucky, check the checkbox, it only checks the currently generated id's. If you wish a history check, leave it unchecked.
Please note, you might run out of ram at some point if you leave it unchecked. I tried to make it cpu friendly so you can abort quickly when needed, just hit the run snippet button again or leave the page.
Math.log2 = Math.log2 || function(n){ return Math.log(n) / Math.log(2); }
Math.trueRandom = (function() {
var crypt = window.crypto || window.msCrypto;
if (crypt && crypt.getRandomValues) {
// if we have a crypto library, use it
var random = function(min, max) {
var rval = 0;
var range = max - min;
if (range < 2) {
return min;
}
var bits_needed = Math.ceil(Math.log2(range));
if (bits_needed > 53) {
throw new Exception("We cannot generate numbers larger than 53 bits.");
}
var bytes_needed = Math.ceil(bits_needed / 8);
var mask = Math.pow(2, bits_needed) - 1;
// 7776 -> (2^13 = 8192) -1 == 8191 or 0x00001111 11111111
// Create byte array and fill with N random numbers
var byteArray = new Uint8Array(bytes_needed);
crypt.getRandomValues(byteArray);
var p = (bytes_needed - 1) * 8;
for(var i = 0; i < bytes_needed; i++ ) {
rval += byteArray[i] * Math.pow(2, p);
p -= 8;
}
// Use & to apply the mask and reduce the number of recursive lookups
rval = rval & mask;
if (rval >= range) {
// Integer out of acceptable range
return random(min, max);
}
// Return an integer that falls within the range
return min + rval;
}
return function() {
var r = random(0, 1000000000) / 1000000000;
return r;
};
} else {
// From http://baagoe.com/en/RandomMusings/javascript/
// Johannes Baagøe <baagoe#baagoe.com>, 2010
function Mash() {
var n = 0xefc8249d;
var mash = function(data) {
data = data.toString();
for (var i = 0; i < data.length; i++) {
n += data.charCodeAt(i);
var h = 0.02519603282416938 * n;
n = h >>> 0;
h -= n;
h *= n;
n = h >>> 0;
h -= n;
n += h * 0x100000000; // 2^32
}
return (n >>> 0) * 2.3283064365386963e-10; // 2^-32
};
mash.version = 'Mash 0.9';
return mash;
}
// From http://baagoe.com/en/RandomMusings/javascript/
function Alea() {
return (function(args) {
// Johannes Baagøe <baagoe#baagoe.com>, 2010
var s0 = 0;
var s1 = 0;
var s2 = 0;
var c = 1;
if (args.length == 0) {
args = [+new Date()];
}
var mash = Mash();
s0 = mash(' ');
s1 = mash(' ');
s2 = mash(' ');
for (var i = 0; i < args.length; i++) {
s0 -= mash(args[i]);
if (s0 < 0) {
s0 += 1;
}
s1 -= mash(args[i]);
if (s1 < 0) {
s1 += 1;
}
s2 -= mash(args[i]);
if (s2 < 0) {
s2 += 1;
}
}
mash = null;
var random = function() {
var t = 2091639 * s0 + c * 2.3283064365386963e-10; // 2^-32
s0 = s1;
s1 = s2;
return s2 = t - (c = t | 0);
};
random.uint32 = function() {
return random() * 0x100000000; // 2^32
};
random.fract53 = function() {
return random() +
(random() * 0x200000 | 0) * 1.1102230246251565e-16; // 2^-53
};
random.version = 'Alea 0.9';
random.args = args;
return random;
}(Array.prototype.slice.call(arguments)));
};
return Alea();
}
}());
Math.guid = function() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.trueRandom() * 16 | 0,
v = c == 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
};
function logit(item1, item2) {
console.log("Do "+item1+" and "+item2+" equal? "+(item1 == item2 ? "OMG! take a screenshot and you'll be epic on the world of cryptography, buy a lottery ticket now!":"No they do not. shame. no fame")+ ", runs: "+window.numberofRuns);
}
numberofRuns = 0;
function test() {
window.numberofRuns++;
var x = Math.guid();
var y = Math.guid();
var test = x == y || historyTest(x,y);
logit(x,y);
return test;
}
historyArr = [];
historyCount = 0;
function historyTest(item1, item2) {
if(window.luckyDog) {
return false;
}
for(var i = historyCount; i > -1; i--) {
logit(item1,window.historyArr[i]);
if(item1 == history[i]) {
return true;
}
logit(item2,window.historyArr[i]);
if(item2 == history[i]) {
return true;
}
}
window.historyArr.push(item1);
window.historyArr.push(item2);
window.historyCount+=2;
return false;
}
luckyDog = false;
document.body.onload = function() {
document.getElementById('runit').onclick = function() {
window.luckyDog = document.getElementById('lucky').checked;
var val = document.getElementById('input').value
if(val.trim() == '0') {
var intervaltimer = window.setInterval(function() {
var test = window.test();
if(test) {
window.clearInterval(intervaltimer);
}
},0);
}
else {
var num = parseInt(val);
if(num > 0) {
var intervaltimer = window.setInterval(function() {
var test = window.test();
num--;
if(num < 0 || test) {
window.clearInterval(intervaltimer);
}
},0);
}
}
};
};
Please input how often the calulation should run. set to 0 for forever. Check the checkbox if you feel lucky.<BR/>
<input type="text" value="0" id="input"><input type="checkbox" id="lucky"><button id="runit">Run</button><BR/>
I don't know if this matters to you, but keep in mind that GUIDs are globally unique, but substrings of GUIDs aren't.
I should mention I bought two external Seagate drives on Amazon, and they had the same device UUID, but differing PARTUUID. Presumably the cloning software they used to format the drives just copied the UUID as well.
Obviously UUID collisions are much more likely to happen due to a flawed cloning or copying process than from random coincidence. Bear that in mind when calculating UUID risks.

Why does the use of an unbuffered channel in the same goroutine result in a deadlock?

I'm sure that there is a simple explanation to this trivial situation, but I'm new to the go concurrency model.
when I run this example
package main
import "fmt"
func main() {
c := make(chan int)
c <- 1
fmt.Println(<-c)
}
I get this error :
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/home/tarrsalah/src/go/src/github.com/tarrsalah/tour.golang.org/65.go:8 +0x52
exit status 2
Why ?
Wrapping c <- in a goroutine makes the example run as we expected
package main
import "fmt"
func main() {
c := make(chan int)
go func(){
c <- 1
}()
fmt.Println(<-c)
}
Again, why ?
Please, I need deep explanation , not just how to eliminate the deadlock and fix the code.
From the documentation :
If the channel is unbuffered, the sender blocks until the receiver has received the value. If the channel has a buffer, the sender blocks only until the value
has been copied to the buffer; if the buffer is full, this means
waiting until some receiver has retrieved a value.
Said otherwise :
when a channel is full, the sender waits for another goroutine to make some room by receiving
you can see an unbuffered channel as an always full one : there must be another goroutine to take what the sender sends.
This line
c <- 1
blocks because the channel is unbuffered. As there's no other goroutine to receive the value, the situation can't resolve, this is a deadlock.
You can make it not blocking by changing the channel creation to
c := make(chan int, 1)
so that there's room for one item in the channel before it blocks.
But that's not what concurrency is about. Normally, you wouldn't use a channel without other goroutines to handle what you put inside. You could define a receiving goroutine like this :
func main() {
c := make(chan int)
go func() {
fmt.Println("received:", <-c)
}()
c <- 1
}
Demonstration
In unbuffered channel writing to channel will not happen until there must be some receiver which is waiting to receive the data, which means in the below example
func main(){
ch := make(chan int)
ch <- 10 /* Main routine is Blocked, because there is no routine to receive the value */
<- ch
}
Now In case where we have other go routine, the same principle applies
func main(){
ch :=make(chan int)
go task(ch)
ch <-10
}
func task(ch chan int){
<- ch
}
This will work because task routine is waiting for the data to be consumed before writes happen to unbuffered channel.
To make it more clear, lets swap the order of second and third statements in main function.
func main(){
ch := make(chan int)
ch <- 10 /*Blocked: No routine is waiting for the data to be consumed from the channel */
go task(ch)
}
This will leads to Deadlock
So in short, writes to unbuffered channel happens only when there is some routine waiting to read from channel, else the write operation is blocked forever and leads to deadlock.
NOTE: The same concept applies to buffered channel, but Sender is not blocked until the buffer is full, which means receiver is not necessarily to be synchronized with every write operation.
So if we have buffered channel of size 1, then your above mentioned code will work
func main(){
ch := make(chan int, 1) /*channel of size 1 */
ch <-10 /* Not blocked: can put the value in channel buffer */
<- ch
}
But if we write more values to above example, then deadlock will happen
func main(){
ch := make(chan int, 1) /*channel Buffer size 1 */
ch <- 10
ch <- 20 /*Blocked: Because Buffer size is already full and no one is waiting to recieve the Data from channel */
<- ch
<- ch
}
In this answer, I will try to explain the error message through which we can peek a little bit into how go works in terms of channels and goroutines
The first example is:
package main
import "fmt"
func main() {
c := make(chan int)
c <- 1
fmt.Println(<-c)
}
The error message is:
fatal error: all goroutines are asleep - deadlock!
In the code, there are NO goroutines at all (BTW this error is in runtime, not compile time). When go runs this line c <- 1, it wants to make sure that the message in the channel will be received somewhere (i.e <-c). Go does NOT know if the channel will be received or not at this point. So go will wait for the running goroutines to finish until either one of the following happens:
all of the goroutines are finished(asleep)
one of the goroutine tries to receive the channel
In case #1, go will error out with the message above, since now go KNOWS that there is no way that a goroutine will receive the channel and it need one.
In case #2, the program will continue, since now go KNOWS that this channel is received. This explain the successful case in OP's example.
Buffering removes synchronization.
Buffering makes them more like Erlang's mailboxes.
Buffered channels can be important for some problems but they are more subtle to reason about
By default channels are unbuffered, meaning that they will only accept sends
(chan <-) if there is a corresponding receive (<- chan) ready to receive the sent value.
Buffered channels accept a limited number of
values without a corresponding receiver for those values.
messages := make(chan string, 2) //-- channel of strings buffering up to 2 values.
Basic sends and receives on channels are blocking.
However, we can use select with a default clause to implement non-blocking sends, receives, and even non-blocking multi-way selects.

Reusing a Go channel causes deadlock

I'm new to golang (whith Java concurrency background). Consider this peace of code :
package main
import "fmt"
func sendenum(num int, c chan int) {
c <- num
}
func main() {
c := make(chan int)
go sendenum(0, c)
x, y := <-c, <-c
fmt.Println(x, y)
}
When I run this code , I get this error
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/home/tarrsalah/src/go/src/github.com/tarrsalah/stackoverflow/chan_dead_lock.go:12 +0x90
exit status 2
I know, adding another go sendenum(0, c) statement fix the issue, ... but
When and Where the deadlock happened ?
After it receives the 0, main keeps on waiting on the receiving end of c for another value to arrive (to put in the y variable), but it never will, as the goroutine running main is the only one left to live.
When you add another go sendenum(0, c), it actually gets a value on the second channel receive, puts it into the y variable, prints x and y out and the program finishes succesfully.
It's not that "reusing" a channel is a problem. It's just a simple deadlock happening because the code prescribes two reads, but only one write to the same channel. The second read can never happen, hence the deadlock.