GoLang FanIn function not working - concurrency

Question

So I'm trying to write a webcrawler using Rob Pike's fanin function.
This is my code -
package main
import (
"net/http"
"encoding/json"
"fmt"
"io/ioutil"
)
func main() {
fanIn(getDuckDuckGo("food"), getGitHub("defunkt"))
}
type DuckDuckGoResponse struct {
RelatedTopics []struct {
Result string `json:"Result"`
FirstUrl string `json:"FirstURL"`
Text string `json:"Text"`
} `json:"RelatedTopics"`
}
type GitHubResponse struct {
Login string `json:"login"`
Email string `json:"email"`
Name string `json:"name"`
}
func fanIn(input1 <-chan DuckDuckGoResponse, input2 <-chan GitHubResponse) <-chan string {
c := make(chan string)
go func() {
for {
select {
case s := <-input1:
fmt.Println(s)
case s := <-input2:
fmt.Println(s)
}
}
}()
return c
}
func getDuckDuckGo(k string) <-chan DuckDuckGoResponse {
resp, err := http.Get("http://api.duckduckgo.com/?q=" + k + "&format=json&pretty=1")
if err != nil {
return nil
}
c := make(chan DuckDuckGoResponse)
var duckDuckParsed DuckDuckGoResponse
jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)
if jsonErr != nil {
fmt.Println("Json error!")
}
defer resp.Body.Close()
if err:= json.Unmarshal(jsonDataFromHttp, &duckDuckParsed); err != nil {
panic(err)
}
return c
}
func getGitHub(k string) <-chan GitHubResponse {
resp, err := http.Get("https://api.github.com/users/?q=" + k)
if err != nil {
return nil
}
c := make(chan GitHubResponse)
var githubParsed GitHubResponse
jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)
if jsonErr != nil {
fmt.Println("Json error!")
}
defer resp.Body.Close()
if err:= json.Unmarshal(jsonDataFromHttp, &githubParsed); err != nil {
panic(err)
}
return c
}
I run this program, and nothing prints.
Why?
Thanks

Answer 1

At first glance, the fanIn function returns a channel that is not being read from in your main loop. So yes, you are invoking the fanIn function which returns a channel, but there is nothing reading off of that channel. For a channel to be useful there needs to be a consumer consuming from the channel while on the other end there needs to be a producer producing on that channel. In other words, sending on a channel can't make progress unless someone on the other end is ready to receive on it.
Next, your getGitHub and getDuckDuckGo return channels, but they don't actually send anything on those channels that they return. Also, what you really need is a way to invoke those functions, have them return a channel and still execute your work. You need to use additional goroutines in order be able to have the http.Get calls do their work.
Lastly, your fanIn function also creates a channel and returns it, however it doesn't actually "fan-in" the results from input1 or input2. Since the fanIn returns a channel of type string, you'll need to write a string into them which could be a field off of DuckDuckGoResponse and GitHubResponse.
I urge you too look at this streamlined example of what you are trying to accomplish: https://talks.golang.org/2012/go-docs/faninboring.go
Last observation you are checking that jsonErr != nil and printing it out but you probably want to return nil as well to prevent the code from continuing on.
I hope this gives you just enough insight to get your code working. Good luck!