How to force error on reading response body - unit-testing

I've written http client wrapper in go and I need to test it thoroughly.
I'm reading the response body with ioutil.ReadAll within the wrapper. I'm having a bit of trouble figuring out how I can force a read from the response body to fail with the help of httptest.
package req
func GetContent(url string) ([]byte, error) {
response, err := httpClient.Get(url)
// some header validation goes here
body, err := ioutil.ReadAll(response.Body)
defer response.Body.Close()
if err != nil {
errStr := fmt.Sprintf("Unable to read from body %s", err)
return nil, errors.New(errStr)
}
return body, nil
}
I'm assuming I can set up a fake server as such:
package req_test
func Test_GetContent_RequestBodyReadError(t *testing.T) {
handler := func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}
ts := httptest.NewServer(http.HandlerFunc(handler))
defer ts.Close()
_, err := GetContent(ts.URL)
if err != nil {
t.Log("Body read failed as expected.")
} else {
t.Fatalf("Method did not fail as expected")
}
}
I'm assuming I need to modify the ResposeWriter. Now, is there any way for me to modify the responseWriter and thereby force the ioutil.ReadAll in the wrapper to fail?
I realize that you seem to think it's a duplicate of this post and while you may believe so or it might be, just marking it as a duplicate doesn't really help me. The code provided in the answers in the "duplicate" post makes very little sense to me in this context.

Check the documentation of Response.Body to see when reading from it might return an error:
// Body represents the response body.
//
// The response body is streamed on demand as the Body field
// is read. If the network connection fails or the server
// terminates the response, Body.Read calls return an error.
//
// The http Client and Transport guarantee that Body is always
// non-nil, even on responses without a body or responses with
// a zero-length body. It is the caller's responsibility to
// close Body. The default HTTP client's Transport may not
// reuse HTTP/1.x "keep-alive" TCP connections if the Body is
// not read to completion and closed.
//
// The Body is automatically dechunked if the server replied
// with a "chunked" Transfer-Encoding.
Body io.ReadCloser
The easiest way is to generate an invalid HTTP response from the test handler.
How to do that? There are many ways, a simple one is to "lie" about the content length:
handler := func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "1")
}
This handler tells it has 1 byte body, but actually it sends none. So at the other end (the client) when attempting to read 1 byte from it, obviously that won't succeed, and will result in the following error:
Unable to read from body unexpected EOF
See related question if you would need to simulate error reading from a request body (not from a response body): How do I test an error on reading from a request body?

To expand upon icza's awesome answer, you can also do this with an httptest.Server object:
bodyErrorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "1")
}))
defer bodyErrorServer.Close()
You can then pass bodyErrorServer.URL in your tests like normal, and you'll always get an EOF error:
package main
import (
"bytes"
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func getBodyFromURL(service string, clientTimeout int) (string, error) {
var netClient = &http.Client{
Timeout: time.Duration(clientTimeout) * time.Millisecond,
}
rsp, err := netClient.Get(service)
if err != nil {
return "", err
}
defer rsp.Body.Close()
if rsp.StatusCode != 200 {
return "", fmt.Errorf("HTTP request error. Response code: %d", rsp.StatusCode)
}
buf, err := ioutil.ReadAll(rsp.Body)
if err != nil {
return "", err
}
return string(bytes.TrimSpace(buf)), nil
}
func TestBodyError(t *testing.T) {
bodyErrorServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "1")
}))
_, err := getBodyFromURL(bodyErrorServer.URL, 1000)
if err.Error() != "unexpected EOF" {
t.Error("GOT AN ERROR")
} else if err == nil {
t.Error("GOT NO ERROR, THATS WRONG!")
} else {
t.Log("Got an unexpected EOF as expected, horray!")
}
}
Playground example here: https://play.golang.org/p/JzPmatibgZn

Related

how can you stub calls to GitHub for testing?

I need to create a Pull Request comment using go-github, and my code works, but now I'd like to write tests for it (yes, I'm aware that tests should come first), so that I don't actually call the real GitHub service during test.
I've read 3 blogs on golang stubbing and mocking, but, being new to golang, I'm a bit lost, despite this discussion on go-github issues. For example, I wrote the following function:
// this is my function
func GetClient(token string, url string) (*github.Client, context.Context, error) {
ctx := context.Background()
ts := oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: token},
)
tc := oauth2.NewClient(ctx, ts)
client, err := github.NewEnterpriseClient(url, url, tc)
if err != nil {
fmt.Printf("error creating github client: %q", err)
return nil, nil, err
}
return client, ctx, nil
}
How could I stub that?
Similarly, I have this:
func GetPRComments(ctx context.Context, client *github.Client) ([]*github.IssueComment, *github.Response, error) {
opts := &github.IssueListCommentsOptions{
ListOptions: github.ListOptions{
Page: 1,
PerPage: 30,
},
}
githubPrNumber, err := strconv.Atoi(os.Getenv("GITHUB_PR_NUMBER"))
if err != nil || githubPrNumber == 0 {
panic("error: GITHUB_PR_NUMBER is not numeric or empty")
}
// use Issues API for PR comments since GitHub docs say "This may seem counterintuitive... but a...Pull Request is just an Issue with code"
comments, response, err := client.Issues.ListComments(
ctx,
os.Getenv("GITHUB_OWNER"),
os.Getenv("GITHUB_REPO"),
githubPrNumber,
opts)
if err != nil {
return nil, nil, err
}
return comments, response, nil
}
How should I stub that?
My thought was to perhaps use dependency injection by creating my own structs first, but I'm not sure how, so currently I have this:
func TestGetClient(t *testing.T) {
client, ctx, err := GetClient(os.Getenv("GITHUB_TOKEN"), "https://example.com/api/v3/")
c, r, err := GetPRComments(ctx, client)
...
}
I would start with an interface:
type ClientProvider interface {
GetClient(token string, url string) (*github.Client, context.Context, error)
}
When testing a unit that needs to call GetClient make sure you depend on your ClientProvider interface:
func YourFunctionThatNeedsAClient(clientProvider ClientProvider) error {
// build you token and url
// get a github client
client, ctx, err := clientProvider.GetClient(token, url)
// do stuff with the client
return nil
}
Now in your test, you can construct a stub like this:
// A mock/stub client provider, set the client func in your test to mock the behavior
type MockClientProvider struct {
GetClientFunc func(string, string) (*github.Client, context.Context, error)
}
// This will establish for the compiler that MockClientProvider can be used as the interface you created
func (provider *MockClientProvider) GetClient(token string, url string) (*github.Client, context.Context, error) {
return provider.GetClientFunc(token, url)
}
// Your unit test
func TestYourFunctionThatNeedsAClient(t *testing.T) {
mockGetClientFunc := func(token string, url string) (*github.Client, context.Context, error) {
// do your setup here
return nil, nil, nil // return something better than this
}
mockClientProvider := &MockClientProvider{GetClientFunc: mockGetClientFunc}
// Run your test
err := YourFunctionThatNeedsAClient(mockClientProvider)
// Assert your result
}
These ideas aren't my own, I borrowed them from those who came before me; Mat Ryer suggested this (and other ideas) in a great video about "idiomatic golang".
If you want to stub the github client itself, a similar approach can be used, if github.Client is a struct, you can shadow it with an interface. If it is already an interface, the above approach works directly.

Trigger http client error for testing with httptest

I have a simple function which takes a URL and fetches the response:
func getUrl(url string) (string, error) {
var theClient = &http.Client{Timeout: 12 * time.Second}
resp, err := theClient.Get(url)
if err != nil {
return "", err
}
defer r.Body.Close()
body, readErr := ioutil.ReadAll(resp.Body)
if readErr != nil {
return "", readErr
}
return string(body), nil
}
Now, I want to trigger an error on the theClient.Get(url) line but I don't know how to. I can trigger an error on the ReadAll() line, by returning no response but with content-length:2.
How can I trigger an error on the theClient.Get(url) line for my unit test?
func TestGetUrl(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "2")
}))
defer server.Close()
gotContent, gotErr := getUrl(server.URL)
wantErr := "unexpected EOF"
if gotErr == nil || gotErr.Error() != wantErr {
t.Errorf("got err %v; wanted %s", gotErr, wantErr)
}
}
Easiest way is to simply pass an invalid URL:
_, err := http.Get("clearly not a valid url")
fmt.Println("Got error:", err != nil) // Got error: true
Another option is to make it timeout by sleeping in your httptest.Server handler, but that doesn't seem like a very nice idea (but you will be able to assert that it was called in the first place).

how to inject an url to httptest.server in golang?

For sentence
resp, err := client.Get(fmt.Sprintf("https://www.xxxxx/day?time=%s", time))
If I want to mock a response to this client.Get() in unit test, I should use httptest.server, but how can I bind the url (https://www.xxxxx/day?time=%s) to the url of httptest.server? so that when I call client.Get() it can return the response I set before.
For some reason I cannot mock a client here.
You don't, usually. You take the base URL from the server and give it to the client:
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestClient(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Verify request, send mock response, etc.
}))
defer server.Close()
var client *http.Client
var time time.Time
baseURL := server.URL // Something like "http://127.0.0.1:53791"
resp, err := client.Get(fmt.Sprintf(baseURL+"/day?time=%s", time))
if err != nil {
t.Fatal(err)
}
// Verify response body if applicable
resp.Body.Close()
}
Like this
func NewTestServerWithURL(URL string, handler http.Handler) (*httptest.Server, error) {
ts := httptest.NewUnstartedServer(handler)
if URL != "" {
l, err := net.Listen("tcp", URL)
if err != nil {
return nil, err
}
ts.Listener.Close()
ts.Listener = l
}
ts.Start()
return ts, nil
}
The http.Client is a struct not an interface which makes mocking it difficult as you have seen. An alternative way of mocking it is passing in the external dependencies that a routine needs, so instead of directly using client.Get, you use clientGet - which is a function pointer that was handed into the routine.
From the unit test you can then create :
mockClientGet(c *http.client, url string) (resp *http.Response, err error) {
// add the test code to return what you want it to.
}
Then in your main code use:
resp, err := clientGet(client, fmt.Sprintf("https://www.xxxxx/day?time=%s", time))
When calling the procedure normally, use the function pointer to http.Client.Get, and for your test pass in a pointer to your mock. It's not ideal, but I've not seen a nicer way around mocking non-interface external calls - and given its an external dependency, injecting it from the outside is not a bad thing.

How do I test an error on reading from a request body?

I'm writing unit tests for http Handlers in golang. When looking at code coverage reports of this I am running into the following issue: When reading the request body from a request, ioutil.ReadAll might return an error that I need to handle. Yet, when I write unit tests for my handler I do not know how to send a request to my handler in a way that it will trigger such an error (premature end of content seems not to generate such an error but will generate an error on unmarshaling the body). This is what I am trying to do:
package demo
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/httptest"
"testing"
)
func HandlePostRequest(w http.ResponseWriter, r *http.Request) {
body, bytesErr := ioutil.ReadAll(r.Body)
if bytesErr != nil {
// intricate logic goes here, how can i test it?
http.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest)
return
}
defer r.Body.Close()
// continue...
}
func TestHandlePostRequest(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(HandlePostRequest))
data, _ := ioutil.ReadFile("testdata/fixture.json")
res, err := http.Post(ts.URL, "application/json", bytes.NewReader(data))
// continue...
}
How can I write a test case for HandlePostRequest that also covers the case of bytesErr not being nil?
You may create and use an http.Request forged by you, which deliberately returns an error when reading its body. You don't necessarily need a whole new request, a faulty body is enough (which is an io.ReadCloser).
Simplest achieved by using the httptest.NewRequest() function where you can pass an io.Reader value which will be used (wrapped to be an io.ReadCloser) as the request body.
Here's an example io.Reader which deliberately returns an error when attempting to read from it:
type errReader int
func (errReader) Read(p []byte) (n int, err error) {
return 0, errors.New("test error")
}
Example that will cover your error case:
func HandlePostRequest(w http.ResponseWriter, r *http.Request) {
defer r.Body.Close()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Printf("Error reading the body: %v\n", err)
return
}
fmt.Printf("No error, body: %s\n", body)
}
func main() {
testRequest := httptest.NewRequest(http.MethodPost, "/something", errReader(0))
HandlePostRequest(nil, testRequest)
}
Output (try it on the Go Playground):
Error reading the body: test error
See related question if you would need to simulate error reading from a response body (not from a request body): How to force error on reading response body

How do you prevent output from being generated in Go until all error checks have completed?

Probably a simple question, but I am having issues delaying the output in a request handler function. I'm using "bufio" to write to when I execute my templates instead of the response writer, but it seems like the buffer can only hold so much before it spits things out. I'm concerned that it will spit out part of the page, then encounter an error, leaving an incomplete and unintelligible response. What is the best strategy for ensuring that everything stays buffered until it's ready to be released into the wild?
If you want to completely buffer the output use bytes.Buffer instead, example:
var bufferPool = &sync.Pool{
New: func() interface{} {
return &bytes.Buffer{}
},
}
func getBuffer() (buf *bytes.Buffer) {
return bufferPool.Get().(*bytes.Buffer)
}
func putBuffer(buf *bytes.Buffer) {
buf.Reset()
bufferPool.Put(buf)
}
func handler(w http.ResponseWriter, req *http.Request) {
buf := getBuffer()
defer putBuffer(buf)
//....
fmt.Fprintf(buf, .....)
buf.WriteTo(w)
}
You could just use a bytes.Buffer
func Handler(w http.ResponseWriter, req *http.Request) {
tmp, err := template.New("hello").Parse("Hello {{.World}}")
if err != nil {
http.Error(w, "Error", 500)
return
}
buf := &bytes.Buffer{}
if err := tmp.Execute(buf, map[string]string{"World": "World"}); err != nil {
http.Error(w, "Error", 500)
return
}
w.Write(buf.Bytes())
}