How to unit test fasthttp-router handler properly? - unit-testing

I am trying to write unit test for my app. I am using fasthttp lib. I also use fasthttp-routing lib. So the problem is that my handler is not standard type of fasthttp.HandlerFunc but routing.Handler.
In order to test HTTP handlers i've written the function that accepts handler fasthttp.RequestHandler parameter. The lib method fasthttp.Serve() accepts handler with type fasthttp.RequestHandler. I use this method to serve incoming connections from the given listener using the given handler. But my handler is of type routing.Handler
My handler:
func deleteExampleBOById(c *routing.Context) error { // Some logic }
My serve() function that i use to serve connections in order to unit test handlers:
func serve(handler fasthttp.RequestHandler, req *http.Request) (*http.Response, error) {
ln := fasthttputil.NewInmemoryListener()
defer ln.Close()
go func() {
err := fasthttp.Serve(ln, handler)
if err != nil {
panic(fmt.Errorf("failed to serve: %v", err))
}
}()
client := http.Client{
Transport: &http.Transport{
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
return ln.Dial()
},
},
}
return client.Do(req)
}
My actual unit test:
func TestHandler(t *testing.T) {
r, err := http.NewRequest("GET", "http://localhost:8181/GoService/example/v1/1", nil)
if err != nil {
t.Error(err)
}
res, err := serve(getExampleBOById, r)
if err != nil {
t.Error(err)
}
body, err := ioutil.ReadAll(res.Body)
if err != nil {
t.Error(err)
}
fmt.Println(string(body))
}
I am not able to serve my handler using function fasthttp.serve(), because of signature differences. I would like to ask any ideas how to convert routung.Handler to fasthttp.HandlerFunc or any other ideas how to unit test my handlers.
I don't have ideas how to solve it

Related

Unit-testing grpc functions in golang

I have created a function that utilizes the grpc package in golang. I don't know if it is relevant but the purpose is the communication with a GoBGP router over grpc. An example is the following function which prints all the peers (neighbors) of the router:
func (gc *Grpc) Peers(conn *grpc.ClientConn) error {
defer conn.Close()
c := pb.NewGobgpApiClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
p := pb.ListPeerRequest{}
peer, err := c.ListPeer(ctx, &p)
if err != nil {
return err
}
for {
res, err := peer.Recv()
if err != nil {
return err
}
fmt.Println(res)
}
return nil
}
Now, I want to create unit tests for the function. To do so, I used google.golang.org/grpc/test/bufconn package, and initialized the following:
type server struct {
pb.UnimplementedGobgpApiServer
}
func (s *server) ListDefinedSet(in *pb.ListDefinedSetRequest, ls pb.GobgpApi_ListDefinedSetServer) error {
return nil
}
var lis *bufconn.Listener
const bufSize = 1024 * 1024
func init() {
lis = bufconn.Listen(bufSize)
s := grpc.NewServer()
pb.RegisterGobgpApiServer(s, &server{})
go func() {
if err := s.Serve(lis); err != nil {
fmt.Println("Server failed!")
}
}()
}
func bufDialer(context.Context, string) (net.Conn, error) {
return lis.Dial()
}
This way, I can run a unit-test creating a connection as follows:
ctx := context.Background()
conn, _ := grpc.DialContext(ctx, "bufnet", grpc.WithContextDialer(bufDialer), grpc.WithInsecure())
Peers(conn)
However, the problem is that the stream seems to be always empty and thus the peer.Recv()
always returns EOF. Is there any way to populate the stream with dummy data? If you have experience, is my methodology correct?

How to write an unit test for a handler that invokes a function that interacts with db in Golang using pgx driver?

I've been trying to write unit tests for my http handler. The code segment is as below:
func (s *Server) handleCreateTicketOption(w http.ResponseWriter, r *http.Request) {
var t ticket.Ticket
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
err = json.Unmarshal(body, &t)
if err != nil {
http.Error(w, er.ErrInvalidData.Error(), http.StatusBadRequest)
return
}
ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
res, err := json.Marshal(ticket)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
log.Printf("%v tickets allocated with name %v\n", t.Allocation, t.Name)
s.sendResponse(w, res, http.StatusOK)
}
Actual logic that interacts with DB. This code segment is invoked by the handler as you can see in the code above. ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
func (t *TicketService) CreateTicketOption(ctx context.Context, ticket ticket.Ticket) (*ticket.Ticket, error) {
tx, err := t.db.dbPool.Begin(ctx)
if err != nil {
return nil, er.ErrInternal
}
defer tx.Rollback(ctx)
var id int
err = tx.QueryRow(ctx, `INSERT INTO ticket (name, description, allocation) VALUES ($1, $2, $3) RETURNING id`, ticket.Name, ticket.Description, ticket.Allocation).Scan(&id)
if err != nil {
return nil, er.ErrInternal
}
ticket.Id = id
return &ticket, tx.Commit(ctx)
}
And that is my unit test for the handler.
func TestCreateTicketOptionHandler(t *testing.T) {
caseExpected, _ := json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 10})
srv := NewServer()
// expected := [][]byte{
// _, _ = json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 20}),
// // json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 20})
// }
tt := []struct {
name string
entry *ticket.Ticket
want []byte
code int
}{
{
"valid",
&ticket.Ticket{Name: "baris", Description: "test-desc", Allocation: 10},
caseExpected,
http.StatusOK,
},
}
var buf bytes.Buffer
for _, tc := range tt {
t.Run(tc.name, func(t *testing.T) {
json.NewEncoder(&buf).Encode(tc.entry)
req, err := http.NewRequest(http.MethodPost, "/ticket_options", &buf)
log.Println("1")
if err != nil {
log.Println("2")
t.Fatalf("could not create request: %v", err)
}
log.Println("3")
rec := httptest.NewRecorder()
log.Println("4")
srv.handleCreateTicketOption(rec, req)
log.Println("5")
if rec.Code != tc.code {
t.Fatalf("got status %d, want %v", rec.Code, tc.code)
}
log.Println("6")
if reflect.DeepEqual(rec.Body.Bytes(), tc.want) {
log.Println("7")
t.Fatalf("NAME:%v, got %v, want %v", tc.name, rec.Body.Bytes(), tc.want)
}
})
}
}
I did research about mocking pgx about most of them were testing the logic part not through the handler. I want to write unit test for both handler and logic itself seperately. However, the unit test I've written for the handler panics as below
github.com/bariis/gowit-case-study/psql.(*TicketService).CreateTicketOption(0xc000061348, {0x1485058, 0xc0000260c0}, {0x0, {0xc000026dd0, 0x5}, {0xc000026dd5, 0x9}, 0xa})
/Users/barisertas/workspace/gowit-case-study/psql/ticket.go:24 +0x125
github.com/bariis/gowit-case-study/http.(*Server).handleCreateTicketOption(0xc000061340, {0x1484bf0, 0xc000153280}, 0xc00018e000)
/Users/barisertas/workspace/gowit-case-study/http/ticket.go:77 +0x10b
github.com/bariis/gowit-case-study/http.TestCreateTicketOptionHandler.func2(0xc000119860)
/Users/barisertas/workspace/gowit-case-study/http/ticket_test.go:80 +0x305
psql/ticket.go:24: tx, err := t.db.dbPool.Begin(ctx)
http/ticket.go:77: ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
http/ticket_test.go:80: srv.handleCreateTicketOption(rec, req)
How can I mock this type of code?
Create an interface which has the required DB functions
Your DB handler implements this interface. You use the handler in actual execution
Create a mock handler using testify/mock and use this in place of DB handler in test cases
From what I can read, you have the following structure:
type Server struct {
TicketService ticket.Service
}
type TicketService struct {
db *sql.Db // ..or similar
}
func (ts *TicketService) CreateTicketOption(...)
The trick to mock this is by ensuring ticket.Service is an interface instead of a struct.
Like this:
type TicketService interface {
CreateTicketOption(ctx context.Context, ticket ticket.Ticket) (*ticket.Ticket, error) {
}
By doing this, your Server expects a TicketService interface.
Then you could do this:
type postgresTicketService struct {
db *sql.Db
}
func (pst *postgresTicketService) CreateTicketOption(...)...
Which means that the postgresTicketService satisfies the requirements to be passed as a ticket.Service to the Server.
This also means that you can do this:
type mockTicketService struct {
}
func (mts *mockTicketService) CreateTicketOption(...)...
This way you decouple the Server from the actual implementation, and you could just init the Server with the mockTicketService when testing and postgresTicketService when deploying.

Golang: mock response testing http client

i'm new to Golang and i'm trying to write a test for a simple HTTP client.
i read a lot of ways of doing so also here in SO but none of them seems to work.
I'm having troubles mocking the client response
This is how my client looks right now:
type API struct {
Client *http.Client
}
func (api *API) MyClient(qp string) ([]byte, error) {
url := fmt.Sprintf("http://localhost:8000/myapi?qp=%s", qp)
resp, err := api.Client.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
// handling error and doing stuff with body that needs to be unit tested
if err != nil {
return nil, err
}
return body, err
}
And this is my test function:
func TestDoStuffWithTestServer(t *testing.T) {
// Start a local HTTP server
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
rw.Write([]byte(`OK`))
}))
defer server.Close()
// Use Client & URL from our local test server
api := API{server.Client()}
body, _ := api.MyClient("1d")
fmt.Println(body)
}
As i said, this is how they look right cause i try lot of ways on doing so.
My problem is that i'm not able to mock the client respose. in this example my body is empty. my understanding was that rw.Write([]byte(OK)) should mock the response 🤔
In the end i solved it like this:
myclient:
type API struct {
Endpoint string
}
func (api *API) MyClient(slot string) ([]byte, error) {
url := fmt.Sprintf("%s/myresource?qp=%s", api.Endpoint, slot)
c := http.Client{}
resp, err := c.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, err
}
test:
func TestDoStuffWithTestServer(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
rw.Write([]byte(`{Result: [{Cluster_name: "cl1", Pings: 2}]}`))
}))
defer server.Close()
api := API{Endpoint: server.URL}
res, _ := api.MyClient("1d")
expected := []byte(`{Result: [{Cluster_name: "cl1", Pings: 2}]}`)
if !bytes.Equal(expected, res) {
t.Errorf("%s != %s", string(res), string(expected))
}
}
still, not 100% sure is the right way of doing so in Go

how can you stub calls to GitHub for testing?

I need to create a Pull Request comment using go-github, and my code works, but now I'd like to write tests for it (yes, I'm aware that tests should come first), so that I don't actually call the real GitHub service during test.
I've read 3 blogs on golang stubbing and mocking, but, being new to golang, I'm a bit lost, despite this discussion on go-github issues. For example, I wrote the following function:
// this is my function
func GetClient(token string, url string) (*github.Client, context.Context, error) {
ctx := context.Background()
ts := oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: token},
)
tc := oauth2.NewClient(ctx, ts)
client, err := github.NewEnterpriseClient(url, url, tc)
if err != nil {
fmt.Printf("error creating github client: %q", err)
return nil, nil, err
}
return client, ctx, nil
}
How could I stub that?
Similarly, I have this:
func GetPRComments(ctx context.Context, client *github.Client) ([]*github.IssueComment, *github.Response, error) {
opts := &github.IssueListCommentsOptions{
ListOptions: github.ListOptions{
Page: 1,
PerPage: 30,
},
}
githubPrNumber, err := strconv.Atoi(os.Getenv("GITHUB_PR_NUMBER"))
if err != nil || githubPrNumber == 0 {
panic("error: GITHUB_PR_NUMBER is not numeric or empty")
}
// use Issues API for PR comments since GitHub docs say "This may seem counterintuitive... but a...Pull Request is just an Issue with code"
comments, response, err := client.Issues.ListComments(
ctx,
os.Getenv("GITHUB_OWNER"),
os.Getenv("GITHUB_REPO"),
githubPrNumber,
opts)
if err != nil {
return nil, nil, err
}
return comments, response, nil
}
How should I stub that?
My thought was to perhaps use dependency injection by creating my own structs first, but I'm not sure how, so currently I have this:
func TestGetClient(t *testing.T) {
client, ctx, err := GetClient(os.Getenv("GITHUB_TOKEN"), "https://example.com/api/v3/")
c, r, err := GetPRComments(ctx, client)
...
}
I would start with an interface:
type ClientProvider interface {
GetClient(token string, url string) (*github.Client, context.Context, error)
}
When testing a unit that needs to call GetClient make sure you depend on your ClientProvider interface:
func YourFunctionThatNeedsAClient(clientProvider ClientProvider) error {
// build you token and url
// get a github client
client, ctx, err := clientProvider.GetClient(token, url)
// do stuff with the client
return nil
}
Now in your test, you can construct a stub like this:
// A mock/stub client provider, set the client func in your test to mock the behavior
type MockClientProvider struct {
GetClientFunc func(string, string) (*github.Client, context.Context, error)
}
// This will establish for the compiler that MockClientProvider can be used as the interface you created
func (provider *MockClientProvider) GetClient(token string, url string) (*github.Client, context.Context, error) {
return provider.GetClientFunc(token, url)
}
// Your unit test
func TestYourFunctionThatNeedsAClient(t *testing.T) {
mockGetClientFunc := func(token string, url string) (*github.Client, context.Context, error) {
// do your setup here
return nil, nil, nil // return something better than this
}
mockClientProvider := &MockClientProvider{GetClientFunc: mockGetClientFunc}
// Run your test
err := YourFunctionThatNeedsAClient(mockClientProvider)
// Assert your result
}
These ideas aren't my own, I borrowed them from those who came before me; Mat Ryer suggested this (and other ideas) in a great video about "idiomatic golang".
If you want to stub the github client itself, a similar approach can be used, if github.Client is a struct, you can shadow it with an interface. If it is already an interface, the above approach works directly.

How to stub a method inside another

I'm writing a web app that will send requests to a third-party service to do some calculations, and send it back to the fronted.
Here are the relevant parts for the test I'm trying to writer.
client.go
func (c *ClientResponse) GetBankAccounts() (*BankAccounts, *RequestError) {
req, _ := http.NewRequest("GET", app.BuildUrl("bank_accounts"), nil)
params := req.URL.Query()
params.Add("view", "standard_bank_accounts")
req.URL.RawQuery = params.Encode()
c.ClientDo(req)
if c.Err.Errors != nil {
return nil, c.Err
}
bankAccounts := new(BankAccounts)
defer c.Response.Body.Close()
if err := json.NewDecoder(c.Response.Body).Decode(bankAccounts); err != nil {
return nil, &RequestError{Errors: &Errors{Error{Message: "failed to decode Bank Account response body"}}}
}
return bankAccounts, nil
}
helper.go
type ClientResponse struct {
Response *http.Response
Err *RequestError
}
type ClientI interface {
ClintDo(req *http.Request) (*http.Response, *RequestError)
}
func (c *ClientResponse) ClientDo(req *http.Request) {
//Do some authentication with third-party service
errResp := *new(RequestError)
client := http.Client{}
resp, err := client.Do(req)
if err != nil {
// Here I'm repourposing the third-party service's error response mapping
errResp.Errors.Error.Message = "internal server error. failed create client.Do"
}
c.Response = resp
c.Err = &errResp
}
I only want to test the GetBankAccounts() method so I want to stub the ClientDo, but I'm at a loss on how to do that. Here's what I have so far with my test case.
client_test.go
type StubClientI interface {
ClintDo(req *http.Request) (*http.Response, *RequestError)
}
type StubClientResponse struct {}
func (c *StubClientResponse) ClientDo(req *http.Request) (*http.Response, *RequestError) {
return nil, nil
}
func TestGetBankAccounts(t *testing.T) {
cr := new(ClientResponse)
accounts, err := cr.GetBankAccounts()
if err != nil {
t.Fatal(err.Errors)
}
t.Log(accounts)
}
The ClintDo still pointing to the actual method on the helper.go, how can I make it use the on in the test?
Update:
I've also tried the following and this doesn't work either, it still sends the request to actual third-party service.
client_test.go
func TestGetBankAccounts(t *testing.T) {
mux := http.NewServeMux()
mux.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
fmt.Fprint(w, toJson(append(BankAccounts{}.BankAccounts, BankAccount{
Url: "https://foo.bar/v2/bank_accounts/1234",
Name: "Test Bank",
})))
}))
server := httptest.NewServer(mux)
cr := new(ClientResponse)
cr.Client = server.Client()
accounts, err := cr.GetBankAccounts()
if err != nil {
t.Fatal(err.Errors)
}
t.Log(accounts)
}
helper.go
type ClientResponse struct {
Client *http.Client
Response *http.Response
Err *RequestError
}
type ClientI interface {
ClintDo(req *http.Request) (*http.Response, *RequestError)
}
func (c *ClientResponse) ClientDo(req *http.Request) {
//Do some authentication with third-party service
errResp := *new(RequestError)
client := c.Client
resp, err := client.Do(req)
if err != nil {
// Here I'm repourposing the third-party service's error response mapping
errResp.Errors.Error.Message = "internal server error. failed create client.Do"
}
c.Response = resp
c.Err = &errResp
}
Update 2
I was able to make some progress from #dm03514 's answer but unfortunately, now I'm getting nil pointer exceptions on the test but not on actual code.
client.go
func (c *ClientResponse) GetBankAccounts() (*BankAccounts, *RequestError) {
req, _ := http.NewRequest("GET", app.BuildUrl("bank_accounts"), nil)
params := req.URL.Query()
params.Add("view", "standard_bank_accounts")
req.URL.RawQuery = params.Encode()
//cr := new(ClientResponse)
c.HTTPDoer.ClientDo(req)
// Panic occurs here
if c.Err.Errors != nil {
return nil, c.Err
}
bankAccounts := new(BankAccounts)
defer c.Response.Body.Close()
if err := json.NewDecoder(c.Response.Body).Decode(bankAccounts); err != nil {
return nil, &RequestError{Errors: &Errors{Error{Message: "failed to decode Bank Account response body"}}}
}
return bankAccounts, nil
}
helper.go
type ClientResponse struct {
Response *http.Response
Err *RequestError
HTTPDoer HTTPDoer
}
type HTTPDoer interface {
//Do(req *http.Request) (*http.Response, *RequestError)
ClientDo(req *http.Request)
}
type ClientI interface {
}
func (c *ClientResponse) ClientDo(req *http.Request) {
// This method hasn't changed
....
}
client_test.go
type StubDoer struct {
*ClientResponse
}
func (s *StubDoer) ClientDo(req *http.Request) {
s.Response = &http.Response{
StatusCode: 200,
Body: nil,
}
s.Err = nil
}
func TestGetBankAccounts(t *testing.T) {
sd := new(StubDoer)
cr := new(ClientResponse)
cr.HTTPDoer = HTTPDoer(sd)
accounts, err := cr.GetBankAccounts()
if err != nil {
t.Fatal(err.Errors)
}
t.Log(accounts)
}
=== RUN TestGetBankAccounts
--- FAIL: TestGetBankAccounts (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12aae69]
There are two common ways to achieve this:
Dependency Injection using interfaces (your example)
Custom http.Transport, which has a hook you can override in your unit tests
It looks like you're close on the interface approach, and are lacking an explicit way to configure the concrete implementation. Consider an interface similiar to your ClientDo:
type HTTPDoer interface {
Do func(req *http.Request) (*http.Response, *RequestError)
}
Dependency injection has the caller configure depedencies and pass them into the resources that actually invoke those dependencies. In this case your ClientResponse struct would have a reference to a HTTPDoer:
type ClientResponse struct {
Response *http.Response
Err *RequestError
HTTPDoer HTTPDoer
}
This allows the caller to configure the concrete implementation that ClientResponse will invoke. In production this will be the actual http.Client but in test it could be anything that implements the Do function.
type StubDoer struct {}
func (s *StubDoer) Do(....)
The unit test could configure the StubDoer, then invoke GetBankAccounts and then make asserstion:
sd := &StubDoer{...}
cr := ClientResponse{
HTTPDoer: sd,
}
accounts, err := cr.GetBankAccounts()
// assertions
The reason it's called Dependency Injection is that the caller initializes the resource (StubDoer) and then provides that resource to the target (ClientResponse). ClientResponse knows nothing about the concrete implementation of HTTPDoer, only that it adheres to the interface!
I wrote a blog post that details dependency injection in the context of unit tests.