I have a program in go that connects to AWS S3 and gets a file.
I'd like to write some tests for it, but I'd like to know, more generally, how to do these mocks in Golang. I know there are some libraries to create mocks but if I remember correctly I read someone suggesting using only standard libraries for unit tests was the best way to go.
So, how would you test a function like this?
func (s S3Input) Sample(key string) ([]byte, error) {
var buf []byte
waBuf := aws.NewWriteAtBuffer(buf)
_, err := s.Downloader.Download(
waBuf,
&s3.GetObjectInput{
Bucket: aws.String(s.Bucket),
Key: aws.String(key),
},
)
if err != nil {
return nil, err
}
return buf, nil
}
Thank you!
One way to do it is to inject the dependencies in your structure, like such:
type S3Inputer interface {
NewWriteAtBuffer(buf []byte) *aws.WriteAtBuffer
String(v string) *string
}
type S3Input struct {
newWriteAtBufferFunc func(buf []byte) *aws.WriteAtBuffer
stringFunc func(v string) *string
}
func (s *S3Input) NewWriteAtBuffer(buf []byte) *WriteAtBuffer {
return s.newWriteAtBufferFunc(buf)
}
func (s *S3Input) String(v string) *string {
return s.stringFunc(v)
}
func (s S3Input) Sample(key string) ([]byte, error) {
var buf []byte
waBuf := s.NewWriteAtBuffer(buf)
_, err := s.Downloader.Download(
waBuf,
&s3.GetObjectInput{
Bucket: s.String(s.Bucket),
Key: s.String(key),
},
)
if err != nil {
return nil, err
}
return buf, nil
}
func main() {
s := &S3Input{
StringFunc: aws.String,
NewWriteAtBufferFunc: aws.NewWriteAtBuffer,
}
// ...
}
This allows you to replace those functions with whatever you want for testing, without the need of any testing framework.
Then, the testing function would look something like this:
func (s S3Input) TestSample(t *testing.T) {
s3Mock := &S3Input{
StringFunc: (func (v string) *string {
return nil
}),
NewWriteAtBufferFunc: (func (buf []byte) *aws.WriteAtBuffer {
return nil
}),
}
res, err := s3Mock.Sample(...) //
// asserts & error checks
}
You could improve it by creating a S3InputMock type instead of reusing the base one, both would implement the S3Inputer interface and your mock could have attributes allowing it to help you with testing. For example, it could count the number of times a function is called, store the arguments it received, have its methods behave differently depending on the attributes you set for easier testing, etc.
Related
I try to test function StartP,
Expect that Start() should be called 1 times, Done() should be called 1 times
but I have trouble that test will block when run this step <-ps.Done()
I expect <-ps.Done() return nil
How can I test function that return chan type?
// production code
func (s *vService) StartP(ctx context.Context, reason string) error {
ps, err := s.factory.CreateVService(ctx)
if err != nil {
return err
}
ps.Start(reason)
err = <-ps.Done() // code stop here to wait ? how can i test ?
if err != nil {
return err
}
return nil
}
// test code
func Test_StartP(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
mockPService := mockpservice.NewMockPInterface(mockCtrl)
vService := &vService {
factory: &servicefactory.FakeServiceFactory{
MockPService: mockPService
}
}
mockPService.EXPECT().Start("reason").Times(1).Return()
mockPService.EXPECT().Done().Times(1).DoAndReturn(func() chan error {
return nil
})
err := vService.StartP(context.Background(), "reason")
assert.Equal(t, nil, err)
}
I use gomock to mock the PServiceInterface
// interface
type PServiceInterface interface {
Start(reason string)
Done() <-chan error
}
gomock gen this function
func (m *MockProvisionServiceInterface) Done() <-chan error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "Done")
ret0, _ := ret[0].(<-chan error)
fmt.Println(ret0,".....mock Done()")
return ret0
}
// I also try this
mockProvisionService.EXPECT().Done().Times(1).DoAndReturn( func() chan error {
fmt.Println("DoAndReturn...err nil")
ch := make(chan error, 1)
ch <- nil
return ch
})
The following shows, I think, the minimum code to implement your test goals.
It does not use any mocking framework because in my experience they tend to obfuscate the test intent, require everybody in the team to learn how to use them and are not needed, at least in Go. One could also wonder what the test is actually testing...
First, let's add some missing production code:
type factoryInterface interface {
CreateVService(ctx context.Context) (PServiceInterface, error)
}
type vService struct {
factory factoryInterface
}
And now the test code, in three parts: the factory, the mock, and the test.
The test factory:
type testFactory struct {
mock PServiceInterface
}
func (f *testFactory) CreateVService(ctx context.Context) (PServiceInterface, error) {
return f.mock, nil
}
The mock:
type ServiceMock struct {
records []string
}
func (sm *ServiceMock) Start(reason string) {
sm.records = append(sm.records, "start")
}
func (sm *ServiceMock) Done() <-chan error {
sm.records = append(sm.records, "done")
ch := make(chan error)
close(ch)
return ch
}
And finally the test:
func TestWithMock(t *testing.T) {
mock := ServiceMock{}
sut := &vService{factory: &testFactory{&mock}}
err := sut.StartP(context.Background(), "banana")
if err != nil {
t.Fatalf("StartP: have: %s; want: no error", err)
}
if have, want := len(mock.records), 2; have != want {
t.Fatalf("number of mock calls: have: %v; want: %v", have, want)
}
if have, want := mock.records[0], "start"; have != want {
t.Fatalf("mock call 1: have: %v; want: %v", have, want)
}
if have, want := mock.records[1], "done"; have != want {
t.Fatalf("mock call 2: have: %v; want: %v", have, want)
}
}
The three assertions on the mock call sequence can be collapsed into one, comparing directly the slice []string{"start", "done"}, if one is using a test library such as the excellent assert package of https://github.com/gotestyourself/gotest.tools
I found the answer, root cause is DoAndReturn something wrong.
Func type should be <-chan error, not chan error
mockProvisionService.EXPECT().Done().Times(1).DoAndReturn( func() <-chan error{
fmt.Println("DoAndReturn...err nil")
ch := make(chan error, 1)
ch <- nil
return ch
})
I've been trying to write unit tests for my http handler. The code segment is as below:
func (s *Server) handleCreateTicketOption(w http.ResponseWriter, r *http.Request) {
var t ticket.Ticket
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
err = json.Unmarshal(body, &t)
if err != nil {
http.Error(w, er.ErrInvalidData.Error(), http.StatusBadRequest)
return
}
ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
res, err := json.Marshal(ticket)
if err != nil {
http.Error(w, er.ErrInternal.Error(), http.StatusInternalServerError)
return
}
log.Printf("%v tickets allocated with name %v\n", t.Allocation, t.Name)
s.sendResponse(w, res, http.StatusOK)
}
Actual logic that interacts with DB. This code segment is invoked by the handler as you can see in the code above. ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
func (t *TicketService) CreateTicketOption(ctx context.Context, ticket ticket.Ticket) (*ticket.Ticket, error) {
tx, err := t.db.dbPool.Begin(ctx)
if err != nil {
return nil, er.ErrInternal
}
defer tx.Rollback(ctx)
var id int
err = tx.QueryRow(ctx, `INSERT INTO ticket (name, description, allocation) VALUES ($1, $2, $3) RETURNING id`, ticket.Name, ticket.Description, ticket.Allocation).Scan(&id)
if err != nil {
return nil, er.ErrInternal
}
ticket.Id = id
return &ticket, tx.Commit(ctx)
}
And that is my unit test for the handler.
func TestCreateTicketOptionHandler(t *testing.T) {
caseExpected, _ := json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 10})
srv := NewServer()
// expected := [][]byte{
// _, _ = json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 20}),
// // json.Marshal(&ticket.Ticket{Id: 1, Name: "baris", Description: "test-desc", Allocation: 20})
// }
tt := []struct {
name string
entry *ticket.Ticket
want []byte
code int
}{
{
"valid",
&ticket.Ticket{Name: "baris", Description: "test-desc", Allocation: 10},
caseExpected,
http.StatusOK,
},
}
var buf bytes.Buffer
for _, tc := range tt {
t.Run(tc.name, func(t *testing.T) {
json.NewEncoder(&buf).Encode(tc.entry)
req, err := http.NewRequest(http.MethodPost, "/ticket_options", &buf)
log.Println("1")
if err != nil {
log.Println("2")
t.Fatalf("could not create request: %v", err)
}
log.Println("3")
rec := httptest.NewRecorder()
log.Println("4")
srv.handleCreateTicketOption(rec, req)
log.Println("5")
if rec.Code != tc.code {
t.Fatalf("got status %d, want %v", rec.Code, tc.code)
}
log.Println("6")
if reflect.DeepEqual(rec.Body.Bytes(), tc.want) {
log.Println("7")
t.Fatalf("NAME:%v, got %v, want %v", tc.name, rec.Body.Bytes(), tc.want)
}
})
}
}
I did research about mocking pgx about most of them were testing the logic part not through the handler. I want to write unit test for both handler and logic itself seperately. However, the unit test I've written for the handler panics as below
github.com/bariis/gowit-case-study/psql.(*TicketService).CreateTicketOption(0xc000061348, {0x1485058, 0xc0000260c0}, {0x0, {0xc000026dd0, 0x5}, {0xc000026dd5, 0x9}, 0xa})
/Users/barisertas/workspace/gowit-case-study/psql/ticket.go:24 +0x125
github.com/bariis/gowit-case-study/http.(*Server).handleCreateTicketOption(0xc000061340, {0x1484bf0, 0xc000153280}, 0xc00018e000)
/Users/barisertas/workspace/gowit-case-study/http/ticket.go:77 +0x10b
github.com/bariis/gowit-case-study/http.TestCreateTicketOptionHandler.func2(0xc000119860)
/Users/barisertas/workspace/gowit-case-study/http/ticket_test.go:80 +0x305
psql/ticket.go:24: tx, err := t.db.dbPool.Begin(ctx)
http/ticket.go:77: ticket, err := s.TicketService.CreateTicketOption(r.Context(), t)
http/ticket_test.go:80: srv.handleCreateTicketOption(rec, req)
How can I mock this type of code?
Create an interface which has the required DB functions
Your DB handler implements this interface. You use the handler in actual execution
Create a mock handler using testify/mock and use this in place of DB handler in test cases
From what I can read, you have the following structure:
type Server struct {
TicketService ticket.Service
}
type TicketService struct {
db *sql.Db // ..or similar
}
func (ts *TicketService) CreateTicketOption(...)
The trick to mock this is by ensuring ticket.Service is an interface instead of a struct.
Like this:
type TicketService interface {
CreateTicketOption(ctx context.Context, ticket ticket.Ticket) (*ticket.Ticket, error) {
}
By doing this, your Server expects a TicketService interface.
Then you could do this:
type postgresTicketService struct {
db *sql.Db
}
func (pst *postgresTicketService) CreateTicketOption(...)...
Which means that the postgresTicketService satisfies the requirements to be passed as a ticket.Service to the Server.
This also means that you can do this:
type mockTicketService struct {
}
func (mts *mockTicketService) CreateTicketOption(...)...
This way you decouple the Server from the actual implementation, and you could just init the Server with the mockTicketService when testing and postgresTicketService when deploying.
How should I unittest following piece of code. I was trying to use coutnerfiter to fake input "*s3.S3" object, but it's not working for me. I am new to coutnerfiter and Go, Can someone please help me on that.
func (l *listContentImp) ListS3Content(client *s3.S3) (bool, error) {
listObj := &s3.ListObjectsV2Input{
Bucket: aws.String(l.bucket),
}
var err error
l.lObj, err = client.ListObjectsV2(listObj)
if err != nil {
return false, err
}
return true, nil
}
You shouldn't pass a reference to the s3.S3 struct. When using the AWS SDK for Go v1 you typically pass the services corresponding interface. For S3 this is s3iface.
The signature of your function would look like this:
func (l *listContentImp) ListS3Content(client s3iface.S3API) (bool, error)
Now every struct that you pass that implements at least one of the methods of s3iface.S3API will work.
At runtime you'll pass the proper service client, but in the unit tests you can just pass a mock:
type mock struct {
s3iface.S3API
output *s3.ListObjectsV2Output
err error
}
func (m mock) ListObjectsV2(*s3.ListObjectsV2Input) (*s3.ListObjectsV2Output, error) {
return m.output, m.err
}
In your test you create the mock and pass it to your function:
func Test_ListObject(t *testing.T) {
l := &listContentImp{...}
m := mock{
output: &s3.ListObjectsV2Output{...},
err: nil
}
result, err := l.ListS3Content(m)
[... add checks here...]
}
I need to create a Pull Request comment using go-github, and my code works, but now I'd like to write tests for it (yes, I'm aware that tests should come first), so that I don't actually call the real GitHub service during test.
I've read 3 blogs on golang stubbing and mocking, but, being new to golang, I'm a bit lost, despite this discussion on go-github issues. For example, I wrote the following function:
// this is my function
func GetClient(token string, url string) (*github.Client, context.Context, error) {
ctx := context.Background()
ts := oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: token},
)
tc := oauth2.NewClient(ctx, ts)
client, err := github.NewEnterpriseClient(url, url, tc)
if err != nil {
fmt.Printf("error creating github client: %q", err)
return nil, nil, err
}
return client, ctx, nil
}
How could I stub that?
Similarly, I have this:
func GetPRComments(ctx context.Context, client *github.Client) ([]*github.IssueComment, *github.Response, error) {
opts := &github.IssueListCommentsOptions{
ListOptions: github.ListOptions{
Page: 1,
PerPage: 30,
},
}
githubPrNumber, err := strconv.Atoi(os.Getenv("GITHUB_PR_NUMBER"))
if err != nil || githubPrNumber == 0 {
panic("error: GITHUB_PR_NUMBER is not numeric or empty")
}
// use Issues API for PR comments since GitHub docs say "This may seem counterintuitive... but a...Pull Request is just an Issue with code"
comments, response, err := client.Issues.ListComments(
ctx,
os.Getenv("GITHUB_OWNER"),
os.Getenv("GITHUB_REPO"),
githubPrNumber,
opts)
if err != nil {
return nil, nil, err
}
return comments, response, nil
}
How should I stub that?
My thought was to perhaps use dependency injection by creating my own structs first, but I'm not sure how, so currently I have this:
func TestGetClient(t *testing.T) {
client, ctx, err := GetClient(os.Getenv("GITHUB_TOKEN"), "https://example.com/api/v3/")
c, r, err := GetPRComments(ctx, client)
...
}
I would start with an interface:
type ClientProvider interface {
GetClient(token string, url string) (*github.Client, context.Context, error)
}
When testing a unit that needs to call GetClient make sure you depend on your ClientProvider interface:
func YourFunctionThatNeedsAClient(clientProvider ClientProvider) error {
// build you token and url
// get a github client
client, ctx, err := clientProvider.GetClient(token, url)
// do stuff with the client
return nil
}
Now in your test, you can construct a stub like this:
// A mock/stub client provider, set the client func in your test to mock the behavior
type MockClientProvider struct {
GetClientFunc func(string, string) (*github.Client, context.Context, error)
}
// This will establish for the compiler that MockClientProvider can be used as the interface you created
func (provider *MockClientProvider) GetClient(token string, url string) (*github.Client, context.Context, error) {
return provider.GetClientFunc(token, url)
}
// Your unit test
func TestYourFunctionThatNeedsAClient(t *testing.T) {
mockGetClientFunc := func(token string, url string) (*github.Client, context.Context, error) {
// do your setup here
return nil, nil, nil // return something better than this
}
mockClientProvider := &MockClientProvider{GetClientFunc: mockGetClientFunc}
// Run your test
err := YourFunctionThatNeedsAClient(mockClientProvider)
// Assert your result
}
These ideas aren't my own, I borrowed them from those who came before me; Mat Ryer suggested this (and other ideas) in a great video about "idiomatic golang".
If you want to stub the github client itself, a similar approach can be used, if github.Client is a struct, you can shadow it with an interface. If it is already an interface, the above approach works directly.
I use the following code which works ok.
This is working example
https://play.golang.org/p/wjvJtDNvJAQ
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type requester interface {
HTTPRequest(c string, i string, mtd string, url string) (p []byte, e error)
}
type impl struct {
client *http.Client
}
// ----This is the function which I need to mock
func (s *ServiceInfo) wrapperFN() {
// Function 1 - get the values
v1, v2 := s.json.parseJson()
// call to http function
s.req.HTTPRequest(v1, v2, "POST", "http://www.mocky.io/v2/5c20eccc2e00005c001e0c84")
}
func (i impl) HTTPRequest(c string, ci string, mtd string, url string) (p []byte, e error) {
req, err := http.NewRequest(mtd, url, nil)
if err != nil {
return nil, err
}
req.SetBasicAuth(c, ci)
res, err := i.client.Do(req)
if err != nil {
return nil, err
}
token, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
defer res.Body.Close()
fmt.Println("success")
return token, nil
}
type parser interface {
parseJson() (string, string)
}
type jsonP struct {
data string
}
func (s jsonP) parseJson() (string, string) {
var result map[string]interface{}
json.Unmarshal([]byte(s.data), &result)
b := result["person"].(map[string]interface{})
for key, value := range b {
return key, value.(string)
}
return "", ""
}
type ServiceInfo struct {
req requester
json parser
}
// When in production pass in concrete implementations.
func NewServiceInfo(http requester, json parser) *ServiceInfo {
return &ServiceInfo{
req: http,
json: json,
}
}
func main() {
httpClient := http.Client{}
js := `{"person":{"p1":"username","p2":"password"},"customers":"10"}`
j := jsonP{data: js}
s := NewServiceInfo(impl{client: &httpClient}, j)
s.wrapperFN()
}
Now i want to test it wrapperFN , what I try I've changed the code to use interface , which works.
This is just example to give a point ( the real code much more complicated)
The problem that I dont understand how to mock function inside wrapperFN like parseJson() , in the real world warpperFN contains several function which I need to mock ,because just calling them in the test will provide error.
How it's best to mock function like parseJson() & HTTPRequest? and assume that inside wrapperFN there is additional functions which is not related...
I need to know if this is the best practice for testing function.
This is the test (which im not sure how to make it right)
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestServiceInfo_wrapperFN(t *testing.T) {
tests := []struct {
name string
s *ServiceInfo
}{
{
name: "wrapper test",
s: &ServiceInfo{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var testHandler http.Handler
srv := httptest.NewServer(testHandler)
defer srv.Close()
iReq := &impl{
client: srv.Client(),
}
v := &ServiceInfo{http: *iReq}
v.wrapperFN()
})
}
}