I am trying to unit test a function which uses https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.ListObjectsV2Pages to list contents of a given s3 bucket.
In my main.go file I have this function:
func listS3Objects(s3Bucket string, s3Prefix string, svc s3iface.S3API) ([]string, error) {
input := &s3.ListObjectsV2Input{
Bucket: aws.String(s3Bucket),
Prefix: aws.String(s3Prefix),
}
var s3Items []string
err := svc.ListObjectsV2Pages(input, func(resp *s3.ListObjectsV2Output, lastPage bool) bool {
for _, s3Item := range resp.Contents {
s3Items = append(s3Items, *s3Item.Key)
}
return true
})
if err != nil {
if awsErr, ok := err.(awserr.Error); ok {
return nil, awsErr
}
return nil, err
}
return s3Items, nil
}
In my main_test.go file I have:
type mockS3Client struct {
s3iface.S3API
}
func (m *mockS3Client) ListObjectsV2Pages(input *s3.ListObjectsV2Input, fn func(*s3.ListObjectsV2Output, bool) bool) error {
var s3Output s3.ListObjectsV2Output
var s3Object s3.Object
var s3Objects []*s3.Object
s3Object.SetKey(*input.Bucket)
s3Objects = append(s3Objects, &s3Object)
s3Output.Contents = s3Objects
fn(&s3Output, true)
return nil
}
The test looks like below:
func TestListS3Objects(t *testing.T) {
testBucket := "testBucket"
testPrefix := "testPrefix"
var mockSvc mockS3Client
s3Items, _ := listS3Objects(testBucket, testPrefix, mockSvc.S3API)
if len(s3Items) != 1 {
t.Errorf("Expected '%v' elements but got '%v'", 1, len(s3Items))
}
if s3Items[0] != testBucket {
t.Errorf("Expected '%v' value but got '%v'", testBucket, s3Items[0])
}
}
This test generates below stack trace:
--- FAIL: TestListS3Objects (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x5a0 pc=0x797066]
goroutine 7 [running]:
testing.tRunner.func1(0xc00017e300)
/usr/local/go/src/testing/testing.go:874 +0x3a3
panic(0x800c20, 0xc657c0)
/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/AirHelp/business-metrics-restore.listS3Objects(0x88d6aa, 0xa, 0x88d6b4, 0xa, 0x0, 0x0, 0x1, 0xc000018dd8, 0x44f278, 0x5aa38b1b10f7, ...)
/workspaces/business-metrics-restore/bmr.go:189 +0x126
github.com/AirHelp/business-metrics-restore.TestListS3Objects(0xc00017e300)
/workspaces/business-metrics-restore/bmr_test.go:59 +0x65
testing.tRunner(0xc00017e300, 0x8a7fa8)
/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:960 +0x350
FAIL github.com/AirHelp/business-metrics-restore 0.044s
FAIL
Error: Tests failed.
I am under the impression there may be some problem with my mocked version of ListObjectsV2Pages but am not able to find the root cause. I tried to keep this mocked function as simple as possible...
Can someone help me?
Thanks!
Best Regards,
Rafal.
how i did it was i embedded s3iface.S3API in a new struct in my application code, and added a slice to save s3items, then provided a separate paginator function
type myS3 struct {
s3iface.S3API
s3items []*s3Item.Key
}
func (s *myS3) findObjects(bucket, prefix string) error {
err := s.ListObjectsV2Pages(&s3.ListObjectsV2Input{
Bucket: &bucket,
Prefix: &prefix,
}, s.Paginator)
return err
}
func (s *myS3) paginator(page *s3.ListObjectsV2Output, lastPage bool) bool {
for _, content := range page.Contents {
s.s3items = append(s.s3items, *content.Key)
}
return lastPage
}
this way, you will only need to test if paginator is filling the s3items field correctly, and to see if whatever function is calling findObjects is handling errors correctly
Related
I am using juniper's netconf package ("github.com/Juniper/go-netconf/netconf") to establish a netconf session in my code.
I wanted to know how can I mock a netconf session in my unit tests.
My methods are:
func TestMyFunction(t *testing.T) {
getSSHConnection = mockGetSSHConnection
got := MyFunction()
want := 123
if !reflect.DeepEqual(got, want) {
t.Errorf("Error expectation not met, want %v, got %v", want, got)
}
}
func mockGetSSHConnection() (*netconf.Session, error) {
var sess netconf.Session
sess.SessionID = 123
return &sess, nil
}
The problem arises when MyFunction() has a line that defers sess.Close() and it's throwing error due to nil pointer dereference
func MyFunction() int {
sess, err := getSSHConnection() // returns (*netconf.Session, error)
if err == nil && sess != nil {
defer sess.Close() -> Problem happens here
// Calls RPC here and rest of the code here
}
return 0
}
So, what changes can I make on mockGetSSHConnection() method so that sess.Close() won't throw error?
The nil pointer error originates within the Close function when Close is called on the underlying Transport. Fortunately Transport is an interface type that you can easily mock and use in an actual instance of the netconf.Session. For example like so:
type MockTransport struct{}
func (t *MockTransport) Send([]byte) error {
return nil
}
func (t *MockTransport) Receive() ([]byte, error) {
return []byte{}, nil
}
func (t *MockTransport) Close() error {
return nil
}
func (t *MockTransport) ReceiveHello() (*netconf.HelloMessage, error) {
return &netconf.HelloMessage{SessionID: 123}, nil
}
func (t *MockTransport) SendHello(*netconf.HelloMessage) error {
return nil
}
func (t *MockTransport) SetVersion(version string) {
}
func mockGetSSHConnection() (*netconf.Session, error) {
t := MockTransport{}
sess := netconf.NewSession(&t)
return sess, nil
}
Note that the function you want to test currently return 0 and not the SessionID of the session. So you should fix that before the test is successful.
You could use OOP and "github.com/stretchr/testify/mock" package
for example create
type SshClientMock struct {
mock.Mock
}
func (s *SshClientMock) GetSSHConnection() {
return //what do you need
}
in your unit test:
sshClient := SshClientMock
sshClient.On("GetSSHConnection").Return(what do you need)
and then call your method
I use Testify to create a unit test for my golang app. I need to create a unit test for this function where it calls a variadic function (function with trailing arguments). I encountered an error when I test it. I'm actually not sure if the error is because of the trailing argument itself or not, but I feel like there's something wrong with the mock.
// svc/callThisFunction.go
// data type of args is []sqkit.SelectOption
func CallThisFunction(ctx context.Context, args ...sqkit.SelectFunctiom) (result string, err error) {
return result, nil
}
// svc/functionToTest.go
// This is the function that I wanna test
func FunctionToTest(ctx context.Context, id int64) (result string, err error) {
args := []sqkit.SelectOption{
sqkit.Where{
fmt.Sprintf("id = %d", id),
},
}
newResult, err := callThisFunctionService.CallThisFunction(ctx, args)
if err != nil {
return newResult, err
}
return newResult, nil
}
// svc/functionToTest_test.go
func Test_FunctionToTest(t *testing.T) {
testCase := []struct {
name string
id int64
onCallThisFunctionMock func(callThisFunctionSvc *mocks.CallThisFunctionSvc)
expectedResult string
wantError bool
expectedError error
}{
{
name: "Success",
id: 1,
onCallThisFunctionMock: func(callThisFunctionSvc *mocks.CallThisFunctionSvc) {
// NOTE: I've created 2 different versions (used separately, not at the same), using mock.Anything() and using actual arguments
// Both of these give the same errors
// Using actual arguments
args := []sqkit.SelectOption{
sqkit.Where{
fmt.Sprintf("id = %d", 1},
},
}
callThisFunctionSvc.On("CallThisFunction", context.Background(), args).Return("Success", nil)
// Using mock.Anything
callThisFunctionSvc.On("CallThisFunction", context.Background(), mock.Anything).Return("Success", nil)
}
}
}
for _, tc := range testCases {
var callThisFunctionSvc = new(mocks.CallThisFunctionSvc)
tc.onCallThisFunctionMock(callThisFunctionSvc)
svc := &svc.FunctionToTest{
CallThisFunction: callThisFunctionSvc,
}
actualResult, actualError := svc.FunctionToTest(context.Background(), tc.id)
if tc.wantEror {
require.Error(t, actualError, tc.expectedError)
} else {
require.NoError(t, actualError)
}
require.Equal(t, tc.expectedResult, actualResult)
}
}
This is the error it gives
=== RUN Test_GenerateDocument
--- FAIL: Test_GenerateDocument (0.00s)
panic:
assert: mock: I don't know what to return because the method call was unexpected.
Either do Mock.On("CallThisFunction").Return(...) first, or remove the GetTemplates() call.
This method was unexpected:
CallThisFunction(*context.emptyCtx,sqkit.Where)
0: (*context.emptyCtx)(0xc0000a4010)
1: sqkit.Where{"id = 1"}
Usually, when I encountered an error like this, it's because I haven't defined the return values of the function calls inside the function I wanna test. But this time I've created it, but it somehow can't read the return. Any idea why?
The error indicates you called CallThisFuncion with params (context.Context, sqkit.Where), but your example is using and setting the expectation for (context.Context, []sqkit.Option). The example with mock.Anything should work, but I believe it's failing because of the context. You'll need to set the expectation with the same context you're passing down. If FunctionToTest is going to be altering the context, I believe you'll need to use mock.Anything instead.
func Test_FunctionToTest(t *testing.T) {
testCase := []struct {
name string
id int64
onCallThisFunctionMock func(context.Context, *mocks.CallThisFunctionSvc)
expectedResult string
wantError bool
expectedError error
}{
{
name: "Success",
id: 1,
onCallThisFunctionMock: func(ctx context.Context, callThisFunctionSvc *mocks.CallThisFunctionSvc) {
args := []sqkit.SelectOption{
sqkit.Where{
fmt.Sprintf("id = %d", 1},
},
}
callThisFunctionSvc.On("CallThisFunction", ctx, args).Return("Success", nil)
}
}
}
for _, tc := range testCases {
var callThisFunctionSvc = new(mocks.CallThisFunctionSvc)
var ctx = context.Background()
tc.onCallThisFunctionMock(ctx, callThisFunctionSvc)
svc := &svc.FunctionToTest{
CallThisFunction: callThisFunctionSvc,
}
actualResult, actualError := svc.FunctionToTest(ctx, tc.id)
if tc.wantEror {
require.Error(t, actualError, tc.expectedError)
} else {
require.NoError(t, actualError)
}
require.Equal(t, tc.expectedResult, actualResult)
}
}
If you want to ensure a context.Context was passed as the first parameter but don't care what context, you could use AnythingOfType.
callThisFunctionSvc.On("CallThisFunction", mock.AnythingOfType("context.Context"), args).Return("Success", nil)
I would like to write a unit test for a controller but I keep getting a runtime error. I found out that it is due to the absence of the Host on the request, the ClientIP() method and request body. How can I set them on the test context?
Here is what I got so far. It fails on the line with Host: c.Request.Host.
Controller:
type loggingControllers struct {
LoggingService services.InterfaceLogging
}
func (l *loggingControllers) RegisterError(c *gin.Context) {
errorEvent := models.Error{
Badges: map[string]string{},
Host: c.Request.Host,
ClientIP: c.ClientIP(),
UserAgent: c.Request.UserAgent(),
Count: 1,
Timestamp: time.Now().Unix(),
}
err := json.NewDecoder(c.Request.Body).Decode(&errorEvent)
if err != nil {
utils.RespondWithError(c, http.StatusInternalServerError, err.Error())
return
}
go l.LoggingService.SaveError(errorEvent)
utils.RespondWithSuccess(c)
}
func GetLoggingControllerMock() loggingControllers {
loggingServiceMock := services.GetLoggingServiceMock()
return loggingControllers{
LoggingService: &loggingServiceMock,
}
}
Unit Test:
func TestLoggingControllers(t *testing.T) {
loggingControllers := GetLoggingControllerMock()
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
loggingControllers.RegisterError(c)
}
Error Message:
--- FAIL: TestLoggingControllers (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x15238ec]
Thanks to Adrian's comment I found the solution.
func TestLoggingControllers(t *testing.T) {
loggingControllers := GetLoggingControllerMock()
w := httptest.NewRecorder()
c, r := gin.CreateTestContext(w)
r.POST("/", loggingControllers.RegisterError)
c.Request, _ = http.NewRequest(http.MethodPost, "/", bytes.NewBuffer([]byte("{}")))
r.ServeHTTP(w, c.Request)
if w.Code != http.StatusOK {
t.Errorf("Expected status %d, got %d", http.StatusOK, w.Code)
}
}
I'm setting up testing in Go.
I use go-sqlmock to test mysql connection and Go Gin as framework. Now I try to test mysql insert logic.
The problem is I need to set mock gin.Context which is used for BindJSON later.
But I can't set this gin.Context so far.
server side: golang
db: mysql
web framework: gin
dao.go
unc PostImageToDBDao(c *gin.Context, db *sql.DB) {
// Because of this logic, I need to set gin.Context with json
var imageData util.ImageData
c.BindJSON(&imageData)
for _, imageName := range imageData.IMAGENAMES {
ins, err := db.Prepare("INSERT INTO images(article_uuid, image_name) VALUES(?,?)")
if err != nil {
log.Fatal(err)
}
ins.Exec(imageData.ARTICLEUUID, imageName.NAME)
}
}
dao_test.go
func TestPostImageToDBDao(t *testing.T) {
db, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
defer db.Close()
prep := mock.ExpectPrepare("^INSERT INTO images*")
prep.ExpectExec().
WithArgs("bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c", "b8119536-fad5-4ffa-ab71-2f96cca19697").
WillReturnResult(sqlmock.NewResult(1, 1))
// try to set context with json post
req, _ := http.NewRequest("POST", "/post/image/db", bytes.NewBuffer([]byte(`[{
"articleUUID": "bea1b24d-0627-4ea0-aa2b-8af4c6c2a41c",
"imageNames": "b8119536-fad5-4ffa-ab71-2f96cca19697",
}]`)))
req.Header.Set("Content-Type", "application/json")
var context *gin.Context
context = &gin.Context{Request: req}
PostImageToDBDao(context, db)
if err := mock.ExpectationsWereMet(); err != nil {
t.Errorf("there were unfulfilled expections: %s", err)
}
}
I expect mock gin.Context to set properly and run go test -v without error, however it fails with the following error:
=== RUN TestPostImageToDBDao
[GIN-debug] [WARNING] Headers were already written. Wanted to override status code 0 with 400
--- FAIL: TestPostImageToDBDao (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x60 pc=0x15bde75]
goroutine 50 [running]:
testing.tRunner.func1(0xc000234100)
/usr/local/go/src/testing/testing.go:830 +0x392
panic(0x16918e0, 0x1ce5850)
/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/gin-gonic/gin.(*Context).AbortWithStatus(0xc00026aee8, 0x190)
/Users/jpskgc/go/pkg/mod/github.com/gin-gonic/gin#v1.4.0/context.go:146 +0x45
github.com/gin-gonic/gin.(*Context).AbortWithError(0xc0000d9ee8, 0x190, 0x1863e00, 0xc0002700a0, 0x1863e00)
/Users/jpskgc/go/pkg/mod/github.com/gin-gonic/gin#v1.4.0/context.go:162 +0x39
github.com/gin-gonic/gin.(*Context).MustBindWith(0xc00026aee8, 0x16328e0, 0xc00022c180, 0x186e060, 0x1d16588, 0x1e316d0, 0x0)
/Users/jpskgc/go/pkg/mod/github.com/gin-gonic/gin#v1.4.0/context.go:561 +0x92
github.com/gin-gonic/gin.(*Context).BindJSON(...)
/Users/jpskgc/go/pkg/mod/github.com/gin-gonic/gin#v1.4.0/context.go:528
article/api/dao.PostImageToDBDao(0xc00026aee8, 0xc000276000)
/Users/jpskgc/article/api/dao/dao.go:54 +0x87
article/api/dao.TestPostImageToDBDao(0xc000234100)
/Users/jpskgc/article/api/dao/dao_test.go:204 +0x4b6
testing.tRunner(0xc000234100, 0x17897e0)
/usr/local/go/src/testing/testing.go:865 +0xc0
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:916 +0x35a
exit status 2
FAIL article/api/dao 0.032s
First, you must instantiate a test *gin.Context and make sure its *http.Request is non-nil:
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Request = &http.Request{
Header: make(http.Header),
}
Then you can mock a POST json body in the following way:
func MockJsonPost(c *gin.Context /* the test context */, content interface{}) {
c.Request.Method = "POST" // or PUT
c.Request.Header.Set("Content-Type", "application/json")
jsonbytes, err := json.Marshal(content)
if err != nil {
panic(err)
}
// the request body must be an io.ReadCloser
// the bytes buffer though doesn't implement io.Closer,
// so you wrap it in a no-op closer
c.Request.Body = io.NopCloser(bytes.NewBuffer(jsonbytes))
}
where the function argument content interface{} is anything that can be marshalled into JSON with json.Marshal(), so in most cases a struct with the proper json tags, or a map[string]interface{}.
Example usage:
func TestMyHandler(t *testing.T) {
w := httptest.NewRecorder()
ctx, _ := gin.CreateTestContext(w)
ctx.Request = &http.Request{
Header: make(http.Header),
}
MockJsonPost(ctx, map[string]interface{}{"foo": "bar"})
MyHandler(ctx)
assert.EqualValues(t, http.StatusOK, w.Code)
}
Related:
How to unit test a Go Gin handler function?
I have a program in go that connects to AWS S3 and gets a file.
I'd like to write some tests for it, but I'd like to know, more generally, how to do these mocks in Golang. I know there are some libraries to create mocks but if I remember correctly I read someone suggesting using only standard libraries for unit tests was the best way to go.
So, how would you test a function like this?
func (s S3Input) Sample(key string) ([]byte, error) {
var buf []byte
waBuf := aws.NewWriteAtBuffer(buf)
_, err := s.Downloader.Download(
waBuf,
&s3.GetObjectInput{
Bucket: aws.String(s.Bucket),
Key: aws.String(key),
},
)
if err != nil {
return nil, err
}
return buf, nil
}
Thank you!
One way to do it is to inject the dependencies in your structure, like such:
type S3Inputer interface {
NewWriteAtBuffer(buf []byte) *aws.WriteAtBuffer
String(v string) *string
}
type S3Input struct {
newWriteAtBufferFunc func(buf []byte) *aws.WriteAtBuffer
stringFunc func(v string) *string
}
func (s *S3Input) NewWriteAtBuffer(buf []byte) *WriteAtBuffer {
return s.newWriteAtBufferFunc(buf)
}
func (s *S3Input) String(v string) *string {
return s.stringFunc(v)
}
func (s S3Input) Sample(key string) ([]byte, error) {
var buf []byte
waBuf := s.NewWriteAtBuffer(buf)
_, err := s.Downloader.Download(
waBuf,
&s3.GetObjectInput{
Bucket: s.String(s.Bucket),
Key: s.String(key),
},
)
if err != nil {
return nil, err
}
return buf, nil
}
func main() {
s := &S3Input{
StringFunc: aws.String,
NewWriteAtBufferFunc: aws.NewWriteAtBuffer,
}
// ...
}
This allows you to replace those functions with whatever you want for testing, without the need of any testing framework.
Then, the testing function would look something like this:
func (s S3Input) TestSample(t *testing.T) {
s3Mock := &S3Input{
StringFunc: (func (v string) *string {
return nil
}),
NewWriteAtBufferFunc: (func (buf []byte) *aws.WriteAtBuffer {
return nil
}),
}
res, err := s3Mock.Sample(...) //
// asserts & error checks
}
You could improve it by creating a S3InputMock type instead of reusing the base one, both would implement the S3Inputer interface and your mock could have attributes allowing it to help you with testing. For example, it could count the number of times a function is called, store the arguments it received, have its methods behave differently depending on the attributes you set for easier testing, etc.