Is there a way to mock and/or spy on methods in golang without using dependency injection. For example lets say that I want to unit test the following method
import (
"github.com/spf13/viper"
"google.golang.org/grpc"
)
func startGrpcServer(ctx context.Context, wg *sync.WaitGroup) {
defer wg.Done()
listen, err := net.Listen("tcp", fmt.Sprintf(":%v", viper.GetString("GRPC_PORT")))
if err != nil {
log.Fatal(err)
}
var opts []grpc.ServerOption
grpcServer := grpc.NewServer(opts...)
reflection.Register(grpcServer)
s := health.NewHealthServer()
proto.RegisterHealthServer(grpcServer, s)
go func() {
<-ctx.Done()
log.Print("Stopping grpc server")
grpcServer.GracefulStop()
}()
log.Print("Starting grpc server")
grpcServer.Serve(listen)
}
I want to make sure of the following
The server is started (spying on the grpcServer.Serve method.
The environment value is ready from viper.GetString method.
As you can see the packages for these methods are global imports and I am not using DI. Also this is legacy code so I cannot go and change all the code to support DI. From my reading and initial researches done you cannot use mock/spy in golang without DI. Is this correct ? or is there some way to do this ?
I came across the following but this is fairly dated How do I mock a function from another package without using dependency injection? and from what I see in this, my options are fairly limited.
Is there a way to mock and/or spy on methods in golang without using dependency injection.
No, there isn't.
Related
I'm working with getstream's Go library in gin gonic and realized that my endpoints will be heavily dependent on stream_chat.Client.
For instance, in the following endpoint (/v1/chat/test-token), a stream_chat.Client must be created so testing this endpoint in unit test would mean creating and maintaining an interface that documents all the methods I use from stream_chat.Client so that I can perform dependency injection with a MockClient that satisfies the same interface and then I can mock the methods chatClient.UpsertUser and chatClient.CreateToken when I write my unit test.
func main() {
config.Load()
server := gin.New()
chatClient, err := stream_chat.NewClient(config.StreamApiKey, config.StreamApiSecret)
if err != nil {
log.Err(err)
os.Exit(2)
}
v1 := server.Group("/v1")
{
v1.GET("/chat/test-token/", func(c *gin.Context) {
_, err := chatClient.UpsertUser(&stream.User{
ID: "test-user",
Role: "admin",
})
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{})
}
token, _ := chatClient.CreateToken("test-user", time.Time{})
c.JSON(http.StatusOK, gin.H{
"token": token,
})
})
}
server.Run(fmt.Sprintf(":%s", config.Port))
}
It seems to me to be quite laborious to document each method that I'd use from stream_chat.Client in order to keep a good test coverage on the endpoints, so I wonder what one should do in this case?
Is maintaining an interface for stream_chat.Client the correct way to go?
Less relevant: Is there a way to properly decouple the gin.HandlerFunc, i.e. func(c *gin.Context) from the creation of stream_chat.Client?
Even less relevant: Is it better to create a singleton stream_chat.Client or should I create a new client for each endpoint that requires a client?
Is maintaining an interface for stream_chat.Client the correct way to go?
If you have a non-interface dependency and you wish to unit test handlers with that, then yes. You need to wrap stream_chat.Client in an interface.
If the third-party struct has a lot of methods, you could split the interface in logical units and inject in each handler only those that are actually needed. The underlying stream_chat.Client implements all of them, but the individual mocks can be kept small and easier to reason about. Personally, I don't think it's worth the overhead. There's plenty of open-source mock generators, above all mock and mockgen, and also tools that generate interfaces from structs.
Is there a way to properly decouple the gin.HandlerFunc, i.e. func(c *gin.Context) from the creation of stream_chat.Client?
You have several options, which you can find here: How to pass arguments to router handlers in Golang using Gin web framework?
In short, the options I prefer are due to better unit-testability are:
make the handlers methods of a struct and your dependencies fields of this struct.
use a provider pattern and set the provider into the Gin context in a middleware
I have an application that defines a type Client struct {} which talks to various other clients in my code that talk to services like github, elasticsearch etc.
Now I have the following ES code in one of my packages
type SinkService interface {
Write(context, index, mapping, doc)
}
type ESSink struct {
client *elastic.Client
}
func NewESSink() *ESSink {}
// checks if the index exists and writes the doc
func (s *ESSink) Write(context, index, mapping, doc) {}
I use this method in my main client that runs the whole application like this c.es.Write(...). Now if I want to write client_test.go I can simply make a mockESSink and use it with some stub code but that won't cover the lines written in my ES code.
How do I unit test my ES code? My ESSink uses an elastic.Client. How do I mock that?
I would like to embed some mock ES client that gives me stub responses and I will be able to test my ESSink.Write method that way.
Based on your question, I assume you're using github.com/olivere/elastic, and you want to be able to test by using stub http responses. When I first read this question, I also have never written Go test code that use ES client. So, in addition to answering this question, I'm also sharing how I find out the answer from the godocs.
First, we can see that elastic.NewClient accepts client option functions. So I checked what kind of client option functions the library provides. Turns out the library provides elastic.SetHttpClient that accepts elastic.Doer. The Doer is an interface that http.Client can implement. From here, the answer becomes clear.
So, you have to:
Change your func NewESSink() to accept http Client or elastic Client.
Write stub http Client (implements elastic.Doer).
ESSink
type ESSink struct {
client *elastic.Client
}
func NewESSink(client *elastic.Client) *ESSink {
return &ESSink{client: client}
}
Stub HttpClient
package stubs
import "net/http"
type HTTPClient struct {
Response *http.Response
Error error
}
func (c *HTTPClient) Do(*http.Request) (*http.Response, error) {
return c.Response, c.Error
}
Your testing code
func TestWrite(t *testing.T) {
// set the body and error according to your test case
stubHttpClient := stubs.HTTPClient{
Response: &http.Response{Body: ...},
Error: ...,
}
elasticClient := elastic.NewClient(elastic.SetHttpClient(stubHttpClient))
esSink := NewESSink(elasticClient)
esSink.Write(...)
}
In your production code, you can use http.Client{} when setting ES http client.
I have a service that only make queries ( read / write ) to influxDB.
I want to unit test this, but I'm not sure how to do it, I've read a bunch of tutos talking about mocking. A lot deals with components like go-sqlmock. But as I am using influxDB, I could not use it.
I also find out other components I've tried to use like goMock or testify to be over-complicated.
What I think to do is to create a Repository Layer, an interface that should implement all the methods I need to run / test, and pass concrete classes with dependency injection.
I think it could work, but is it the easiest way to do it ?
I guess having Repositories everywhere, even for small services, just for them to be testable, seems to be over-engineered.
I can give you code if needed, but I think my question is a bit more theorical than practical. It is about the easiest way to mock a custom DB for unit testing.
To expand on #Markus W Mahlberg answer:
If the goal is to verify the queries are valid and actually execute against influx there's no shortcut for actually performing these against influx. These are usually considered to be "integration" tests. I have found with docker-compose that these tests can be just as reliable as unit tests, and fast enough to be integrated into CI. Having the tests execute in CI enables local engineers to easily run these tests to verify their query changes as well.
I guess having Repositories everywhere, even for small services, just for them to be testable, seems to be over-engineered.
I have found this to be pretty polarizing discussion. A test implementation IS a concrete implementation and paves the way for reliable, repeatable tests that support easily isolating and exercising specific components of your code.
I want to unit test this, but I'm not sure how to do it,
I think this is pretty nuanced, IMO unit testing queries provides negative value. Value comes from using a repository interface to allow your unit tests to explicitly configure responses that you would receive from influx in order to fully exercise your application code. This provides no feedback on influx, which is why the integration tests are essential in order to verify that your application can validly configure, connect, and query against influx. This validation implicitly happens when you deploy your application, at which point it becomes way more expensive in terms of feedback than verifying it locally and in CI with integration tests.
I created a diagram to try and illustrate these differences:
Unit tests with repository are focused on your application code and provide little feedback/value on anything to do with influx. Integration tests are useful for verifying your client (perhaps being extended to your application depending on where the tests are exercising but I prefer to bound it to the client since you already have the static feedback from go on the interfaces and calls). Then finally, as #Markus points out, the step to e2e tests is pretty small from integration tests, and allow you to test your full service.
By its very definition, if you test your integration with an external resource, we are talking of integration tests, not unit tests. So we have two problems to solve here.
Unit tests
What you typically do is to have a data access layer which accepts interfaces, which in turn are easy to mock and you can unittest your application logic.
package main
import (
"errors"
"fmt"
)
var (
values = map[string]string{"foo": "bar", "bar": "baz"}
Expected = errors.New("Expected error")
)
type Getter interface {
Get(name string) (string, error)
}
// ErrorGetter implements Getter and always returns an error to test the error handling code of the caller.
// ofc, you could (and prolly should) use some mocking here in order to be able to test various other cases
type ErrorGetter struct{}
func (e ErrorGetter) Get(name string) (string, error) {
return "", Expected
}
// MapGetter implements Getter and uses a map as its datasource.
// Here you can see that you actually get an advantage: you decouple your logic from the data source,
// making refactoring (and debugging) **much** easier WTSHTF.
type MapGetter struct {
data map[string]string
}
func (m MapGetter) Get(name string) (string, error) {
if v, ok := m.data[name]; ok {
return v, nil
}
return "", fmt.Errorf("No value found for %s", name)
}
type retriever struct {
g Getter
}
func (r retriever) retrieve(name string) (string, error) {
return r.g.Get(name)
}
func main() {
// Assume this is test code. No tests possible on playground ;)
bad := retriever{g: ErrorGetter{}}
s, err := bad.retrieve("baz")
if s != "" || err == nil {
panic("Something went seriously wrong")
}
// Needs to fail as well, as "baz" is not in values
good := retriever{g: MapGetter{values}}
s, err = good.retrieve("baz")
if s != "" || err == nil {
panic("Something went seriously wrong")
}
s, err = good.retrieve("foo")
if s != "bar" || err != nil {
panic("Something went seriously wrong")
}
}
In the example above, I actually had to implement two Getters to cover all test cases, since I could not use a mocking library, but you get the picture.
As for the over engineering: Plain and simple, no, that is not overengineering. It is what I personally call proper craftsmanship. It will pay in the long run to get used to it. Maybe not in this project, but in one to come.
Integration tests
Dodgy. What I tend to do is to make sure my queries are correct before I commit them ;)
In the rare case I really want to verify my queries in a CI for example, I usually create a Makefile which in turn spins up a docker(-compose) which provides the stuff I want to integrate against and then runs the tests.
I am learning to do unit testing using mocks in Go for the first time, using gomock's mockgen utility. My unit tests work fine except for one of them. The method-under-test has two dependencies: one on a database, and the other on an external service it makes rest api calls to. The mock for the database (mockRepo) works fine in that the method-under-test properly invokes the mock instead of the actual repo code. The mock for the rest client, however, continues to invoke the actual rest client and not the mock code. I can't figure out why. Can someone explain why and help fix?
Here is my unit test:
func TestService_CreateWorkspace(t *testing.T) {
ts := NewTestService(t)
defer ts.mockCtrl.Finish()
ts.mockClient.EXPECT().POST(gomock.Any(), gomock.Any(), gomock.Any()).Times(1).Return(&http.Response{StatusCode: 200}, nil)
testWs := TestWorkspaces()["max-ws"]
ts.mockRepo.EXPECT().Create(testWs).Times(1).Return(&testWs, nil)
ws, err := ts.service.CreateWorkspace(&testWs)
assert.Equal(t, testWs, ws)
assert.NoError(t, err)
}
Here is the code for NewTestService:
type TestService struct {
mockCtrl *gomock.Controller
mockClient *MockRestClient
mockRepo *MockRepository
service Service
}
func NewTestService(t *testing.T) *TestService {
mockCtrl := gomock.NewController(t)
mockRepo := NewMockRepository(mockCtrl)
mockClient := NewMockRestClient(mockCtrl)
return &TestService{
mockCtrl: mockCtrl,
mockClient: mockClient,
mockRepo: mockRepo,
service: NewService(mockRepo),
}
}
Is there an issue with assigning the same mock controller to two different mock objects? Not really sure what's going on here. Any help appreciated.
I resolved this as Adrian in the comments above suggested. I was missing a way to pass the mock client into the NewService and ended up adding a client parameter to NewService.
I am using mocks in unit testing in Go. How do I understand the difference between stub and mock in the implementation code in Go?
Intention of mocks and stubs in GO is the same as different programming languages:
stub is replacement for some dependency in your code that will be used during test execution. It is typically built for one particular test and unlikely can be reused for another because it has hardcoded expectations and assumptions.
mock takes stubs to next level. It adds means for configuration, so you can set up different expectations for different tests. That makes mocks more complicated, but reusable for different tests.
Let's check how that works on example:
In our case, we have http handler that internally makes http calls to another web service. To test handler we want to isolate handler code from dependency we do not control (external web service). We can do that by either using stub or mock.
Our handler code is the same for stub and mock. We should inject http.Client dependency to be be able to isolate it in unit test:
func New(client http.Client) http.Handler {
return &handler{
client: client,
}
}
type handler struct {
client http.Client
}
func (h handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
...
// work with external web service that cannot be executed in unit test
resp, err := h.client.Get("http://example.com")
...
}
Our replacement for run-time http.Client is straight-forward in stub:
func TestHandlerStub(t *testing.T) {
mux := http.NewServeMux()
mux.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// here you can put assertions for request
// generate response
w.WriteHeader(http.StatusOK)
}))
server := httptest.NewServer(mux)
r, _ := http.NewRequest(http.MethodGet, "https://some.com", nil)
w := httptest.NewRecorder()
sut := New(server.Client())
sut.ServeHTTP(w, r)
//assert handler response
}
Mock story is more complicated. I am skipping code for mock implementation but this is how its interface may look like:
type Mock interface {
AddExpectation(path string, handler http.HandlerFunc)
Build() *http.Client
}
This is code for test using Mock:
func TestHandlerMock(t *testing.T) {
mock := NewMock()
mock.AddExpectation("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// here you can put assertions for request
// generate response
w.WriteHeader(http.StatusOK)
}))
r, _ := http.NewRequest(http.MethodGet, "https://some.com", nil)
w := httptest.NewRecorder()
sut := New(mock.Build())
sut.ServeHTTP(w, r)
//assert handler response
}
For this simple sample it adds no much value. But think about more complicated cases. You can build much cleaner tests code and cover more cases with less lines.
This is how tests setup may look like if we have to call 2 services and evolved our mock a little bit:
mock.AddExpectation("/first", firstSuccesfullHandler).AddExpectation("/second", secondSuccesfullHandler)
mock.AddExpectation("/first", firstReturnErrorHandler).AddExpectation("/second", secondShouldNotBeCalled)
mock.AddExpectation("/first", firstReturnBusy).AddExpectation("/first", firstSuccesfullHandler)AddExpectation("/second", secondSuccesfullHandler)
You can imagine how many times you have to copy-paste handler logic in tests if we do not have our tiny mock helper. That copy-pasted code makes our tests fridgile.
But building your own mocks is not the only options. You can rely on existing mocking packages like DATA-DOG/go-sqlmock that mocks SQL.