I have an application that defines a type Client struct {} which talks to various other clients in my code that talk to services like github, elasticsearch etc.
Now I have the following ES code in one of my packages
type SinkService interface {
Write(context, index, mapping, doc)
}
type ESSink struct {
client *elastic.Client
}
func NewESSink() *ESSink {}
// checks if the index exists and writes the doc
func (s *ESSink) Write(context, index, mapping, doc) {}
I use this method in my main client that runs the whole application like this c.es.Write(...). Now if I want to write client_test.go I can simply make a mockESSink and use it with some stub code but that won't cover the lines written in my ES code.
How do I unit test my ES code? My ESSink uses an elastic.Client. How do I mock that?
I would like to embed some mock ES client that gives me stub responses and I will be able to test my ESSink.Write method that way.
Based on your question, I assume you're using github.com/olivere/elastic, and you want to be able to test by using stub http responses. When I first read this question, I also have never written Go test code that use ES client. So, in addition to answering this question, I'm also sharing how I find out the answer from the godocs.
First, we can see that elastic.NewClient accepts client option functions. So I checked what kind of client option functions the library provides. Turns out the library provides elastic.SetHttpClient that accepts elastic.Doer. The Doer is an interface that http.Client can implement. From here, the answer becomes clear.
So, you have to:
Change your func NewESSink() to accept http Client or elastic Client.
Write stub http Client (implements elastic.Doer).
ESSink
type ESSink struct {
client *elastic.Client
}
func NewESSink(client *elastic.Client) *ESSink {
return &ESSink{client: client}
}
Stub HttpClient
package stubs
import "net/http"
type HTTPClient struct {
Response *http.Response
Error error
}
func (c *HTTPClient) Do(*http.Request) (*http.Response, error) {
return c.Response, c.Error
}
Your testing code
func TestWrite(t *testing.T) {
// set the body and error according to your test case
stubHttpClient := stubs.HTTPClient{
Response: &http.Response{Body: ...},
Error: ...,
}
elasticClient := elastic.NewClient(elastic.SetHttpClient(stubHttpClient))
esSink := NewESSink(elasticClient)
esSink.Write(...)
}
In your production code, you can use http.Client{} when setting ES http client.
Related
I am new to Golang and have been exploring but not clear about mocking in unit tests. Can anyone explain following specific questions ?
Question1: For writing unit tests in Golang, why we need to have interfaces to mock methods, why not only struct ?
Question2: Why we inject the interface in struct(where we call external method)
With struct -
type GlobalData struct {}
var (
GlobalObj = GlobalData{}
)
func (g GlobalData) GetGlobalData(a string) string{
return a
}
With interface definition-
type GlobalInterface interface {
GetGlobalData(a string) string
}
type GlobalData struct {}
var (
GlobalObj = GlobalData{}
)
func (g GlobalData) GetGlobalData(a string) string{
return a
}
Thanks
Question 1: For writing unit tests in Golang, why we need to have interfaces to mock methods, why not only struct ?
Answer: Its not mandatory
Question 2: Why we inject the interface in struct(where we call external method)
Answer: Because, it helps you to replace the actual function call (that might trigger some out of scope actions as a part of unit test , such as database call, some API call etc) by injecting a MockStruct (which will be implementing the same interface that is there in the actual code). Polymorphism in simple words.
So, you create a MockStruct and define your own mockMethods to it. As polymorphism, your unit test pick MockStruct without complaining. Calling actual DB or http endpoints do not come under unit testing.
Just for reference, I can point you to one of my github codebase where I wrote a small test case for a file. As you can see I mocked :
GuestCartHandler interface , that allowed me to not call the actual implementation
Mocked sql connection using "github.com/DATA-DOG/go-sqlmock" package. This helped me to avoid establishing actual db client (so, no dependency of database while unit testing)
Let me know if you get the idea conceptually or do you need some more clarification.
If you have methods on types in package user let's say, ex.
package user
type User struct {
name string
}
func (u *User) GetUserProfile() UserProfile{}
And now on import in catalog package :
package catalog
import user
func getUserCatalog(user user.User) []catalog {
user.GetUserProfile()
}
Now to test getUserCatalog method there are 2 ways:
1. var getUserProfileFunc = user.GetUserProfile
using this approach mock can be easily passed at test run time like:
getUserProfile = func() UserProfile {
return fakeUserProfile
}
this is the easiest way to test it.
Now there is another way using interface, in package user add an interface like
type UserInterface interface {
GetUserProfile() UserProfile
}
if User package is a library on which you don't have control then create your own interface, type and use this.
In this case testing in catalog package will become like:
because now methods will be invoked from UserInterface type not from UserType, hence while testing :
UserInterface = fakeUserStruct
and follow below steps
//1. define type of func to return
type typeGetUserProfile func() UserProfile
//2. create a var to return
var mockedGetUserProfile typeGetUserProfile
//3. create a type
type FakeUser struct{}
//4. implement method interface
func (user *FakeUserStruct) GetUserProfile() UserProfile{
return mockedGetUserProfile
}
now when running test :
mockerGetUserProfile = func() UserProfile {
return fakeUserProfile
}
There is mock library which helps in creating boilerplate code for mocking. Check this https://github.com/stretchr/testify
There are many other mock library, but I had used this one, this was really cool.
I hope this helps.
if not please let me know, i'll give some example code and push it to Github.
Also please check https://levelup.gitconnected.com/utilizing-the-power-of-interfaces-when-mocking-and-testing-external-apis-in-golang-1178b0db5a32
I'm trying to implement unit tests in Go for an existing service which uses a connection pool struct and a connection struct from an existing library (call these LibraryPool and LibraryConnection) to connect to an external service.
To use these, the service functions in the main code uses a unique, global instance of the pool, which has a GetConnection() method, like this:
// Current Main Code
var pool LibraryPool // global, instantiated in main()
func someServiceFunction(w http.ResponseWriter, r *http.Request) {
// read request
// ...
conn := pool.GetConnection()
conn.Do("some command")
// write response
// ...
}
func main() {
pool := makePool() // builds and returns a LibraryPool
// sets up endpoints that use the service functions as handlers
// ...
}
I'd like to unit-test these service functions without connecting to the external service, and so I'd like to mock the LibraryPool and LibraryConnection. To allow for this, I was thinking of changing the main code to something like this:
// Tentative New Main Code
type poolInterface interface {
GetConnection() connInterface
}
type connInterface interface {
Do(command string)
}
var pool poolInterface
func someServiceFunction(w http.ResponseWriter, r *http.Request) {
// read request
// ...
conn := pool.GetConnection()
conn.Do("some command")
// write response
// ...
}
func main() {
pool := makePool() // still builds a LibraryPool
}
In the tests, I would use mock implementations MockPool and MockConnection of these interfaces, and the global pool variable would be instantiated using MockPool. I would instantiate this global pool in a setup() function, inside of a TestMain() function.
The problem is that in the new main code, LibraryPool does not properly implement poolInterface, because GetConnection() returns a connInterface instead of a LibraryConnection (even though LibraryConnection is a valid implementation of connInterface).
What would be a good way to approach this kind of testing? The main code is flexible too, by the way.
Well, I'll try to answer by completely explain how I see this design. Sorry in advance if this is too much and not to the point..
Entity / Domain
The core of the app, will include the entity struct, won't import ANY outer layer package, but can be imported by every package (almost)
Application / Use case
The "service". Will be responsible mainly for the app logic, won't know about the transport(http), will "talk" with the DB through interface. Here you can have the domain validation, for example if resource is not found, or text is too short. Anything related to business logic.
transport
Will handle the http request, decode the request, get the service to do his stuff, and encode the response. Here you can return 401 if there is a missing required param in the request, or the user is not authorized, or something...
infrastructure
DB connection
Maybe some http engine and router and stuff.
Totally app-agnostic, don't import any inner package, not even Pseron
For example, let's say we want to do something as simple as insert person to the db.
package person will only include the person struct
package person
type Person struct{
name string
}
func New(name string) Person {
return Person{
name: name,
{
}
About the db, let's say you use sql, I recommend to make a package named sql to handle the repo. (if you use postgress, use 'postgress package...).
The personRepo will get the dbConnection which will be initialized in main and implement DBAndler. only the connection will "talk" with the db directly, the repository main goal is to be gateway to the db, and speak in application-terms. (the connection is app-agnostic)
package sql
type DBAndler interface{
exec(string, ...interface{}) (int64, error)
}
type personRepo struct{
dbHandler DBHandler
}
func NewPersonRepo(dbHandler DBHandler) &personRepo {
return &personRepo{
dbHandler: dbHandler,
}
}
func (p *personRepo) InsertPerson(p person.Person) (int64, error) {
return p.dbHandler.Exec("command to insert person", p)
}
The service will get this repository as a dependancy (as interface) in the initailzer, and will interact with it to accomplish the business logic
package service
type PersonRepo interface{
InsertPerson(person.Person) error
}
type service struct {
repo PersonRepo
}
func New(repo PersonRepo) *service {
return &service{
repo: repo
}
}
func (s *service) AddPerson(name string) (int64, error) {
person := person.New(name)
return s.repo.InsertPerson(person)
}
Your transport handler will be initialized with the service as a dependancy, and he will handle the http request.
package http
type Service interface{
AddPerson(name string) (int64, error)
}
type handler struct{
service Service
}
func NewHandler(s Service) *handler {
return &handler{
service: s,
}
}
func (h *handler) HandleHTTP(w http.ResponseWriter, r *http.Request) {
// read request
// decode name
id, err := h.service.AddPerson(name)
// write response
// ...
}
And in main.go you will tie everything together:
Initialize db connection
Initialize personRepo with this connection
Initialize service with the repo
Initialize the transport with the service
package main
func main() {
pool := makePool()
conn := pool.GetConnection()
// repo
personRepo := sql.NewPersonRepo(conn)
// service
personService := service.New(personRepo)
// handler
personHandler := http.NewPersonHandler(personService)
// Do the rest of the stuff, init the http engine/router by passing this handler.
}
Note that every package struct was initialized with an interface but returned a struct, and also the interfaces were declared in the package which used them, not in the package which implemented them.
This makes it easy to unit test these package. for example, if you want to test the service, you don't need to worry about the http request, just use some 'mock' struct that implements the interface that the service depend on (PersonRepo), and you good to go..
Well, I hope it helped you even a little bit, it may seem confusing at first, but in time you will see how this seems like a large piece of code, but it helps when you need to add functionality or switching the db driver and such.. I recommend you to read about domain driven design in go, and also hexagonal arch.
edit:
In addition, this way you pass the connection to the service, the service doesn't import and use the global DB pool. Honestly, I don't know why it is so common, I guess it has its advantages and it is better to some application, but generally I think that letting your service depend on some interface, without actually know what is going on, is much a better practice.
i am using spring cloud's eureka and feign to communicate between some services (lets say A and B). Now id like to unittest my service layer of a single service (A). The problem is, that this service (A) is using a feign client to request some information of the other service (B).
Running the unittests without any special configuration throws the following exception: java.lang.RuntimeException: com.netflix.client.ClientException: Load balancer does not have available server for client: service-b => but i do not want any server to run.
My question is: Is there a way to mock the feign client, so i can unittest my service (A) without running an eureka instance and service (B)?
Edit:
I ended up creating a stub for the feign client. The stub is marked as a primary component to force spring instantiating the stub within my tests.
This is the solution i came up with.
//the feign client
#FeignClient("user")
public interface UserClient {
UserEntity getUser();
}
//the implementation i use for the tests
#Component
#Primary //mark as primary implementation
public class UserClientTestImpl implements UserClient {
#Override public UserEntity getUser() {
return someKindOfUser;
}
}
The question is ... do you even need to mock? I often see that people mention "mock" as the first solution to anything that "should not be part of the unit test". Mocking is a technique, not the solution to everything. (see here).
If you are still at the early stages of your code, just refactor and use something else instead of depending on the concrete instance of the Feign Client. You might use an interface, an abstract class, a trait or whatever you want. Don't depend on the object itself, otherwise you have to "mock it".
public interface IWebClient {
public String get(...);
public String post(...);
}
To the question: but I will have other code that will do exactly the same (except that it will be on the concrete instance of Feign), what do I do then?
Well, you can write a functional test and call an instance of a web server that you can setup locally - or use Wiremock, as mentioned by Marcin Grzejszczak in one of the answers.
public class FeignClientWrapper implements IWebClient {
private feign = something
public String get() {
feign.get( ... )
}
public String post() {
feign.post( ... )
}
}
Unit tests are used to test algorithms, if/else, loops: how units work. Don't write code to make mocks fit - it must be the other way around: your code should have less dependencies, and you should mock only when you need to verify the behavior (otherwise you can use a stub or a fake object): do you need to verify the behavior? Do you need to test that a particular method gets called in your code? Or that a particular method gets called with X, Y, and Z for 3 times in a row? Well, then yes, mocking is ok.
Otherwise, use a fake object: what you want is to test just the call/response and maybe the status code. All you probably want is to test how your code reacts to different outputs (e.g., the field "error" is present or not in a JSON response), different status codes (assuming that the Client documentation is right: 200 OK when GET, 201 when POST, etc).
Mocking a feign client is really useful in microservice component tests. You want to test one microservice without having to start all the other microservices.
If you're using Spring (and it looks like you are), the #MockBean annotation together with a bit of Mockito code will do the job.
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment =
SpringBootTest.WebEnvironment.DEFINED_PORT)
public class TestYourComponent {
#Configuration
#Import({YourConfiguration.class})
public static class TestConfiguration {
}
#MockBean
private UserClient userClient;
#Test
public void someTest()
{
//...
mockSomeBehavior();
//...
}
private void mockSomeBehavior() {
Mockito.doReturn(someKindOfUser).when(userClient).getUser();
}
}
If you need to use a mock you can use Wiremock to stub the response for a given request - http://wiremock.org/stubbing.html. That way you will do integration tests with real HTTP requests sent. For unit testing the answer from #Markon is very good.
I'm studying the outyet example project from https://github.com/golang/example/tree/master/outyet. The test file does not cover the case where http.Head(url) returns an error. I would like to extend the unit tests to cover the if statement where the error is logged (https://github.com/golang/example/blob/master/outyet/main.go#L100). I would like to mock http.Head(), but I'm not sure how to do this. How can this be done?
The http.Head function simply calls the Head method on the default HTTP client (exposed as http.DefaultClient). By replacing the default client within your test, you can change the behaviour of these standard library functions.
In particular, you will want a client that sets a custom transport (any object implementing the http.RoundTripper interface). Something like the following:
type testTransport struct{}
func (t testTransport) RoundTrip(request *http.Request) (*http.Response, error) {
# Check expectations on request, and return an appropriate response
}
...
savedClient := http.DefaultClient
http.DefaultClient = &http.Client{
Transport: testTransport{},
}
# perform tests that call http.Head, http.Get, etc
http.DefaultClient = savedClient
You could also use this technique to mock network errors by returning an error from your transport rather than an HTTP response.
I'm trying to create a simple mock for unit testing some code using the VMware vSphere API client - govmomi - but I'm having trouble finding a usable pattern.
A simple use case for the client library would be to retrieve the installed licenses for a vSphere cluster:
vclient, err := govmomi.NewClient(*vcurl, true)
if err != nil {
return err
}
lic, err := vclient.LicenseManager().ListLicenses()
NewClient() returns a pointer to a Client structure, Client.LicenseManager() returns an instance of a LicenseManager structure, and LicenseManager.ListLicenses() returns a slice of structures containing the license info. Coming from a Python background, I'd usually monkey patch the ListLicenses() method on LicenseManger for a mock, but I can't seem to come up with a comparable pattern or methodology in Go.
To this point, I've tried creating a wrapper structure VCenterClient with the govmomi Client structure as an anonymous member and a "constructor" function NewVCenter() to create new instances of the wrapper structure with logic for mocks:
import (
"net/url"
"github.com/vmware/govmomi"
"github.com/vmware/govmomi/vim25/types"
)
type VCenterClient struct {
VCenterClientInterface
}
type VCenterClientInterface interface {
LicenseManager() LicenseManager
}
type LicenseManager interface {
ListLicenses() ([]types.LicenseManagerLicenseInfo, error)
}
type VCenterClientMock struct{}
type LicenseManagerMock struct{}
func (v *VCenterClientMock) LicenseManager() LicenseManager {
return LicenseManagerMock{}
}
func (l LicenseManagerMock) ListLicenses() ([]types.LicenseManagerLicenseInfo, error) {
return make([]types.LicenseManagerLicenseInfo, 0), nil
}
func NewVCenterClient(uri string, mock bool) *VCenterClient {
if mock {
return &VCenterClient{&VCenterClientMock{}}
}
vcurl, _ := url.Parse(uri)
vclient, _ := govmomi.NewClient(*vcurl, true)
return &VCenterClient{vclient}
}
...but I having trouble using interfaces to properly abstract the nested structures in the govmomi library. I know the above will not work as govmomi.LicenseManager() returns a structure of type govmomi.LicenseManager and my VCenterClientInterface.LicenseManager() method returns an interface of type LicenseManager. However, I'm struggling to find an alternative.
Any help on a better design pattern or proper use of interfaces in this case would be much appreciated.
This library is a SOAP client (http://godoc.org/github.com/vmware/govmomi/vim25/soap#Client). Abstract at the HTTP layer with net/http/httptest (http://golang.org/pkg/net/http/httptest/) or by using your own HTTPRoundtripper to mock the response.