I'm using client-go (the k8s client for go) to programmatically retrieve and update some secrets from my cluster. While doing this, I'm facing the need of unit-testing my code, and after some investigation I stumbled upon client-go's fake client. However, I haven't been able to mock errors yet. I've followed the instructions from this issue, but without any success.
Here you have my business logic:
func (g goClientRefresher) RefreshNamespace(ctx context.Context, namespace string) (err error, warnings bool) {
client := g.kubeClient.CoreV1().Secrets(namespace)
secrets, err := client.List(ctx, metav1.ListOptions{LabelSelector: "mutated-by=confidant"})
if err != nil {
return fmt.Errorf("unable to fetch secrets from cluster: %w", err), false
}
for _, secret := range secrets.Items {
// business logic here
}
return nil, warnings
}
And the test:
func TestWhenItsNotPossibleToFetchTheSecrets_ThenAnErrorIsReturned(t *testing.T) {
kubeClient := getKubeClient()
kubeClient.CoreV1().(*fakecorev1.FakeCoreV1).
PrependReactor("list", "secret", func(action testingk8s.Action) (handled bool, ret runtime.Object, err error) {
return true, &v1.SecretList{}, errors.New("error listing secrets")
})
r := getRefresher(kubeClient)
err, warnings := r.RefreshNamespace(context.Background(), "target-ns")
require.Error(t, err, "an error should have been raised")
}
However, when I run the test I'm getting a nil error. Am I doing something wrong?
I've finally found the error... it is in the resource name of the reactor function, I had secret and it should be the plural secrets instead... :facepalm:. So this is the correct version of the code:
func TestWhenItsNotPossibleToFetchTheSecrets_ThenAnErrorIsReturned(t *testing.T) {
kubeClient := getKubeClient()
kubeClient.CoreV1().(*fakecorev1.FakeCoreV1).
PrependReactor("list", "secrets", func(action testingk8s.Action) (handled bool, ret runtime.Object, err error) {
return true, &v1.SecretList{}, errors.New("error listing secrets")
})
// ...
}
Related
I'm using gin-gonic for a server, and testify for testing and mocks, along with "testing" and "net/http/httptest"
The part of the interface that mocks the method:
func (m *MockInterface) Method(ctx context.Context, id string, deleted bool) ([]models.Entity, error) {
args := m.Called(ctx, id, deleted)
var entities []models.Entity
if args.Get(0) != nil {
entities = args.Get(0).([]models.Entity)
}
var err error
if args.Get(1) != nil {
err = args.Error(1)
}
return entities, err
}
Setting it up in a test - the server is setup outside of this t.Run, there are tests before this that run fine.
t.Run("TestName", func(t *testing.T) {
mockInterface := new(mocks.MockInterface)
mockInterface.On("Method", mock.AnythingOfType("*context.timerCtx"), id.String(), true).Return(mockResp, nil)
// a response writer to capture the response
rr := httptest.NewRecorder()
url := "SomeURLString"
// make the request to the Method handler
request, err := http.NewRequest(http.MethodGet, url, nil)
assert.NoError(t, err)
router.ServeHTTP(rr, request)
assert.NoError(t, err)
assert.Equal(t, http.StatusOK, rr.Code)
mockInterface.AssertExpectations(t)
})
This is where it panics:
router.ServeHTTP(rr, request)
mock: Unexpected Method Call
-----------------------------
Method(*context.timerCtx,string,bool)
0: &context.timerCtx{cancelCtx:context.cancelCtx{Context:(*context.emptyCtx)...etc}}
1: "MatchingID"
2: true
The closest call I have is:
Method(mock.AnythingOfTypeArgument,string,bool)
0: "*context.timerCtx"
1: "MatchingID"
2: false
When I go into debug mode, mockInterface.Mock.ExpectedCalls[0].Arguments[2] is true, just as I set it. And then it panics and says it's not... while it is still true!
I've gone far enough into the stack to verify that the handler called method with the boolean as true, so it ought to pass. Instead it panics, and I'm not sure where to go from here to figure out why.
Does anyone know what is going on here? Is there some kind of odd interaction between gin and testify that I'm missing? Thank you.
I have the following function that makes multiple calls to AWS IAM. I am able to run unit tests on single calls. However when I run a test on the one below I get a panic: "runtime error, invalid memory or nil pointer dereference"
func (iamDependency *iamService) CreateMyUser(userName string) (string, error){
//first IAM call
err:=iamDependency.GetUser(&iam.GetUserInput{UserName: userName})
if err != nil {
fmt.Println("Failed to get user, attempting to create")
//second IAM call
err:=iamDependency.CreateUser(&iam.CreateUserInput{UserName: userName})
if err != nil {
log.Fatalf("Failed to create user\n", err )
}
}
}
Here is my mock and test:
type mockSomeOutput{}
type mockedIamCall{
iamiface.IAMAPI
Response mockSomeOutput
}
func TestCreateMyUser(t *testing.T){
t.Run("Successful user create", fun(t *testing.T){
mo:= mockOutput{}
m:= mockedIamCall{Response: mo}
d:= iamService{
iamInstance: m,
}
mockedUser:="TestUser"
_, err:= d.ResetCredentials(&mockedUser)
if err != nil {
t.Fatal("Everything should be ok")
}
})
}
I'm wondering whether there are any tricks or guidelines for making unit tests for this kind of function in Golang.
Appreciate any help.
You probably want to try using: https://github.com/golang/mock
You can creating mock implementation for the iamiface.IAMAPI (from the actual interface) then expecting the function calls and mocking the response.
Creating the mock implementation of the interface using mockgen.
mockgen -source={path to IAM API interface}
And then you can expect the function calls with something like this on the test cases:
function TestExample(t *testing.T) {
ctrl := gomock.NewController(t)
mockIAMAPI := mock_package.NewMockIAMAPI(ctrl)
mockIAMAPI.EXPECT().GetUser(expectedInput).Return(mockResponse).Times(1)
}
#Raymond thanks for the response, it was insightful. However I seem to have found a simpler answer to my own question. I created my own interface
type UserCreator interface{
GetUser(*iam.GetUserInput) (*iam.GetUserOutput, error)
CreateUser(*iam.CreateUserInput) (*iam.CreateUserInput, error)
}
func CreateMyUser(iamSvc UserCreator, userName string) (string, error){
//first IAM call
_, err:=iamSvc.GetUser(&iam.GetUserInput{UserName: userName})
if err != nil {
fmt.Println("Failed to get user, attempting to create")
//second IAM call
_, err:=iamSvc.CreateUser(&iam.CreateUserInput{UserName: userName})
if err != nil {
log.Fatalf("Failed to create user\n", err )
}
}
}
And then for my test I just implement the interface, override these methods, and pass a mock:
type mockUserCreator{
Response string
}
func (m * mockUserCreator) GetUser(input *iam.GetUserInput)(*iam.GetUserOutput, error){
return &iam.GetUserOutput{}, nil
}
func (m * mockUserCreator) CreateUser(input *iam.CreateUserInput)(*iam.CreateUserOutput, error){
return &iam.CreateUserOutput{}, nil
}
func TestCreateMyUser(t *testing.T){
testcases:=[]struct{
TestName string
}{
{
TestName:"Some test"
}
}
for _, tt := range testcases{
t.Run(tt.TestName, func(t *testing.T){
m := mockUserCreator{}
mockUser := "TestUser"
_, err:= CreateMyUser(&m, mockUser)
if err != nil {
t.Error("TestCreateMyUser returned and error: %s", err)
}
}
}
}
I need to create a Pull Request comment using go-github, and my code works, but now I'd like to write tests for it (yes, I'm aware that tests should come first), so that I don't actually call the real GitHub service during test.
I've read 3 blogs on golang stubbing and mocking, but, being new to golang, I'm a bit lost, despite this discussion on go-github issues. For example, I wrote the following function:
// this is my function
func GetClient(token string, url string) (*github.Client, context.Context, error) {
ctx := context.Background()
ts := oauth2.StaticTokenSource(
&oauth2.Token{AccessToken: token},
)
tc := oauth2.NewClient(ctx, ts)
client, err := github.NewEnterpriseClient(url, url, tc)
if err != nil {
fmt.Printf("error creating github client: %q", err)
return nil, nil, err
}
return client, ctx, nil
}
How could I stub that?
Similarly, I have this:
func GetPRComments(ctx context.Context, client *github.Client) ([]*github.IssueComment, *github.Response, error) {
opts := &github.IssueListCommentsOptions{
ListOptions: github.ListOptions{
Page: 1,
PerPage: 30,
},
}
githubPrNumber, err := strconv.Atoi(os.Getenv("GITHUB_PR_NUMBER"))
if err != nil || githubPrNumber == 0 {
panic("error: GITHUB_PR_NUMBER is not numeric or empty")
}
// use Issues API for PR comments since GitHub docs say "This may seem counterintuitive... but a...Pull Request is just an Issue with code"
comments, response, err := client.Issues.ListComments(
ctx,
os.Getenv("GITHUB_OWNER"),
os.Getenv("GITHUB_REPO"),
githubPrNumber,
opts)
if err != nil {
return nil, nil, err
}
return comments, response, nil
}
How should I stub that?
My thought was to perhaps use dependency injection by creating my own structs first, but I'm not sure how, so currently I have this:
func TestGetClient(t *testing.T) {
client, ctx, err := GetClient(os.Getenv("GITHUB_TOKEN"), "https://example.com/api/v3/")
c, r, err := GetPRComments(ctx, client)
...
}
I would start with an interface:
type ClientProvider interface {
GetClient(token string, url string) (*github.Client, context.Context, error)
}
When testing a unit that needs to call GetClient make sure you depend on your ClientProvider interface:
func YourFunctionThatNeedsAClient(clientProvider ClientProvider) error {
// build you token and url
// get a github client
client, ctx, err := clientProvider.GetClient(token, url)
// do stuff with the client
return nil
}
Now in your test, you can construct a stub like this:
// A mock/stub client provider, set the client func in your test to mock the behavior
type MockClientProvider struct {
GetClientFunc func(string, string) (*github.Client, context.Context, error)
}
// This will establish for the compiler that MockClientProvider can be used as the interface you created
func (provider *MockClientProvider) GetClient(token string, url string) (*github.Client, context.Context, error) {
return provider.GetClientFunc(token, url)
}
// Your unit test
func TestYourFunctionThatNeedsAClient(t *testing.T) {
mockGetClientFunc := func(token string, url string) (*github.Client, context.Context, error) {
// do your setup here
return nil, nil, nil // return something better than this
}
mockClientProvider := &MockClientProvider{GetClientFunc: mockGetClientFunc}
// Run your test
err := YourFunctionThatNeedsAClient(mockClientProvider)
// Assert your result
}
These ideas aren't my own, I borrowed them from those who came before me; Mat Ryer suggested this (and other ideas) in a great video about "idiomatic golang".
If you want to stub the github client itself, a similar approach can be used, if github.Client is a struct, you can shadow it with an interface. If it is already an interface, the above approach works directly.
How can I validate and get info from a JWT received from Amazon Cognito?
I have setup Google authentication in Cognito, and set the redirect uri to to hit API Gateway, I then receive a code which I POST to this endpoint:
https://docs.aws.amazon.com/cognito/latest/developerguide/token-endpoint.html
To receive the JWT token, in a RS256 format. I am now struggling to validate, and parse the token in Golang. I’ve tried to parse it using jwt-go, but it appears to support HMAC instead by default and read somewhere that they recommend using frontend validation instead. I tried a few other packages and had similar problems.
I came across this answer here: Go Language and Verify JWT but assume the code is outdated as that just says panic: unable to find key.
jwt.io can easily decode the key, and probably verify too. I’m not sure where the public/secret keys are as Amazon generated the token, but from what I understand I need to use a JWK URL to validate too? I’ve found a few AWS specific solutions, but they all seem to be hundreds of lines long. Surely it isn’t that complicated in Golang is it?
Public keys for Amazon Cognito
As you already guessed, you'll need the public key in order to verify the JWT token.
https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-verifying-a-jwt.html#amazon-cognito-user-pools-using-tokens-step-2
Download and store the corresponding public JSON Web Key (JWK) for your user pool. It is available as part of a JSON Web Key Set (JWKS).
You can locate it at
https://cognito-idp.{region}.amazonaws.com/{userPoolId}/.well-known/jwks.json
Parse keys and verify token
That JSON file structure is documented in the web, so you could potentially parse that manually, generate the public keys, etc.
But it'd probably be easier to just use a library, for example this one:
https://github.com/lestrrat-go/jwx
And then jwt-go to deal with the JWT part: https://github.com/dgrijalva/jwt-go
You can then:
Download and parse the public keys JSON using the first library
keySet, err := jwk.Fetch(THE_COGNITO_URL_DESCRIBED_ABOVE)
When parsing the token with jwt-go, use the "kid" field from the JWT header to find the right key to use
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodRS256); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
kid, ok := token.Header["kid"].(string)
if !ok {
return nil, errors.New("kid header not found")
}
keys := keySet.LookupKeyID(kid);
if !ok {
return nil, fmt.Errorf("key with specified kid is not present in jwks")
}
var publickey interface{}
err = keys.Raw(&publickey)
if err != nil {
return nil, fmt.Errorf("could not parse pubkey")
}
return publickey, nil
The type assertion in the code provided by eugenioy and Kevin Wydler did not work for me: *jwt.SigningMethodRS256 is not a type.
*jwt.SigningMethodRS256 was a type in the initial commit. From the second commit on (back in July 2014) it was abstracted and replaced by a global variable (see here).
This following code works for me:
func verify(tokenString string, keySet *jwk.Set) {
tkn, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if token.Method.Alg() != "RSA256" { // jwa.RS256.String() works as well
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
kid, ok := token.Header["kid"].(string)
if !ok {
return nil, errors.New("kid header not found")
}
keys := keySet.LookupKeyID(kid)
if len(keys) == 0 {
return nil, fmt.Errorf("key %v not found", kid)
}
var raw interface{}
return raw, keys[0].Raw(&raw)
})
}
Using the following dependency versions:
github.com/dgrijalva/jwt-go/v4 v4.0.0-preview1
github.com/lestrrat-go/jwx v1.0.4
This is what I did with only the latest (v1.0.8) github.com/lestrrat-go/jwx. Note that github.com/dgrijalva/jwt-go does not seem to be maintained anymore and people are forking it to make the updates they need.
package main
import (
...
"github.com/lestrrat-go/jwx/jwk"
"github.com/lestrrat-go/jwx/jwt"
)
...
keyset, err := jwk.Fetch("https://cognito-idp." + region + ".amazonaws.com/" + userPoolID + "/.well-known/jwks.json")
parsedToken, err := jwt.Parse(
bytes.NewReader(token), //token is a []byte
jwt.WithKeySet(keyset),
jwt.WithValidate(true),
jwt.WithIssuer(...),
jwt.WithClaimValue("key", value),
)
//check err as usual
//here you can call methods on the parsedToken to get the claim values
...
Token claim methods
eugenioy's answer stopped working for me because of this refactor. I ended up fixing with something like this
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodRS256); !ok {
return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
}
kid, ok := token.Header["kid"].(string)
if !ok {
return nil, errors.New("kid header not found")
}
keys := keySet.LookupKeyID(kid);
if len(keys) == 0 {
return nil, fmt.Errorf("key %v not found", kid)
}
// keys[0].Materialize() doesn't exist anymore
var raw interface{}
return raw, keys[0].Raw(&raw)
})
A newer method to achieve verification and access the token is to use Gin Cognito JWT Authentication Middleware:
package main
import (
jwtCognito "github.com/akhettar/gin-jwt-cognito"
"github.com/gin-gonic/gin"
"github.com/golang-jwt/jwt"
"log"
)
func main() {
r := gin.Default()
// Create the authentication middleware
mw, err := jwtCognito.AuthJWTMiddleware(<iss>, <user_pool_id>, <region>)
if err != nil {
panic(err)
}
r.GET("/someGet", mw.MiddlewareFunc(), func(c *gin.Context) {
// Get the token
tokenStr, _ := c.Get("JWT_TOKEN")
token := tokenStr.(*jwt.Token)
// Cast the claims
claims := token.Claims.(jwt.MapClaims)
log.Printf("userCognitoId=%v", claims["cognito:username"])
log.Printf("userName=%v", claims["name"])
c.Status(http.StatusOK)
})
// By default it serves on :8080
r.Run()
}
This is what worked for me:
import (
"errors"
"fmt"
"github.com/dgrijalva/jwt-go"
"github.com/gin-gonic/gin"
"github.com/lestrrat-go/jwx/jwk"
"net/http"
"os"
)
func verifyToken(token *jwt.Token) (interface{}, error) {
// make sure to replace this with your actual URL
// https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-verifying-a-jwt.html#amazon-cognito-user-pools-using-tokens-step-2
jwksURL := "COGNITO_JWKS_URL"
set, err := jwk.FetchHTTP(jwksURL)
if err != nil {
return nil, err
}
keyID, ok := token.Header["kid"].(string)
if !ok {
return nil, errors.New("expecting JWT header to have string kid")
}
keys := set.LookupKeyID(keyID)
if len(keys) == 0 {
return nil, fmt.Errorf("key %v not found", keyID)
}
if key := set.LookupKeyID(keyID); len(key) == 1 {
return key[0].Materialize()
}
return nil, fmt.Errorf("unable to find key %q", keyID)
}
I am calling it like this (using AWS Lambda gin) in my case. If you are using a different way of managing requests, make sure to replace that with http.Request or any other framework that you might be using:
func JWTVerify() gin.HandlerFunc {
return func(c *gin.Context) {
tokenString := c.GetHeader("AccessToken")
_, err := jwt.Parse(tokenString, verifyToken)
if err != nil {
c.AbortWithStatus(http.StatusUnauthorized)
}
}
}
This is my go.mod:
module MY_MODULE_NAME
go 1.12
require (
github.com/aws/aws-lambda-go v1.20.0
github.com/aws/aws-sdk-go v1.36.0
github.com/awslabs/aws-lambda-go-api-proxy v0.9.0
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/gin-gonic/gin v1.6.3
github.com/google/uuid v1.1.2
github.com/lestrrat-go/jwx v0.9.2
github.com/onsi/ginkgo v1.14.2 // indirect
github.com/onsi/gomega v1.10.3 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
)
Here's an example using github.com/golang-jwt/jwt, (formally known as github.com/dgrijalva/jwt-go,) and a JWKs like the one AWS Cognito provides.
It'll refresh the AWS Cognito JWKs once every hour, refresh when a JWT signed with an unknown kid comes in, and have a global rate limit of 1 HTTP request to refresh the JWKs every 5 minutes.
package main
import (
"fmt"
"log"
"time"
"github.com/golang-jwt/jwt/v4"
"github.com/MicahParks/keyfunc"
)
func main() {
// Get the JWKS URL from your AWS region and userPoolId.
//
// See the AWS docs here:
// https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-verifying-a-jwt.html
regionID := "" // TODO Get the region ID for your AWS Cognito instance.
userPoolID := "" // TODO Get the user pool ID of your AWS Cognito instance.
jwksURL := fmt.Sprintf("https://cognito-idp.%s.amazonaws.com/%s/.well-known/jwks.json", regionID, userPoolID)
// Create the keyfunc options. Use an error handler that logs. Refresh the JWKS when a JWT signed by an unknown KID
// is found or at the specified interval. Rate limit these refreshes. Timeout the initial JWKS refresh request after
// 10 seconds. This timeout is also used to create the initial context.Context for keyfunc.Get.
options := keyfunc.Options{
RefreshErrorHandler: func(err error) {
log.Printf("There was an error with the jwt.Keyfunc\nError: %s", err.Error())
},
RefreshInterval: time.Hour,
RefreshRateLimit: time.Minute * 5,
RefreshTimeout: time.Second * 10,
RefreshUnknownKID: true,
}
// Create the JWKS from the resource at the given URL.
jwks, err := keyfunc.Get(jwksURL, options)
if err != nil {
log.Fatalf("Failed to create JWKS from resource at the given URL.\nError: %s", err.Error())
}
// Get a JWT to parse.
jwtB64 := "eyJraWQiOiJmNTVkOWE0ZSIsInR5cCI6IkpXVCIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiJLZXNoYSIsImF1ZCI6IlRhc2h1YW4iLCJpc3MiOiJqd2tzLXNlcnZpY2UuYXBwc3BvdC5jb20iLCJleHAiOjE2MTkwMjUyMTEsImlhdCI6MTYxOTAyNTE3NywianRpIjoiMWY3MTgwNzAtZTBiOC00OGNmLTlmMDItMGE1M2ZiZWNhYWQwIn0.vetsI8W0c4Z-bs2YCVcPb9HsBm1BrMhxTBSQto1koG_lV-2nHwksz8vMuk7J7Q1sMa7WUkXxgthqu9RGVgtGO2xor6Ub0WBhZfIlFeaRGd6ZZKiapb-ASNK7EyRIeX20htRf9MzFGwpWjtrS5NIGvn1a7_x9WcXU9hlnkXaAWBTUJ2H73UbjDdVtlKFZGWM5VGANY4VG7gSMaJqCIKMxRPn2jnYbvPIYz81sjjbd-sc2-ePRjso7Rk6s382YdOm-lDUDl2APE-gqkLWdOJcj68fc6EBIociradX_ADytj-JYEI6v0-zI-8jSckYIGTUF5wjamcDfF5qyKpjsmdrZJA"
// Parse the JWT.
token, err := jwt.Parse(jwtB64, jwks.Keyfunc)
if err != nil {
log.Fatalf("Failed to parse the JWT.\nError: %s", err.Error())
}
// Check if the token is valid.
if !token.Valid {
log.Fatalf("The token is not valid.")
}
log.Println("The token is valid.")
// End the background refresh goroutine when it's no longer needed.
jwks.EndBackground()
}
I've built a quick and easy API in Go that queries ElasticSearch. Now that I know it can be done, I want to do it correctly by adding tests. I've abstracted some of my code so that it can be unit-testable, but I've been having some issues mocking the elastic library, and as such I figured it would be best if I tried a simple case to mock just that.
import (
"encoding/json"
"github.com/olivere/elastic"
"net/http"
)
...
func CheckBucketExists(name string, client *elastic.Client) bool {
exists, err := client.IndexExists(name).Do()
if err != nil {
panic(err)
}
return exists
}
And now the test...
import (
"fmt"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"testing"
)
type MockClient struct {
mock.Mock
}
func (m *MockClient) IndexExists(name string) (bool, error) {
args := m.Mock.Called()
fmt.Println("This is a thing")
return args.Bool(0), args.Error(1)
}
func TestMockBucketExists(t *testing.T) {
m := MockClient{}
m.On("IndexExists", "thisuri").Return(true)
>> r := CheckBucketExists("thisuri", m)
assert := assert.New(t)
assert.True(r, true)
}
To which I'm yielded with the following error: cannot use m (type MockClient) as type *elastic.Client in argument to CheckBucketExists.
I'm assuming this is something fundamental with my use of the elastic.client type, but I'm still too much of a noob.
This is an old question, but couldn't find the solution either.
Unfortunately, this library is implemented using a struct, that makes mocking it not trivial at all, so the options I found are:
(1) Wrap all the elastic.SearchResult Methods on an interface on your own and "proxy" the call, so you end up with something like:
type ObjectsearchESClient interface {
// ... all methods...
Do(context.Context) (*elastic.SearchResult, error)
}
// NewObjectsearchESClient returns a new implementation of ObjectsearchESClient
func NewObjectsearchESClient(cluster *config.ESCluster) (ObjectsearchESClient, error) {
esClient, err := newESClient(cluster)
if err != nil {
return nil, err
}
newClient := objectsearchESClient{
Client: esClient,
}
return &newClient, nil
}
// ... all methods...
func (oc *objectsearchESClient) Do(ctx context.Context) (*elastic.SearchResult, error) {
return oc.searchService.Do(ctx)
}
And then mock this interface and responses as you would with other modules of your app.
(2) Another option is like pointed in this blog post that is mock the response from the Rest calls using httptest.Server
for this, I mocked the handler, that consist of mocking the response from the "HTTP call"
func mockHandler () http.HandlerFunc{
return func(w http.ResponseWriter, r *http.Request) {
resp := `{
"took": 73,
"timed_out": false,
... json ...
"hits": [... ]
...json ... ,
"aggregations": { ... }
}`
w.Write([]byte(resp))
}
}
Then you create a dummy elastic.Client struct
func mockClient(url string) (*elastic.Client, error) {
client, err := elastic.NewSimpleClient(elastic.SetURL(url))
if err != nil {
return nil, err
}
return client, nil
}
In this case, I've a library that builds my elastic.SearchService and returns it, so I use the HTTP like:
...
ts := httptest.NewServer(mockHandler())
defer ts.Close()
esClient, err := mockClient(ts.URL)
ss := elastic.NewSearchService(esClient)
mockLibESClient := es_mock.NewMockSearcherClient(mockCtrl)
mockLibESClient.EXPECT().GetEmployeeSearchServices(ctx).Return(ss, nil)
where mockLibESClient is the library I mentioned, and we stub the mockLibESClient.GetEmployeeSearchServices method making it return the SearchService with that will return the expected payload.
Note: for creating the mock mockLibESClient I used https://github.com/golang/mock
I found this to be convoluted, but "Wrapping" the elastic.Client was in my point of view more work.
Question: I tried to mock it by using https://github.com/vburenin/ifacemaker to create an interface, and then mock that interface with https://github.com/golang/mock and kind of use it, but I kept getting compatibility errors when trying to return an interface instead of a struct, I'm not a Go expect at all so probably I needed to understand the typecasting a little better to be able to solve it like that. So if any of you know how to do it with that please let me know.
The elasticsearch go client Github repo contains an official example of how to mock the elasticsearch client. It basically involves calling NewClient with a configuration which stubs the HTTP transport:
client, err := elasticsearch.NewClient(elasticsearch.Config{
Transport: &mocktrans,
})
There are primarily three ways I discovered to create a Mock/Dumy ES client. My response does not include integration tests against a real Elasticsearch cluster.
You can follow this article so as to mock the response from the Rest calls using httptest.Server, to eventually create a dummy elastic.Client struct
As mentioned by the package author in this link, you can work on "specifying an interface that has two implementations: One that uses a real ES cluster, and one that uses callbacks used in testing. Here's an example to get you started:"
type Searcher interface {
Search(context.Context, SearchRequest) (*SearchResponse, error)
}
// ESSearcher will be used with a real ES cluster.
type ESSearcher struct {
client *elastic.Client
}
func (s *ESSearcher) Search(ctx context.Context, req SearchRequest) (*SearchResponse, error) {
// Use s.client to run against real ES cluster and perform a search
}
// MockedSearcher can be used in testing.
type MockedSearcher struct {
OnSearch func(context.Context, SearchRequest) (*SearchResponse, error)
}
func (s *ESSearcher) Search(ctx context.Context, req SearchRequest) (*SearchResponse, error) {
return s.OnSearch(ctx, req)
}
Finally, as mentioned by the author in the same link you can "run a real Elasticsearch cluster while testing. One particular nice way might be to start the ES cluster during testing with something like github.com/ory/dockertest. Here's an example to get you started:"
package search
import (
"context"
"fmt"
"log"
"os"
"testing"
"github.com/olivere/elastic/v7"
"github.com/ory/dockertest/v3"
"github.com/ory/dockertest/v3/docker"
)
// client will be initialize in TestMain
var client *elastic.Client
func TestMain(m *testing.M) {
pool, err := dockertest.NewPool("")
if err != nil {
log.Fatalf("unable to create new pool: %v", err)
}
options := &dockertest.RunOptions{
Repository: "docker.elastic.co/elasticsearch/elasticsearch-oss",
Tag: "7.8.0",
PortBindings: map[docker.Port][]docker.PortBinding{
"9200": {{HostPort: "9200"}},
},
Env: []string{
"cluster.name=elasticsearch",
"bootstrap.memory_lock=true",
"discovery.type=single-node",
"network.publish_host=127.0.0.1",
"logger.org.elasticsearch=warn",
"ES_JAVA_OPTS=-Xms1g -Xmx1g",
},
}
resource, err := pool.RunWithOptions(options)
if err != nil {
log.Fatalf("unable to ES: %v", err)
}
endpoint := fmt.Sprintf("http://127.0.0.1:%s", resource.GetPort("9200/tcp"))
if err := pool.Retry(func() error {
var err error
client, err = elastic.NewClient(
elastic.SetURL(endpoint),
elastic.SetSniff(false),
elastic.SetHealthcheck(false),
)
if err != nil {
return err
}
_, _, err = client.Ping(endpoint).Do(context.Background())
if err != nil {
return err
}
return nil
}); err != nil {
log.Fatalf("unable to connect to ES: %v", err)
}
code := m.Run()
if err := pool.Purge(resource); err != nil {
log.Fatalf("unable to stop ES: %v", err)
}
os.Exit(code)
}
func TestAgainstRealCluster(t *testing.T) {
// You can use "client" variable here
// Example code:
exists, err := client.IndexExists("cities-test").Do(context.Background())
if err != nil {
t.Fatal(err)
}
if !exists {
t.Fatal("expected to find ES index")
}
}
The line
func CheckBucketExists(name string, client *elastic.Client) bool {
states that CheckBucketExists expects a *elastic.Client.
The lines:
m := MockClient{}
m.On("IndexExists", "thisuri").Return(true)
r := CheckBucketExists("thisuri", m)
pass a MockClient to the CheckBucketExists function.
This is causing a type conflict.
Perhaps you need to import github.com/olivere/elastic into your test file and do:
m := &elastic.Client{}
instead of
m := MockClient{}
But I'm not 100% sure what you're trying to do.