Questions
I would like to know the use case of t.Cleanup introduced in Go1.14. What is the convenience of t.Cleanup as compared to using defer?
https://golang.org/pkg/testing/#T.Cleanup.
Sample
For example, let's say we create a temporary directory, and when we test it, we want to delete the temporary directory we created.
t.Cleanup can be used to write a test as follows, but it also works as defer os.RemoveAll(tempDir).
package mypkg
import (
"io/ioutil"
"os"
"testing"
)
func TestDirwalk(t *testing.T) {
tempDir, err := ioutil.TempDir(".", "temp")
if err != nil {
t.Errorf("create tempDir: %v", err)
}
t.Cleanup(func() { os.RemoveAll(tempDir) })
// something...
}
Cleanup functions are also called if your test panics, so in your case both would work.
The advantage of using T.Cleanup() becomes clear if your test calls other functions, passing testing.T along. Obviously using defer in those functions would be executed before those functions return, but if you register cleanup functions using T.Cleanup(), then they will be called only at the end of your test.
Think of T.Cleanup() as an "improved" and extended version of defer. It also documents that the passed functions are for cleanup purposes.
t.Cleanup is useful for cleaning up resources allocated by a helper function when the test does not care about the resource itself.
Example
Consider testing a service layer. The service uses a *sql.DB but does not create it itself.
package testutils
import (
"testing"
"my/db"
"my/domain"
)
func NewTestSubject(t *testing.T) *domain.Service {
t.Helper()
sqldb := newDatabase(t)
s, _ := domain.NewService(sqldb)
return s
}
func newDatabase(t *testing.T) *sql.DB {
t.Helper()
d, _ := db.Create()
t.Cleanup(func() {
d.Close()
})
}
Without t.Cleanup newTestSubject would have to return (*domain.Service, *sql.DB), leaking details about domain.Service's construction.
Besides what others have pointed out, t.Cleanup() is also useful when dealing with parallel subtests, where the cleanup should only run after all subtests have completed. Consider
func TestSomething(t *testing.T){
setup()
defer cleanup()
t.Run("parallel subtest 1", func(t *testing.T){
t.Parallel()
(...)
})
t.Run("parallel subtest 2", func(t *testing.T){
t.Parallel()
(...)
})
}
which doesn't work because the test function will return while the the subtests are still running, causing the resources required by the subtests to get wiped out by defer cleanup().
Before t.Cleanup() a way to solve this was to wrap the subtests on another test
func TestSomething(t *testing.T){
setup()
defer cleanup()
t.Run("parallel tests", func(t *testing.T){
t.Run("subtest 1", func(t *testing.T){
t.Parallel()
(...)
})
t.Run("subtest 2", func(t *testing.T){
t.Parallel()
(...)
})
})
}
which looks ok, but with t.Cleanup() it gets way better
func TestSomething(t *testing.T){
setup()
t.Cleanup(cleanup)
t.Run("parallel subtest 1", func(t *testing.T){
t.Parallel()
(...)
})
t.Run("parallel subtest 2", func(t *testing.T){
t.Parallel()
(...)
})
}
Related
Let's say I have a function if that is executed asynchronous as a go routine:
func f(wg *sync.WaitGroup){
defer wg.Done()
// Do sth
}
main(){
var wg sync.WaitGroup
wg.Add(1)
go f(&wg)
wg.Wait() // Wait until f is done
// ...
}
How would I create a unit test for f that makes sure wg.Done() is called?
One option is to call wg.Done() in the test directly after the f is called. If f fails to call wg.Done() the test will panic which is not nice.
Another option would be to create an interface for sync.WaitGroup but that seems a bit weird.
How would I create a unit test for f that makes sure wg.Done() is called?
Something like this:
func TestF(t *testing.T) {
wg := &sync.WaitGroup{}
wg.Add(1)
// run the task asynchronously
go f(wg)
// wait for the WaitGroup to be done, or timeout
select {
case <-wrapWait(wg):
// all good
case <-time.NewTimer(500 * time.Millisecond).C:
t.Fail()
}
}
// helper function to allow using WaitGroup in a select
func wrapWait(wg *sync.WaitGroup) <-chan struct{} {
out := make(chan struct{})
go func() {
wg.Wait()
out <- struct{}{}
}()
return out
}
You don't inspect the WaitGroup directly, which you can't do anyway. Instead you assert that the function behaves as expected, given the expected input.
In this case, the expected input is the WaitGroup argument and the expected behavior is that wg.Done() gets called eventually. What does that mean, in practice? It means that if the function is successful a WaitGroup with count 1 will reach 0 and allow wg.Wait() to proceed.
The statement defer wg.Done() at the beginning of f already makes sure that the test is resilient to errors or crashes. The addition of a timeout is simply to make sure the test will complete within a reasonable time, i.e. that it doesn't stall your test suite for too long. Personally, I prefer using explicit timeouts, either with timers or with contexts, to 1) avoid problems if someone forgets to set timeouts at the CI level, 2) make the time ceiling available to anyone who checks out the repo and runs the test suite, i.e. avoid dependencies on IDE configs or whatnot.
I have this function
override fun trackEvent(trackingData: TrackingData) {
trackingData.eventsList()
}
And I could have my test as below.
#Test
fun `My Test`() {
// When
myObject.trackEvent(myTrackingMock)
// Then
verify(myTrackingMock, times(1)).eventsList()
}
However, if I make it into a
override fun trackEvent(trackingData: TrackingData) {
GlobalScope.launch{
trackingData.eventsList()
}
}
How could I still get my test running? (i.e. can make the launch Synchronous?)
I created my own CoroutineScope and pass in (e.g. CoroutineScope(Dispatchers.IO) as a variable myScope)
Then have my function
override fun trackEvent(trackingData: TrackingData) {
myScope.launch{
trackingData.eventsList()
}
}
Then in my test I mock the scope by create a blockCoroutineScope as below.
class BlockCoroutineDispatcher : CoroutineDispatcher() {
override fun dispatch(context: CoroutineContext, block: Runnable) {
block.run()
}
}
private val blockCoroutineScope = CoroutineScope(BlockCoroutineDispatcher())
For my test, I'll pass the blockCoroutineScope in instead as myScope. Then the test is executed with launch as a blocking operation.
To approach the answer, try asking a related question: "How would I unit-test a function that has
Thread { trackingData.eventsList() }
in it?"
Your only hope is running a loop that repeatedly checks the expected condition, for some period time, until giving up and declaring the test failed.
When you wrote GlobalScope.launch, you waived your interest in Kotlin's structured concurrency, so you'll have to resort to unstructured and non-deterministic approaches of testing.
Probably the best recourse is to rewrite your code to use a scope under your control.
I refactored my method to
suspend fun deleteThing(serial: String): String? = coroutineScope {
This way, I can launch coroutines with launch
val jobs = mutableListOf<Job>()
var certDeleteError: String? = null
certs.forEach { certArn ->
val job = launch {
deleteCert(certArn, serial)?.let { error ->
jobs.forEach { it.cancel() }
certDeleteError = error
}
}
jobs.add(job)
}
jobs.joinAll()
For the test, I can then just use runTest and it runs all of the coroutines synchronously
#Test
fun successfullyDeletes2Certs() = runTest {
aws.deleteThing("s1")
Now you just need to mind your context where you are calling the deleteThing function. For me, it was a ktor request, so I could just call launch there also.
delete("vehicles/{vehicle-serial}/") {
launch {
aws.deleteThing(serial)
}
}
I am implementing a API in Golang. I have a endpoint where I am calling a method with parameters of other package. Now I need to check that, that method has been called in the request.
Below is the small similar scenario what I am doing and what I am expecting.
My handler
package myPackage
import (
"log"
"github.com/myrepo/notifier" // my another package
)
func MyHandler(writer http.ResponseWriter, request *http.Request) {
// ...
// ...
notifier.Notify(4, "sdfsdf")
// ...
// ...
}
Testing handler
func TestMyHandler(t *testing.T) {
// Here I want to
req, _ := http.NewRequest("GET", "/myendpoint", nil)
// ... Want to test that notifier.Notify is called
// ...
}
In the TestMyHandler, I want to check that notifier.Notify has called.
Findings
I tried to understand the AssertNumberOfCalls, func (*Mock) Called, and func (*Mock) MethodCalled but I am not sure how to use them :(.
I am a newbie in Golang and really exicted to do that.
Please let me know if I missed anything or you may need more information for more understaing.
This is a good opportunity to use dependency injection and interfaces.
Namely, we need to extract the concept of a Notifier
(warning: code not tested directly)
type Notifier interface {
Notify(int, string)() error
}
Now to avoid any confusion with the notifier library, use a local alias.
import "github.com/myrepo/notifier" mynotifier
Then, because the library you're using exports it as a function, not within a struct, we'll need to make a struct that implements our interface
type myNotifier struct {}
func (mn *myNotifier) Notify(n int, message string) error {
return mynotifier.Notify(n, message)
}
Then you modify your function:
func MyHandler(writer http.ResponseWriter, request *http.Request, notifier Notifier) {
// ...
// ...
notifier.Notify(4, "sdfsdf")
// ...
// ...
}
Then in your test, you're now free to send in a spy Notifier
type spyNotifier struct {
called boolean
}
func (n *spyNotifier) Notify(n int, msg string) error {
n.called = true
return
}
Want to test that notifier.Notify is called.
No you don't. You are interested in that the handler does what it should do and this seems to consist of two things:
Return the right response (easy to test with a net/http/httptest.ResponseRecorder), and
Has some noticeable side effect, here issue some notification.
To test 2. you test that the notification was issued, not that some function was called.
Whatever notify.Notify results in (e.g. a database entry, a file, some HTTP call) should be tested. Formally this is no longer unit testing but testing for side effects is never strict unit testing.
What you can do: Wrap your handler logic into some object and observe that objects state. Ugly. Don't.
This approach is similar to Mathew's answer, but uses the mock package from testify instead. Here, you create a mock implementation of the Notifier, register the method call, and assert that the method has been called with the expected arguments.
Implementation
package handler
import (
"net/http"
"github.com/stretchr/testify/mock"
)
// the interface for the Notifier
type Notifier interface {
Notify(int, string) error
}
// the mock implementation of the interface above
type MockNotifier struct {
mock.Mock
}
// stub the notify method to ensure it can be expected later in the test
func (mockNotifier *MockNotifier) Notify(arg1 int, arg2 string) error {
args := mockNotifier.Called(arg1, arg2)
return args.Error(0)
}
// this handler which accepts a Notifier for dependency injection
type Handler struct {
notifier Notifier
}
// the MyHandler implementation which calls the notifier instance
func (h *Handler) MyHandler(writer http.ResponseWriter, request *http.Request) {
// this is what we want to test!
h.notifier.Notify(4, "sdfsdf")
}
Test
package handler_test
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestMyHandler(t *testing.T) {
t.Parallel()
mockNotifier := MockNotifier{}
handler := Handler{ notifier: &mockNotifier }
// register the mock to expect a call to Notify with the given arguments and return nil as the error
mockNotifier.On("Notify", 4, "sdfsdf").Return(nil)
// setup the test arguments
request := httptest.NewRequest(http.MethodGet, "/someapi", nil)
writer := httptest.NewRecorder()
// call the handler
handler.MyHandler(writer, request)
// this is the important part!!
// this ensures that the mock Notify method was called with the correct arguments, otherwise the test will fail
mockNotifier.AssertExpectations(t)
}
I follow the MVP pattern + UseCases to interact with a Model layer. This is a method in a Presenter I want to test:
fun loadPreviews() {
launch(UI) {
val items = previewsUseCase.getPreviews() // a suspending function
println("[method] UseCase items: $items")
println("[method] View call")
view.showPreviews(items)
}
}
My simple BDD test:
fun <T> givenSuspended(block: suspend () -> T) = BDDMockito.given(runBlocking { block() })
infix fun <T> BDDMockito.BDDMyOngoingStubbing<T>.willReturn(block: () -> T) = willReturn(block())
#Test
fun `load previews`() {
// UseCase and View are mocked in a `setUp` method
val items = listOf<PreviewItem>()
givenSuspended { previewsUseCase.getPreviews() } willReturn { items }
println("[test] before Presenter call")
runBlocking { presenter.loadPreviews() }
println("[test] after Presenter call")
println("[test] verify the View")
verify(view).showPreviews(items)
}
The test passes successfully but there's something weird in the log. I expect it to be:
"[test] before Presenter call"
"[method] UseCase items: []"
"[method] View call"
"[test] after Presenter call"
"[test] verify the View"
But it turns out to be:
[test] before Presenter call
[test] after Presenter call
[test] verify the View
[method] UseCase items: []
[method] View call
What's the reason of this behaviour and how should I fix it?
I've found out that it's because of a CoroutineDispatcher. I used to mock UI context with EmptyCoroutineContext. Switching to Unconfined has solved the problem
Update 02.04.20
The name of the question suggests that there'll be an exhaustive explanation how to unit test a suspending function. So let me explain a bit more.
The main problem with testing a suspending function is threading. Let's say we want to test this simple function that updates a property's value in a different thread:
class ItemUpdater(val item: Item) {
fun updateItemValue() {
launch(Dispatchers.Default) { item.value = 42 }
}
}
We need to somehow replace Dispatchers.Default with an another dispatcher only for testing purposes. There're two ways how we can do that. Each has its pros and cons, and which one to choose depends on your project & style of coding:
1. Inject a Dispatcher.
class ItemUpdater(
val item: Item,
val dispatcher: CoroutineDispatcher // can be a wrapper that provides multiple dispatchers but let's keep it simple
) {
fun updateItemValue() {
launch(dispatcher) { item.value = 42 }
}
}
// later in a test class
#Test
fun `item value is updated`() = runBlocking {
val item = Item()
val testDispatcher = Dispatchers.Unconfined // can be a TestCoroutineDispatcher but we still keep it simple
val updater = ItemUpdater(item, testDispatcher)
updater.updateItemValue()
assertEquals(42, item.value)
}
2. Substitute a Dispatcher.
class ItemUpdater(val item: Item) {
fun updateItemValue() {
launch(DispatchersProvider.Default) { item.value = 42 } // DispatchersProvider is our own global wrapper
}
}
// later in a test class
// -----------------------------------------------------------------------------------
// --- This block can be extracted into a JUnit Rule and replaced by a single line ---
// -----------------------------------------------------------------------------------
#Before
fun setUp() {
DispatchersProvider.Default = Dispatchers.Unconfined
}
#After
fun cleanUp() {
DispatchersProvider.Default = Dispatchers.Default
}
// -----------------------------------------------------------------------------------
#Test
fun `item value is updated`() = runBlocking {
val item = Item()
val updater = ItemUpdater(item)
updater.updateItemValue()
assertEquals(42, item.value)
}
Both of them are doing the same thing - they replace the original Dispatchers.Default in test classes. The only difference is how they do that. It's really really up to you which of them to choose so don't get biased by my own thoughts below.
IMHO: The first approach is a little too much cumbersome. Injecting dispatchers everywhere will result into polluting most of the classes' constructors with an extra DispatchersWrapper only for a testing purpose. However Google recommends this way at least for now. The second style keeps things simple and it doesn't complicate the production classes. It's like an RxJava's way of testing where you have to substitute schedulers via RxJavaPlugins. By the way, kotlinx-coroutines-test will bring the exact same functionality someday in future.
I see you found out on you own, but I'd like to explain a bit more for the people that might run into the same problem
When you do launch(UI) {}, a new coroutine is created and dispatched to the "UI" Dispatcher, that means that your coroutine now runs on a different thread.
Your runBlocking{} call create a new coroutine, but runBlocking{} will wait for this coroutine to end before continuing, your loadPreviews() function creates a coroutine, start it and then return immediately, so runBlocking() just wait for it and return.
So while runBlocking{} has returned, the coroutine that you created with launch(UI){} is still running in a different thread, that's why the order of your log is messed up
The Unconfined context is a special CoroutineContext that simply create a dispatcher that execute the coroutine right there on the current thread, so now when you execute runBlocking{}, it has to wait for the coroutine created by launch{} to end because it is running on the same thread thus blocking that thread.
I hope my explanation was clear, have a good day
At the moment I try to establish best practices for unit testing go handlers.
I need to mock dependencies, but therefore I have to be able to access / mock these dependencies.
There I some solutions I do not want to consider like for example global variables / application state. Or having all handlers as functions of a struct holding the dependencies member variables.
I was kind of satisfied with a solution where I injected the needed dependencies of a handler in the following way:
func helloHandler(db *DbService) http.HandlerFunc {
return func(w http.ResponseWriter, r *httpRequest) {
// handler code goes here
}
}
Then I am able to provide this handler for routing:
http.HandleFunc("/hello", helloHander(myDbService))
I can also test it easily:
helloHandler(myDbService)(req, respRecorder)
But when I use for example gorilla mux (what I haven't considered from the beginning) I get another problem.
Gorilla/mux adds extra semantics like filtering by Method (GET / POST), providing path parameters etc.
So I should need to test the resulting gorilla router.
But then I am again not able to inject my mock dependencies anymore.
When I get the router back everything (dependencies) is already set up.
But I also don't want to rebuild my Router in my tests because of DRY!
So for now I don't really have a good solution for the problem of conveniently setting up my routes but also being able to mock the dependencies.
Any idea?
I personally changed my Router function into a method with a Receiver, so I can create Mock receivers:
Production code:
func main(){
router := mux.NewRouter()
version.AddRouter(router, contextRoot)
someStruct := SomeStruct{SomeDependency{}}
router.HandleFunc(contextRoot, someStruct.HandleRequests).
Methods(http.MethodGet)
}
func (s SomeStruct) HandleRequests(writer http.ResponseWriter, reader *http.Request) {
...
}
func (s SomeDependency) SomeFunctionCalledFromHandler(...) SomeReturnStruct {
...
return SomeReturnStruct{}
}
Unit test:
type mockWriter struct {}
type someMockDependency struct {}
func Test_HandleRequest1(t *testing.T) {
someStructMockDeps := SomeStruct{someMockDependency{}}
someStructMockDeps.HandleRequests(mockWriter{}, &http.Request{URL: &url.URL{RawQuery:"http://dummy-query.com"}});
assert.Equal(...)
}
func (someMockDependency) SomeFunctionCalledFromHandler(...) SomeReturnStruct {
...
return SomeReturnStruct{}
}
Integration test:
func TestHandlerFuncIntg(t *testing.T) {
if testing.Short() {
println("skipping")
t.Skip()
}
req, err := http.NewRequest("GET", "/hello?param=value", nil)
if err != nil {
t.Fatal(err)
}
rr := httptest.NewRecorder()
someStructMockDeps := SomeStruct{someMockDependency{}}
handler := http.HandlerFunc(someStructMockDeps.HandleRequests)
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
assert.Equal(t, "This is my result", rr.Body.String())
}
func (someMockDependency) SomeFunctionCalledFromHandler(...) SomeReturnStruct {
...
return SomeReturnStruct{}
}
SomeStruct declares the dependencies (as Dependency Injection) so they can be overridden for tests.