I'm writing a unit test for a function that calls out to module as part of a side effect of invoking it:
defmodule HeimdallrWeb.VerifyController do
use HeimdallrWeb, :controller
def verify(conn, _params) do
[forwarded_host | _tail] = get_req_header(conn, "x-forwarded-host")
case is_preview_page?(forwarded_host) do
{:ok, false} ->
conn |> send_resp(200, "")
{:ok, %Heimdallr.Commits.Commit{} = commit} ->
Heimdallr.Commits.touch_commit(commit)
conn |> send_resp(200, "")
{:not_found, _reason} ->
conn |> send_resp(200, "")
end
end
end
The side effect is triggered from the line Heimdallr.Commits.touch_commit(commit).
A few questions about this:
Should my unit test be concerned with testing the effects of the touch_commit method.
If so, should I think about passing in a generic "touch" function to verify method to make it easier to test. This might be difficult due to the nature of Phoenix / Elixirs routing system, I haven't investigated.
If I was using Rails / Ruby / Rspec then I'd set an expectation that a class level method would be called on the HeimdallrCommits module.
My concern and reason for writing the test is that in the future I may accidentally remove the functionality that is touching a commit by deleting or commenting out the line etc.
I would say 1: No. And that is to keep the complexity low in your test. You only want to test (whatever it is you want to test) in a method, the rest should be mocked ignored. What you could do is verify what your method have invoked touch_commit - should be part of a good mocking framework. Those are my 5 cent, sorry to say that I am not familiar with phoenix/elixir so I can't give you any working examples. Like the verify method i Mockito or Moq is what I am thinking of..
Related
I have a test case where I need to mock downloading an image. The issue is when I mock this download function, it makes the other functions in that file undefined, but I also need to call the other functions in the test as they originally exist without mocking.
Is there a way to mock only one function from App.Functions in the example below and keep the rest of the functions working the same?
The code looks like this for setting up the mock:
setup_with_mocks(
[
{App.Functions, [], [download_file: fn _url -> :ok end]}
],
context
)
Seems that you are using Mock (https://hexdocs.pm/mock/Mock.html). In that case you can use the passthrough option:
test_with_mock "test_name", App.Functions, [:passthrough], [download_file: fn _url -> :ok end] do
end
I don't know if the option is available also for setup_with_mocks.
More info here: https://github.com/jjh42/mock#passthrough---partial-mocking-of-a-module
Sometimes when you encounter difficulty in mocking functions for testing it can indicate an organizational problem in your code, e.g. a violation of the single-responsibility-principle. Pondering things like this starts to venture into more philosophical territory (which Stackoverflow is not geared towards), but generally it's helpful to isolate your modules in a way that is compatible with testing -- some of the common code/repo organizational patterns fall into place more easily when giving due consideration to facilitating testing.
As already noted, Mock allows the passthrough option.
The Mox package does not have a viable solution to this particular use case -- even its skipping-optional-callbacks option does not really fit the bill.
Another option is to go the more manual route: pass an opt (or read one out of the Application config) that can be overridden at runtime to facilitate testing. This tactic smells to me a bit like Javascript's heavy reliance on passing callback functions, but it can work in a pinch, e.g. something like:
def download(url, opts \\ []) do
http_client = Keyword.get(opts, :client, HTTPoison)
http_client.get(url)
end
# OR
def download(url) do
http_client = Application.get_env(:myapp, :http_client, HTTPoison)
http_client.get(url)
end
Then in your tests:
test "download a file" do
assert {:ok, _} = MyApp.download("http://example", client: HttpClientMock)
end
# OR...
setup do
starting_value = Application.get_env(:myapp, :http_client)
on_exit(fn ->
Application.put_env(:myapp, :http_client, starting_value)
end)
end
test "download a file" do
Application.put_env(:myapp, :http_client, ClientMock)
# ...
end
This has the disadvantage of punting compile-time errors into runtime (which might be a worthwhile tradeoff to achieve test coverage), and this approach can become disorganized, so use with care.
Generally, I've found Mox's approach to rely on behaviours/callbacks to lead to cleaner tests and cleaner code, but your mileage and use-cases may vary.
I have a service that only make queries ( read / write ) to influxDB.
I want to unit test this, but I'm not sure how to do it, I've read a bunch of tutos talking about mocking. A lot deals with components like go-sqlmock. But as I am using influxDB, I could not use it.
I also find out other components I've tried to use like goMock or testify to be over-complicated.
What I think to do is to create a Repository Layer, an interface that should implement all the methods I need to run / test, and pass concrete classes with dependency injection.
I think it could work, but is it the easiest way to do it ?
I guess having Repositories everywhere, even for small services, just for them to be testable, seems to be over-engineered.
I can give you code if needed, but I think my question is a bit more theorical than practical. It is about the easiest way to mock a custom DB for unit testing.
To expand on #Markus W Mahlberg answer:
If the goal is to verify the queries are valid and actually execute against influx there's no shortcut for actually performing these against influx. These are usually considered to be "integration" tests. I have found with docker-compose that these tests can be just as reliable as unit tests, and fast enough to be integrated into CI. Having the tests execute in CI enables local engineers to easily run these tests to verify their query changes as well.
I guess having Repositories everywhere, even for small services, just for them to be testable, seems to be over-engineered.
I have found this to be pretty polarizing discussion. A test implementation IS a concrete implementation and paves the way for reliable, repeatable tests that support easily isolating and exercising specific components of your code.
I want to unit test this, but I'm not sure how to do it,
I think this is pretty nuanced, IMO unit testing queries provides negative value. Value comes from using a repository interface to allow your unit tests to explicitly configure responses that you would receive from influx in order to fully exercise your application code. This provides no feedback on influx, which is why the integration tests are essential in order to verify that your application can validly configure, connect, and query against influx. This validation implicitly happens when you deploy your application, at which point it becomes way more expensive in terms of feedback than verifying it locally and in CI with integration tests.
I created a diagram to try and illustrate these differences:
Unit tests with repository are focused on your application code and provide little feedback/value on anything to do with influx. Integration tests are useful for verifying your client (perhaps being extended to your application depending on where the tests are exercising but I prefer to bound it to the client since you already have the static feedback from go on the interfaces and calls). Then finally, as #Markus points out, the step to e2e tests is pretty small from integration tests, and allow you to test your full service.
By its very definition, if you test your integration with an external resource, we are talking of integration tests, not unit tests. So we have two problems to solve here.
Unit tests
What you typically do is to have a data access layer which accepts interfaces, which in turn are easy to mock and you can unittest your application logic.
package main
import (
"errors"
"fmt"
)
var (
values = map[string]string{"foo": "bar", "bar": "baz"}
Expected = errors.New("Expected error")
)
type Getter interface {
Get(name string) (string, error)
}
// ErrorGetter implements Getter and always returns an error to test the error handling code of the caller.
// ofc, you could (and prolly should) use some mocking here in order to be able to test various other cases
type ErrorGetter struct{}
func (e ErrorGetter) Get(name string) (string, error) {
return "", Expected
}
// MapGetter implements Getter and uses a map as its datasource.
// Here you can see that you actually get an advantage: you decouple your logic from the data source,
// making refactoring (and debugging) **much** easier WTSHTF.
type MapGetter struct {
data map[string]string
}
func (m MapGetter) Get(name string) (string, error) {
if v, ok := m.data[name]; ok {
return v, nil
}
return "", fmt.Errorf("No value found for %s", name)
}
type retriever struct {
g Getter
}
func (r retriever) retrieve(name string) (string, error) {
return r.g.Get(name)
}
func main() {
// Assume this is test code. No tests possible on playground ;)
bad := retriever{g: ErrorGetter{}}
s, err := bad.retrieve("baz")
if s != "" || err == nil {
panic("Something went seriously wrong")
}
// Needs to fail as well, as "baz" is not in values
good := retriever{g: MapGetter{values}}
s, err = good.retrieve("baz")
if s != "" || err == nil {
panic("Something went seriously wrong")
}
s, err = good.retrieve("foo")
if s != "bar" || err != nil {
panic("Something went seriously wrong")
}
}
In the example above, I actually had to implement two Getters to cover all test cases, since I could not use a mocking library, but you get the picture.
As for the over engineering: Plain and simple, no, that is not overengineering. It is what I personally call proper craftsmanship. It will pay in the long run to get used to it. Maybe not in this project, but in one to come.
Integration tests
Dodgy. What I tend to do is to make sure my queries are correct before I commit them ;)
In the rare case I really want to verify my queries in a CI for example, I usually create a Makefile which in turn spins up a docker(-compose) which provides the stuff I want to integrate against and then runs the tests.
My problem is specific to Go-kit and how to organize code within.
I'm trying to write a unit test for the following function:
func MakeHandler(svc Service, logger kitlog.Logger) http.Handler {
orderHandler := kithttptransport.NewServer(
makeOrderEndpoint(svc),
decodeRequest,
encodeResponse,
)
r := mux.NewRouter()
r.Handle("/api/v1/order/", orderHandler).Methods("GET")
return r
}
What would be the correct way of writing a proper unit test? I have seen examples such as the following:
sMock := &ServiceMock{}
h := MakeHandler(sMock, log.NewNopLogger())
r := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/api/v1/order/", bytes.NewBuffer([]byte("{}")))
h.ServeHTTP(r, req)
And then testing the body and headers of the request. But this doesn't seem like a proper unit test, as calls other parts of the code (orderHandler). Is it possible to just validate what's returned from MakeHandler() instead of during a request?
TL;DR: Yes, that test is in the right direction. You shouldn't try to test the
internals of the returned handler since that third party package may change in ways you didn't expect in the future.
Is it possible to just validate what's returned from MakeHandler() instead of
during a request?
Not in a good way. MakeHandler() returns an interface and ideally you'd use
just the interface methods in your tests.
You could look at the docs of the type returned by mux.NewRouter() to see if
there are any fields or methods in the concrete type that can give you the
information, but that can turn out to be a pain - both for understanding the
tests (one more rarely used type to learn about) and due to how future
modifications to the mux package may affect your code without breaking the
tests.
What would be the correct way of writing a proper unit test?
Your example is actually in the right direction. When testing MakeHandler(),
you're testing that the handler returned by it is able to handle all the paths
and calls the correct handler for each path. So you need to call the
ServeHTTP() method, let it do its thing and then test to see it worked
correctly. Only introspecting the handler does not guarantee correctness during
actual usage.
You may need to make actually valid requests though so you're able to understand
which handler was called based on the response body or header. That should
bring the test to a quite reasonable state. (I think you already have that)
Similarly, I'd add a basic sub test for each route that's added in the future.
Detailed handler tests can be written in separate funcs.
I modified the code from here to look very much like the code I have and am trying to create Unit Tests for. While I would like to ask "what is the best way" to test this, I will be quite satisfied with any way to create a decent Unit Test! I looked at Spock, GroovyTestCase and GMock, while each of them may be quite capable of creating such a test I found the documentation for such an example lacking in all cases.
class Searcher {
def http = new HTTPBuilder('http://ajax.googleapis.com')
def _executeSearch = { searchQ -> {
req ->
uri.path = '/ajax/services/search/web'
uri.query = searchQ
requestContentType = URLENC
response.success = { resp, reader ->
assert resp.statusLine.statusCode == 200
reader.text
}
response.failure = { resp -> println "FAILURE! ${resp.properties}"
resp.statusLine.statusCode }
}
}
def executeSearch(query) {
// http.setHeaders(Accept: 'application/json') // I want JSON back, but this not important
http.request(GET, _executeGetCommand(query))
}
}
What I want/need to do is to mock 'http' in such a way that I can test:
1. uri.query is getting properly set via the passed in data
2. calls to response.success return mocked test data
3. calls to failure get executed and return the failure code
I am probably approaching this entirely incorrectly and will be open to the "right way" of going about unit testing such code. Please bear with me though as this is new to me!
I would use Spock mock's for your test case. They should be quite straightforward, since you don't need any actual network interaction during unit testing.
Spock mocks are documented pretty well in
the new spock docs
If you do want to test web services from the client side, check out geb, which works well as a companion to spock. Geb is documented in the Book of Geb Integration testing over the web obviously involves more moving parts, so you're right to start by mocking out the server side.
I am looking specifically for frameworks that allow me to take advantage of unique features of the language. I am aware of FsUnit. Would you recommend something else, and why?
My own unit testing library, Unquote, takes advantage of F# quotations to allow you to write test assertions as plain, statically checked F# boolean expressions and automatically produces nice step-by-step test failure messages. For example, the following failing xUnit test
[<Fact>]
let ``demo Unquote xUnit support`` () =
test <# ([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0] #>
produces the following failure message
Test 'Module.demo Unquote xUnit support' failed:
([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0]
[4; 3; 2; 1] = [4..1]
[4; 3; 2; 1] = []
false
C:\File.fs(28,0): at Module.demo Unquote xUnit support()
FsUnit and Unquote have similar missions: to allow you to write tests in an idiomatic way, and to produce informative failure messages. But FsUnit is really just a small wrapper around NUnit Constraints, creating a DSL which hides object construction behind composable function calls. But it comes at a cost: you lose static type checking in your assertions. For example, the following is valid in FsUnit
[<Test>]
let test1 () =
1 |> should not (equal "2")
But with Unquote, you get all of F#'s static type-checking features so the equivalent assertion would not even compile, preventing us from introducing a bug in our test code
[<Test>] //yes, Unquote supports both xUnit and NUnit automatically
let test2 () =
test <# 1 <> "2" #> //simple assertions may be written more concisely, e.g. 1 <>! "2"
// ^^^
//Error 22 This expression was expected to have type int but here has type string
Also, since quotations are able to capture more information at compile time about an assertion expression, failure messages are a lot richer too. For example the failing FsUnit assertion 1 |> should not (equal 1) produces the message
Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed:
Expected: not 1
But was: 1
C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\FsUnit.fs(11,0): at FsUnit.should[a,a](FSharpFunc`2 f, a x, Object y)
C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\VerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()
Whereas the failing Unquote assertion 1 <>! 1 produces the following failure message (notice the cleaner stack trace too)
Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed:
1 <> 1
false
C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\VerifyNunitSupport\VerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()
And of course from my first example at the beginning of this answer, you can see just how rich and complex Unquote expressions and failure messages can get.
Another major benefit of using plain F# expressions as test assertions over the FsUnit DSL, is that it fits very well with the F# process of developing unit tests. I think a lot of F# developers start by developing and testing code with the assistance of FSI. Hence, it is very easy to go from ad-hoc FSI tests to formal tests. In fact, in addition to special support for xUnit and NUnit (though any exception-based unit testing framework is supported as well), all Unquote operators work within FSI sessions too.
I haven't yet tried Unquote, but I feel I have to mention FsCheck:
http://fscheck.codeplex.com/
This is a port of Haskells QuickCheck library, where rather than specifying what specific tests to carry out, you specify what properties about your function should hold true.
To me, this is a bit harder than using traditional tests, but once you figure out the properties, you'll have more solid tests. Do read the introduction: http://fscheck.codeplex.com/wikipage?title=QuickStart&referringTitle=Home
I'd guess a mix of FsCheck and Unquote would be ideal.
You could try my unit testing library Expecto; it's has some features you might like:
F# syntax throughout, tests as values; write plain F# to generate tests
Use the built-in Expect module, or an external lib like Unquote for assertions
Parallel tests by default
Test your Hopac code or your Async code; Expecto is async throughout
Pluggable logging and metrics via Logary Facade; easily write adapters for build systems, or use the timing mechanism for building an InfluxDB+Grafana dashboard of your tests' execution times
Built in support for BenchmarkDotNet
Build in support for FsCheck; makes it easy to build tests with generated/random data or building invariant-models of your object's/actor's state space
Hello world looks like this
open Expecto
let tests =
test "A simple test" {
let subject = "Hello World"
Expect.equal subject "Hello World" "The strings should equal"
}
[<EntryPoint>]
let main args =
runTestsWithArgs defaultConfig args tests