Is there a way to execute test cases in GoLang in a pre-defined order.
P.S: I am writing test cases for life cycle of a event. So I have different api's for all the CURD operations. I want to run these test cases in a particular order as only if an event is created it can be destroyed.
Also can I get some value from one test case and pass it as input to another. (example:- To test the delete event api, I need a event_id which i get when I call create_event test case)
I am new to GoLang, can someone please guide me through.
Thanks in advance
The only way to do it is to encapsulate all your tests into one test function, that calls sub-functions in the right order and with the right context, and pass the testing.T pointer to each so they can fail. The down-side is that they will all appear as one test. But in fact that is the case - tests are stateless as far as the testing framework is concerned, and each function is a separate test case.
Note that although the tests may run in the order they are written in, I found no documentation stating that this is actually a contract of some sort. So even though you can write them in order and keep the state as external global variables - that's not recommended.
The only flexibility the framework gives you since go 1.4 is the TestMain method that lets you run before/after steps, or setup/teardown:
func TestMain(m *testing.M) {
if err := setUp(); err != nil {
panic(err)
}
rc := m.Run()
tearDown()
os.Exit(rc)
}
But that won't give you what you want. The only way to do that safely is to do something like:
// this is the whole stateful sequence of tests - to the testing framework it's just one case
func TestWrapper(t *testing.T) {
// let's say you pass context as some containing struct
ctx := new(context)
test1(t, ctx)
test2(t, ctx)
...
}
// this holds context between methods
type context struct {
eventId string
}
func test1(t *testing.T, c *context) {
// do your thing, and you can manipulate the context
c.eventId = "something"
}
func test2(t *testing.T, c *context) {
// do your thing, and you can manipulate the context
doSomethingWith(c.eventId)
}
Related
I apologize for the impracticality of this example, but it's structurally identical to something I have written and tested recently, and am trying to improve.
In summary, I have a rather large public-facing Go method (PrepareFoodOrders) that has one job...in this case doing backend processing for pending restaurant orders. The consumer of this Method really only needs to know how many orders can be successfully made. To determine if an order can be made, a few things need to happen: locating a recipe, checking out equipment, and checking inventory for each ingredient in the recipe. I have broken these into helper methods like findIngredientsForRecipe and findEquipmentToPrepareRecipe. All databases such as orderDatabase, recipeDatabase, ingredientDatabase, and equipmentDatabase are interfaces, and I'm using the Testify Mock library in order to mock those DB calls.
I noticed when writing table driven tests for this method...the test setup becomes extremely bloated. I not only have to set up Mock arguments and returns for all of the Database calls scattered throughout the tested Method and its related helpers, but have to set them up multiple times. For example, a test case might include 3 orders returned from orderDatabase.GetAllPendingOrders(), and those orders' recipes may contain 15, 3, and 8 ingredients respectively (so intotal 26 mock calls to ingredientDatabase.GetAvailableCount() in findIngredientsForRecipe()).
In addition to sheer amount of Mock setups I have to worry about to write that particular test case, I have to determine if various database calls are even going to be made based on if various errors happen. For example, in findIngredientsForRecipe(), if there's a database error getting the amount of an ingredient, the entire helper method will error out and the ingredient inventory won't be updated.
It would take a lot of time to write up what my test table would look like here, but essentially I've found myself writing giant 100+ line long nested structures containing information on how to set up each individual mock database call (should this DB call be made, what should the returns/errors be, what are the expected arguments), etc.
Overall, I feel like I am very much writing my test to match the implementation details of the code, making the test extremely fragile, and with so many internal Mock dependencies that have to be set up to rigorously test the code, the test is nearly impossible to read/maintain. If the order of two database calls are switched in the code...the test will blow up.
I can only see two paths forward:
Make my test less rigorous. With Go's testify package, when I set up "expectations" for a Mock call, I can make the call optional, so I don't hit an assert if a call isn't hit because a previous one early-returns. This can make my test less tightly coupled to the specific order of Database calls and if they will be called but I lose the ability to test how this function is interacting with all the databases...important functionality that the API doesn't return. I wouldn't be able to detect if one of my database calls was unreachable, so as long as it didn't affect the return of my function (that last orderDatabase.Remove call for example). The senior engineers on my team very much discouraged making Mock calls optional, and preferred my test to be more explicit about asserting that certain functions were/were not called.
Split up the test and test functions. In general...we keep unit tests in the same package as the code they are testing. I could reduce the complexity of my test table by unit testing all the helper methods individually, which is what one Senior guy recommended. Then my core test for PrepareFoodOrders() can set up simpler "success or error" execution paths for findIngredientsForRecipe and findEquipmentToPrepareRecipe, and the separate tests for those can get more involved in testing all the ways those can fail . The problem I have with this is that they are supposed to be non-exported helpers, implementation details of the main method. If this were C code, those helper functions wouldn't be in a header file my test would have access to. This just seems fundamentally wrong. I could promote these helpers to public/Exported, but the API doesn't need them...and since they are in the same package, I wouldn't be able to mock or stub them out easily, hence test setup would still be nasty.
// This method tracks the preparation of ordered food items, included finding a recipe,
// decreasing inventory values according to the recipe, and marking the order as completed
// if able to make. Returns how many items were successfully made.
func PrepareFoodOrders() (int, error) {
// Query the database for all orders made to the kitchen
pendingOrders, err := orderDatabase.GetAllPendingOrders()
if err != nil {
return 0, err
}
var completedOrderCount int
for _, order := range pendingOrders {
// Query the database to get a recipe for the ordered item
menuItemID := order.MenuItemCode
recipe, err := recipeDatabase.GetRecipeForMenuItem(menuItemID)
if err != nil {
// Could not find a recipe...too bad, can't make the order, continue to the next one
continue
}
// Handle getting all the ingredients to make the recipe and updating inventory
if err := findIngredientsForRecipe(recipe.ID); err != nil {
// Couldn't get all the stuff to make the recipe...too bad, continue to next order
continue
}
// Maybe something else needs to happen, like getting required pots/pans/utensils
// this would make database calls to track which equipment is being "borrowed" for
// the chef
if err := findEquipmentToPrepareRecipe(recipe.ID); err != nil {
continue
}
// made it this far, mark the order as completed
if err := orderDatabase.Remove(order.ID) {
// the food can be made and is being prepared...but log an error so people
// can figure out why duplicate orders are being processed
}
completedOrderCount++
}
return completedOrderCount, nil
}
// helper method to get ingredients for a recipe and update inventory.
// returns an error if there aren't enough ingredients or something.
func findIngredientsForRecipe(recipe *Recipe) error {
ingredients := recipe.IngredientsList
for _, ingredient := range ingredients {
// check inventory to see if there are enough of this ingredient
if availableCount, err := ingredientDatabase.GetAvailableCount(ingredient.ID); err != nil {
return errors.New("database failure")
}
if availableCount < ingredient.RequiredNum {
return errors.New("not enough of incredient %v", ingredient.Name)
}
// update inventory
if err := ingredientDatabase.UpdateInventory(ingredient.ID, ingredient.RequiredNum); err != nil {
// who knows, maybe we would want to cancel the recipe instead so nobody gets surprised
// when the inventory fails to update and thinks there are more eggs than there actually are!
continue
}
}
return nil
}
... Other helpers would be down here
Anyways, any advice on how to break this function up and simplify my unit test for it would be appreciated. Also, I can't imagine how somebody would do "test driven development" on a method like this when there are so many implementation-specific Mock dependencies to worry about.
I am trying to mock the Go function os.GetEnv() in my test files so that I can get the desired value for a particular environment variable.
For example I have defined.
abc := os.GetEnv("XYZ_URL")
Here I should be able to get the needed value for the variable abc. Also I have several places with the GetEnv functions.
It will be really helpful if someone can give me a workaround without the help of any Go framework.
First, you can't mock that function. You can only mock something exposed as an interface.
Second, you probably don't need to. Mocks, broadly speaking, are over-used, and should be avoided whenever possible.
When testing environment variables, you have few options.
If you're using Go 1.17 or newer, you can take advantage of the new Setenv function, which sets the environment variable for the duration of the current test only:
func TestFoo(t *testing.T) {
t.Setenv("XYZ_URL", "http://example.com")
/* do your tests here */
}
For older versions of Go, consider these options:
Create an object which can be mocked/doubled, which exposes the necessary functionality. Example:
type OS interface {
Getenv(string) string
}
type defaultOS struct{}
func (defaultOS) Getenv(key string) string {
return os.Getenv(key)
}
// Then in your code, replace `os.Getenv()` with:
myos := defaultOS{}
value := myos.Getenv("XYZ_URL")
And in your tests, create a custom implementation that satisfies the interface, but provides the values you need for testing.
This approach is useful for some things (like wrapping the time package), but is probably a bad approach for os.Getenv.
Make your functions not depend on os.Getenv, and instead just pass the value in. Example, instead of:
func connect() (*DB, error) {
db, err := sql.Connect(os.Getenv("XYZ_URL"), ...)
/* ... */
return db, err
}
use:
func connect(url string) (*DB, error) {
db, err := sql.Connect(url, ...)
/* ... */
return db, err
}
In a sense, this only "moves" the problem--you may still want to test the caller, which uses os.Getenv(), but you can at least reduce the surface area of your API that depends on this method, which makes the third approach easier.
Set the environment expliclitly during your tests. Example:
func TestFoo(t *testing.T) {
orig := os.Getenv("XYZ_URL")
os.Setenv("XYZ_URL", "http://example.com")
t.Cleanup(func() { os.Setenv("XYZ_URL", orig) })
/* do your tests here */
}
This approach does have limitations. In particular, it won't work to run multiple of these tests in parallel, so you still want to minimize the number of these tests you run.
This means that approaches 2 and 3 in conjunction with each other can be very powerful.
A final option is to create a function variable, that can be replaced in tests. I've talked about this in another post, but for your example it could look like this:
var getenv = os.Getenv
/* ... then in your code ... */
func foo() {
value := getenv("XYZ_URL") // Instead of calling os.Getenv directly
}
and in a test:
func TestFoo(t *testing.T) {
getenv = func(string) string { return "http://example.com/" }
/* ... your actual tests ... */
}
This has many of the same limitations as option #3, in that you cannot run multiple tests in parallel, as they will conflict.
I'm building an UI for cli apps. I completed the functions but I couldn't figure out how to test it.
Repo: https://github.com/erdaltsksn/cui
func Success(message string) {
color.Success.Println("√", message)
os.Exit(0)
}
// Error prints a success message and exit status 1
func Error(message string, err ...error) {
color.Danger.Println("X", message)
if len(err) > 0 {
for _, e := range err {
fmt.Println(" ", e.Error())
}
}
os.Exit(1)
}
I want to write unit tests for functions. The problem is functions contains print and os.Exit(). I couldn't figure out how to write test for both.
This topic: How to test a function's output (stdout/stderr) in unit tests helps me test print function. I need to add os.Exit()
My Solution for now:
func captureOutput(f func()) string {
var buf bytes.Buffer
log.SetOutput(&buf)
f()
log.SetOutput(os.Stderr)
return buf.String()
}
func TestSuccess(t *testing.T) {
type args struct {
message string
}
tests := []struct {
name string
args args
output string
}{
{"Add test cases.", args{message: "my message"}, "ESC[1;32m"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
want := tt.output
got := captureOutput(func() {
cui.Success(tt.args.message)
})
got := err
if got.Error() != want {
t.Error("Got:", got, ",", "Want:", want)
}
})
}
}
The usual answer in TDD is that you take your function and divide it into two parts; one part that is easy to test, but is not tightly coupled to specific file handles, or a specific implementation of os::Exit; the other part is tightly coupled to these things, but is so simple it obviously has no deficiencies.
Your "unit tests" are mistake detectors that measure the first part.
The second part you write once, inspect it "by hand", and then leave it alone. The idea here being that things are so simple that, once implemented correctly, they don't need to change.
// Warning: untested code ahead
func Foo_is_very_stable() {
bar_is_easy_to_test(stdin, stdout, os.exit)
}
func bar_is_easy_to_test(in *File, out *File , exit func(int)) {
// Do complicated things here.
}
Now, we are cheating a little bit -- os.exit is special magic that never returns, but bar_is_easy_to_test doesn't really know that.
Another design that is a bit more fair is to put the complicated code into a state machine. The state machine decides what to do, and the host invoking the machine decides how to do that....
// More untested code
switch state_machine.next() {
case OUT:
println(state_machine.line())
state_machine.onOut()
case EXIT:
os.exit(state_machine.exitCode())
Again, you get a complicated piece that is easy to test (the state machine) and a much simpler piece that is stable and easily verified by inspection.
This is one of the core ideas underlying TDD - that we deliberately design our code in such a way that it is "easy to test". The justification for this is the claim that code that is easy to test is also easy to maintain (because mistakes are easily detected and because the designs themselves are "cleaner").
Recommended viewing
Boundaries, by Gary Bernhardt
Building Protocol Libraries..., by Cory Benfield
What you have there is called a "side effect" - a situation when execution of your application spans beyond it's environment, it's address space. And the thing is, you don't test side effects. It is not always possible, and when it is - it is unreasonably complicated and ugly.
The basic idea is to have your side effects, like CLI output or os.Exit() (or network connections, or accessing files), decoupled from you main body of logic. There are plenty of ways to do it, the entire "software design" discipline is devoted to that, and #VoiceOfUnreason gives a couple of viable examples.
In your example I would go with wrapping side effects in functions and arranging some way to inject dependencies into Success() & Error(). If you want to keep those two just plain functions, then it's either a function argument or a global variable holding a function for exiting (as per #Peter's comment), but I'd recommend going OO way, employing some patterns and achieving much greater flexibility for you lib.
I'm pretty new to Go and still learning about how things work in Go, so with that said I've been looking in to Go testing approach and how mocking would work for the past few weeks and most of the information I found based on functions being concrete.
E.g. everything is a function is either passed as a receiver or a parameter, however, the problem I face is my function uses a switch case to determine what function it should be called, so it's not passed from outside.
func (n *Notification) Notify(m Message) error {
switch n.Service {
case "slack":
var s slack.Slack
s.User = m.User
s.Host = m.Host
s.Provider = m.Provider
s.SystemUser = m.SystemUser
return s.SlackSend(n.Url)
default:
return errors.New(codes.CODE5)
}
}
Above code is what the function I want to test looks like and I'm having a hard time figuring out how I could mock the SlackSend() function.
I've come across some article say I should write the function in the test file and when I'm happy with what function do, I should write the real code. This doesn't sound right to me because for me it feels like I have the same code in two places and test only uses the one in test and I could change the real code and break it without testes detecting that.
I mainly work on Python and I'm used to using things like Mock/MagicMock that can intercept the function call and replace on at runtime, so apologies in advance if I don't quite get the Go's TDD approach.
This is what test code would look like if anyone wonders:
type MockSlack struct {
*slack.Slack
}
func (s *MockSlack) SlackSend(url string) error {
if url != "" {
return nil
} else {
return errors.New("url empty")
}
}
func TestNotify(t *testing.T) {
m := Message{}
n := Notification{
Service: "slack",
Url: "https://dummy.io",
}
if err := n.Notify(m); err != nil {
t.Errorf("SlackSend, expected: %s, got: %s", "nil", err.Error())
}
}
Obviously the MockSlack structure is not taking effect because it's not really passed in anywhere.
If anyone has any advice on how I could mock this or what I should do instead would be much appreciated.
Thanks in advance.
UPDATE:
Background
This isn't a web server/applicaition of a sort. It's an SSH authentication plugin so it would be a server-side application. As for Notify method, it server's the purpose of a mapper. So it can call Slack, MS Teams, AWS SNS which give caller less conditions to process and how and where the notification it sent is decided by the Notify method.
If you are not able to change the Notify Method to make it testing friendly. One option would be to consider using monkeypatching. Here is an example provided for *net.Dialer:
func main() {
var d *net.Dialer // Has to be a pointer to because `Dial` has a pointer receiver
monkey.PatchInstanceMethod(reflect.TypeOf(d), "Dial", func(_ *net.Dialer, _, _ string) (net.Conn, error) {
return nil, fmt.Errorf("no dialing allowed")
})
_, err := http.Get("http://google.com")
fmt.Println(err) // Get http://google.com: no dialing allowed
}
WARNING: It's not safe to use it outside of a testing environment.
How I can test my NewClient constructor for my Client struct ?
package busybus
import (
"bufio"
"net"
)
type Client struct {
counter integer
conn net.Conn
bufin *bufio.Reader
bufout *bufio.Writer
messages chan string
state string
}
func NewClient(conn net.Conn, messages chan string) *Client {
return &Client{
counter: 0,
conn: conn,
bufin: bufio.NewReader(conn),
bufout: bufio.NewWriter(conn),
messages: messages,
state: "waiting",
}
}
I tried some testing like this:
package busybus
import (
"net"
"testing"
)
func TestNewClient(t *testing.T) {
ln, _ := net.Listen("tcp", ":65535")
conn, _ := ln.Accept()
messages := make(chan string)
client := NewClient(conn, messages)
if client.conn != conn {
t.Errorf("NewClient(%q, %q).conn == %q, want %q", conn, messages, client.conn, conn)
}
}
But this hangs during the test runs due to ln.Accept, it seems a totally wrong approach anyway... any suggestion how I can test this constructor ?
Some code is so short and simple that ensuring that the test is correct is more complicated than ensuring that the code itself is correct.
Your constructor is such code.
What I would do is either not test it at all, or just call it (using some dummy implementation of net.Conn) to ensure that it doesn't blow up in smoke when called (therefore called a smoke test).
Later you can test it as part of a bigger test (integration test), where you have a real server to connect to, and which uses your client struct to talk to it.
If you ever find a bug that is due to this constructor, then first add a test that demonstrates the problem, then fix it. Then you will have your test :-)
A test should check that the constructor "works as intended". Checking the value of client.conn is hardly checking any intended behavior, as the value of an unexported field is not behavior. It just checks HOW the struct and constructor is implemented, not WHAT it implements. Test the what, not the how.
By the way, you could maybe embed a *ReadWriter in your client.
client.buf = bufio.NewReadWriter(...)
In addition to what #bjarke-ebert said.
There are no constructors in Go, they are just normal functions. Test them as you would test any other function. If a function is too complex to test then probably there is a way to change the design to make it more testable.
Also from my experience tests that check internal details of implementation too much (like the test from the question) are very hard to maintain as they constantly need to be updated.