I want to create a test for the following function which creates a file at a particular path:
exists, err := IsFileExists(path)
if err != nil {
return err
}
if exists {
return errors.New("File already exists")
}
file, err := os.Create(path)
defer file.Close()
return err
I know in JUnit there's the TemporaryFolder rule to help you set up the file and directory structure you need for your test like so:
public class DummyFileClassTest {
#Rule
public TemporaryFolder folder = new TemporaryFolder();
#Test
public void someMethod() {
// given
final File file1 = folder.newFile("myfile1.txt");
final File file2 = folder.newFile("myfile2.txt");
... etc...
}
}
Is there something similar in Go? Or a better approach?
testing provides TempDir to get a test directory which will be cleaned up after each test.
import(
"testing"
"path"
)
func TestConvert(t *testing.T) {
tmpdir := t.TempDir()
...
// tmpdir will be cleaned up
}
You can then do path.Join(tmpdir, "test.file") to make a file with a specific name in the temp directory, or ioutil.TempFile(tmpdir, "*") to make a random one.
Note, your function contains a race condition. If two processes both check for the file at the same time they will both conclude it does not exist and both try to create the file. One will create the file, the other will truncate it. This is "check then do".
Instead, create the file and then check for an error. "Do then check".
Related
I am working on learning go with a simple program that is doing some file reading and am working on adding unit testing to my program. I have ran into an issue/question while doing this. I want to unit test the function below and my question is that the function takes a name of the file which is then opened and processed. During testing I do not want to actually pass it a real file. I am wondering is this something I can somehow mock so that I can just pass it a "fake" file and have it process that instead? Thanks!
func openAndReadFile(fileName string) [][]string {
file, err := os.Open(fileName)
if err != nil {
fmt.Printf("Failed to read file: %s", fileName)
}
r := csv.NewReader(file)
lines, err := r.ReadAll()
if err != nil {
log.Fatal(err)
}
return lines
}
You need to refactor your code and make more suitable for testing.
Here is how I would do it:
func openAndReadFile(fileName string) [][]string {
file, err := os.Open(fileName)
if err != nil {
fmt.Printf("Failed to open file: %s", fileName)
}
lines, err := readFile(file)
if err != nil {
fmt.Printf("Failed to read file: %s", fileName)
}
return lines
}
func readFile(reader io.Reader) ([][]string, error) {
r := csv.NewReader(reader)
lines, err := r.ReadAll()
if err != nil {
log.Fatal(err)
}
return lines, err
}
Then for testing you can simply use any data structure that implements the io.reader interface. For example, I use a bytes buffer, but you can choose a network connection:
func TestReadFile(t *testing.T) {
var buffer bytes.Buffer
buffer.WriteString("fake, csv, data")
content, err := readFile(&buffer)
if err != nil {
t.Error("Failed to read csv data")
}
fmt.Print(content)
}
The function you have shown is dominated by interactions: Interactions with the file system and interactions with the csv reader. To be sure that these interactions work nicely you will later anyway have to do some integration-testing against the file system and the csv reader. Think about which bugs you are hoping to find, and you will see that bugs are more likely on the interaction level: Is the order of file,err correct, or should it be the other way around? Is nil really the value indicating no error? Do you have to give more arguments to Open? etc.
Therefore, I would not concentrate on unit-testing this function. However, this function is a good candidate to be mocked to make unit-testing the surrounding code easier. Thus, mock openAndReadFile to unit-test the surrounding code, and test openAndReadFile using integration-testing.
I'd strongly suggest using an interface instead of the filename string like the other answers here are recommending, but if you really must do this the only way is likely with a temp file. The decision to use a string file name has locked the code into assuming something to be present on the file system and has pushed in the responsibility of file management.
I am trying to learn how to write tests for my code in order to write better code, but I just seem to have the hardest time figuring out how to actually test some code I have written. I have read so many tutorials, most of which seem to only cover functions that add two numbers or mock some database or server.
I have a simple function I wrote below that takes a text template and a CSV file as input and executes the template using the values of the CSV. I have "tested" the code by trial and error, passing files, and printing values, but I would like to learn how to write proper tests for it. I feel that learning to test my own code will help me understand and learn faster and better. Any help is appreciated.
// generateCmds generates configuration commands from a text template using
// the values from a CSV file. Multiple commands in the text template must
// be delimited by a semicolon. The first row of the CSV file is assumed to
// be the header row and the header values are used for key access in the
// text template.
func generateCmds(cmdTmpl string, filename string) ([]string, error) {
t, err := template.New("cmds").Parse(cmdTmpl)
if err != nil {
return nil, fmt.Errorf("parsing template: %v", err)
}
f, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("reading file: %v", err)
}
defer f.Close()
records, err := csv.NewReader(f).ReadAll()
if err != nil {
return nil, fmt.Errorf("reading records: %v", err)
}
if len(records) == 0 {
return nil, errors.New("no records to process")
}
var (
b bytes.Buffer
cmds []string
keys = records[0]
vals = make(map[string]string, len(keys))
)
for _, rec := range records[1:] {
for k, v := range rec {
vals[keys[k]] = v
}
if err := t.Execute(&b, vals); err != nil {
return nil, fmt.Errorf("executing template: %v", err)
}
for _, s := range strings.Split(b.String(), ";") {
if cmd := strings.TrimSpace(s); cmd != "" {
cmds = append(cmds, cmd)
}
}
b.Reset()
}
return cmds, nil
}
Edit: Thanks for all the suggestions so far! My question was flagged as being too broad, so I have some specific questions regarding my example.
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err? For example:
type tmplTest struct {
name string // test name
tmpl string // the text template
filename string // CSV file with template values
expected []string // expected configuration commands
err error // expected error
}
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()? Same idea for template.Parse(), template.Execute(), etc.
Edit 2: Below is a test function I came up with. My two question from the first edit still stand.
package cmd
import (
"testing"
"strings"
"path/filepath"
)
type tmplTest struct {
name string // test name
tmpl string // text template to execute
filename string // CSV containing template text values
cmds []string // expected configuration commands
}
var tests = []tmplTest{
{"empty_error", ``, "", nil},
{"file_error", ``, "fake_file.csv", nil},
{"file_empty_error", ``, "empty.csv", nil},
{"file_fmt_error", ``, "fmt_err.csv", nil},
{"template_fmt_error", `{{ }{{`, "test_values.csv", nil},
{"template_key_error", `{{.InvalidKey}}`, "test_values.csv", nil},
}
func TestGenerateCmds(t *testing.T) {
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cmds, err := generateCmds(tc.tmpl, filepath.Join("testdata", tc.filename))
if err != nil {
// Unexpected error. Fail the test.
if !strings.Contains(tc.name, "error") {
t.Fatal(err)
}
// TODO: Otherwise, check that the function failed at the expected point.
}
if tc.cmds == nil && cmds != nil {
t.Errorf("expected no commands; got %d", len(cmds))
}
if len(cmds) != len(tc.cmds) {
t.Errorf("expected %d commands; got %d", len(tc.cmds), len(cmds))
}
for i := range cmds {
if cmds[i] != tc.cmds[i] {
t.Errorf("expected %q; got %q", tc.cmds[i], cmds[i])
}
}
})
}
}
You basically need to have some sample files with the contents you want to test, then in your test code you can call the generateCmds function passing in the template string and the files to then verify that the results are what you expect.
It is not so much different as the examples you probably saw for simpler cases.
You can place the files under a testdata folder inside the same package (testdata is a special name that the Go tools will ignore during build).
Then you can do something like:
func TestCSVProcessing(t *testing.T) {
templateStr := `<your template here>`
testFile := "testdata/yourtestfile.csv"
result, err := generateCmds(templateStr, testFile)
if err != nil {
// fail the test here, unless you expected an error with this file
}
// compare the "result" contents with what you expected
// failing the test if it does not match
}
EDIT
About the specific questions you added later:
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err?
Yes, it'd make sense to include both the expected strings to be returned as well as the expected error (if any).
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()?
I don't think you'll be able to "initialize" an equivalent error for each case. Sometimes the libraries might use internal types for their errors making this impossible. Easiest would be to "initialize" a regular error with the same value returned in its Error() method, then just compare the returned error's Error() value with the expected one.
in one of my golang Projects i went to mock the os.FileInfo for testcases.
I am not sure if I understand the interface handling of golang correctly.
As far as I know the following piece of code should work, but I get an Compilererror saying that the interface does not match.
I modified this go-doc example a bit in case you want to test it out yourself.
package main
import (
"fmt"
"io/ioutil"
"log"
)
type file interface{
Name() string
}
func readFiles() []file{
files, err := ioutil.ReadDir(".")
if err != nil {
log.Fatal(err)
}
return files
}
func main() {
files := readFiles()
for _, file := range files {
fmt.Println(file.Name())
}
}
Following at the golang doc, the ioutil.ReadDir(".") should Return a slice of os.FileInfo which should be a specialisation of my selfwritten file interface.
Can anyone help me out of this hell of misconceptions and entanglements, please?
Thank you very much guys!
os.FileInfo is interface and it can converted to file. But you try to convert []os.FileInfo to []file. Slice is not interface, and can not be asserted.
To elaborate on the other answer and comments, os.FileInfo does fulfill the file interface you've defined, but a slice of os.FileInfo does not implicitly convert to a slice of your file interface. A simple fix would be to have your readFiles do the conversion explicitly, e.g.
func readFiles() []file{
fileInfos, err := ioutil.ReadDir(".")
if err != nil {
log.Fatal(err)
}
files := make([]file, len(fileInfos))
for i, fi := range fileInfos {
files[i] = fi
}
return files
}
I'm working on converting a pet project of mine from Python to Go just to help me get a bit familiar with the language. An issue I am currently facing is that it's escaping my forward slashes. So it will receive a string like:
/location/to/something
and it then becomes
%2flocation%2fto%2fsomething
Now, it's only doing this when it's in a link (from what I've been reading this escaping is contextual) so this is what the line in the HTML template looks like:
<tr><td>{{.FileName}}</td></tr>
If possible, how can I prevent this in either the template or the code itself?
This is what my templating function looks like (yes, I know it's hackish)
func renderTemplate(w http.ResponseWriter, tmpl string) {
t, err := template.ParseFiles(templates_dir+"base.html", templates_dir+tmpl)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
if tmpl == "view.html" {
err = t.Execute(w, FileList)
} else {
err = t.Execute(w, nil)
}
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
As the value of .FullFilePath, pass a value of type template.URL instead of string, which will tell the html/template package not to escape it.
For example:
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileFullPath": template.URL("/location/to/something"),
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Output (try it on the Go Playground):
<tr><td>something.txt</td></tr>
Note that even though forward slashes / are allowed in URLs, the reason why the template package still encodes them is because it analyses the URL and sees that the value you want to include is the value of a URL parameter (file=XXX), and so it also encodes the slashes (so that everything you pass in will be part of the value of the file URL parameter).
If you plan to acquire this file path at the server side from URL parameters, then what the template package does is the correct and proper way.
But know that by doing this, you'll lose the safety that prevents code injection into URLs. If you're the one providing the values and you know they are safe, there is no problem. But if the data comes from a user input for example, never do this.
Also note that if you pass the whole URL (and not just a part of it), it will work without using template.URL (try this variant on the Go Playground):
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileURL": "/file?file=/location/to/something",
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Also note that the recommended way in my opinion would be to include the file path as part of the URL path and not as the value of a parameter, so instead you should create urls like this:
/file/location/to/something
Map your handler (which serves the file content, see this answer as an example) to the /file/ pattern, and when it is matched and your handler is called, cut off the /file/ prefix from the path r.URL.Path, and the rest will be the full file path. If you choose this, you also won't need the template.URL conversion (because the value you include is not a value of a URL parameter anymore):
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileFullPath": "/location/to/something",
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Try this on the Go Playground.
Also very important: never parse templates in your handler functions! For details see:
It takes too much time when using "template" package to generate a dynamic web page to client in golang
OK, So the solution I've found (and please post if there's a better one) is based on an answer here.
I changed the struct I was using from:
type File struct {
FullFilePath string
FileName string
}
To this:
type File struct {
FullFilePath template.HTML
FileName string
}
And moved the html into the FullFilePath name, and then placed that in template.HTML so each FullFilePath name I was generating was done like so:
file := File{template.HTML("<a href=\"/file?file=" + path + "\"</a>"), f.Name()}
And my template file line was changed to this:
<tr><td>{{.FullFilePath}}{{.FileName}}</td></tr>
I would like a unit test that verifies a particular command line flag is within an enumeration.
Here is the code I would like to write tests against:
var formatType string
const (
text = "text"
json = "json"
hash = "hash"
)
func init() {
const (
defaultFormat = "text"
formatUsage = "desired output format"
)
flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")
}
func main() {
flag.Parse()
}
The desired test would pass only if -format equalled one of the const values given above. This value would be available in formatType. An example correct call would be: program -format text
What is the best way to test the desired behaviors?
Note: Perhaps I have phrased this poorly, but the displayed code it not the unit test itself, but the code I want to write unit tests against. This is a simple example from the tool I am writing and wanted to ask if there were a good way to test valid inputs to the tool.
Custom testing and processing of flags can be achieved with the flag.Var function in the flag package.
Flag.Var "defines a flag with the specified name and usage string. The type and value of the flag are represented by the first argument, of type Value, which typically holds a user-defined implementation of Value."
A flag.Value is any type that satisfies the Value interface, defined as:
type Value interface {
String() string
Set(string) error
}
There is a good example in the example_test.go file in the flag package source
For your use case you could use something like:
package main
import (
"errors"
"flag"
"fmt"
)
type formatType string
func (f *formatType) String() string {
return fmt.Sprint(*f)
}
func (f *formatType) Set(value string) error {
if len(*f) > 0 && *f != "text" {
return errors.New("format flag already set")
}
if value != "text" && value != "json" && value != "hash" {
return errors.New("Invalid Format Type")
}
*f = formatType(value)
return nil
}
var typeFlag formatType
func init() {
typeFlag = "text"
usage := `Format type. Must be "text", "json" or "hash". Defaults to "text".`
flag.Var(&typeFlag, "format", usage)
flag.Var(&typeFlag, "f", usage+" (shorthand)")
}
func main() {
flag.Parse()
fmt.Println("Format type is", typeFlag)
}
This is probably overkill for such a simple example, but may be very useful when defining more complex flag types (The linked example converts a comma separated list of intervals into a slice of a custom type based on time.Duration).
EDIT: In answer to how to run unit tests against flags, the most canonical example is flag_test.go in the flag package source. The section related to testing custom flag variables starts at Line 181.
You can do this
func main() {
var name string
var password string
flag.StringVar(&name, "name", "", "")
flag.StringVar(&password, "password", "", "")
flag.Parse()
for _, v := range os.Args {
fmt.Println(v)
}
if len(strings.TrimSpace(name)) == 0 || len(strings.TrimSpace(password)) == 0 {
log.Panicln("no name or no passward")
}
fmt.Printf("name:%s\n", name)
fmt.Printf("password:%s\n", password)
}
func TestMainApp(t *testing.T) {
os.Args = []string{"test", "-name", "Hello", "-password", "World"}
main()
}
You can test main() by:
Making a test that runs a command
Which then calls the app test binary, built from go test, directly
Passing the desired flags you want to test
Passing back the exit code, stdout, and stderr which you can assert on.
NOTE This only works when main exits, so that the test does not run infinitely, or gets caught in a recursive loop.
Given your main.go looks like:
package main
import (
"flag"
"fmt"
"os"
)
var formatType string
const (
text = "text"
json = "json"
hash = "hash"
)
func init() {
const (
defaultFormat = "text"
formatUsage = "desired output format"
)
flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")
}
func main() {
flag.Parse()
fmt.Printf("format type = %v\n", formatType)
os.Exit(0)
}
Your main_test.go may then look something like:
package main
import (
"fmt"
"os"
"os/exec"
"path"
"runtime"
"strings"
"testing"
)
// This will be used to pass args to app and keep the test framework from looping
const subCmdFlags = "FLAGS_FOR_MAIN"
func TestMain(m *testing.M) {
// Only runs when this environment variable is set.
if os.Getenv(subCmdFlags) != "" {
runAppMain()
}
// Run all tests
exitCode := m.Run()
// Clean up
os.Exit(exitCode)
}
func TestMainForCorrectness(tester *testing.T) {
var tests = []struct {
name string
wantCode int
args []string
}{
{"formatTypeJson", 0, []string{"-format", "json"}},
}
for _, test := range tests {
tester.Run(test.name, func(t *testing.T) {
cmd := getTestBinCmd(test.args)
cmdOut, cmdErr := cmd.CombinedOutput()
got := cmd.ProcessState.ExitCode()
// Debug
showCmdOutput(cmdOut, cmdErr)
if got != test.wantCode {
t.Errorf("unexpected error on exit. want %q, got %q", test.wantCode, got)
}
})
}
}
// private helper methods.
// Used for running the application's main function from other test.
func runAppMain() {
// the test framework has process its flags,
// so now we can remove them and replace them with the flags we want to pass to main.
// we are pulling them out of the environment var we set.
args := strings.Split(os.Getenv(subCmdFlags), " ")
os.Args = append([]string{os.Args[0]}, args...)
// Debug stmt, can be removed
fmt.Printf("\nos args = %v\n", os.Args)
main() // will run and exit, signaling the test framework to stop and return the exit code.
}
// getTestBinCmd return a command to run your app (test) binary directly; `TestMain`, will be run automatically.
func getTestBinCmd(args []string) *exec.Cmd {
// call the generated test binary directly
// Have it the function runAppMain.
cmd := exec.Command(os.Args[0], "-args", strings.Join(args, " "))
// Run in the context of the source directory.
_, filename, _, _ := runtime.Caller(0)
cmd.Dir = path.Dir(filename)
// Set an environment variable
// 1. Only exist for the life of the test that calls this function.
// 2. Passes arguments/flag to your app
// 3. Lets TestMain know when to run the main function.
subEnvVar := subCmdFlags + "=" + strings.Join(args, " ")
cmd.Env = append(os.Environ(), subEnvVar)
return cmd
}
func showCmdOutput(cmdOut []byte, cmdErr error) {
if cmdOut != nil {
fmt.Printf("\nBEGIN sub-command out:\n%v", string(cmdOut))
fmt.Print("END sub-command\n")
}
if cmdErr != nil {
fmt.Printf("\nBEGIN sub-command stderr:\n%v", cmdErr.Error())
fmt.Print("END sub-command\n")
}
}
I'm not sure whether we agree on the term 'unit test'. What you want to achieve seems to me
more like a pretty normal test in a program. You probably want to do something like this:
func main() {
flag.Parse()
if formatType != text || formatType != json || formatType != hash {
flag.Usage()
return
}
// ...
}
Sadly, it is not easily possible to extend the flag Parser with own value verifiers
so you have to stick with this for now.
See Intermernet for a solution which defines a custom format type and its validator.