I am developing web service that will receive JSON. Go converts types too strict.
So I did following function to convert interface{} in bool
func toBool(i1 interface{}) bool {
if i1 == nil {
return false
}
switch i2 := i1.(type) {
default:
return false
case bool:
return i2
case string:
return i2 == "true"
case int:
return i2 != 0
case *bool:
if i2 == nil {
return false
}
return *i2
case *string:
if i2 == nil {
return false
}
return *i2 == "true"
case *int:
if i2 == nil {
return false
}
return *i2 != 0
}
return false
}
I believe that function is still not perfect and I need functions to convert interface{} in string, int, int64, etc
So my question: Is there library (set of functions) in Go that will convert interface{} to certain types
UPDATE
My web service receive JSON. I decode it in map[string]interface{} I do not have control on those who encode it.
So all values I receive are interface{} and I need way to cast it in certain types.
So it could be nil, int, float64, string, [...], {...} and I wish to cast it to what it should be. e.g. int, float64, string, []string, map[string]string with handling of all possible cases including nil, wrong values, etc
UPDATE2
I receive {"s": "wow", "x":123,"y":true}, {"s": 123, "x":"123","y":"true"}, {a:["a123", "a234"]}, {}
var m1 map[string]interface{}
json.Unmarshal(b, &m1)
s := toString(m1["s"])
x := toInt(m1["x"])
y := toBool(m1["y"])
arr := toStringArray(m1["a"])
objx package makes exactly what you want, it can work directly with JSON, and will give you default values and other cool features:
Objx provides the objx.Map type, which is a map[string]interface{}
that exposes a powerful Get method (among others) that allows you to
easily and quickly get access to data within the map, without having
to worry too much about type assertions, missing data, default values
etc.
This is a small example of the usage:
o := objx.New(m1)
s := o.Get("m1").Str()
x := o.Get("x").Int()
y := o.Get("y").Bool()
arr := objx.New(m1["a"])
A example from doc working with JSON:
// use MustFromJSON to make an objx.Map from some JSON
m := objx.MustFromJSON(`{"name": "Mat", "age": 30}`)
// get the details
name := m.Get("name").Str()
age := m.Get("age").Int()
// get their nickname (or use their name if they
// don't have one)
nickname := m.Get("nickname").Str(name)
Obviously you can use something like this with the plain runtime:
switch record[field].(type) {
case int:
value = record[field].(int)
case float64:
value = record[field].(float64)
case string:
value = record[field].(string)
}
But if you check objx accessors you can see a complex code similar to this but with many case of usages, so i think that the best solution is use objx library.
Fast/Best way is 'Cast' in time execution (if you know the object):
E.g.
package main
import "fmt"
func main() {
var inter (interface{})
inter = "hello"
var value string
value = inter.(string)
fmt.Println(value)
}
Try here
I came here trying to convert from interface{} to bool and Reflect gave me a clean way to do it:
Having:
v := interface{}
v = true
The solution 1:
if value, ok := v.(bool); ok {
//you can use variable `value`
}
The solution 2:
reflect.ValueOf(v).Bool()
Then reflect offers a function for the Type you need.
Related
I have this interface for a cache object which is mirroring the lru cache from github.com/hashicorp/golang-lru:
type Cacher interface {
Add(key, value interface{})
Get(key interface{}) (value interface{}, ok bool)
}
In main.go I create the object when some conditions are met otherwise it stays null:
import lru "github.com/hashicorp/golang-lru"
...
var cache *lru.ARCCache
if someCondition {
cache, _ = lru.NewARC(int(cacheSize))
}
... later on
r.Cache = cache
Now, in another package , I check if the cache is nil before doing any operations on it:
if r.Cache != nil {
v, found := r.Cache.Get(...)
}
This leads to a invalid memory address or nil pointer dereference error because the type is not nil but the value is.
My question is how to check if r.Cache is nil without having to import github.com/hashicorp/golang-lru in that package (which makes using the Cacher interface pointless): if r.Cache != (*lru.ARCCache)(nil)
Avoid checking for nil concrete value in interface because a nil value may be a valid implementation of the interface. Here's a somewhat contrived example of where nil is valid:
type exampleCacher struct { }
func (c *exampleCacher) Get(key interface{}) (value interface{}, ok bool) }
if c == nil {
return nil, false
}
...
}
A better fix to the problem is to ensure that the code only assigns valid values to r.Cache.
The code in the question always sets r.Cache to a non-nil value because the code assigns a concrete type to r.Cache. See the FAQ entry on nil error for an explanation.
Fix the problem by declaring cache as a Cacher.
var cache Cacher
As I mention in my comment above, another fix is:
if cache != nil {
r.Cache = cache
}
Just be aware that (*lru.ARCCache)(nil) != Cacher(nil).
So just don't assign (*lru.ARCCache)(nil) to r.Cache.
Here is the fix:
if cache != nil {
r.Cache = cache
}
I'm having a hard time to get the current Cognito user attributes from within my lambda function, that is written in Go. I'm currently doing:
userAttributes = request.RequestContext.Authorizer["claims"]
And if I want to get the email:
userEmail = request.RequestContext.Authorizer["claims"].(map[string]interface{})["email"].(string)
I don't think this is a good way or even an acceptable way - it must have a better way to do it.
You can use 3rd party library to convert map[string]interface{} to a concrete type. Check the mitchellh/mapstructure library, it will help you to implement in a better way.
So, you could improve your code with this code :
import "github.com/mitchellh/mapstructure"
type Claims struct {
Email string
// other fields
ID int
}
func claims(r request.Request) (Claims, error) {
input := r.RequestContext.Authorizer["claims"]
output := Claims{}
err := mapstructure.Decode(input, &output)
if err != nil {
return nil, err
}
return output, nil
}
And somewhere in your handlers, you could get your claims by calling this method
func someWhere(){
userClaims, err := claims(request)
if err != nil {
// handle
}
// you can now use : userClaims.Email, userClaims.ID
}
Don't forget to change func claims request parameter type according to yours (r parameter).
I am trying to learn how to write tests for my code in order to write better code, but I just seem to have the hardest time figuring out how to actually test some code I have written. I have read so many tutorials, most of which seem to only cover functions that add two numbers or mock some database or server.
I have a simple function I wrote below that takes a text template and a CSV file as input and executes the template using the values of the CSV. I have "tested" the code by trial and error, passing files, and printing values, but I would like to learn how to write proper tests for it. I feel that learning to test my own code will help me understand and learn faster and better. Any help is appreciated.
// generateCmds generates configuration commands from a text template using
// the values from a CSV file. Multiple commands in the text template must
// be delimited by a semicolon. The first row of the CSV file is assumed to
// be the header row and the header values are used for key access in the
// text template.
func generateCmds(cmdTmpl string, filename string) ([]string, error) {
t, err := template.New("cmds").Parse(cmdTmpl)
if err != nil {
return nil, fmt.Errorf("parsing template: %v", err)
}
f, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("reading file: %v", err)
}
defer f.Close()
records, err := csv.NewReader(f).ReadAll()
if err != nil {
return nil, fmt.Errorf("reading records: %v", err)
}
if len(records) == 0 {
return nil, errors.New("no records to process")
}
var (
b bytes.Buffer
cmds []string
keys = records[0]
vals = make(map[string]string, len(keys))
)
for _, rec := range records[1:] {
for k, v := range rec {
vals[keys[k]] = v
}
if err := t.Execute(&b, vals); err != nil {
return nil, fmt.Errorf("executing template: %v", err)
}
for _, s := range strings.Split(b.String(), ";") {
if cmd := strings.TrimSpace(s); cmd != "" {
cmds = append(cmds, cmd)
}
}
b.Reset()
}
return cmds, nil
}
Edit: Thanks for all the suggestions so far! My question was flagged as being too broad, so I have some specific questions regarding my example.
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err? For example:
type tmplTest struct {
name string // test name
tmpl string // the text template
filename string // CSV file with template values
expected []string // expected configuration commands
err error // expected error
}
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()? Same idea for template.Parse(), template.Execute(), etc.
Edit 2: Below is a test function I came up with. My two question from the first edit still stand.
package cmd
import (
"testing"
"strings"
"path/filepath"
)
type tmplTest struct {
name string // test name
tmpl string // text template to execute
filename string // CSV containing template text values
cmds []string // expected configuration commands
}
var tests = []tmplTest{
{"empty_error", ``, "", nil},
{"file_error", ``, "fake_file.csv", nil},
{"file_empty_error", ``, "empty.csv", nil},
{"file_fmt_error", ``, "fmt_err.csv", nil},
{"template_fmt_error", `{{ }{{`, "test_values.csv", nil},
{"template_key_error", `{{.InvalidKey}}`, "test_values.csv", nil},
}
func TestGenerateCmds(t *testing.T) {
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cmds, err := generateCmds(tc.tmpl, filepath.Join("testdata", tc.filename))
if err != nil {
// Unexpected error. Fail the test.
if !strings.Contains(tc.name, "error") {
t.Fatal(err)
}
// TODO: Otherwise, check that the function failed at the expected point.
}
if tc.cmds == nil && cmds != nil {
t.Errorf("expected no commands; got %d", len(cmds))
}
if len(cmds) != len(tc.cmds) {
t.Errorf("expected %d commands; got %d", len(tc.cmds), len(cmds))
}
for i := range cmds {
if cmds[i] != tc.cmds[i] {
t.Errorf("expected %q; got %q", tc.cmds[i], cmds[i])
}
}
})
}
}
You basically need to have some sample files with the contents you want to test, then in your test code you can call the generateCmds function passing in the template string and the files to then verify that the results are what you expect.
It is not so much different as the examples you probably saw for simpler cases.
You can place the files under a testdata folder inside the same package (testdata is a special name that the Go tools will ignore during build).
Then you can do something like:
func TestCSVProcessing(t *testing.T) {
templateStr := `<your template here>`
testFile := "testdata/yourtestfile.csv"
result, err := generateCmds(templateStr, testFile)
if err != nil {
// fail the test here, unless you expected an error with this file
}
// compare the "result" contents with what you expected
// failing the test if it does not match
}
EDIT
About the specific questions you added later:
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err?
Yes, it'd make sense to include both the expected strings to be returned as well as the expected error (if any).
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()?
I don't think you'll be able to "initialize" an equivalent error for each case. Sometimes the libraries might use internal types for their errors making this impossible. Easiest would be to "initialize" a regular error with the same value returned in its Error() method, then just compare the returned error's Error() value with the expected one.
Let's imagine I have a map: map[string]string. I would like to get the list of sorted keys for this map. So I could do something like this:
func SortedMapKeys(m map[string]string) (keyList []string) {
for key := range m {
keyList = append(keyList, key)
}
sort.Strings(keyList)
return
}
Then I will have another map of type map[string]bool. I would like to get it's keys also. But the problem is that function SortedMapKeys accepts a map[string]string argument. So I need to write exactly the same function, with the only one difference - it will accept a map[string]bool.
For obvious reasons it's not an option. If one day I would like to change the logic of how I get and sort my keys, I will need to track and update all these functions. Also, I will have to write the same unit tests for all these functions which actually do the same thing since their bodies are 100% equal (code duplicate).
Is there any way to create a generic function which could accept an map[string] of whatever?
Since map[string]bool and map[string]string and map[string]Whatever are all distinct types, the only way to create a single function to sort the keys of all possible map[string]* types is via reflection.
func SortedMapKeys(m interface{}) (keyList []string) {
keys := reflect.ValueOf(m).MapKeys()
for _, key := range keys {
keyList = append(keyList, key.Interface().(string))
}
sort.Strings(keyList)
return
}
For an in-between solution, since there are probably only a few combinations of types you're concerned with, you can use a type switch to extract the keys
func SortedMapKeys(m interface{}) (keyList []string) {
switch m := m.(type) {
case map[string]string:
for k := range m {
keyList = append(keyList, k)
}
case map[string]bool:
for k := range m {
keyList = append(keyList, k)
}
default:
panic("unknown map type")
}
sort.Strings(keyList)
return
}
Here's my 0.02$. Since keys extracting logic is unlikely to change and you want to keep everything in one place you can create variant and choose non-nil map from it:
type MapVariant struct {
Bool map[string]bool
String map[string]string
}
func SortedMapKeys(variant MapVariant) (keyList []string) {
if variant.String != nil {
for k := range variant.String {
keyList = append(keyList, k)
}
goto SORT
}
if variant.Bool != nil {
for k := range variant.Bool {
keyList = append(keyList, k)
}
goto SORT
}
SORT:
sort.Strings(keyList)
return
}
Of course you can avoid goto statements by adding more conditions but I personally find it clearer.
Then you can use the function like:
SortedMapKeys(MapVariant{
Bool: map[string]bool{"a": true, "b": false}
})
SortedMapKeys(MapVariant{
String: map[string]string{"c": "v1", "b": "v2"}
})
I would like a unit test that verifies a particular command line flag is within an enumeration.
Here is the code I would like to write tests against:
var formatType string
const (
text = "text"
json = "json"
hash = "hash"
)
func init() {
const (
defaultFormat = "text"
formatUsage = "desired output format"
)
flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")
}
func main() {
flag.Parse()
}
The desired test would pass only if -format equalled one of the const values given above. This value would be available in formatType. An example correct call would be: program -format text
What is the best way to test the desired behaviors?
Note: Perhaps I have phrased this poorly, but the displayed code it not the unit test itself, but the code I want to write unit tests against. This is a simple example from the tool I am writing and wanted to ask if there were a good way to test valid inputs to the tool.
Custom testing and processing of flags can be achieved with the flag.Var function in the flag package.
Flag.Var "defines a flag with the specified name and usage string. The type and value of the flag are represented by the first argument, of type Value, which typically holds a user-defined implementation of Value."
A flag.Value is any type that satisfies the Value interface, defined as:
type Value interface {
String() string
Set(string) error
}
There is a good example in the example_test.go file in the flag package source
For your use case you could use something like:
package main
import (
"errors"
"flag"
"fmt"
)
type formatType string
func (f *formatType) String() string {
return fmt.Sprint(*f)
}
func (f *formatType) Set(value string) error {
if len(*f) > 0 && *f != "text" {
return errors.New("format flag already set")
}
if value != "text" && value != "json" && value != "hash" {
return errors.New("Invalid Format Type")
}
*f = formatType(value)
return nil
}
var typeFlag formatType
func init() {
typeFlag = "text"
usage := `Format type. Must be "text", "json" or "hash". Defaults to "text".`
flag.Var(&typeFlag, "format", usage)
flag.Var(&typeFlag, "f", usage+" (shorthand)")
}
func main() {
flag.Parse()
fmt.Println("Format type is", typeFlag)
}
This is probably overkill for such a simple example, but may be very useful when defining more complex flag types (The linked example converts a comma separated list of intervals into a slice of a custom type based on time.Duration).
EDIT: In answer to how to run unit tests against flags, the most canonical example is flag_test.go in the flag package source. The section related to testing custom flag variables starts at Line 181.
You can do this
func main() {
var name string
var password string
flag.StringVar(&name, "name", "", "")
flag.StringVar(&password, "password", "", "")
flag.Parse()
for _, v := range os.Args {
fmt.Println(v)
}
if len(strings.TrimSpace(name)) == 0 || len(strings.TrimSpace(password)) == 0 {
log.Panicln("no name or no passward")
}
fmt.Printf("name:%s\n", name)
fmt.Printf("password:%s\n", password)
}
func TestMainApp(t *testing.T) {
os.Args = []string{"test", "-name", "Hello", "-password", "World"}
main()
}
You can test main() by:
Making a test that runs a command
Which then calls the app test binary, built from go test, directly
Passing the desired flags you want to test
Passing back the exit code, stdout, and stderr which you can assert on.
NOTE This only works when main exits, so that the test does not run infinitely, or gets caught in a recursive loop.
Given your main.go looks like:
package main
import (
"flag"
"fmt"
"os"
)
var formatType string
const (
text = "text"
json = "json"
hash = "hash"
)
func init() {
const (
defaultFormat = "text"
formatUsage = "desired output format"
)
flag.StringVar(&formatType, "format", defaultFormat, formatUsage)
flag.StringVar(&formatType, "f", defaultFormat, formatUsage+" (shorthand)")
}
func main() {
flag.Parse()
fmt.Printf("format type = %v\n", formatType)
os.Exit(0)
}
Your main_test.go may then look something like:
package main
import (
"fmt"
"os"
"os/exec"
"path"
"runtime"
"strings"
"testing"
)
// This will be used to pass args to app and keep the test framework from looping
const subCmdFlags = "FLAGS_FOR_MAIN"
func TestMain(m *testing.M) {
// Only runs when this environment variable is set.
if os.Getenv(subCmdFlags) != "" {
runAppMain()
}
// Run all tests
exitCode := m.Run()
// Clean up
os.Exit(exitCode)
}
func TestMainForCorrectness(tester *testing.T) {
var tests = []struct {
name string
wantCode int
args []string
}{
{"formatTypeJson", 0, []string{"-format", "json"}},
}
for _, test := range tests {
tester.Run(test.name, func(t *testing.T) {
cmd := getTestBinCmd(test.args)
cmdOut, cmdErr := cmd.CombinedOutput()
got := cmd.ProcessState.ExitCode()
// Debug
showCmdOutput(cmdOut, cmdErr)
if got != test.wantCode {
t.Errorf("unexpected error on exit. want %q, got %q", test.wantCode, got)
}
})
}
}
// private helper methods.
// Used for running the application's main function from other test.
func runAppMain() {
// the test framework has process its flags,
// so now we can remove them and replace them with the flags we want to pass to main.
// we are pulling them out of the environment var we set.
args := strings.Split(os.Getenv(subCmdFlags), " ")
os.Args = append([]string{os.Args[0]}, args...)
// Debug stmt, can be removed
fmt.Printf("\nos args = %v\n", os.Args)
main() // will run and exit, signaling the test framework to stop and return the exit code.
}
// getTestBinCmd return a command to run your app (test) binary directly; `TestMain`, will be run automatically.
func getTestBinCmd(args []string) *exec.Cmd {
// call the generated test binary directly
// Have it the function runAppMain.
cmd := exec.Command(os.Args[0], "-args", strings.Join(args, " "))
// Run in the context of the source directory.
_, filename, _, _ := runtime.Caller(0)
cmd.Dir = path.Dir(filename)
// Set an environment variable
// 1. Only exist for the life of the test that calls this function.
// 2. Passes arguments/flag to your app
// 3. Lets TestMain know when to run the main function.
subEnvVar := subCmdFlags + "=" + strings.Join(args, " ")
cmd.Env = append(os.Environ(), subEnvVar)
return cmd
}
func showCmdOutput(cmdOut []byte, cmdErr error) {
if cmdOut != nil {
fmt.Printf("\nBEGIN sub-command out:\n%v", string(cmdOut))
fmt.Print("END sub-command\n")
}
if cmdErr != nil {
fmt.Printf("\nBEGIN sub-command stderr:\n%v", cmdErr.Error())
fmt.Print("END sub-command\n")
}
}
I'm not sure whether we agree on the term 'unit test'. What you want to achieve seems to me
more like a pretty normal test in a program. You probably want to do something like this:
func main() {
flag.Parse()
if formatType != text || formatType != json || formatType != hash {
flag.Usage()
return
}
// ...
}
Sadly, it is not easily possible to extend the flag Parser with own value verifiers
so you have to stick with this for now.
See Intermernet for a solution which defines a custom format type and its validator.