The sdk by default marshals time.Time values as RFC3339 strings. How can you choose to marshal and unmarshal in other ways e.g. millis since epoch?
The SDK mentions the Marshaler and Unmarshaler interfaces but does not explain how to use them.
(As I was about to post my question I figured out the answer by looking into how UnixTime worked).
To use a custom Marshaler and Unmarshaler you can create a custom type.
type MillisTime time.Time
func (e MillisTime) MarshalDynamoDBAttributeValue(av *dynamodb.AttributeValue) error {
millis := timeAsMillis(time.Time(e))
millisStr := fmt.Sprintf("%d", millis)
av.N = &millisStr
return nil
}
func (e *MillisTime) UnmarshalDynamoDBAttributeValue(av *dynamodb.AttributeValue) error {
millis, err := strconv.ParseInt(*av.N, 10, 0)
if err != nil {
return err
}
*e = MillisTime(millisAsTime(millis))
return nil
}
func timeAsMillis(t time.Time) int64 {
nanosSinceEpoch := t.UnixNano()
return (nanosSinceEpoch / 1_000_000_000) + (nanosSinceEpoch % 1_000_000_000)
}
func millisAsTime(millis int64) time.Time {
seconds := millis / 1_000
nanos := (millis % 1_000) * 1_000_000
return time.Unix(seconds, nanos)
}
NOTE: Example above uses the new number literal syntax introduced in go 1.13.
You can easily marshal and unmarshal structs using MarshalMap and UnmarshalMap but the downside is that the fields in your struct type have to use MillisTime instead of time.Time. Conversion is not easy but is possible.
The SDK defines a UnixTime type which will handle marshaling and unmarshaling between time.Time <=> seconds since epoch.
Related
This question already has answers here:
Is there an easy way to stub out time.Now() globally during test?
(10 answers)
Golang testing programs that involves time
(1 answer)
Closed 21 days ago.
I have a function that is calculating the number of days since a particular timestamp, where the timestamp is coming from an external API (parsed as string in json return from API)
I have been following this article on how to test functions that use time.Now():
https://medium.com/go-for-punks/how-to-test-functions-that-use-time-now-ea4f2453d430
My function looks like this:
type funcTimeType func() time.Time // per suggested in article
func ageOfReportDays(dateString string, funcTime funcTimeType) {
// date string will look like this:
//"2022-08-30 09:05:27.567995"
parseLayout := "2006-01-02 15:04:05.000000"
t, err := time.Parse(parseLayout, dateString)
if err != nil {
fmt.Printf("Error parsing datetime value %v: %w", timeStr, err)
}
days := int(time.Since(t).Abs().Hours() / 24)
//fmt.Println(days)
return days, nil
}
As you can see, I am not using the funcTime funcTimeType in my actual function, as indicated in the article, because I cannot figure out how my function would be implemented with that.
The unit test I would hope to run would be something like this:
func Test_ageOfReportDays(t *testing.T) {
t.Run("timestamp age in days test", func(t *testing.T) {
parseLayout := "2006-01-02 15:04:05.000000"
dateString := "2022-08-30 09:05:27.567995" // example of recent timestamp
mockNow := func() time.Time {
fakeTime, _ := time.Parse(parseLayout, "2023-01-20 09:00:00.000000")
return fakeTime
}
// now I want to use "fakeTime" to spoof "time.Now()" so I can test my function
got: ageOfReportDays(dateString, mockNow)
expected: 152
if got != expected {
t.Errorf("expected '%d' but got '%d'", expected, got)
}
}
Obviously the logic is not quite with my code vs article author's code.
Is there a good way for me to write a unit test for this funcition, based on how the article is suggesting to mock time.Now()?
You are pretty close. Changing time.Since(t) to funcTime().Sub(t) would probably get you passed the finish line.
From time package docs:
time.Since returns the time elapsed since t. It is shorthand for time.Now().Sub(t).
Example function:
import (
"fmt"
"time"
)
const parseLayout = "2006-01-02 15:04:05.000000"
type funcTimeType func() time.Time // per suggested in article
func ageOfReportDays(dateString string, funcTime funcTimeType) (int, error) {
t, err := time.Parse(parseLayout, dateString)
if err != nil {
return 0, fmt.Errorf("parsing datetime value %v: %w", dateString, err)
}
days := int(funcTime().Sub(t).Hours() / 24)
//fmt.Println(days)
return days, nil
}
And a test:
import (
"testing"
"time"
)
func Test_ageOfReportDays(t *testing.T) {
t.Run("timestamp age in days test", func(t *testing.T) {
dateString := "2022-08-30 09:05:27.567995" // example of recent timestamp
mockNow := func() time.Time {
fakeTime, _ := time.Parse(parseLayout, "2023-01-20 09:00:00.000000")
return fakeTime
}
// now I want to use "fakeTime" to spoof "time.Now()" so I can test my function
got, _ := ageOfReportDays(dateString, mockNow)
expected := 142
if got != expected {
t.Errorf("expected '%d' but got '%d'", expected, got)
}
})
}
I'm having a hard time to get the current Cognito user attributes from within my lambda function, that is written in Go. I'm currently doing:
userAttributes = request.RequestContext.Authorizer["claims"]
And if I want to get the email:
userEmail = request.RequestContext.Authorizer["claims"].(map[string]interface{})["email"].(string)
I don't think this is a good way or even an acceptable way - it must have a better way to do it.
You can use 3rd party library to convert map[string]interface{} to a concrete type. Check the mitchellh/mapstructure library, it will help you to implement in a better way.
So, you could improve your code with this code :
import "github.com/mitchellh/mapstructure"
type Claims struct {
Email string
// other fields
ID int
}
func claims(r request.Request) (Claims, error) {
input := r.RequestContext.Authorizer["claims"]
output := Claims{}
err := mapstructure.Decode(input, &output)
if err != nil {
return nil, err
}
return output, nil
}
And somewhere in your handlers, you could get your claims by calling this method
func someWhere(){
userClaims, err := claims(request)
if err != nil {
// handle
}
// you can now use : userClaims.Email, userClaims.ID
}
Don't forget to change func claims request parameter type according to yours (r parameter).
I'm trying to test one of my package to reach 100%.
However, I can't find how I can do this without being "against the system" (functions pointers, etc.).
I tried to do something similar to this, but I can't reach 100% because of "real" functions :
var fs fileSystem = osFS{}
type fileSystem interface {
Open(name string) (file, error)
Stat(name string) (os.FileInfo, error)
}
type file interface {
io.Closer
io.Reader
io.ReaderAt
io.Seeker
Stat() (os.FileInfo, error)
}
// osFS implements fileSystem using the local disk.
type osFS struct{}
func (osFS) Open(name string) (file, error) { return os.Open(name) }
func (osFS) Stat(name string) (os.FileInfo, error) { return os.Stat(name) }
(From https://talks.golang.org/2012/10things.slide#8)
Does someone would have a suggestion ? :)
Thanks !
I attempted to do same thing, just to try it. I achieved it by referencing all system file calls as interfaces and having method accept an interface. Then if no interface was provided the system method was used. I am brand new to Go, so I'm not sure if it violates best practices or not.
import "io/ioutil"
type ReadFile func (string) ([]byte, error)
type FileLoader interface {
LoadPath(path string) []byte
}
// Initializes a LocalFileSystemLoader with default ioutil.ReadFile
// as the method to read file. Optionally allows caller to provide
// their own ReadFile for testing.
func NewLocalFileSystemLoader(rffn ReadFile) *localFileSystemLoader{
var rf ReadFile = ioutil.ReadFile
if rffn != nil {
rf = rffn
}
return &localFileSystemLoader{
ReadFileFn: rf}
}
type localFileSystemLoader struct {
ReadFileFn ReadFile
}
func (lfsl *localFileSystemLoader) LoadPath(path string) []byte {
dat, err := lfsl.ReadFileFn(path)
if err != nil {
panic(err)
}
return dat
}
I'd like to marshal a variety of objects to file, then unmarshal them, and convert them back to their original type by getting the type of the variables which were marshalled.
The key point is that I'd like to convert the unmarshalled object to the type of a specified variable, without specifying the type.
The short pseudocode:
// Marshal this
item := Book{"The Myth of Sisyphus", "Albert Camus"}
// Then unmarshal and convert to the type of the item variable.
itemType := reflect.TypeOf(item)
newItem itemType = unmarshalledItem.(itemType) // This is the problem.
fmt.Println("Unmarshalled is:", reflect.TypeOf(newItem)) // Should print *main.Book
The full code:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"reflect"
)
type Book struct {
Title string
Author string
}
func main() {
// Create objects to marshal.
book := Book{"The Myth of Sisyphus", "Albert Camus"}
box := make(map[string]interface{})
box["The Myth of Sisyphus"] = &book
itemType := reflect.TypeOf(box["The Myth of Sisyphus"])
fmt.Println("Book is:", itemType)
// Marshal objects to file.
err := Write(&book)
if err != nil {
fmt.Println("Unable to save store.", err)
return
}
// Unmarshal objects from file.
untyped := make(map[string]interface{})
bytes, err := ioutil.ReadFile("store.txt")
if err != nil {
fmt.Println("Unable to load store.", err)
return
}
err = json.Unmarshal(bytes, &untyped)
if err != nil {
fmt.Println("Err in store unmarshal.", err)
return
}
// Get Title property of unmarshalled object,
// and use that to get variable type from box map.
for k, v := range untyped {
if k == "Title" {
itemTitle := v.(string)
fmt.Println("Cast item having title:", itemTitle)
targetType := reflect.TypeOf(box[itemTitle])
fmt.Println("Type to cast to is:", targetType)
// Convert untyped to targetType.
// This is the problem.
typed targetType = untyped.(targetType)
fmt.Println("Unmarshalled is:", reflect.TypeOf(typed)) // Should print *main.Book
}
}
}
func Write(b *Book) error {
data, err := json.Marshal(b)
if err != nil {
return err
}
newFilename := "store.txt"
f, err := os.OpenFile(newFilename, os.O_CREATE|os.O_TRUNC, 0660)
if err != nil {
return err
}
_, err = f.WriteString(string(data) + "\n")
if err != nil {
return err
}
return nil
}
This might work in a dynamically typed language, but it wont work here because Go is statically typed.
The book is not stored as a Book, its stored as a json string and the json unmarshaller has no idea that its a Book unless you tell it so. i.e. it doesn't know to map the fields to a Book object.
You can't cast the unmarshalled 'untyped' into a Book because it is not a Book it is a map[string]interface{} which happens to look exactly like a Book.
What you would need to do is
unmarshal the json into a map[string]interface{} then read the title.
use an if statement or a switch statement based on the type
unmarshal the json again into an object of that type (e.g. a Book)
Something like this I guess:
// check the type
if targetType.String() == "*main.Book" {
// unmarshall it again as a Book
var typedBook Book
_ = json.Unmarshal(bytes, &typedBook)
fmt.Println("Unmarshalled is:", reflect.TypeOf(typedBook)) // Should print *main.Book
} else if targetType.String() == "*main.Magazine" {
// unmarshal a magazine or whatever
}
I am developing web service that will receive JSON. Go converts types too strict.
So I did following function to convert interface{} in bool
func toBool(i1 interface{}) bool {
if i1 == nil {
return false
}
switch i2 := i1.(type) {
default:
return false
case bool:
return i2
case string:
return i2 == "true"
case int:
return i2 != 0
case *bool:
if i2 == nil {
return false
}
return *i2
case *string:
if i2 == nil {
return false
}
return *i2 == "true"
case *int:
if i2 == nil {
return false
}
return *i2 != 0
}
return false
}
I believe that function is still not perfect and I need functions to convert interface{} in string, int, int64, etc
So my question: Is there library (set of functions) in Go that will convert interface{} to certain types
UPDATE
My web service receive JSON. I decode it in map[string]interface{} I do not have control on those who encode it.
So all values I receive are interface{} and I need way to cast it in certain types.
So it could be nil, int, float64, string, [...], {...} and I wish to cast it to what it should be. e.g. int, float64, string, []string, map[string]string with handling of all possible cases including nil, wrong values, etc
UPDATE2
I receive {"s": "wow", "x":123,"y":true}, {"s": 123, "x":"123","y":"true"}, {a:["a123", "a234"]}, {}
var m1 map[string]interface{}
json.Unmarshal(b, &m1)
s := toString(m1["s"])
x := toInt(m1["x"])
y := toBool(m1["y"])
arr := toStringArray(m1["a"])
objx package makes exactly what you want, it can work directly with JSON, and will give you default values and other cool features:
Objx provides the objx.Map type, which is a map[string]interface{}
that exposes a powerful Get method (among others) that allows you to
easily and quickly get access to data within the map, without having
to worry too much about type assertions, missing data, default values
etc.
This is a small example of the usage:
o := objx.New(m1)
s := o.Get("m1").Str()
x := o.Get("x").Int()
y := o.Get("y").Bool()
arr := objx.New(m1["a"])
A example from doc working with JSON:
// use MustFromJSON to make an objx.Map from some JSON
m := objx.MustFromJSON(`{"name": "Mat", "age": 30}`)
// get the details
name := m.Get("name").Str()
age := m.Get("age").Int()
// get their nickname (or use their name if they
// don't have one)
nickname := m.Get("nickname").Str(name)
Obviously you can use something like this with the plain runtime:
switch record[field].(type) {
case int:
value = record[field].(int)
case float64:
value = record[field].(float64)
case string:
value = record[field].(string)
}
But if you check objx accessors you can see a complex code similar to this but with many case of usages, so i think that the best solution is use objx library.
Fast/Best way is 'Cast' in time execution (if you know the object):
E.g.
package main
import "fmt"
func main() {
var inter (interface{})
inter = "hello"
var value string
value = inter.(string)
fmt.Println(value)
}
Try here
I came here trying to convert from interface{} to bool and Reflect gave me a clean way to do it:
Having:
v := interface{}
v = true
The solution 1:
if value, ok := v.(bool); ok {
//you can use variable `value`
}
The solution 2:
reflect.ValueOf(v).Bool()
Then reflect offers a function for the Type you need.