I'm having a hard time to get the current Cognito user attributes from within my lambda function, that is written in Go. I'm currently doing:
userAttributes = request.RequestContext.Authorizer["claims"]
And if I want to get the email:
userEmail = request.RequestContext.Authorizer["claims"].(map[string]interface{})["email"].(string)
I don't think this is a good way or even an acceptable way - it must have a better way to do it.
You can use 3rd party library to convert map[string]interface{} to a concrete type. Check the mitchellh/mapstructure library, it will help you to implement in a better way.
So, you could improve your code with this code :
import "github.com/mitchellh/mapstructure"
type Claims struct {
Email string
// other fields
ID int
}
func claims(r request.Request) (Claims, error) {
input := r.RequestContext.Authorizer["claims"]
output := Claims{}
err := mapstructure.Decode(input, &output)
if err != nil {
return nil, err
}
return output, nil
}
And somewhere in your handlers, you could get your claims by calling this method
func someWhere(){
userClaims, err := claims(request)
if err != nil {
// handle
}
// you can now use : userClaims.Email, userClaims.ID
}
Don't forget to change func claims request parameter type according to yours (r parameter).
Related
I am trying to learn how to write tests for my code in order to write better code, but I just seem to have the hardest time figuring out how to actually test some code I have written. I have read so many tutorials, most of which seem to only cover functions that add two numbers or mock some database or server.
I have a simple function I wrote below that takes a text template and a CSV file as input and executes the template using the values of the CSV. I have "tested" the code by trial and error, passing files, and printing values, but I would like to learn how to write proper tests for it. I feel that learning to test my own code will help me understand and learn faster and better. Any help is appreciated.
// generateCmds generates configuration commands from a text template using
// the values from a CSV file. Multiple commands in the text template must
// be delimited by a semicolon. The first row of the CSV file is assumed to
// be the header row and the header values are used for key access in the
// text template.
func generateCmds(cmdTmpl string, filename string) ([]string, error) {
t, err := template.New("cmds").Parse(cmdTmpl)
if err != nil {
return nil, fmt.Errorf("parsing template: %v", err)
}
f, err := os.Open(filename)
if err != nil {
return nil, fmt.Errorf("reading file: %v", err)
}
defer f.Close()
records, err := csv.NewReader(f).ReadAll()
if err != nil {
return nil, fmt.Errorf("reading records: %v", err)
}
if len(records) == 0 {
return nil, errors.New("no records to process")
}
var (
b bytes.Buffer
cmds []string
keys = records[0]
vals = make(map[string]string, len(keys))
)
for _, rec := range records[1:] {
for k, v := range rec {
vals[keys[k]] = v
}
if err := t.Execute(&b, vals); err != nil {
return nil, fmt.Errorf("executing template: %v", err)
}
for _, s := range strings.Split(b.String(), ";") {
if cmd := strings.TrimSpace(s); cmd != "" {
cmds = append(cmds, cmd)
}
}
b.Reset()
}
return cmds, nil
}
Edit: Thanks for all the suggestions so far! My question was flagged as being too broad, so I have some specific questions regarding my example.
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err? For example:
type tmplTest struct {
name string // test name
tmpl string // the text template
filename string // CSV file with template values
expected []string // expected configuration commands
err error // expected error
}
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()? Same idea for template.Parse(), template.Execute(), etc.
Edit 2: Below is a test function I came up with. My two question from the first edit still stand.
package cmd
import (
"testing"
"strings"
"path/filepath"
)
type tmplTest struct {
name string // test name
tmpl string // text template to execute
filename string // CSV containing template text values
cmds []string // expected configuration commands
}
var tests = []tmplTest{
{"empty_error", ``, "", nil},
{"file_error", ``, "fake_file.csv", nil},
{"file_empty_error", ``, "empty.csv", nil},
{"file_fmt_error", ``, "fmt_err.csv", nil},
{"template_fmt_error", `{{ }{{`, "test_values.csv", nil},
{"template_key_error", `{{.InvalidKey}}`, "test_values.csv", nil},
}
func TestGenerateCmds(t *testing.T) {
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cmds, err := generateCmds(tc.tmpl, filepath.Join("testdata", tc.filename))
if err != nil {
// Unexpected error. Fail the test.
if !strings.Contains(tc.name, "error") {
t.Fatal(err)
}
// TODO: Otherwise, check that the function failed at the expected point.
}
if tc.cmds == nil && cmds != nil {
t.Errorf("expected no commands; got %d", len(cmds))
}
if len(cmds) != len(tc.cmds) {
t.Errorf("expected %d commands; got %d", len(tc.cmds), len(cmds))
}
for i := range cmds {
if cmds[i] != tc.cmds[i] {
t.Errorf("expected %q; got %q", tc.cmds[i], cmds[i])
}
}
})
}
}
You basically need to have some sample files with the contents you want to test, then in your test code you can call the generateCmds function passing in the template string and the files to then verify that the results are what you expect.
It is not so much different as the examples you probably saw for simpler cases.
You can place the files under a testdata folder inside the same package (testdata is a special name that the Go tools will ignore during build).
Then you can do something like:
func TestCSVProcessing(t *testing.T) {
templateStr := `<your template here>`
testFile := "testdata/yourtestfile.csv"
result, err := generateCmds(templateStr, testFile)
if err != nil {
// fail the test here, unless you expected an error with this file
}
// compare the "result" contents with what you expected
// failing the test if it does not match
}
EDIT
About the specific questions you added later:
Would a test table be useful in a function like this? And, if so, would the test struct need to include the returned cmds string slice and the value of err?
Yes, it'd make sense to include both the expected strings to be returned as well as the expected error (if any).
How do you handle errors that are supposed to be returned for specific test cases? For example, os.Open() returns an error of type *PathError if an error is encountered. How do I initialize a *PathError that is equivalent to the one returned by os.Open()?
I don't think you'll be able to "initialize" an equivalent error for each case. Sometimes the libraries might use internal types for their errors making this impossible. Easiest would be to "initialize" a regular error with the same value returned in its Error() method, then just compare the returned error's Error() value with the expected one.
I'm currently looking into creating some unit tests for my service in Go, as well as other functions that build up on top of that functionality, and I'm wondering what is the best way to unit test that in Go? My code looks like:
type BBPeripheral struct {
client *http.Client
endpoint string
}
type BBQuery struct {
Name string `json:"name"`
}
type BBResponse struct {
Brand string `json:"brand"`
Model string `json:"model"`
...
}
type Peripheral struct {
Brand string
Model string
...
}
type Service interface {
Get(name string) (*Peripheral, error)
}
func NewBBPeripheral(config *peripheralConfig) (*BBPeripheral, error) {
transport, err := setTransport(config)
if err != nil {
return nil, err
}
BB := &BBPeripheral{
client: &http.Client{Transport: transport},
endpoint: config.Endpoint[0],
}
return BB, nil
}
func (this *BBPeripheral) Get(name string) (*Peripheral, error) {
data, err := json.Marshal(BBQuery{Name: name})
if err != nil {
return nil, fmt.Errorf("BBPeripheral.Get Marshal: %s", err)
}
resp, err := this.client.Post(this.endpoint, "application/json", bytes.NewBuffer(data))
if resp != nil {
defer resp.Body.Close()
}
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf(resp.StatusCode)
}
var BBResponse BBResponse
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
err = json.Unmarshal(body, &BBResponse)
if err != nil {
return nil, err
}
peripheral := &Peripheral{}
peripheral.Model = BBResponse.Model
if peripheral.Model == "" {
peripheral.Model = NA
}
peripheral.Brand = BBResponse.Brand
if peripheral.Brand == "" {
peripheral.Brand = NA
}
return peripheral, nil
}
Is the most efficient way of testing this code and the code that uses these functions to spin up a separate goroutine to act like the server, use http.httptest package, or something else? that's the first time that i try to write a test then i don't realy know how.
It really completely depends. Go provides pretty much all the tools you need to test your application at every single level.
Unit Tests
Design is important because there aren't many tricks to dynamically provide mock/stub objects. You can override variables for tests, but it unlocks all sorts of problems with cleanup. I would focus on IO free unit tests to check that your specific logic works.
For example, you could test BBPeripheral.Get method by making client an interface, requiring it during instantiation, and providing a stub one for the test.
func Test_BBPeripheral_Get_Success(*testing.T) {
bb := BBPeripheral{client: &StubSuccessClient, ...}
p, err := bb.Get(...)
if err != nil {
t.Fail()
}
}
Then you could create a stub error client that exercises error handling in the Get method:
func Test_BBPeripheral_Get_Success(*testing.T) {
bb := BBPeripheral{client: &StubErrClient, ...}
_, err := bb.Get(...)
if err == nil {
t.Fail()
}
}
Component/Integration Tests
These tests can help exercise that each individual unit in your package can work together in unison. Since your code talks over http, Go provides the httptest package that could be used.
To do this the test could create an httptest server with a handler registered to provide the response that this.endpoint expects. You could then exercise your code using its public interface by requesting a NewBBPeripheral, passing in this.endpoint corresponding to the Server.URL property.
This allows you to simulate your code talking to a real server.
Go Routine Tests
Go makes it so easy to write concurrent code, and makes it just as easy to test it. Testing the top level code that spawns a go routine that exercises NewBBPeripheral could look very much like the test above. In addition to starting up a test server your test will have to wait your your asynchronous code to complete. If you don't have a service wide way to cancel/shutdown/signal complete then one may be required to test it using go routines.
RaceCondition/Load Testing
Using go's built in bechmark test combined with -race flag, you can easily exercise your code, and profile it for race conditions, leveraging the tests you wrote above.
One thing to keep in mind, if the implementation of your application is still in flux, writing unit tests may cost a large amount of time. Creating a couple tests, which exercise the public interface of your code, should allow you to easily verify that your application is working, while allowing the implementation to change.
I need to create JSON data using few keys of map and need to incorporate into html generated. I am using pongo2 library and want to write custom filter to achieve the same.
<script> {{ CategoryMapping|MycustomFilter }} </script>
and coded custom filter like below.
func init() {
pongo2.RegisterFilter("superfilter", GetCategoryJsonData)
}
func GetCategoryJsonData(CatAttributeMapping *map[string]interface{}, param *int) (*string, *error) {
.....
}
But I am getting below error.
src/util/TemplateFilters.go:10: cannot use GetCategoryJsonData (type func(*int, *int) (*string, *error)) as type pongo2.FilterFunction in argument to pongo2.RegisterFilter
I am following below documentation - https://godoc.org/github.com/flosch/pongo2#FilterFunction
I am new to go and unable to understand what wrong I am doing here. Please guide me for the same.
The problem is that your filter function does not accept or return the right types to match what pongo2 is requiring. Let's walk through the docs and see what they want.
First, take a look at the godoc for RegisterFilterFunction. It says
func RegisterFilter(name string, fn FilterFunction)
This is in the pongo2 package so you should read this as RegisterFilter is a function that accepts two arguments and returns no values. The first argument name is of the builtin type string and the second argument fn is of the type pongo2.FilterFunction. But what is a pongo2.FilterFunction? Well clicking on it we see further down in the doc
type FilterFunction func(in *Value, param *Value) (out *Value, err *Error)
In Go you can make your own types based on any other types including functions. So what pongo2 has done is to create a named type called FilterFunction that is any func which accepts two arguments (both of type *pongo2.Value) and returns two values (one of type *pongo2.value and one of type *pongo2.Error).
To bring it all together we would do something like this:
package main
import (
"fmt"
"log"
"strings"
"github.com/flosch/pongo2"
)
func init() {
pongo2.RegisterFilter("scream", Scream)
}
// Scream is a silly example of a filter function that upper cases strings
func Scream(in *pongo2.Value, param *pongo2.Value) (out *pongo2.Value, err *pongo2.Error) {
if !in.IsString() {
return nil, &pongo2.Error{
ErrorMsg: "only strings should be sent to the scream filter",
}
}
s := in.String()
s = strings.ToUpper(s)
return pongo2.AsValue(s), nil
}
func main() {
tpl, err := pongo2.FromString("Hello {{ name|scream }}!")
if err != nil {
log.Fatal(err)
}
// Now you can render the template with the given
// pongo2.Context how often you want to.
out, err := tpl.Execute(pongo2.Context{"name": "stack overflow"})
if err != nil {
log.Fatal(err)
}
fmt.Println(out) // Output: Hello STACK OVERFLOW!
}
I'm working on converting a pet project of mine from Python to Go just to help me get a bit familiar with the language. An issue I am currently facing is that it's escaping my forward slashes. So it will receive a string like:
/location/to/something
and it then becomes
%2flocation%2fto%2fsomething
Now, it's only doing this when it's in a link (from what I've been reading this escaping is contextual) so this is what the line in the HTML template looks like:
<tr><td>{{.FileName}}</td></tr>
If possible, how can I prevent this in either the template or the code itself?
This is what my templating function looks like (yes, I know it's hackish)
func renderTemplate(w http.ResponseWriter, tmpl string) {
t, err := template.ParseFiles(templates_dir+"base.html", templates_dir+tmpl)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
if tmpl == "view.html" {
err = t.Execute(w, FileList)
} else {
err = t.Execute(w, nil)
}
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
As the value of .FullFilePath, pass a value of type template.URL instead of string, which will tell the html/template package not to escape it.
For example:
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileFullPath": template.URL("/location/to/something"),
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Output (try it on the Go Playground):
<tr><td>something.txt</td></tr>
Note that even though forward slashes / are allowed in URLs, the reason why the template package still encodes them is because it analyses the URL and sees that the value you want to include is the value of a URL parameter (file=XXX), and so it also encodes the slashes (so that everything you pass in will be part of the value of the file URL parameter).
If you plan to acquire this file path at the server side from URL parameters, then what the template package does is the correct and proper way.
But know that by doing this, you'll lose the safety that prevents code injection into URLs. If you're the one providing the values and you know they are safe, there is no problem. But if the data comes from a user input for example, never do this.
Also note that if you pass the whole URL (and not just a part of it), it will work without using template.URL (try this variant on the Go Playground):
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileURL": "/file?file=/location/to/something",
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Also note that the recommended way in my opinion would be to include the file path as part of the URL path and not as the value of a parameter, so instead you should create urls like this:
/file/location/to/something
Map your handler (which serves the file content, see this answer as an example) to the /file/ pattern, and when it is matched and your handler is called, cut off the /file/ prefix from the path r.URL.Path, and the rest will be the full file path. If you choose this, you also won't need the template.URL conversion (because the value you include is not a value of a URL parameter anymore):
func main() {
t := template.Must(template.New("").Parse(templ))
m := map[string]interface{}{
"FileName": "something.txt",
"FileFullPath": "/location/to/something",
}
if err := t.Execute(os.Stdout, m); err != nil {
panic(err)
}
}
const templ = `<tr><td>{{.FileName}}</td></tr>`
Try this on the Go Playground.
Also very important: never parse templates in your handler functions! For details see:
It takes too much time when using "template" package to generate a dynamic web page to client in golang
OK, So the solution I've found (and please post if there's a better one) is based on an answer here.
I changed the struct I was using from:
type File struct {
FullFilePath string
FileName string
}
To this:
type File struct {
FullFilePath template.HTML
FileName string
}
And moved the html into the FullFilePath name, and then placed that in template.HTML so each FullFilePath name I was generating was done like so:
file := File{template.HTML("<a href=\"/file?file=" + path + "\"</a>"), f.Name()}
And my template file line was changed to this:
<tr><td>{{.FullFilePath}}{{.FileName}}</td></tr>
Help. I can't get the right signature using the test parameters provided by Amazon and Go.
My signature hash function is as follows. I use SHA-256 and base64 encoding as per Amazon documentation.
func HashSignature(str string, secret string) string {
mac := hmac.New(sha256.New, []byte(secret))
_, err := mac.Write([]byte(str))
if err != nil { return "" }
hash := base64.StdEncoding.EncodeToString(mac.Sum(nil))
hash = url.QueryEscape(hash)
return hash
}
My signature test function is as follows. I use the canonical string below in Ruby code and it generates the correct expected signature. So the problem seems to be with the output of my HashSignature() function, but I don't see what I'm doing wrong there.
func TestAmazonSignature(t *testing.T) {
/* here is the canonical string from Amazon documentation which should yield the expected signature below
GET
webservices.amazon.com
/onca/xml
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&AssociateTag=mytag-20&ItemId=0679722769&Operation=ItemLookup&ResponseGroup=Images%2CItemAttributes%2COffers%2CReviews&Service=AWSECommerceService&Timestamp=2014-08-18T12%3A00%3A00Z&Version=2013-08-01
*/
SECRET_KEY := "1234567890"
CANONICAL_STR := "GET\nwebservices.amazon.com\n/onca/xml\nAWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&AssociateTag=mytag-20&ItemId=0679722769&Operation=ItemLookup&ResponseGroup=Images%2CItemAttributes%2COffers%2CReviews&Service=AWSECommerceService&Timestamp=2014-08-18T12%3A00%3A00Z&Version=2013-08-01"
EXPECTED := "j7bZM0LXZ9eXeZruTqWm2DIvDYVUU3wxPPpp%2BiXxzQc%3D"
if RESULT := HashSignature(CANONICAL_STR, SECRET_KEY); RESULT != EXPECTED {
t.Errorf("\nEXPECTED:\n%v\nRESULT:\n%v", EXPECTED, RESULT)
} else { fmt.Println("TestAmazonSignature: Signature: OK") }
}
Here's a playground link with all this code.
Looks fine to me, try running:
https://play.golang.org/p/w0mQAYx2GQ
I added necessary imports and a main function