I just read the article: Build You Own Web Framework In Go and for sharing values among handlers I picked the context.Context and I'm using it in the following way to share values across handlers and middlewares:
type appContext struct {
db *sql.DB
ctx context.Context
cancel context.CancelFunc
}
func (c *appContext)authHandler(next http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request {
defer c.cancel() //this feels weird
authToken := r.Header.Get("Authorization") // this fakes a form
c.ctx = getUser(c.ctx, c.db, authToken) // this also feels weird
next.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
func (c *appContext)adminHandler(w http.ResponseWriter, r *http.Request) {
defer c.cancel()
user := c.ctx.Value(0).(user)
json.NewEncoder(w).Encode(user)
}
func getUser(ctx context.Context, db *sql.DB, token string) context.Context{
//this function mimics a database access
return context.WithValue(ctx, 0, user{Nome:"Default user"})
}
func main() {
db, err := sql.Open("my-driver", "my.db")
if err != nil {
panic(err)
}
ctx, cancel := context.WithCancel(context.Background())
appC := appContext{db, ctx, cancel}
//....
}
Everything is working and handlers are loading faster than using gorilla/context So my questions are:
Is this approach safe?
Is it really necessary to defer the c.cancel() function the way I'm doing it?
Can I use it to implement a custom web framework by using controllers like struct to share values with models?
Note: go 1.7.0-rc2 does clarify a bit how to release resources associated with Contexts (Sameer Ajmani):
Some users don't realize that creating a Context with a CancelFunc attaches a subtree to the parent, and that that subtree is not released until the CancelFunc is called or the parent is canceled.
Make this clear early in the package docs, so that people learning about this package have the right conceptual model.
The documentation now includes:
Incoming requests to a server should create a Context, and outgoing calls to servers should accept a Context.
The chain of function calls between them must propagate the Context, optionally replacing it with a derived Context created using WithCancel, WithDeadline, WithTimeout, or WithValue.
These Context values form a tree: when a Context is canceled, all Contexts derived from it are also canceled.
The WithCancel, WithDeadline, and WithTimeout functions return a derived Context and a CancelFunc.
Calling the CancelFunc cancels the new Context and any Contexts derived from it, removes the Context from the parent's tree, and stops any associated timers.
Failing to call the CancelFunc leaks the associated resources until the parent Context is canceled or the timer fires.
You have a problem with your code because you are storing the user into the app context. With multiple users at the same time, it doesn't work. The context must be related to the request to not be overwrote by other requests. The user must be stored in a request context. In my articles I use the following gorilla function: context.Set(r, "user", user). r is the request.
If you want to use context.Context in your app, you should use their gorilla wrapper (you can find it at the end of this article: https://blog.golang.org/context).
Also, you don't need a context with cancel. context.Background() is okay for the root context.
Two major use cases of Context package are:
For storing request scoped values - Using context.WithValue()
For cancellation - using context.WithCancel(), context.WithTimeout(), context.WithDeadline()
Use of context forms a context tree with context.Background() as root. WithValue() ,context.WithCancel(), WithTimeout(), WithDeadline() are derived context from root context which can be further divide. An example of each can make it clear so ast to what is the correct use context. Came across this guide which provides the use of all discussed above with proper examples.
Source: https://golangbyexample.com/using-context-in-golang-complete-guide/
Related
I'm working with getstream's Go library in gin gonic and realized that my endpoints will be heavily dependent on stream_chat.Client.
For instance, in the following endpoint (/v1/chat/test-token), a stream_chat.Client must be created so testing this endpoint in unit test would mean creating and maintaining an interface that documents all the methods I use from stream_chat.Client so that I can perform dependency injection with a MockClient that satisfies the same interface and then I can mock the methods chatClient.UpsertUser and chatClient.CreateToken when I write my unit test.
func main() {
config.Load()
server := gin.New()
chatClient, err := stream_chat.NewClient(config.StreamApiKey, config.StreamApiSecret)
if err != nil {
log.Err(err)
os.Exit(2)
}
v1 := server.Group("/v1")
{
v1.GET("/chat/test-token/", func(c *gin.Context) {
_, err := chatClient.UpsertUser(&stream.User{
ID: "test-user",
Role: "admin",
})
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{})
}
token, _ := chatClient.CreateToken("test-user", time.Time{})
c.JSON(http.StatusOK, gin.H{
"token": token,
})
})
}
server.Run(fmt.Sprintf(":%s", config.Port))
}
It seems to me to be quite laborious to document each method that I'd use from stream_chat.Client in order to keep a good test coverage on the endpoints, so I wonder what one should do in this case?
Is maintaining an interface for stream_chat.Client the correct way to go?
Less relevant: Is there a way to properly decouple the gin.HandlerFunc, i.e. func(c *gin.Context) from the creation of stream_chat.Client?
Even less relevant: Is it better to create a singleton stream_chat.Client or should I create a new client for each endpoint that requires a client?
Is maintaining an interface for stream_chat.Client the correct way to go?
If you have a non-interface dependency and you wish to unit test handlers with that, then yes. You need to wrap stream_chat.Client in an interface.
If the third-party struct has a lot of methods, you could split the interface in logical units and inject in each handler only those that are actually needed. The underlying stream_chat.Client implements all of them, but the individual mocks can be kept small and easier to reason about. Personally, I don't think it's worth the overhead. There's plenty of open-source mock generators, above all mock and mockgen, and also tools that generate interfaces from structs.
Is there a way to properly decouple the gin.HandlerFunc, i.e. func(c *gin.Context) from the creation of stream_chat.Client?
You have several options, which you can find here: How to pass arguments to router handlers in Golang using Gin web framework?
In short, the options I prefer are due to better unit-testability are:
make the handlers methods of a struct and your dependencies fields of this struct.
use a provider pattern and set the provider into the Gin context in a middleware
I'm pretty new to Go and still learning about how things work in Go, so with that said I've been looking in to Go testing approach and how mocking would work for the past few weeks and most of the information I found based on functions being concrete.
E.g. everything is a function is either passed as a receiver or a parameter, however, the problem I face is my function uses a switch case to determine what function it should be called, so it's not passed from outside.
func (n *Notification) Notify(m Message) error {
switch n.Service {
case "slack":
var s slack.Slack
s.User = m.User
s.Host = m.Host
s.Provider = m.Provider
s.SystemUser = m.SystemUser
return s.SlackSend(n.Url)
default:
return errors.New(codes.CODE5)
}
}
Above code is what the function I want to test looks like and I'm having a hard time figuring out how I could mock the SlackSend() function.
I've come across some article say I should write the function in the test file and when I'm happy with what function do, I should write the real code. This doesn't sound right to me because for me it feels like I have the same code in two places and test only uses the one in test and I could change the real code and break it without testes detecting that.
I mainly work on Python and I'm used to using things like Mock/MagicMock that can intercept the function call and replace on at runtime, so apologies in advance if I don't quite get the Go's TDD approach.
This is what test code would look like if anyone wonders:
type MockSlack struct {
*slack.Slack
}
func (s *MockSlack) SlackSend(url string) error {
if url != "" {
return nil
} else {
return errors.New("url empty")
}
}
func TestNotify(t *testing.T) {
m := Message{}
n := Notification{
Service: "slack",
Url: "https://dummy.io",
}
if err := n.Notify(m); err != nil {
t.Errorf("SlackSend, expected: %s, got: %s", "nil", err.Error())
}
}
Obviously the MockSlack structure is not taking effect because it's not really passed in anywhere.
If anyone has any advice on how I could mock this or what I should do instead would be much appreciated.
Thanks in advance.
UPDATE:
Background
This isn't a web server/applicaition of a sort. It's an SSH authentication plugin so it would be a server-side application. As for Notify method, it server's the purpose of a mapper. So it can call Slack, MS Teams, AWS SNS which give caller less conditions to process and how and where the notification it sent is decided by the Notify method.
If you are not able to change the Notify Method to make it testing friendly. One option would be to consider using monkeypatching. Here is an example provided for *net.Dialer:
func main() {
var d *net.Dialer // Has to be a pointer to because `Dial` has a pointer receiver
monkey.PatchInstanceMethod(reflect.TypeOf(d), "Dial", func(_ *net.Dialer, _, _ string) (net.Conn, error) {
return nil, fmt.Errorf("no dialing allowed")
})
_, err := http.Get("http://google.com")
fmt.Println(err) // Get http://google.com: no dialing allowed
}
WARNING: It's not safe to use it outside of a testing environment.
I'm trying to implement unit tests in Go for an existing service which uses a connection pool struct and a connection struct from an existing library (call these LibraryPool and LibraryConnection) to connect to an external service.
To use these, the service functions in the main code uses a unique, global instance of the pool, which has a GetConnection() method, like this:
// Current Main Code
var pool LibraryPool // global, instantiated in main()
func someServiceFunction(w http.ResponseWriter, r *http.Request) {
// read request
// ...
conn := pool.GetConnection()
conn.Do("some command")
// write response
// ...
}
func main() {
pool := makePool() // builds and returns a LibraryPool
// sets up endpoints that use the service functions as handlers
// ...
}
I'd like to unit-test these service functions without connecting to the external service, and so I'd like to mock the LibraryPool and LibraryConnection. To allow for this, I was thinking of changing the main code to something like this:
// Tentative New Main Code
type poolInterface interface {
GetConnection() connInterface
}
type connInterface interface {
Do(command string)
}
var pool poolInterface
func someServiceFunction(w http.ResponseWriter, r *http.Request) {
// read request
// ...
conn := pool.GetConnection()
conn.Do("some command")
// write response
// ...
}
func main() {
pool := makePool() // still builds a LibraryPool
}
In the tests, I would use mock implementations MockPool and MockConnection of these interfaces, and the global pool variable would be instantiated using MockPool. I would instantiate this global pool in a setup() function, inside of a TestMain() function.
The problem is that in the new main code, LibraryPool does not properly implement poolInterface, because GetConnection() returns a connInterface instead of a LibraryConnection (even though LibraryConnection is a valid implementation of connInterface).
What would be a good way to approach this kind of testing? The main code is flexible too, by the way.
Well, I'll try to answer by completely explain how I see this design. Sorry in advance if this is too much and not to the point..
Entity / Domain
The core of the app, will include the entity struct, won't import ANY outer layer package, but can be imported by every package (almost)
Application / Use case
The "service". Will be responsible mainly for the app logic, won't know about the transport(http), will "talk" with the DB through interface. Here you can have the domain validation, for example if resource is not found, or text is too short. Anything related to business logic.
transport
Will handle the http request, decode the request, get the service to do his stuff, and encode the response. Here you can return 401 if there is a missing required param in the request, or the user is not authorized, or something...
infrastructure
DB connection
Maybe some http engine and router and stuff.
Totally app-agnostic, don't import any inner package, not even Pseron
For example, let's say we want to do something as simple as insert person to the db.
package person will only include the person struct
package person
type Person struct{
name string
}
func New(name string) Person {
return Person{
name: name,
{
}
About the db, let's say you use sql, I recommend to make a package named sql to handle the repo. (if you use postgress, use 'postgress package...).
The personRepo will get the dbConnection which will be initialized in main and implement DBAndler. only the connection will "talk" with the db directly, the repository main goal is to be gateway to the db, and speak in application-terms. (the connection is app-agnostic)
package sql
type DBAndler interface{
exec(string, ...interface{}) (int64, error)
}
type personRepo struct{
dbHandler DBHandler
}
func NewPersonRepo(dbHandler DBHandler) &personRepo {
return &personRepo{
dbHandler: dbHandler,
}
}
func (p *personRepo) InsertPerson(p person.Person) (int64, error) {
return p.dbHandler.Exec("command to insert person", p)
}
The service will get this repository as a dependancy (as interface) in the initailzer, and will interact with it to accomplish the business logic
package service
type PersonRepo interface{
InsertPerson(person.Person) error
}
type service struct {
repo PersonRepo
}
func New(repo PersonRepo) *service {
return &service{
repo: repo
}
}
func (s *service) AddPerson(name string) (int64, error) {
person := person.New(name)
return s.repo.InsertPerson(person)
}
Your transport handler will be initialized with the service as a dependancy, and he will handle the http request.
package http
type Service interface{
AddPerson(name string) (int64, error)
}
type handler struct{
service Service
}
func NewHandler(s Service) *handler {
return &handler{
service: s,
}
}
func (h *handler) HandleHTTP(w http.ResponseWriter, r *http.Request) {
// read request
// decode name
id, err := h.service.AddPerson(name)
// write response
// ...
}
And in main.go you will tie everything together:
Initialize db connection
Initialize personRepo with this connection
Initialize service with the repo
Initialize the transport with the service
package main
func main() {
pool := makePool()
conn := pool.GetConnection()
// repo
personRepo := sql.NewPersonRepo(conn)
// service
personService := service.New(personRepo)
// handler
personHandler := http.NewPersonHandler(personService)
// Do the rest of the stuff, init the http engine/router by passing this handler.
}
Note that every package struct was initialized with an interface but returned a struct, and also the interfaces were declared in the package which used them, not in the package which implemented them.
This makes it easy to unit test these package. for example, if you want to test the service, you don't need to worry about the http request, just use some 'mock' struct that implements the interface that the service depend on (PersonRepo), and you good to go..
Well, I hope it helped you even a little bit, it may seem confusing at first, but in time you will see how this seems like a large piece of code, but it helps when you need to add functionality or switching the db driver and such.. I recommend you to read about domain driven design in go, and also hexagonal arch.
edit:
In addition, this way you pass the connection to the service, the service doesn't import and use the global DB pool. Honestly, I don't know why it is so common, I guess it has its advantages and it is better to some application, but generally I think that letting your service depend on some interface, without actually know what is going on, is much a better practice.
I'm studying the outyet example project from https://github.com/golang/example/tree/master/outyet. The test file does not cover the case where http.Head(url) returns an error. I would like to extend the unit tests to cover the if statement where the error is logged (https://github.com/golang/example/blob/master/outyet/main.go#L100). I would like to mock http.Head(), but I'm not sure how to do this. How can this be done?
The http.Head function simply calls the Head method on the default HTTP client (exposed as http.DefaultClient). By replacing the default client within your test, you can change the behaviour of these standard library functions.
In particular, you will want a client that sets a custom transport (any object implementing the http.RoundTripper interface). Something like the following:
type testTransport struct{}
func (t testTransport) RoundTrip(request *http.Request) (*http.Response, error) {
# Check expectations on request, and return an appropriate response
}
...
savedClient := http.DefaultClient
http.DefaultClient = &http.Client{
Transport: testTransport{},
}
# perform tests that call http.Head, http.Get, etc
http.DefaultClient = savedClient
You could also use this technique to mock network errors by returning an error from your transport rather than an HTTP response.
Is there a way to execute test cases in GoLang in a pre-defined order.
P.S: I am writing test cases for life cycle of a event. So I have different api's for all the CURD operations. I want to run these test cases in a particular order as only if an event is created it can be destroyed.
Also can I get some value from one test case and pass it as input to another. (example:- To test the delete event api, I need a event_id which i get when I call create_event test case)
I am new to GoLang, can someone please guide me through.
Thanks in advance
The only way to do it is to encapsulate all your tests into one test function, that calls sub-functions in the right order and with the right context, and pass the testing.T pointer to each so they can fail. The down-side is that they will all appear as one test. But in fact that is the case - tests are stateless as far as the testing framework is concerned, and each function is a separate test case.
Note that although the tests may run in the order they are written in, I found no documentation stating that this is actually a contract of some sort. So even though you can write them in order and keep the state as external global variables - that's not recommended.
The only flexibility the framework gives you since go 1.4 is the TestMain method that lets you run before/after steps, or setup/teardown:
func TestMain(m *testing.M) {
if err := setUp(); err != nil {
panic(err)
}
rc := m.Run()
tearDown()
os.Exit(rc)
}
But that won't give you what you want. The only way to do that safely is to do something like:
// this is the whole stateful sequence of tests - to the testing framework it's just one case
func TestWrapper(t *testing.T) {
// let's say you pass context as some containing struct
ctx := new(context)
test1(t, ctx)
test2(t, ctx)
...
}
// this holds context between methods
type context struct {
eventId string
}
func test1(t *testing.T, c *context) {
// do your thing, and you can manipulate the context
c.eventId = "something"
}
func test2(t *testing.T, c *context) {
// do your thing, and you can manipulate the context
doSomethingWith(c.eventId)
}