I'm rebuilding an app that supports customer specific templating (themes) from node.js to Go.
I'm currently using render to render my template files but what I actually need to do access templates that are stored in an object store such as Cloudfiles.
In node.js I've done this with express and I'm overriding the render() method but I've not been able to figure out how to do this in Go.
I essentially need to do something like this:
func (c *Controller) MyRouteHandler (rw http.ResponseWriter, req *http.Request) {
// retrieve the store from the context (assigned in middleware chain)
store := context.Get(req, "store").(*Store)
... do some stuff like load the entity from the database
// retrieve the template from the Object store and
// create the template instance (template.New("template").Parse(...))
tpl := c.ObjectStore.LoadTemplate(store, entity.TemplateFile)
// I know render's .HTML function takes the path to the template
// so I'll probably need to show the html a different way
c.HTML(rw, http.StatusOK, tpl, &PageContext{Title: "My Page", Entity: &entity})
}
I can dynamically include sub-templates by doing something like this if needed: http://play.golang.org/p/7BCPHdKRi2 but it doesn't seem like a great way if I'm honest.
I've searched for a solution to this but keep hitting road blocks. Any advice / assistance would be great.
Edit:
In essence, I'm asking the following:
How can I load a specific template from a datastore on a per-request basis.
How can I then send that as a response to the client
How can I load a specific template from a datastore on a per-request basis.
//take HTTP for example:
resp, err := http.Get("http://mytemplates.com/template1")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
templateString := string(body)
How can I then send that as a response to the client
tmpl, err := template.New("name").Parse(templateString)
tmpl.Execute(rw, &yourDataModel{})
Related
I want to use the data provided by the client to serve a file for them to download (say .txt) without creating or persisting any files on my server. How can I do it?
For instance, I want to omit the os.Create() defer f.Close() part from the following code with Gin framework:
func serveFile(c *gin.Context) {
path := "/tmp/hello.txt"
f, _ := os.Create(path)
defer f.Close()
f.WriteString("hello world")
f.Sync()
// ...
c.File(path)
}
After setting proper response headers eg
w.Header().Set("Content-Disposition", "attachment; filename=hello.txt")
w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
you can
w.Write("hello world")
you can also use http.ServeContent
https://cs.opensource.google/go/go/+/refs/tags/go1.17.1:src/net/http/fs.go;l=192v
Use c.Data() to write a string to the response:
c.Data(http.StatusOK, "text/plain", []byte("hello world"))
I want to write a golang function which return the service according to the input:
func GetService(service string) <what to write here> {
session, _ := session.NewSession(&aws.Config{Region: aws.String(Region)})
switch {
case service == 'ecr':
var svc *ecr.ECR
svc = ecr.New(session)
return svc
case service == 'ecs':
var svc *ecs.ECS
svc = ecs.New(session)
return svc
}
}
It is kind of factory for aws services, but what is their common type which has to be returned?
What you can do is write two functions where each will return either of these.
Other option is to return an interface and let the user cast the value as he knows what it is. Optionally interface can have some methods defined if object share methods with same name.
Last approach that I see is most fitting is a modular approach that I will also demonstrate. But with pseudo code.
func NewSession
session, err = (create session)
(handle error)
return session
// now lets use this
func MyUseCase
ecr = NewEcr(NewSession())
(use ecr)
The thing you are abstracting in your code should not be abstracted at all. User should create his instance as it is simpler then casting it and even faster.
The main goal of abstraction is simplify top level code and eliminate repetition. Keep that in mind.
I am creating a web application that lists applicants and their position on a waiting list.
We need to be able to add new applicants to this list and remove applicants from the list. There will be under 10k applicants in the list.
Specifics:
I plan to write the app in Golang.
The list needs to be safe, I the program shuts down, it should be recoverable.
The app should contain this data for every applicant: Name, Student ID, position.
Questions:
How do I secure the list (lock?) so it is updated correctly for both if two updates to it is made at the same time?
Should I save the data in a database or use a file?
I need your help!
UPDATE:
Mockup code:
package main
import (
"log"
"sync"
"time"
"github.com/boltdb/bolt"
)
type applicant struct {
FirstName string
LastName string
StudentID string
Position int
}
type priorityList struct {
sync.Mutex
applicants []applicant
}
func (l *priorityList) newApplicant(fn string, ln string, sid string) error {
// add applicant to priorityList
return nil
}
func (l *priorityList) removeApplicant(sid string) error {
// remove applicant from priorityList
return nil
}
func (l *priorityList) editApplicant(sid string) error {
// edit applicant in priorityList
return nil
}
func main() {
// Database
db, err := bolt.Open("priorityList.db", 0600, &bolt.Options{Timeout: 1 * time.Second})
if err != nil {
log.Fatal(err)
}
defer db.Close()
}
If you use a file, you could use a Mutex to block concurrent writes.
Otherwise a database would be fine. For example BoltDB could be suitable. It is pure go and runs withing your program.
There are many approaches. You can use file and protect it with Go mutex or system lock. You can memory map the file for performance. You either can use BoltDB which is nice peace of software and provide needed machinery and can work in-process. If you write rare and mostly read, then constant DB https://github.com/colinmarc/cdb also looks interesting.
But, classic SQL DB has some advantages
You can use third party store for data and easely migrate when needed
You can access your data from third party app or just plain SQL
request
You can think about data schema and code logic separately
I'm looking for help on implementing something that automatically includes versioned filenames in a Go HTML template. For example, in my template I have something like this in the head:
<link rel="stylesheet" href="{{ .MyCssFile }}" />
The stylesheets themselves have a chunk of MD5 hash appended to the name from a gulp script called gulp-rev
stylesheet-d861367de2.css
The purpose is to ensure new changes are picked up by browsers, but also allow caching. Here is an example implementation in Django for a better explanation:
https://docs.djangoproject.com/en/1.9/ref/contrib/staticfiles/#manifeststaticfilesstorage
A subclass of the StaticFilesStorage storage backend which stores the file names it handles by appending the MD5 hash of the file’s content to the filename. For example, the file css/styles.css would also be saved as css/styles.55e7cbb9ba48.css.
The purpose of this storage is to keep serving the old files in case some pages still refer to those files, e.g. because they are cached by you or a 3rd party proxy server. Additionally, it’s very helpful if you want to apply far future Expires headers to the deployed files to speed up the load time for subsequent page visits.
Now I'm wondering how to best pull this off in Go? I intend to serve the files from the built in file server.
My current thoughts are:
Have a loop that checks for the newest stylesheet file in a directory. Sounds slow.
Do some kind of redirect/rewrite to a generically named file (as in file.css is served on a request to file-hash.css).
Have Go manage the asset naming itself, appending the hash or timestamp.
Maybe its better handled with nginx or something else?
Write a template function to resolve the name. Here's an example template function:
func resolveName(p string) (string, error) {
i := strings.LastIndex(p, ".")
if i < 0 {
i = len(p)
}
g := p[:i] + "-*" + p[i:]
matches, err := filepath.Glob(g)
if err != nil {
return "", err
}
if len(matches) != 1 {
return "", fmt.Errorf("%d matches for %s", len(matches), p)
}
return matches[0], nil
}
and here's how to use it in a template when registered as the function "resolveName":
<link rel="stylesheet" href="{{ .MyCssFile | resolveName }}" />
playground example
This function resolves the name of the file every time the template is rendered. A more clever function might cache names as they are resolved or walk the directory tree at startup to prebuild a cache.
I knew it's too old, but maybe this library will help you. It allows to collect and hash static files. Also it has function to reverse file path from the orignal location to the hashed location:
staticFilesPrefix := "/static/"
staticFilesRoot := "output/dir"
storage := NewStorage(staticFilesRoot)
err := storage.LoadManifest()
funcs := template.FuncMap{
"static": func(relPath string) string {
return staticFilesPrefix + storage.Resolve(relPath)
},
}
tmpl := template.Must(
template.New("").Funcs(funcs).ParseFiles("templates/main.tpl")
)
Now you can call static function in templates like this {{static "css/style.css"}}. The call will be converted to /static/css/style.d41d8cd98f00b204e9800998ecf8427e.css.
I am currently trying to test a piece of my code that runs a query on the datastore before putting in a new entity to ensure that duplicates are not created. The code I wrote works fine in the context of the app, but the tests I wrote for that methods are failing. It seems that I cannot access data put into the datastore through queries in the context of the testing package.
One possibility might lie in the output from goapp test which reads: Applying all pending transactions and saving the datastore. This line prints out after both the get and put methods are called (I verified this with log statements).
I tried closing the context and creating a new one for the different operations, but unfortunately that didn't help either. Below is a simple test case that Puts in an object and then runs a query on it. Any help would be appreciated.
type Entity struct {
Value string
}
func TestEntityQuery(t *testing.T) {
c, err := aetest.NewContext(nil)
if err != nil {
t.Fatal(err)
}
defer c.Close()
key := datastore.NewIncompleteKey(c, "Entity", nil)
key, err = datastore.Put(c, key, &Entity{Value: "test"})
if err != nil {
t.Fatal(err)
}
q := datastore.NewQuery("Entity").Filter("Value =", "test")
var entities []Entity
keys, err := q.GetAll(c, &entities)
if err != nil {
t.Fatal(err)
}
if len(keys) == 0 {
t.Error("No keys found in query")
}
if len(entities) == 0 {
t.Error("No entities found in query")
}
}
There is nothing wrong with your test code. The issue lies in the Datastore itself. Most queries in the HR Datastore are not "immediately consistent" but eventually consistent. You can read more about this in the Datastore documentation.
So basically what happens is that you put an entity into the Datastore, and the SDK's Datastore "simulates" the latency that you can observe in production, so if you run a query right after that (which is not an ancestor query), the query result will not include the new entity you just saved.
If you put a few seconds sleep between the datastore.Put() and q.GetAll(), you will see the test passes. Try it. In my test it was enough to sleep just 100ms, and the test always passed. But when writing tests for such cases, use the StronglyConsistentDatastore: true option as can be seen in JonhGB's answer.
You would also see the test pass without sleep if you'd use Ancestor queries because they are strongly consistent.
The way to do this is to force the datastore to be strongly consistent by setting up the context like this:
c, err := aetest.NewContext(&aetest.Options{StronglyConsistentDatastore: true})
if err != nil {
t.Fatal(err)
}
Now the datastore won't need any sleep to work, which is faster, and better practice in general.
Update: This only works with the old aetest package which was imported via appengine/aetest. It does not work with the newer aetest package which is imported with google.golang.org/appengine/aetest. App Engine has changed from using an appengine.Context to using a context.Context, and consequently the way that the test package now works is quite different.
To compliment #JohnGB's answer in the latest version of aetest, there are more steps to get a context with strong consistency. First create an instance, then create a request from that instance, which you can use to produce a context.
inst, err := aetest.NewInstance(
&aetest.Options{StronglyConsistentDatastore: true})
if err != nil {
t.Fatal(err)
}
defer inst.Close()
req, err := inst.NewRequest("GET", "/", nil)
if err != nil {
t.Fatal(err)
}
ctx := appengine.NewContext(req)