How do I set the Request.FormFile when trying to test an endpoint?
Partial code:
func (a *EP) Endpoint(w http.ResponseWriter, r *http.Request) {
...
x, err := strconv.Atoi(r.FormValue("x"))
if err != nil {
a.ren.Text(w, http.StatusInternalServerError, err.Error())
return
}
f, fh, err := r.FormFile("y")
if err != nil {
a.ren.Text(w, http.StatusInternalServerError, err.Error())
return
}
defer f.Close()
...
}
How do I use the httptest lib to generate a post request that has value that I can get in FormFile?
You don't need to mock the complete FormFile struct as suggested by the other answer. The mime/multipart package implements a Writer type that lets you create a FormFile. From the docs
CreateFormFile is a convenience wrapper around CreatePart. It creates
a new form-data header with the provided field name and file name.
func (w *Writer) CreateFormFile(fieldname, filename string) (io.Writer, error)
Then, you can pass this io.Writer to httptest.NewRequest, which accepts a reader as an argument.
request := httptest.NewRequest("POST", "/", myReader)
To do this, you can either write the FormFile to an io.ReaderWriter buffer or use an io.Pipe. Here is a complete example that makes use of pipes:
func TestUploadImage(t *testing.T) {
// Set up a pipe to avoid buffering
pr, pw := io.Pipe()
// This writer is going to transform
// what we pass to it to multipart form data
// and write it to our io.Pipe
writer := multipart.NewWriter(pw)
go func() {
defer writer.Close()
// We create the form data field 'fileupload'
// which returns another writer to write the actual file
part, err := writer.CreateFormFile("fileupload", "someimg.png")
if err != nil {
t.Error(err)
}
// https://yourbasic.org/golang/create-image/
img := createImage()
// Encode() takes an io.Writer.
// We pass the multipart field
// 'fileupload' that we defined
// earlier which, in turn, writes
// to our io.Pipe
err = png.Encode(part, img)
if err != nil {
t.Error(err)
}
}()
// We read from the pipe which receives data
// from the multipart writer, which, in turn,
// receives data from png.Encode().
// We have 3 chained writers!
request := httptest.NewRequest("POST", "/", pr)
request.Header.Add("Content-Type", writer.FormDataContentType())
response := httptest.NewRecorder()
handler := UploadFileHandler()
handler.ServeHTTP(response, request)
t.Log("It should respond with an HTTP status code of 200")
if response.Code != 200 {
t.Errorf("Expected %s, received %d", 200, response.Code)
}
t.Log("It should create a file named 'someimg.png' in uploads folder")
if _, err := os.Stat("./uploads/someimg.png"); os.IsNotExist(err) {
t.Error("Expected file ./uploads/someimg.png' to exist")
}
}
This function makes use of the image package to generate a file dynamically taking advantage of the fact that you can pass an io.Writer to png.Encode. In the same vein, you could pass your multipart Writer to generate the bytes in a CSV format (NewWriter in package "encoding/csv"), generating a file on the fly, without needing to read anything from your filesystem.
If you have a look at the implementation of the FormFile function you'll see that it reads the exposed MultipartForm field.
https://golang.org/src/net/http/request.go?s=39022:39107#L1249
// FormFile returns the first file for the provided form key.
1258 // FormFile calls ParseMultipartForm and ParseForm if necessary.
1259 func (r *Request) FormFile(key string) (multipart.File, *multipart.FileHeader, error) {
1260 if r.MultipartForm == multipartByReader {
1261 return nil, nil, errors.New("http: multipart handled by MultipartReader")
1262 }
1263 if r.MultipartForm == nil {
1264 err := r.ParseMultipartForm(defaultMaxMemory)
1265 if err != nil {
1266 return nil, nil, err
1267 }
1268 }
1269 if r.MultipartForm != nil && r.MultipartForm.File != nil {
1270 if fhs := r.MultipartForm.File[key]; len(fhs) > 0 {
1271 f, err := fhs[0].Open()
1272 return f, fhs[0], err
1273 }
1274 }
1275 return nil, nil, ErrMissingFile
1276 }
In your test you should be able to create a test instance of multipart.Form and assign it to your request object - https://golang.org/pkg/mime/multipart/#Form
type Form struct {
Value map[string][]string
File map[string][]*FileHeader
}
Of course this will require that you use a real filepath which isn't great from a testing perspective. To get around this you could define an interface to read FormFile from a request object and pass a mock implementation into your EP struct.
Here is a good post with a few examples on how to do this: https://husobee.github.io/golang/testing/unit-test/2015/06/08/golang-unit-testing.html
I combined these and other answers into an Echo example without pipes or goroutines:
func Test_submitFile(t *testing.T) {
path := "testfile.txt"
body := new(bytes.Buffer)
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("object", path)
assert.NoError(t, err)
sample, err := os.Open(path)
assert.NoError(t, err)
_, err = io.Copy(part, sample)
assert.NoError(t, err)
assert.NoError(t, writer.Close())
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", body)
req.Header.Set(echo.HeaderContentType, writer.FormDataContentType())
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
c.SetPath("/submit")
if assert.NoError(t, submitFile(c)) {
assert.Equal(t, 200, rec.Code)
assert.Contains(t, rec.Body.String(), path)
fi, err := os.Stat(expectedPath)
if os.IsNotExist(err) {
t.Fatal("Upload file does not exist", expectedPath)
}
assert.Equal(t, wantSize, fi.Size())
}
}
By combining the previous answers, this worked for me:
filePath := "file.jpg"
fieldName := "file"
body := new(bytes.Buffer)
mw := multipart.NewWriter(body)
file, err := os.Open(filePath)
if err != nil {
t.Fatal(err)
}
w, err := mw.CreateFormFile(fieldName, filePath)
if err != nil {
t.Fatal(err)
}
if _, err := io.Copy(w, file); err != nil {
t.Fatal(err)
}
// close the writer before making the request
mw.Close()
req := httptest.NewRequest(http.MethodPost, "/upload", body)
req.Header.Add("Content-Type", mw.FormDataContentType())
res := httptest.NewRecorder()
// router is of type http.Handler
router.ServeHTTP(res, req)
Related
I have the following function:
func CreateByID(w http.ResponseWriter, r *http.Request) {
formFile, _, err := r.FormFile("audio")
if err != nil {
return
}
defer formFile.Close()
defer r.MultipartForm.RemoveAll()
tmpFile, err := os.CreateTemp("/tmp")
if err != nil {
return
}
defer os.Remove(tmpFile.Name())
io.Copy(tmpFile, formFile)
err = validateFile(tmpFile.Name())
if err != nil {
return
}
}
func validateFile(path string) error {
fs, err := os.Stat(path)
if err != nil {
return
}
if fs.Size() < allowedSize {
return fmt.Errorf("file is too small")
}
}
which is basically creating a temp file from the HTTP request and validating it before further processing. This works fine except for the unit test. When I'm testing this function - I'm getting always filesize = 0.
Below you can see the Test case:
func TestCreateByID(t *testing.T) {
pr, pw := io.Pipe()
writer := multipart.NewWriter(pw)
go func() {
defer writer.Close()
_, err := writer.CreateFormFile("audio", "/tmp/audioFile")
if err != nil {
t.Error(err)
}
generateAudioSample("/tmp/audioFile")
}()
request := httptest.NewRequest(http.MethodGet, "/{id}")
request.Header.Add("Content-Type", writer.FormDataContentType())
response := httptest.NewRecorder()
# Calling The Actual Function
CreateByID(response, request)
handler := http.NewServeMux()
handler.ServeHTTP(response, request)
if response.Code != 200 {
t.Errorf("Expected %d, received %d", 200, response.Code)
return
}
}
generateAudioSample function is working fine because I can see the file on the filesystem and its size it bigger than 0 but for some reason this function io.Copy(tmpFile, formFile) doesn't seem to handle FormFile correctly, and is creating just an empty file on the filesystem and obviously fail the test. What else should I add to the test function in order to pass the test?
i'm new to Golang and i'm trying to write a test for a simple HTTP client.
i read a lot of ways of doing so also here in SO but none of them seems to work.
I'm having troubles mocking the client response
This is how my client looks right now:
type API struct {
Client *http.Client
}
func (api *API) MyClient(qp string) ([]byte, error) {
url := fmt.Sprintf("http://localhost:8000/myapi?qp=%s", qp)
resp, err := api.Client.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
// handling error and doing stuff with body that needs to be unit tested
if err != nil {
return nil, err
}
return body, err
}
And this is my test function:
func TestDoStuffWithTestServer(t *testing.T) {
// Start a local HTTP server
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
rw.Write([]byte(`OK`))
}))
defer server.Close()
// Use Client & URL from our local test server
api := API{server.Client()}
body, _ := api.MyClient("1d")
fmt.Println(body)
}
As i said, this is how they look right cause i try lot of ways on doing so.
My problem is that i'm not able to mock the client respose. in this example my body is empty. my understanding was that rw.Write([]byte(OK)) should mock the response 🤔
In the end i solved it like this:
myclient:
type API struct {
Endpoint string
}
func (api *API) MyClient(slot string) ([]byte, error) {
url := fmt.Sprintf("%s/myresource?qp=%s", api.Endpoint, slot)
c := http.Client{}
resp, err := c.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, err
}
test:
func TestDoStuffWithTestServer(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
rw.Write([]byte(`{Result: [{Cluster_name: "cl1", Pings: 2}]}`))
}))
defer server.Close()
api := API{Endpoint: server.URL}
res, _ := api.MyClient("1d")
expected := []byte(`{Result: [{Cluster_name: "cl1", Pings: 2}]}`)
if !bytes.Equal(expected, res) {
t.Errorf("%s != %s", string(res), string(expected))
}
}
still, not 100% sure is the right way of doing so in Go
I have a simple function which takes a URL and fetches the response:
func getUrl(url string) (string, error) {
var theClient = &http.Client{Timeout: 12 * time.Second}
resp, err := theClient.Get(url)
if err != nil {
return "", err
}
defer r.Body.Close()
body, readErr := ioutil.ReadAll(resp.Body)
if readErr != nil {
return "", readErr
}
return string(body), nil
}
Now, I want to trigger an error on the theClient.Get(url) line but I don't know how to. I can trigger an error on the ReadAll() line, by returning no response but with content-length:2.
How can I trigger an error on the theClient.Get(url) line for my unit test?
func TestGetUrl(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "2")
}))
defer server.Close()
gotContent, gotErr := getUrl(server.URL)
wantErr := "unexpected EOF"
if gotErr == nil || gotErr.Error() != wantErr {
t.Errorf("got err %v; wanted %s", gotErr, wantErr)
}
}
Easiest way is to simply pass an invalid URL:
_, err := http.Get("clearly not a valid url")
fmt.Println("Got error:", err != nil) // Got error: true
Another option is to make it timeout by sleeping in your httptest.Server handler, but that doesn't seem like a very nice idea (but you will be able to assert that it was called in the first place).
How do I set the Request.FormFile when trying to test an endpoint?
Partial code:
func (a *EP) Endpoint(w http.ResponseWriter, r *http.Request) {
...
x, err := strconv.Atoi(r.FormValue("x"))
if err != nil {
a.ren.Text(w, http.StatusInternalServerError, err.Error())
return
}
f, fh, err := r.FormFile("y")
if err != nil {
a.ren.Text(w, http.StatusInternalServerError, err.Error())
return
}
defer f.Close()
...
}
How do I use the httptest lib to generate a post request that has value that I can get in FormFile?
You don't need to mock the complete FormFile struct as suggested by the other answer. The mime/multipart package implements a Writer type that lets you create a FormFile. From the docs
CreateFormFile is a convenience wrapper around CreatePart. It creates
a new form-data header with the provided field name and file name.
func (w *Writer) CreateFormFile(fieldname, filename string) (io.Writer, error)
Then, you can pass this io.Writer to httptest.NewRequest, which accepts a reader as an argument.
request := httptest.NewRequest("POST", "/", myReader)
To do this, you can either write the FormFile to an io.ReaderWriter buffer or use an io.Pipe. Here is a complete example that makes use of pipes:
func TestUploadImage(t *testing.T) {
// Set up a pipe to avoid buffering
pr, pw := io.Pipe()
// This writer is going to transform
// what we pass to it to multipart form data
// and write it to our io.Pipe
writer := multipart.NewWriter(pw)
go func() {
defer writer.Close()
// We create the form data field 'fileupload'
// which returns another writer to write the actual file
part, err := writer.CreateFormFile("fileupload", "someimg.png")
if err != nil {
t.Error(err)
}
// https://yourbasic.org/golang/create-image/
img := createImage()
// Encode() takes an io.Writer.
// We pass the multipart field
// 'fileupload' that we defined
// earlier which, in turn, writes
// to our io.Pipe
err = png.Encode(part, img)
if err != nil {
t.Error(err)
}
}()
// We read from the pipe which receives data
// from the multipart writer, which, in turn,
// receives data from png.Encode().
// We have 3 chained writers!
request := httptest.NewRequest("POST", "/", pr)
request.Header.Add("Content-Type", writer.FormDataContentType())
response := httptest.NewRecorder()
handler := UploadFileHandler()
handler.ServeHTTP(response, request)
t.Log("It should respond with an HTTP status code of 200")
if response.Code != 200 {
t.Errorf("Expected %s, received %d", 200, response.Code)
}
t.Log("It should create a file named 'someimg.png' in uploads folder")
if _, err := os.Stat("./uploads/someimg.png"); os.IsNotExist(err) {
t.Error("Expected file ./uploads/someimg.png' to exist")
}
}
This function makes use of the image package to generate a file dynamically taking advantage of the fact that you can pass an io.Writer to png.Encode. In the same vein, you could pass your multipart Writer to generate the bytes in a CSV format (NewWriter in package "encoding/csv"), generating a file on the fly, without needing to read anything from your filesystem.
If you have a look at the implementation of the FormFile function you'll see that it reads the exposed MultipartForm field.
https://golang.org/src/net/http/request.go?s=39022:39107#L1249
// FormFile returns the first file for the provided form key.
1258 // FormFile calls ParseMultipartForm and ParseForm if necessary.
1259 func (r *Request) FormFile(key string) (multipart.File, *multipart.FileHeader, error) {
1260 if r.MultipartForm == multipartByReader {
1261 return nil, nil, errors.New("http: multipart handled by MultipartReader")
1262 }
1263 if r.MultipartForm == nil {
1264 err := r.ParseMultipartForm(defaultMaxMemory)
1265 if err != nil {
1266 return nil, nil, err
1267 }
1268 }
1269 if r.MultipartForm != nil && r.MultipartForm.File != nil {
1270 if fhs := r.MultipartForm.File[key]; len(fhs) > 0 {
1271 f, err := fhs[0].Open()
1272 return f, fhs[0], err
1273 }
1274 }
1275 return nil, nil, ErrMissingFile
1276 }
In your test you should be able to create a test instance of multipart.Form and assign it to your request object - https://golang.org/pkg/mime/multipart/#Form
type Form struct {
Value map[string][]string
File map[string][]*FileHeader
}
Of course this will require that you use a real filepath which isn't great from a testing perspective. To get around this you could define an interface to read FormFile from a request object and pass a mock implementation into your EP struct.
Here is a good post with a few examples on how to do this: https://husobee.github.io/golang/testing/unit-test/2015/06/08/golang-unit-testing.html
I combined these and other answers into an Echo example without pipes or goroutines:
func Test_submitFile(t *testing.T) {
path := "testfile.txt"
body := new(bytes.Buffer)
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("object", path)
assert.NoError(t, err)
sample, err := os.Open(path)
assert.NoError(t, err)
_, err = io.Copy(part, sample)
assert.NoError(t, err)
assert.NoError(t, writer.Close())
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", body)
req.Header.Set(echo.HeaderContentType, writer.FormDataContentType())
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
c.SetPath("/submit")
if assert.NoError(t, submitFile(c)) {
assert.Equal(t, 200, rec.Code)
assert.Contains(t, rec.Body.String(), path)
fi, err := os.Stat(expectedPath)
if os.IsNotExist(err) {
t.Fatal("Upload file does not exist", expectedPath)
}
assert.Equal(t, wantSize, fi.Size())
}
}
By combining the previous answers, this worked for me:
filePath := "file.jpg"
fieldName := "file"
body := new(bytes.Buffer)
mw := multipart.NewWriter(body)
file, err := os.Open(filePath)
if err != nil {
t.Fatal(err)
}
w, err := mw.CreateFormFile(fieldName, filePath)
if err != nil {
t.Fatal(err)
}
if _, err := io.Copy(w, file); err != nil {
t.Fatal(err)
}
// close the writer before making the request
mw.Close()
req := httptest.NewRequest(http.MethodPost, "/upload", body)
req.Header.Add("Content-Type", mw.FormDataContentType())
res := httptest.NewRecorder()
// router is of type http.Handler
router.ServeHTTP(res, req)
I have an HTTP handler function (POST) which allows a user to upload a folder from a web browser application. The folder is passed from JavaScript code as an array of files in a folder and on the backend (Go API) it is accepted as a []*multipart.FileHeader. I am struggling in writing a Go unit test for this function. How can I pass a folder as input from a test function? I need help in creating the httpRequest in the right format.
I have tried to use / set values for an array of FileHeader, but some attributes are not allowed to be imported. So there must be a different way of testing this handler that I am not aware of.
Handler Function for folder upload:
func FolderUpload(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
// SOME LOGIC
files := r.MultipartForm.File["multiplefiles"] // files is of the type []*multipart.FileHeader
// SOME LOGIC TO PARSE THE FILE NAMES TO RECREATE THE SAME TREE STRUCTURE ON THE SERVER-SIDE AND STORE THEM AS A FOLDER
Unit Test function for the same handler:
func TestFolderUpload(t *testing.T) {
// FolderPreCondition()
request, err := http.NewRequest("POST", uri, body) //Q: HOW TO CREATE THE BODY ACCEPTABLE BY THE ABOVE HANDLER FUNC?
// SOME ASSERTION LOGIC
}
You should write your file to request:
func newFileUploadRequest(url string, paramName, path string) (*http.Request, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
body := new(bytes.Buffer)
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile(paramName, filepath.Base(path))
if err != nil {
return nil, err
}
_, err = io.Copy(part, file)
if err != nil {
return nil, err
}
err = writer.Close()
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", url, body)
if err != nil {
return nil, err
}
req.Header.Add("Content-Type", writer.FormDataContentType())
return req, err
}
then use it:
req, err := newFileUploadRequest("http://localhost:1234/upload", "multiplefiles", path)
client := &http.Client{}
resp, err := client.Do(req)
It works for me, hope it helps you)