Post a multipart/form-data HTTP request with WinHTTP - c++

I have been trying for the past few days to send an HTTP POST request to my SpringBoot application with the Win32 API, but I'm always receiving the same error. The request is a multipart consisting of a binary file and a JSON. Sending the request via Postman works with no problems, and I'm able to receive the file and the JSON correctly.
I have looked at almost every post and question regarding how to construct an HTTP request with WinHTTP, and it seems they all do exactly what I did, but for some reason I'm getting a weirdly constructed request, as seen in the WireShark image below.
It seems as if the request is not recognized as several parts, but rather as one chunk of data.
Looking in WireShark at the correct request which was sent with postman, shows that the request consists of all parts, just as it supposed to.
Here is my code:
LPCWSTR additionalHeaders = L"Accept: application/json\r\nContent-Type: multipart/form-data; boundary=----------------------------346435246262465368257857\r\n";
if (data->type == RequestType::MULTIPART) {
size_t filesize = 0;
char* fileData = fileToString("img.png", "rb", &filesize);
WinHttpAddRequestHeaders(hRequest, additionalHeaders, -1L, WINHTTP_ADDREQ_FLAG_ADD);
char postData1[] =
"----------------------------346435246262465368257857\r\n"
"Content-Disposition: form-data; name=\"file\"; filename=\"img.png\"\r\n"
"Content-Type: image/png\r\n\r\n";
char postData2[] =
"\r\n----------------------------346435246262465368257857\r\n"
"Content-Disposition: form-data; name=\"newData\"\r\n"
"Content-Type: application/json\r\n\r\n"
"{\"dataType\":\"DEVICE_SETTINGS\"}"
"\r\n----------------------------346435246262465368257857--\r\n";
if (hRequest)
bResults = WinHttpSendRequest(hRequest,
WINHTTP_NO_ADDITIONAL_HEADERS,
0, WINHTTP_NO_REQUEST_DATA, 0,
lstrlenA(postData1) + lstrlenA(postData2) + filesize, NULL);
DWORD dwBytesWritten = 0;
if (bResults)
bResults = WinHttpWriteData(hRequest, postData1, lstrlenA(postData1), &dwBytesWritten);
if (bResults)
bResults = WinHttpWriteData(hRequest,(LPCVOID) fileData, filesize, &dwBytesWritten);
if (bResults)
bResults = WinHttpWriteData(hRequest, postData2, lstrlenA(postData2), &dwBytesWritten);
}
errorMessageID = ::GetLastError();
// End the request.
if (bResults)
bResults = WinHttpReceiveResponse(hRequest, NULL);
Valid request sent with Postman
Invalid request sent with WinHTTP
Postman configuration 1
Postman configuration 2
SpringBoot controller
Here is the error I receive in my SpringBoot app log:
2022-03-04 14:48:34.520 WARN 25412 --- [nio-8010-exec-6] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.web.multipart.support.MissingServletRequestPartException: Required request part 'file' is not present]
I would really appreciate any help here solving this mystery!

The MIME boundaries in your postdata1 and postdata2 strings are incomplete, which is why WireShark and the SpringBoot app are not parsing your data correctly.
Every MIME boundary in the body data must start with a leading --, followed by the value you specified in the Content-Type's boundary attribute, followed by a trailing -- in the final termination boundary.
Let's look at an simpler example that doesn't use any - in the boundary value at all, this should make it clearer to you:
POST /resource HTTP/1.1
Host: ...
Content-Type: multipart/form-data; boundary=myboundary\r\n";
--myboundary
Content-Disposition: form-data; name="file"; filename="img.png"
Content-Type: image/png
<file data>
--myboundary
Content-Disposition: form-data; name="newData"
Content-Type: application/json
<json data>
--myboundary--
As you can see above, and in Postman's data, the leading -- is present on each boundary, but is missing in your data:
Postman's boundary attribute declares a value with 26 leading -s, and each boundary in the body data begins with 28 leading -s.
Your boundary attribute declares a value with 28 leading -s, and each boundary in the body data also begins with 28 leading -s.
Hence, the leading -- is missing from each boundary in your data.
Simply remove 2 -s from the value in your Content-Type's boundary attribute, and then you should be fine.

Related

libcurl HTTP request set content-disposition and content-type in MIME data

I'm trying to upload an image on twitter using libcurl, I used the twurl command line tool to generate an HTTP request and see how it should look like, what I get is this:
POST /1.1/media/upload.json HTTP/1.1
Accept: */
Content-Type: multipart/form-data, boundary="00Twurl342528555775455418lruwT99"
Authorization: OAuth oauth_body_hash="XXX", oauth_consumer_key="XXX", oauth_nonce="XXX", oauth_signature="XXX", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1603308767", oauth_token="XXX", oauth_version="1.0"
Connection: close
Host: upload.twitter.com
Content-Length: 612739
--00Twurl342528555775455418lruwT99
Content-Disposition: form-data; name="media"; filename="image.png"
Content-Type: application/octet-stream
binary data of image.png
--00Twurl342528555775455418lruwT99--
The request that I can generate via libcurl (got it using curl verbose) for the moment is this one:
POST /1.1/media/upload.json HTTP/2
Host: upload.twitter.com
accept: */*
authorization: OAuth oauth_consumer_key="XXX",oauth_nonce="XXX",oauth_signature="XXX",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1603372043",oauth_token="XXX",oauth_version="1.0"
content-length: 268
content-type: multipart/form-data; boundary=------------------------d1b0fc28e693c24a
Using the following code:
curl_mime *mime = nullptr;
curl_mimepart *part = nullptr;
mime = curl_mime_init(request_handle);
part = curl_mime_addpart(mime);
curl_mime_name(part, "media");
curl_mime_filename(part, "image.png");
curl_easy_setopt(request_handle, CURLOPT_MIMEPOST, mime);
The problem is that I don't know how to make my request similar to the first one with libcurl, how do I specify Content-Type and Content-Disposition ?
Edit: solution
Full code
curl_mime* mime = nullptr;
curl_mimepart* part = nullptr;
/* initialize mime part */
mime = curl_mime_init(request_handle);
part = curl_mime_addpart(mime);
/* content-disposition: form-data; name="media"; filename="image.png" */
curl_mime_name(part, "media");
curl_mime_filename(part, "image.png");
/* add file content */
curl_mime_filedata(part, "image.png");
/* content-type: application/octet-stream */
curl_mime_type(part, "application/octet-stream");
/* link the MIME data to your curl handle */
curl_easy_setopt(request_handle, CURLOPT_MIMEPOST, mime);
I didn't do it to highlight the functions to use, but check function return.
how do I specify Content-Type and Content-Disposition ?
Just read the fine manual (which you can navigate to from the fine example postit2.c)
CURLcode curl_mime_type(curl_mimepart * part, const char * mimetype);
curl_mime_type sets a mime part's content type.
CURLcode curl_mime_filename(curl_mimepart * part, const char * filename);
curl_mime_filename sets a mime part's remote file name. When remote file name is set, content data is processed as a file, whatever is the part's content source. A part's remote file name is transmitted to the server in the associated Content-Disposition generated header.
The official libcurl tutorial is also a nice read.

HTTP POST REQUEST C++

I have been stuck on a small piece of code trying it out.
char httpRequest[] = "POST http://myhost.com/ HTTP/1.1\r\n"
"Host: myhost.com\r\n"
"Content-Type: application/x-www-form-urlencoded\r\n"
"Content-Length: 10\r\n"
"Connection: close\r\n"
"\r\n"
"text1=test";
sock_error2 = send(tcpsock, httpRequest, strlen(httpRequest) + 1, 0);
The request never reaches the destination server, i have packet dump on both sides.
Whenever i change content-length to 0, the packet is received by the server without the body of course. What am i doing in wrong in content-length or carriage returns?

How do I correctly setup the multipart formdata with libcurl (in C++) to upload a binary file

To provide a little background. I am not very experienced with Ethernet communications so I apologize in advance for that. I'm working on a project where I need to figure out how to upload a binary file.
I'm trying to upload a large binary file (~34MB) to an embedded device. I have a python code snippet that works but I'm trying to implement the same capability in a different application using C++. Using WireShark, I've captured the header from the python program that works as well as the header that I end up with in my C++ code which doesn't work as needed.
Success Uploading with Python
Here is the python code that works:
response = session.post('http://10.42.42.1:81/__FileUpload',
files={"upfile": open(filename, 'rb')},
stream=False)
Here is the header information extracted from the message which initiates the successful file upload:
POST /__FileUpload HTTP/1.1
Host: 10.42.42.1:81
Content-Length: 34112690
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.7.0 CPython/2.7.13 Windows/10
Connection: keep-alive
Content-Type: multipart/form-data; boundary=6a659e345a35419e99b66546c1bd9d4e
--6a659e345a35419e99b66546c1bd9d4e
Content-Disposition: form-data; name="upfile"; filename="TestFile.bin"
No Success with C++ Code
Here is the essence of the code that I'm using to upload the file in C++:
curl_mime *multipart;
curl_mimepart *part;
// Specify the target URL
std::string str(comms.BaseURL() + kFileUploadEndpoint);
curl_easy_setopt(pCurl, CURLOPT_URL, str.c_str());
multipart = curl_mime_init(pCurl);
part = curl_mime_addpart(multipart);
curl_mime_name(part, "upfile");
curl_mime_data(part, ("filename=\"" + FileName + "\"").c_str(), CURL_ZERO_TERMINATED);
part = curl_mime_addpart(multipart);
curl_mime_data_cb(part, fileSize, ReadCallback, SeekCallback, NULL, pFile);
curl_easy_setopt(pCurl, CURLOPT_MIMEPOST, multipart);
curl_easy_setopt(pCurl, CURLOPT_TIMEOUT, 90L);
res = curl_easy_perform(pCurl);
...
Here is the header information from running the C++ code:
--------------------------4977715f070a13da
Content-Disposition: form-data; name="upfile"
filename="TestFile.bin"
--------------------------4977715f070a13da
Content-Disposition: form-data
I realized that the header above does not contain the URL endpoint and such and noticed that apparently the message was split into two pieces. Here is the header content from the message sent before the message above.
POST /__FileUpload HTTP/1.1
Host: 10.42.42.1:81
Accept: */*
Content-Length: 31546130
Content-Type: multipart/form-data; boundary=------------------------4977715f070a13da
Expect: 100-continue
I can check the status from the embedded device during the upload and the one thing that I notice in particular is that when the upload is successful with python then the embedded device reports the filename being uploaded in the status content reply. However, when I run the C++ code the filename is blank when I check the status. Therefore the embedded device is obviously not able to extract the filename from the C++ message.
What the Embedded Device is Looking For
While I don't have access to the source code of the device I did get the following information from someone who does. He indicated that this is what the embedded device is looking for. It didn't help me to figure out how to get things working but it might help someone else more knowledgeable in this area.
<FORM METHOD=POST name="install" enctype="multipart/form-data" target="HiddenFrame" action="/__FileUpload" onsubmit="InstallAction(); return true;">
File to upload: <INPUT TYPE=FILE NAME="upfile" size=50><p>
<INPUT TYPE=SUBMIT VALUE="Submit" >
</FORM>
I would prefer using the libcurl 'curl_mime_...' methods to setup the file upload only because that approach is recommended over using the older HTTP post methods. However, I'm perfectly okay with using the older HTTP post methods if that is easier to do. I just want to get it working.
Thanks in advance for your time.
I ended up finding the solution.
It is posted here for anyone who could use it:
curl_mime *multipart;
curl_mimepart *part;
multipart = curl_mime_init(pCurl);
part = curl_mime_addpart(multipart);
curl_mime_name(part, "upfile");
curl_mime_filename(part, FileName.c_str());
curl_mime_data_cb(part, lSize, ReadCallback, SeekCallback, NULL, pFile);
part = curl_mime_addpart(multipart);
curl_easy_setopt(pCurl, CURLOPT_MIMEPOST, multipart);
// Now send the message
res = curl_easy_perform(pCurl);
// Free the post data
curl_mime_free(multipart);
...
curl_mime_data(part, ("filename=\"" + FileName + "\"").c_str(), CURL_ZERO_TERMINATED);
I presume this wants to set the mime part's file name, and then you should rather use curl_mime_filename, because the file name is not data. Each part has a name and data, but also meta-data such as file name. Setting the file name only of course then requires that you set the data separately.
If you rather want to set the data as well as the file name, then instead do it with curl_mime_filedata.
Also, take a look at the official libcurl example postit2.c.

Invalid header name in HTTP request

I'm manually making the following request using sockets in C++.
struct addrinfo hints, *res;
int sockfd;
char buf[2056];
int byte_count;
//get host info, make socket and connect it
memset(&hints, 0,sizeof hints);
hints.ai_family=AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
getaddrinfo("some_web_service.azurewebsites.net","80", &hints, &res);
sockfd = socket(res->ai_family,res->ai_socktype,res->ai_protocol);
char *header = "POST /api/factura HTTP/1.1\n"
"Host: some_web_service.azurewebsites.net\n"
"Content-Type: application/json\n"
"Accept: */*\n"
"Cache-Control: no-cache\n"
"Content-Length: 1188\n"
"{\"IdTransaccion\": \"6828836174244758\",\"InstitucionFinanciera\": \"Banco Azteca\",\"NumTarjeta\": \"0077\",\"Fecha\":\"2019-10-06T17:00:00\",\"SubTotal\": \"2000.00\",\"Moneda\": \"MXN\",\"Total\": \"2320.00\",\"TipoDeComprobante\": \"I\",\"FormaPago\": \"01\",\"MetodoPago\": \"PUE\",\"LugarExpedicion\": \"06300\",\"Emisor\": {\"Rfc\": \"LAN8507268IA\",\"Nombre\": \"Banco Azteca, S.A de C.V.\",\"RegimenFiscal\": \"601\"},\"Receptor\": {\"Rfc\": \"VEV0603275K9\",\"Nombre\": \"Test Receptor\",\"UsoCFDI\": \"G03\"},\"Conceptos\":{\"Concepto\":[{\"ClaveProdServ\":\"84111506\",\"ClaveUnidad\":\"E48\",\"Cantidad\":\"1\",\"Descripcion\":\"Producto1\",\"ValorUnitario\":\"1000.00\",\"Importe\":\"1000.00\",\"Impuestos\":{\"Traslados\":{\"Traslado\":{\"Base\":\"1000.00\",\"Impuesto\":\"002\",\"TipoFactor\":\"Tasa\",\"TasaOCuota\":\"0.160000\",\"Importe\":\"160.00\"}}}},{\"ClaveProdServ\":\"01010101\",\"ClaveUnidad\":\"E48\",\"Cantidad\":\"1\",\"Descripcion\":\"Producto2\",\"ValorUnitario\":\"1000.00\",\"Importe\":\"1000.00\",\"Impuestos\":{\"Traslados\":{\"Traslado\":{\"Base\":\"1000.00\",\"Impuesto\":\"002\",\"TipoFactor\":\"Tasa\",\"TasaOCuota\":\"0.160000\",\"Importe\":\"160.00\"}}}}]},\"Impuestos\":{\"TotalImpuestosTrasladados\":\"320.00\",\"Traslados\": {\"Traslado\": {\"Impuesto\": \"002\",\"TipoFactor\": \"Tasa\",\"TasaOCuota\": \"0.160000\",\"Importe\": \"320.00\"}}}}\n";
send(sockfd,header,strlen(header),0);
//all right ! now that we're connected, we can receive some data!
byte_count = recv(sockfd,buf,sizeof(buf),0);
byte_count = recv(sockfd,buf,sizeof(buf),0);
But the message received is:
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Date: Mon, 07 Oct 2019 21:34:20 GMT
Connection: close
Content-Length: 339
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Header</h2>
<hr><p>HTTP Error 400. The request has an invalid header name.</p>
</BODY></HTML>
Can anyone tell me what I'm doing wrong with my headers? I already tried using \r\n instead of \n but that doesn't seem to do nothing.
I believe you're missing an empty line between your headers and message body. Per the RFC (https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4):
Request (section 5) and Response (section 6) messages use the generic
message format of RFC 822 [9] for transferring entities (the payload
of the message). Both types of message consist of a start-line, zero
or more header fields (also known as "headers"), an empty line (i.e.,
a line with nothing preceding the CRLF) indicating the end of the
header fields, and possibly a message-body.
Try adding an extra \n after your last header line:
"Content-Length: 1188\n\n"
To ensure compatibility with applications that adhere strictly to the RFC, you should use \r\n (CRLF) as mentioned in the quote!

Send POST multipart/form-data request using restbed C++

I am working on a C++ rest client using restbed lib that will send a base64 encoded image using a POST request.
The code I wrote so far is :
auto request = make_shared< Request >(Uri("http://127.0.0.1:8080/ProcessImage"));
request->set_header("Accept", "*/*");
request->set_header("Content-Type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
request->set_header("Cache-Control", "no-cache");
request->set_method("POST");
string test = "------WebKitFormBoundary7MA4YWxkTrZu0gW"
"Content-Disposition:form-data;name=\"image\""
""
"testMessage"
"------WebKitFormBoundary7MA4YWxkTrZu0gW--";
request->set_body(imgContent);
auto response = Http::sync(request)
I am not sure how I should set the request body. I tried with simple image="blabla" and also with this long version message I took from postman.
But in every case I received a "error 400 Bad request" answer.
Update:
Tested also with this version of code but with no success:
auto request = make_shared< Request >(Uri("http://127.0.0.1:8080/ProcessImage"));
request->set_header("Accept", "*/*");
request->set_header("Host","127.0.0.1:8080");
request->set_method("POST");
request->set_header("Content-Type", "multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW");
request->set_header("Cache-Control", "no-cache");
string imgContent = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\n"
"Content-Disposition: form-data; name=\"image\"\r\n"
"\r\n"
"test\r\n"
"------WebKitFormBoundary7MA4YWxkTrZu0gW--\r\n";
request->set_body(imgContent
auto response = Http::sync(request);
The response I get from the server:
*** Response ***
Status Code: 400
Status Message: BAD REQUEST
HTTP Version: 1.0
HTTP Protocol: HTTP
Header 'Content-Length' > '192'
Header 'Content-Type' > 'text/html'
Header 'Date' > 'Sun, 04 Feb 2018 21:09:45 GMT'
Header 'Server' > 'Werkzeug/0.14.1 Python/3.5.4'
Body:<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not understand.</p>
²²²²∩x...
Also on the server side (which is using python flask) I added:
encoded_img = request.form.get('image') and printed the string. The print result was: "None"
Your body content is missing explicit line break characters at the end of each line. C++ does not insert them automatically for you.
Also, if you are going to send base64 data, you should include a Content-Transfer-Encoding header, too.
Try this:
string imgContent = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\n"
"Content-Disposition: form-data; name=\"image\"\r\n"
"Content-Transfer-Encoding: base64\r\n"
"\r\n"
"<base64 image data here>\r\n"
"------WebKitFormBoundary7MA4YWxkTrZu0gW--\r\n";
request->set_body(imgContent);