how to parse HTTP POST(file upload) stream? - c++

I am using actionscript engine to upload a file, the engine will select the file and send the file over network thru HTTP POST command, the document says the POST message is like:
POST /handler.cfm HTTP/1.1
Accept: text/*
Content-Type: multipart/form-data;
boundary=----------Ij5ae0ae0KM7GI3KM7ei4cH2ei4gL6
User-Agent: Shockwave Flash
Host: www.example.com
Content-Length: 421
Connection: Keep-Alive
Cache-Control: no-cache
------------Ij5GI3GI3ei4GI3ei4KM7GI3KM7KM7
Content-Disposition: form-data; name="Filename"
MyFile.jpg
------------Ij5GI3GI3ei4GI3ei4KM7GI3KM7KM7
Content-Disposition: form-data; name="photo"; filename="MyFile.jpg"
Content-Type: application/octet-stream
FileDataHere
------------Ij5GI3GI3ei4GI3ei4KM7GI3KM7KM7
Content-Disposition: form-data; name="Upload"
Submit Query
------------Ij5GI3GI3ei4GI3ei4KM7GI3KM7KM7--
In server side I have a C++ program listens on port 80 and parse the POST message. I only want the file name and file data. How to decode the file data using c++, is it base64 encoded and is there a library can do it for me? I want to decode the binary, and write it to the file, thanks!

No, there is no encoding. The body of each subpart of the multipart message is included as verbatim bytes. Consequently you have to be careful to choose a boundary string that doesn't occur anywhere in the file data.
To parse a multipart/form-data form submission you will enough of a MIME parser to parse the headers, pick out parameters you want like boundary and name, and split the received message by the boundary string. This is not completely trivial, so you may want to consider existing libraries.
(Unfortunately, what browsers actually do in HTTP differs a bit from the standard MIME rules outlined in RFC 1341. In particular, field name and filename parameters tend to include non-ASCII characters and unescaped quotes. But if you are generating the POST yourself hopefully you can steer clear of these areas of contention.)

In the absence of a "Content_Transfer_Encoding" header, it is assumed data is encoded in "7bit" (RFC 1521; RFC 1867; RFC 2616).
I don't know if tere's a C library to parse / decode HTTP POSTs. I'm pretty sure there are though :)

You can upload unlimited size of data/file. Try this solution
const char* ctype = "multipart/form-data; boundary=----WebKitFormBoundaryfm9qwXVLSbFKKR88";
size_t content_length = 1459606;
http_payload* hp = new http_payload(ctype, content_length);
if (hp->is_multipart()) {
int ret = hp->read_all("C:\\temp\\");
if (ret < 0) {
std::cout << hp->get_last_error() << std::endl;
hp->clear();
}
else {
std::string dir_str("C:\\upload_dir\\");
ret = hp->read_files([&dir_str](http_posted_file* file) {
std::string path(dir_str.c_str());
path.append(file->get_file_name());
file->save_as(path.c_str());
file->clear(); path.clear();
std::string().swap(path);
});
hp->clear();
std::cout << "Total file uploaded :" << ret << std::endl;
}
}
else {
int ret = hp->read_all();
if (ret < 0) {
std::cout << hp->get_last_error() << std::endl;
hp->clear();
}
else {
std::cout << "Posted data :" << hp->get_body() << std::endl;
hp->clear();
}
}
https://github.com/safeonlineworld/web_jsx/blob/0d08773c95f4ae8a9799dbd29e0a4cd84413d108/src/web_jsx/core/http_payload.cpp#L402

Related

How to get rid of the POST details in the json message?

I am receiving a msg contains JSON and I have a problem to parse into JSON details. The msg I receive is :
POST /api/school/2/order HTTP/1.1
User-Agent: Guzzle/4.0 curl/7.21.4 PHP/5.5.7
content-type: application/json
Content-Length: 12
{
"json msg is here"
}
I want to git rid of the first 4 lines ..
Any suggestions ?
That is because you are retrieving the data as plain text.
Then you would use libCURL to get the actual JSON data. Then parse it with some C++ library like jsoncpp.

How to send attachment using AWS SES in NodeJS?

As per the given doc at "https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SES.html#sendTemplatedEmail-property". It was using "sendTemplatedEmail" API we can send email using templates. It was successful. But I could not figure out how to add attachments to it.
In the 4th point of the "sendTemplatedEmail" API doc it says "The total size of the message, including attachments, must be less than 10 MB". How to add the attachment here in this sendTemplatedEmail API?
Also there is a API called "sendRawEmail". But that does not suit my requirement. I need to use templates and also attach documents. Does any one know what to do ??
Take a look at the SendRawEmail example:
/* The following example sends an email with an attachment: */
var params = {
Destinations: [],
FromArn: "",
RawMessage: {
Data: <Binary String>
},
ReturnPathArn: "",
Source: "",
SourceArn: ""
};
ses.sendRawEmail(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
/*
data = {
MessageId: "EXAMPLEf3f73d99b-c63fb06f-d263-41f8-a0fb-d0dc67d56c07-000000"
}
*/
});
Reference: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SES.html
Important: you need to understand the MIME type standards to include your attachment. Take a look at this article.
MIME was defined in 1992 by the Internet Engineering Task Force
(IETF). The distinguishing characteristic of a MIME message is the
presence of the MIME headers. As long as your mail recipients also
have e-mail software that is MIME-compliant (and most e-mail software
is), you can swap files containing attachments automatically.
EDIT: This article explains how to include attachment in your body.
MIME completes the illusion of file attachments by allowing the
message body to be divided into distinct parts, each with their own
headers. The content type multipart/mixed means that the content of
the body is divided into blocks separated by "--" + a unique string
guaranteed to not be found anywhere else in the message. If you say
that your boundary string is "MyBoundaryString", then all occurrences
of that string will be treated as a boundary. So it better not be in
the message the user typed or it won't be decoded correctly.
Wikipedia also gives an example:
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=frontier
This is a message with multiple parts in MIME format.
--frontier
Content-Type: text/plain
This is the body of the message.
--frontier
Content-Type: application/octet-stream
Content-Transfer-Encoding: base64
PGh0bWw+CiAgPGhlYWQ+CiAgPC9oZWFkPgogIDxib2R5PgogICAgPHA+VGhpcyBpcyB0aGUg
Ym9keSBvZiB0aGUgbWVzc2FnZS48L3A+CiAgPC9ib2R5Pgo8L2h0bWw+Cg==
--frontier--
I assume you are familiar with Base64.

Sending POST multipart/form-data request using c++ code

I am trying to write a client, which will connect to a server and make a HTTP POST multipart/form-data request. Please go through following code. With this code, I am not getting any error response.
I am able to get some response, but not exactly what I expet.
With this code I am able to access the URL, but I think server is not reading the parameters (payload).
I am not sure whether I am passing an appropriate payload.
Please help me to write an appropriate payload. Thanks.
void BuildHTTPRequest::BuildHeader1(stringstream &hdr,string payload)
{
hdr << "POST /home/one/user/send_event HTTP/1.1\r\n";
hdr << "Host:host.domain.com\r\n";
hdr << "Port:80\r\n";
hdr << "Content-Type:multipart/form-data; boundary=463762444806\r\n";
hdr << "Content-Length:" << payload.size() << "\r\n";
hdr << "\r\n";
hdr << payload;
}
void BuildHTTPRequest::BuildPayload1(string *payload)
{
(*payload).append("Content-Disposition: form-data; name=\"interface_version\"\r\n\r\n");
(*payload).append("01.0.01\r\n");
(*payload).append("-----------------------------463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"terminal_id\"\r\n\r\n");
(*payload).append("1\r\n");
(*payload).append("-----------------------------463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"device\"\r\n\r\n");
(*payload).append("display\r\n");
(*payload).append("-----------------------------463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"start_session\"\r\n\r\n");
(*payload).append("true\r\n");
(*payload).append("-----------------------------463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"first_contact\"\r\n\r\n");
(*payload).append("true\r\n");
(*payload).append("-----------------------------463762444806--\r\n\r\n");
}
You should also have a boundary delimiter before the first payload data and since you aren't putting any preamble before the multipart data, you need 2 CRLF between your request headers and your payload.
The requirement that the encapsulation boundary begins with a CRLF
implies that the body of a multipart entity must itself begin with a
CRLF before the first encapsulation line -- that is, if the "preamble"
area is not used, the entity headers must be followed by TWO CRLFs.
This is indeed how such entities should be composed.
Also, why do use more than 2 hyphens before your boundary value for the delimiter ?
Specs says:
The Content-Type field for multipart entities requires one parameter,
"boundary", which is used to specify the encapsulation boundary. The
encapsulation boundary is defined as a line consisting entirely of two
hyphen characters ("-", decimal code 45) followed by the boundary
parameter value from the Content-Type header field.
...
Thus, a typical multipart Content-Type header field might look like
this:
Content-Type: multipart/mixed; boundary=gc0p4Jq0M2Yt08jU534c0p
This indicates that the entity consists of several parts, each itself with a structure that is
syntactically identical to an RFC 822 message, except that the header
area might be completely empty, and that the parts are each preceded
by the line
--gc0p4Jq0M2Yt08jU534c0p
Try this instead:
void BuildHTTPRequest::BuildPayload1(string *payload)
{
(*payload).append("\r\n--463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"interface_version\"\r\n\r\n");
(*payload).append("01.0.01\r\n");
(*payload).append("--463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"terminal_id\"\r\n\r\n");
(*payload).append("1\r\n");
(*payload).append("--463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"device\"\r\n\r\n");
(*payload).append("display\r\n");
(*payload).append("--463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"start_session\"\r\n\r\n");
(*payload).append("true\r\n");
(*payload).append("--463762444806\r\n");
(*payload).append("Content-Disposition: form-data; name=\"first_contact\"\r\n\r\n");
(*payload).append("true\r\n");
(*payload).append("--463762444806--\r\n\r\n");
}

Accessing QNetworkRequest data before it's sent

Is there any way to see the data that will be sent (or has been sent) during (or after) a call to QNetworkAccessManager::post(QNetworkRequest,QByteArray) on the client side?
In other words, I would like to see the raw HTTP request in it's entirety:
POST /somepage.php HTTP/1.1
Host: example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 19
name=need&just=tosee
It was a while since I had to debug my requests and things may have been changed in Qt, but I had to access the different parts of the requests using various functions in order to obtain all the details.
I created a wrapper for the post function which would print the details before posting the request. Here's a code snippet that extracts and prints the URL, raw headers and data for instance:
void debugRequest(QNetworkRequest request, QByteArray data = QByteArray())
{
...
qDebug() << request.url().toString();
const QList<QByteArray>& rawHeaderList(request.rawHeaderList());
foreach (QByteArray rawHeader, rawHeaderList) {
qDebug() << request.rawHeader(rawHeader);
}
qDebug() << data;
...
}

HttpClient and extracting audio from server response

I am using Apache HttpClient to connect to a server for downloading a .wav file. I am using HTTP POST method in my program.
The server correctly responds with the following header and body:
> HTTP/1.1 200 OK\r\n Content-Disposition: attachment;
> filename=saveme1.mp3\r\n Content-Length: 6264\r\n
> Content-Transfer-Encoding: binary\r\n Content-Type: audio/mp3\r\n
How do I now extract the saveme1.mp3 file from the HTTP response? I am using the following code:
ResponseHandler<String> responseHandler = new BasicResponseHandler();
byte[] data = httpclient.execute(httppost, responseHandler).getBytes();
However, I am getting garbage when I am writing the data to a file.
FileOutputStream fileoutputstream = new FileOutputStream(outputFile);
for (int i = 0; i < data.length; i++)
fileoutputstream.write(data[i]);
If you want download mp3 I Think easiest way is :
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
Now you have entity and can call entity.getContent(); This give you you a inputStream , now you can save this stream with every method you want , ofcurse you need mime type and filename to save your file. if you have problem with filename and mime type tell me to add some sample code.
You are getting MIME attachment that you need to parse first. The BasicResponseHandler just return the response string, but you need the body of the attachment that contains the binary of your .mp3. You would need to do the following steps:
Understand the MIME format. You could skim the Wikipedia Entry for gaining quick familiarity
Once you understood, you need to create a MIME Parser. This would basically extract each part of the MIME message especially the body of your attachment. I think there should be something out there that you could reuse. You probably should look MimeMultipart. The only thing that I am not sure about it is whether it handles "binary" encoding in your message.
Create your own extension of ResponseHandler that will utilize the MIME Parser that you have in the previous step