Parse Exception from a simple POST request - c++

Question

I am using sockets in C++ to communicate with Elasticsearch. I can communicate with my ELK instance (it is running in a docker compose, the docker files are here).
Here is the request that I am sending:
POST /twitter/_doc/1? HTTP/1.1
Content-Type: application/json; charset=UTF-8
Host: 127.0.0.1:9200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
'{ "post_date" : "2017-11-16T14:12:12", "message" : "trying out Elasticsearch FROM CPPPPP" }'
Connection: close
And here is the response that I am getting from the server:
HTTP/1.1 400 Bad Request
content-type: application/json; charset=UTF-8
content-length: 163
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}
I can confirm that I am sending and receiving this data by counting the bytes being sent and received.
I am confused because when I put that request into the Kibana console, it runs fine.
Any ideas on what would be wrong with the request that I am sending?
Here is the C++ code where I create the request:
std::string post_http = "";
post_http += "POST /twitter/_doc/1? HTTP/1.1\n";
post_http += "Content-Type: application/json; charset=UTF-8\n";
post_http += "Host: ";
post_http += aURL;
post_http += ":9200\n";
post_http += "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\n";
post_http += "'{ \"post_date\" : \"2017-11-16T14:12:12\", ";
post_http += "\"message\" : \"trying out Elasticsearch FROM CPPPPP\" }'";
post_http += "\r\nConnection: close\r\n\r\n";
As far as I can tell it would be an issue with how I am creating the request in the code above.

Answer 1

Based on your code, it appears you mixed body inside headers.
Thus you actually have no body in your HTTP message.
Please see https://en.wikipedia.org/wiki/HTTP_message_body for a description of how an HTTP message is composed, specifically pay attention to the need pof having an EMPTY line between headers and body.
In your case, a solution may be to change your post_http string to:
std::string post_http = "";
post_http += "POST /twitter/_doc/1? HTTP/1.1\r\n";
post_http += "Content-Type: application/json; charset=UTF-8\r\n";
post_http += "Host: " + aURL + ":9200\r\n";
post_http += "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\r\n";
post_http += "Connection: close\r\n"
post_http += "\r\n"; // EMPTY line between header and body.
post_http += "'{ \"post_date\" : \"2017-11-16T14:12:12\", ";
post_http += "\"message\" : \"trying out Elasticsearch FROM CPPPPP\" }'";
I've also changed line endings to be all consistent "\r\n", as specified in the linked article.
It may also help to define a string const to give a name to the line ending sequence, e.g.:
const char* HTTP_LINE_END = "\r\n";

Answer 2

The problem was coming from not having a Content-Body in the header, as well as how long the Socket was waiting for a response. It seems that contrary to the WinSock documentation, you should not call the shutdown method if you are waiting for a response from the server.
Here is what I ended up having for my finished method:
assert(strcmp(aMethod, "GET") || strcmp(aMethod, "POST") || strcmp(aMethod, "PUT"));
// TODO: Investigate if I actually need to reconnect the socket every time a request is made?
ConnectSocket();
char Request[MAX_REQUEST_LEN];
strcpy_s(Request, MAX_REQUEST_LEN, aMethod);
strcat_s(Request, MAX_REQUEST_LEN, " ");
strcat_s(Request, MAX_REQUEST_LEN, aIndexParam);
strcat_s(Request, MAX_REQUEST_LEN, RequestHeader);
strcat_s(Request, MAX_REQUEST_LEN, "Content-Length: ");
strcat_s(Request, MAX_REQUEST_LEN, std::to_string(strlen(aMsg)).c_str());
strcat_s(Request, MAX_REQUEST_LEN, "\r\n");
strcat_s(Request, MAX_REQUEST_LEN, ConnectionClose);
strcat_s(Request, MAX_REQUEST_LEN, aMsg);
strcat_s(Request, MAX_REQUEST_LEN, "\r\n");
As you can see, I moved to just using char * instead of std::String, which gave me about a 30% performance boost. Also, There is now a Content-Length tag in the header.
This is what my final headers look like:
POST /twitter/_doc/ HTTP/1.1
Content-Type: application/json; charset=UTF-8
Content-Encoding: identity
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
Host: HTTP/1.1
127.0.0.1:9200
Content-Length: 112
Connection: close
{ "user" : "kimchy", "post_date" : "2018-09-16T14:12:12", "message" : "trying out Elasticsearch FROM c++ BOII" }