HTTP Response Format incorrect? - c++

I was looking at creating WebServers with C++.
I followed this tutorial by Sloan Kelly, and referenced this answer here.
I wrote (improperly adapted) the following code (showing the sending part only):
std::stringstream make_response;
make_response << "HTTP/1.1 200 OK\r\n";
make_response << "Cache-Control: no-cache, private\r\n";
make_response << "Content-Type: text/html\r\n";
make_response << "Content-Length: 88\r\n";
make_response << "\r\n";
make_response << "Hello!";
std::string finished_response = make_response.str();
send(client, finished_response.c_str(), finished_response.length()+1, 0);
What is the problem here? When I connect to 127.0.0.1:8000, the screen is entirely blank.
I know that the request reaches me, as I have printed it in the console:
C:\Users\Jaideep Shekhar\OneDrive\Documents\Projects\ProjectC++\Library\v0.1\main>webserver.exe
Listening for incoming connections...
Client connected!
Client says: GET / HTTP/1.1
Host: 127.0.0.1:8000
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.75 Safari/537.36
Sec-Fetch-Mode: navigate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Sec-Fetch-Site: cross-site
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
#_M
Client disconnected.
You can copy the whole source code from here and open up your terminal:
g++ webserver.cpp -o webserver.exe -lws2_32 && webserver.exe
Note that this code is for Windows OS.
Edit: Thanks to a comment, I solved the problem (see code below). Any improvements or problems in the code are welcome to be pointed out.
Edit2: How would I send images? I am opening the file opened with std::ios::binary, but it is not being recieved by the browser.
Thanks!

Okay, looks like the format and headers were all correct after all. I made a mistake in responding that I am sending 88 bytes of content, while I sent 5.
In fact, to avoid these issues, I have now put the HTML in its separate file.
The relevant part looks like this now:
// Transfer the whole HTML to a std::string
std::fstream grab_content("home.html");
std::stringstream make_content;
make_content << grab_content.rdbuf();
std::string finished_content;
finished_content = make_content.str();
// Create the HTTP Response
std::stringstream make_response;
make_response << "HTTP/1.1 200 OK\r\n";
make_response << "Cache-Control: no-cache, private\r\n";
make_response << "Content-Type: text/html\r\n";
make_response << "Content-Length: " << finished_content.length() << "\r\n";
make_response << "\r\n";
make_response << finished_content;
// Transfer it to a std::string to send
std::string finished_response = make_response.str();
// Send the HTTP Response
send(client, finished_response.c_str(), finished_response.length(), 0); // flag = 0

Related

HTTP POST REQUEST C++

I have been stuck on a small piece of code trying it out.
char httpRequest[] = "POST http://myhost.com/ HTTP/1.1\r\n"
"Host: myhost.com\r\n"
"Content-Type: application/x-www-form-urlencoded\r\n"
"Content-Length: 10\r\n"
"Connection: close\r\n"
"\r\n"
"text1=test";
sock_error2 = send(tcpsock, httpRequest, strlen(httpRequest) + 1, 0);
The request never reaches the destination server, i have packet dump on both sides.
Whenever i change content-length to 0, the packet is received by the server without the body of course. What am i doing in wrong in content-length or carriage returns?

Invalid header name in HTTP request

I'm manually making the following request using sockets in C++.
struct addrinfo hints, *res;
int sockfd;
char buf[2056];
int byte_count;
//get host info, make socket and connect it
memset(&hints, 0,sizeof hints);
hints.ai_family=AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
getaddrinfo("some_web_service.azurewebsites.net","80", &hints, &res);
sockfd = socket(res->ai_family,res->ai_socktype,res->ai_protocol);
char *header = "POST /api/factura HTTP/1.1\n"
"Host: some_web_service.azurewebsites.net\n"
"Content-Type: application/json\n"
"Accept: */*\n"
"Cache-Control: no-cache\n"
"Content-Length: 1188\n"
"{\"IdTransaccion\": \"6828836174244758\",\"InstitucionFinanciera\": \"Banco Azteca\",\"NumTarjeta\": \"0077\",\"Fecha\":\"2019-10-06T17:00:00\",\"SubTotal\": \"2000.00\",\"Moneda\": \"MXN\",\"Total\": \"2320.00\",\"TipoDeComprobante\": \"I\",\"FormaPago\": \"01\",\"MetodoPago\": \"PUE\",\"LugarExpedicion\": \"06300\",\"Emisor\": {\"Rfc\": \"LAN8507268IA\",\"Nombre\": \"Banco Azteca, S.A de C.V.\",\"RegimenFiscal\": \"601\"},\"Receptor\": {\"Rfc\": \"VEV0603275K9\",\"Nombre\": \"Test Receptor\",\"UsoCFDI\": \"G03\"},\"Conceptos\":{\"Concepto\":[{\"ClaveProdServ\":\"84111506\",\"ClaveUnidad\":\"E48\",\"Cantidad\":\"1\",\"Descripcion\":\"Producto1\",\"ValorUnitario\":\"1000.00\",\"Importe\":\"1000.00\",\"Impuestos\":{\"Traslados\":{\"Traslado\":{\"Base\":\"1000.00\",\"Impuesto\":\"002\",\"TipoFactor\":\"Tasa\",\"TasaOCuota\":\"0.160000\",\"Importe\":\"160.00\"}}}},{\"ClaveProdServ\":\"01010101\",\"ClaveUnidad\":\"E48\",\"Cantidad\":\"1\",\"Descripcion\":\"Producto2\",\"ValorUnitario\":\"1000.00\",\"Importe\":\"1000.00\",\"Impuestos\":{\"Traslados\":{\"Traslado\":{\"Base\":\"1000.00\",\"Impuesto\":\"002\",\"TipoFactor\":\"Tasa\",\"TasaOCuota\":\"0.160000\",\"Importe\":\"160.00\"}}}}]},\"Impuestos\":{\"TotalImpuestosTrasladados\":\"320.00\",\"Traslados\": {\"Traslado\": {\"Impuesto\": \"002\",\"TipoFactor\": \"Tasa\",\"TasaOCuota\": \"0.160000\",\"Importe\": \"320.00\"}}}}\n";
send(sockfd,header,strlen(header),0);
//all right ! now that we're connected, we can receive some data!
byte_count = recv(sockfd,buf,sizeof(buf),0);
byte_count = recv(sockfd,buf,sizeof(buf),0);
But the message received is:
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Date: Mon, 07 Oct 2019 21:34:20 GMT
Connection: close
Content-Length: 339
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Header</h2>
<hr><p>HTTP Error 400. The request has an invalid header name.</p>
</BODY></HTML>
Can anyone tell me what I'm doing wrong with my headers? I already tried using \r\n instead of \n but that doesn't seem to do nothing.
I believe you're missing an empty line between your headers and message body. Per the RFC (https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4):
Request (section 5) and Response (section 6) messages use the generic
message format of RFC 822 [9] for transferring entities (the payload
of the message). Both types of message consist of a start-line, zero
or more header fields (also known as "headers"), an empty line (i.e.,
a line with nothing preceding the CRLF) indicating the end of the
header fields, and possibly a message-body.
Try adding an extra \n after your last header line:
"Content-Length: 1188\n\n"
To ensure compatibility with applications that adhere strictly to the RFC, you should use \r\n (CRLF) as mentioned in the quote!

Parse Exception from a simple POST request

I am using sockets in C++ to communicate with Elasticsearch. I can communicate with my ELK instance (it is running in a docker compose, the docker files are here).
Here is the request that I am sending:
POST /twitter/_doc/1? HTTP/1.1
Content-Type: application/json; charset=UTF-8
Host: 127.0.0.1:9200
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
'{ "post_date" : "2017-11-16T14:12:12", "message" : "trying out Elasticsearch FROM CPPPPP" }'
Connection: close
And here is the response that I am getting from the server:
HTTP/1.1 400 Bad Request
content-type: application/json; charset=UTF-8
content-length: 163
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}
I can confirm that I am sending and receiving this data by counting the bytes being sent and received.
I am confused because when I put that request into the Kibana console, it runs fine.
Any ideas on what would be wrong with the request that I am sending?
Here is the C++ code where I create the request:
std::string post_http = "";
post_http += "POST /twitter/_doc/1? HTTP/1.1\n";
post_http += "Content-Type: application/json; charset=UTF-8\n";
post_http += "Host: ";
post_http += aURL;
post_http += ":9200\n";
post_http += "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\n";
post_http += "'{ \"post_date\" : \"2017-11-16T14:12:12\", ";
post_http += "\"message\" : \"trying out Elasticsearch FROM CPPPPP\" }'";
post_http += "\r\nConnection: close\r\n\r\n";
As far as I can tell it would be an issue with how I am creating the request in the code above.
Based on your code, it appears you mixed body inside headers.
Thus you actually have no body in your HTTP message.
Please see https://en.wikipedia.org/wiki/HTTP_message_body for a description of how an HTTP message is composed, specifically pay attention to the need pof having an EMPTY line between headers and body.
In your case, a solution may be to change your post_http string to:
std::string post_http = "";
post_http += "POST /twitter/_doc/1? HTTP/1.1\r\n";
post_http += "Content-Type: application/json; charset=UTF-8\r\n";
post_http += "Host: " + aURL + ":9200\r\n";
post_http += "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\r\n";
post_http += "Connection: close\r\n"
post_http += "\r\n"; // EMPTY line between header and body.
post_http += "'{ \"post_date\" : \"2017-11-16T14:12:12\", ";
post_http += "\"message\" : \"trying out Elasticsearch FROM CPPPPP\" }'";
I've also changed line endings to be all consistent "\r\n", as specified in the linked article.
It may also help to define a string const to give a name to the line ending sequence, e.g.:
const char* HTTP_LINE_END = "\r\n";
The problem was coming from not having a Content-Body in the header, as well as how long the Socket was waiting for a response. It seems that contrary to the WinSock documentation, you should not call the shutdown method if you are waiting for a response from the server.
Here is what I ended up having for my finished method:
assert(strcmp(aMethod, "GET") || strcmp(aMethod, "POST") || strcmp(aMethod, "PUT"));
// TODO: Investigate if I actually need to reconnect the socket every time a request is made?
ConnectSocket();
char Request[MAX_REQUEST_LEN];
strcpy_s(Request, MAX_REQUEST_LEN, aMethod);
strcat_s(Request, MAX_REQUEST_LEN, " ");
strcat_s(Request, MAX_REQUEST_LEN, aIndexParam);
strcat_s(Request, MAX_REQUEST_LEN, RequestHeader);
strcat_s(Request, MAX_REQUEST_LEN, "Content-Length: ");
strcat_s(Request, MAX_REQUEST_LEN, std::to_string(strlen(aMsg)).c_str());
strcat_s(Request, MAX_REQUEST_LEN, "\r\n");
strcat_s(Request, MAX_REQUEST_LEN, ConnectionClose);
strcat_s(Request, MAX_REQUEST_LEN, aMsg);
strcat_s(Request, MAX_REQUEST_LEN, "\r\n");
As you can see, I moved to just using char * instead of std::String, which gave me about a 30% performance boost. Also, There is now a Content-Length tag in the header.
This is what my final headers look like:
POST /twitter/_doc/ HTTP/1.1
Content-Type: application/json; charset=UTF-8
Content-Encoding: identity
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
Host: HTTP/1.1
127.0.0.1:9200
Content-Length: 112
Connection: close
{ "user" : "kimchy", "post_date" : "2018-09-16T14:12:12", "message" : "trying out Elasticsearch FROM c++ BOII" }

Python-Requests Post request failed with 403 Forbidden

I was trying to use python-requests to login https://www.custommade.com/, but it keeps giving me a "403 forbidden error". I got the post_url and content of payload from httpfox
import requests
post_url = 'https://www.custommade.com/secure/login/api/'
client = requests.session()
csrftoken = client.get('https://www.custommade.com/').cookies['csrftoken']
header_info = {
'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36',
'Content-type': 'application/json'
}
payload = {'_method':'login','csrftoken': csrftoken,'email': MYEMAIL,'password':MYPWS}
r = client.post(post_url, json=payload, headers = header_info)
print r.status_code
Could someone help? I tried to login other website and this works fine.
If you would print the reponse text you are getting you would see that they print error to you that you are not accepting cookies.
When you are doing something like this - always try to simulate the browser as best as possible - that means you have to set-up all the headers and also do the steps the browser does.
So first open the webpage in your browser. Open the dev tools, network tab.
Now click on the login -> you see that the browser does request to the /secure/proxy
So your program has to do it too. Than to the actual request. Ensure that your request looks so much as the request from the browser - check the headers. You can see that they send the token there. (btw they do not send it in the post data as you did in your script). Also they are probably checking some other headers, because when you remove them - it doesn't work. So easiest way is to put all the headers as the browser.
Don't forget about the cookies. But this is done automatically because you are using session from requests.
Anyway this is working code:
import requests
post_url = 'https://www.custommade.com/secure/login/api/'
client = requests.session()
client.get('https://www.custommade.com/')
r = client.get('https://www.custommade.com/secure/proxy/')
csrftoken = r.cookies['csrftoken']
header_info = {
"Host" : "www.custommade.com",
"Connection" : " keep-alive",
"Origin" : " https://www.custommade.com",
"User-Agent" : " Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36",
"Content-Type" : " application/x-www-form-urlencoded",
"Accept" : " */*",
"X-Requested-With" : " XMLHttpRequest",
"X-CSRFToken" : csrftoken,
"DNT" : " 1",
"Referer" : " https://www.custommade.com/secure/proxy/",
"Accept-Encoding" : " gzip, deflate, br",
"Accept-Language" : " en-US,en;q=0.8,cs-CZ;q=0.6,cs;q=0.4,sk;q=0.2,ru;q=0.2",
}
payload = {'_method':'login','email': 'sdfasdf#safs.com','password':'asfdasf', 'remember':True}
r = client.post(post_url, data=payload, headers = header_info)
print r.text
print r.status_code
Print:
{"errors": "Oops! Something went wrong. Please ensure you are sending JSON data."}
400
^^ Means the password is wrong

How do I send a POST with boost::asio::ip::tcp::iostream?

From the Boost docs, you can send a GET through an iostream quite easily:
ip::tcp::iostream stream;
stream.expires_from_now(boost::posix_time::seconds(60));
stream.connect("www.boost.org", "http");
stream << "GET /LICENSE_1_0.txt HTTP/1.0\r\n";
stream << "Host: www.boost.org\r\n";
stream << "Accept: */*\r\n";
stream << "Connection: close\r\n\r\n";
stream.flush();
std::cout << stream.rdbuf();
When I modify the above to connect to my IIS server, it works fine. The problem comes in when I try to send a POST to my server. Then I get the error message "HTTP Error 400. The request verb is invalid."
Various online discussions make it seem that the problem is with separator characters in headers, but removing all question marks fixed nothing.
Is there something that I'm missing here? This forum discussion makes it look like POSTing with an iostream should be doable. Google hasn't been much use since post is such an overloaded word online.
Edit - here's an example of my POST. With a GET, the server will pick it up and the handler will complain that it wants a POST (as it should).
boost::asio::ip::tcp::iostream stream;
stream.connect("myurl.com", "http");
stream << "POST /.api/api.svc/objects/723aa707-4978-4062-bcc6-67b05783c4ec/comments/add\r\n";
stream << "Host: myurl.com\r\n";
stream << "Accept: */*\r\n";
stream << "Content-Type: application/x-www-form-urlencoded; charset=utf-8\r\n";
stream << "Content-Length: 51\r\n";
stream << "Connection: close\r\n\r\n";
stream << "message=%3Cp%3EHello%3C%2Fp%3E";
stream.flush();
std::cout << stream.rdbuf();
As #Omaha spotted, the POST request is invalid. The POST line should look something like:
stream << "POST /.api/api.svc/objects/723aa707-4978-4062-bcc6-67b05783c4ec/comments/add HTTP/1.0\r\n"
to be a valid HTTP request. See HTTP/1.1: Request