Boost asio custom HTTP server reading HTTP post requests - c++

My C++ application requires of an HTTP server and I decided to make my own, which is correctly working when sending HTTP Get requests but has some problems when reading HTTP Post requests.
The problem is that when sending HTTP Posts requests, the last header isn't read properly, so I can't get the post request.
This is my code:
void HttpSession::readHeaders(std::shared_ptr<HttpSession> pThis) {
boost::asio::async_read_until(pThis->socket_, pThis->buff_, '\r\n\r\n',
[pThis](const boost::system::error_code &e, std::size_t s) {
std::istream headersStream(&pThis->buff_);
std::string header;
while(std::getline(headersStream, header, '\n')) {
if(header != "\r") {
if(header.back() == '\r') header = header.substr(0, header.length() - 1);
qDebug() << QString::fromStdString(header);
//Some stuff to get content-length to contentLength_
}
}
if(contentLength_ > 0) {
qDebug() << "Reading:";
readBody(pThis);
}
std::shared_ptr<std::string> str = std::make_shared<std::string>(pThis->headers_.getResponse());
boost::asio::async_write(pThis->socket_, boost::asio::buffer(str->c_str(), str->length()),
[pThis, str](const boost::system::error_code &e, std::size_t s) {
qDebug() << "Written";
});
}
void HttpSession::readBody(std::shared_ptr<HttpSession> pThis) {
boost::asio::async_read(pThis->socket_, pThis->data_, boost::asio::transfer_at_least(1),
[pThis](const boost::system::error_code &e, std::size_t s) {
std::istream body(&pThis->data_);
std::string line;
body >> line;
qDebug() << QString::fromStdString(line);
});
}
The buff_ and data_ variable are declared as: boost::asio::streambuf. And the HttpSession class is the one which stores the headers and handles all the webpage serving. I didn't include the code of that class since it's not the problem.
The output of an HTTP Post request to /url is this one:
"POST /url HTTP/1.1"
"Host: 192.168.1.41:8080"
"Connection: keep-alive"
"Content-Length: 10"
"Cache-Control: max-age=0"
"Origin: http://192.168.1.41:8080"
"Upgrade-Insecure-Requests: 1"
"User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"
"Content-Type: application/x-www-form-urlencoded"
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
"Referer: http://192.168.1.41:8080/"
"Accept-Encoding: gzip, deflate"
"Accept-Langua"
Reading:
"ge:"
So as you can see, the Accept-Language header is not read properly although in this image you can see that the POST request is made properly.
Note that when sending GET requests everything works properly. So my guess is that it has something to do with reading asynchronously, since the async_read_until doesn't block, so async_read of the readBody function, reads before it should. Should I read synchronously, then? Will I be able to handle more than one client either way if I create one HttpSession class for each client? (I really don't need to support more than one user, but still, it would be nice).
For the client handling I do like this:
void HttpServer::run() {
using namespace boost::asio;
io_service io_service;
ip::tcp::endpoint endpoint{ip::tcp::v4(), 8080};
ip::tcp::acceptor acceptor{io_service, endpoint};
acceptor.listen();
runServer(acceptor, io_service);
io_service.run();
while(true) QThread::msleep(5000);
}
void HttpServer::runServer(boost::asio::ip::tcp::acceptor& acceptor, boost::asio::io_service& io_service) {
std::shared_ptr<HttpSession> ses = std::make_shared<HttpSession>(io_service);
acceptor.async_accept(ses->socket_,
[ses, &acceptor, &io_service](const boost::system::error_code& accept_error) {
runServer(acceptor, io_service);
if(!accept_error) HttpSession::interact(ses);
});
}
All kind of help will be appreciated! If you have any code improvement, please tell me. Thanks!

I think the problem is with '\r\n\r\n'. That's not a string but a char.
The compiler should normally warn you about this, with something like:
warning: implicit conversion from 'int' to 'char' changes value from 218762506 to 10 [-Wconstant-conversion]
Try replacing that with "\r\n\r\n".

Related

Why Boost.Asio SSL request returns 405 Not Allowed?

I am trying to send HTTPS request to a server and receive the page contents by only using Boost.Asio(not Network.Ts or Beast or others) by these code :
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <iostream>
int main() {
boost::system::error_code ec;
using namespace boost::asio;
// what we need
io_service svc;
ssl::context ctx(ssl::context::method::tlsv1);
ssl::stream<ip::tcp::socket> ssock(svc, ctx);
ip::tcp::endpoint endpoint(boost::asio::ip::make_address("157.90.94.153",ec),443);
ssock.lowest_layer().connect(endpoint);
ssock.handshake(ssl::stream_base::handshake_type::client);
// send request
std::string request("GET /index.html HTTP/1.1\r\n\r\n");
boost::asio::write(ssock, buffer(request));
// read response
std::string response;
do {
char buf[1024];
size_t bytes_transferred = ssock.read_some(buffer(buf), ec);
if (!ec) response.append(buf, buf + bytes_transferred);
} while (!ec);
// print and exit
std::cout << "Response received: '" << response << "'\n";
}
But I keep getting 405 Not Allowed on my local PC and 400 Bad Request on Coliru.
What did I do wrong?
... "GET /index.html HTTP/1.1\r\n\r\n"
This is not a valid HTTP/1.1 request. It must at least also contain a Host field and the value of the field must match the servers expectation, i.e.
"GET /index.html HTTP/1.1\r\nHost: example.com\r\n\r\n"
In general, HTTP might look easy but is actually complex and has several pitfalls. If you really need to do HTTP by your own please study the standard.

C++ Networking: Request method POST

I'm trying to communicate with some server. To get username (with permissions), I need to do something like registration: HTTP request method POST send json-like body containing {"devicetype": "devicename"}. O tried to do it with ASIO library.
asio::error_code ec;
asio::io_context context;
asio::ip::tcp::endpoint endpoint(asio::ip::make_address("ipAddress", ec), 80);
asio::ip::tcp::socket socket(context);
if (!ec)
{
std::cout << "Succesfully connected\n";
}
else
{
std::cout << "Failed to connect to address: \n" << ec.message() << std::endl;
}
if (socket.is_open())
{
std::string sRequest =
"POST /api HTTP/1.1\r\n"
"Host: ipAddress \r\n"
"Body: {\"devicetype\": \"devicename\"}"
"Connection: close\r\n\r\n";
socket.write_some(asio::buffer(sRequest.data(), sRequest.size()), ec);
/*Reading received message and getting error message from server*/
}
Error says: "invalid/missing parameters in body". The parameters are correct. The problem is probably with message formatting I am sending (sRequest). How can I specify json body to message?
Thanks for help.
What you have shown is not a properly formatted HTTP request. There is no Body header in HTTP. The JSON data needs to go after the \r\n\r\n that terminates the headers. And you need to add Content-Type and Content-Length headers so the server knows what kind of data you are posting and how large it is.
Try this instead:
std::string json = "{\"devicetype\": \"devicename\"}";
std::string sRequest =
"POST /api HTTP/1.1\r\n"
"Host: ipAddress\r\n"
"Content-Type: application/json\r\n"
"Content-Length: " + std::to_string(json.size()) + "\r\n"
"Connection: close\r\n\r\n" + json;

sending a GET command to an ssl server to get result

I am trying to send a get request to acounts.google.com to be able to implement a library for C++ OAuth to learn it.
I get the following code from this post: Creating a HTTPS request using Boost Asio and OpenSSL and modified it as follow:
int main()
{
try
{
std::string request = "/o/oauth2/v2/auth";
boost::system::error_code ec;
using namespace boost::asio;
// what we need
io_service svc;
ssl::context ctx(svc, ssl::context::method::sslv23_client);
ssl::stream<ip::tcp::socket> ssock(svc, ctx);
ip::tcp::resolver resolver(svc);
auto it = resolver.resolve({ "accounts.google.com", "443" }); // https://accouts.google.com:443
boost::asio::connect(ssock.lowest_layer(), it);
ssock.handshake(ssl::stream_base::handshake_type::client);
// send request
std::string fullResuest = "GET " + request + " HTTP/1.1\r\n\r\n";
boost::asio::write(ssock, buffer(fullResuest));
// read response
std::string response;
do
{
char buf[1024];
size_t bytes_transferred = ssock.read_some(buffer(buf), ec);
if (!ec) response.append(buf, buf + bytes_transferred);
std::cout << "Response received: '" << response << "'\n"; // I add this to see what I am getting from the server, so it should not be here.
} while (!ec);
// print and exit
std::cout << "Response received: '" << response << "'\n";
}
catch (const std::exception& e)
{
std::cerr << e.what() << std::endl;
if (std::string const * extra = boost::get_error_info<my_tag_error_info>(e))
{
std::cout << *extra << std::endl;
}
}
}
The problem that I have is as follow:
1- The results that I am getting is not what I am getting when I visit https://accounts.google.com/o/oauth2/v2/auth using a web browser. I essentially getting a message that they can not find the requested URL /o/oauth2/v2/auth
<p>The requested URL <code>/o/oauth2/v2/auth</code> was not found on this server. <ins>ThatÔÇÖs all we know.</ins>
How should I setup the GET commend so I can get the same result that I am getting with a browser?
2- The application hangs getting data from server, apparently the following loop is not right:
do
{
char buf[1024];
size_t bytes_transferred = ssock.read_some(buffer(buf), ec);
if (!ec) response.append(buf, buf + bytes_transferred);
} while (!ec);
What is the correct way of reading responce from the web server which is fast and read all data?
Edit 1
For reference based on accepted answer, I fixed the problem using the correct GET header as shown below:
// send request
std::string fullResuest = "GET " + request + " HTTP/1.1\r\n";
fullResuest+= "Host: " + server + "\r\n";
fullResuest += "Accept: */*\r\n";
fullResuest += "Connection: close\r\n\r\n";
boost::asio::write(ssock, buffer(fullResuest));
A HTTP/1.1 request must have a Host header. A simple experiment with OpenSSL will show the problem, i.e. the missing header:
$ openssl s_client -connect accounts.google.com:443
...
GET /o/oauth2/v2/auth HTTP/1.1
... The requested URL <code>/o/oauth2/v2/auth</code> was not found on this server. <ins>That’s all we know.</ins>
When adding the Host header instead we get a different response:
$ openssl s_client -connect accounts.google.com:443
...
GET /o/oauth2/v2/auth HTTP/1.1
Host: accounts.google.com
... >Required parameter is missing: response_type<
Apart from that HTTP/1.1 implicitly uses HTTP keep-alive, i.e. server and client might keep the connection open after the response is done. This means you should not read until the end of connection but should instead properly parse the HTTP header, extract the Content-length header and/or Transfer-Encoding header and behave according to their values. Or if you want it simpler use HTTP/1.0 instead.
For more information see the HTTP/1.1 standard.

Error:403 response for Boost HTTPS GET request

I have implemented a HTTPS client using boost::asio library, implementation went fine , yet the problem arises when I send a GET request that invokes some servlet like entity.
GET request
request_stream << "GET "<<"https://172.198.71.135:8085/jrnal_content/test?data=MFBLUQ==&iv=aHU5Rw=="<<" HTTP/1.0\r\n"
The request was successfully written from my end to the server, but the response I have been getting is 403 ("Forbidden"), later sometime I sent another GET request.
request_stream << "GET "<<"https://172.198.71.135:8085/users/Sign_in"<<" HTTP/1.0\r\n"
For which I've got 200, I really do not know what went wrong in my first request (that received 403), I am sure there is no problem with certificate and stuff,I've never implemented an HTTP client/server before,so I want to make sure whether am making a proper request call.
Please see my code below
boost::asio::io_service io_service1;
boost::asio::io_service &io_service(io_service1);
boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);
boost::asio::ssl::context& context_=ctx;
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> socket_(io_service,context_);
int main()
{
context_.set_options(boost::asio::ssl::context::default_workarounds| boost::asio::ssl::context::no_sslv2
| boost::asio::ssl::context::single_dh_use);
context_.set_password_callback(my_password_callback);
context_.use_certificate_chain_file("SSL\\rich.crt");
context_.use_private_key_file("SSL\\rich.key", boost::asio::ssl::context::pem);
boost::asio::ssl::context ctx(boost::asio::ssl::context::sslv23);
tcp::resolver resolver_(io_service);
tcp::resolver::query query("172.198.71.135", "https");
tcp::resolver::iterator endpoint_iterator = resolver_.resolve(query);
boost::asio::connect(socket_.lowest_layer(),endpoint_iterator);
socket_.lowest_layer().set_option(tcp::no_delay(true));
socket_.set_verify_mode(boost::asio::ssl::verify_peer);
socket_.set_verify_callback(boost::bind(verify_certificate));//(boost::asio::ssl::rfc2818_verification("172.198.71.135"));
socket_.handshake(boost::asio::ssl::stream_base::client);
boost::asio::placeholders::error, boost::asio::placeholders::iterator));
string path="https://172.198.71.135:8085/jrnal_content/test?";
temp=path+"data="+data+"&"+"iv="+Iv;
boost::asio::streambuf request;
std::ostream request_stream(&request);
request_stream << "GET "+temp+" HTTP/1.0\r\n" ;
request_stream << "Host: " <<"172.198.71.135"<< "\r\n";
request_stream << "Accept: */*\r\n";
request_stream << "Connection: close\r\n\r\n";
const char* header=boost::asio::buffer_cast<const char*>(request.data());
cout<<header<<endl;
try{
boost::asio::write(socket_, request);
request_stream.clear();
t=sizeof(request);
request.consume(t);
}catch(runtime_error e)
{ ss<<e.what();
string err=ss.str();
err="";
ss.str("");
}
boost::asio::streambuf response;
try{
boost::asio::read_until(socket_, response, "\r\n");
}catch(runtime_error e)
{
ss<<e.what();
string err=ss.str();
err="";
ss.str("");
}
std::istream response_stream(&response);
std::string http_version;
response_stream >> http_version;
unsigned int status_code;
response_stream >> status_code;
cout<<status_code<<" status_code"<<endl;
std::string status_message;
std::getline(response_stream, status_message);
if (!response_stream || http_version.substr(0, 5) != "HTTPS/")
{
}
if (status_code==200)
{
ss<<ID;
string SID=ss.str();
ss.str("");
boost::asio::read_until(socket_, response, "\r\n\r\n");
response_stream.clear();
t=sizeof(response);
response.commit(t);
}
else{
Sleep(6000);
continue;
}
Sleep(7000);
}
return 0;
}
string my_password_callback(size_t t, boost::asio::ssl::context_base::password_purpose p)//std::size_t max_length,ssl::context::password_purpose purpose )
{
std::string password;
return "12345";
}
bool verify_certificate()
{
return true;
}
Thanks to sehe for the valuable suggestions he has given, So At last, I found what caused the issue.
tcp::resolver::query query("172.198.71.135", "https");
The above query gets resolved to 172.198.71.135:443 and the port NO 443 is blocked intentionally.
request_stream << "GET "<<"https://172.198.71.135:8085/jrnal_content/test?data=MFBLUQ==&iv=aHU5Rw=="<<" HTTP/1.0\r\n"
So things get changed as
tcp::resolver::query query("172.198.71.135", "https");
to
tcp::resolver::query query("172.198.71.135", "8085");
The "three-digit mess" is what's commonly referred to as "a number". In this case, the number is HTTP Status Code of the webserver's response.
If you read up on "HTTP" and perhaps even that HTTP response code, you'll find that it means Forbidden:
A web server may return a 403 Forbidden HTTP status code in response to a request from a client for a web page or resource to indicate that the server can be reached and understood the request, but refuses to take any further action. Status code 403 responses are the result of the web server being configured to deny access, for some reason, to the requested resource by the client.
Make sure to supply the required HTTP Authentication

Connecting to an HTTPS server with boost::asio

I want to connect to an HTTPS server using boost::asio. I managed to successfully shake hands with the server, but I just can't manage to get the server to respond to my POST request.
This is the related code (I left out debugging and try-catch to save some space):
HTTPSClient::HTTPSClient()
{
ssl::context context(ssl::context::sslv23);
context.set_verify_mode(ssl::verify_peer);
context.set_default_verify_paths();
context.load_verify_file("certificate.pem");
mSSLSocket = new ssl::stream<ip::tcp::socket>(mIOService, context);
}
void HTTPSClient::SendRequest(const ptree &crPTree, const std::string cHost,
const std::string cURI)
{
tcp::resolver resolver(mIOService);
tcp::resolver::query query(cHost, "https");
resolver.async_resolve(query, boost::bind(&HTTPSClient::HandleResolve, this,
placeholders::error, placeholders::iterator, request));
}
void HTTPSClient::HandleResolve(const error_code &crError,
const iterator &criEndpoints, HTTPSRequest &rRequest)
{
async_connect(mSSLSocket->lowest_layer(), criEndpoints,
boost::bind(&HTTPSClient::HandleConnect, this, placeholders::error,
rRequest));
}
void HTTPSClient::HandleConnect(const error_code &crError, HTTPSRequest &rRequest)
{
mSSLSocket->lowest_layer().set_option(ip::tcp::no_delay(true));
mSSLSocket->set_verify_callback(ssl::rfc2818_verification(rRequest.mcHost));
mSSLSocket->handshake(ssl::stream_base::client);
// Write the json into a stringstream
std::ostringstream json;
boost::property_tree::write_json(json, rRequest.mcPTree);
std::string result;
result = json.str();
// Form the request
streambuf request;
std::ostream requestStream(&request);
requestStream << "POST " << rRequest.mcURI << " HTTP/1.1\r\n";
requestStream << "Host: " << rRequest.mcHost << "\r\n";
requestStream << "Accept: application/json\r\n";
requestStream << "Content-Type: application/json; charset=UTF-8\r\n";
requestStream << "Content-Length: " << result.length() << "\r\n";
requestStream << result << "\r\n\r\n";
write(*mSSLSocket, request);
streambuf response;
read_until(*mSSLSocket, response, "\r\n");
std::istream responseStream(&response);
}
read_until hangs until it throws the error read_until: End of file. Everything before that goes successfully, including the SSL handshake (which I just recently figured out).
I used to do everything asynchronously until I started debugging, and started trying to backtrace to the problem, to no avail. It would be awesome if someone could help me out after two painful days of debugging.
EDIT
I just realized it might be useful to add the contents of requestStream after composing the header:
POST /authenticate HTTP/1.1
Host: <hostname>
Accept: application/json
Content-Type: application/json; charset=UTF-8
Content-Length: 136
{
"username": "vijfhoek",
"password": "test123",
<other json content>
}
You need a double linefeed before the body (POST contents)
POST /authenticate HTTP/1.1
Host: <hostname>
Accept: application/json
Content-Type: application/json; charset=UTF-8
Content-Length: 136
{
"username": "vijfhoek",
"password": "test123",
<other json content>
}
Otherwise, the content will have been received by the server as header lines and the server just keeps waiting for 136 bytes of content data (also make sure that Content-Length is accurate, which it isn't in this example)
So, basically:
requestStream << "Content-Length: " << result.length() << "\r\n";
requestStream << "\r\n"; // THIS LINE ADDED
I managed to figure out what I was doing wrong. For some reason, I couldn't get boost to write data using the boost::asio::streambuf and std::ostream approach. Instead, I put the POST data in a std::string and sent it like this:
write(*mSSLSocket, boost::asio::buffer(requestString));
Which worked out fine.