Socket connection 301 or 400 - c++

i have a problem with sending GET to my domain. With 80 port, i receive 301 moved permanently, with 443 port, i receive 400 Bad Request. When i turn off redirect to https, all ok.
std::string HttpGet(std::string szHost, std::string szApi) {
std::string request;
std::string response;
int resp_leng;
char buffer[1024];
struct sockaddr_in serveraddr;
int sock;
WSADATA wsaData;
if (WSAStartup(MAKEWORD(2, 0), &wsaData) != 0) L::Print(XorStr("web_parser: WSAStartup() failed"));
struct hostent* host;
host = gethostbyname(szHost.c_str());
int port = 80;
request += "GET " + szApi + " HTTP/1.0\r\n";
request += "Host: " + szHost + "\r\n";
request += "User-Agent: WebParser [Build: 1.1] \r\n";
request += "\r\n";
if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) L::Print(XorStr("web_parser: socket() failed"));
memset(&serveraddr, 0, sizeof(serveraddr));
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = *((unsigned long*)host->h_addr);
serveraddr.sin_port = htons((unsigned short)port);
struct in_addr** addr_list;
addr_list = (struct in_addr**)host->h_addr_list;
response = "";
if (connect(sock, (struct sockaddr*)&serveraddr, sizeof(serveraddr)) < 0) L::Print(XorStr("web_зarser: connect() failed"));
if (send(sock, request.c_str(), request.length(), 0) != request.length()) L::Print(XorStr("web_parser: send() sent a different number of bytes than expected"));
resp_leng = 1024;
while (resp_leng > 0) {
resp_leng = recv(sock, (char*)&buffer, 1024, 0);
if (resp_leng > 0) response += std::string(buffer).substr(0, resp_leng);
}
closesocket(sock);
WSACleanup();
const char* buf = response.c_str(); // Get HTTP response from URL
const char* content = strstr(buf, "\r\n\r\n");
if (content != NULL) {
content += 4; // Offset by 4 bytes to start of content
}
else {
content = buf; // Didn't find end of header, write out everything
}
return content;
}
Thank you.

The result codes 400 and 301 are correct in this case as #Steffen Ullrich wrote in his comment.
When you try to connect to the port 80 and the server is configured to do redirect to https then the server indeed tries to redirect to https and it sends a response '301 Moved Permanently' with a header 'Location: https://....'. The standard response phrase is 'Moved Permanently' in this case. A few responses may cause redirect. These responses have codes 300 - 399 and they are called 'redirect' responses.
When you try to connect to the port 443 then the communication does not start with HTTP protocol but the SSL/TLS handshake must be be done before HTTP communication starts and HTTP communication must be sent through the SSL/TLS session when it is established. You got 400 Bad Request because your code does not try to do SSL/TLS handshake.
There are a few options how to support SSL/TLS in your application
Implement SSL/TLS layer from scratch just using the socket API. That is a very difficult task even for a experienced developer. Perhaps you can try learning experience but do not do that for a real application.
Use an existing SSL/TLS library for example windows built-in SSL stuff. See SSL api for winsock?. It is easier than the first option but still quite complex task.
Use a library that support HTTP and HTTPS. There are lot of libraries. For example Microsoft provides WinHTTP. It is a recommended way to implement an HTTP/HTTPS clients because it is not necessary to reinvent a wheel.

Related

C Raw Sockets HTTP Request Always Returning Status 400

I normally write a bunch of Typescript/Javascript code, and I took the HTTP protocol for granted. I decided to write a C++ program that can make GET requests to a given URL with sockets to learn more about how HTTP works. I've coded up what I think is the basic structure of what I should be doing:
Create a socket
Connect the socket to host
Send request byte by byte
Received response byte by byte
The issue is that the request always returns status code 400:
>>>> Request sent:
GET /links HTTP/1.1
>>>> Response received:
HTTP/1.1 400 Bad Request
Server: cloudflare
Date: Sun, 11 Oct 2020 05:57:11 GMT
Content-Type: text/html
Content-Length: 155
Connection: close
CF-RAY: -
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>cloudflare</center>
</body>
</html>
Here's the code I'm using to create the socket and connect it to the host:
// ---- Create the socket
int sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd < 0) error("!!!! ERROR opening socket");
// ---- Look up (and parse) the server host, route
std::string host = parseHost(url);
std::string route = parseRoute(url);
struct hostent *server = gethostbyname(host.c_str()); // host.c_str())
if (server == NULL)
error("!!!! ERROR, no such host");
// ---- Fill in the server address structure
struct sockaddr_in serv_addr;
memset(&serv_addr, 0, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(PORT); // PORT
memcpy(&serv_addr.sin_addr.s_addr, server->h_addr, server->h_length);
// ---- Connect the socket to the given url's host
const int didConnect = connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr));
if (didConnect < 0) error("!!!! ERROR connecting to host");
If that logic seems correct, then here's the code that sends the request and receives the response:
// ---- Send the GET request byte by byte
int bytes, sent, received;
char response[MAX_CONTENT_LENGTH];
int total = sizeof(response) - 1;
sent = 0;
std::string message = "GET " + route + "HTTP/1.1" + "\r\n\r\n";
do
{
bytes = write(sockfd, message.c_str() + sent, total - sent);
if (bytes < 0) error("!!!! ERROR writing message to socket");
if (bytes == 0) break;
sent += bytes;
} while (sent < total);
std::cout << ">>>> Request sent: \n" << message << std::endl;
// ---- Receive the response byte by byte
memset(response, 0, sizeof(response));
total = sizeof(response) - 1;
received = 0;
do {
bytes = read(sockfd, response + received, total - received);
if (bytes < 0)
error("!!!! ERROR reading response from socket");
if (bytes == 0)
break;
received += bytes;
} while (received < total);
if (received == total)
error("!!!! ERROR storing complete response from socket (not enough space allocated)");
TLDR: Am I sending the right char array to the server? As of right now it has this format: "GET /route HTTP/1.1\r\n\r\n". Am I forgetting any headers/other information that lets the server know that it's a GET request?
>>>> Request sent:
GET /links HTTP/1.1
This is not a valid HTTP request. It is missing the Host header with the target domain name, i.e. it should be at least something like this
GET /links HTTP/1.1
Host: www.example.com
Apart from that your code simply tries to read data until the server closes the connection or the size of your internal buffer is reached. But HTTP/1.1 by default uses persistent connection. So the server might simply keep the connection open after the response is sent because it expects your next request, which means that your program will simply block doing nothing.
Note that HTTP is far more complex than it looks. Please study the actual standard if you want to implement it instead of using existing libraries. That's what standards are for.

Access web server by domain name instead of IP address with C/++ sockets [duplicate]

How do I make an http request using sockets on linux? currently, I'm getting
HTTP/1.1 301 Moved Permanently
//etc
Location: https://server.com
here's relevant part of code(the function is too big to post here):
/* Socket file descriptor. */
int sock;
struct sockaddr_in sockaddr;
struct hostent *host; /* Host information. */
sock = socket(AF_INET, /* IPV4 protocol. */
SOCK_STREAM, /* TCP socket. */
0); /* O for socket() function choose the correct protocol based on the socket type. */
if(sock == INVALID_SOCKET) return SOCK_GENERROR;
if((host = gethostbyname(server)) == NULL) {
close(sock);
return SOCK_HOSTNFOUND;
}
/* zero buffer */
memset(&sockaddr, 0, sizeof(sockaddr));
sockaddr.sin_family = AF_INET;
memcpy(&sockaddr.sin_addr,
host -> h_addr,
host -> h_length );
sockaddr.sin_port = htons(port);
if(connect(sock, (struct sockaddr *)&sockaddr, sizeof(sockaddr)) == INVALID_SOCKET) {
close(sock);
return SOCK_FERRCONN;
}
if(send(sock, sendbuf, bufsize, 0) == INVALID_SOCKET) {
close(sock);
return SOCK_FERRWRITE;
}
if((readed = recv(sock, recvbuffer, sizeof(recvbuffer), 0)) <= 0)
break;
in the call, server="server.com"; and port=80;
I tried to remove as possible my onw routines and type from this code to make more clean for you.
https requests look just like http requests, but with transparent encryption of the actual communication between the client and the server, and on a different default port. The good news is that transparent encryption allows you to program just like you're writing a regular HTTP client. The bad news is that the encryption is complex enough that you need a specialized library to implement it for you.
One such library is OpenSSL. Using OpenSSL, the minimal code for a client would look like this:
#include <openssl/ssl.h>
// first connect to the remote as usual, but use the port 443 instead of 80
// initialize OpenSSL - do this once and stash ssl_ctx in a global var
SSL_load_error_strings ();
SSL_library_init ();
SSL_CTX *ssl_ctx = SSL_CTX_new (SSLv23_client_method ());
// create an SSL connection and attach it to the socket
SSL *conn = SSL_new(ssl_ctx);
SSL_set_fd(conn, sock);
// perform the SSL/TLS handshake with the server - when on the
// server side, this would use SSL_accept()
int err = SSL_connect(conn);
if (err != 1)
abort(); // handle error
// now proceed with HTTP traffic, using SSL_read instead of recv() and
// SSL_write instead of send(), and SSL_shutdown/SSL_free before close()
HTTPS is just like HTTP, but its encapsulated in a cryptographic SSL layer. You will need to use a lib like OpenSSL to make those HTTPS connections.
OpenSSL will provide functions that replace the socket.h ones, to connect, read and write regular HTTP (or whatever other protocol you want to use) through a SSL channel, making the handling of the SSL part transparent to you.

Receive (recv) full request (e.g. curl HTTP)

How should this be done?
I want to receive a (rather long) HTTP request and cannot get this to work.
The problem: Without flags, recv does not read the whole message. I guess this is normal behavior. From what I understand using the MSG_WAITALL flag causes it to block until everything is received. However, in that case the call blocks forever (until I ctrl+c the client (curl) process.
Below there is a (still lengthy, but rather minimal) example snippet.
Sorry it mixes C and C++ style but I wanted to avoid own errors and sticked largely to example code with as few modifications as possible.
// Example largely based on: beej.us/guide/bgnet/output/html/singlepage/bgnet.html
// Mixes c-style and c++ due to my modifications.
// Only intended to reproduce a problem.
#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <sys/wait.h>
#include <sstream>
#include <iostream>
#include <cstring>
#define PORT "7004" // the port users will be connecting to
#define BACKLOG 10 // how many pending connections queue will hold
// get sockaddr, IPv4 or IPv6:
void *get_in_addr(struct sockaddr *sa)
{
if (sa->sa_family == AF_INET) {
return &(((struct sockaddr_in*)sa)->sin_addr);
}
return &(((struct sockaddr_in6*)sa)->sin6_addr);
}
int main(void)
{
int sockfd, new_fd; // listen on sock_fd, new connection on new_fd
struct addrinfo hints, *servinfo, *p;
struct sockaddr_storage their_addr; // connector's address information
socklen_t sin_size;
struct sigaction sa;
int yes=1;
char s[INET6_ADDRSTRLEN];
int rv;
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE; // use my IP
if ((rv = getaddrinfo(NULL, PORT, &hints, &servinfo)) != 0) {
fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
return 1;
}
// loop through all the results and bind to the first we can
for(p = servinfo; p != NULL; p = p->ai_next) {
if ((sockfd = socket(p->ai_family, p->ai_socktype,
p->ai_protocol)) == -1) {
perror("server: socket");
continue;
}
if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes,
sizeof(int)) == -1) {
perror("setsockopt");
exit(1);
}
if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
close(sockfd);
perror("server: bind");
continue;
}
break;
}
freeaddrinfo(servinfo); // all done with this structure
if (p == NULL) {
fprintf(stderr, "server: failed to bind\n");
exit(1);
}
if (listen(sockfd, BACKLOG) == -1) {
perror("listen");
exit(1);
}
printf("server: waiting for connections...\n");
while(1) { // main accept() loop
sin_size = sizeof their_addr;
new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size);
inet_ntop(their_addr.ss_family,
get_in_addr((struct sockaddr *)&their_addr),
s, sizeof s);
printf("server: got connection from %s\n", s);
const uint EASILY_ENOUGH = 1000000;
char* buffer = new char[EASILY_ENOUGH + 1];
auto bytesRead = recv(new_fd, buffer, EASILY_ENOUGH, MSG_WAITALL); // FREEZES UNTIL I KILL THE CLIENT
// auto bytesRead = recv(new_fd, buffer, EASILY_ENOUGH, 0); // DOES NOT READ FULL REQUEST
if (bytesRead == -1) {
perror("recv");
close(new_fd);
continue;
}
buffer[bytesRead] = 0;
printf("bytes read: %ld\n", bytesRead);
printf("request: %s\n", buffer);
delete[] buffer;
std::string content = "some content.";
std::ostringstream os;
os << "HTTP/1.1 200 OK\r\n" << "Content-Length: " << content.size() << "\r\n"
<< "Connection: close\r\n" << "Content-Type: " << "plain/text"
<< "; charset=" << "utf-8" << "\r\n"
<< "Access-Control-Allow-Origin: *" << "\r\n" << "\r\n" << content;
std::string response = os.str();
printf("Response has size: %ld\n", response.size());
auto bytesSent = send(new_fd, response.c_str(), response.size(), 0);
printf("Method send claims it has sent %ld bytes\n", bytesSent);
close(new_fd);
}
return 0;
}
The curl resquest i used:
curl 'http://<myhost>:7004/___THE START___jsondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwa___THE END'
With MSG_WAITALL my server produces the output:
server: waiting for connections...
server: got connection from <my ip>
and hangs, once I CTRL+C curl it progresses with:
bytes read: 6379
request: GET /___THE START___jsondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwasondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwa___THE END HTTP/1.1
Host: <myhost>:7004
User-Agent: curl/7.49.0
Accept: */*
Response has size: 144
Method send claims it has sent 144 bytes
Without MSG_WAITALL and the same request, the server produces the output:
server: waiting for connections...
server: got connection from <my ip>
bytes read: 1448
request: GET /___THE START___jsondsadasdasdzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwaghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewjfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiwehfoewhfiuewhfiewhfhfiuewhfiwehfiuewwehgurwadzughjdssftzghuijerdftzghujiesdhufdewojfoewfhoiw
Response has size: 144
Method send claims it has sent 144 bytes
And curl does not receive the HTTP response.
However, a shorter request properly receives the response:
curl 'http://myhost:7004/___THE START___THE END__'
> some content.
And the server properly received the full request:
server: got connection from <my ip>
bytes read: 128
request: GET /___THE START___THE END__ HTTP/1.1
Host: <myhost>:7004
User-Agent: curl/7.49.0
Accept: */*
Response has size: 144
Method send claims it has sent 144 bytes
I do understand that it is normal that not all TCP packets arrive right away. However, what is the correct way for me to assemble the full request? I also tried non-blocking variants but usually ran into situations where no more data was ready for reading. If necessary I can produce sample code for this, similarly.
PS: The problem manifests when request length and connection badness exceed a threshold. I cannot reproduce it on my own machine with queries to localhost, and depending on my connection, the request length where problems start to manifest varies.
HTTP is a protocol, it has structure and rules to it. Read RFC 2616, particularly Section 4 "HTTP Message".
Your recv code is not doing anything to follow the protocol at all. You can't just blindly read an arbitrary buffer of data and expect it to be the complete HTTP request correctly. You have to read the request according to the rules of the protocol. Specifically, you need to:
read a CRLF delimited line of text. This will contain the requested method, resource, and HTTP version.
then read a variable length list of CRLF delimited request headers. The list is terminated by a CRLF CRLF sequence.
then analyze the request method and headers to determine whether the request has an entity body. If so, the headers will tell you how it is encoded over the connection (see Section 4.4 "Message Length"), so you know how it needs to be read and when you need to stop reading.
then process the completed request, and send your response.
then close the connection, unless either:
the request asked for HTTP 1.0 and Connection: keep-alive is present in the request headers.
the request asked for HTTP 1.1+ and Connection: close is not present in the request headers.

SSL3_READ_BYTES:tlsv1 alert internal error, secure sockets, OpenSSL, C++, getting HTTPS page

I want to write a simple application, which grabs the content of the HTTPS page: https://httpbin.org/html.
I know how to access its insecure (HTTP) version (http://httpbin.org/html), I just need to send a request using sockets:
std::stringstream ss;
ss << "GET /html HTTP/1.1" << "\r\n"
<< "Host: httpbin.org\r\n"
<< "Connection: close"
<< "\r\n\r\n";
std::string request = ss.str();
// socket creation, connect
send(sock, request.c_str(), request.size(), 0);
And it works. However, when using secure sockets (#include <openssl/ssl.h>) I got the following error: error:14094438:SSL routines:SSL3_READ_BYTES:tlsv1 alert internal error when trying result = SSL_connect(ssl);
Here's the code:
#include <sys/stat.h>
#include <netdb.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <string>
#include <iostream>
#include <sstream>
int main(int argc, char **argv)
{
int sock;
char buff[1024];
struct sockaddr_in address;
struct hostent *host = NULL;
std::string servername = "httpbin.org";
if (!inet_aton(servername.c_str(), &address.sin_addr))
if (host = gethostbyname(servername.c_str()))
address.sin_addr = *(struct in_addr*)host->h_addr;
else
return -1;
address.sin_port = htons(443);
address.sin_family = AF_INET;
std::stringstream ss;
ss << "GET /html HTTP/1.1" << "\r\n"
<< "Host: httpbin.org\r\n"
<< "Connection: close"
<< "\r\n\r\n";
std::string request = ss.str();
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) != -1)
{
if (connect(sock, (struct sockaddr *)&address, sizeof(struct sockaddr_in)) != -1)
{
SSL_library_init();
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
SSL_CTX *ctx = SSL_CTX_new(TLSv1_2_method());
if (!ctx)
return -1;
int result = -1;
SSL *ssl = SSL_new(ctx);
SSL_set_fd(ssl, sock);
result = SSL_connect(ssl);
if (result == 0)
{
long error = ERR_get_error();
const char* error_str = ERR_error_string(error, NULL);
printf("%s\n", error_str);
return 0;
}
SSL_write(ssl, request.c_str(), request.size());
SSL_read(ssl, buff, 1024);
printf("%s\n", buff);
memset(buff, 0, 1024);
SSL_free(ssl);
SSL_CTX_free(ctx);
close(sock);
}
}
return 0;
}
... SSL routines:SSL3_READ_BYTES:tlsv1 alert internal error when trying result = SSL_connect(ssl);
This means that the server sends a TLSv1 alert back probably because it does not like the information send by the client in the handshake (ClientHello). This might be because the client only offers ciphers which the server does not supports, that the client uses a protocol version not supported or very often also because the server requires the TLS SNI extension to determine which certificate to provide to the client. Testing with other tools indicates that this is actually the case. The problem thus can be solved in your code by using the SNI extension:
SSL *ssl = SSL_new(ctx);
SSL_set_fd(ssl, sock);
// ADD THIS
SSL_ctrl(ssl, SSL_CTRL_SET_TLSEXT_HOSTNAME, TLSEXT_NAMETYPE_host_name, (void*)servername.c_str());
result = SSL_connect(ssl);
In case it is ever useful to anyone, I had the same error (irregularly ) in my C++ program and it was driving me mad:
error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version
I tried all combinations of OpenSSL, different TLS versions etc, etc.
In the end, I found the Ping I had implemented ( to check the socket was still up ) was being called while another thread's SSL_read / SSL_write() was active on the same SSL object.
This error is generated for a handshake failure, which is caused by a miss-match between what the client offers and what the server accepts.
For some situations, openssl will do a pretty good job of negotiating a working set between the client and server. For other situations, it's a hard failure and you will need to ammend your code. I have seen this error generated due to:
Protocol and cipher missmatch
SSL_set_min_proto_version()
SSL_set_max_proto_version()
SSL_set_cipher_list()
SNI required by the server but not offered by the client
SSL_set_tlsext_host_name()
Reused session offered by client but sessions not implemented on server
SSL_set_session()

recv function blocks with keep-alive (HTTP)

I'm learning about sockets in Windows, also HTTP protocol.
So I was doing some tests with keep-alive but for some reason recv hangs for about 5 seconds, here it's:
VOID TestWinsock()
{
WSADATA WsaData;
addrinfo hints;
addrinfo* hResult;
SOCKET hsocket;
int Result;
WSAStartup(MAKEWORD(2, 2), &WsaData);
RtlZeroMemory(&hints, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = IPPROTO_TCP;
Result = getaddrinfo("localhost", "80", &hints, &hResult);
if (Result != 0)
{
return;
}
hsocket = socket(hResult->ai_family, hResult->ai_socktype, hResult->ai_protocol);
if (hsocket == INVALID_SOCKET)
{
return;
}
Result = connect(hsocket, hResult->ai_addr, (int)hResult->ai_addrlen);
if (Result == SOCKET_ERROR)
{
return;
}
char* POSTContent;
char HTTPRequestBuffer[1024];
char RecvBuffer[1024];
char* HTTPRequest =
"POST %s HTTP/1.0\r\n"
"Host: %s\r\n"
"connection: keep-alive\r\n"
"Content-type: application/x-www-form-urlencoded\r\n"
"Content-Length: %u\r\n"
"\r\n"
"%s"
"\r\n"
"\r\n";
POSTContent = "variable1=10";
wsprintfA(
HTTPRequestBuffer,
HTTPRequest,
"/tests/test.php",
"127.0.0.1",
strlen(POSTContent),
POSTContent
);
int sent = send(hsocket, HTTPRequestBuffer, strlen(HTTPRequestBuffer), 0);
RtlZeroMemory(RecvBuffer, sizeof(RecvBuffer));
// here recv blocks for about 5 seconds, yeah, it reads only 100 bytes, tried with something like sizeof(RecvBuffer) as well.
recv(hsocket, RecvBuffer, 100, 0);
....
}
I want to know how to solve this problem or which is the proper way to use keep-alive.
In your HTTP request, you send an additional \r\n\r\n which you do not include in your Content-Length, thus the data and and the length announced does not match. The HTTP standard says that :
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body.
The MUST is important, it means the server is totally in its right to reject your request. The fact that you specified keep-alive can also mean that the server waits for a next, valid request. Since you only send 4 bytes, the server end up closing the connection anyway after 5s, and probably sends an error (this part is only speculation about what actually happens).