Following HTTP response header using socket - c++

I have a problem in my code I did send HTTP request to facebook server but the problem i get HTTP/1.1 301 Moved Permanently
message.
What i want to do is to allow follow the header response link and give me the redirected page HTML,
Also I tried to send another HTTP request but I get no response so i thought there is function option that allow me to follow the header link.
So basically what i'm trying to do is this send http post request to facebook server with my user and password log in follow header redirect link give me response.
if it's possible please guide me to the right way to do it.
PS: I already read some RFC and it's still not clear.
Thanks for your time.
My code
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <netdb.h>
#include <iostream>
#include <string>
#include <cstring>
#include <cstdlib>
using namespace std;
int main()
{
int sockid, status;
unsigned strSize;
ssize_t count; //
struct sockaddr_in addr; // struct
ssize_t size = sizeof(addr);
sockid = socket(PF_INET, SOCK_STREAM, 0); // create socket
if(sockid < 0) //if error
{
cout<<"Cannot create sucket\n";
close(sockid);
exit(EXIT_FAILURE);
}
// access struct
addr.sin_family = AF_INET;
addr.sin_port = htons(80);
if(inet_aton("31.13.90.36",&addr.sin_addr) == 0)
{
cout << "Wrong address";
close(sockid);
exit(-1);
}
status = connect(sockid, (sockaddr*)&addr, sizeof(addr)); // attempt to establish a connection
// check
if(status < 0)
{
cout << "failed to establish a connection";
close(sockid);
exit(EXIT_FAILURE);
}
// sending HTTP request
char msg[] = "POST /login.php HTTP/1.1\r\n"
"HOST: m.facebook.com\r\n"
"Content-type: application/x-www-form-urlencoded\r\n"
"Content-Length: 147\r\n"
"Connection: Keep-Alive\r\n"
"\r\n"
"lsd=AVp5UV4F&version=1&ajax=0&width=0&pxr=0&gps=0&dimensions=0&m_ts=1481464104&li=KFlNWFL78UFJkrUnTV_sFFDQ&email=Minglixe&pass=test123&login=Log+In";
const int bufSize = 4000;
char * buf;
buf = new char [bufSize];
//unsigned bufSize = strlen(buf);
strSize = strlen(msg);
send(sockid, msg, strSize, 0); // send request
while((count = recv(sockid, buf, bufSize, 0)) > 0)// receive the request
{
buf[count] = '\0';
cout << buf << flush;
}
if(count < 0 )
{
cout << "error" ;
exit(EXIT_FAILURE);
}
delete[] buf;
close(sockid);
}

You will have to issue another HTTP request, a GET request, to the redirected URL given in the 301 response.
You will need to close this socket, and create a new socket for the new request. If the redirected URL is for the same domain, you could implement HTTP/1.1 pipelining and save yourself the trouble of tearing down one socket and creating a new one, but that would be overkill.
In addition, it is a near certainty that the 301 response also gave you a handful of cookies, which Facebook's server will expect you to return to it, in the redirected request. Otherwise none of what you're trying to do will have any chance of working.
Unfortunately, there are no shortcuts here, and no instant gratification. In order to do this correctly, you need to understand, and be familiar with, pretty much, the entirety of HTTP. The canonical reference for HTTP consists of RFC 7230 and RFC 7231 (there's also RFC 7232, RFC 7233, RFC 7234, RFC 7235, RFC 7236 and RFC 7237, but I don't believe you need them for your particular purpose). Rather than copy-pasting chunks of it here, I will direct you to the authoritative source, in RFC 7230 and 7231.
HTTP is not really that complicated, but it is not something that can be absorbed in a few hours, like POP3. Be prepared to invest a few days on reviewing that technical specification for HTTP, and studying it. When I implemented my own full-featured HTTP 1.1 client, it took me (in my spare time), a few months to work my way through the RFC 2616 predecessor, and hack up a cookie-supporting HTTP 1.1 client.
Furthermore, in order for your HTTP client to handle cookies, the canonical reference for how to handle cookies is RFC 6265. You will need to learn that too.
P.S. Hopefully the redirected URL is not an https URL, or Facebook permits non-encrypted connections to fetch content after logging in. Otherwise, I'm afraid, you will need to pull in your platform's SSL libraries, and also code a TLS/SSL session on top of the whole thing.

Related

linux sockets cpp recv() - cant recv fully data via http [duplicate]

This question already has an answer here:
Differ between header and content of http server response (sockets)
(1 answer)
Closed 1 year ago.
I'm making this socket HTTP client (very basic). When recv()'ing response data from example.com it works fine and writes it all to a buffer but when I try to revc any bigger amounts of data it stops at around 1500 bytes.
Right now all I'm trying to do is get the response written into the buffer (headers and all). Not trying to parse anything. But that isn't working. It works for a few iterations but then stops or hangs. I'm asking for help identifying the issue with this receive_response() function that causes these behaviors.
This is the function that revc's the HTTP response:
void tcp_client::receive_response(char *buffer) {
int bytes_recv = 0;
int total_bytes_recv = 0;
for (;;) {
bytes_recv = recv(sock, &buffer[total_bytes_recv], CHUNK_SIZE, 0);
if (bytes_recv <= 0) {
break;
} else {
total_bytes_recv += bytes_recv;
}
}
}
The main function:
int main(int argc, char **argv) {
http_client http;
char response[100000] = {0};
http.connect_to_host("go.com", 80);
http.send_request("GET / HTTP/1.1\r\n\r\n");
http.receive_response(response);
std::cout << response << std::endl;
return 0;
}
Thank you
You seem to expect the server to close the connection after the response is transmitted. A typical HTTP 1.1 server doesn't do that by default; they keep the connection open for further requests, unless the client explicitly asks otherwise via Connection: close header.
So, you receive all the data, and then the next recv call is sitting there, waiting for more data to arrive.
An HTTP 1.1 client is expected to detect the end of response via Content-Length header, or by decoding a chunked response as indicated by Transfer-Encoding: chunked header.

C++ Server Response Is displaying Header Information On Client Page WebSocket Winsock Windows VS

I am attempting to send an HTML page over a WebSocket using C++
EDIT: I've updated the response header to **not** specify
how many characters are being sent via char response[] = "...";
I've added to the header the Content-Type and Content-Length headers
with length value being 1024. The result is still the same.
The server sends the response to the web page client and the
web page prints the header information instead of using the HTML tags.
I'm aware C++ isn't exactly a preferred language for this, but it's the language I have to use.
I have already managed to "establish?" a connection with a client webpage in google chrome. The server is getting the initial request from the client and is able to send back a response that the client is accepting. It's just displaying the wrong part.
Request Header:
Host: hostinfo.com:8xxx
Connection: keep-alive
Upgrade-Insecure-Request: 1
DNT: 1
User-Agent: Mozilla/5.0 (chrome etc...)
Accept: (encrypted message)
Server Response:
HTTP/1.1 200 OK
Upgrade-Insecure-Request: 1
Connection: keep-alive
Content-Length: 1024
Content-Type: text/html
(space for separation of header and HTML info using \r\n\r\n (space
after n))
test
(and the end page response \r\n\r\n (space after n)
However, when the webpage connects, instead of displaying "test" it's displaying the entire set of characters in the header including "HTTP/1.1 200 OK... etc"
I'm new to HTTP protocols and handshakes, I've gotten to this point in just a few days using articles like this, and this, and reading the RFC indexes but there doesn't seem to be any answers for this specific problem. I could understand handshake errors that resulted in a failure to send data between the server and client but sending the header instead of the HTML code is beyond me. Thanks for any help/insight.
TLDR:
Why would the client be displaying the header information rather than using the HTML code to display an HTML page?
Client-Side HTML:
<form action="http://xxxxxxx.com:8xxx" method="POST">
<div>
<input>
</div>
<div>
<button>Test Connection</button>
</div>
</form>
Server-Side C++ Code:
Header File
#pragma once
#undef UNICODE
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winsock2.h>
#include <ws2tcpip.h>
#include <iostream>
#include <string>
#include <stdlib.h>
#include <stdio.h>
// Need to link with Ws2_32.lib
#pragma comment (lib, "Ws2_32.lib")
// #pragma comment (lib, "Mswsock.lib")
class html_connection
{
public:
static int client_handler_thread(int index);
static int server();
};
Server-Side C++ Code:
Class File
#include "pch.h"
#include "html_connection.h"
#pragma warning(disable : 4996)
SOCKET Connections[100]; // client threads (up to 100)
int ConnectionCounter = 0;
int html_connection::server()
{
WSADATA ws;
WORD dllV = MAKEWORD(2, 1);
if (WSAStartup(dllV, &ws) != 0)
{
MessageBoxA(NULL, "Winsock startup failed", "Error", MB_OK | MB_ICONERROR);
}
SOCKADDR_IN addr; // create an address
int addrSize = sizeof(addr); //length of address
addr.sin_addr.s_addr = inet_addr("xxx.x.x.xx"); //Broadcast IP
addr.sin_port = htons(8000); //Port
addr.sin_family = AF_INET; // IPv4 socket
SOCKET s = socket(AF_INET, SOCK_STREAM, NULL); // listener socket
bind(s, (SOCKADDR*)&addr, sizeof(addr)); // binds address to socket
listen(s, SOMAXCONN); // max possible listeners
SOCKET newC; // client socket
for (int i = 0; i < 100; i++) // ensures limited number of clients can connect
{
newC = accept(s, (SOCKADDR*)&addr, &addrSize); // connect
if (newC == 0) // if client con failed
{
std::cout << "connect failed" << std::endl;
}
else // connected
{
Connections[i] = newC; // store new connection in array of connections
ConnectionCounter += 1;
CreateThread(NULL, NULL, (LPTHREAD_START_ROUTINE)client_handler_thread, (LPVOID)(i), NULL, NULL);
}
}
return 0;
}
int html_connection::client_handler_thread(int index)
{
// buffer for the client request
char clientInput[256];
// Response to the client
char response[] = "HTTP/1.1 200 OK \r\n Connection: keep-alive \r\n Content-Length:1024 \r\n Upgrade-Insecure-Requests: 1 \r\n Content-Type: text/html \r\n \r\n\r\n <html> </html> \r\n\r\n ";
while (true)
{
// gets the request from the client and prints it to the console
recv(Connections[index], clientInput, sizeof(clientInput), 0);
std::cout << clientInput << std::endl;
// send server response
send(Connections[index], response, sizeof(response), 0);
}
closesocket(Connections[index]);
}
Output: Client - Webpage (initial page - pre-request)
Output: Client - Webpage (post-request)
Output: Server - Console output (single client connection)

SSL3_READ_BYTES:tlsv1 alert internal error, secure sockets, OpenSSL, C++, getting HTTPS page

I want to write a simple application, which grabs the content of the HTTPS page: https://httpbin.org/html.
I know how to access its insecure (HTTP) version (http://httpbin.org/html), I just need to send a request using sockets:
std::stringstream ss;
ss << "GET /html HTTP/1.1" << "\r\n"
<< "Host: httpbin.org\r\n"
<< "Connection: close"
<< "\r\n\r\n";
std::string request = ss.str();
// socket creation, connect
send(sock, request.c_str(), request.size(), 0);
And it works. However, when using secure sockets (#include <openssl/ssl.h>) I got the following error: error:14094438:SSL routines:SSL3_READ_BYTES:tlsv1 alert internal error when trying result = SSL_connect(ssl);
Here's the code:
#include <sys/stat.h>
#include <netdb.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <string>
#include <iostream>
#include <sstream>
int main(int argc, char **argv)
{
int sock;
char buff[1024];
struct sockaddr_in address;
struct hostent *host = NULL;
std::string servername = "httpbin.org";
if (!inet_aton(servername.c_str(), &address.sin_addr))
if (host = gethostbyname(servername.c_str()))
address.sin_addr = *(struct in_addr*)host->h_addr;
else
return -1;
address.sin_port = htons(443);
address.sin_family = AF_INET;
std::stringstream ss;
ss << "GET /html HTTP/1.1" << "\r\n"
<< "Host: httpbin.org\r\n"
<< "Connection: close"
<< "\r\n\r\n";
std::string request = ss.str();
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) != -1)
{
if (connect(sock, (struct sockaddr *)&address, sizeof(struct sockaddr_in)) != -1)
{
SSL_library_init();
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
SSL_CTX *ctx = SSL_CTX_new(TLSv1_2_method());
if (!ctx)
return -1;
int result = -1;
SSL *ssl = SSL_new(ctx);
SSL_set_fd(ssl, sock);
result = SSL_connect(ssl);
if (result == 0)
{
long error = ERR_get_error();
const char* error_str = ERR_error_string(error, NULL);
printf("%s\n", error_str);
return 0;
}
SSL_write(ssl, request.c_str(), request.size());
SSL_read(ssl, buff, 1024);
printf("%s\n", buff);
memset(buff, 0, 1024);
SSL_free(ssl);
SSL_CTX_free(ctx);
close(sock);
}
}
return 0;
}
... SSL routines:SSL3_READ_BYTES:tlsv1 alert internal error when trying result = SSL_connect(ssl);
This means that the server sends a TLSv1 alert back probably because it does not like the information send by the client in the handshake (ClientHello). This might be because the client only offers ciphers which the server does not supports, that the client uses a protocol version not supported or very often also because the server requires the TLS SNI extension to determine which certificate to provide to the client. Testing with other tools indicates that this is actually the case. The problem thus can be solved in your code by using the SNI extension:
SSL *ssl = SSL_new(ctx);
SSL_set_fd(ssl, sock);
// ADD THIS
SSL_ctrl(ssl, SSL_CTRL_SET_TLSEXT_HOSTNAME, TLSEXT_NAMETYPE_host_name, (void*)servername.c_str());
result = SSL_connect(ssl);
In case it is ever useful to anyone, I had the same error (irregularly ) in my C++ program and it was driving me mad:
error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version
I tried all combinations of OpenSSL, different TLS versions etc, etc.
In the end, I found the Ping I had implemented ( to check the socket was still up ) was being called while another thread's SSL_read / SSL_write() was active on the same SSL object.
This error is generated for a handshake failure, which is caused by a miss-match between what the client offers and what the server accepts.
For some situations, openssl will do a pretty good job of negotiating a working set between the client and server. For other situations, it's a hard failure and you will need to ammend your code. I have seen this error generated due to:
Protocol and cipher missmatch
SSL_set_min_proto_version()
SSL_set_max_proto_version()
SSL_set_cipher_list()
SNI required by the server but not offered by the client
SSL_set_tlsext_host_name()
Reused session offered by client but sessions not implemented on server
SSL_set_session()

How do I get a Win7 app to communicate with a website?

I have sought in vain for a book entitled "Website Communication for Dummies". Can anyone suggest some good reading material / tutorial for me to consult?
Here is where I am at: I have a 32-bit Windows app I have written in C++ using Visual Studio 2010 C++ Express. The app facilitates User selection of an URL in text format == i.e., www.maps.google.com -- and then creates a socket and connects it, etc. The problem is that I can use the "send" command w/o error, but I have no idea what content to send in the 2nd argument, which is a const char[].
I've tried simple commands like "dump" and "refresh" for various websites, but the recv() function merely returns 0 (bytes received) after a long delay.
Thanks for attending to this.
To understand what sort of data goes back and forth between web server and a client, look at the RFC (or start with a tutoral).
When you have the understanding of the protocol and played with raw sockets, look for C or C++ implementations. libcurl would be one such. I also think Windows has build-in support for HTTP clients in Windows SDK.
You'll probably want to send something like
GET / HTTP/1.1
to get a proper http response. However most sites will disregard requests that don't include certain HTTP headers (e.g. Host). I would advise looking up http client libraries in C++ to do some of the grunt work for you, writing your own http request building code is very much reinventing the wheel.
First use send( GET / HTTP/1.1.. ) to the webserver to make a request, and after that use recv(Socket,buffer..) to download the website HTML code into a buffer.
send(Socket,"GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n", strlen("GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n"),0);
Winsock code:
#include <winsock2.h>
#include <windows.h>
#include <iostream>
#pragma comment(lib,"ws2_32.lib")
using namespace std;
int main (){
WSADATA wsaData;
if (WSAStartup(MAKEWORD(2,2), &wsaData) != 0) {
cout << "WSAStartup failed.\n";
system("pause");
return 1;
}
SOCKET Socket=socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
struct hostent *host;
host = gethostbyname("www.google.com");
SOCKADDR_IN SockAddr;
SockAddr.sin_port=htons(80);
SockAddr.sin_family=AF_INET;
SockAddr.sin_addr.s_addr = *((unsigned long*)host->h_addr);
cout << "Connecting...\n";
if(connect(Socket,(SOCKADDR*)(&SockAddr),sizeof(SockAddr)) != 0){
cout << "Could not connect";
system("pause");
return 1;
}
cout << "Connected.\n";
send(Socket,"GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n", strlen("GET / HTTP/1.1\r\nHost: www.google.com\r\nConnection: close\r\n\r\n"),0);
char buffer[10000];
int nDataLength;
while ((nDataLength = recv(Socket,buffer,10000,0)) > 0){
int i = 0;
while (buffer[i] >= 32 || buffer[i] == '\n' || buffer[i] == '\r') {
cout << buffer[i];
i += 1;
}
}
closesocket(Socket);
WSACleanup();
system("pause");
return 0;
}

Sending POST data using winsock (and receiving them with PHP server side script)

So I thought I would play with HTTP a bit and try to send simple plain(not encoded) text from my program to a server. However, something is not right and I don't know what.
Here is the server side PHP script, I have tested it by sending POST data from HTML form and it worked just great, so I guess there isn't anything wrong on server side.
<?php
$file = 'postData.txt';
$somecontent = $_POST['dat'];
$fp = fopen($file, 'w') or die('Could not open file!');
fwrite($fp, "$somecontent") or die('Could not write to file');
fclose($fp);
?>
Here is the program(this code includes some unused parts like reading file content in buffer etc., that's cuz I am playing with it all the time and changing stuff every 5 seconds, don't mind it):
#include <windows.h>
#include "WinSock2.h"
#include <stdio.h>
#include <stdint.h>
#include <iostream>
int main()
{
WSADATA wsa;
if (WSAStartup(MAKEWORD(2, 2), &wsa) != 0)
return 0;
SOCKET fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
if (fd < 0) throw;
SOCKADDR_IN service;
service.sin_family = AF_INET;
service.sin_port = htons(80);
LPHOSTENT host = gethostbyname("123.17.25.123");
if (!host) throw;
service.sin_addr = *((LPIN_ADDR)*host->h_addr_list);
if (connect(fd, (SOCKADDR *)&service, sizeof(service)) < 0) throw;
FILE *f = fopen("file.txt", "rb");
if (!f) throw;
uint32_t len = 0;
fseek(f, 0x00, SEEK_END);
len = ftell(f);
fseek(f, 0x00, SEEK_SET);
char header[1024];
char *buffer = new char[len];
fread(buffer, sizeof(char), len, f);
sprintf(header,
"POST /recv.php HTTP/1.1\r\n"
"Host: 123.17.25.123\r\n"
"User-Agent: Mozilla Firefox/4.0\r\n"
"Content-Length: %d\r\n"
"Content-Type: application/x-www-form-urlencoded\r\n"
"Accept-Charset: utf-8\r\n\r\n",
len+4);
std::cout << header << std::endl;
send(fd, header, strlen(header), 0);
send(fd, "dat=", 4, 0);
send(fd, buffer, strlen(buffer), 0);
send(fd, "\r\n", 2, 0);
delete [] buffer;
fprintf(stderr, "Done\n");
closesocket(fd);
WSACleanup();
return 0;
}
So, what's wrong with it? Does anyone have any idea? :P
edit1: I monitored the traffic with wireshark and tried to run the program few times, but no packets were captured. Strange, it does not even send anything anywhere.
edit2: Thanks to TokenMacGuy got it working. Code above is lame, but it will read all file content and send it as POST data to your server, hopefully it will be useful for noobs like me to learn. Thank you once again!
You aren't recieving the data because you aren't actually sending any data. Although buffer appears in the sprintf, there's no format specifier to consume it (only the length is formatted).
Try removing the buffer from the sprintf call altogether, and then call send twice, once for the headers (as you already do) and again to send the actual data.
Or maybe you don't intend to send any data from the file you read. You just want to get that dat=somedatar. The problem is that you indicate the content-type as text/plain, in which case the server won't interpret it at all. The content type should probably be application/x-www-form-urlencoded. Since the dat parameter is part of the body, the content-length header must include it. If the content length doesn't match the actual number of bytes sent as content, conforming servers ignore the whole request (usually returning a 400-499 range response code).
POST should have two newlines before the data, not after it.
"Accept-Charset: utf-8\r\n\r\n"
"dat=somedata\r\n",