I'm making an HTTP proxy in C++; when a client sends a GET or CONNECT request to the proxy, the proxy parses the HTTP header of the packet, resolve the hostname in it, opens another socket to the server destination and send client's request. Then the proxy will send server's response to the client.
Here's the GET and CONNECT requests from the client sent by the proxy to the server:
GET http://www.gstatic.com/generate_204 HTTP/1.1
CONNECT cr-input.getspeakit.com:443 HTTP/1.1
But when I parse a GET response from server, I find a 400 status code, i.e. Bad Request: this seems to be (from Wikipedia):
a malformed request syntax, invalid request message framing, or deceptive request routing.
Do I send wrong arguments to the server in the GET request?
GET. The syntax is not wrong but if the request has not been faked by you and it is really going to www.gstatic.com you can check yourself that any kind of requests generates a 40x status code. That's a domain used for Google for offloading static content. Whether that's still the case and why it returns 40x for the requests. Go to Google.
CONNECT. If you are forwarding the CONNECT to the server, this is wrong. A CONNECT is meant to open an end to end binary connection bypassing your proxy. The sequence would be:
Get connect request from client
Open TCP connection to IP:Port (after DNS resolution obviously)
Return "200 OK" to the client if the connection was successfully opened or an error code of your choice (plus optionally some explanation in Text/HTML form for the end user)
If data is received from either end -> forward it to the other end until one of the connections is closed, when you close the other end.
Related
We are using Kerberos to authenticate HTTPS request. We wish to reuse the connection for requests to same endpoint by using same curl handle. This works fine for HTTP connections. However, each rest request we sent will send two request, first will get 401 to get back authentication token and then 200 for the data. Then curl will close the connection and reconnect again for following rest request. This is extremely inefficient, is there a way to cut down all the authentication cost? Thanks a lot!
It is needed to make a HTTP2-response, which will force a client to reconnect to the same server to the same address.
In case of HTTP/1.1 it could be done sending 307 Temporary Redirect response with Connection: close header.
In HTTP/2 Connection: close header is ignored and redirect is performed without reconnection, which brings to redirect loop error.
Also, I've tried to send 421 Misdirected Request response to client with the same url, but Chrome browser is do nothing after receiving this response.
What is the most proper way to force a HTTP/2 client to reconnect? Which server response can be send? Maybe some kind of GOAWAY frame?
Following the graceful shutdown procedure that RFC 7540 recommends should result in a reconnection:
A server that is attempting to gracefully shut down a connection
SHOULD send an initial GOAWAY frame with the last stream identifier
set to 2^31-1 and a NO_ERROR code. This signals to the client that
a shutdown is imminent and that initiating further requests is
prohibited. After allowing time for any in-flight stream creation
(at least one round-trip time), the server can send another GOAWAY
frame with an updated last stream identifier. This ensures that a
connection can be cleanly shut down without losing requests.
Regarding 421 handling in Chrome, this bug https://bugs.chromium.org/p/chromium/issues/detail?id=546991 was opened to have Chrome re-open a new connection to the server, it's seen some activity recently.
Yes, the way on HTTP/2 to ask a client to reconnect for further requests is to send a GOAWAY frame to it. How this works depends on your server side implementation or framework for HTTP/2 support. E.g. a framework could intercept your Connection: close header and treat it as a request to close the connection after the request. But I guess most HTTP/2 implementations wouldn't like to do that, since they guess the header targets only the current request scope and not the whole connection. Alternatively the framework could provide a way in the request handler to not only access the request and response data but also to get a reference to the HTTP/2 connection, which can be used to send the GOAWAY.
This may help you Google HTTP2
Recently i started to write a proxy(web debugging) software.
and handled the GET request well.
sometimes i get CONNECT messages from the client, so i connect to the target server and reply the client by "200 Connection Established".
is that it all??
because after it i don't get any messages
from the server or the client.
so i got confused.
I want to know all the steps of https CONNECT message request and responses until an https site(like https://google.com) gets loaded.
Thank you.
The CONNECT request is used to set up a connection tunnel. This is used mainly to allow access to https sites through an http proxy.
The web proxy is expected to set up a proxy connection to the indicated host, and then proxy the traffic between the two connections, until one or the other terminates.
After establishing the connection, you expect to see either the client or the server start sending something to the other. Your proxy code should be prepared, at any time, to receive more data from either the client or the server, to be forwarded to the other party. If not, your proxy is not doing something correctly. It is not your proxy's job to handle https negotiation. Once the connection is established, your proxy must transparently proxy all the data, and it is the client's and the server's task to talk https with each other.
I've been searching for a solution on StackOverFlow and couldn't seem to find an answer, I am using WinSock2 to log into a website and the server responds with "Connection: close" (Even if I send Connection: keep-alive) in the header. Any messages attempting to recv after returns 0. (0 = Graceful close)
Questions:
Is the connection suppose to drop after a POST request?
How do you send subsequent GET/POST requests after it has dropped?
So do you have to recycle the socket and re-establish everything again like my example below?
Example of list of events (that i'd imagine i'd need to do):
WSAStartup
Find Address
Create socket
Connect Socket with address
Send Post Request
---Connection Closes after Post---
Destroy Socket
Create socket
Connect Socket with address
Send Get Request with Authentication Cookie
This is what I picture the event chain would look like, but I am not 100% sure how browsers handle all this. But I learn from other users experience and input so if anyone knows exactly what happens let me know. Thank you for your time,
Is the connection suppose to drop after a POST request?
It can, yes. HTTP is a stateless protocol, there is no guarantee that the connection will stay alive after a response, even if a keep-alive is requested. Whether or not to close the connection after sending the response is up to the server to decide, if the client does not request the connection to be closed.
How do you send subsequent GET/POST requests after it has dropped?
You have no choice but to reconnect to the server, and everything that involves (TCP handshakes, SSL/TLS handshakes, etc) before you can send the new request.
If you:
send an HTTP 1.0 request that does not explicitly state Connection: keep-alive, or receive an HTTP 1.0 response that does not explicitly state Connection: keep-alive
send an HTTP 1.1 request that explicitly states Connection: close, or receive an HTTP 1.1 response that explicitly states Connection: close
Then you must close your end of the connection after reading the response.
Even if the response indicates a keep-alive is in effect, the connection could still timeout and be closed before you send your next request on the same connection.
So, any time you want to send a new request, if the connection has already been closed previously, or you get a connection error while sending the request, close the socket, reconnect, and resend the request.
So do you have to recycle the socket and re-establish everything again like my example below?
Potentially, yes.
I am not 100% sure how browsers handle all this.
Exactly as described above.
This is discussed further in RFC 2616 Section 8 "Connections".
So far, my proxy only deals HTTP connections on port 80: I'd like to improve it and make it manage HTTPS requests. Here's how it works:
the proxy is listening on a given port; Chrome can "see" the proxy thanks to SwitchyOmega plugin and connect the traffic on that given port.
The proxy reads the packet's header, gets the request (I parse only GET requests until now), e.g. GET http://www.google.it HTTP/1.1, gets the hostname parsing www.google.it, finds the IP address by resolving the hostname with gethostbyname and assumes the server port is number 80.
Now the proxy send to the server what it received from client by opening a socket: this socket is opened, binded, and then connected to the IP address I resolved before from the hostname.
I read here how to turn a socket into an SSL socket: after socket, bind, listen and accept syscalls, set what you need and pass the socket's file descriptor to SSL_set_fd so I can read and write data through the new SSL file descriptor.
What (most above all) bothers me is the creation of SSL context: if SSLv23_server_method is for servers and SSLv23_client_method is for clients, what should I use for my proxy?
I found no particular proxies configuration in OpenSSL documentation.
Thanks in advance for your help.
Edit: more detailed info about how the proxy works.
SSL/TLS has no concept of proxies, only point-to-point connections. Your proxy has a connection to the client, and a separate connection to the target server, and it is simply passing data between the two connections as needed. So the proxy has to secure (or not) those connections independently of each other, and how it must do that depends on how the client is asking the proxy to relay HTTP.
If the client sends a GET/POST/etc request to your proxy requesting an absolute HTTP URL, your proxy needs to connect to the target server without using SSL/TLS, and then relay the client's HTTP request and server's response back and forth. The client may or may not connect to your proxy using SSL/TLS. If it does, that session is only between the client and your proxy, and the data read from, and sent to, the client is encrypted/decrypted accordingly.
If the client sends a GET/POST/etc request to your proxy requesting an absolute HTTPS URL, your proxy needs to connect to the target server and establish its own SSL/TLS session with the server, and then relay the client's HTTP request and server's response back and forth. The proxy is acting as a client to the server, so use a client-based method (sslv23_client_method(), etc). The real client may or may not connect to your proxy using SSL/TLS. If it does, that session is only between the client and your proxy, and the data read from, and sent to, the client is encrypted/decrypted accordingly, separate from the encryption used on the server connection.
If the client sends a CONNECT request to your proxy, the proxy needs to connect to the requested host/port and then pass raw data back and forth as-is. No SSL/TLS is involved on the proxy's part. If the proxy handled the client's or server's SSL/TLS data, it would be acting as a MITM attacker, something SSL/TLS is designed to prevent. If the connection to the server is successful, the client and server (not the proxy) would secure their respective endpoints with SSL/TLS so they are talking to each other directly (needed for exchanging certificates and keys, and validating identities). The SSL/TLS handshake, and subsequent encrypted HTTP request/response data, would pass through the proxy as-is. The proxy would only be able to see the raw encrypted data, not the HTTP data, since only the client and server have the keys needed to decrypt the data.