I am developing a small and lightweight HTTP Server running on the Nintendo Switch, therefore on a UNIX-like system (freeBSD kernel). I built a HTTP server without any library using the raw socket(), bind(), listen(), etc. methods.
I noticed that when I open up the address of the server using Google Chrome, that it first sends a standard HTTP GET request with all headers, and after that a second, 0-byte long request (probably not even a request, it just opens up a second connection to the server, then sends nothing).
Due to my implementation, that's not easy to handle since I receive all data from the connection using recv(), which blocks until all data is read. Hence, on the second request Chrome sends, it will block "forever" (there is a timeout), because Chrome sends no data.
Is that a known bug to Chrome, and is there any way I could detect and/or prevent this from happening? I tested it with Firefox and Edge and these browsers didn't open a second socket.
Thanks!
The solution was easier than I thought. You can just ignore the second socket, and pass MSG_DONTWAIT as a flag to the recv() call. More on that here.
Additionaly, my problem was that I was returning a HTTP 501 response to all requests which had no URL. For the second, 0-length socket, that did obviously break it and made Chrome show an error. I added handling to it wouldn't return an error when there is a 0-length request. It will just silently close it.
In our client/server application, we use TLS/TCP protocol for messaging. There is a message shift occurs between applications after a while (messages are sent and received in the correct order at the beginning) i.e. the client sends the 1000th message to the server and receives the response of message 999th. The suspect is on the client side, in which we implement TCP and TLS layers independently i.e. do not bind TCP socket to SSL object (via SSL_set_fd()) but using BIOs. When the client app gets the response from server (pretty sure that message is processed in the server correctly, client TCP layer receives the message correctly etc.), the message is forwarded to SSL layer. The client app firstly write the message to BIO:
BIO_write (readBio, data, length);
Then in another function of SSL layer, the message is read using SSL_read():
res = SSL_read (ssl, buffer, length);
The read operation is done successfully, but my goal is to check whether there is another record(s) to be read in the BIO. I considered to use the method SSL_pending() but it seems that this one should be used in order to check if there are still bytes in the SAME record. If our suspects are correct, I would like to check if there is another record in the BIO so that all messages are processed without any delay. Can you help me on this topic? Thanks in advance.
SSL_pending tells you if there are data from the current decryted SSL record which got not yet read by SSL_read. BIO_pending can be used to find out if there are already data in the BIO which are not processed by the SSL layer. To find out if the are unread data at the socket level use MSG_PEEK.
I have a client/server application that communicates via tcp/ip sockets over an often unreliable wireless network.
To make it responsive in case of a connection error,I created this protocol.
1) client sends a request
2) server confirms reception of request (1 second timeout)
3) server processes while client wait(may take up to 10 seconds) (20 seconds timeout)
4) server sends response
Sometimes the request command get lost (the client sends it over an open connection but the server never receives it), but with this protocol I know immediately if the command has been received and is going to be processed.
What I'm asking (I've made some test with RESTSharp and ServiceStack) is: is possible to do something like this with a webservice? where a client before the response receives a confirmation that the request has been received?
Thanks,
Mattia
It's strange to see this type of error handling in Application code since if a HTTP request wasn't successful it will already throw an Exception.
To get a confirmation receipt in Application code you would need to add a 2nd request implementing some kind of Push Event / Comet technique to get the server to notify the client of different events, e.g. what Request Ids have been received.
I am currently trying to implement my own webserver in C++ - not for productive use, but for learning.
I basically open a socket, listen, wait for a connection and open a new socket from which I read the data sent by the client. So far so good. But how do I know the client has finished sending data and not simply temporarily stopped sending more because of some other reason?
My current example: When the client sends a POST-request, it first sends the headers, then two times "\r\n" in a row and then the request body. Sometimes the body does not contain any data. So if the client is temporarily unable to send anything after it sent the headers - how do I know it is not yet finished with its request?
Does this solely depend on the used protocol (HTTP) and it is my task to find this out on the basis of the data I received, or is there something like an EOF for sockets?
If I cannot get the necessary Information from the socket, how do I protect my program from faulty clients? (Which I guess I must do regardless of this, since it might be an attacker and not a faulty client sending wrong data.) Is my only option to keep reading until the request is complete by definition of the protocol or a timeout (defined by me) is reached?
I hope this makes sense.
Btw: Please don't tell me to use some library - I want to learn the basics.
The protocol (HTTP) tells you when the client has stopped sending data. You can't get the info from the socket as the client will leave it open waiting for a response.
As you say, you must guard against errant clients not sending proper requests. Typically in the case of an incomplete request a timeout is applied to the read. If you haven't received anything in 30 seconds, say, then close the socket and ignore it.
For an HTTP post, there should be a header (Content-Length) saying how many bytes to expect after the the end of the headers. If its a POST and there is no Content-Length, then reject it.
"Does this solely depend on the used protocol (HTTP) and it is my task to find this out on the basis of the data I received,"
Correct. You can find the HTTP spec via google;
http://www.w3.org/Protocols/rfc2616/rfc2616.html
"or is there something like an EOF for sockets?"
There is as it behaves just like a file ... but that's not applicable here because the client isn't closing the connection; you're sending the reply ON that connection.
With text based protocols like HTTP you are at the mercy of the client. Most well formatted POST will have a content-length so you know how much data is coming. However the client can just delay sending the data, or it may have had its Ethernet cable removed or just hang, in which case that socket is sitting there indefinitely. If it disconnects nicely then you will get a socket closed event/response from the recv().
Most well designed servers in that case will have a receive timeout, and if the socket is idle for more than say 30 seconds it will close that socket, so resources are not leaked by misbehaving clients.
I have a requirement to send some 100 bytes data over internet .My machine is connected to internet.
I can do this with HTTP by sending requests and receiving responses.
But my requirement is just to send data not receive response.
I am thinking of doing this using UDP Client server program. But to do that I need to host UDP client on internet?
Is there any other way to do that?
any suggestions?
Cheap answer to send 100 bytes of data on the internet.
C:\Windows\system32>ping -n 1 -l 100 -4 google.com
Pinging google.com [209.85.171.99] with 100 bytes of data:
Reply from 209.85.171.99: bytes=56 (sent 100) time=174ms TTL=233
Ping statistics for 209.85.171.99:
Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 174ms, Maximum = 174ms, Average = 174ms
Anything that happens on the internet requires a client and a server.
One box is in the role of client, the other is in the role of server for your specific transaction.
Usually (but not always) your local box is a client and some other box is the server.
Software MUST be running on both to implement some protocol for exchanging data.
A server can listen on TCP or UDP sockets, with some restrictions. Some port numbers are privileged. Some port numbers are blocked by firewalls.
Port 80, while rarely blocked by firewalls is a privileged port. Generally, you need a web server (e.g., Apache) or privileges to listen on port 80.
"Sending 100 bytes" can be done using a lot of available protocols: Echo, Telnet, FTP, HTTP to name a few.
The big advantage of HTTP is that port 80 is very often open. With other protocols you have to rely on the operators to open the port.
In order to send data but not receive a response, you can simply write your program in such a way that it does not listen for a response. This doesn't mean one won't be sent to you, just that you won't get it.
For example, you can make sure you don't call "recv" on the socket. Also, you can use "shutdown" to disable reads on the socket. Depending on the underlying implementation, going the "shutdown" route might cause all incoming packets to simply be dropped.
As far as how to send the packets, really any sort of protocol will work. Of course, you need to know of a destination server on the Internet, but you've got plenty of options. Perhaps the simplest route to take is what you have suggested: HTTP (perhaps use www.google.com as your destination server).
You need a client (you) and a server (other end). For UDP, you send datagrams over the Internet (using IP). UDP doesn't provide the safety that TCP does, but doesn't require a response (but such responses are part of their protocols, not yours).
I would suggest using TCP to save you some headache.
Also, make sure you're not behind a firewall, else your packets won't make it to their destination as you'd expect.
Hmmm...
You want to send short messages over the internet, but without any response.
Your application wouldn't by any chance be some kind of spyware, would it?
Use UDP. Open a socket, send the data, close the socket. That's it. Here is a Python version of the client:
import socket
data = 100*'x'
address = ('192.168.0.123', 8080) # Host, port
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
sock.connect(address)
sock.send(data)
sock.close()
On the Wikipedia page about UDP there is some corresponding WinSock code.
Of course the other side must be reachable, and there must be someone listening there, otherwise the target machine will reply with an ICMP "port unreachable" packet (at least if it complies with standards).
If you want a UDP listener on the internet, it will have to be hosted somewhere.
You can get HTTP hosting much easier, it's everywhere, UDP you may need your own machine or at least a VM.
curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.
See examples here