A given server normally creates only one listening socket, which
then exists for the lifetime of the server. The kernel creates one
connected socket for each client connection that is accepted
I'm new to socket programming, and reading above on the book, since port number is limited to 1-65535, every connection consume one and there is of course more concurrent connection than this, how can web server overcome this problem
I'm new to socket programming, and reading above on the book, since port number is limited to 1-65535, every connection consume one
No. Every inbound connection uses the same port as the listening port. The book is talking about new sockets, not new ports. You misread.
and there is of course more concurrent connection than this, how can web server overcome this problem.
There is no problem to overcome.
Related
I want to write a simple program in c++ that use tcp socket to communicate with the same program on another computer in lan.
To create the tcp socket I could make the user write the ip and the port to make the connection. But I also need to be able to autodetect in the local area network if there is any computer also running the program.
My idea was:
when the program is autodetecting for available connection in lan, it will send all ips a message via udp to a specific port, meanwhile it will also keep listening to a port waiting to eventual answer.
when the program on the other computer is opened for lan connection, it will keep listening to the a port in case another computer is trying to detect, then it will send also via udp the response messagee notifying the possibility of connection.
All the security system is another problem for which I don't need answer now.
// Client 1:
// Search for all ips in local network
// create udp socket
// send check message
// thread function listening for answers
// if device found than show to menu
// continue searching process
// Client 2 (host) :
// user enable lan connection
// create udp socket
// thread function listening for detection requests
// if request structure is right send back identification message
// continue listening for request
My question - Is there a more efficient or standard way to do something like that?
Testing whether another computer is listening on a given port is what hackers do all day to try to take over the world...
When writing a software like you describe, though, you want to specify the IP and port information. A reason to search and automatically find a device would be if you are implementing a printer, for example. In that case, as suggested by Hero, you could use broadcasting. However, in that case, you use UDP (because TCP does not support that feature).
The software on one side must have a server, which in TCP parlance means a listen() call followed by an accept() until a connection materialized.
The client on the other side can then attempt a connect(). If the connect works, then the software on the other side is up and running.
If you need both to be able to attempt a connection, then both must implement the client and server (which is doable if you use ppoll() [or the old select()] you know which event is happening and can act on it, no need for threads or fork()).
On my end, I wrote the eventdispatcher library to do all those things under the hood. I also want many computers to communicate between each others, so I have a form of RPC service I call communicatord. This service is at the same time a client and a server. It listens on a port and tries to connect to other systems. If the other system has a lower IP address, it is considered a server. Otherwise, it is viewed as a client and I disconnect after sending a GOSSIP message. That way the client (larger IP address) can in turn connect to the server. This communicator service allows all my other services to communicate without having to re-implement the communication layer between computer over and over again.
I want to write a tcp server and client application, which has several different connections to each other where the client uses the same port number.
So far I understand it, the server has a listener port and when the client calls it, then I get a new socket for this new connection on the server side, when I call
accept();
Right? So on Server side I can identify my connection with this new socket and send data through it.
Now my understanding problem with the client side. There I get my socket when I call
socket(AF_INET, SOCK_STREAM, 0)
so I have only one socket. In the
connect()
I can specify remote adress and so on. So when I understand it correctly I can use one socket to make several connects to different adresses/port pairs to create different connections. Right?
But how can I now see in the Client from which logical connection I receive my data or how can I send it when 2 logical connections use the same local port at the client? On serverside I have 2 sockets when I have 2 accept called but what about the client side? For send and receive I have only one socket handle?
Or do I have to call socket() for each logical connection on the client?
I can specify remote adress and so on. So when I understand it correctly I can use one socket to make several connects to different adresses/port pairs to create different connections. Right?
No. A socket is the combination of IP address plus port number.
Or do I have to call socket() for each logical connection on the client?
Yes.
It seems to me your confusion arises because you think for example that a certain port is used for SMTP connections and a certain port is used for HTTP connections.
Well, that port alone is NOT defining for you a socket to the server. The IP address of the server is changing.
As an example, consider the following scenario:
You want to connection to Stackoverflow:
Your PC – IP1+port 50500 ——– Stackoverflow IP2 + port 80 (standard http port)
That is the combination IP1+50500 = the socket on the client computer and IP2 + port 80 = destination socket on the Stackoverflow server.
Now you want to connect to gnu.org:
your PC – IP1+port 50501 ——–gnu.org IP3 +port 80 (standard http port)
The combination IP1+50501 = the socket on the client computer and IP3 + port 80 = destination socket on the gnu.org server.
Better check out Beej's Network Programming to learn more. It is a must-read for anyone working with sockets.
So when I understand it correctly I can use one socket to make several connects to different adresses/port pairs to create different connections. Right?
No. A TCP socket can only be used once. When its connection has finished, or even if connect() just fails to make a connection, you must close the socket and create a new one if you want to make a new connection.
But how can I now see in the Client from which logical connection I receive my data or how can I send it when 2 logical connections use the same local port at the client?
Every TCP connection will have its own unique socket allocated for it. It is your responsibility to keep track of them.
On serverside I have 2 sockets when I have 2 accept called but what about the client side?
The exact same thing happens on the client side, too. You need to create and connect a separate socket for every TCP connection you make. So, you will have a new pair of socket()/connect() calls for every connection.
For send and receive I have only one socket handle?
No, you will have a separate socket for each connection, just like on the server side.
Or do I have to call socket() for each logical connection on the client?
Yes, and connect(), too.
I will not talk about a specific programming language rather I will give a general answer that is applicable for all:
In networking what you care about is the socket (IP+Port) this should be unique whether it is server/client socket or UDP/TCP socket.
For server sockets you must assign a port. For client sockets usually you do not specifically assign a port but it will be assigned by the operating system automatically. However, you can still assign a port to a client socket manually (e.g. in case some port numbers are blocked by the firewall)
In the server process:
you can get the server socket info and the connected client socket info
In the client process:
you can get the client socket info and the server (you want to connect to) socket info (of course you should know the server socket info previously otherwise how will you connect to it).
You can send/receive from/to client sockets. After the server gets the connected client socket it can send/receive through it. Same for the client side it can send/receive through its socket.
The "socket" abstraction is an unfortunate relic of past network stack design. It mixes two different sorts of objects.
A listening socket on the server has a port, and potentially an IP address of the local interface. However, this can also be 0.0.0.0 when listening on all interfaces.
A connected socket is associated with a TCP connection, and therefore has 4 parameters: {local IP, local port, remote IP, remote port}.
Now on the client side, you typically don't care about local IP or local port, so these are locally assigned on connect. And yes, these local parameters can in fact be reused for multiple connections. Only the 4-tuple of {local IP, local port, remote IP, remote port} needs to be unique. The OS will map that unique tuple to your SOCKET.
But since you need a new 4-tuple for every connection, it also follows you need a new SOCKET on both sides, for every connection, on both client and server.
I am trying to write a distributed server system (consisting of server 1="main", and server 2="replacement" for now). Don't mind some dirty methods, it's just to achieve a basic function a distributed server would achive.
Currently I have both servers running via SO_REUSEADDR and a TCP Socket (maybe UDP will solve my problem, but I wanna try it either way with TCP first) on the same machine.
Main server sends establishes a connection with the Replacement server and clients connecting to it.
Now what I want to achieve: The main server should send the socket of the connecting clients to the replacement server, so in case the main server can't work anymore (timeout or what ever) the replacement server can work with the clients and send/recv with them.
The socket I send from main to the replacement server is the one I get with ClientSocket = ::accept(ListenSocket, NULL, NULL); where ClientSocket is a SOCKET (UINT_PTR).
The replacement server can't send/recv to the clients even though the main server gets terminated midway.
Is that because each server, even though they run on the same port, need to be connected via a ::connect from the clients?
EDIT: If my theory is true, this should be solved by using UDP instead of TCP as well, no?
EDIT2: With distributed server I mean a server which in case of a failure will be replaced by another without the clients task getting interrupted.
EDIT3: If there is a better and more simple way to do this, I'd like to know about that as well. I'm not too fond of sockets and server communication as of now that's how I came up with my idea to solve it.
You cannot send a socket between machines. A socket is an OS concept. It represents (in this case) a connection between the server and a client. This connection cannot be resumed on a different machine that has a different IP address because a TCP connection is defined to be established between a pair of addresses and a pair of ports.
The UINT_PTR on one machine means nothing to another machine. It is an opaque value. It is an OS handle.
UDP does not solve the problem since the client needs to notice that the IP address is is communicating with has changed.
Even if you manage that you have the problem that a server failure kills all data on that server. The other server cannot resume with the exact same data. This is a basic problem of distributed computing. You cannot keep the state of all three machines completely in sync.
Make the clients tolerate interruptions by retrying. Make the servers stateless and put all data into a database.
This is a very hard problem to solve. Seek off-the-shelve solutions.
TCP standard has "simultaneous open" feature.
The implication of the feature, client trying to connect to local port, when the port is from ephemeral range, can occasionally connect to itself (see here).
So client think it's connected to server, while it actually connected to itself. From other side, server can not open its server port, since it's occupied/stolen by client.
I'm using RHEL 5.3 and my clients constantly tries to connect to local server.
Eventually client connects to itself.
I want to prevent the situation. I see two possible solutions to the problem:
Don't use ephemeral ports for server ports.
Agree ephemeral port range and configure it on your machines (see ephemeral range)
Check connect() as somebody propose here.
What do you thinks?
How do you handle the issue?
P.S. 1
Except of the solution, which I obviously looking for,
I'd like you to share your real life experience with the problem.
When I found the cause of the problem, I was "astonished" on my work place people are not familiar with it. Polling server by connecting it periodically is IMHO common practice,
so how it's that the problem is not commonly known.
When I stumbled into this I was flabbergasted. I could figure out that the outgoing
port number accidentally matches the incoming port number, but not why the TCP
handshake (SYN SYN-ACK ACK) would succeed (ask yourself: who is sending the ACK if
there is nobody doing a listen() and accept()???)
Both Linux and FreeBSD show this behavior.
Anyway, one solution is to stay out of the high range of port numbers for servers.
I noticed that Darwin side-steps this issue by not allowing the outgoing port
to be the same as the destination port. They must have been bitten by this as well...
An easy way to show this effect is as follows:
while true
do
telnet 127.0.0.1 50000
done
And wait for a minute or so and you will be chatting with yourself...
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
hello?
hello?
Anyway, it makes good job interview material.
Bind the client socket to port 0 (system assigns), check the system assigned port, if it matches the local server port you already know the server is down and and can skip connect().
For server you need to bind() socket to port. Once addr:port pair had socket bound, it will no longer be used for implicit binding in connect().
No problem, no trouble.
Note that this solution is theoretical and I have not tested it on my own. I've not experienced it before (or did not realize) and hopefully I won't experience it anymore.
I'm assuming that you cannot edit neither the client source code nor the server source. Additionally I'm assuming the real problem is the server which cannot start.
Launch the server with a starter application. If the target port that the server will bind is being used by any process, create an RST (reset packet) by using raw sockets.
The post below briefly describes what an RST packet is (taken from http://forum.soft32.com/linux/killing-socket-connection-cmdline-ftopict473059.html)
You have to look at a "raw socket" packet generator.
And you have to be superuser.
You probably need a network sniffer as well.
http://en.wikipedia.org/wiki/Raw_socket
http://kerneltrap.org/node/3072 - TCP RST attacks
http://search.cpan.org/dist/Net-RawIP/lib/Net/RawIP.pm - a Perl module
http://mixter.void.ru/rawip.html - raw IP in C
In the C version, you want a TH_RST packet.
RST is designed to handle the following case.
A and B establish a connection.
B reboots, and forgets about this.
A sends a packet to B to port X from port Y.
B sends a RST packet back, saying "what are you talking about? I don't
have a connection with you. Please close this connection down."
So you have to know/fake the IP address of B, and know both ports X
and Y. One of the ports will be the well known port number. The other
you have to find out. I thnk you also need to know the sequence
number.
Typically people do this with a sniffer. You could use a switch with a
packet mirroring function, or run a sniffer on either host A or B.
As a note, Comcast did this to disable P2P traffic.
http://www.eff.org/wp/packet-forgery-isps-report-comcast-affair
In our case we don't need to use a sniffer since we know the information below:
So you have to know/fake the IP address of B, and know both ports X
and Y
X = Y and B's IP address is localhost
Tutorial on http://mixter.void.ru/rawip.html describes how to use Raw Sockets.
NOTE that any other process on the system might also steal our target port from ephemeral pool. (e.g. Mozilla Firefox) This solution will not work on this type of connections since X != Y B's IP address is not localhost but something like 192.168.1.43 on eth0. In this case you might use netstat to retrieve X, Y and B's IP address and then create a RST packet accordingly.
Hmm, that is an odd problem. If you have a client / server on the same machine and it will always be on the same machine perhaps shared memory or a Unix domain socket or some other form of IPC is a better choice.
Other options would be to run the server on a fixed port and the client on a fixed source port. Say, the server runs on 5000 and the client runs on 5001. You do have the issue of binding to either of these if something else is bound to them.
You could run the server on an even port number and force the client to an odd port number. Pick a random number in the ephemeral range, OR it with 1, and then call bind() with that. If bind() fails with EADDRINUSE then pick a different odd port number and try again.
This option isn't actually implemented in most TCPs. Do you have an actual problem?
That's an interesting issue! If you're mostly concerned that your server is running, you could always implement a heartbeat mechanism in the server itself to report status to another process. Or you could write a script to check and see if your server process is running.
If you're concerned more about the actual connection to the server being available, I'd suggest moving your client to a different machine. This way you can verify that your server at least has some network connectivity.
In my opinion, this is a bug in the TCP spec; listening sockets shouldn't be able to send unsolicited SYNs, and receiving a SYN (rather than a SYN+ACK) after you've sent one should be illegal and result in a reset, which would quickly let the client close the unluckily-chosen local port. But nobody asked for my opinion ;)
As you say, the obvious answer is not to listen in the ephemeral port range. Another solution, if you know you'll be connecting to a local machine, is to design your protocol so that the server sends the first message, and have a short timeout on the client side for receiving that message.
The actual problem you are having seems to be that while the server is down, something else can use the ephemeral port you expect for your server as the source port for an outgoing connection. The detail of how that happens is separate to the actual problem, and it can happen in ways other than the way you describe.
The solution to that problem is to set SO_REUSEADDR on the socket. That will let you create a server on a port that has a current outgoing connection.
If you really care about that port number, you can use operating specific methods to stop it being allocated as an ephemeral port.
I am writing a server in linux that is supposed to serve an API.
Initially, I wanted to make it Multi-threaded on a single port, meaning that I'd have multiple threads working on various request received on a single port.
One of my friends told me that it not the way it is supposed to work. He told me that when a request is received, I first have to follow a Handshake procedure, create a thread that is listening to some other port dedicated to the request and then redirect the requested client to the new port.
Theoretically, it's very interesting but I could not find any information on how to implement the handshake and do the redirection. Can someone help?
If I'm not wrong in interpreting your responses, once I create a multithreaded server with a main thread listening to a port, and creates a new thread to handle requests, I'm essentially making it multithreaded on a single port?
Consider the scenario where I get a large number of requests every second. Isn't it true that every request on the port should now wait for the "current" request to complete? If not, how would the communication still be done: Say a browser sends a request, so the thread handling this has to first listen to the port, block it, process it, respond and then unblock it.
By this, eventhough I'm having "multithreads" , all I'm using is one single thread at a time apart from the main thread because the port is being blocked.
What your friend told you is similar to passive FTP - a client tells the server that it needs a connection, the server sends back the port number and the client creates a data connection to that port.
But all you wanted to do is a multithreaded server. All you need is one server socket listening and accepting connections on a given port. As soon as the automatic TCP handshake is finished, you'll get a new socket from the accept function - that socket will be used for communication with the client that has just connected. So now you only have to create a new thread, passing that client socket to the thread function. In your server thread, you will then call accept again in order to accept another connection.
TCP/IP does the handshake, if you can't think of any reason to do a handshake than your application does not demand it.
An example of an application specific handshake could be for user authentication.
What your colleague is suggesting sounds like the way FTP works. This is not a good thing to do -- the internet these days is more or less used for protocols which use a single port, and having a command port is bad. One of the reasons is because statefull firewalls aren't designed for multi-port applications; they have to be extended for each individual application that does things this way.
Look at ASIO's tutorial on async TCP. There one part accept connections on TCP and spawns handlers that each communicate with a single client. That's how TCP-servers usually work (including HTTP/web, the most common tcp protocol.)
You may disregard the asynchronous stuff of ASIO if you're set on creating a thread per connection. It doesn't apply to your question. (Going fully async and have one worker-thread per core is nice, but it might not integrate well with the rest of your environment.)