Ok, so I created a very simple WAR which serves a simple Hello World .jsp. With all the HTML it's about 200bytes.
Deployed it on my server running Jetty 7.5.x jdk 6u27
On my client computer create simple JMeter test plan with: Thread Group, HTTP Request, Response Assertion, Summary Report Client also running jdk6u27
I set up the thread group to 5 threads running for 60secs and I got 5800 requests/sec
Then I setup 10 threads and got 6800 requests/sec
The moment I disable Keep-Alive in JMeter on the HTTP Request sampler. I seem to get lots of big pauses on the client side I suppose, it doesn't seem the server is receiving anything. I get less pauses at 5 threads or barely none but at 10 threads it hangs pretty much all the time.
What does this mean exactly?
Keep in mind I'm technically creating a REST service and I was getting the same issue, so I though maybe I was doing something funky in my service, till I figured out it's a Keep-Alive issue as it's doing it pretty much on a staic web app. So in reality I will have 1 client request 1 server response. The client will not be keeping the connection open.
My guess is that since Keep-Alive is what allows HTTP Connection (and thereby, socket) reuse, you are running out of available ephemeral port numbers -- there are only 64k port numbers, and since connections must have unique client/server port combos (and server port is fixed), you can quickly go through those. Now, if ports were reusable as soon as connection was closed by one side, it would not matter: however, as per TCP spec, both sides MUST wait for configurable amount of time (default: 2 minutes) until reuse is considered safe.
For more details you can read a TCP book (like "Stevens book"); above is a simplification.
Related
I am doing a TCP client - server simulation. In the simulation, I have created 2 clients and 2 servers. And I have programmed that read requests will go to server 1 and write requests will go to server 2. Thus, the client will always renew it's socket and make a new connection to the servers.
However, after the client has made 66561 times of connections to the server, instead of sending request packets, it will just simply send some empty ACK packets.
I expected both the clients to be able to send up to millions of requests, but currently, both the clients are only able to send up to 13k requests. Can anyone give me tips or advices?
Nagle's algorithm
Solutions:
Do not use small package in your app protocol
Use socket option TCP_NODELAY on both side client/server
Sounds like most previously created connections are still taking the resource (not released from system). From the information you give,
However, after the client has made 66561 times of connections to the server, instead of sending request packets, it will just simply send some empty ACK packets.
Looks like about 1000+ connections are released. Probably because of the 2msl time is due. If this is the case, suggest you explicitly release a connect before you create a new one.
Copy and paste your C/S part code would help the analyse.
Let's say I have a chat app with registration and it does long-polling to an Apache server. I've done some reading but I'm still confused and want to be extremely sure. From my understanding, it can either be :
Any amount of client can do long-polling to that server and it won't affect the limit because all the clients only have 1 concurrent connection each to the server. So if I open the chat app in 7 IE8/chrome/firefox in d same computer OR in different computer EACH and connect to the same url/domain, it won't be affected but if I open the chat in 7 tabs in IE8/chrome/firefox only then it will be affected.
Same as the above but the limit will only be affected if I open 7 IE8/chrome/firefox browsers in 7 computers by 7 different accounts. Which means only 6 different users can connect to the chat app at the same time.
I'm leaning heavily to the first one. Can you help me correct/expand on either both or if both are wrong, kindly add number 3? Thank you!
This limitation is a restriction put in place by each browser vendor. The typical connection limit for a browser instance is set to 6 socket connections to the same domain. These six connections make up the browsers socket pool. This socket pool is managed by the socket pool manager and are used across all browser processes. This is to maximize the efficiency of the TCP connection by reusing established connections, as well as other performance benefits.
According to the HTTP 1.1 specification the maximum number of connections should be limited to 2.
Clients that use persistent connections SHOULD limit the number of
simultaneous connections that they maintain to a given server. A
single-user client SHOULD NOT maintain more than 2 connections with
any server or proxy. These guidelines are intended to improve HTTP
response times and avoid congestion.
However, this spec was approved in June 1999 during the infancy of the internet, and browser vendors like Chrome have since increased this number to six.
Currently these are set to 32 sockets per proxy, 6 sockets per
destination host, and 256 sockets per process (not implemented exactly
correct, but good enough).
With that said, each socket pool is managed by each browser. Depending on the browsers connection limit (a minimum of two). You should be able to open 8 connections by opening two tabs in IE, Chrome, Firefox, and Safari. Your max connection is limited by the browser itself. Also keep in mind the server can only handle so many concurrent connections at once. Don't accidentally DoS yourself :)
If you absolutely need to go beyond the connection limitation you could look into domain sharding. Which basically tricks the browser into opening new more connections by providing a different the host name with the request. I wouldn't advise using it though, as the browser has set these limitations to maximize performance and reuse existing connections. Tread lightly.
First, thanks all the Netty contributors for the great library. I have been happily using it for several weeks.
Recently, I started to load test my system but now I'm experiencing some scalability problem with Netty. I tried to fork as many simultaneous Netty clients as possible to connect to a Netty server. For small number of clients (<50), the system just works fine. However, for large number of clients (>100), I find the client side always prompts the "ClosedChannelException":
java.nio.channels.ClosedChannelException
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$1.operationComplete(NioClientSocketPipelineSink.java:157)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:367)
at org.jboss.netty.channel.DefaultChannelFuture.setSuccess(DefaultChannelFuture.java:316)
at org.jboss.netty.channel.AbstractChannel$ChannelCloseFuture.setClosed(AbstractChannel.java:351)
at org.jboss.netty.channel.AbstractChannel.setClosed(AbstractChannel.java:188)
at org.jboss.netty.channel.socket.nio.NioSocketChannel.setClosed(NioSocketChannel.java:146)
at org.jboss.netty.channel.socket.nio.NioWorker.close(NioWorker.java:592)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.close(NioClientSocketPipelineSink.java:415)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processConnectTimeout(NioClientSocketPipelineSink.java:379)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:299)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
I am wondering how to make Netty support more simultaneous client connections, such as 10K. I am using the newest version of Netty. Following is the testing scenario:
Each client sends a four letter string to the server and the server handler does nothing upon receiving the string. Each of the server and the clients is running on a high performance machine with eight-core and 16GB memory. The two machines are connected by a Gigabyte network.
Do you have any hints?
1) You can tweak the connectTimeout in the client bootstrap to make sure there is no network/server issues
clientBootStrap.setOption("connectTimeoutMillis", optimumTimout);
2) By setting the backlog value in the Netty server, you can increase the queue of incoming connection size, so clients will have better chance of connecting to the server
serverBootStrap.setOption("backlog", 1000);
3) You have said that your application is creating many connections simultaneously, Client Boss thread may lag behind, if the application is connecting too fast.
Netty 3.2.7 Final allows to set more than one Client Boss thread in NioClientSocketChannelFactory constructor to avoid this issue.
I have a blocking client/server connected locally via Winsock. The client uses firefox to retrieve data from websites, passing certain data along to the server for extra processing. The server always responds, and the processing can take anywhere from 1/10th second to a few minutes. The client has no winsock connection to anything but the server; all web data is retrieved to hard-drive via firefox.
This setup works quite well until, seemingly randomly, the client's recv returns -1 (SOCKET_ERROR) with error code 10054 (WSAECONNRESET). This means the server supposedly terminated connection, but the server is actually still waiting to recv as if nothing is wrong. The connection has failed in this way as early as 5 minutes in or after working for as long as about an hour and a half. The client sends about 10 different types of requests to the server, and failure has occurred on a variety of them. The frequency of requests is roughly constant, probably an average of 10-15 a minute. When the connection breaks, neither computer experiences internet problems and remote desktop does not disconnect.
Initially I thought memory leaks, but after extensive debugging I am reasonably certain no more exist. Firefox is engaged in considerable HTTP traffic at times, so I thought maybe that could be filling available socket bufferspace or something -- seems doubtful but at this point I'm really not sure. So, could it be more memory leaks, maybe a hidden buffer overrun, too much web traffic? What is causing my Winsock app to randomly fail?
Sounds like a firewall at work.
Many firewalls are configured to terminate idle connections (i.e. open TCP sessions on which no data is transferred for awhile). Especially if it's an HTTP connection, which are typically not persistent.
I am trying to create a simple board game (a kind of checkers), where users will be able to play online with each other using flex application as a client.
I am using django application to process the game on the server side. And I come across the problem, if one user made a move, I can send it to a server, but how do I let the opponent know about it?
The way I am thinking to do it is to create a timer and send requests to the server asking was opponents move done or not....But here we have 2 limitations:
1) Each client would produce big amount of requests (not sure how server will work if I have e.g 100 such clients)
2) If players will chose game with a time limit for example 5 minutes/per game it will be very important to show them situation on the board as soon as it changes (without a pause), but timer will send request only on timer event, so if for example I will chose tick interval to 5 seconds it will mean that 5 seconds another side will not be aware of the situation changes.
Think of it this way. If you poll every 1 or 2 seconds, that should be quick enough not to be noticed by either client. A simple REST request checking for changes is bloody quick and a modern web servers should be able to handle 100 such requests without issue.
Implement it with the timer now, run some performance tests and worry about servers after you're done.
If you are worried later, you can always have graduated timers. e.g., check after 100ms, 200ms, 400ms, 800ms, 1600ms, etc... with a cap at 5 seconds or something.
Take a look at this code for some ideas maybe, since chat uses similar concepts: http://anantgarg.com/2009/05/13/gmail-facebook-style-jquery-chat/
One way is to use a TCP Socket from the client to connect back to your server. Have the client listen for data, and have the server send updates whenever needed. This may require firewall changes (to allow the port you'll be using) and a server which accepts multiple persistent client connections. This may only work for a fixed smallish number of clients, since if you are keeping multiple connections open it will incur some server overhead.
If you have firewall restrictions and need to use HTTP ports, you can investigate Comet implementations. What I proposed in the first paragraph is more or less the same thing - Comet just does it over HTTP and standardises some aspects of the communication.