Sorry for improper description of my question.
What my program do is that connect a server, send some data and close connection. I simplified my code as below:
WSAStartup(MAKEWORD(2, 2), &wsaData);
SOCKET s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
connect(s, (const sockaddr*)&dstAddr, sizeof(dstAddr));
send(s, (const char*)pBuffer, fileLen, 0);
shutdown(s, SD_SEND);
closesocket(s);
WSACleanup();
Only partial data was received by server before found a RST causing communication shutdown.
I wrote a simulate server program to accept connection and receive data, but the simulator could get all data. Because I couldn't access server's source code, I didn't know if something made wrong in it. Is there a way I can avoid this error by adding some code in client, or can I prove that there is something wrong in server program?
Setting socket's linger option can fix the bug. But I need to give a magic number for the value of linger time.
linger l;
l.l_onoff = 1;
l.l_linger = 30;
setsockopt(socket, SOL_SOCKET, SO_LINGER, (const char*)&l, sizeof(l));
WSASend returns before sending data to device actually
Correct.
I created a non-blocking socket and tried to send data to server.
WSASocket(AF_INET, SOCK_STREAM, IPPROTO_TCP, NULL, 0, WSA_FLAG_OVERLAPPED)
No you didn't. You created an overlapped I/O socket.
After executed, returnValue was SOCKET_ERROR and WSAGetLastError() returned WSA_IO_PENDING. Then I called WSAWaitForMultipleEvents to wait for event being set. After it returned WSA_WAIT_EVENT_0, I called WSAGetOverlappedResult to get actual sent data length and it is the same value with I sent.
So all the data got transferred into the socket send buffer.
I called WSASocket first, then WSASend/WSAWaitForMultipleEvents/WSAGetOverlappedResult several times to send a bunch of data, and closesocket at the end.
So at the end of that process all the data and the close had been transferred to the socket send buffer.
But server couldn't receive all data, I used Wireshark to view tcp packets and found that client sent RST before all packet were sent out.
That could be for a number of reasons none of which is determinable without seeing some code.
If I slept 1 minute before calling closesocket, then server would receive all data.
Again this would depend on what else had happened in your code.
It seemed like that WSASend/WSAWaitForMultipleEvents/WSAGetOverlappedResult returned before sending data to server actually.
Correct.
The data were saved in buffer and waiting for being sent out.
Correct.
When I called closesocket, communication was shut down.
Correct.
They didn't work as my expectation.
So your expectation was wrong.
What did I go wrong? This problem only occurred in specific PCs, the application run well in others.
Impossible to answer without seeing some code. The usual reason for issuing an RST is that the peer had written data to a connection that you had already closed: in other words, an application protocol error; but there are other possibilities.
Related
I have a UDP client C++ code - based on WSA sockets - that works well. The code was originally written in VS6 and I recently recompiled it in VS2010 for 64bit environment, with only little adjustments.
Now, the sendto() fails to send something, if there is no Sleep(..) or any equivalent delay after the sendto() and before closesocket(). "Fails" means, that sendto() returns the proper amount of data, but I see no message on the network (I used wireshark to check this).
This is my code:
void CTest::SendHello()
{
SOCKET sSocket;
sSocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
struct sockaddr_in addr;
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = inet_addr(m_strDstIpAddr);
addr.sin_port = htons(m_nTxPort);
int nMsgLen = 8;
char pTxBuffer[8];
*((DWORD*) &pTxBuffer[ 0]) = 0x11223344;
*((DWORD*) &pTxBuffer[ 4]) = 0;
int nSent = sendto(sSocket, pTxBuffer, nMsgLen, 0, (struct sockaddr *) &addr, sizeof(addr));
Sleep(10); // <- this seems to be necessary
if (nSent != nMsgLen)
{
CString s = "error sending HELO\n";
AfxMessageBox(s);
}
closesocket(sSocket);
}
Without the Sleep(), the code does not send anything, yet it returns no errors. With the Sleep() it works. Also, this is happens in release version, when compiled for debug, the code also works without the Sleep().
It seems, as if the closesocket() shuts the socket down, before the message is finally sent, but I thought sendto() is a synchronous function. I tried using SO_LINGER, but this is not applicable for SOCK_DGRAM sockets.
Since the code is inside a DLL, I can't create the socket in ctor and delete it in the dtor, because SendHello() might be called from different thread contexts, and I like to avoid to make the code too complicated.
thanks for any help
With UDP there is no ordering of data between the sender and receiver and data sent using UDP datagram sockets isn't guaranteed to arrive. All sleep is doing in your case is practically providing enough time for the data to arrive at the other end. If you want confirmation of receipt and error checking, then you can either code a scheme for UDP or use TCP. In fact, you can turn off the server completely and your client will happily fire UDP packets out without error even though there is nobody listening.
to insure a connection, look at connect(). Nothing prevents using connect with UDP and you can then use send() recv().
I am very new to networking and have an issue with sending messages during a while loop.
To my knowledge I should do something along the lines of this:
Create Socket()
Connect()
While
Do logic
Send()
End while
Close Socket()
However it sends once and returns -1 there after.
The code will only work when I create the socket in the loop.
While
Create Socket()
Connect()
Do logic
Send()
Close Socket()
End while
Here is a section of the code I am using but doesn't work:
//init winsock
WSAStartup(MAKEWORD(2, 0), &wsaData);
//open socket
sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
//connect
memset(&serveraddr, 0, sizeof(serveraddr));
serveraddr.sin_family = AF_INET;
serveraddr.sin_addr.s_addr = inet_addr(ipaddress);
serveraddr.sin_port = htons((unsigned short) port);
connect(sock, (struct sockaddr *) &serveraddr, sizeof(serveraddr));
while(true) {
if (send(sock, request.c_str(), request.length(), 0)< 0 /*!= request.length()*/) {
OutputDebugString(TEXT("Failed to send."));
} else {
OutputDebugString(TEXT("Activity sent."));
}
Sleep(30000);
}
//disconnect
closesocket(sock);
//cleanup
WSACleanup();
The function CheckForLastError() returns:10053
WSAECONNABORTED
Software caused connection abort.
An established connection was aborted by the software in your host computer, possibly due to a data transmission time-out or protocol error
Thanks
I have been looking for a solution to this problem too. I am having the same problem with my server. When trying to send a response from inside the loop, the client seems never to receive it.
As I understand the problem, according to user207421's suggestions, when you establish a connection between a client and a server, the protocol should have enough information to let the client know when the server has finished sending the response. If you see this example, you have a minimum HTTP server that responds to requests. In this case, you can use a browser or an application like Postman. And if you see the response message, you will see a header called Connection. Setting its value to close tells the client which one is the last message from the server for that request. The message is being sent, but the client keeps waiting, maybe because there is no closing element the client can recognize. I was also missing the Content-Length header. My HTTP response message was wrong, and the client was lost.
This diagram shows what needs to be outside the loop and what needs to be inside.
To understand how and why your program fails,you have to understand the functions you use.
Some of them are blocking functions and some are them not. Some of them need previous calles of other functions and some of them don't.
Now from what i understand we are talking about a client here,not a server.
The client has only non blocking functions in this case. That means that whenever you call a function,it will be executed without waiting.
So send() will send data the second it is called and the stream will go on to the next line of code.
If the information to be sent was not yet ready...you will have a problem,since nothing will be sent.
To solve it you could use some sort of a delay. The problem with delays is that they are Blocking functions meaning your stream will stop once it hits the delay. To solve it you can create a thread and lock it untill the information is ready to be sent.
But that would do the job for one send(). You will send the info and thats that.
If you want to hold the communication and send repeatedly info,you will need to create a while loop. once you have a while loop you dont have to worry about anything. That is because you can verify that the information is ready with a stream control and you can use send over and over again before terminating the connection.
Now the question is what is happening on the server side of things?
"ipaddress" should hold the ip of the server. The server might reject your request to connect.Or worst he might accept your request but he is listening with diffrent settings in relation to your client.Meaning that maybe the server is not reciving (does not have recv() function)information and you are trying to send info... that might resault in errors/crashes and what not.
I'm having trouble with receiving data over a network using Winsock2, with Windows. I'm trying to use a simple client and server system to implement a file transfer program. With our current code, the last packet coming in doesn't get appended to the file because it's not the size of the buffer. So, the file transfer doesn't quite completely, throws an error, and breaks. It's not always the very last packet, sometimes it's earlier.
Here is a snippet of the Server code:
int iResult;
ifstream sendFile(path, ifstream::binary);
char* buf;
if (sendFile.is_open()) {
printf("File Opened!\n");
// Sends the file
while (sendFile.good()) {
buf = new char[1024];
sendFile.read(buf, 1024);
iResult = send(AcceptSocket, buf, (int)strlen(buf)-4, 0 );
if (iResult == SOCKET_ERROR) {
wprintf(L"send failed with error: %d\n", WSAGetLastError());
closesocket(AcceptSocket);
WSACleanup();
return 1;
}
//printf("Bytes Sent: %d\n", iResult);
}
sendFile.close();
}
And here is a snippet of the Client code:
int iResult;
int recvbuflen = DEFAULT_BUFLEN;
char recvbuf[DEFAULT_BUFLEN] = "";
do {
iResult = recv(ConnectSocket, recvbuf, recvbuflen, 0);
if ( iResult > 0){
printf("%s",recvbuf);
myfile.write(recvbuf, iResult);
}
else if ( iResult == 0 ) {
wprintf(L"Connection closed\n");
} else {
wprintf(L"recv failed with error: %d\n", WSAGetLastError());
}
} while( iResult > 0 );
myfile.close();
When trying to transfer a file that is a dictionary, it can break at random times. For example, one run broke early in the S's and appended weird characters to the end, which isn't rare:
...
sayable
sayer
sayers
sayest
sayid
sayids
saying
sayings
╠╠╠╠╠╠╠╠recv failed with error: 10054
What can I do to handle these errors and weird characters?
The error is happening on the server side. You're getting a "Connection reset by peer" error.
This line - buf = new char[1024]; - is clearly problematic and is likely causing the server to crash because it runs out of memory. There is no clean up happening. Start by adding the appropriate delete statement, probably best placed after the send call. If that doesn't fix it I would use a small test file and step through that while loop in the server code.
P.S. A better solution than using new and delete in your loop is to reuse the existing buff. The compiler might optimize this mistake out but if it doesn't you're severely hindering the applications performance. I think you actually should just move buf = new char[1024]; outside of the loop. buf is a char pointer so read will continue to overwrite the contents of buf if you pass it buf. Re allocating the buffer over and over is not good.
With regard to the error MSDN says:
An existing connection was forcibly closed by the remote host. This normally results if the peer application on the remote host is suddenly stopped, the host is rebooted, the host or remote network interface is disabled, or the remote host uses a hard close (see setsockopt for more information on the SO_LINGER option on the remote socket). This error may also result if a connection was broken due to keep-alive activity detecting a failure while one or more operations are in progress. Operations that were in progress fail with WSAENETRESET. Subsequent operations fail with WSAECONNRESET.
First, using the new operator in a loop might not be good, especially without a corresponding delete. I'm not a C++ expert, though (only C) but I think it is worth checking.
Second, socket error 10054 is "connection reset by peer" which tells me that the server is not performing what is called a graceful close on the socket. With a graceful close, WinSock will wait until all pending data has been received by the other side before sending the FIN message that breaks the connection. It is likely that your server is is just closing immediately after the final buffer is given to WinSock without any time for it to get transmitted. You'll want to look into the SO_LINGER socket options -- they explain the graceful vs non-graceful closes.
Simply put, you either need to add your own protocol to the connection so that the client can acknowledge receipt of the final data block, or the server side needs to call setsocketopt() to set a SO_LINGER timeout so that WinSock will wait for the TCP/IP acknowledgement from the client side for the final block of data before issuing the socket close across the network. If you don't do at least ONE of those things, then this problem will occur.
There's also another article about that here that you might want to look at:
socket error 10054
Good luck!
The server (192.168.1.5:3001), is running Linux 3.2, and is designed to only accept one connection at a time.
The client (192.168.1.18), is running Windows 7. The connection is a wireless connection. Both programs are written in C++.
It works great 9 in 10 connect/disconnect cycles. The tenth-ish (randomly happens) connection has the server accept the connection, then when it later actually writes to it (typically 30+s later), according to Wireshark (see screenshot) it looks like it's writing to an old stale connection, with a port number that the client has FINed (a while ago), but the server hasn't yet FINed. So the client and server connections seems to get out of sync - the client makes new connections, and the server tries writing to the previous one. Every subsequent connection attempt fails once it gets in this broken state. The broken state can be initiated by going beyond the maximum wireless range for a half a minute (as before 9 in 10 cases this works, but it sometimes causes the broken state).
Wireshark screenshot behind link
The red arrows in the screenshot indicate when the server started sending data (Len != 0), which is the point when the client rejects it and sends a RST to the server. The coloured dots down the right edge indicate a single colour for each of the client port numbers used. Note how one or two dots appear well after the rest of the dots of that colour were (and note the time column).
The problem looks like it's on the server's end, since if you kill the server process and restart, it resolves itself (until next time it occurs).
The code is hopefully not too out-of-the-ordinary. I set the queue size parameter in listen() to 0, which I think means it only allows one current connection and no pending connections (I tried 1 instead, but the problem was still there). None of the errors appear as trace prints where "// error" is shown in the code.
// Server code
mySocket = ::socket(AF_INET, SOCK_STREAM, 0);
if (mySocket == -1)
{
// error
}
// Set non-blocking
const int saveFlags = ::fcntl(mySocket, F_GETFL, 0);
::fcntl(mySocket, F_SETFL, saveFlags | O_NONBLOCK);
// Bind to port
// Union to work around pointer aliasing issues.
union SocketAddress
{
sockaddr myBase;
sockaddr_in myIn4;
};
SocketAddress address;
::memset(reinterpret_cast<Tbyte*>(&address), 0, sizeof(address));
address.myIn4.sin_family = AF_INET;
address.myIn4.sin_port = htons(Port);
address.myIn4.sin_addr.s_addr = INADDR_ANY;
if (::bind(mySocket, &address.myBase, sizeof(address)) != 0)
{
// error
}
if (::listen(mySocket, 0) != 0)
{
// error
}
// main loop
{
...
// Wait for a connection.
fd_set readSet;
FD_ZERO(&readSet);
FD_SET(mySocket, &readSet);
const int aResult = ::select(getdtablesize(), &readSet, NULL, NULL, NULL);
if (aResult != 1)
{
continue;
}
// A connection is definitely waiting.
const int fileDescriptor = ::accept(mySocket, NULL, NULL);
if (fileDescriptor == -1)
{
// error
}
// Set non-blocking
const int saveFlags = ::fcntl(fileDescriptor, F_GETFL, 0);
::fcntl(fileDescriptor, F_SETFL, saveFlags | O_NONBLOCK);
...
// Do other things for 30+ seconds.
...
const int bytesWritten = ::write(fileDescriptor, buffer, bufferSize);
if (bytesWritten < 0)
{
// THIS FAILS!! (but succeeds the first ~9 times)
}
// Finished with the connection.
::shutdown(fileDescriptor, SHUT_RDWR);
while (::close(fileDescriptor) == -1)
{
switch(errno)
{
case EINTR:
// Break from the switch statement. Continue in the loop.
break;
case EIO:
case EBADF:
default:
// error
return;
}
}
}
So somewhere between the accept() call (assuming that is exactly the point when the SYN packet is sent), and the write() call, the client's port gets changed to the previously-used client port.
So the question is: how can it be that the server accepts a connection (and thus opens a file descriptor), and then sends data through a previous (now stale and dead) connection/file descriptor? Does it need some sort of option in a system call that's missing?
I'm submitting an answer to summarize what we've figured out in the comments, even though it's not a finished answer yet. It does cover the important points, I think.
You have a server that handles clients one at a time. It accepts a connection, prepares some data for the client, writes the data, and closes the connection. The trouble is that the preparing-the-data step sometimes takes longer than the client is willing to wait. While the server is busy preparing the data, the client gives up.
On the client side, when the socket is closed, a FIN is sent notifying the server that the client has no more data to send. The client's socket now goes into FIN_WAIT1 state.
The server receives the FIN and replies with an ACK. (ACKs are done by the kernel without any help from the userspace process.) The server socket goes into the CLOSE_WAIT state. The socket is now readable, but the server process doesn't notice because it's busy with its data-preparation phase.
The client receives the ACK of the FIN and goes into FIN_WAIT2 state. I don't know what's happening in userspace on the client since you haven't shown the client code, but I don't think it matters.
The server process is still preparing data for a client that has hung up. It's oblivious to everything else. Meanwhile, another client connects. The kernel completes the handshake. This new client will not be getting any attention from the server process for a while, but at the kernel level the second connection is now ESTABLISHED on both ends.
Eventually, the server's data preparation (for the first client) is complete. It attempts to write(). The server's kernel doesn't know that the first client is no longer willing to receive data because TCP doesn't communicate that information! So the write succeeds and the data is sent out (packet 10711 in your wireshark listing).
The client gets this packet and its kernel replies with RST because it knows what the server didn't know: the client socket has already been shut down for both reading and writing, probably closed, and maybe forgotten already.
In the wireshark trace it appears that the server only wanted to send 15 bytes of data to the client, so it probably completed the write() successfully. But the RST arrived quickly, before the server got a chance to do its shutdown() and close() which would have sent a FIN. Once the RST is received, the server won't send any more packets on that socket. The shutdown() and close() are now executed, but don't have any on-the-wire effect.
Now the server is finally ready to accept() the next client. It begins another slow preparation step, and it's falling further behind schedule because the second client has been waiting a while already. The problem will keep getting worse until the rate of client connections slows down to something the server can handle.
The fix will have to be for you to make the server process notice when a client hangs up during the preparation step, and immediately close the socket and move on to the next client. How you will do it depends on what the data preparation code actually looks like. If it's just a big CPU-bound loop, you have to find some place to insert a periodic check of the socket. Or create a child process to do the data preparation and writing, while the parent process just watches the socket - and if the client hangs up before the child exits, kill the child process. Other solutions are possible (like F_SETOWN to have a signal sent to the process when something happens on the socket).
Aha, success! It turns out the server was receiving the client's SYN, and the server's kernel was automatically completing the connection with another SYN, before the accept() had been called. So there definitely a listening queue, and having two connections waiting on the queue was half of the cause.
The other half of the cause was to do with information which was omitted from the question (I thought it was irrelevant because of the false assumption above). There was a primary connection port (call it A), and the secondary, troublesome connection port which this question is all about (call it B). The proper connection order is A establishes a connection (A1), then B attempts to establish a connection (which would become B1)... within a time frame of 200ms (I already doubled the timeout from 100ms which was written ages ago, so I thought I was being generous!). If it doesn't get a B connection within 200ms, then it drops A1. So then B1 establishes a connection with the server's kernel, waiting to be accepted. It only gets accepted on the next connection cycle when A2 establishes a connection, and the client also sends a B2 connection. The server accepts the A2 connection, then gets the first connection on the B queue, which is B1 (hasn't been accepted yet - the queue looked like B1, B2). That is why the server didn't send a FIN for B1 when the client had disconnected B1. So the two connections the server has are A2 and B1, which are obviously out of sync. It tries writing to B1, which is a dead connection, so it drops A2 and B1. Then the next pair are A3 and B2, which are also invalid pairs. They never recover from being out of sync until the server process is killed and the TCP connections are all reset.
So the solution was to just change a timeout for waiting on the B socket from 200ms to 5s. Such a simple fix that had me scratching my head for days (and fixed it within 24 hours of putting it on stackoverflow)! I also made it recover from stray B connections by adding socket B to the main select() call, and then accept()ing it and close()ing it immediately (which would only happen if the B connection took longer than 5s to establish). Thanks #AlanCurry for the suggestion of adding it to the select() and adding the puzzle piece about the listen() backlog parameter being a hint.
I am hooking WSASend, and WSARecv in C++ using the same method I've used to hook the client's WSASend and WSARecv functions. In the client I am able to get the IP, Port, and Socket from the SOCKET structure passed by WSASend/WSARecv; however, for the server when I try to use getpeername or getsockname() they both return the error 10057 (Socket not connected)...
I'm fairly sure that the hook is correct on the server, since it prints the bytes successfully, and I'm also sure the socket SHOULD be valid seeing how client and server establish a successful connection.
Is there a way to resolve this problem by any other alternative methods? I've been looking around the internet to find a solution, but I haven't seen anyone with the same problem.
I've tried this:
sockaddr *address = new sockaddr;
int peer_len;
getpeername(s, address, &peer_len);
int err = WSAGetLastError();
if(err==0)
{
char *Str = inet_ntoa(((sockaddr_in*)address)->sin_addr);
printf("[%s", Str);
printf(":%d]",ntohs(((sockaddr_in*)address)->sin_port));
}
else
{
printf("Error %i\n",err);
}
(Using both getpeername and getsockname)Both result in the same socket not connected error.
I'm planning on using the packets the C++ dll gets and forward the information to the C# dll since it'll be easier to manage on that (for me anyways), but I'd need to distinguish each packet with it's socket id.
You can only do that on the connected socket, i.e. the one returned from the accept() call, not on the listening "server" socket.