IOCP AcceptEx not creating completion upon connect - c++

I am currently trying some new libraries (IOCP) for socket programming. And I've stumbled upon the AcceptEx functionality to enable async connections.
As the documentation says:
The AcceptEx function uses overlapped I/O, unlike the accept function. If your application uses AcceptEx, it can service a large number of clients with a relatively small number of threads. As with all overlapped Windows functions, either Windows events or completion ports can be used as a completion notification mechanism.
But I am not receving any completion when a client connects. I do however get a completion when the client sends data..
This is my code:
DWORD dwBytes;
GUID GuidAcceptEx = WSAID_ACCEPTEX;
int iResult = WSAIoctl(m_hSocket, SIO_GET_EXTENSION_FUNCTION_POINTER,
&GuidAcceptEx, sizeof (GuidAcceptEx),
&m_lpfnAcceptEx, sizeof (m_lpfnAcceptEx),
&dwBytes, NULL, NULL);
if (iResult == SOCKET_ERROR)
{
CloseSocket();
}
And then:
WSAOVERLAPPED olOverlap;
memset(&olOverlap, 0, sizeof (olOverlap));
char lpOutputBuf[1024];
int outBufLen = 1024;
DWORD dwBytes;
BOOL bRet = m_lpfnAcceptEx( m_hSocket, hSocket, lpOutputBuf,
outBufLen - ((sizeof (sockaddr_in) + 16) * 2),
sizeof (sockaddr_in) + 16, sizeof (sockaddr_in) + 16,
&dwBytes, &olOverlap);
if ( bRet == FALSE )
{
DWORD dwRet = WSAGetLastError();
if( dwRet != WSA_IO_PENDING )
{
return dwRet;
}
}
Any suggestion of what to do to receive completions?
EDIT:
I bind the hSocket to the completionport after m_lpfnAcceptEx()

Firstly the WSAOVERLAPPED and data buffer that you're declaring on the stack above your call to AcceptEx() will not be in existence when a completion occurs (unless you are calling GetQueuedCompletionStatus() in the same function, which would be a trifle odd). You need to dynamically allocate them or pool them.
Secondly you state that you associate the socket to the completion port after you call AcceptEx(). That's wrong. You need to do these things before you call AcceptEx().
Create a socket with WSA_FLAG_OVERLAPPED set.
Bind it to the address you want to listen on.
Call listen on it with your desired backlog.
Load AcceptEx() dynamically using the listening socket and a call to WSAIoctl (not strictly necessary and the code you show should work but this way you can be sure you get your listening socket from the same underlying winsock provider and that it supports AcceptEx().
Load GetAcceptExSockaddrs() in the same way as you load AcceptEx() - you'll need it once the accept completes.
Associate the listening socket to your IOCP.
Now you can post a number of AcceptEx() calls using the listening socket and new 'accept' socket which you create like this:
Create a socket with WSA_FLAG_OVERLAPPED set.
Associate the socket to your IOCP.
As stated above you need to ensure that the buffer and the OVERLAPPED are unique per call and last until the completion occurs.
When the completion occurs you have to do the following....
Call setsockopt() with SO_UPDATE_ACCEPT_CONTEXT on the accepted socket using the listening socket as the data...
Deblock your addresses using GetAcceptExSockaddrs().
Process any data (if you allocated enough space in the buffer for data).
Note that by design AcceptEx() can be used to accept a new connection and return the initial data from that connection in one operation (this leads to slightly better performance in situations where you know you will always want some data before you can start doing things but is horribly complex to manage if you want to defend aginst the denial of service attack that can be launched simply by connecting and NOT sending data - I wrote about this here).
If you do not want AcceptEx() to wait for data to arrive then simply provide a data buffer that is ONLY big enough for the addresses to be returned and pass 0 as the 'buffer size'. This will cause the AcceptEx() to operate like an overlaped accept() and return as soon as the connection is established.
Note that Martin James' initial comment to your question is in fact the answer you're looking for. Don't pass outBufLen - ((sizeof (sockaddr_in) + 16) * 2), pass 0.

Related

UnrealEngine4: Recv function would keep blocking when TCP server shutdown

I use a blocking FSocket in client-side that connected to tcp server, if there's no message from server, socket thread would block in function FScoket::Recv(), if TCP server shutdown, socket thread is still blocking in this function. but when use blocking socket of BSD Socket API, thread would pass from recv function and return errno when TCP server shutdown, so is it the defect of FSocket?
uint32 HRecvThread::Run()
{
uint8* recv_buf = new uint8[RECV_BUF_SIZE];
uint8* const recv_buf_head = recv_buf;
int readLenSeq = 0;
while (Started)
{
//if (TcpClient->Connected() && ClientSocket->GetConnectionState() != SCS_Connected)
//{
// // server disconnected
// TcpClient->SetConnected(false);
// break;
//}
int32 bytesRead = 0;
//because use blocking socket, so thread would block in Recv function if have no message
ClientSocket->Recv(recv_buf, readLenSeq, bytesRead);
.....
//some logic of resolution for tcp msg bytes
.....
}
delete[] recv_buf;
return 0
}
As I expected, you are ignoring the return code, which presumably indicates success or failure, so you are looping indefinitely (not blocking) on an error or end of stream condition.
NB You should allocate the recv_buf on the stack, not dynamically. Don't use the heap when you don't have to.
There is a similar question on the forums in the UE4 C++ Programming section. Here is the discussion:
https://forums.unrealengine.com/showthread.php?111552-Recv-function-would-keep-blocking-when-TCP-server-shutdown
Long story short, in the UE4 Source, they ignore EWOULDBLOCK as an error. The code comments state that they do not view it as an error.
Also, there are several helper functions you should be using when opening the port and when polling the port (I assume you are polling since you are using blocking calls)
FSocket::Connect returns a bool, so make sure to check that return
value.
FSocket::GetLastError returns the UE4 Translated error code if an
error occured with the socket.
FSocket::HasPendingData will return a value that informs you if it
is safe to read from the socket.
FSocket::HasPendingConnection can check to see your connection state.
FSocket::GetConnectionState will tell you your active connection state.
Using these helper functions for error checking before making a call to FSocket::Recv will help you make sure you are in a good state before trying to read data. Also, it was noted in the forum posts that using the non-blocking code worked as expected. So, if you do not have a specific reason to use blocking code, just use the non-blocking implementation.
Also, as a final hint, using FSocket::Wait will block until your socket is in a desirable state of your choosing with a timeout, i.e. is readable or has data.

WSARecv sometimes return "invalid handle (error no 6)" for a socket associated with an IOCP port. (C++)

I'm trying to write a server which can support many clients connections simultaneously so I'm trying to do it with IOCP. So let me brief about my code flow and then I can explain my problem. First of all, server is opening a port for listening and waiting on an "accept" call for new incoming connections. For reference I have used same code as mentioned here So it accepts every new incoming connection and returns a new socket descriptor (sd), and then it marks as nonblocking with:
arg = 1;
ioctlsocket(sd, FIONBIO, &arg);
and then enable TCP_NODELAY:
level = IPPROTO_TCP;
optName = TCP_NODELAY;
value = 1;
setsockopt(sd, level, optName, (const char*)&value, sizeof(value));
thereafter associating with an IOCP port as:
CreateIoCompletionPort((HANDLE)sd, iocp_port, (DWORD)completion_key, 4);
completion_key is a class object which is nothing but a container, it contains data buffer, overlapped-buffer, query-type recv/send etc.
and in last issuing a read call:
WSARecv(sd, wsabuf, 1, &bytes, &flags, overlapped, NULL);
wsabuf and overlapped are part of completion_key object.
In 90% cases it works fine i.e. when there is some incoming data available on this socket "GetQueuedCompletionStatus" gets unblocked and it has valid data. But sometimes WSARecv call returns with an error and GetLastError() returns 6 which is "invalid handle" error. I'm bit bewildered why it's happening so.
The way I'm creating an iocp port:
iocp_port = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 0);
and there are threads which are waiting on "GetQueuedCompletionStatus".
I monitored all system calls which were happening in background. WSARecv internally calls NtDeviceIoControlFile and there is an argument "Event" which is same as what is passed in lpOverlapped structure of WSARecv as hEvent. I wasn't setting hEvent to NULL, so it was taking some garbage value, when it was NULL then NtDeviceIoControlFile returned successfully and for other cases it returned "INVALID_HANDLE" error. Unfortunately, it was NULL most of the time.

Socket is invalid while hooking WSASend/WSARecv on the server

I am hooking WSASend, and WSARecv in C++ using the same method I've used to hook the client's WSASend and WSARecv functions. In the client I am able to get the IP, Port, and Socket from the SOCKET structure passed by WSASend/WSARecv; however, for the server when I try to use getpeername or getsockname() they both return the error 10057 (Socket not connected)...
I'm fairly sure that the hook is correct on the server, since it prints the bytes successfully, and I'm also sure the socket SHOULD be valid seeing how client and server establish a successful connection.
Is there a way to resolve this problem by any other alternative methods? I've been looking around the internet to find a solution, but I haven't seen anyone with the same problem.
I've tried this:
sockaddr *address = new sockaddr;
int peer_len;
getpeername(s, address, &peer_len);
int err = WSAGetLastError();
if(err==0)
{
char *Str = inet_ntoa(((sockaddr_in*)address)->sin_addr);
printf("[%s", Str);
printf(":%d]",ntohs(((sockaddr_in*)address)->sin_port));
}
else
{
printf("Error %i\n",err);
}
(Using both getpeername and getsockname)Both result in the same socket not connected error.
I'm planning on using the packets the C++ dll gets and forward the information to the C# dll since it'll be easier to manage on that (for me anyways), but I'd need to distinguish each packet with it's socket id.
You can only do that on the connected socket, i.e. the one returned from the accept() call, not on the listening "server" socket.

TCP IOCP won't receive after acceptex

I'm trying to write an IOCP server. Basically, I have it accepting new connections. For the purpose of my testing, I'm running and connecting to 127.0.0.1.
I create the pseudo socket prior to calling AcceptEx(). Once a connection is accepted, the new pseudo socket is used for communication. This new socket is associated with an io completion port [CreateIoCompletionPort], I then assign it a few options, [SO_EXCLUSIVEADDRUSE] and [SO_CONDITIONAL_ACCEPT], and then I call WSARecv() to accept incoming data.
The problem is that once my remote connection connects to the server, it sends data, but that data is never received. I'm wondering if someone could offer some ideas as to why it's not receiving data? Perhaps my logic is flawed? I stepped through my code several times. no errors are recorded.
EDIT: Fixed the wording. I create the socket before AcceptEx() call.
Basic logic in my code:
// Create socket, associate with IOCP
WSASocket(af, type, proto, lpProtoInfo, g, dwFlags);
HANDLE hIOCP = GetPool()->GetQueueHandle();
CreateIoCompletionPort(hSource, hIOCP, 0, 0) != NULL;
// Server bind and listen
bind(m_shSocket, pAddr, nAddrLen);
listen(m_shSocket, nBacklog);
// Creation of the pseudo socket
SOCKET s = ::WSASocket(m_iSocketAf, m_iSocketType, m_iSocketProto, m_pWpi, m_SocketGroup, m_dwSocketFlags);
DWORD dwBytes;
BOOL bRet = m_fnAcceptEx(m_shSocket, s, chOutput, 0, sizeof(SOCKADDR_STORAGE) + 16, sizeof(SOCKADDR_STORAGE) + 16, &dwBytes, m_pcbAccept);
// ... New Connection comes in, it's accepted ...
// Associate new pseudo socket with IOCP
HANDLE hNewIOCP = GetPool()->GetQueueHandle();
CreateIoCompletionPort((HANDLE) s, hNewIOCP , 0, 0) != NULL;
// ... Remote socket sends ...
// ... Remote socket and Pseudo socket call WSARecv ...
// ... Pseudo socket does not receive ...
NOTE: I tried sending from the pseudo socket to the remote socket, same problem as sending data in the reverse way.
You need to post some code but your description doesn't make sense. That's NOT how AcceptEx() based servers operate.
With an AcceptEx() based server you create your accepted socket before you post the AcceptEx(). You then post the AcceptEx() with the listening socket and the new socket and a buffer which allows you to receive the remote address and, optionally, data.
So if you are describing your code in your original question then your code is wrong or you're not using AcceptEx(). I'm currently ignoring the 'few options' that you throw into the mix as they simply further confuse things at present without any code to analyse.
You might be interested in downloading my free IOCP based server framework, which includes working AcceptEx() and traditional Accept() based server code. You can get it from here: http://www.serverframework.com/products---the-free-framework.html
Are you calling GetQueuedCompletionStatus to get the data?
In case you are not doing this just to learn for yourself, I would also recommend that you use boost::asio - an excellent library that allows you to let someone else do the tedious code for handling the io completion ports.
I figured it out. I'm an idiot. I was sending zero bytes.

Non blocking socket from within a DLL (no window)

I have a DLL wich connects to a server through a single socket.
I am facing the following problem : If server's IP address & port are false or server is down, the application using this DLL freezes until half a minute.
My idea is to use non-blocking sockets to avoid that problem, by showing a window indicating connection progress, and also allowing to cancel this process.
But how could I use WSAAsyncSelect function as I have no window handler ?
If you want to use WSAAsyncSelect type sockets then your Dll will need to create at least one message window to handle socket events. As the window will never be visible, its WindowProc would consist only of a handler for your custom message (WM_USER+1) probably that you passed to WSAAsyncSelect - everything else goes straight to DefWindowProc.
You are going to have to create a modeless progress window on connect anyway to show your connecting UI.
It seems to me that your root problem is that IO operations are blocking your UI thread. I would try to move the connection to a separate thread as that should not block the UI, but run in parallel with it. It's a good idea to keep IO operations separate from the UI thread anyway. You can communicate between the two threads using the normal mechanisms such as semaphores.
Take a look at boost threads if you can, they're quite easy to use.
I suggest using an appropriate library, such as boost::asio which is also crossplatform and offers async connection handling capabilities
Another approach with a nonblocking socket is to use the select() function.
You can determine the completion of the connection by checking to see if the socket is writeable,
and you can also specify a timeout on the select.
I would agree that using a non-blocking socket, and then select() is the way to go in C. Here's some basic sample code that does a non-blocking connect on Windows with a 15 second timeout.
int s;
long on = 1L;
int socketsFound;
struct timeval timeout;
fd_set wfds;
struct addrinfo *addrInfo,
s = socket(addrInfo->ai_family, addrInfo->ai_socktype, addrInfo->ai_protocol);
if (s < 0)
{
/* Error */
return ;
}
if (ioctlsocket(s, FIONBIO, (u_long *)on))
{
/* Error */
return ;
}
if (connect(s, addrInfo->ai_addr, addrInfo->ai_addrlen) < 0)
{
if (WSAGetLastError()!= WSAEWOULDBLOCK)
{
/* Connection failed */
return;
}
}
/* 15 second timeout */
timeout.tv_sec = (long)15;
timeout.tv_usec = 0;
FD_ZERO(&wfds);
FD_SET(s, &wfds);
socketsFound = select((int)s+1, NULL, &wfds, NULL, &timeout);
if (socketsFound > 0 && FD_ISSET( s, &wfds ))
{
/* Connected */
}
Using WSAAsyncSelect isn't your only choice for non-blocking sockets in Winsock. It's actually the old Winsock 1.1/Win16 way of doing asynchronous sockets.
The Winsock 2+/Win32 way is to used overlapped I/O. See this article for a description of overlapped I/O with sockets.
Pass HWND_MESSAGE as the parent window to CreateWindow. This will create a message queue without a window. You will still need a WndProc, of course, because that's where you will process the messages.