The WaitNamedPipe function allows a pipe client application to synchronously wait for an available connection on a named pipe server. You then call CreateFile to open the pipe as a client. Pseudocode:
// loop works around race condition with WaitNamedPipe and CreateFile
HANDLE hPipe;
while (true) {
if (WaitNamedPipe says connection is ready) {
hPipe = CreateFile(...);
if (hPipe ok or last error is NOT pipe busy) {
break; // hPipe is valid or last error is set
}
} else {
break; // WaitNamedPipe failed
}
}
The problem is that these are all blocking, synchronous calls. What is a good way to do this asynchronously? I can't seem to find an API that uses overlapped I/O to do this, for example. For example, for pipe servers the ConnectNamedPipe function provides an lpOverlapped parameters allowing for a server to asynchronously wait for a client. The pipe server can then call WaitForMultipleObjects and wait for the I/O operation to complete, or any other event to be signaled (for example, an event signaling the thread to cancel pending I/O and terminate).
The only way I can think of is to call WaitNamedPipe in a loop with a short, finite timeout and check other signals if it times out. Alternatively, in a loop call CreateFile, check other signals, and then call Sleep with a short delay (or WaitNamedPipe). For example:
HANDLE hPipe;
while (true) {
hPipe = CreateFile(...);
if (hPipe not valid and pipe is busy) {
// sleep 100 milliseconds; alternatively, call WaitNamedPipe with timeout
Sleep(100);
// TODO: check other signals here to see if we should abort I/O
} else
break;
}
But this method stinks to high heaven in my opinion. If a pipe isn't available for awhile, the thread continues to run - sucking up CPU, using power, requiring memory pages to remain in RAM, etc. In my mind, a thread that relies on Sleep or short timeouts does not perform well and is a sign of sloppy multi-threaded programming.
But what's the alternative in this case?
WaitNamedPipe is completely useless, and will just use all the cpu if you specify a timeout and there's no server waiting for it.
Just call CreateFile over and over with a Sleep like you're doing, and move it to other threads as you see appropriate. There is no API alternative.
The only "benefit" WaitNamedPipe provides is if you want to know if you can connect to a named pipe but you explicitly don't want to actually open a connection. It's junk.
If you really want to be thorough, your only options are
Ensure that whatever program is opening the named pipe is always calling CreateNamedPipe again immediately after it's named pipe is connected to.
Have your program actually check if that program is running.
If your intent is really not to have additional connections, still call CreateNamedPipe, and when someone connects, tell them to go away until they're waited a given amount of time, the close the pipe.
Why can't the server just create more pipes? The performance hit in the scenario you describe isn't a problem if it is rare.
I.e. if there are usually enough pipes to go round what does it matter if you use CreateFile/Sleep instead of WaitForMultipleObjects? The performance hit will not matter.
I also have to question the need for overlapped IO in a client. How many servers is it communicating with at a time? If the answer is less than, say, 10 you could reasonably create a thread per connection.
Basically I am saying I think the reason there is no overlapped WaitforNamedPipe is because there is no reasonable use-case which requires it.
You can open the pipe file system at \\.\pipe\ and then use DeviceIoControl to send FSCTL_PIPE_WAIT.
Related
I have a loop which basically calls this every few seconds (after the timeout):
while(true){
if(finished)
return;
switch(select(FD_SETSIZE, &readfds, 0, 0, &tv)){
case SOCKET_ERROR : report bad stuff etc; return;
default : break;
}
// do stuff with the incoming connection
}
So basically for every few seconds (which is specified by tv), it reactivates the listening.
This is run on thread B (not a main thread). There are times when I want to end this acceptor loop immediately from thread A (main thread), but seems like I have to wait until the time interval finishes..
Is there a way to disrupt the select function from another thread so thread B can quit instantly?
The easiest way is probably to use pipe(2) to create a pipe and add the read end to readfds. When the other thread wants to interrupt the select() just write a byte to it, then consume it afterward.
Yes, you create a connected pair of sockets. Then thread B writes to one side of socket and thread A adds the other side socket to select. So once B writes to socket A exits select, do not forget to read this byte from socket.
This is the most standard and common way to interrupt selects.
Notes:
Under Unix, use socketpair to create a pair of sockets, under windows it is little bit tricky but googling for Windows socketpair would give you samples of code.
Can't you just make the timeout sufficiently short (like 10ms or so?).
These "just create a dummy connection"-type solution seem sort of hacked. I personally think that if an application is well designed, concurrent tasks never have to be interrupted forcefully, the just has worker check often enough (this is also a reason why boost.threads do not have a terminate function).
Edit Made this answer CV. It is bad, but it might help other to understand why it is bad, which is explained in the comments.
You can use shutdown(Sock, SHUT_RDWR) call from main thread to come out of waiting select call which will also exit your another thread before the timeout so you don't need to wait till timeout expires.
cheers. :)
Say that I have two threads, the main thread and a thread that is dedicated to continuously listening on a standard TCP socket. Now, say that at some point I want to shutdown everything. From the main thread, I would like to close the connection the listening thread is working on, then join the thread and end the program.
However, this is tricky, since I don't know how to make the listening thread return from the call to read. That call won't return unless something actually is received, and in principle I could be waiting for a long long time until the other endpoint decides to send me something.
When I was working with UDP sockets, I used to work around this problem by sending a packet on that port from my loopback interface, therefore triggering a return from recvfrom. However, this is terribly inelegant and it cannot be done on a TCP socket.
I know that another workaround could be to set a timeout with setsockopt: in this way I am guaranteed that the call will eventually return, but this is inelegant as well, and also quite inefficient since I could be waiting for several seconds before being able to join the thread.
So I was wondering if there is some way to trigger an EAGAIN on a socket read call, not unlike the one I would get on a timeout, so that on my main thread I could just call some force_returnon my socket descriptor and the call to read on the other thread would return?
I usually solve this problem by creating a pipe() and using select() in the reading thread. The reading thread must select on both the TCP socket and one end of the pipe. Whenever you want to shut down the reader, set a flag and write some data to the other end of the pipe.
Setup:
#include <unistd.h>
int signalPipe[2];
...
pipe(signalPipe);
Reader:
while(running)
{
FD_ZERO(&fds);
FD_SET(tcpSocket, &fds);
FD_SET(signalPipe[0], &fds);
select(max(tcpSocket, signalPipe[0]) + 1, &fds, NULL, NULL, NULL);
...
}
Other thread:
// We want to stop now.
running = false;
write(signalPipe[1], "foo", 3);
using socket with the overlapped operation selected the event-based completion notification;
Have 2 events, one for data, the other to cancel long send/recv:
HANDLE events[] = { m_hDataEvent, m_hInterruptEvent };
then calling WSASend,
WSASend(m_Socket, &DataBuf, 1, NULL, 0, &SendOverlapped, NULL);
followed by
WSAWaitForMultipleEvents(2, events, FALSE, INFINITE, FALSE);
which is setup to return on any one event signaled.
Now assume send is in progress, and m_hInterruptEvent is signaled.
WSAWaitForMultipleEvents returns, technically the function calling send can return as well and delete internally allocated buffers.
What is not clear to me, the WSASend may still be working in background, and deleting buffers will cause data corruption in best case.
What would be the proper way to stop the background Send/Receive, if the socket needs to be used for something else immediately?
I looked at the CancelIO(), but the MSDN never mentions it in relation to Sockets, does it work with file based IO only?
It makes no sense to try to cancel it once sent. Even if you succeeded you would have a problem because the receiving application would not have any idea that the transmission was interrupted. Your new message will be mistaken for the end of the old message.
If you feel the need to cancel long sends, you should probably look at your application design.
Send in chunks and check for cancellation in between chunks. Ensure you have a way of communicating to the receiver that the transmission was cancelled.
Close the socket to cancel. Again, ensure the client has a way to know that this is an interrupted transmission (for example if the client knows the total length in advance they will recognise an interrupted transmission).
Just wait for it to succeed in the background and don't worry. If you have urgent messages use a separate connection for them.
For your particular question "What would be the proper way to stop the background Send/Receive, if the socket needs to be used for something else immediately", the answer is: Sockets are cheap - Just use two - one for the slow transmission the other for the urgent messages.
I have a loop which basically calls this every few seconds (after the timeout):
while(true){
if(finished)
return;
switch(select(FD_SETSIZE, &readfds, 0, 0, &tv)){
case SOCKET_ERROR : report bad stuff etc; return;
default : break;
}
// do stuff with the incoming connection
}
So basically for every few seconds (which is specified by tv), it reactivates the listening.
This is run on thread B (not a main thread). There are times when I want to end this acceptor loop immediately from thread A (main thread), but seems like I have to wait until the time interval finishes..
Is there a way to disrupt the select function from another thread so thread B can quit instantly?
The easiest way is probably to use pipe(2) to create a pipe and add the read end to readfds. When the other thread wants to interrupt the select() just write a byte to it, then consume it afterward.
Yes, you create a connected pair of sockets. Then thread B writes to one side of socket and thread A adds the other side socket to select. So once B writes to socket A exits select, do not forget to read this byte from socket.
This is the most standard and common way to interrupt selects.
Notes:
Under Unix, use socketpair to create a pair of sockets, under windows it is little bit tricky but googling for Windows socketpair would give you samples of code.
Can't you just make the timeout sufficiently short (like 10ms or so?).
These "just create a dummy connection"-type solution seem sort of hacked. I personally think that if an application is well designed, concurrent tasks never have to be interrupted forcefully, the just has worker check often enough (this is also a reason why boost.threads do not have a terminate function).
Edit Made this answer CV. It is bad, but it might help other to understand why it is bad, which is explained in the comments.
You can use shutdown(Sock, SHUT_RDWR) call from main thread to come out of waiting select call which will also exit your another thread before the timeout so you don't need to wait till timeout expires.
cheers. :)
To simplify, this is a situation where a NamedPipe SERVER is waiting for a NamedPipe CLIENT to write to the pipe (using WriteFile())
The Windows API that is blocking is ReadFile()
The Server has created the synchronous pipe (no overlapped I/O) with blocking enabled
The client has connected, and now the server is waiting for some data.
In the normal flow of things, the client sends some data and the server processes it and then returns to ReadFile() to wait for the next chunk of data.
Meanwhile an event occurs (user input for example) and the NamedPipe SERVER must now execute some other code, which it cannot do while the ReadFile() is blocking.
At this point I need to mention that the NamedPipe Client is not my application, so I have no control over it. I cannot make it send a few bytes to unblock the server. It is just going to sit there and send no data. Since I do not have control of the Client implementation I cannot change anything on that end.
One solution would be to create a separate thread in which all ReadFile() operations are performed. That way when the event occurs, I can just process the code. The problem with that, is that the event also requires a separate thread, so now I have two additional threads for each instance of this server. Since this needs to be scalable, this is undesirable.
From another thread I have tried calling
DisconnectNamedPipe()
and
CloseHandle()
they both will not return (until the client writes to the pipe.)
I cannot connect to the same pipe and write a few bytes because:
"All instances of a named pipe share the same pipe name, but each instance has
its own buffers and handles, and provides a separate conduit for client/server
communication."
http://msdn.microsoft.com/en-us/library/aa365590.aspx
I need a way to fake it out, So the $64k dollar question is:
How can I break the blocking of ReadFile()?
Try this before ReadFile :
BOOL WINAPI PeekNamedPipe(
__in HANDLE hNamedPipe,
__out_opt LPVOID lpBuffer,
__in DWORD nBufferSize,
__out_opt LPDWORD lpBytesRead,
__out_opt LPDWORD lpTotalBytesAvail,
__out_opt LPDWORD lpBytesLeftThisMessage
);
if(TotalBytesAvail > 0)
ReadFile(....);
-AV-
Take a look on CancelSynchronousIo
Marks pending synchronous I/O
operations that are issued by the
specified thread as canceled.
And CancelIo/CancelIoEx:
To cancel all pending asynchronous I/O
operations, use either:
CancelIo — this function only cancels
operations issued by the calling
thread for the specified file handle.
CancelIoEx — this function cancels all
operations issued by the threads for
the specified file handle.
http://msdn.microsoft.com/en-us/library/aa363794(VS.85).aspx
http://msdn.microsoft.com/en-us/library/aa365467(VS.85).aspx
Mike,
You can't cancel synchronous ReadFile. But you can switch to asynchronous (overlapped) operations. By doing this, you can implement a pretty scalable architecture.
Possible algorithm (just an idea):
For each new client call ReadFile
WaitForMultipleObjects where the handles are overlapped.hEvent + your
custom events
Iterate over signalled events, and schedule them for execution by threads from a threads pool.
This way you can have only few threads to receive connections and read data, while the actual data processing can be done by the threads pool.
The problem with that, is that the
event also requires a separate thread,
so now I have two additional threads
for each instance of this server.
Since this needs to be scalable, this
is undesirable.
Never in my career have I found that "more threads" == "less scalable". How many of these "server" instances do you have?
Normally, an operation needs to be performed in a separate thread if that operation is going to block and the system needs to be responsive while the operation is blocked.
Asynchronous I/O operations do not have to block any thread if they use I/O Completion Ports. See: http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx
What happening is the server outbound pipe is left open waiting for connection while your client is trying to connect to the server inbound pipe (which is no longer existent)... What you need to do is flush out your outbound pipe in order to loop back to your inbound. You can flush out on the client side by reading the file (remember to loop the connect establishment because there is a "handshake" in there, and it will never work the first time)
Just use SetNamedPipeHandleState function
https://learn.microsoft.com/en-us/windows/win32/api/namedpipeapi/nf-namedpipeapi-setnamedpipehandlestate
Use the PIPE_NOWAIT flag when calling this function.
hNamedPipe should be the handle that returned from CreateFile function.
After that, the call to ReadFile will not block the thread when no data available.