How should one monitor data that went through a FIFO ? Simply open and keep watching doesn't work, since if the monitor reads all bytes, the actual program that needs data will fail to receive the data.
I am not sure what kind of FIFO you have there (pipe? socket? maybe you should elaborate more on your question in general), but the only case where I know about forward-reading is with sockets.
You can use recv() with the flag MSG_PEEK with the following effect:
This flag causes the receive operation to return data from the
beginning of the receive queue without removing that data from the
queue. Thus, a subsequent receive call will return the same data.
You can implement IPC with sockets, too (unix(7)), so you might want to add them to your project (if you are using linux/unix). If you want to know how to use sockets then you should read the man page: socket(2) and socket(7) or in case of Windows, recv() and socket().
You might also want to try to use 2 FIFO's, one to your monitor and the other one from your monitor to your actual program. Then you simply read all incoming data with your monitor and filter the relevant parts and write them directly to your actual program. This might come in handy if you have multiple receivers inside your actual programs and want to split up the incoming data.
If you simply want to know whether there is data to read, you can use select(2) or pselect(2) or maybe poll(2), or select()
You should use one of the following system calls:
select()- source: man -s 2 select
pselect()- source: man -s 2 pselect
select() and pselect() allow a program to monitor multiple
file
descriptors, waiting until one or more of the file descriptors become
"ready" for some class of I/O operation (e.g., input possible). A file
descriptor is considered ready if it is possible to perform the corre-
sponding I/O operation (e.g., read(2)) without blocking.
Note they are all I/O blocking calls.
ppoll()- man -s 2 ppoll
poll()- man -s 2 poll
Also read the difference between the both set of system calls: http://www.unixguide.net/network/socketfaq/2.14.shtml
And using pselect or ppoll is always better than select and poll for safer uses.
Related
In linux, how does one generate an event to break out a select / poll / epoll loop on thread termination? Processes have a pidfd and SIGCHILD. Is there something similar for threads?
Edit: this is to directly monitor the thread termination event.
Well the most obvious solution, that comes to mind, is that one of the file descriptors being polled/selected for would be a very special file descriptor, that's reserved for that particular purpose. When you want to "break out" of the select/poll/epoll you simply need to make the appropriate arrangements for this, very special, file descriptor to become available for reading, and this will make it happen.
After select/poll/epoll returns you'll check that file descriptor, just like you would check any other one, and proceed according to whatever should happen in that event. So the only remaining part of this question is what kind of a very special file descriptor would this be?
Well, since you tagged your question with linux, you have many Linux-specific options to choose from.
You can turn off native signal handling in your process, and create a signal file descriptor. Then a sent signal to the process translates to the signal file descriptor becoming available for reading, and reading from it, as documented in the manual page, tells you that the signal has been received.
An event file descriptor could be another option, this one's more suitable for different threads in the same process notifying each other.
Both event and signal file descriptors are eminently pollable/selectable. And there's always the old-school approach of creating a pipe(), selecting/polling the read end of the pipe, and writing to the write end of the pipe to effect the notification.
Is there a way to check the number of bytes available from a USB device (printer in our case)?
We're using CreateFile and ReadFile and WriteFile for IO communications with our USB device, which works. But We can't figure out how much data is available without actually doing a read. We can't use GetFileSize, as even the documentation says you can't use it for a :
"nonseeking device such as a pipe or a communications device"...
So that doesn't work. Any suggestions? Are we doing our USB I/O incorrectly? Is there a better way to Read/Write to USB?
You first need to open up the port in asynchronous mode. To do that, pass the flag FILE_FLAG_OVERLAPPED to CreateFile. Then, when you call ReadFile, pass in a pointer to an OVERLAPPED structure. This does an asynchronous read and immediately returns ERROR_IO_PENDING without blocking (or, if the OS already has the data buffered, you might get lucky and get a successful read -- be prepared to handle that case).
Once the asynchronous I/O has started, you can then periodically check if it has completed with GetOverlappedResult.
This allows you to answer the question "are X bytes of data available?" for a particular value of X (the one passed to ReadFile). 95% of the time, that's good enough, since you're looking for data in a particular format. The other 5% of the time, you'll need to add another layer of abstraction top, where you keep doing asynchronous reads and store the data in a buffer.
Note that asynchronous I/O is very tricky to get right, and there's a lot of edge cases to consider. Carefully read all of the documentation for these functions to make sure your code is correct.
Can you use C#? If so you can access the USB port using System.IO.SerialPort class, and then set up a DataReceived event handler for incoming data. There is a BytesToRead property that tells you how much data is waiting to be read.
All of this must be available in native code, if I can find it I'll edit this.
EDIT: the best I can find for native is ReadPrinter - I don't see how to check if data is there, this will block if it's not.
I have a process that continuously needs to write information. Furthermore, there is a second process which sometimes connects to the "information channel" of the writing process and should read the information that are written since it's connected. This process might also deconnect and reconnect several times again.
I am currently realizing this with a named pipe, by using mkfifo() in my c++ program. Unfortunately if I call open() on this fifo it blocks until a process opens the fifo for reading. This is quite normal for named pipes, but I need this open command to be non-blocking.
Do you know an alternative to mkfifo in this case?
Heinrich
You could use Unix-domain sockets, or regular TCP sockets on loopback interface.
You can use shared memory or mmap. It should contain offset to the oldest data, and the block of memory for data
fifo is limited to 64k (depends on distribution and some settings).
I finally used the unix message queue, Reader and Writer can.be started totally independet and everything can be performed non blocking
I'm designing event loop for asynchronous socket IO using epoll/devpoll/kqueue/poll/select (including windows-select).
I have two options of performing, IO operation:
Non-blocking mode, poll on EAGAIN
Set socket to non-blocking mode.
Read/Write to socket.
If operation succeeds, post completion notification to event loop.
If I get EAGAIN, add socket to "select list" and poll socket.
Polling mode: poll and then execute
Add socket to select list and poll it.
Wait for notification that it is readable writable
read/write
Post completion notification to event loop of sucseeds
To me it looks like first would require less system calls when using in normal mode,
especially for writing to socket (buffers are quite big).
Also it looks like that it would be possible to reduce the overhead over number of "select"
executions, especially it is nice when you do not have something that scales well
as epoll/devpoll/kqueue.
Questions:
Are there any advantages of the second approach?
Are there any portability issues with non-blocking operations on sockets/file descriptors over numerous operating systems: Linux, FreeBSD, Solaris, MacOSX, Windows.
Notes: Please do not suggest using existing event-loop/socket-api implementations
I'm not sure there's any cross-platform problem; at the most you would have to use Windows Sockets API, but with the same results.
Otherwise, you seem to be polling in either case (avoiding blocking waits), so both approaches are fine. As long as you don't put yourself in a position to block (ex. read when there's no data, write when buffer's full), it makes no difference at all.
Maybe the first approach is easier to code/understand; so, go with that.
It might be of interest to you to check out the documentation of libev and the c10k problem for interesting ideas/approaches on this topic.
The first design is the Proactor Pattern, the second is the Reactor Pattern
One advantage of the reactor pattern is that you can design your API such that you don't have to allocate read buffers until the data is actually there to be read. This reduces memory usage while you're waiting for I/O.
from my experience with low latency socket apps:
for writes - try to write directly into the socket from writing thread (you need to obtain event loop mutex for that), if write is incomplete subscribe to write readiness with event loop (select/waitformultipleobjects) and write from event loop thread when socket gets writable
for reads - be always "subscribed" for read readiness for all sockets, so you always read from within event loop thread when the socket gets readable
We're trying to read data from 2 usb mice connected to a linux box (this data is used for odometry/localization on a robot). So we need to continuously read from each mouse how much it moved. The problem is that when a mouse is not moving, it doesn't send any data, so the file stream from which we get the data blocks execution and therefore the program can't do the odometry calculations (which involve time measurement for speed).
Is there a way to set a timeout on the input stream (we're using ifstream in C++ and read from /dev/input/mouse), so that we're able to know when the mouse doesn't move, instead of waiting for an event to be received? Or do we need to mess up with threads (arggh...)? Any other suggestions are welcome!
Thanks in advance!
A common way to read from multiple file descriptors in linux is to use select(). I suggest starting with the manpage. The basic system flow is as follows:
1) Initialize devices
2) Obtain list of device file descriptors
3) Setup the time out
4) Call select with file descriptors and timeout as parameters - it will block until there is data on one of the file descriptors or the time out is reached
5) Determine why select returned and act accordingly (i.e. call read() on the file descriptor that has data). You may need to internally buffer the result of read until an entire data gram is obtained.
6) loop back to 4.
This can become your programs main loop. If you already have a different main loop you, can run the above without looping, but your will need to insure that the function is called frequently enough such that you do not lose data on the serial ports. You should also insure that your update rate (i.e. 1/timeout) is fast enough for your primary task.
Select can operate on any file descriptor such network sockets and anything else that exposes an interface through a file descriptor.
What you're looking for would be an asynchronous way to read from ifstream, like socket communication. The only thing that could help would be the readsome function, perhaps it returns if no data is available, but I doubt this helps.
Using threads would be the best way to handle this.
Take a look at the boost Asio library. This might help you deal with the threading suggested by schnaeder.
No, there is no such method. You'll have to wait for an event, or create a custom Timer class and wait for a timeout to repoll, or use threads.