I tried to read file contents using Libaio, but I found that even if I didn't call io_getevents method, the expected contents are obtained.
Is it necessary to call the io_getevents method after I call the io_submit method?
If yes, why this issue happened?
If no, when should I call io_getevents to the read result? Can I repead to call it multi times?
here is the demo code:
#include <stdio.h>
#include <libaio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
#define error() printf("error [%s : %d]\n", __FILE__, __LINE__)
#define BUFF_SIZE 51
#define BUFF_CNT 50
#define READLEN 4194304
int main(int argc, char *argv[])
{
int fd = open(__FILE__, O_RDONLY);
if (fd == -1) {
error();
return -1;
}
io_context_t ctx = 0;
int err = io_setup(BUFF_CNT, &ctx);
if (err != 0) {
error();
return -1;
}
struct iocb *io = (struct iocb *)malloc(sizeof(struct iocb));
if (NULL == io) {
printf("alloc struct iocb failed");
return -1;
}
memset(io, 0x00, sizeof(io));
char double_check_m_buf[READLEN];
io_prep_pread(io, fd, double_check_m_buf, READLEN, 0);
int rc = io_submit(ctx, 1, &io);
if (rc < 0) {
printf("aio send read one block failed");
return -1;
}
printf("aio send read one block success, len: %d \n content: %s",strlen(double_check_m_buf), double_check_m_buf);
There are two things that may be going on here.
There are a bunch of cases where io_submit can't do asynchronous I/O. When those happen, it falls back to synchronous I/O. It's up to the caller to realize the requested I/O happened inline of the call.
In your case, you need to open the file with O_RDONLY|O_DIRECT. That will likely do the trick, presuming your filesystem supports AIO.
If you open __FILE__ with O_DIRECT, io_submit will bypass the kernel's buffer cache. It may still complete rapidly. Depending on what the underlying storage is, it may very well complete before you can inspect the buffer. It's an off-chance, but still, it's possible.
The only way to be sure your I/O has completed is to call io_getevents. That's the only way to retrieve the error if it fails. Here, __FILE__ is likely far shorter than 4 MiB, so you'll need the length of the read that's returned in that structure as well.
Related
I have the given code:
#include <winsock2.h>
#include <sys/time.h>
#include <iostream>
int main()
{
WSADATA wsaData;
if (WSAStartup(MAKEWORD(2, 2), &wsaData) != 0)
{
std::cout << "WSA Initialization failed!" << std::endl;
WSACleanup();
}
timeval time;
time.tv_sec = 1;
time.tv_usec = 0;
int retval = select(0, NULL, NULL, NULL, &time);
if (retval == SOCKET_ERROR)
{
std::cout << WSAGetLastError() << std::endl;
}
return 0;
}
It prints 10022, which means error WSAEINVAL. According to this page, I can get this error only if:
WSAEINVAL: The time-out value is not valid, or all three descriptor parameters were null.
However, I have seen a few examples calling select() without any FD_SETs. Is it possible somehow? I need to do it in a client-side code to let the program sleep for short periods while it is not connected to the server.
However, I have seen a few examples calling select() without any
FD_SETs.
It will work in most OS's (that aren't Windows).
Is it possible somehow [under Windows]?
Not directly, but it's easy enough to roll your own wrapper around select() that gives you the behavior you want even under Windows:
int proper_select(int largestFileDescriptorValuePlusOne, struct fd_set * readFS, struct fd_set * writeFS, struct fd_set * exceptFS, struct timeVal * timeout)
{
#ifdef _WIN32
// Note that you *do* need to pass in the correct value
// for (largestFileDescriptorValuePlusOne) for this wrapper
// to work; Windows programmers sometimes just pass in a dummy value,
// because the current Windows implementation of select() ignores the
// parameter, but that's a portability-killing hack and wrong,
// so don't do it!
if ((largestFileDescriptorValuePlusOne <= 0)&&(timeout != NULL))
{
// Windows select() will error out on a timeout-only call, so call Sleep() instead.
Sleep(((timeout->tv_sec*1000000)+timeout->tv_usec)/1000);
return 0;
}
#endif
// in all other cases we just pass through to the normal select() call
return select(maxFD, readFS, writeFS, exceptFS, timeout);
}
... then just call proper_select() instead of select() and you're golden.
From the notorious and offensive Winsock 'lame list':
Calling select() with three empty FD_SETs and a valid TIMEOUT structure as a sleezy delay function.
Inexcusably lame.
Note the mis-spelling. The document is worth reading, if you can stand it, just to see the incredible depths hubris can attain. In case they've recanted, or discovered that they didn't invent the Sockets API, you could try it with empty FD sets instead of null parameters, but I don't hold out much hope.
Currently I try to write a serial port communication in VC++ to transfer data from PC and robot via XBee transmitter. But after I wrote some commands to poll data from robot, I didn't receive anything from the robot (the output of filesize is 0 in the code.). Because my MATLAB interface works, so the problem should happen in the code not the hardware or communication. Would you please give me help?
01/03/2014 Updated: I have updated my codes. It still can not receive any data from my robot (the output of read is 0). When I use "cout<<&read" in the while loop, I obtain "0041F01C1". I also don't know how to define the size of buffer, because I don't know the size of data I will receive. In the codes, I just give it a random size like 103. Please help me.
// This is the main DLL file.
#include "StdAfx.h"
#include <iostream>
#define WIN32_LEAN_AND_MEAN //for GetCommState command
#include "Windows.h"
#include <WinBase.h>
using namespace std;
int main(){
char init[]="";
HANDLE serialHandle;
// Open serial port
serialHandle = CreateFile("\\\\.\\COM8", GENERIC_READ | GENERIC_WRITE, 0, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
// Do some basic settings
DCB serialParams;
DWORD read, written;
serialParams.DCBlength = sizeof(serialParams);
if((GetCommState(serialHandle, &serialParams)==0))
{
printf("Get configuration port has a problem.");
return FALSE;
}
GetCommState(serialHandle, &serialParams);
serialParams.BaudRate = CBR_57600;
serialParams.ByteSize = 8;
serialParams.StopBits = ONESTOPBIT;
serialParams.Parity = NOPARITY;
//set flow control="hardware"
serialParams.fOutX=false;
serialParams.fInX=false;
serialParams.fOutxCtsFlow=true;
serialParams.fOutxDsrFlow=true;
serialParams.fDsrSensitivity=true;
serialParams.fRtsControl=RTS_CONTROL_HANDSHAKE;
serialParams.fDtrControl=DTR_CONTROL_HANDSHAKE;
if (!SetCommState(serialHandle, &serialParams))
{
printf("Set configuration port has a problem.");
return FALSE;
}
GetCommState(serialHandle, &serialParams);
// Set timeouts
COMMTIMEOUTS timeout = { 0 };
timeout.ReadIntervalTimeout = 30;
timeout.ReadTotalTimeoutConstant = 30;
timeout.ReadTotalTimeoutMultiplier = 30;
timeout.WriteTotalTimeoutConstant = 30;
timeout.WriteTotalTimeoutMultiplier = 30;
SetCommTimeouts(serialHandle, &timeout);
if (!SetCommTimeouts(serialHandle, &timeout))
{
printf("Set configuration port has a problem.");
return FALSE;
}
//write packet to poll data from robot
WriteFile(serialHandle,">*>p4",strlen(">*>p4"),&written,NULL);
//check whether the data can be received
char buffer[103];
do {
ReadFile (serialHandle,buffer,sizeof(buffer),&read,NULL);
cout << read;
} while (read!=0);
//buffer[read]="\0";
CloseHandle(serialHandle);
return 0;
}
GetFileSize is documented not to be valid when used with a serial port handle. Use the ReadFile function to receive serial port data.
You should use strlen instead of sizeof here:
WriteFile(serialHandle,init,strlen(init),&written,NULL)
You would be even better off creating a function like this:
function write_to_robot (const char * msg)
{
DWORD written;
BOOL ok = WriteFile(serialHandle, msg, strlen(msg), &written, NULL)
&& (written == strlen(msg));
if (!ok) printf ("Could not send message '%s' to robot\n", msg);
}
But that's only the appetizer. The main trouble is, as MDN says:
You cannot use the GetFileSize function with a handle of a nonseeking device such as a pipe or a communications device.
If you want to read from the port, you can simply use ReadFile until it returns zero bytes.
If you already know the max size of your robot's response, try reading that many characters.
Continue reading until the read reports an actual number of bytes read inferior to the size of the buffer. For instance:
#define MAX_ROBOT_ANSWER_LENGTH 1000 /* bytes */
const char * read_robot_response ()
{
static char buffer[MAX_ROBOT_ANSWER_LENGTH];
DWORD read;
if (!ReadFile (serialHandle, buffer, sizeof(buffer), &read, NULL))
{
printf ("something wrong with the com port handle");
exit (-1);
}
if (read == sizeof(buffer))
{
// the robot response is bigger than it should
printf ("this robot is overly talkative. Flushing input\n");
// read the rest of the input so that the next answer will not be
// polluted by leftovers of the previous one.
do {
ReadFile (serialHandle, buffer, sizeof(buffer), &read, NULL);
} while (read != 0);
// report error
return "error: robot response exceeds maximal length";
}
else
{
// add a terminator to string in case Mr Robot forgot to provide one
buffer[read] = '\0';
printf ("Mr Robot said '%s'\n", buffer);
return buffer;
}
}
This simplistic function returns a static variable, which will be overwritten each time you call read_robot_response.
Of course the proper way of doing things would be to use blocking I/Os instead of waiting one second and praying for the robot to answer in time, but that would require a lot more effort.
If you feel adventurous, you can use overlapped I/O, as this lenghty MDN article thoroughly explores.
EDIT: after looking at your code
// this reads at most 103 bytes of the answer, and does not display them
if (!ReadFile(serialHandle,buffer,sizeof(buffer),&read,NULL))
{
printf("Reading data to port has a problem.");
return FALSE;
}
// this could display the length of the remaining of the answer,
// provided it is more than 103 bytes long
do {
ReadFile (serialHandle,buffer,sizeof(buffer),&read,NULL);
cout << read;
}
while (read!=0);
You are displaying nothing but the length of the response beyond the first 103 characters received.
This should do the trick:
#define BUFFER_LEN 1000
DWORD read;
char buffer [BUFFER_LEN];
do {
if (!ReadFile(
serialHandle, // handle
buffer, // where to put your characters
sizeof(buffer) // max nr of chars to read
-1, // leave space for terminator character
&read, // get the number of bytes actually read
NULL)) // Yet another blody stupid Microsoft parameter
{
// die if something went wrong
printf("Reading data to port has a problem.");
return FALSE;
}
// add a terminator after last character read,
// so as to have a null terminated C string to display
buffer[read] = '\0';
// display what you actually read
cout << buffer;
}
while (read!=0);
I advised you to wrap the actual calls to serial port accesses inside simpler functions for a reason.
As I said before, Microsoft interfaces are a disaster. They are verbose, cumbersome and only moderately consistent. Using them directly leads to awkward and obfuscated code.
Here, for instance, you seem to have gotten confused between read and buffer
read holds the number of bytes actually read from the serial port
buffer holds the actual data.
buffer is what you will want to display to see what the robot answered you
Also, you should have a documentation for your robot stating which kind of answers you are supposed to expect. It would help to know how they are formatted, for instance whether they are null-terminated strings or not. That could dispense to add the string terminator.
I am trying to guess how much data is in pipe, and I don't want to use while(read) because it is blocking until EOF.
Is there any way to do that?
I real I want something like this:
i = pipe1.size();
pipe1.read(i);
I say again, I don't want to use while (read) because it is blocking until EOF.
The amount of data coming from a pipe could be infinite, just a like a stream, there's no concept of size in a pipe. if you don't want it to block if there's nothing to read you should set the O_NONBLOCK flag when calling pipe2():
pipe2(pipefd, O_NONBLOCK);
This way when you call read() if there's no data it would fail and set errno to EWOULDBLOCK
if (read(fd, ...) < 0) {
if (errno == EWOULDBLOCK) {
//no data
}
//other errors
}
From the man page:
O_NONBLOCK: Set the O_NONBLOCK file status flag on the two new open
file descriptions. Using this flag saves extra calls to fcntl(2) to
achieve the same result.
You could also use select() on a blocking pipe to timeout.
This could help you, however it is unix specific:
#include <iostream>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <errno.h>
int pipe_fd; /* your pipe's file descriptor */
......
int nbytes = 0;
//see how much data is waiting in buffer
if ( ioctl(pipe_fd, FIONREAD, &nbytes) < 0 )
{
std::cout << "error occured: " << errno;
}
else
{
std::cout << nbytes << " bytes waiting in buffer";
}
Using inotify to monitor a directory for any new file created in the directory by adding a watch on the directory by
fd = inotify_init();
wd = inotify_add_watch(fd, "filename_with_path", IN_CLOSE_WRITE);
inotify_add_watch(fd, directory_name, IN_CLOSE_WRITE);
const int event_size = sizeof(struct inotify_event);
const int buf_len = 1024 * (event_size + FILENAME_MAX);
while(true) {
char buf[buf_len];
int no_of_events, count = 0;
no_of_events = read(fd, buf, buf_len);
while(count < no_of_events) {
struct inotify_event *event = (struct inotify_event *) &buf[count];
if (event->len) {
if (event->mask & IN_CLOSE_WRITE) {
if (!(event->mask & IN_ISDIR)) {
//It's here multiple times
}
}
}
count += event_size + event->len;
}
When I scp a file to the directory, this loops infinitely. What is the problem with this code ? It shows the same event name and event mask too. So , it shows that the event for the same, infinite times.
There are no break statements. If I find an event, I just print it and carry on waiting for another event on read(), which should be a blocking call. Instead, it starts looping infinitely. This means, read doesn't block it but returns the same value for one file infinitely.
This entire operation runs on a separate boost::thread.
EDIT:
Sorry all. The error I was getting was not because of the inotify but because of sqlite which was tricky to detect at first. I think I jumped the gun here. With further investigation, I did find that the inotify works perfectly well. But the error actually came from the sqlite command : ATTACH
That command was not a ready-only command as it was supposed to. It was writing some meta data to the file. So inotify gets notification again and again. Since they were happening so fast, it screwed up the application.I finally had to breakup the code to understand why.
Thanks everyone.
I don't see anything wrong with your code...I'm running basically the same thing and it's working fine. I'm wondering if there's a problem with the test, or some part of the code that's omitted. If you don't mind, let's see if we can remove any ambiguity.
Can you try this out (I know it's almost the same thing, but just humor me) and let me know the results of the exact test?
1) Put the following code into test.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
#include <sys/inotify.h>
int main (int argc, char *argv[])
{
char target[FILENAME_MAX];
int result;
int fd;
int wd; /* watch descriptor */
const int event_size = sizeof(struct inotify_event);
const int buf_len = 1024 * (event_size + FILENAME_MAX);
strcpy (target, ".");
fd = inotify_init();
if (fd < 0) {
printf ("Error: %s\n", strerror(errno));
return 1;
}
wd = inotify_add_watch (fd, target, IN_CLOSE_WRITE);
if (wd < 0) {
printf ("Error: %s\n", strerror(errno));
return 1;
}
while (1) {
char buff[buf_len];
int no_of_events, count = 0;
no_of_events = read (fd, buff, buf_len);
while (count < no_of_events) {
struct inotify_event *event = (struct inotify_event *)&buff[count];
if (event->len){
if (event->mask & IN_CLOSE_WRITE)
if(!(event->mask & IN_ISDIR)){
printf("%s opened for writing was closed\n", target);
fflush(stdout);
}
}
count += event_size + event->len;
}
}
return 0;
}
2) Compile it with gcc:
gcc test.c
3) kick it off in one window:
./a.out
4) in a second window from the same directory try this:
echo "hi" > blah.txt
Let me know if that works correctly to show output every time the file is written to and does not loop as your code does. If so, there's something important your omiting from your code. If not, then there's some difference in the systems.
Sorry for putting this in the "answer" section, but too much for a comment.
My guess is that read is returning -1 and since you dont ever try to fix the error, you get another error on the next call to read which also returns -1.
I'm using the following code to write data through a named pipe from one application to another. The thread where the writing is taken place should never be exited. But if r_write() returns less than it should, the thread/program stops for some reason. How can I make the thread continue once write has returned less than it should?
ssize_t r_write(int fd, char *buf, size_t size)
{
char *bufp;
size_t bytestowrite;
ssize_t byteswritten;
size_t totalbytes;
for (bufp = buf, bytestowrite = size, totalbytes = 0;
bytestowrite > 0;
bufp += byteswritten, bytestowrite -= byteswritten) {
byteswritten = write(fd, bufp, bytestowrite);
if ((byteswritten) == -1 && (errno != EINTR))
return -1;
if (byteswritten == -1)
byteswritten = 0;
totalbytes += byteswritten;
}
return totalbytes;
}
void* sendData(void *thread_arg)
{
int fd, ret_val, count, numread;
string word;
char bufpipe[5];
ret_val = mkfifo(pipe, 0777); //make the sprout pipe
if (( ret_val == -1) && (errno != EEXIST))
{
perror("Error creating named pipe");
exit(1);
}
while(1)
{
if(!sproutFeed.empty())
{
string s;
s.clear();
s = sproutFeed.front();
int sizeOfData = s.length();
snprintf(bufpipe, 5, "%04d", sizeOfData);
char stringToSend[strlen(bufpipe) + sizeOfData +1];
bzero(stringToSend, sizeof(stringToSend));
strncpy(stringToSend,bufpipe, strlen(bufpipe));
strncat(stringToSend,s.c_str(),strlen(s.c_str()));
strncat(stringToSend, "\0", strlen("\0"));
int fullSize = strlen(stringToSend);
cout << "sending string" << stringToSend << endl;
fd = open(pipe,O_WRONLY);
int numWrite = r_write(fd, stringToSend, strlen(stringToSend) );
if(numWrite != fullSize)
{
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
cout << "NOT FULL SIZE WRITE " << endl; //program ends here??
}
else
{
sproutFeed.pop();
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
}
}
else
{
sleep(1);
}
}
}
If the write() returns a positive (non-zero, non-negative) value for the number of bytes written, it was successful, but there wasn't room for all the data. Try again, writing the remainder of the data from the buffer (and repeat as necessary). Don't forget, a FIFO has a limited capacity - and writers will be held up if necessary.
If the write() returns a negative value, the write failed. The chances are that you won't be able to recover, but check errno for the reason why.
I think the only circumstance where write() can return zero is if you have the file descriptor open with O_NONBLOCK and the attempt to write would block. You might need to scrutinize the manual page for write() to check for any other possibilities.
What your thread does then depends on why it experienced a short write, and what you want to do about it.
The write to the FIFO failed. Investigate the value of errno to find out why. Look in errno.h on your system to decipher the value of errno. If the program is ending upon trying to write to the console, the reason may be related.
Also, your loop doesn't appear to be closing the file descriptor for the FIFO (close(fd)).
Finally, you mention multithreading. The standard library stream cout on your system may not (and probably isn't) thread-safe. In that case, writing to the console concurrently from multiple threads will cause unpredictable errors.
You need to make the file descriptor non-blocking. You can do it like this:
fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | O_NONBLOCK);
Explanation
This is how fcntl works (not a complete description - look at man fcntl for that). First of all, the includes:
#include <unistd.h>
#include <fcntl.h>
reading the file descriptor's flags
Use F_GETFL to get the file descriptor's flags. From man fcntl:
F_GETFL
Read the file descriptor's flags.
RETURN VALUE
For a successful call, the return value depends on the operation:
F_GETFL Value of flags.
and this is how it's used:
int fd_flags = fcntl(fd, F_GETFL);
writing the file descriptor's flags
Use F_SETFL to set the O_NONBLOCK flag. Again, quoting from man fcntl:
F_SETFL
Set the file status flags part of the descriptor's flags to the
value specified by arg. Remaining bits (access mode, file cre?
ation flags) in arg are ignored. On Linux this command can
only change the O_APPEND, O_NONBLOCK, O_ASYNC, and O_DIRECT
flags.
and this is how it's used:
fcntl(fd, F_SETFL, fd_flags | O_NONBLOCK);