I am using inotify to monitor a local file, for example "/root/temp" using
inotify_add_watch(fd, "/root/temp", mask).
When this file is deleted, the program will be blocked by read(fd, buf, bufSize) function. Even if I create a new "/root/temp" file, the program is still block by read function. I am wondering if inotify can detect that the monitored file is created and the read function can get something from fd so that read will not be blocked forever.
Here is my code:
uint32_t mask = IN_ALL_EVENTS;
int fd = inotify_init();
int wd = inotify_add_watch(fd, "/root/temp", mask);
char *buf = new char[1000];
int nbytes = read(fd, buf, 500);
I monitored all events.
The problem is that read is a blocking operation by default.
If you don't want it to block, use select or poll before read. For example:
struct pollfd pfd = { fd, POLLIN, 0 };
int ret = poll(&pfd, 1, 50); // timeout of 50ms
if (ret < 0) {
fprintf(stderr, "poll failed: %s\n", strerror(errno));
} else if (ret == 0) {
// Timeout with no events, move on.
} else {
// Process the new event.
struct inotify_event event;
int nbytes = read(fd, &event, sizeof(event));
// Do what you need...
}
Note: untested code.
In order to see a new file created, you need to watch the directory, not the file. Watching a file should see when it is deleted (IN_DELETE_SELF) but may not spot if a new file is created with the same name.
You should probably watch the directory for IN_CREATE | IN_MOVED_TO to see newly created files (or files moved in from another place).
Some editors and other tools (e.g. rsync) may create a file under a different name, then rename it.
Related
I have a server and needs to feed data from clients to a library; however, that library only supports reading files (it uses open to access the file).
Since the data can get pretty big, I rather not write it out to a temporary file, read it in with the library then delete it afterwards. Instead I would like to do something similar to a ramdisk where there's a file in which the content is actually in memory.
However, there can be multiple clients sending over large data, I don't think constantly calling mount and umount to create a ramdisk for each client is efficient. Is there a way for me to mount an existing memory buffer as a file without writing to disk?
The library does not support taking in a file descriptor nor FILE*. It will only accept a path which it feeds directly to open
I do have the library's source code and attempted to add in a function that uses fmemopen; however, fmemopen returns a FILE* with no file descriptor. The internals of the library works only with file descriptors and it is too complex to change/add support to use FILE*
I looked at mmap, but it appears to be no different than writing out the data to a file
Using mount requires sudo access and I prefer not to run the application as sudo
bool IS_EXITING = false;
ssize_t getDataSize( int clientFD ) { /* ... */}
void handleClient( int clientFD ) {
// Read in messages to get actual data size
ssize_t dataSize = getDataSize( clientFD );
auto* buffer = new char[ dataSize ];
// Read in all the data from the client
ssize_t bytesRead = 0;
while( bytesRead < dataSize ) {
int numRead = read( clientFD, buffer + bytesRead, dataSize - bytesRead );
bytesRead += numRead;
// Error handle if numRead is <= 0
if ( numRead <= 0 ) { /* ... */ }
}
// Mount the buffer and get a file path... How to do this
std::string filePath = mountBuffer( buffer );
// Library call to read the data
readData( filePath );
delete[ ] buffer;
}
void runServer( int socket )
while( !IS_EXITING ) {
auto clientFD = accept( socket, nullptr, nullptr );
// Error handle if clientFD <= 0
if ( clientFD <= 0 ) { /* ... */ }
std::thread clientThread( handleClient, clientFD );
clientThread.detach( );
}
}
Use /dev/fd. Get the file descriptor of the socket, and append that to /dev/fd/ to get the filename.
If the data is in a memory buffer, you could create a thread that writes to a pipe. Use the file descriptor of the read end of the pipe with /dev/fd.
I'm reading CAN-BUS traffic under SocketCAN and C++ in GNU/Linux. I've found that the read call is blocking, and I'm struggling to figure out how to stop my program properly when I don't want to keep reading.
Of course, I could hit Ctrl+C if I've invoked the program from the terminal, but the point is to find a way to do it programmatically when some condition is met (e.g., record for 5 seconds, or when some event happens, like a flag is raised). A timeout could work, or something like a signal, but I don't know how to do it properly.
// Read (blocking)
nbytes = read(s, &frame, sizeof(struct can_frame));
You don't.
Use a method like select or epoll to determine whether the socket has activity before beginning the read. Then it will not actually block.
The select/epoll call is itself blocking, but can be given a timeout so that you always have an escape route (or, in the case of epoll, the lovely epollfd for immediate triggering of a breakout).
Read is always blocking... you want to only read if data is waiting... so consider doing a poll on the socket first to see if data is available and if so THEN read it. You can loop over doing the poll until you no longer want to read anymore...
bool pollIn(int fd)
{
bool returnValue{false};
struct pollfd *pfd;
pfd = calloc(1, sizeof(struct pollfd));
pfd.fd = fd;
pfd.events = POLLIN;
int pollReturn{-1};
pollReturn = poll(pfd, 1, 0);
if (pollReturn > 0)
{
if (pfd.revents & POLLIN)
{
returnValue = true;
}
}
free(pfd);
return(returnValue);
}
The above should return if there is data waiting at the socket file descriptor.
while(!exitCondition)
{
if(pollIn(fd))
{
nbytes = read(fd, &frame, sizeof(struct can_frame));
// other stuff you need to do with your read
}
}
I'm writing a small Server/Client Filetransfer using Winsocket for class and it basicly works except that i can't receiv any more messages on the socket after I receiv the file and write it to my HDD.
The transmit code loosk like this:
long size = GetFileSize(hFile, NULL);
TransmitFile(_socket, hFile, size, 0, NULL, NULL, TF_DISCONNECT);
ok = ::recv(_socket, cantwortfilename, 100, 0); // getting a confirmation (1)
cantwortfilename[ok] = '\0';
cout << cantwortfilename << endl;
char test[] = "ok!";
::send(_socket, test, strlen(test), 0); // to test if server receives again (2)
I tried it with 0 instead of size with same results.
now to the receiving on the server side:
HANDLE hFile = CreateFile(filepathlong, GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
while (1)
{
ReadFile((HANDLE)_socket, &buffer, MAX_PATH, &dwbr, &AsyncInfo);
GetOverlappedResult((HANDLE)_socket, &AsyncInfo, &dwbr, TRUE);
if (dwbr == 0) break;
WriteFile(hFile, buffer, dwbr, &dwbr, NULL);
}
char test[] = "alles ok!";
::send(_socket, test, strlen(test), 0); // sending the confirmation (1)
CloseHandle(hFile);
i = ::recv(_socket, test, 10, 0); // receiving the test (2)
test[i] = '\0';
cout << "empfangen:" << test << endl;
The transfer of the file works fine as far as i can tell (tried rar jpg .h) and the
::send(_socket, test, strlen(test), 0); // sending the confirmation (1)
gets out fine too.
but the receiving after that gives me nothing? or something empty?
I would guess something empty since the recv doesn't block the program either.
But "i" will be 0 when i give it out.
to check if i made some kind of error with the stuff in the while(1) loop i tried another way to receiv the file.
the 2nd try:
int r;
ofstream file(filepath, std::ios::out | std::ios::binary | std::ios::trunc);
char *memblock;
int size = 7766; // was the size of the file i was testing with
memblock = new char[size];
memset(memblock, 0, size);
if (file.is_open()){
//while memory blocks are still being received
while ((r = recv(_socket, memblock, 256, 0)) != 0)
{
//if there's a socket error, abort
if (r == SOCKET_ERROR)
{
cout << "error" << endl;
}
//write client's file blocks to file on server
file.write(memblock, r);
}
delete[] memblock;
//finished sending memory blocks; file is completely transferred.
file.close();
}
after that, again a send a recv with the same result.
The file worked but receiving again got me something empty.
So can anyone tell me why and how can fix this? If possible with the least change possible?
Thanks,
Martin
EDIT: Code that is working for me now:
char csize[256];
rc = recv(_socket, csize, 256, 0);
csize[rc] = '\0';
int size = atoi(csize);
int bytes_read = 0, len = 0;
int r;
ofstream file(filepath, std::ios::out | std::ios::binary | std::ios::trunc);
char *memblock;
memblock = new char[size];
memset(memblock, 0, size);
if (file.is_open()){
while (bytes_read < size){
len = recv(_socket, memblock + bytes_read, size - bytes_read, 0);
bytes_read += len;
}
file.write(memblock, size);
delete[] memblock;
file.close();
}
Your transmitter is calling TransmitFile() with the TF_DSCONNECT flag. That will close the socket after the file has finished being sent. The transmitter can't send any more data once TransmitFile() has exited (I am not sure if TransmitFile() only does a half-duplex close to close just the send direction but leaves the receive direction open, or if it does a full-duplex close on both send and receive directions - I would err on the side of caution and assume the latter).
Based on what you have shown, you need to use shutdown() and closesocket() instead of TF_DISCONNECT so you have more control over when and how the disconnect actually occurs.
Also, TransmitFile() does not send the file size, only the file data, so your receiver has no way of knowing when the file data has actually finished being received, unless it waits for a timeout (which is not a reliable solution). You need to change your transmitter to send the file size before sending the file data, then change your receiver to read the file size first and then loop until it has received the specified number of bytes.
I'm using the libssh library in MFC C++, specifically the SFTP wrapper. I have code working when not using threading, but I want to use AfxBeginThread to allow user action to continue.
I have confirmed I'm passing the exact same filenames and target save paths using the thread or not - I've tested this by hardcoding the file to save and the destination to save directly in BeginDownload().
Is there a specific way the SFTP wrapper for the libssh must be used to let the download perform within a thread? I would like to allow users to download multiple files at the same time from the same instance of the ssh objects, and just launch a new thread each time a file needs to be downloaded.
Here is what I'm using for the actual downloading. m_sftp_sesssion is my sftp_session object created upon logging in.
bool CSFtpManager::BeginDownload(CString filename, CString savefilename)
{
sftp_dir dir = sftp_opendir(m_sftp_session, "../../../directory");
sftp_attributes attrs = sftp_readdir(m_sftp_session, dir);
int access_type = O_RDONLY;
sftp_file file;
file = sftp_open(m_sftp_session, "../../../directory/myfile.zip", access_type, 0);
const char* x2 = ssh_get_error(m_ssh_session);
if(file == NULL)
{
int a;
}
int nbytes;
char buffer[1024];
FILE * pFile;
pFile = fopen(CStringA(savefilename), "wb");
nbytes = sftp_read(file, buffer, sizeof(buffer)); //this is crashing inside a thread
while(nbytes > 0)
{
fwrite(buffer, sizeof(char), sizeof(buffer), pFile);
nbytes = sftp_read(file, buffer, sizeof(buffer));
}
fclose(pFile);
sftp_close(file);
return true;
}
Here is my login method where my libssh objects are being set in member variables for later use. Included is a commented line calling my BeginDownload. Calling it outside a thread (i.e. clicking a login button) does work.
bool CSFtpManager::login()
{
int verbosity = SSH_LOG_PROTOCOL;
m_ssh_session = ssh_new();
ssh_options_set(m_ssh_session, SSH_OPTIONS_HOST, "ftp3 host..");
//setting username and password...
ssh_options_set(m_ssh_session, SSH_OPTIONS_LOG_VERBOSITY, &verbosity);
ssh_options_set(m_ssh_session, SSH_OPTIONS_PORT, &m_iPort);
int x = ssh_connect(m_ssh_session);
const char* x2 = ssh_get_error(m_ssh_session);
if(rc != SSH_AUTH_SUCCESS)
{
return false;
}
m_sftp_session;
m_sftp_session = sftp_new(m_ssh_session);
//Calling BeginDownload here does work
//BeginDownload(L"file.txt", L"T:\\file.txt");
... }
Here is what I'm using to call the exact same function via a thread
CWinThread* pThread2 = AfxBeginThread(BeginSFTPDownload, bundle);
And the method being ran via the above:
UINT CMainFrame::BeginSFTPDownload( LPVOID pParam)
{
pSft->BeginDownload(L"test.txt", L"T:\\test.txt");
return 0;
}
My program uses inotify for monitoring changes on files.
My code is as following:
fd = inotify_init();
wd = inotify_add_watch(fd, "./test.txt", IN_ALL_EVENTS);
len = read(fd, buff, BUFF_SIZE);
while (i < len) {
struct inotify_event *pevent = (struct inotify_event *) &buff[i];
//process events
i += sizeof(struct inotify_event) + pevent->len;
}
However, after I got several events for the first change in the monitored files (event IN_OPEN, IN_MODIFY, IN_ACESS, etc.), later changes in the monitored files does generate any other events ==> my program hang out at the read function (the read is blocked)
Could you guys help me explain this error.
please show what operations actually happen to "./test.txt" and more about your code (buff, BUFF_SIZE). And here is some point for you to check:
There are some special events like IN_IGNORED which may remove watch from the file
I suggest to use select/poll/epoll to monitor the fd rather than BLOCKING read on it.
Use fcntl to make the file descriptor non-blocking.