Flush not invoking file change - c++

I am writing a program where one process A reads the data appended to file by another process B.I am using ReadDirectoryChangesW for the notification.The problem is that the notification is not being generated until I close the handle in B although I am flushing contents to file using fflush.The code is a given below
File Writer:
int _tmain(int argc, _TCHAR* argv[])
{
FILE *fp;
fp=_fsopen("log.txt", "a", _SH_DENYNO);
char str[4096];
for(int i=1;i<4096;i++)
str[i]=i;
while(true){
fwrite(str,1,4096,fp);
fflush(fp);
Sleep(2000);
}
return 0;
}
File Reader:
#include "stdafx.h"
#include <windows.h>
#include <stdlib.h>
#include <stdio.h>
#include <tchar.h>
#include <conio.h>
#include <assert.h>
#include <share.h>
void _tmain(int argc, TCHAR *argv[])
{
FILE *fp;
fp=_fsopen("C:\\Users\\dell\\Documents\\Visual Studio 2012\\Projects\\FileWriter\\FileWriter\\log.txt", "r", _SH_DENYNO);
int last_size=0,new_size=0;
if(fp==NULL)
return ;
HANDLE m_hMonitoredDir = CreateFile(TEXT("C:\\Users\\dell\\Documents\\Visual Studio 2012\\Projects\\FileWriter\\FileWriter"), FILE_LIST_DIRECTORY,
FILE_SHARE_DELETE|FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL, OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS, NULL );
if ( m_hMonitoredDir == INVALID_HANDLE_VALUE )
{
DWORD dwErr = GetLastError();
printf("error");
return;
}
char szBuf[ MAX_PATH ];
DWORD dwBytesRead = 0;
int flag=0;
char *buffer;
while ( ReadDirectoryChangesW( m_hMonitoredDir, szBuf, MAX_PATH, FALSE, FILE_NOTIFY_CHANGE_LAST_WRITE,&dwBytesRead, NULL, NULL ))
{
PFILE_NOTIFY_INFORMATION pstFileNotif = (PFILE_NOTIFY_INFORMATION)( szBuf );
if ( pstFileNotif->Action == FILE_ACTION_MODIFIED )
{
char szNotifFilename[ MAX_PATH ] = { 0 };
if ( int iNotifFilenameLen = WideCharToMultiByte( CP_OEMCP, NULL,
pstFileNotif->FileName,
pstFileNotif->FileNameLength / sizeof( WCHAR ),
szNotifFilename, sizeof( szNotifFilename ) / sizeof( char ),
NULL, NULL ) )
{
if ( strcmp("log.txt", szNotifFilename ) == 0 )
{
fseek(fp, 0, SEEK_END);
new_size = ftell(fp);
fseek(fp,last_size,SEEK_SET);
int size=new_size-last_size;
buffer=new char[size+1];
fread(buffer,1,size,fp);
buffer[size]='\0';
printf("%s",buffer);
free(buffer);
}
}
}
}
}
Can anyone help me get notifications as soon as I use fflush in B ?

I don't think this is possible. According to the documentation on FILE_NOTIFY_CHANGE_LAST_WRITE (emphasis mine):
Any change to the last write-time of files in the watched directory or subtree causes a change notification wait operation to return. The operating system detects a change to the last write-time only when the file is written to the disk. For operating systems that use extensive caching, detection occurs only when the cache is sufficiently flushed.
fflush() ensures that the file data is passed back to the operating system, but it does not guarantee that the data gets written to the disk, since typically a lot of caching is involved:
Buffers are normally maintained by the operating system, which determines the optimal time to write the data automatically to disk: when a buffer is full, when a stream is closed, or when a program terminates normally without closing the stream. The commit-to-disk feature of the run-time library lets you ensure that critical data is written directly to disk rather than to the operating-system buffers.
As others have said in the comments, you may be better of using named pipes for your goals, since you're only dealing with a single known file.

You can force a commit-to-disk by calling _flushall ( http://msdn.microsoft.com/en-us/library/s9xk9ehd.aspx )
or see this article ( http://support.microsoft.com/kb/148505 ) on how to force a commit-to-disk. You need to link with commode.obj to force fflush to commit-to-disk automatically.
The alternative might be to fclose the file each time, and reopen the file in append mode, if you are only doing it every 2 seconds (the overhead is small).

See here: https://jeffpar.github.io/kbarchive/kb/066/Q66052/
For us, linking with commode.obj didn't work.
However, this approach did:
When we opened our file using fopen, we included the "c" mode option as the LAST OPTION:
fopen( path, "wc") // w - write mode, c - allow immediate commit to disk
Then when you want to force a flush to disk, call
_flushall()
We made this call before calling
fclose()
We experienced the exact issue you described and this approach fixed it.
From that above site:
"Microsoft C/C++ version 7.0 introduces the "c" mode option for the fopen()
function. When an application opens a file and specifies the "c" mode, the
run-time library writes the contents of the file buffer to disk when the
application calls the fflush() or _flushall() function. "

Related

Change in temperature file not detected

I'm writing a C++ program to run on my raspberry pi 3b+ that monitors the temperature of the CPU in real-time.In order to avoid polling, I'm using the sys/inotify library to watch /sys/class/thermal/thermal_zone0/temp for file updates.
However, this doesn't seem to pick up changes to the file.
To test this:
I polled the file repeatedly (using cat) while running main, and I saw that the value in the file did change, but the change was not detected by main.
I tried tail -f /sys/class/thermal/thermal_zone0/temp, but this did not detect any changes either.
I created a script to periodically write to another file, and had my program watch this file, and it detected changes there.
Is it possible that this file is being updated without propagating an event that is detectable by inotify? I am trying to avoid having to implement a periodic polling of this file to monitor for changes at all cost.
temperatureMonitor.cpp
#include <iostream>
#include <unistd.h>
#include <sys/inotify.h>
#include <sys/types.h>
#include "./temperatureMonitor.hpp"
#define EVENT_SIZE ( sizeof (struct inotify_event) )
#define BUF_LEN ( 1024 * ( EVENT_SIZE + 16 ) )
namespace Performance {
void TemperatureMonitor::monitor_temperature(void(*callback)(double)){
};
void TemperatureMonitor::monitor_temperature_file(){
int length, i = 0;
int fd;
int wd;
char buffer[BUF_LEN];
fd = inotify_init();
wd = inotify_add_watch(fd,"/sys/class/thermal/thermal_zone0/temp" , IN_MODIFY|IN_CREATE|IN_DELETE);
while (true){
length = read(fd, buffer, BUF_LEN);
std::cout << "detected file change\n";
};
};
};
main.cpp
#include "Performance/temperatureMonitor.hpp"
#define EVENT_SIZE ( sizeof (struct inotify_event) )
#define BUF_LEN ( 1024 ∗ ( EVENT_SIZE + 16 ) )
int main(){
Performance::TemperatureMonitor tm = Performance::TemperatureMonitor();
tm.monitor_temperature_file();
return 0;
};
fwriter.go
package main
import (
"os"
"time"
)
func main() {
f, err := os.Create("foo.txt")
if err != nil {
panic(err)
}
for {
f.Write([]byte("test\n"))
time.Sleep(time.Second)
}
}
It's not a real file, whenever you read it, the driver is asked to produce the data in it. There is no way to get notified when its contents would change. Polling is the answer.
https://www.kernel.org/doc/html/latest/filesystems/sysfs.html
On read(2), the show() method should fill the entire buffer. Recall
that an attribute should only be exporting one value, or an array of
similar values, so this shouldn’t be that expensive.
This allows userspace to do partial reads and forward seeks
arbitrarily over the entire file at will. If userspace seeks back to
zero or does a pread(2) with an offset of ‘0’ the show() method will
be called again, rearmed, to fill the buffer.
You don't have to open and close it every time, seeking to the beginning should refresh it. Although this will probably not save much.

Processing audio file in memory with lib sox

I am trying to process audio file in memory with SOX C++ API and I stuck at the very beginning. The goal is to load an audio file from disk, apply few effects (tempo/gain adjustments) in memory. Here is the code I started with, but I receive a strange error when creating out stream:
formats: can't open output file `': No such file or directory
What could be an issue here? I am testing it on Mac. Here is the code:
#include <iostream>
#include <sox.h>
#include <assert.h>
int main() {
sox_format_t * in, *out;
sox_effect_t * e;
sox_init();
in = sox_open_read("/path/to/file.wav", NULL, NULL, NULL);
sox_format_t *out_format = (sox_format_t *)malloc(sizeof(sox_format_t));
memcpy(out_format, in, sizeof(sox_format_t));
char * buffer;
size_t buffer_size;
out = sox_open_memstream_write(&buffer, &buffer_size, &in->signal, NULL, "sox", NULL);
//chain = sox_create_effects_chain(&in->encoding, &out->encoding);
//e = sox_create_effect(sox_find_effect("input"));
return 0;
}
sox_open_memstream_write() uses either fmemopen() or open_memstream() depending on the parameters you pass.
Some (or all) versions of OSX do not have these functions.
The same is true for Windows.
You can find the relevant code in file src/formats.c, function open_write(), look for the #ifdef HAVE_FMEMOPEN conditionals.

Read lines from a file opened with CreateFile [duplicate]

In Unix, if you have a file descriptor (e.g. from a socket, pipe, or inherited from your parent process), you can open a buffered I/O FILE* stream on it with fdopen(3).
Is there an equivalent on Windows for HANDLEs? If you have a HANDLE that was inherited from your parent process (different from stdin, stdout, or stderr) or a pipe from CreatePipe, is it possible to get a buffered FILE* stream from it? MSDN does document _fdopen, but that works with integer file descriptors returned by _open, not generic HANDLEs.
Unfortunately, HANDLEs are completely different beasts from FILE*s and file descriptors. The CRT ultimately handles files in terms of HANDLEs and associates those HANDLEs to a file descriptor. Those file descriptors in turn backs the structure pointer by FILE*.
Fortunately, there is a section on this MSDN page that describes functions that "provide a way to change the representation of the file between a FILE structure, a file descriptor, and a Win32 file handle":
_fdopen, _wfdopen: Associates a stream with a file that was
previously opened for low-level I/O and returns a pointer to the open
stream.
_fileno: Gets the file descriptor associated with a stream.
_get_osfhandle: Return operating-system file handle associated
with existing C run-time file descriptor
_open_osfhandle: Associates C run-time file descriptor with an
existing operating-system file handle.
Looks like what you need is _open_osfhandle followed by _fdopen to obtain a FILE* from a HANDLE.
Here's an example involving HANDLEs obtained from CreateFile(). When I tested it, it shows the first 255 characters of the file "test.txt" and appends " --- Hello World! --- " at the end of the file:
#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <cstdio>
int main()
{
HANDLE h = CreateFile("test.txt", GENERIC_READ | GENERIC_WRITE, 0, 0,
OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if(h != INVALID_HANDLE_VALUE)
{
int fd = _open_osfhandle((intptr_t)h, _O_APPEND | _O_RDONLY);
if(fd != -1)
{
FILE* f = _fdopen(fd, "a+");
if(f != 0)
{
char rbuffer[256];
memset(rbuffer, 0, 256);
fread(rbuffer, 1, 255, f);
printf("read: %s\n", rbuffer);
fseek(f, 0, SEEK_CUR); // Switch from read to write
const char* wbuffer = " --- Hello World! --- \n";
fwrite(wbuffer, 1, strlen(wbuffer), f);
fclose(f); // Also calls _close()
}
else
{
_close(fd); // Also calls CloseHandle()
}
}
else
{
CloseHandle(h);
}
}
}
This should work for pipes as well.
Here is a more elegant way of doing this instead of CreateFile: specify "N" in fopen(). It's a Microsoft-specific extension to fopen, but since this code is platform-specific anyway, it's ok. When called with "N", fopen adds _O_NOINHERIT flag when calling _open internally.
Based on this:
Windows C Run-Time _close(fd) not closing file

Communication with Arduino using Linux

This is the first time I'm communicating with Arduino using my computer. I use Ubuntu 14.04. This is the C program for writing to the file. The Arduino shows up ttyACM0.
While compiling using gcc the compiler shows an error saying:
Segmentation fault(core dumped)
How do I rectify this error.
#include<unistd.h>
#include<stdio.h>
int main() {
char data[] = {'f','b','r'}; //Random data we want to send
FILE *file;
file = fopen("/dev/ttyACM0","w"); //Opening device file
int i = 0;
for(i = 0 ; i < 3 ; i++) {
fprintf(file,"%c",data[i]); //Writing to the file
fprintf(file,"%c",','); //To separate digits
sleep(1);
}
fclose(file);
}
Pardon my ignorance. I tried researching on it. Couldn't make it work. Thanks in advance for your help.
You're getting a NULL return from the fopen() that NULL is being passed to fprintf() which is expecting a valid FILE* and messing up causing the SEGV.
If you use fopen you should check what it returns so you can give the user a something more useful than "segmentation fault".
The probable cause of the fopen() failure is you don't have permission to play with the serial port.
Normally you need the group dialout to be able to access the serial port.
As root do:
usermod -a -G dialoutyourusername
Then log out and back in so you get the new group.
Consider using minicom or microcom (on any of the several other serial terminal programs) to access the serial port instead of writing your own.
I also suggest you have the Arduino send a hello message when it boots up so you can be sure you have the right baud rate etc...
You did not put any success check on the return value of fopen("/dev/ttyACM0","w");. In case fopen() fails, using file further is undefined behavior, causing segmentation fault. Do something like
file = fopen("/dev/ttyACM0","w"); //Opening device file
if (file)
{
//do something with file
}
else
return 0;
Also, add a return 0 before ending main().
// the following code:
// compiles cleanly
// performs appropriate error checking
// has proper return statement
#include <unistd.h> // sleep()
#include <stdio.h> // fopen(), fclose(), fprintf(), perror()
#include <stdlib.h> // exit() and EXIT_FAILURE
int main()
{
char data[] = {'f','b','r'}; //Random data we want to send
FILE *file;
if( NULL == (file = fopen("/dev/ttyACM0","w") ) ) //Opening device file
{ // then fopen failed
perror("fopen failed for ttyACM0" );
exit( EXIT_FAILURE );
}
// implied else, fopen successful
int i = 0;
for(i = 0 ; i < 3 ; i++)
{
if( 0 >= fprintf(file,"%c",data[i]) ) //Writing to the file
{ // fprintf failed
perror("fprintf data failed" );
exit( EXIT_FAILURE );
}
// implied else, fprintf successful for data
if( 0 >= fprintf(file,"%c",',') ) //To separate digits
{ // then, fprintf failed
perror( "fprintf for comma failed");
exit( EXIT_FAILURE );
}
// implied else, fprintf successful for comma
sleep(1);
} // end for
fclose(file);
return(0);
} // end function: main
On failure fopen returns NULL, so you are potentially dereferencing a NULL pointer, the correct way of doing that, is checking the result of fopen. I would however suggest low level IO for this kind of thing something like
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
int main()
{
char data[] = {'f','b','r'}; //Random data we want to send
int fd;
int i;
fd = open("/dev/ttyACM0", O_WRONLY); //Opening device file
if (fd == -1)
{
perror("cannot open /dev/ttyACM0");
return -1;
}
for(i = 0 ; i < 3 ; i++)
{
write(fd, &(data[i]), 1);
write(fd, ",", 1);
sleep(1);
}
close(fd);
return 0;
}
on error open returns a special value -1 so you should abort writing to it.
I'm pretty sure in your case there will be a permission denied error, since normally the /dev/tty* belong to group dialout and they have group write permission by default, but since probably your user doesn't belong to that group you don't have write access to /dev/ttyACM0.

Watching a file for changes in Windows 7

I have a Visual Studio 2008 C++ application for Windows 7 where I would like to watch a file for changes.
The file may be changed like this:
std::ofstream myfile_;
void LogData( const char* data )
{
myfile_ << data << std::endl;
// note that the file output buffer is flushed by std::endl, but the file is not closed.
}
I have tried watching the file's directory using both ReadDirectoryChangesW and FindFirstChangeNotification with FILE_NOTIFY_CHANGE_SIZE | FILE_NOTIFY_CHANGE_LAST_WRITE | FILE_NOTIFY_CHANGE_LAST_ACCESS | FILE_NOTIFY_CHANGE_SECURITY | FILE_NOTIFY_CHANGE_CREATION | FILE_NOTIFY_CHANGE_FILE_NAME flags. But, neither of those APIs will detect file changes until the file handle is actually closed.
Is there any way to detect a change when the file is actually written, but before the file handle is closed?
Thanks,
PaulH
Update
On #Edwin's suggestion, I'm attempting to use the Journal feature. But, I'm having a couple issues.
FSCTL_READ_USN_JOURNAL returns instantly. It does not block. (though, this may be related to issue 2)
Regardless of where my handle points to (I have tried opening a handle to the directory "C:\Foo\Bar" and to the file "C:\Foo\Bar\MyFile.txt") I seem to get any changes made to the the C: volume. Is there a way to limit what FSCTL_READ_USN_JOURNAL gives me?
Error checking omitted for brevity.
boost::shared_ptr< void > directory(
::CreateFileW( L"C:\\Foo\\Bar\\Myfile.txt",
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL ),
::CloseHandle );
USN_JOURNAL_DATA journal = { 0 };
DWORD returned = 0;
::DeviceIoControl( directory.get(), FSCTL_QUERY_USN_JOURNAL, NULL, 0, &journal, sizeof( journal ), &returned, NULL );
BYTE buffer[ 4096 ] = { 0 };
READ_USN_JOURNAL_DATA read = { 0, USN_REASON_DATA_EXTEND | USN_REASON_DATA_TRUNCATION, FALSE, 0, 0, journal.UsnJournalID };
::DeviceIoControl( directory.get(), FSCTL_READ_USN_JOURNAL, &read, sizeof( read ), &buffer, sizeof( buffer ), &returned, NULL );
for( USN_RECORD* record = ( USN_RECORD* )( buffer + sizeof( USN ) );
( ( BYTE* )record - buffer ) < returned;
record = ( USN_RECORD* )( ( BYTE* )record + record->RecordLength ) )
{
ATLTRACE( L"%s\r\n", record->FileName );
}
Example output (none of these are in the C:\Foo\Bar directory):
AeXProcessList.txt`
AeXProcessList.txt`
AeXAMInventory.txt`
AeXAMInventory.txt`
AeXProcessList.txt`
AeXProcessList.txtP
access.log`
mysqlgeneral.log
E804.tmp
apache_error.log
E804.tmp
CHROME.EXE-5FE9909D.pfh
CHROME.EXE-5FE9909D.pfp
SyncData.sqlite3-journal
CHROME.EXE-5FE9909D.pfh
CHROME.EXE-5FE9909D.pfP
1211.tmp
SyncData.sqlite3-journal
AeXAMInventory.txt
You can use
Change Journal Operations
(see MSDN docs)
That's the only 100% garanteed way to detect any change in the filesystem.
But it's pretty complicated.
To read data for a specific file or directory, I believe you want to use FSCTL_READ_FILE_USN_DATA instead of FSCTL_READ_USN_JOURNAL. I believe the latter always retrieves data for an entire volume. That does not, however, fill in the TimeStamp, Reason, or SourceInfo fields of the USN record you get. If you need those, I believe you can read them with FSCTL_READ_USN_JOURNAL, specifying the exact USN you want to read.
No, because until you close the file handle there is no guarantee a single byte ever gets written by the OS.
The exception would probably be by calling flush on your file handle and then call the Windows API function FlushFileBuffers, but unless the program writing into the file does this no bytes probably get written.
This can be done with a filter driver that monitors the FASTIO_WRITE and IRP_MJ_WRITE operations. Here is a pretty good how-to article.