Read lines from a file opened with CreateFile [duplicate] - c++

In Unix, if you have a file descriptor (e.g. from a socket, pipe, or inherited from your parent process), you can open a buffered I/O FILE* stream on it with fdopen(3).
Is there an equivalent on Windows for HANDLEs? If you have a HANDLE that was inherited from your parent process (different from stdin, stdout, or stderr) or a pipe from CreatePipe, is it possible to get a buffered FILE* stream from it? MSDN does document _fdopen, but that works with integer file descriptors returned by _open, not generic HANDLEs.

Unfortunately, HANDLEs are completely different beasts from FILE*s and file descriptors. The CRT ultimately handles files in terms of HANDLEs and associates those HANDLEs to a file descriptor. Those file descriptors in turn backs the structure pointer by FILE*.
Fortunately, there is a section on this MSDN page that describes functions that "provide a way to change the representation of the file between a FILE structure, a file descriptor, and a Win32 file handle":
_fdopen, _wfdopen: Associates a stream with a file that was
previously opened for low-level I/O and returns a pointer to the open
stream.
_fileno: Gets the file descriptor associated with a stream.
_get_osfhandle: Return operating-system file handle associated
with existing C run-time file descriptor
_open_osfhandle: Associates C run-time file descriptor with an
existing operating-system file handle.
Looks like what you need is _open_osfhandle followed by _fdopen to obtain a FILE* from a HANDLE.
Here's an example involving HANDLEs obtained from CreateFile(). When I tested it, it shows the first 255 characters of the file "test.txt" and appends " --- Hello World! --- " at the end of the file:
#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <cstdio>
int main()
{
HANDLE h = CreateFile("test.txt", GENERIC_READ | GENERIC_WRITE, 0, 0,
OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if(h != INVALID_HANDLE_VALUE)
{
int fd = _open_osfhandle((intptr_t)h, _O_APPEND | _O_RDONLY);
if(fd != -1)
{
FILE* f = _fdopen(fd, "a+");
if(f != 0)
{
char rbuffer[256];
memset(rbuffer, 0, 256);
fread(rbuffer, 1, 255, f);
printf("read: %s\n", rbuffer);
fseek(f, 0, SEEK_CUR); // Switch from read to write
const char* wbuffer = " --- Hello World! --- \n";
fwrite(wbuffer, 1, strlen(wbuffer), f);
fclose(f); // Also calls _close()
}
else
{
_close(fd); // Also calls CloseHandle()
}
}
else
{
CloseHandle(h);
}
}
}
This should work for pipes as well.

Here is a more elegant way of doing this instead of CreateFile: specify "N" in fopen(). It's a Microsoft-specific extension to fopen, but since this code is platform-specific anyway, it's ok. When called with "N", fopen adds _O_NOINHERIT flag when calling _open internally.
Based on this:
Windows C Run-Time _close(fd) not closing file

Related

C++ redirect or disable stdio temporarily

In our C++ project under MacOS X, we are using stdio for interaction with clients. Yet, the dylib we are using is also using stdio to print "open log files". This breaks down all communication between our clients and the application. I looked at the examples of redirecting stdio to a file or temporarily disabling stdio. Yet, we could not succeed in it.
So, how can we disable or redirect stdio temporarily while interacting with the dylib.
void RedirectStandardStreamsToDEVNULL(int *_piOriginalSTDIN_FILENO, int *_piOriginalSTDOUT_FILENO, int *_piOriginalSTDERR_FILENO)
{
//flushing pending things before redirection.
//fflush(stdin);
fflush(stdout);
fflush(stderr);
*_piOriginalSTDIN_FILENO = dup(STDIN_FILENO);
*_piOriginalSTDOUT_FILENO = dup(STDOUT_FILENO);
*_piOriginalSTDERR_FILENO = dup(STDERR_FILENO);
int devnull = open("/dev/null", O_RDWR);
dup2(devnull, STDIN_FILENO);
dup2(devnull, STDOUT_FILENO);
dup2(devnull, STDERR_FILENO);
close(devnull);
}
void RestoreStandardStreams(int *_piOriginalSTDIN_FILENO, int *_piOriginalSTDOUT_FILENO, int *_piOriginalSTDERR_FILENO)
{
//flushing pending things before restoring.
//fflush(stdin);
fflush(stdout);
fflush(stderr);
dup2(*_piOriginalSTDIN_FILENO, STDIN_FILENO);
dup2(*_piOriginalSTDOUT_FILENO, STDOUT_FILENO);
dup2(*_piOriginalSTDERR_FILENO, STDERR_FILENO);
}
void myfunction()
{
int iOriginalSTDIN_FILENO = -1;
int iOriginalSTDOUT_FILENO = -1;
int iOriginalSTDERR_FILENO = -1;
RedirectStandardStreamsToDEVNULL(&iOriginalSTDIN_FILENO, &iOriginalSTDOUT_FILENO, &iOriginalSTDERR_FILENO);
//all of your code which prints to stdout or stderr will be directed to /dev/null
RestoreStandardStreams(&iOriginalSTDIN_FILENO, &iOriginalSTDOUT_FILENO, &iOriginalSTDERR_FILENO);
}
Important point is to identify the functions which are called inside dylib from your code. Now, surround such functions with Redirect and Restore functions described above.
OSX is a POSIX systems, and like all POSIX systems standard output is file descriptor STDOUT_FILENO (which is a macro defined as 1).
What you can do is duplicate STDOUT_FILENO to another file descriptor, open a temporary file and duplicating (using dup2) the temporary file as STDOUT_FILENO. Then whenever there is output to standard out (using plain write, C printf or C++ std::cout) it will be put in the temporary file.
When done with the temporary "redirection" you simply duplicate the saved standard output (from the first dup call) back into STDOUT_FILENO. and close and remove the temporary file.
Something like the following:
int saved_stdout = dup(STDOUT_FILENO);
int temp_file = open("/tmp/temp_stdout", O_WRONLY, 0600);
dup2(temp_file, STDOUT_FILENO); // Replace standard out
// Code here to write to standard output
// It should all end up in the file /tmp/temp_stdout
dup2(saved_stdout, STDOUT_FILENO); // Restore old standard out
close(temp_file)
unlink("/tmp/temp_stdout"); // Remove file

Open() syscall filedesriptor

I have to correct the return value of the open() syscall of a posix OS. I understood from the man-Pages that it has to return the file descriptor, and , in case of an error the system call will return -1 and set the errno value. The problem is that I do not know how to get the file descriptor for the opened nod. I checked all the files and didn't found a method that can assign a fd to processes.
Here is the method :
int syscalls::open(const char *path, int oflags, mode_t mode){
syscall_message msg;
msg.call.type = syscalls::open_call;
msg.open_data.path_name = &path[0];
msg.open_data.flags = oflags;
msg.open_data.create_mode = mode;
syscaller::call_system(msg);
return msg.error.number;
}
syscall_message is a struct that holds the data info for the system call. syscalls is the namesapace where all the system calls are. syscaller is used to send the call to the kernel, unsing the call_system method.
The call_system method:
syscalls::open_call:
{
//get the file
i_fs_node_ptr file = i_fs::open_node( msg.open_data.path_name );
//add the file handle
if ( file )
{
cur_process->push_filehandle(
file,
msg.open_data.flags,
msg.open_data.create_mode );
}
else
{
msg.error.type = syscalls::e_no_such_entry;
}
}
I don't know what you mean by "I can't get the file descriptor". As you have mentioned open() returns it. It's simply stored in an integer variable. If this variable is equal to -1, then something has gone wrong. For example if you have
int file = open(path, O_SYNC, O_DIRECT, O_RDONLY);
but you do not have the reading permissions for the file named path the variable file will get a value of -1. Additional manipulation on an opened file can be done via read() (if file was opened in read mode) and write() (if file was opened in write mode). I suggest you read the documentation on the open() function more carefully. If you need more control over the file descriptor I suggest you use fopen():
Discussion on the difference between open() and fopen()
Tutorial on fopen()
Documentation on fopen()

Flush not invoking file change

I am writing a program where one process A reads the data appended to file by another process B.I am using ReadDirectoryChangesW for the notification.The problem is that the notification is not being generated until I close the handle in B although I am flushing contents to file using fflush.The code is a given below
File Writer:
int _tmain(int argc, _TCHAR* argv[])
{
FILE *fp;
fp=_fsopen("log.txt", "a", _SH_DENYNO);
char str[4096];
for(int i=1;i<4096;i++)
str[i]=i;
while(true){
fwrite(str,1,4096,fp);
fflush(fp);
Sleep(2000);
}
return 0;
}
File Reader:
#include "stdafx.h"
#include <windows.h>
#include <stdlib.h>
#include <stdio.h>
#include <tchar.h>
#include <conio.h>
#include <assert.h>
#include <share.h>
void _tmain(int argc, TCHAR *argv[])
{
FILE *fp;
fp=_fsopen("C:\\Users\\dell\\Documents\\Visual Studio 2012\\Projects\\FileWriter\\FileWriter\\log.txt", "r", _SH_DENYNO);
int last_size=0,new_size=0;
if(fp==NULL)
return ;
HANDLE m_hMonitoredDir = CreateFile(TEXT("C:\\Users\\dell\\Documents\\Visual Studio 2012\\Projects\\FileWriter\\FileWriter"), FILE_LIST_DIRECTORY,
FILE_SHARE_DELETE|FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL, OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS, NULL );
if ( m_hMonitoredDir == INVALID_HANDLE_VALUE )
{
DWORD dwErr = GetLastError();
printf("error");
return;
}
char szBuf[ MAX_PATH ];
DWORD dwBytesRead = 0;
int flag=0;
char *buffer;
while ( ReadDirectoryChangesW( m_hMonitoredDir, szBuf, MAX_PATH, FALSE, FILE_NOTIFY_CHANGE_LAST_WRITE,&dwBytesRead, NULL, NULL ))
{
PFILE_NOTIFY_INFORMATION pstFileNotif = (PFILE_NOTIFY_INFORMATION)( szBuf );
if ( pstFileNotif->Action == FILE_ACTION_MODIFIED )
{
char szNotifFilename[ MAX_PATH ] = { 0 };
if ( int iNotifFilenameLen = WideCharToMultiByte( CP_OEMCP, NULL,
pstFileNotif->FileName,
pstFileNotif->FileNameLength / sizeof( WCHAR ),
szNotifFilename, sizeof( szNotifFilename ) / sizeof( char ),
NULL, NULL ) )
{
if ( strcmp("log.txt", szNotifFilename ) == 0 )
{
fseek(fp, 0, SEEK_END);
new_size = ftell(fp);
fseek(fp,last_size,SEEK_SET);
int size=new_size-last_size;
buffer=new char[size+1];
fread(buffer,1,size,fp);
buffer[size]='\0';
printf("%s",buffer);
free(buffer);
}
}
}
}
}
Can anyone help me get notifications as soon as I use fflush in B ?
I don't think this is possible. According to the documentation on FILE_NOTIFY_CHANGE_LAST_WRITE (emphasis mine):
Any change to the last write-time of files in the watched directory or subtree causes a change notification wait operation to return. The operating system detects a change to the last write-time only when the file is written to the disk. For operating systems that use extensive caching, detection occurs only when the cache is sufficiently flushed.
fflush() ensures that the file data is passed back to the operating system, but it does not guarantee that the data gets written to the disk, since typically a lot of caching is involved:
Buffers are normally maintained by the operating system, which determines the optimal time to write the data automatically to disk: when a buffer is full, when a stream is closed, or when a program terminates normally without closing the stream. The commit-to-disk feature of the run-time library lets you ensure that critical data is written directly to disk rather than to the operating-system buffers.
As others have said in the comments, you may be better of using named pipes for your goals, since you're only dealing with a single known file.
You can force a commit-to-disk by calling _flushall ( http://msdn.microsoft.com/en-us/library/s9xk9ehd.aspx )
or see this article ( http://support.microsoft.com/kb/148505 ) on how to force a commit-to-disk. You need to link with commode.obj to force fflush to commit-to-disk automatically.
The alternative might be to fclose the file each time, and reopen the file in append mode, if you are only doing it every 2 seconds (the overhead is small).
See here: https://jeffpar.github.io/kbarchive/kb/066/Q66052/
For us, linking with commode.obj didn't work.
However, this approach did:
When we opened our file using fopen, we included the "c" mode option as the LAST OPTION:
fopen( path, "wc") // w - write mode, c - allow immediate commit to disk
Then when you want to force a flush to disk, call
_flushall()
We made this call before calling
fclose()
We experienced the exact issue you described and this approach fixed it.
From that above site:
"Microsoft C/C++ version 7.0 introduces the "c" mode option for the fopen()
function. When an application opens a file and specifies the "c" mode, the
run-time library writes the contents of the file buffer to disk when the
application calls the fflush() or _flushall() function. "

How do I CloseHandle a handle that was converted to a FILE*?

I am trying to redirect the output of a child process and return a FILE* to the parent.
I am essentially following the code example at Creating a Child Process with Redirected Input and Output. The only modification is that i encapsulated the whole thing in a function and do
return _fdopen(_open_osfhandle((intptr_t)g_hChildStd_OUT_Rd, _O_RDONLY), "rb");
The problem is that I want to cleanup and fclose is throwing an error. Was I supposed to close handle after I did the _fdopen? How am I supposed to clean up here?
The correct way to close a file stream opened with _fdopen is to call fclose. If that's not working for you, then you have other problems.
Do not close the file descriptor after calling _fdopen. The documentation doesn't exactly say, but the FILE object owns that file descriptor and will close it when you call fclose. Likewise, the file descriptor owns the file handle, so you shouldn't call CloseHandle, either.
May be quite late to answer, but just adding for reference. I looked into call stack of fclose, and it is actually making a call to CloseHandle
auto NThdl = CreateFileW(L"D:\\s.txt", GENERIC_READ | GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);
if (NThdl != INVALID_HANDLE_VALUE)
{
int iohdl = _open_osfhandle((intptr_t) NThdl, O_RDWR | O_BINARY);
if (iohdl != 0xFFFFFFFF)
{
FILE *hLstFile = _wfdopen(iohdl, L"w+bc");
fwrite("Bogus", 5, 1, hLstFile);
fclose(hLstFile);
// Exception
CloseHandle(NThdl);
}
}
And attempt to call CloseHandle after fclose would cause an exception saying that Handle is invalid.

fork() and pipes() in c

What is fork and what is pipe?
Any scenarios explaining why their use is necessary will be appreciated.
What are the differences between fork and pipe in C?
Can we use them in C++?
I need to know this is because I want to implement a program in C++ which can access live video input, convert its format and write it to a file.
What would be the best approach for this?
I have used x264 for this. So far I have implemented the part of conversion on a file format.
Now I have to implement it on a live stream.
Is it a good idea to use pipes? Capture video in another process and feed it to the other?
A pipe is a mechanism for interprocess communication. Data written to the pipe by one process can be read by another process. The primitive for creating a pipe is the pipe function. This creates both the reading and writing ends of the pipe. It is not very useful for a single process to use a pipe to talk to itself. In typical use, a process creates a pipe just before it forks one or more child processes. The pipe is then used for communication either between the parent or child processes, or between two sibling processes. A familiar example of this kind of communication can be seen in all operating system shells. When you type a command at the shell, it will spawn the executable represented by that command with a call to fork. A pipe is opened to the new child process and its output is read and printed by the shell. This page has a full example of the fork and pipe functions. For your convenience, the code is reproduced below:
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
/* Read characters from the pipe and echo them to stdout. */
void
read_from_pipe (int file)
{
FILE *stream;
int c;
stream = fdopen (file, "r");
while ((c = fgetc (stream)) != EOF)
putchar (c);
fclose (stream);
}
/* Write some random text to the pipe. */
void
write_to_pipe (int file)
{
FILE *stream;
stream = fdopen (file, "w");
fprintf (stream, "hello, world!\n");
fprintf (stream, "goodbye, world!\n");
fclose (stream);
}
int
main (void)
{
pid_t pid;
int mypipe[2];
/* Create the pipe. */
if (pipe (mypipe))
{
fprintf (stderr, "Pipe failed.\n");
return EXIT_FAILURE;
}
/* Create the child process. */
pid = fork ();
if (pid == (pid_t) 0)
{
/* This is the child process.
Close other end first. */
close (mypipe[1]);
read_from_pipe (mypipe[0]);
return EXIT_SUCCESS;
}
else if (pid < (pid_t) 0)
{
/* The fork failed. */
fprintf (stderr, "Fork failed.\n");
return EXIT_FAILURE;
}
else
{
/* This is the parent process.
Close other end first. */
close (mypipe[0]);
write_to_pipe (mypipe[1]);
return EXIT_SUCCESS;
}
}
Just like other C functions you can use both fork and pipe in C++.
there are stdin and stdout for common input and output.
A common style is like this:
input->process->output
But with pipe, it becomes:
input->process1->(tmp_output)->(tmp-input)->process2->output
pipe is the function that returns the two temporary tmp-input and tmp-output, i.e. fd[0] and fd[1].