My C++ program calls another program via system(). That program writes out to a file. I can't modify the called program.
My program then needs to read in the file that was written. At the present, my program reads in the file so quickly that the changes by the other program haven't yet hit the file system, and my program reads in an outdated version.
How can I force the called program to finish writing before my program tries to read? Or perhaps how can the calling program check that the file is finished being written?
I switched to system() from fork/execv and CreateProcess specifically so the program would block and I wouldn't have to watch for termination. So, I could switch back if that would help.
Right now, my only idea would be to delete the file before calling system() -- that way it wouldn't exist unless writing had completed.
Any solution needs to have versions for Mac, Windows, and Linux.
you should use wait() method to implement this. wait method make the parent process wait and sleeps it until its child process finishes. you should use fork() and then exec the another copy of process generated.
As far as I know if you try to open the file not in reading mode but in writing mode it will fail if the file is already being written by another process.
I would suggest to have a small (a few milliseconds) wait time to make sure the file was opened by the other process and then try to write on it and when you can actually write on the file then you can read it.
I'm not sure how things work on Mac but I assume it would be the same as Linux and Windows here.
It would be much better to use fork and exec instead of system so my suggestion is just a simple way (but not recommended for an important project) to go around the problem.
Related
I'm developing an interactive program that takes inputs from the user (through stdin) and outputs data accordingly (using stdout). While the program is running I need to open a background application (and leave it running while the main program is running). I have successfully done this using popen (using the "r" mode), however at times I get stdin conflicts. For example when the user enters an input intended for the main program, sometimes, the program treats it as an input to the background program. The user should never have to interact directly with the background program. Is there a way to completely decouple both the input and the output of the background program from stdin and stdout, while still being able to read and write using a file descriptor to the background process?
Just code exactly what you need. The popen function is a convenience function that you can use what it happens to do exactly what you happen to need. Otherwise, use pipe, fork, dup2, close, and whatever exec-family function you want.
You may find it helpful to look at a few implementations of popen/pclose to see how they work. Adjust as needed.
Simply don't use popen. Instead, use fork + one of the exec family of functions, which doesn't connect the io streams.
i want to do a simple program that will run another program (easy task in C++), hook the process and, when the program is closed, run another program (needed to sync a file modified by the first program).
The second program prompt out a popout asking a simple Yes/No buttons.
There is any way?
I am writing an article about hooking : http://ntvalk.blogspot.nl/2013/11/hooking-explained-detouring-library.html
It describes the probably most common methods for hooking under Windows. (jmp/vtable/etc)
gldraphael already described a method of running another program.
Check this out: http://faq.cprogramming.com/cgi-bin/smartfaq.cgi?answer=1044654269&id=1043284392
It lists the various ways you can run another program.
How can I check if a file is still being written? I need to wait for a file to be created, written and closed again by another process, so I can go on and open it again in my process.
In general, this is a difficult problem to solve. You can ask whether a file is open, under certain circumstances; however, if the other process is a script, it might well open and close the file multiple times. I would strongly recommend you use an advisory lock, or some other explicit method for the other process to communicate when it's done with the file.
That said, if that's not an option, there is another way. If you look in the /proc/<pid>/fd directories, where <pid> is the numeric process ID of some running process, you'll see a bunch of symlinks to the files that process has open. The permissions on the symlink reflect the mode the file was opened for - write permission means it was opened for write mode.
So, if you want to know if a file is open, just scan over every process's /proc entry, and every file descriptor in it, looking for a writable symlink to your file. If you know the PID of the other process, you can directly look at its proc entry, as well.
This has some major downsides, of course. First, you can only see open files for your own processes, unless you're root. It's also relatively slow, and only works on Linux. And again, if the other process opens and closes the file several times, you're stuck - you might end up seeing it during the closed period, and there's no easy way of knowing if it'll open it again.
You could let the writing process write a sentinel file (say "sentinel.ok") after it is finished writing the data file your reading process is interested in. In the reading process you can check for the existence of the sentinel before reading the data file, to ensure that the data file is completely written.
#blu3bird's idea of using a sentinel file isn't bad, but it requires modifying the program that's writing the file.
Here's another possibility that also requires modifying the writer, but it may be more robust:
Write to a temporary file, say "foo.dat.part". When writing is complete, rename "foo.dat.part" to "foo.dat". That way a reader either won't see "foo.dat" at all, or will see a complete version of it.
You can try using inotify
http://en.wikipedia.org/wiki/Inotify
If you know that the file will be opened once, written and then closed, it would be possible for your app to wait for the IN_CLOSE_WRITE event.
However if the behaviour of the other application doing the writing of the file is more like open,write,close,open,write,close....then you'll need some other mechanism of determining when the other app has truly finished with the file.
I have a c++ program (very complicated, and lengthy both in code and execution time).
Once in a while this program stops and calls a user-specified shell script.
Before calling the script, my program creates a .out file with current data. I call the script via system() command. The script then reads the .out file, and creates its own script.out file and exits.
Then the system() function call ends, and my program reads and parses the script.out file.
Question: is there a better way to execute communication between my c++ program and a random shell script?
My intent is to have full communication between the two. Script could virtually "ask" the program "What data do you have right now?" and the program would reply with some strict convention. Then the script could say "Add this data...", or "delete all your previous data" etc.etc.
The reason I need this is because the shell script tells the program to modify its data. The exact data that was put in the original .out file. So after the modification is done -- the actual data held by the program does not correspond to the data written in the .out file.
Thanks!
P.S.
I swear I've searched around, but everyone suggests an intermediate file.
There are certainly ways to do that without intermediate files. The most common approach is to use command line arguments for input, and pipes for standard output; others also use pipes for input. The most straight-forward alternative to system then is to use popen.
On a unix-like system? Perhaps pipe (2) will work for you?
From the man page (Mac OS X 10.5 version):
SYNOPSIS
#include <unistd.h>
int pipe(int fildes[2]);
DESCRIPTION
The pipe() function creates a pipe (an object that allows unidirectional
data flow) and allocates a pair of file descriptors. The first descrip-
tor connects to the read end of the pipe; the second connects to the
write end.
You will, of course, have to follow the creation of the pipes with a fork and exec pair. Probably this has already been answered in detail, and now you know what to search on...
It's been a while since I did this, but:
In the main process, before forking the sub-process you call pipe twice. Now you have two pipes and control both ends of both of them.
You fork.
The main process will read from one pipe and write from the other. It doesn't matter which is which, but you need to be clear about this.
The child process will call one of the exec family of function to replace it's image with that of the shell you want to run but first you will use dup2 to replace it's standard input and output with the ends of the two pipes (again, this is where you need to be clear about which pipe is which).
At his point you have two processes, the main process can send things into one pipe ad they will be received on the standard input of the script, and anything the script writes to it's standard output will be sent up the other pipe to the controlling process. So they take turns, just like interacting with the shell.
You can use pipes or (maybe more convenient) sockets - for example frontends to gdb, or expect do that. It would require changes to your shell scripts, and switching from system() to more low-level fork() and exec().
It's rather complicated so please, be more specific about your environment and what you need to clarify.
You are asking the question on Interprocess Communication (IPC).
There are a lot of ways to do that. You can do a simply search and Internet will return you most answers.
If I am not wrong, Google chrome uses a technique called Named Pipe.
Anyway, I think the most "portable way" is probably a file. But if you know you are working on which operating system, you can definitely use most of the IPC techniques.
I have a project that I thought was going to be relatively easy, but is turning out to be more of a pain that I had hoped. First, most of the code I'm interacting with is legacy code that I don't have control over, so I can't do big paradigm changes.
Here's a simplified explanation of what I need to do: Say I have a large number of simple programs that read from stdin and write to stdout. (These I can't touch). Basically, input to stdin is a command like "Set temperature to 100" or something like that. And the output is an event "Temperature has been set to 100" or "Temperature has fallen below setpoint".
What I'd like to do is write an application that can start a bunch of these simple programs, watch for events and then send commands to them as necessary. My initial plan was to something like popen, but I need a bidrectional popen to get both read and write pipes. I hacked something together that I call popen2 where I pass it the command to run and two FILE* that get filled with the read and write stream. Then all I need to do is write a simple loop that reads from each of the stdouts from each of the processes, does the logic that it needs and then writes commands back to the proper process.
Here's some pseudocode
FILE *p1read, *p1write;
FILE *p2read, *p2write;
FILE *p3read, *p3write;
//start each command, attach to stdin and stdout
popen2("process1",&p1read,&p1write);
popen2("process2",&p2read,&p2write);
popen2("process3",&p3read,&p3write);
while (1)
{
//read status from each process
char status1[1024];
char status2[1024];
char status3[1024];
fread(status1,1024,p1read);
fread(status2,1024,p2read);
fread(status3,1024,p3read);
char command1[1024];
char command2[1024];
char command3[1024];
//do some logic here
//write command back to each process
fwrite(command1,p1write);
fwrite(command2,p2write);
fwrite(command3,p3write);
}
The real program is more complicated where it peeks in the stream to see if anything is waiting, if not, it will skip that process, likewise if it doesn't need to send a command to a certain process it doesn't. But this code gives the basic idea.
Now this works great on my UNIX box and even pretty good on a Windows XP box with cygwin. However, now I need to get it to work on Win32 natively.
The hard part is that my popen2 uses fork() and execl() to start the process and assign the streams to stdin and stdout of the child processes. Is there a clean way I can do this in windows? Basically, I'd like to create a popen2 that works in windows the same way as my unix version. This way the only windows specific code would be in that function and I could get away with everything else working the same way.
Any Ideas?
Thanks!
On Windows, you invoke CreatePipe first (similar to pipe(2)), then CreateProcess. The trick here is that CreateProcess has a parameter where you can pass stdin, stdout, stderr of the newly-created process.
Notice that when you use stdio, you need to do fdopen to create the file object afterwards, which expects file numbers. In the Microsoft CRT, file numbers are different from OS file handles. So to return the other end of CreatePipe to the caller, you first need _open_osfhandle to get a CRT file number, and then fdopen on that.
If you want to see working code, check out _PyPopen in
http://svn.python.org/view/python/trunk/Modules/posixmodule.c?view=markup
I think you've made a very good start to your problem by using the popen2() function to abstract away the cross-platform issues. I was expecting to come and suggest 'sockets', but I'm sure that's not relevant after reading the question. You could use sockets instead of pipes - it would be hidden in the popen2() function.
I am 99% sure you can implement the required functionality on Windows, using Windows APIs. What I cannot do is point you to the right functions reliably. However, you should be aware the Microsoft has most of the POSIX-like API calls available, but the name is prefixed with '_'. There are also native API calls that achieve the effects of fork and exec.
Your comments suggest that you are aware of issues with availability of data and possible deadlocks - be cautious.