A program generates a text file after every 15 iterations. It overwrites the output.txt (formed at 15th step) with a new output.txt (formed at 30th step), due to using the same name. I can't modify the file name within the program. Can I run some script concurrently with the program on my Ubuntu system that monitors my directory and moves the output.txt file to a desired directory when it is formed or changes the output file name?
I can't modify the file name within the program.
(I take this to mean you are required to not change the file name, not that you don't know how.)
You've marked this posting as C++.
While it is possible to run some script to monitor a directory, coordinating the name change and running a thread or another process (from C++) can be much more challenging than other choices.
How about a simpler approach:
I suggest using std::stringstream to generate a unique pfn (path-file-name) for each time you want to write a file. For instance, an incrementing number can be appended to the unmodifiable-file-name.
Something like:
std::string uniqueFileName(void)
{
std::stringstream ss;
// vvvvvvvvvv -- unmodifiable-file-name is not changed
ss << "output.txt" << ++fileCount;
uniqueFileName = ss.str();
return(uniqueFileName);
}
Good luck.
PS
If you feel you must write the file first in the correct file name, and then change the file name to something unique ... yes, you can rename the file from within this program (i.e. trivial synchronization)
I would use popen() as I feel it provides more feedback, and I've used it before.
Others prefer something like system() (there are about 6 of these).
In either case, use the command to rename the existing file (to each you provide a bash command, like mv fromPfn toPfn, or maybe you'll need cp.
For each, your code must not proceed until the command has completed.
Related
Is there any good way i can make some data created by my c++ program available to the shell after exiting the program?
I have a c++ program, inside which i have a string containing a path:
std::string path = "/home/some/path"
I want this path to be available after the c++ program exits main and i am returned to the shell, so that i can use that path (e.g. cd to that path).
I have considered/tried the following approaches:
I tried making an environment variable in c++ program using setenv(). However the environment variable only exists while in the c++ program, and it is apparently not possible to make those changes visible in the shell after exiting the program.
(considered) writing the path to a temporary file, so that a bash script could later access the details of the path from it. However i have read many suggestions to not do that due to security vulnerabilities.
I tried calling the bash script from within the c++ program, using system(). This does not work if i try to cd to that directory (exiting the program will keep me in the same directory as before).
I figure that if i am desperate, i could have my program cout the path, and use the solutions as described here:
$ ./program | tee output.txt
Then the path is stored inside the file. This works technically, but has the undesirable effect of creating a file and printing the path to the screen, and is basically creating a temporary file.
another option to, again, cout in my program, and use command substitution. running in the shell
$ var=$(./program)
storing the path in var. This didnt work because my program does many things including requiring user input before calling
std::cout<< path << std::endl;.
Particularly, i have observed this approach to not display a curses window, which is required for the program.
the only solution that has worked is piping the output to tee.
Environment variables are only an input, they cannot be used to return any information from a program.
You are already using std::cin and std::cout for user input, and std::cerr should be reserved for error messages. However, you can have the shell open more filedescriptors, and have your program write to those. However, doing this with pure C++ is not possible. But if you don't mind using POSIX C functions:
#include <cstdio>
int main() {
FILE *f = fdopen(3, "w");
fprintf(f, "some path\n");
}
And then use it like so:
./program 3> output.txt
This of course creates an undesirable file. I don't think there is any way to store the output from an extra filedescriptor directly to a variable in bash. However, you could create a temporary file inside /dev/shm, so it will never be written to disk, or create a FIFO object and redirect the output from the program to the FIFO, and then read it back. For some examples of how to do this, see this question.
You could write the output that you want the user to see to stderr instead of stdout. Only output what you need your shell script to see to stdout:
#include <iostream>
int main() {
std::clog << "Enter data: "; // clog prints to stderr like cerr
std::string line;
std::getline(std::cin, line);
std::cout << line << '\n';
}
Then this will work:
var=$(./program)
I have a program that needs to run a program we'll call externalProg in parallel on our linux (CentOS) cluster - or rather, it needs to run many instances of externalProg, each on different cores. Each "thread" creates 3 files based on a few parameters - the inputs to externalProg, a command file to tell externalProg how to execute my file, and a bash script to set up the environment (calls a setup script provided by the manufacturer) and actually call externalProg with my inputs.
Since this needs to be parallel with an unknown number of concurrent threads and I don't want to risk overwriting another thread's files, I am creating temp files using
mkstemp("PREFIX_XXXXXX")
for these input files. After the external program runs, I extract the relevant data and store it, and close the temp files (therefore deleting them).
We'll call the files created (Which actually have a name based on the template above)
tmpInputs - Inputs to externalProg
tmpCommand - Input that tells externalProg how to execute tmpInputs
tmpBash - bash script to set up and call externalProg with my inputs
The file tmpBash looks something like
source /path/to/setup/script # Sets up environment variables
externalProg < /path/to/tmpCommand
where tmpCommand is just a simple text file.
The problem I'm having is actually executing the bash script. Within my program, I call
ostringstream launchcmd;
launchcmd << "bash " << path_to_tmpBash
system(launchcmd.str().c_str());
But nothing happens. No error, no warning, no 'file not found' or permission denied or anything. I have verified that the files are being created and have the correct content. The rest of the code after system() is executed successfully (Though it fails since externalProg wasn't run).
Strangely, if I go back to the terminal and type
bash /path/to/tmpBash
then externalProg is executed successfully. I have also cout'd the launchcmd string, copy and pasted that in to the terminal, which also works successfully. For some reason, this only fails when called within my program.
After a bit of experimentation, I've determined that system() calls /bin/sh on our cluster. If I change launchcmd to look like
/path/to/tmpBash
(So that the full command should look like /bin/sh /path/to/tmpBash), I get a permission denied error, which is no surprise. The problem is that I can't chmod +x the tmpBash file while it's still open, and if I close the file, it gets deleted - so I'm not sure how to address that.
Is there something obviously wrong I'm doing, or does system() have some nuance that I'm missing?
edit: I wanted to add that I can successfully call things like
system("echo $PATH")
and get the expected results (in this case, my default $PATH).
Two separate ideas:
Change your SHELL environment variable to be /bin/bash, then call system(),
or:
Use execve directly `execve('/bin/bash', ['/path/to/tmpBash'], environ)
I've got a series of cpp source file and I want to write another program to JUDGE if they can run correctly (give input and compare their output with standart output) . so how to:
call/spawn another program, and give a file to be its standard input
limit the time and memory of the child process (maybe setrlimit thing? is there any examples?)
donot let the process to read/write any file
use a file to be its standard output
compare the output with the standard output.
I think the 2nd and 3rd are the core part of this prob. Is there any way to do this?
ps. system is Linux
To do this right, you probably want to spawn the child program with fork, not system.
This allows you to do a few things. First of all, you can set up some pipes to the parent process so the parent can supply the input to the child, and capture the output from the child to compare to the expected result.
Second, it will let you call seteuid (or one of its close relatives like setreuid) to set the child process to run under a (very) limited user account, to prevent it from writing to files. When fork returns in the parent, you'll want to call setrlimit to limit the child's CPU usage.
Just to be clear: rather than directing the child's output to a file, then comparing that to the expected output, I'd capture the child's output directly via a pipe to the parent. From there the parent can write the data to a file if desired, but can also compare the output directly to what's expected, without going through a file.
std::string command = "/bin/local/app < my_input.txt > my_output_file.txt 2> my_error_file.txt";
int rv = std::system( command.c_str() );
1) The system function from the STL allows you to execute a program (basically as if invoked from a shell). Note that this approach is inherenly insecure, so only use it in a trusted environment.
2) You will need to use threads to be able to achieve this. There are a number of thread libraries available for C++, but I cannot give you recommendation.
[After edit in OP's post]
3) This one is harder. You either have to write a wrapper that monitors read/write access to files or do some Linux/Unix privilege magic to prevent it from accessing files.
4) You can redirect the output of a program (that it thinks goes to the standard output) by adding > outFile.txt after the way you would normally invoke the program (see 1)) -- e.g. otherapp > out.txt
5) You could run diff on the saved file (from 3)) to the "golden standard"/expected output captured in another file. Or use some other method that better fits your need (for example you don't care about certain formatting as long as the "content" is there). -- This part is really dependent on your needs. diff does a basic comparing job well.
What is the fastest method in C++, to read a new line from a file which is written by another process. Or how my program can be notified that there is a new line in file so read it? (in linux)
The fastest method is to use pipes or events (for Windows apps).
If you still want use files, first of all that you really need, making sure, that a file has been really modified (use seek and compare it with prew value). Than go to the 'last val of seek' and read it.
And it will be better use mutex (if you read data from file).
Assuming the OS supports concurrent file access, all you should need to do is seek to EOF, wait for the stat to change then try to read from the file. You might want to add in a sleep to slow down the loop.
The 'tail' command on POISX (with the -f option) implements this - source code is available.
From the top of my head, did u tried something like this:
Count the lines in a file, store it.
Get the size of the file (google it, i dont want to ruin the fun :D ).
Then try to read from the last line u stored when size of the file changes... and again and again.
Have fun :)
Use inotify to get notification about file changes and then reread from your last pos if the file is now larger then before.
I'm trying to edit the /etc/fstab file on a CentOS installation using C++. The idea being that based on another config file I will add entries that don't exist in the fstab, or edit entries in the fstab file where the mount point is the same. This lets us set the system up properly on initial bootup.
I've found setmntent() and getmntent() for iterating over the exiting entries so I can easily check whether an entry in fstab also exists in my config file. And I can then use addmntent() to add any entry that doesn't already exist - the documentation says nothing about this being able to edit an entry, only add a new entry to the end of the file. There seems to be no way to edit an existing entry or delete an entry. It seems odd that this feature doesn't exist, only the CR and not the UD of CRUD.
I'd rather not have to write my own parser if I can at all help it.
My other alternative is to:
open the file using setmntent()
read the whole of fstab into memory using getmentent() and perform any additions and/or edits
close the file using endmntent()
open /etc/fstab for writing
close /etc/fstab (thus emptying the file)
open the fstab using setmntent()
loop through the entries I read in previously and write them out using addmntent()
Which although probably fine, just seems a bit messy.
When modifying system configuration files such as /etc/fstab keep in mind that these are critical state and, should your "edit" be interrupted by a power loss, might result in a failure to reboot.
The way to deal with this is:
create an empty output:
FILE* out = setmntent("/etc/fstab.new", "rw");
open the original for input:
FILE* in = setmntent("/etc/fstab", "r");
copy the contents:
while (m = getmntent(in)) { addmntent(out, m); }
make sure the output has it all:
fflush(out); endmntent(out); endmntent(in);
atomically replace /etc/fstab:
rename("/etc/fstab.new", "/etc/fstab");
It's left as an exercise to the reader to change the body of the while loop to make a modification to an existing element, to substitute a specifically crafted mntent or whatever. If you have specific questions on that please ask.
UN*X semantics for rename() guarantee that even in the case of power loss, you'll have either the original version or your new updated one.
There's a reason why there is no modifymntent() - because that would encourage bad programming / bad ways of changing system critical files. You say at the end of your post "... probably fine ..." - not. The only safe way to change a system configuration file is to write a complete modified copy, sync that to safe storage, and then use rename to replace the old one.