I have to read a script from the user and call a QProcess passing that script as a file.
For example, the user insert this, say, Python script
import sys
print(sys.copyright)
and I have to put that script in a file, and call the python interpreter using that file.
I thought to use a QTemporaryFile, because that file will serve just when launching the process, and I have no need to keep it open.
The question is: is it safe to open a QTemporaryFile, write something in it, pass that file to a process (which will continue indefinitely) and then destroy the temporary file? What if the process will need that file again? What if the process keep the file open?
I reckon that, if kept open by the process, no problem will arise: probably the QTemporaryFile will unlink the path, but data will still be accessible in memory.
But what if the process will try to open the file again?
Here a snippet as example (wrote on the fly)
QString script = QInputDialog::getText(blah);
QTemporaryFile tmp;
if (tmp.open()) {
tmp.write(script.toUtf8());
QStringList params;
params << tmp.fileName();
QProcess *proc = new QProcess("/usr/bin/python3");
proc->start(params);
}
As I understand it, in the case of the 'autoRemove' flag (which is on by default), the QTemporaryFile will exist so long as the instance of QTemporaryFile exists. Therefore, in the code you originally presented, when tmp goes out of scope, the file will be removed. Calling open / close on the object will not delete the file.
You could dynamically allocate the file with QTemporaryFile* pTmp = new QTemporaryFile and then delete it later, if you know when the python script has finished with it.
Ouch, I just noted the autoRemove flag in the QTemporaryFile. I guess this could be a solution: if set to false, the file will not be removed from the disk, so the process is free to reuse the file - I think.
Temporary files should be stored in system's default location, so I guess that the files are not removed until a reboot (at least, I believe Linux works this way).
This is just an idea, but I will wait for other answers or confirmations.
Related
I am trying to learn how to handle and work on files, now I know how to open them, write on them and read. What I would like to do is, how can I delete data/content from the file, when I have finished to use the program?
I use the a txt file to save some informations that I used them, when I need during the execution of the program, but when I finish, I would like to delete the data saved, which are simply numbers. I was thinking to remove the file each time, and to create, but I think it's not the ideal. Any suggestions?
Using std::filesystem::resize_file:
std::filesystem::resize_file(your_file, 0);
In such a case, you usually just re-write the file as a whole. If it is a small file, you can read in the entire content, modify the content and write the file back.
With large files, if you fear that you are consuming too much memory, you can read in the file in chunks of appropriate size, but you'd write these chunks to another, temporary file. When being finished, you delete the old file and move the temporary file to the location of the old file.
You can combine both aproaches, too: Read the entire file, write it to temporary at once, then delete and move; if anything goes wrong while writing the temporary, you'd still have the old file as backup...
You can open the file in writing mode (w) and then close it . It will truncate all previous data.
It's generally a good idea to clean up temporary files once your program ends. I certainly wouldn't leave an empty temporary file hanging around. You can easily remove file (e.g. with boost::filesystem::remove or std::filesystem::remove). If you really just want to 'clear' a file then:
void clear_file(const std::string& filename)
{
std::ofstream file {filename};
}
Will do the job.
I'm in the process of trying to copy an hdf5 binary file on a local machine to a remote computing blade. I am using libssh to copy the desired directory or files out after they are generated by my Qt application. Using libssh I am able to open an ssh_session, authenticate it, open a channel and send remote commands.
for (QStringList::iterator it = ipList.begin(); it != ipList.end(); ++it)
{
ssh_session my_session = new ssh_new();
QString ip_address = *it;
ssh_options_set(my_session, SSH_OPTIONS_HOST, ip_address.toStdString().c_str());
// Connect...Authenticate using public key....
QString command = QString("rm -r %2; cp -r %1 %1; cp /local/file.txt /remote/file.txt").arg(local_dir, remote_dir);
execute_remote_command(my_session, command.toStdString().c_str());
// Open channel and execute command
ssh_disconnect(my_session);
ssh_free(my_session);
}
This command is being executed for each individual computing blade. In between each of the calls I am closing and opening an ssh session to the next blade. The files make it out the blades but they appear to be corrupt. They are the exact same file size. I haven't figured out a way to compare the individual bytes to see just how corrupt they are, any tips there would be appreciated as well.
When I run my ssh copy commands in a separate test terminal program the files appear to make it intact and are readable on the blades. The issue only seems to occur when the files are moved from within the Qt GUI program.
EDIT: So delving a little bit deeper into what is wrong, it appears that the file on the remote server is not the same size. It appears to be missing a large portion of the bytes. On top of that when I examine what is there byte by byte with the local version of the file, almost all of the bytes differ.
Turns out the answer was that the HDF5 writer wasn't being closed properly before the SSH commands were being called. I fixed the problem by dynamically allocating the custom H5 class that someone else wrote and made sure to delete it before the SSH commands were called. Turns out whoever wrote the HDF5 read and write class didn't handle file opening and closing properly and didn't provide functions to do so.
Below is an example of what I am talking about.
HDF5writer_class *hdf5_writer = new HDF5writer_class();
hdf5_writer->create_file("/local/machine/hdf5_file.h5");
// ... add the data to the file
delete hdf5_writer;
// Open SSH Session and run the copy commands
Long story short, make sure the file you are writing is closed and released for use before you try to copy it.
On my home Linux laptop, I like to write wrapper programs and GUI helpers for things I use frequently. However, I don't like Bash scripting very much, so I do a lot of stuff in C++. However, a lot of times, this requires me to use the system() function from the cstdlib.
This system() command is awesome, but I wanted a way to call system() and receive the stdout/stderror. The system() command only returns the return code from the command. So, in a Bash script, one can do:
myVar=$(ls -a | grep 'search string')
echo $myVar
and myVar will output whatever the stdout was for the command. So I began writing a wrapper class that will add a pipe-to-file to the end of the command, open the file, read all of the piped stdout, and return it as either one long string or as a vector of strings. The intricacies of the class are not really relevant here (I don't think anyway), but the above example would be done like this:
SystemCommand systemCommand;
systemCommand.setCommand("ls -a | grep \'search string\' ");
systemCommand.execute();
std::cout << systemCommand.outputAsString() << std::endl;
Behind the scenes, when systemCommand.execute() is called, the class ensures that the command will properly pipe all stdout/stderr to a randomly generated filename, in the current working directory. So for example, the above command would end up being
"ls -a | grep 'search string' >> 1452-24566.txt 2>&1".
The class then goes attempts to open and read from that file, using ifstream:
std::ifstream readFromFile;
readFromFile.open(_outputFilename);
if (readFromFile.is_open()) {
//Read all contents of file into class member vector
...
readFromFile.close();
//Remove temporary file
...
} else {
//Handle read failure
}
So here is my main question will std::ifstream ever fail to open a recently created file in the current working directory? If so, what would be a way to make it more robust (specifically on Linux)?
A side/secondary question: Is there a very simplified way to achieve what I'm trying to achieve without using file pipes? Perhaps some stuff available in unistd.h? Thanks for your time.
So here is my main question will std::ifstream ever fail to open a recently created file in the current working directory?
Yes.
Mount a USB thumb drive (or some other removable media)
cd to the mount
Execute your program. While it's executing, remove the drive.
Watch the IO error happen.
There's a ton of other reasons too. Filesystem corruption, hitting the file descriptor limit, etc.
If so, what would be a way to make it more robust (specifically on Linux)?
Make temporary files in /tmp, whose entire purpose is for temporary files. Or don't create a file at all, and use pipes for communication instead (Like what popen does, like harmic suggested). Even so, there are no guarantees; try to gracefully handle errors.
I'd like to make a copy of some/path/myfile in $TMPDIR/myprog-<random-string>.ext, such that I can then pass it on to a 3rd party procedure that chokes on extensionless files.
Here's what I'd like to work:
QString originalPath = "some/path/myfile";
QTemporaryFile f(
QDir::temp().absoluteFilePath("mprog-XXXXXX.ext")
);
// f.open(); ?
QFile(originalPath).copy(f.fileName());
However, I now have a problem - either the file doesn't yet exist, and thus hasn't been assigned a temporary fileName(), or the file name has been assigned but the file itself already exists, preventing the new file being copied on top.
How can I copy a file to a temporary location in QT, and have the temporary copy removed when the destructor of QTemporaryFile is called?
If the file doesn't exist, create the QTemporaryFile object exactly as you have done, open it and then close it immediately. This will generate the random filename and create it on the disk.
When your QTemporaryFile object gets destroyed, the file will be deleted from the disk.
Unfortunately, Qt (5.3) doesn't support copying to an existing file. The only correct, race-free use of QTemporaryFile is to open() it, creating it in the process, and then operate on it.
You'll need to implement the copy yourself, I'm afraid :(
On Windows, if you expect there to be some gain from using CopyFileEx, I have a complete example that wraps it for Qt's perusal, with progress signals.
The real question is: do you really need to create a copy? Wouldn't a hard link do? Since Qt runs, for the most part, on Unices and Windows, you can create a symbolic link wrapper that will wrap POSIX link() and winapi CreateHardLink. If hard link creation fails, or the temporary folder is on a different volume, you can then try CreateSymbolicLink. Of course you'd need to look up CreateSymbolicLinkW using QLibrary if you intend your executable to start un XP at all. If that fails, you're either running on XP or on a FAT partition, and the final fallback is copying.
Would it be out of the question to rename the file, run the 3rd-party application on it, then rename it back?
If you're not going to be actually using the file / stream to the file QTemporaryFile created for you, you're better off using QUuid instead to just generate a guaranteed unique filename. This generates a unique filename, roughly equivalent to QTemporaryFile:
QUuid uuid = QUuid::createUuid();
QString tempFileFullPath = QDir::toNativeSeparators(QDir::tempPath() + "/" + qApp->applicationName().replace(" ", "") + "_" + uuid.toString(QUuid::WithoutBraces) + ".dat");
I like the temporary filename to not contain spaces, so I removed that from the app name used for the filename prefix. (You should call QApplication::setApplicationName in your main() function so this will work.)
Also, you should probably change the .dat extension to something suitable for your file type.
I have a folder to which files are copied. I want to watch it and process files as soon as they are copied to the directory. I can detect when a file is in the directory, whether through polling (my current implementation) or in some tests using Windows APIs from a few samples I've found online.
The problem is that I detect when the file is first created and its still being copied. This makes my program, that needs to access the file, through errors (because the file is not yet complete). How can I detect not when the copying started but when the copying ended? I'm using C++ on Windows, so the answer may be platform dependent but, if possible, I'd prefer it to be platform agnostic.
You could use either lock files or a special naming convention. The easiest is the latter and would work something like this:
Say you want to move a file named "fileA.txt" When copying it to the destination directory, instead, copy it to "fileA.txt.partial" or something like that. When the copy is complete, rename the file from "fileA.txt.partial" to "fileA.txt". So the appearance of "fileA.txt" is atomic as far as the watching program can see.
The other option as mentioned earlier is lock files. So when you copy a file named "fileA.txt", you first create a file called "fileA.txt.lock". When the copying is done, you simply delete the lock file. When the watching program see "fileA.txt", it should check if "fileA.txt.lock" exists, if it does, it can wait or revisit that file in the future as needed.
You should not be polling. Use FindFirstChangeNotification (http://msdn.microsoft.com/en-us/library/windows/desktop/aa364417%28v=vs.85%29.aspx) to watch a directory for changes.
Then use the Wait functions (http://msdn.microsoft.com/en-us/library/windows/desktop/ms687069%28v=vs.85%29.aspx) to wait on change notifications to happen.
Overview and examples here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365261%28v=vs.85%29.aspx
I'm not sure how exactly file write completion can be determined. Evan Teran's answer is a good idea.
You can use something like this, This is tested and working
bool IsFileDownloadComplete(const std::wstring& dir, const std::wstring& fileName)
{
std::wstring originalFileName = dir + fileName;
std::wstring tempFileName = dir + L"temp";
while(true)
{
int ret = rename(convertWstringToString(originalFileName).c_str(), convertWstringToString(tempFileName).c_str());
if(ret == 0)
break;
Sleep(10);
}
/** File is not open. Rename to original. */
int ret = rename(convertWstringToString(tempFileName).c_str(), convertWstringToString(originalFileName).c_str());
if(ret != 0)
throw std::exception("File rename failed");
return true;
}