This question already has answers here:
How to create a temporary directory in C++?
(6 answers)
Closed 3 years ago.
I'm trying to create a unique temporary directory in the system temp folder and have been reading about the security and file creation issues of tmpnam().
I've written the below code and was wondering whether it would satisfy these issues, is my use of the tmpnam() function correct and the throwing of the filesystem_error? Should I be adding checks for other things (e.g. temp_directory_path, which also throws an exception)?
// Unique temporary directory path in the system temporary directory path.
std::filesystem::path tmp_dir_path {std::filesystem::temp_directory_path() /= std::tmpnam(nullptr)};
// Attempt to create the directory.
if (std::filesystem::create_directories(tmp_dir_path)) {
// Directory successfully created.
return tmp_dir_path;
} else {
// Directory could not be created.
throw std::filesystem_error("directory could not be created.");
}
From cppreference.com:
Although the names generated by std::tmpnam are difficult to guess, it is possible that a file with that name is created by another process between the moment std::tmpnam returns and the moment this program attempts to use the returned name to create a file.
The problem is not with how you use it, but the fact that you do.
For example, in your code sample, if a malicious user successfully guesses and creates the directory right in between the first and second line, it might deny service (DOS) from your application, which might be critical, or not.
Instead, there is a way to do this without races on POSIX-compliant systems:
For files see mkstemp(3) for more information
For dirs see mkdtemp(3) for more information.
Your code is fine. Because you try to create the directory the OS will arbitrate between your process and another process trying to create the same file so, if you win, you own the file and if you lose you get an error.
I wrote a similar function recently. Whether you throw an exception or not depends on how you want to use this function. You could, for example simply return either an open or closed std::fstream and use std::fstream::is_open as a measure of success or return an empty pathname on failure.
Looking up std::filesystem::create_directories it will throw its own exception if you don't supply a std::error_code parameter so you don't need to throw your own exception:
std::filesystem::path tmp_dir_path {std::filesystem::temp_directory_path() /= std::tmpnam(nullptr)};
// Attempt to create the directory.
std::filesystem::create_directories(tmp_dir_path));
// If that failed an exception will have been thrown
// so no need to check or throw your own
// Directory successfully created.
return tmp_dir_path;
Related
As you can see by the title I have issues when copying files from a directory to another. The code works perfectly fine but when the files are in use, obviously I get an error. Is there any way I can skip these files? (that are in use) and simply move on to the next?
I have tried looking for a solution to my issue but it seems there isn't a function from what I've seen on the internet for filesystem, any information given helps. I provided the code below for copying files.
std::filesystem::copy("C:\\Users\\"+ user+"\\AppData\\Local\\tee\\test\\User Data", "C:\\Users\\"+user+"\\AppData\\Local\\tee\\test\\Application", std::filesystem::copy_options::overwrite_existing | std::filesystem::copy_options::recursive);
If you can't copy you will get an exception, if you don't catch that the program chrashes. You can skip the files if you catch the exception with a simple try-catch statement. eg.:
try{
std::filesystem::copy(/*Params*/);
}catch(...){
std::cout<<"cant-copy '"<<filename<<"'";
/*Handle it if you don't want to just ignore (sleep and try later or ask the user what to to...)*/
}
I use boost log by this config.
[Sinks.2]
Filter="%Severity% >= 2"
Destination=TextFile
AutoFlush=true
Format="[%TimeStamp%] [%ThreadID%] <%Severity%> %Message%"
Asynchronous=false
Target="logs"
FileName="logs/quo.%Y%m%dT%H%M%S.%a.%5N.log.detail"
RotationTimePoint="00:00:00"
RotationSize=104857600
MinFreeSpace=4294967296
MaxSize=4294967296
ScanForFiles=All
when date change to next day. my program crash by exception:
terminate called after throwing an instance of
'boost::filesystem::filesystem_error'
what(): boost::filesystem::last_write_time: No such file or directory: "/root/work/hy-trade/bin/debug/logs/quo.20181027T173106.Sat.00000.log.detail"
I check my disk space, find the free space less than MinFreeSpace in config and the file quo.20181027T173106.Sat.00000.log.detail not exists.
how to avoid this exception?
version of boost is 1.67
thank you
It looks like someone had already deleted the log file before it was rotated. It may have been an external process, or Boost.Log.
With Boost.Log, this can happen if you have multiple file sinks that write log files into the same directory, which is also used as the target directory for the rotated files (i.e. the FileName parameter includes the path specified in the Target parameter, and there are multiple sinks that use that path). The problem is because, according to ScanForFiles=All, the library scans the target directory for any files but does not update the file counter to be used for creating new files. This means that if the file "quo.20181027T173106.Sat.00000.log.detail" was present in that directory when your process started then it would be considered as an old file, even if upon starting your process would be still writing new logs to that file. Then, when a file rotation happens and storage limits are exceeded (e.g. if MinFreeSpace is not satisfied), that file may be deleted. The rotation has to happen on another sink that still stores files into the same "logs" directory.
To solve the problem you can do one of the following:
Use ScanForFiles=Matching in your settings so that the file counter is updated after scanning. This will make sure that new log files have unique names and don't get deleted prematurely.
Write log files to a different directory from your target storage. I.e. specify FileName so that it doesn't point to the same directory as Target.
Also, you may want to add exception handling to avoid crashing in case of errors (which may still happen for whatever reason on filesystem operations). See here and here for more info (also, follow the links in those sections).
i when run this test program create the below error.
import shutil
src=r"G:\aaa"
dst=r"F:\zzz"
shutil.copytree(src,dst, symlinks=False, ignore=None)
FileExistsError: [WinError 183] Cannot create a file when that file already exists:
but the the folder of F:\zzz is empty!!!
shutil.copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, dirs_exist_ok=False)
dirs_exist_ok dictates whether to raise an exception in case dst or any missing parent directory already exists.
Python 3.8 now have support of dirs_exist_ok parameter. This won't give that error anymore and overwrite the destination folder in case it already exists.
Hence you need to use:
shutil.copytree(src, dst, dirs_exist_ok=True)
"Cannot create a file when that file already exists" is a generic Windows message which is confusing because it's the same for directories or regular files. (Windows isn't known for very helpful error messages, you have to make do with that)
from the online help of shutil.copytree:
>>> help(shutil.copytree)
Help on function copytree in module shutil:
copytree(src, dst, symlinks=False, ignore=None, copy_function=, ignore_dangling_symlinks=False)
Recursively copy a directory tree.
The destination directory must not already exist.
So first time it probably works, but other times you need to perform
shutil.rmtree(dst)
to remove the destination directory prior to copying the tree (note that Windows is annoying with permissions and that files with read-only attribute can choke shutil.rmtree, which I personally copied the code into a custom version (you're encouraged to do so in the online help) to add a os.chmod(path,0o777) prior to deleting regular files.
Note: Since problem is solved, I've added comments to my original posts.
According to "http://msdn.microsoft.com/en-us/library/6tkkkc1y%28v=vs.90%29.aspx", it stated as this:
*You must call _findclose after you are finished using either the _findfirst or _findnext function (or any variants). This frees up resources used by these functions in your application.*
--comment: it is vague, but what microsoft is trying to say is: some users just need to find the first file(they don't need to call _findnext), then call _findclose; some users called _findnext (they MUST have already called _findfirst), after finished using that, call _findclose. Actually _findnext can be called multiple times, while _findclose is only responsible to a handle, which is created by _findfirst.
And following is a piece of code that is widely used to list files in the directory. -- comment: it is correct.
For example, if there are 2 files and 1 directory in the directory, then:
.
..
ddd
file1.txt
file2.txt
_findfirst is called once. the handle's corresponding fileinfo is system directory "." (is that right?)
--comment: no. the handle is a group of files+directories, the fileinfo is acting as the "cursor". (fileinfo always contained the "name" field, I bet the implementation of _findnext is using the "name" to find the next in a group of files+directories specified by the handle)
_findnext is called 4 times. (the first argument is always the handle corresponding to ".", is that right?)
--comment: yes + no. The first argument is always the same handle; the handle is NOT corresponding to any fileinfo, but to a group of them.
My questions are:
Does "_findclose" be called ONCE is enough?
*--comment:* yes.
if _findnext will not change the handle value, how can it "remember" where to start to find the next file(or directory)? (sorry, maybe I was thinking in the "linked list" pattern.)
*--comment:* I bet is using fileinfo's name field. Just as in Windows Explorer, we sort the contents in a folder, given a file name, we can know their position in the list, so we can "find next".
Are there any harm to call _findclose more times than needed? (like crash or something)
*--comment:* a stupid question. Sorry!
Or is the following code wrong at all? If yes, what's the correct way to implement it?
--It is correct code.
// List the files in the directory
intptr_t file;
_finddata_t filedata;
file = _findfirst(desc.c_str(),&filedata);
if (file != -1)
{
do
{
cout << filedata.name << endl;
// Or put the file name in a vector here
} while (_findnext(file,&filedata) == 0);
}
else
{
cout << "No described files found" << endl;
}
_findclose(file);
I asked this because I've met an issue that an application is freezing a directory which can not be deleted if the process is alive. However, I can guaranteed that "_findclose" is called on every return value from "_findfirst". If I add "_findclose" after calling "_findnext", then will fix the issue perfectly. How can you help me to explain it?
--comment: pardon. don't use "guarantee" too easy. That's where the bug is.
Note: I don't have problem to understand what is a handle, like open a file, read/write/read/write..., close the file handle. I just find the documentations describing these three APIs are vague.
--comment: go to improve your english.
Thank you in advance.
Your calls to _findclose should match with your calls to _findfirst -- i.e., each time you call _findfirst, you should have a matching call to _findclose.
In the code above, since you have only one call to _findfirst, it's correct to have only one call to _findclose.
If you were doing a recursive search of subdirectories, then you'd end up with multiple calls to _findfirst as you descend the hierarchy, and matching calls to _findclose as you finish and ascend back up the hierarchy.
You only need to call _findclose once, when you are finished.
On Windows, a directory may be locked if it is the current directory of your process. Try calling _chdir.
If that doesn't work... are you opening any of the files in the directory you're searching? An open file may lock the directory as well.
It may be useful to let Process Explorer get a look at your app. It can tell you for sure what handle you have left open.
I am using Qsettings for non gui products to store its settings into xml files. This is written as a library which gets used in C, C++ programs. There will be 1 xml file file for each product. Each product might have more than one sub products and they are written into xml by subproduct grouping as follows -
File: "product1.xml"
<product1>
<subproduct1>
<settings1>..</settings1>
....
<settingsn>..</settingsn>
</subproduct1>
...
<subproductn>
<settings1>..</settings1>
....
<settingsn>..</settingsn>
</subproductn>
</product1>
File: productn.xml
<productn>
<subproduct1>
<settings1>..</settings1>
....
<settingsn>..</settingsn>
</subproduct1>
...
<subproductn>
<settings1>..</settings1>
....
<settingsn>..</settingsn>
</subproductn>
</productn>
The code in one process does the following -
settings = new QSettings("product1.xml", XmlFormat);
settings.setValue("settings1",<value>)
sleep(20);
settings.setValue("settings2", <value2>)
settings.sync();
When the first process goes to sleep, I start another process which does the following -
settings = new QSettings("product1.xml", XmlFormat);
settings.remove("settings1")
settings.setValue("settings3", <value3>)
settings.sync();
I would expect the settings1 to go away from product1.xml file but it still persist in the file - product1.xml at the end of above two process. I am not using QCoreApplication(..) in my settings library. Please point issues if there is anything wrong in the above design.
This is kind of an odd thing that you're doing, but one thing to note is that the sync() call is what actually writes the file to disk. In this case if you want your second process to actually see the changes you've made, then you'll need to call sync() before your second process accesses the file in order to guarantee that it will actually see your modifications. Thus I would try putting a settings.sync() call right before your sleep(20)
Maybe you have to do delete settings; after the sync() to make sure it is not open, then do the writing in the other process?
Does this compile? What implementation of XmlFormat are you using and which OS? There must be some special code in your project for storing / reading to and from Xml - there must be something in this code which works differently from what you expect.