How can I copy a directory using Boost Filesystem - c++

How can I copy a directory using Boost Filesystem?
I have tried boost::filesystem::copy_directory() but that only creates the target directory and does not copy the contents.

bool copyDir(
boost::filesystem::path const & source,
boost::filesystem::path const & destination
)
{
namespace fs = boost::filesystem;
try
{
// Check whether the function call is valid
if(
!fs::exists(source) ||
!fs::is_directory(source)
)
{
std::cerr << "Source directory " << source.string()
<< " does not exist or is not a directory." << '\n'
;
return false;
}
if(fs::exists(destination))
{
std::cerr << "Destination directory " << destination.string()
<< " already exists." << '\n'
;
return false;
}
// Create the destination directory
if(!fs::create_directory(destination))
{
std::cerr << "Unable to create destination directory"
<< destination.string() << '\n'
;
return false;
}
}
catch(fs::filesystem_error const & e)
{
std::cerr << e.what() << '\n';
return false;
}
// Iterate through the source directory
for(
fs::directory_iterator file(source);
file != fs::directory_iterator(); ++file
)
{
try
{
fs::path current(file->path());
if(fs::is_directory(current))
{
// Found directory: Recursion
if(
!copyDir(
current,
destination / current.filename()
)
)
{
return false;
}
}
else
{
// Found file: Copy
fs::copy_file(
current,
destination / current.filename()
);
}
}
catch(fs::filesystem_error const & e)
{
std:: cerr << e.what() << '\n';
}
}
return true;
}
Usage:
copyDir(boost::filesystem::path("/home/nijansen/test"), boost::filesystem::path("/home/nijansen/test_copy")); (Unix)
copyDir(boost::filesystem::path("C:\\Users\\nijansen\\test"), boost::filesystem::path("C:\\Users\\nijansen\\test2")); (Windows)
As far as I see, the worst that can happen is that nothing happens, but I won't promise anything! Use at your own risk.
Please note that the directory you're copying to must not exist. If directories within the directory you are trying to copy can't be read (think rights management), they will be skipped, but the other ones should still be copied.
Update
Refactored the function respective to the comments. Furthermore the function now returns a success result. It will return false if the requirements for the given directories or any directory within the source directory are not met, but not if a single file could not be copied.

Since C++17 you don't need boost for this operation anymore as filesystem has been added to the standard.
Use std::filesystem::copy
#include <exception>
#include <filesystem>
namespace fs = std::filesystem;
int main()
{
fs::path source = "path/to/source/folder";
fs::path target = "path/to/target/folder";
try {
fs::copy(source, target, fs::copy_options::recursive);
}
catch (std::exception& e) { // Not using fs::filesystem_error since std::bad_alloc can throw too.
// Handle exception or use error code overload of fs::copy.
}
}
See also std::filesystem::copy_options.

I see this version as an improved upon version of #nijansen's answer. It also supports the source and/or destination directories to be relative.
namespace fs = boost::filesystem;
void copyDirectoryRecursively(const fs::path& sourceDir, const fs::path& destinationDir)
{
if (!fs::exists(sourceDir) || !fs::is_directory(sourceDir))
{
throw std::runtime_error("Source directory " + sourceDir.string() + " does not exist or is not a directory");
}
if (fs::exists(destinationDir))
{
throw std::runtime_error("Destination directory " + destinationDir.string() + " already exists");
}
if (!fs::create_directory(destinationDir))
{
throw std::runtime_error("Cannot create destination directory " + destinationDir.string());
}
for (const auto& dirEnt : fs::recursive_directory_iterator{sourceDir})
{
const auto& path = dirEnt.path();
auto relativePathStr = path.string();
boost::replace_first(relativePathStr, sourceDir.string(), "");
fs::copy(path, destinationDir / relativePathStr);
}
}
The main differences are exceptions instead of return values, the use of recursive_directory_iterator and boost::replace_first to strip the common part of the iterator path, and relying on boost::filesystem::copy() to do the right thing with different file types (preserving symlinks, for instance).

This is a non-Boost version I'm using based on Doineann's code. I'm using std::filesystem but couldn't use a simple fs::copy(src, dst, fs::copy_options::recursive); because I wanted to filter which files are copied by file extension inside the loop.
void CopyRecursive(fs::path src, fs::path dst)
{
//Loop through all the dirs
for (auto dir : fs::recursive_directory_iterator(src))
{
//copy the path's string to store relative path string
std::wstring relstr = dir.path().wstring();
//remove the substring matching the src path
//this leaves only the relative path
relstr.erase(0, std::wstring(src).size());
//combine the destination root path with relative path
fs::path newFullPath = dst / relstr;
//Create dir if it's a dir
if (fs::is_directory(newFullPath))
{
fs::create_directory(newFullPath);
}
//copy the files
fs::copy(dir.path(), newFullPath, fs::copy_options::recursive | fs::copy_options::overwrite_existing);
}
}
relstr.erase(0, std::wstring(src).size()); is a working Boost-less replacement for the boost::replace_first() call used in the other answer

Related

open file by std::ifstream occasionaly fails on folder detected by FindFirstChangeNotification (Windows)

I have an application that must monitor some folders (in Windows) to detect if a file was created in that folder (real use is to detect incoming FTP files).
If a file is detected , it is read, then deleted .
Occasionally, I get a file reading error on a file that was detected.
Question is: Why?
To simulate the error, I created a simple program to reproduce it:
std::vector<std::filesystem::path> watch;
void main()
{
watch.push_back("D:\\test1"); //must exist
watch.push_back("D:\\test2");
watch_dir();
}
this example monitors 2 folders.
To simulate incoming files on the folder, another program copies files to that folder
continuously at configurable intervals (say 100 milliseconds).
To detect folder changes , WIN32 API functions FindFirstChangeNotification and WaitForMultipleObjects are used, based on this Microsoft example
https://learn.microsoft.com/en-us/windows/win32/fileio/obtaining-directory-change-notifications
detection function adapted from the example (Note: WaitForMultipleObjects blocks until a change is detected)
void watch_dir()
{
HANDLE handle[2];
memset(handle, 0, 2 * sizeof(HANDLE));
for (size_t idx = 0; idx < watch.size(); idx++)
{
std::string str = watch.at(idx).string();
LPTSTR path = (LPTSTR)str.c_str();
std::cout << "watch path " << path << std::endl;
handle[idx] = FindFirstChangeNotification(
path, // directory to watch
FALSE, // do not watch subtree
FILE_NOTIFY_CHANGE_FILE_NAME); // watch file name changes
if (handle[idx] == INVALID_HANDLE_VALUE)
{
assert(0);
ExitProcess(GetLastError());
}
}
while (TRUE)
{
std::cout << "Waiting for notification..." << std::endl;
DWORD wait_status = WaitForMultipleObjects(watch.size(), handle, FALSE, INFINITE);
std::cout << "Directory " << watch.at(wait_status) << " changed" << std::endl;
if (FindNextChangeNotification(handle[wait_status]) == FALSE)
{
assert(0);
ExitProcess(GetLastError());
}
std::filesystem::path path = watch.at(wait_status);
send_files_in_path(path);
}
}
Once a change is detected by the function above, then all files in the folder are listed
and read, by these functions
void send_files_in_path(const std::filesystem::path& ftp_path)
{
std::vector<std::filesystem::path> list = get_files(ftp_path);
for (size_t idx = 0; idx < list.size(); idx++)
{
std::string buf;
read_file(list.at(idx).string(), buf);
std::this_thread::sleep_for(std::chrono::milliseconds(10));
std::filesystem::remove(list.at(idx));
}
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//get_files
//get all ".txt" files inside a FTP folder
/////////////////////////////////////////////////////////////////////////////////////////////////////
std::vector<std::filesystem::path> get_files(const std::filesystem::path& base_archive_path)
{
std::vector<std::filesystem::path> list;
try
{
for (const auto& entry : std::filesystem::recursive_directory_iterator(base_archive_path))
{
std::filesystem::path path = entry.path();
if (!entry.is_regular_file())
{
continue;
}
std::string fname = entry.path().filename().string();
size_t len = fname.size();
size_t pos = len - 4;
//check if last 4 characters are ".txt"
if (fname.find(".txt", pos) == std::string::npos && fname.find(".TXT", pos) == std::string::npos)
{
continue;
}
SPDLOG_INFO("loading: " + entry.path().string());
list.push_back(path);
}//this path
} //try
catch (const std::exception& e)
{
SPDLOG_ERROR(e.what());
}
return list;
}
The function where the error happens is
int read_file(const std::string& fname, std::string& buf)
{
std::ifstream ifs;
std::ios_base::iostate mask = ifs.exceptions() | std::ios::failbit;
ifs.exceptions(mask);
std::this_thread::sleep_for(std::chrono::milliseconds(0));
std::cout << "opening : " << fname << std::endl;
try
{
ifs.open(fname);
if (!ifs.is_open())
{
std::cout << "open fail: " << fname << std::endl;
return -1;
}
}
catch (const std::exception& e)
{
std::cout << e.what() << std::endl;
return -1;
}
std::stringstream ss;
ss << ifs.rdbuf();
ifs.close();
buf = ss.str();
return 0;
}
the try/catch block, again, occasionally , is triggered with the error
ios_base::failbit set: iostream stream error
removing the try/catch block, and the open mask (just to try), then
ifs.is_open
fails.
A temporary solution was to detect the cases where the open() failed and repeat it.. which succeeds, because the file does exist.
Calling this with a small delay before the open call has the effect of reducing the open fails
std::this_thread::sleep_for(std::chrono::milliseconds(10));
ifs.open(fname);
But still would like to find out the reason for the occasional failure

CD Command in C++ command line

I'm trying to make a CD command for a shell I call POSH (Pine's own shell).
If the previous cd command didn't end in a /, it will append the two paths and throw (cd pine then cd src will error because path would be /home/dingus/pinesrc instead of /home/dingus/pine/src).
I know why this happens, but can't seem to fix it.
Here is the source code:
inline void cd(string cmd)
{
if(cmd.length() < 3) return;
string arg = cmd.substr(3);
if(fs::is_directory(fs::status( string(path).append(arg))))
{
path.append(arg);
}
else cout << "Error: \"" << arg << "\" is not a directory" << endl;
}
string(path).append(arg) is performing string concatenation. You are not appending any of your own slashes between filesystem elements. So, if the path is /home/dingus/, then cd pine would just append pine to the end producing /home/dingus/pine, and then cd src would just append src to the end producing /home/dingus/pinesrc, as you observed.
You would need to do something more like this instead:
string curr_path;
inline void cd(string cmd)
{
if (cmd.length() < 4) return;
string arg = cmd.substr(3);
string new_path = curr_path;
if ((!new_path.empty()) && (new_path.back() != '\\')) {
new_path += '\\';
}
new_path += arg;
if (fs::is_directory(fs::status(new_path))) {
curr_path = new_path;
}
else {
cout << "Error: \"" << arg << "\" is not a directory" << endl;
}
}
However, since you are using the <filesystem> library anyway, you should be using std::filesystem::path (especially since std::filesystem::status() only accepts std::filesystem::path to begin with). Let the library handle any path concatenations for you, eg:
fs::path curr_path;
inline void cd(string cmd)
{
if (cmd.length() < 4) return;
string arg = cmd.substr(3);
fs::path new_path = curr_path / arg;
if (fs::is_directory(new_path)) {
curr_path = new_path;
}
else {
cout << "Error: \"" << arg << "\" is not a directory" << endl;
}
}
std::filesystem::path implements operator/ to insert a slash if one is not already present.

Copy directory content

I want to copy the content o directory(tmp1) to another directory(tmp2). tmp1 may contain files and others directories. I want to copy the content of tmp1 (including the mode) using C/C++. If tmp1 contains a tree of directories I want to copy them recursively.
What is the simplest solution?
I found a solution to open the directory and read every entry and copy it with cp command. Any simpler solutions?
I recommend using std::filesystem (merged to ISO C++ as of C++17!)
Shamelessly copied from http://en.cppreference.com/w/cpp/filesystem/copy:
std::filesystem::copy("/dir1", "/dir3", std::filesystem::copy_options::recursive);
Read more about it:
https://gcc.gnu.org/onlinedocs/gcc-6.1.0/libstdc++/api/a01832.html
experimental::filesystem linker error
Recently I had the same need, so I have developed the next chunk of code in order to solve the problem. I hope it helps to another people in the same situation.
#include <iostream>
#include <dirent.h>
#include <string.h>
#include <sys/stat.h>
#include <windows.h>
using namespace std;
bool is_dir(const char* path);
void copyFile_(string inDir, string outDir);
void copyDir_(const char *inputDir, string outDir);
int main()
{
string srcDir = "C:\\testDirectory";
string destDir = "C:\\destDir";
copyDir_(srcDir.c_str(), destDir);
return 0;
}
void copyDir_(const char *inputDir, string outDir)
{
DIR *pDIR;
struct dirent *entry;
string tmpStr, tmpStrPath, outStrPath, inputDir_str = inputDir;
if (is_dir(inputDir) == false)
{
cout << "This is not a folder " << endl;
return;
}
if( pDIR = opendir(inputDir_str.c_str()) )
{
while(entry = readdir(pDIR)) // get folders and files names
{
tmpStr = entry->d_name;
if( strcmp(entry->d_name, ".") != 0 && strcmp(entry->d_name, "..") != 0 )
{
tmpStrPath = inputDir_str;
tmpStrPath.append( "\\" );
tmpStrPath.append( tmpStr );
cout << entry->d_name;
if (is_dir(tmpStrPath.c_str()))
{
cout << "--> It's a folder" << "\n";
// Create Folder on the destination path
outStrPath = outDir;
outStrPath.append( "\\" );
outStrPath.append( tmpStr );
mkdir(outStrPath.c_str());
copyDir_(tmpStrPath.c_str(), outStrPath);
}
else
{
cout << "--> It's a file" << "\n";
// copy file on the destination path
outStrPath = outDir;
outStrPath.append( "\\" );
outStrPath.append( tmpStr );
copyFile_(tmpStrPath.c_str(), outStrPath.c_str());
}
}
}
closedir(pDIR);
}
}
bool is_dir(const char* path)
{
struct stat buf;
stat(path, &buf);
return S_ISDIR(buf.st_mode);
}
void copyFile_(string inDir, string outDir)
{
CopyFile(inDir.c_str(), outDir.c_str(), 1);
DWORD Error = GetLastError();
}

How do I ignore hidden files (and files in hidden directories) with Boost Filesystem?

I am iterating through all files in a directory recursively using the following:
try
{
for ( bf::recursive_directory_iterator end, dir("./");
dir != end; ++dir )
{
const bf::path &p = dir->path();
if(bf::is_regular_file(p))
{
std::cout << "File found: " << p.string() << std::endl;
}
}
} catch (const bf::filesystem_error& ex) {
std::cerr << ex.what() << '\n';
}
But this includes hidden files and files in hidden directories.
How do I filter out these files? If needed I can limit myself to platforms where hidden files and directories begin with the '.' character.
Unfortunately there doesn't seem to be a cross-platform way of handling "hidden". The following works on Unix-like platforms:
First define:
bool isHidden(const bf::path &p)
{
bf::path::string_type name = p.filename();
if(name != ".." &&
name != "." &&
name[0] == '.')
{
return true;
}
return false;
}
Then traversing the files becomes:
try
{
for ( bf::recursive_directory_iterator end, dir("./");
dir != end; ++dir)
{
const bf::path &p = dir->path();
//Hidden directory, don't recurse into it
if(bf::is_directory(p) && isHidden(p))
{
dir.no_push();
continue;
}
if(bf::is_regular_file(p) && !isHidden(p))
{
std::cout << "File found: " << p.string() << std::endl;
}
}
} catch (const bf::filesystem_error& ex) {
std::cerr << ex.what() << '\n';
}
Let's assume for now that you want to ignore files which start with a '.'. This is the standard indication in Unix for a hidden file. I suggest writing a recursive function to visit each file. In pseudocode, it looks something like this:
visitDirectory dir
for each file in dir
if the filename of file does not begin with a '.'
if file is a directory
visitDirectory file
else
do something with file (perhas as a separate function call?)
This avoids the need to search the whole path of a file to determine whether or not we want to deal with it. Instead, we simply skip any directories which are "hidden."
I can think of several iterative solutions as well, if that's what you prefer. One is to have a stack or queue to keep track of which directory to visit next. Basically this emulates the recursive version with your own data structure. Alternatively, if you are stuck on parsing the full path of the file, simply make sure you get the absolute path. This will guarantee that you don't encounter a directory with a name like './' or '../', which would cause problems with checking for a hidden file.

C++: Error with Boost Filesystem copy_file

I'm running into some trouble with the copy_file function. My program is very simple, I'm just attempting to copy a text file from one spot to another.
The following code brings up a "Debug Error!" because abort() was called.
int main()
{
path src_path = "C:\\src.txt";
path dst_path = "C:\\dst.txt";
cout << "src exists = " << exists( src_path ) << endl; // Prints True
boost::filesystem::copy_file( src_path, dst_path );
return 0;
}
If I look at some other examples of code on Stackoverflow I cannot notice what I'm doing wrong. I feel like I'm missing something obvious here.
I have Boost v1.47 installed and I'm using Visual C++ 2010.
I'm guessing that the target file exists.
The docs:
template <class Path1, class Path2> void copy_file(const Path1& from_fp, const Path2& to_fp);
Requires: Path1::external_string_type and Path2::external_string_type are the same type.
Effects: The contents and attributes of the file from_fp resolves to are copied to the file to_fp resolves to.
Throws: basic_filesystem_error<Path> if from_fp.empty() || to_fp.empty() || !exists(from_fp) || !is_regular_file(from_fp) || exists(to_fp)
A simple test like so:
#include <iostream>
#include <boost/filesystem.hpp>
int main()
{
using namespace boost::filesystem;
path src_path = "test.in";
path dst_path = "test.out";
std::cout << "src exists = " << std::boolalpha << exists( src_path ) << std::endl; // Prints true
try
{
boost::filesystem::copy_file( src_path, dst_path );
} catch (const boost::filesystem::filesystem_error& e)
{
std::cerr << "Error: " << e.what() << std::endl;
}
return 0;
}
Prints:
src exists = true
Error: boost::filesystem::copy_file: File exists: "test.in", "test.out"
on the second run :)
I think if you are using boost::filesystem2 it should be
boost::filesystem2::copy(src_path,dest_path);
copy_file should have been deprecated.