I'm trying to retrieve all files from the root (C:/) in C++
First of all, I retrieve all logical drives in the computer, then I use the std::filesystem library (specifically the recursive_directory_iterator function in order to loop in directories)
DWORD dwSize = MAX_PATH;
char szLogicalDrives[MAX_PATH] = { 0 };
DWORD dwResult = GetLogicalDriveStrings(dwSize, szLogicalDrives);
if (dwResult > 0 && dwResult <= MAX_PATH)
{
char* szSingleDrive = szLogicalDrives;
while (*szSingleDrive)
{
szSingleDrive[strlen(szSingleDrive) - 1] = 0;
printf(szSingleDrive);
for (fs::directory_entry p : std::filesystem::recursive_directory_iterator(szSingleDrive))
{
string filePath = p.path().string();
// Vérification du type de l'objet
if (fs::is_regular_file(p.path()))
{
cout << filePath << endl;
}
// get the next drive
szSingleDrive += strlen(szSingleDrive) + 1;
}
}
}
However, the output I get is the path of my project.
Eg : C:x64\Debug\myProject.exe
Desired output : C:\Users, C:\Windows, C:\Program Files...
In order to resolve the issue I had to launch VS 2019 in admin (or launch the .exe in admin) + disable Windows Defender.
To avoid UAC exception, I also added skip_permission_denied in filesystem option.
However, my program still encounter "Sharing Violation error"
Related
I wrote a simple code to do some operation on every file in every folder (subfolders).
It's perfectly works until the path comes with 'SPACE
' character program crashs and INVALID_HANDLE_VALUE has been called. This is function:
int dirListFiles(char* startDir)
{
HANDLE hFind;
WIN32_FIND_DATAA wfd;
char path[MAX_PATH];
sprintf(path, "%s\\*", startDir);
std::string fileName;
std::string s_path = startDir;
std::string fullPath;
fprintf(stdout, "In Directory \"%s\"\n\n", startDir);
if ((hFind = FindFirstFileA(path, &wfd)) == INVALID_HANDLE_VALUE)
{
printf("FindFirstFIle failed on path = \"%s\"\n", path);
abort();
}
BOOL cont = TRUE;
while (cont == TRUE)
{
if ((strncmp(".", wfd.cFileName, 1) != 0) && (strncmp("..", wfd.cFileName, 2) != 0))
{
if (wfd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
{
sprintf(path, "%s\\%s", startDir, wfd.cFileName);
dirListFiles(path);
}
else
{
fileName = wfd.cFileName;
fullPath = s_path + "\\" + fileName;
std::string fileExt = PathFindExtension(fullPath.c_str());
if (fileExt == ".cpp")
{
... Some operation on file
}
}
}
cont = FindNextFile(hFind, &wfd);
}
FindClose(hFind);
For example, If FindNextFile wants to Open Program Files (x86) which has space between file name cause error and program exit. What Can I do for supporting spaces? What Is Problem?
Space is legal character in directory and file names.
First I propose to modify slightly your code:
if ((hFind = FindFirstFileA(path, &wfd)) == INVALID_HANDLE_VALUE)
{
printf("FindFirstFIle failed on path = \"%s\". Error %d\n", path, GetLastError());
return 0; // I think you shouldn't abort on error, just skip this dir.
}
Now check error codes reported by your program.
For some paths I have got error #5 (access denied). Examples:
c:\Program Files (x86)\Google\CrashReports\*
c:\ProgramData\Microsoft\Windows Defender\Clean Store\*
c:\Windows\System32\config\*
Got two cases with code #123 (Invalid name) for path names unmanageable by FindFirstFileA. To correct this behavior it would be better to use wide version of function FindFirstFileW. See both answers for c++ folder only search. For new Windows applications you should use wide version of API, converting with MultiByteToWideChar and WideCharToMultiByte if needed.
You have also logic error. Code skips all directories and files starting with dot.
After having written many files to a series of folders on EFS (10k or so). Readdir stops returning all of the files in each directory.
I have a C++ application that in one part of its process it generates a lot of files and each file is given a symlink. After that I need to get a list of the file in a folder to then select a subset to rename. When I run the function that gets the list of files, it does not return all the files that are actually there. This code runs fine on my local machine, but on an AWS server with a mounted EFS drive, it stops working after a while.
In order to troubleshoot this issue, I have made my code only write one file at a time. I have also setup my code to use getFiles() to give me a count of how many files there are in a folder after writing each batch of files (around 17 files). When the number of files reaches ~950 files, getFiles() starts listing ~910 files and no longer increments. When its writing files, the files are varied but fairly small (2 bytes - 300K) and its writing about 200 files a second. Each file also has a symlink created to it.
When reading and writing files I am using posix open(), write(), read() and close(). I have verified that I do in fact close all files after reading or writing.
I am trying to figure out:
1. Why is readdir not working? Or why is it not listing all the files?
2. What is different about EFS that could be causing issues?
These are the functions I am using to get the list of files in a folder:
DIR * FileUtil::getDirStream(std::string path) {
bool success = false;
if (!folderExists(path)){
return NULL;
}
DIR * dir = opendir(path.c_str());
success = dir != NULL;
int count = 0;
while(!success){
int fileRetryDelay = BlazingConfig::getInstance()->getFileRetryDelay();
const int sleep_milliseconds = (count+1)*fileRetryDelay;
std::this_thread::sleep_for(std::chrono::milliseconds(sleep_milliseconds));
std::cout<<"Was unable to get Dir stream for "<<path<<std::endl;
dir = opendir(path.c_str());
success = dir != NULL;
count++;
if(count > 6){
break;
}
}
if(success == -1){
std::cout<<"Can't get Dir stream for "<<path<<". Error was: "<<errno<<std::endl;
}
return dir;
}
int FileUtil::getDirEntry(DIR * dirp, struct dirent * & prevDirEntry, struct dirent * & dirEntry){
bool success = false;
if (dirp == NULL){
return -1;
}
int returnCode = readdir_r(dirp, prevDirEntry, &dirEntry);
success = (dirEntry == NULL && returnCode == 0) || dirEntry != NULL;
int count = 0;
while(!success){
int fileRetryDelay = BlazingConfig::getInstance()->getFileRetryDelay();
const int sleep_milliseconds = (count+1)*fileRetryDelay;
std::this_thread::sleep_for(std::chrono::milliseconds(sleep_milliseconds));
std::cout<<"Was unable to get dirent with readdir"<<std::endl;
returnCode = readdir_r(dirp, prevDirEntry, &dirEntry);
success = (dirEntry == NULL && returnCode == 0) || dirEntry != NULL;
count++;
if(count > 6){
break;
}
}
if(success == -1){
std::cout<<"Can't get dirent with readdir. Error was: "<<errno<<std::endl;
}
return returnCode;
}
std::vector<std::string> FileUtil::getFiles(std::string baseFolder){
DIR *dir = getDirStream(baseFolder);
std::vector <std::string> subFolders;
if (dir != NULL) {
struct dirent *prevDirEntry = NULL;
struct dirent *dirEntry = NULL;
int len_entry = offsetof(struct dirent, d_name) + fpathconf(dirfd(dir), _PC_NAME_MAX) + 1;
prevDirEntry = (struct dirent *)malloc(len_entry);
int returnCode = getDirEntry(dir, prevDirEntry, dirEntry);
while (dirEntry != NULL) {
if( dirEntry->d_type == DT_REG || dirEntry->d_type == DT_LNK){
std::string name(dirEntry->d_name);
subFolders.push_back(name);
}
returnCode = getDirEntry(dir, prevDirEntry, dirEntry);
}
free(prevDirEntry);
closedir (dir);
} else {
std::cout<<"Could not open directory err num is"<<errno<<std::endl;
/* could not open directory */
perror ("");
}
return subFolders;
}
The functions were written this way to try to be as robust as possible, since there can be many threads performing file operations, I wanted to be able to have the code retry in case of any failures. Unfortunately when getFiles() returns the wrong result, it does not give me any indication of failure.
Note: when I use readdir as opposed to readdir_r I still have the same issue.
My executable built in C++/WinAPI will check for a file placed in the same folder and I use PathFileExists for that. When I run it on a normal computer it finds the file but when I publish the executable on RemoteApp and I run it from Web Access the file is not found. What would I be missing?
// This is the file I want to find (located in the same directory as the EXE)
wstring myfile = L"myfile.conf";
BOOL abspath = FALSE;
// Trying to get the absolute path first
DWORD nBufferLength = MAX_PATH;
wchar_t szCurrentDirectory[MAX_PATH + 1];
if (GetCurrentDirectory(nBufferLength, szCurrentDirectory) == 0) {
szCurrentDirectory[MAX_PATH + 1] = '\0';
} else {
abspath = true;
}
if (abspath) {
// Create the absolute path to the file
myfile = L'\\' + myfile;
myfile = szCurrentDirectory + myfile ;
MessageBox(hWnd, ConvertToUNC(myfile).c_str(), L"Absolute Path", MB_ICONINFORMATION);
} else {
// Get the UNC path
myfile = ConvertToUNC(myfile);
MessageBox(hWnd, myfile.c_str(), L"UNC Path", MB_ICONINFORMATION);
}
// Try to find file
int retval = PathFileExists(myfile.c_str());
if (retval == 1) {
// Do something
} else {
// File not found
}
The ConvertToUNC function is copied from here.
What I see is that, although the executable lies somewhere else, the absolute path is considered to be C:\Windows. I really don't know what is causing this. The server is Windows 2012 R2 and, like I said, applications are run through RemoteApp Web Access. The returned UNC path is just the name of the file (no volume or folder)
I am working on a application in VC++ where network drives are used to access files. The drives are assigned manually by the users and then select the drive in the application. This results in drives not always being mapped to the same servers.
How would I go about obtaining the UNC path to such a file? This is mostly for identification purposes.
here's the function I use to convert a normal path to an UNC path:
wstring ConvertToUNC(wstring sPath)
{
WCHAR temp;
UNIVERSAL_NAME_INFO * puni = NULL;
DWORD bufsize = 0;
wstring sRet = sPath;
//Call WNetGetUniversalName using UNIVERSAL_NAME_INFO_LEVEL option
if (WNetGetUniversalName(sPath.c_str(),
UNIVERSAL_NAME_INFO_LEVEL,
(LPVOID) &temp,
&bufsize) == ERROR_MORE_DATA)
{
// now we have the size required to hold the UNC path
WCHAR * buf = new WCHAR[bufsize+1];
puni = (UNIVERSAL_NAME_INFO *)buf;
if (WNetGetUniversalName(sPath.c_str(),
UNIVERSAL_NAME_INFO_LEVEL,
(LPVOID) puni,
&bufsize) == NO_ERROR)
{
sRet = wstring(puni->lpUniversalName);
}
delete [] buf;
}
return sRet;;
}
Suggest you use WNetGetConnection.
API description is here: http://msdn.microsoft.com/en-us/library/windows/desktop/aa385453(v=vs.85).aspx
I've written an application that uses the WIN32 api to create a temporarily directory hierarchy. Now, when wanting to delete the directories when shutting down the application I'm running into some problems.
So lets say I have a directory hierarchy: C:\temp\directory\subdirectory\
I'm using this recursive function:
bool Dir::deleteDirectory(std::string& directoryname, int flags)
{
if(directoryname.at(directoryname.size()-1) != '\\') directoryname += '\\';
if ((flags & CONTENTS) == CONTENTS)
{
WIN32_FIND_DATAA fdata;
HANDLE dhandle;
directoryname += "\\*";
dhandle = FindFirstFileA(directoryname.c_str(), &fdata);
// Loop through all the files in the main directory and delete files & make a list of directories
while(true)
{
if(FindNextFileA(dhandle, &fdata))
{
std::string filename = fdata.cFileName;
if(filename.compare("..") != 0)
{
std::string filelocation = directoryname.substr(0, directoryname.size()-2) + StringManip::reverseSlashes(filename);
// If we've encountered a directory then recall this function for that specific folder.
if(!isDirectory(filelocation)) DeleteFileA(filename.c_str());
else deleteDirectory(filelocation, DIRECTORY_AND_CONTENTS);
}
} else if(GetLastError() == ERROR_NO_MORE_FILES) break;
}
directoryname = directoryname.substr(0, directoryname.size()-2);
}
if ((flags & DIRECTORY) == DIRECTORY)
{
HANDLE DirectoryHandle;
DirectoryHandle = CreateFileA(directoryname.c_str(),
FILE_LIST_DIRECTORY,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED,
NULL);
bool DeletionResult = (RemoveDirectoryA(directoryname.c_str()) != 0)?true:false;
CloseHandle(DirectoryHandle);
return DeletionResult;
}
return true;
}
This function iterates over the directory contents of the temp directory; and for each directory in the temp directory it keeps recalling itself until it's at the lowest directory; subdirectory in the example.
There are also 3 flags defined
enum DirectoryDeletion
{
CONTENTS = 0x1,
DIRECTORY = 0x2,
DIRECTORY_AND_CONTENTS = (0x1 | 0x2)
};
When using this function, it only removes the lowest subdirectory and I can't remove the ones higher in hierarchy because it says that the directory is not empty. When I go and look to the directory 'subdirectory' is only removed after the application ends. However, when I try to encapsulate this in a non recursive simple main application I have no problems at all with deleting the directories.
There's a Windows API, SHFileOperation, that will do a recursive folder delete for you.
LONG DeleteDirectoryAndAllSubfolders(LPCWSTR wzDirectory)
{
WCHAR szDir[MAX_PATH+1]; // +1 for the double null terminate
SHFILEOPSTRUCTW fos = {0};
StringCchCopy(szDir, MAX_PATH, wzDirectory);
int len = lstrlenW(szDir);
szDir[len+1] = 0; // double null terminate for SHFileOperation
// delete the folder and everything inside
fos.wFunc = FO_DELETE;
fos.pFrom = szDir;
fos.fFlags = FOF_NO_UI;
return SHFileOperation( &fos );
}
You're not closing dhandle from all those FindFirstFile calls, so each directory has a reference to it when you try to delete it.
And, why do you need to create DirectoryHandle? It's not needed, and will probably also block the directory deletion.
When your app closes, those handles are forced close, and (I guess) the last attempted delete then succeeds.
SHFileOperations works great on Windows 7. In fact in the IFileOperation documentation says
IFileOperation can only be applied in a single-threaded apartment (STA) situation. It cannot be used for a multithreaded apartment (MTA) situation. For MTA, you still must use SHFileOperation.
However my issue with SHFileOperations is it doesn't seem to support paths longer than 260 characters, and does not support \?\ prefix for long filenames.
This is a real pain....but a recursive function is still needed if you want ability to handle paths longer than 260 characters (Which NTFS supports - but not Windows Explorer, command prompt commands etc)
Well, I found several bugs in this code.. here is what I found
bool Dir::deleteDirectory(std::string& directoryname, int flags)
{
if(directoryname.at(directoryname.size()-1) != '\\') directoryname += '\\';
if ((flags & CONTENTS) == CONTENTS)
{
WIN32_FIND_DATAA fdata;
HANDLE dhandle;
//BUG 1: Adding a extra \ to the directory name..
directoryname += "*";
dhandle = FindFirstFileA(directoryname.c_str(), &fdata);
//BUG 2: Not checking for invalid file handle return from FindFirstFileA
if( dhandle != INVALID_HANDLE_VALUE )
{
// Loop through all the files in the main directory and delete files & make a list of directories
while(true)
{
if(FindNextFileA(dhandle, &fdata))
{
std::string filename = fdata.cFileName;
if(filename.compare("..") != 0)
{
//BUG 3: caused by BUG 1 - Removing too many characters from string.. removing 1 instead of 2
std::string filelocation = directoryname.substr(0, directoryname.size()-1) + filename;
// If we've encountered a directory then recall this function for that specific folder.
//BUG 4: not really a bug, but spurious function call - we know its a directory from FindData already, use it.
if( (fdata.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0)
DeleteFileA(filelocation.c_str());
else
deleteDirectory(filelocation, DIRECTORY_AND_CONTENTS);
}
} else if(GetLastError() == ERROR_NO_MORE_FILES) break;
}
directoryname = directoryname.substr(0, directoryname.size()-2);
//BUG 5: Not closing the FileFind with FindClose - OS keeps handles to directory open. MAIN BUG
FindClose( dhandle );
}
}
if ((flags & DIRECTORY) == DIRECTORY)
{
HANDLE DirectoryHandle;
DirectoryHandle = CreateFileA(directoryname.c_str(),
FILE_LIST_DIRECTORY,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED,
NULL);
//BUG 6: Not checking CreateFileA for invalid handle return.
if( DirectoryHandle != INVALID_HANDLE_VALUE )
{
bool DeletionResult = (RemoveDirectoryA(directoryname.c_str()) != 0)?true:false;
CloseHandle(DirectoryHandle);
return DeletionResult;
}
else
{
return true;
}
}
return true;
}
Try calling FindClose to close handle returned by FindFileFileA.
I don't see a FindClose for your dhandle. An open handle means that the directory is still in use.
MSDN says: "When the search handle is no longer needed, close it by using the FindClose function, not CloseHandle."
(CloseHandle appears to be correct for your DirectoryHandle further down, but not for the dhandle used in the Find loop.)
The main issue has already been answered, but here's something I noticed. Your main while loop seems a bit fragile to me...
while(true)
{
if(FindNextFileA(dhandle, &fdata))
{
//...
} else if(GetLastError() == ERROR_NO_MORE_FILES) break;
}
This ends if FindNextFile ends because there are no more files in the directory. But what if it ever ends for some other reason? If something abnormal happens, it seems you could end up with an infinite loop.
I'd think if FindNextFile fails for any reason, then you'll want to stop the loop and start returning through the recursive calls. So I'd suggest simply removing the GetLastError test and just making it "else break;"
Actually, after a moment's thought I would probably just reduce it to:
while(FindNextFileA(dhandle, &fdata))
{
//...
}