I write for myself a small program in C ++, which could perform some operations on files that it finds (in my filter), and that's stumbled on the mechanism of searching for files. At start the program asks the full path, and then by file type recursively looking for them in all subdirectories of the selected directory. The trouble is that after performing an operation (cycle fopen - operation - fclose) can not rename or delete the file. The program simply exits with code 0. It is I sin on the file searching mechanism, as is likely, the function uses image for the time of its implementation and does not delete or rename the file. I tried different options to manage files through WinAPI, std (fstream) and just fopen / fclose. Nothing comes out.
Code snippet:
int main() {
char sPath[MAX_PATH] = "C:\\TmpDir";
char sExt[10] = "doc";
char sEXT[10] = "DOC";
GetFileList(sPath, sExt, sEXT);
printf("Results= %d\n", rez);
system("pause");
return 0;
}
void GetFileList(LPTSTR sPath, LPTSTR sExt, LPTSTR sEXT) {
WIN32_FIND_DATA pFILEDATA;
HANDLE hFile = FindFirstFile(strcat(sPath, "\\*.*"), &pFILEDATA);
sPath[strlen(sPath) - strlen(strstr(sPath, "*.*"))] = '\0';
if (hFile != INVALID_HANDLE_VALUE) {
char * chBuf;
do {
if (strlen(pFILEDATA.cFileName) == 1 && strchr(pFILEDATA.cFileName, '.') != NULL)
if (FindNextFile(hFile, &pFILEDATA) == 0)
break;
if (strlen(pFILEDATA.cFileName) == 2 && strstr(pFILEDATA.cFileName, "..") != NULL)
if (FindNextFile(hFile, &pFILEDATA) == 0)
break;
if (pFILEDATA.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
GetFileList(strcat(sPath, pFILEDATA.cFileName), sExt, sEXT);
sPath[strlen(sPath) - strlen(pFILEDATA.cFileName) - 1] = '\0';
} else {
if ((chBuf = strrchr(pFILEDATA.cFileName, '.'))) {
if (strstr(chBuf + 1, sExt) || strstr(chBuf + 1, sEXT)) {
CharToOem(sPath, sPath);
printf("%s", sPath);
OemToChar(sPath, sPath);
CharToOem(pFILEDATA.cFileName, pFILEDATA.cFileName);
printf("%s\n", pFILEDATA.cFileName);
/* Какая-то операция с файлом.
...
Конец операции с файлом. */
rez++;
}
}
}
} while (FindNextFile(hFile, &pFILEDATA));
}
}
Related
I wrote a simple code to do some operation on every file in every folder (subfolders).
It's perfectly works until the path comes with 'SPACE
' character program crashs and INVALID_HANDLE_VALUE has been called. This is function:
int dirListFiles(char* startDir)
{
HANDLE hFind;
WIN32_FIND_DATAA wfd;
char path[MAX_PATH];
sprintf(path, "%s\\*", startDir);
std::string fileName;
std::string s_path = startDir;
std::string fullPath;
fprintf(stdout, "In Directory \"%s\"\n\n", startDir);
if ((hFind = FindFirstFileA(path, &wfd)) == INVALID_HANDLE_VALUE)
{
printf("FindFirstFIle failed on path = \"%s\"\n", path);
abort();
}
BOOL cont = TRUE;
while (cont == TRUE)
{
if ((strncmp(".", wfd.cFileName, 1) != 0) && (strncmp("..", wfd.cFileName, 2) != 0))
{
if (wfd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
{
sprintf(path, "%s\\%s", startDir, wfd.cFileName);
dirListFiles(path);
}
else
{
fileName = wfd.cFileName;
fullPath = s_path + "\\" + fileName;
std::string fileExt = PathFindExtension(fullPath.c_str());
if (fileExt == ".cpp")
{
... Some operation on file
}
}
}
cont = FindNextFile(hFind, &wfd);
}
FindClose(hFind);
For example, If FindNextFile wants to Open Program Files (x86) which has space between file name cause error and program exit. What Can I do for supporting spaces? What Is Problem?
Space is legal character in directory and file names.
First I propose to modify slightly your code:
if ((hFind = FindFirstFileA(path, &wfd)) == INVALID_HANDLE_VALUE)
{
printf("FindFirstFIle failed on path = \"%s\". Error %d\n", path, GetLastError());
return 0; // I think you shouldn't abort on error, just skip this dir.
}
Now check error codes reported by your program.
For some paths I have got error #5 (access denied). Examples:
c:\Program Files (x86)\Google\CrashReports\*
c:\ProgramData\Microsoft\Windows Defender\Clean Store\*
c:\Windows\System32\config\*
Got two cases with code #123 (Invalid name) for path names unmanageable by FindFirstFileA. To correct this behavior it would be better to use wide version of function FindFirstFileW. See both answers for c++ folder only search. For new Windows applications you should use wide version of API, converting with MultiByteToWideChar and WideCharToMultiByte if needed.
You have also logic error. Code skips all directories and files starting with dot.
After having written many files to a series of folders on EFS (10k or so). Readdir stops returning all of the files in each directory.
I have a C++ application that in one part of its process it generates a lot of files and each file is given a symlink. After that I need to get a list of the file in a folder to then select a subset to rename. When I run the function that gets the list of files, it does not return all the files that are actually there. This code runs fine on my local machine, but on an AWS server with a mounted EFS drive, it stops working after a while.
In order to troubleshoot this issue, I have made my code only write one file at a time. I have also setup my code to use getFiles() to give me a count of how many files there are in a folder after writing each batch of files (around 17 files). When the number of files reaches ~950 files, getFiles() starts listing ~910 files and no longer increments. When its writing files, the files are varied but fairly small (2 bytes - 300K) and its writing about 200 files a second. Each file also has a symlink created to it.
When reading and writing files I am using posix open(), write(), read() and close(). I have verified that I do in fact close all files after reading or writing.
I am trying to figure out:
1. Why is readdir not working? Or why is it not listing all the files?
2. What is different about EFS that could be causing issues?
These are the functions I am using to get the list of files in a folder:
DIR * FileUtil::getDirStream(std::string path) {
bool success = false;
if (!folderExists(path)){
return NULL;
}
DIR * dir = opendir(path.c_str());
success = dir != NULL;
int count = 0;
while(!success){
int fileRetryDelay = BlazingConfig::getInstance()->getFileRetryDelay();
const int sleep_milliseconds = (count+1)*fileRetryDelay;
std::this_thread::sleep_for(std::chrono::milliseconds(sleep_milliseconds));
std::cout<<"Was unable to get Dir stream for "<<path<<std::endl;
dir = opendir(path.c_str());
success = dir != NULL;
count++;
if(count > 6){
break;
}
}
if(success == -1){
std::cout<<"Can't get Dir stream for "<<path<<". Error was: "<<errno<<std::endl;
}
return dir;
}
int FileUtil::getDirEntry(DIR * dirp, struct dirent * & prevDirEntry, struct dirent * & dirEntry){
bool success = false;
if (dirp == NULL){
return -1;
}
int returnCode = readdir_r(dirp, prevDirEntry, &dirEntry);
success = (dirEntry == NULL && returnCode == 0) || dirEntry != NULL;
int count = 0;
while(!success){
int fileRetryDelay = BlazingConfig::getInstance()->getFileRetryDelay();
const int sleep_milliseconds = (count+1)*fileRetryDelay;
std::this_thread::sleep_for(std::chrono::milliseconds(sleep_milliseconds));
std::cout<<"Was unable to get dirent with readdir"<<std::endl;
returnCode = readdir_r(dirp, prevDirEntry, &dirEntry);
success = (dirEntry == NULL && returnCode == 0) || dirEntry != NULL;
count++;
if(count > 6){
break;
}
}
if(success == -1){
std::cout<<"Can't get dirent with readdir. Error was: "<<errno<<std::endl;
}
return returnCode;
}
std::vector<std::string> FileUtil::getFiles(std::string baseFolder){
DIR *dir = getDirStream(baseFolder);
std::vector <std::string> subFolders;
if (dir != NULL) {
struct dirent *prevDirEntry = NULL;
struct dirent *dirEntry = NULL;
int len_entry = offsetof(struct dirent, d_name) + fpathconf(dirfd(dir), _PC_NAME_MAX) + 1;
prevDirEntry = (struct dirent *)malloc(len_entry);
int returnCode = getDirEntry(dir, prevDirEntry, dirEntry);
while (dirEntry != NULL) {
if( dirEntry->d_type == DT_REG || dirEntry->d_type == DT_LNK){
std::string name(dirEntry->d_name);
subFolders.push_back(name);
}
returnCode = getDirEntry(dir, prevDirEntry, dirEntry);
}
free(prevDirEntry);
closedir (dir);
} else {
std::cout<<"Could not open directory err num is"<<errno<<std::endl;
/* could not open directory */
perror ("");
}
return subFolders;
}
The functions were written this way to try to be as robust as possible, since there can be many threads performing file operations, I wanted to be able to have the code retry in case of any failures. Unfortunately when getFiles() returns the wrong result, it does not give me any indication of failure.
Note: when I use readdir as opposed to readdir_r I still have the same issue.
I'm trying to implement the "Open In Folder" functionality that you seen in firefox and download managers. This is the code that I've come up so far, and I decided to use nautilux program to open the file.
int File::openTempFile(std::string temp_file_dir)
{
std::string file_path = temp_file_dir + "/" ;
file_path = file_path + this->additional_info ;
// if there is already an temporary file then delete it//
if( temporary_file != "" )
{
// :TODO: open temporary file stack //
// so when the application dinit we could remove those //
// remove(temporary_file.c_str() );
}
/* write temporary file */
FILE* fp = fopen (file_path.c_str(), "w");
if( fp== NULL)
return FALSE;
fwrite( m_data, 1, m_size, fp);
fclose(fp);
// now open it using natulus //
char * parmList[] = {strdup("nautilus"),strdup(file_path.c_str() )} ;
int pid;
if(( pid= fork() ) == -1)
perror("fork failed");
if( pid ==0 ){
int a = execvp("nautilus" , parmList);
printf("exevp failed to load the temporary file");
}
temporary_file = file_path ;
return TRUE;
}
but the error is natulux open three windows and can't figure out where is my bug.
Any idea ?
I am trying to encrypt all the files in a particular folder and that folder has sub folders when i try to print all the files it works good but when i try to encrypt all the files it keeps on encrypting the same file again and again
void dirListFiles(wchar_t *startDir) {
HANDLE hFind;
WIN32_FIND_DATA wfd;
wchar_t path[99999];
char *enName;
const char *extension = ".enc";
int wcsChars;
wsprintf(path, L"%s\\*", startDir);
if ((hFind = FindFirstFile(path, &wfd)) == INVALID_HANDLE_VALUE) {
return;
}
do {
if ((wcsncmp(L".", wfd.cFileName, 1) !=0) && (wcsncmp(L"..", wfd.cFileName, 2) != 0) ) {
wsprintf(path, L"%s\\%s", startDir, wfd.cFileName);
if (wfd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
dirListFiles(path);
} else {
wcsChars = wcslen(path);
char *szTo = new char[wcsChars + 1];
szTo[wcsChars] = '\0';
WideCharToMultiByte(CP_ACP, 0, path, -1, szTo, wcsChars, NULL, NULL);
enName = (char *)malloc(strlen(szTo) + 1 + 4);
strcpy(enName, szTo);
strcat(enName, extension);
// If i add this line it keeps on encrypting the same file
//fencrypt(szTo, enName, (unsigned const char*)"1234567812345678");
printf("%s\n", enName);
delete[] szTo;
free(enName);
}
}
} while(FindNextFile(hFind, &wfd));
if (GetLastError() != ERROR_NO_MORE_FILES) {
FindClose(hFind);
return;
}
}
If i add fencrypt(szTo, enName, (unsigned const char*)"1234567812345678"); then it encrypts the files in the main folder that is G:\WinApp but when it enters G:\WinApp\ipch\winapp-1918e0a3 it keeps on encrypting the same file again and again there is no file in G:\WinApp\ipch\ only a folder winapp-1918e0a3 this is my encryption function please tell where am i wrong
void fencrypt(char* read, char* write, const unsigned char* enc_key) {
RAND_bytes(iv, AES_BLOCK_SIZE);
readFile = fopen(read,"rb");
writeFile = fopen(write,"wb");
fwrite(iv, 1, 8, writeFile);
fwrite("\0\0\0\0\0\0\0\0", 1, 8, writeFile);
AES_set_encrypt_key(enc_key, 256, &key);
init_ctr(&state, iv);
while(1) {
bytes_read = fread(indata, 1, AES_BLOCK_SIZE, readFile);
AES_ctr128_encrypt(indata, outdata, bytes_read, &key, state.ivec, state.ecount, &state.num);
bytes_written = fwrite(outdata, 1, bytes_read, writeFile);
if (bytes_read < AES_BLOCK_SIZE) {
break;
}
}
fclose(writeFile);
fclose(readFile);
}
I'm far from 100% sure, but I have a feeling that when you create a new file in your directory, the FindNextFile gets confused and finds the same filename again because it's not actually the same file. You may need to either collect a list of all files in a directory and process them one at a time, or keep a list of what you have already done and skip over the ones you have done.
The FindNextFile call will probably see the newly appeared encrypted file and therefore you will keep processing the new files also.
Since you are appending an ".enc" extension to your encrypted files, it would be relatively simple to skip files that already have this extension.
I am cross compiling a package (libqmi).(With simple compiling it was fine.)
The problem came out when I tried to comply a c++ part.
I got the message that is
"read is not declared".
I know it does not need to be included in case of C, but what about C++?
I tried to add the headers by hand: fcntl.h and unistd.h too, without any solution.
(the compiler found them and included, but the error message is still left)
Do you have any idea what the problem can be behind this?
I do not think the problem is wrong, as it is a realised and good with host compiler.
EDIT:
thanks the comments.
host: Linux, x86,
target: Linux, arm
unistd.h header does not solved the problem:
I also tried type alloc, maybe there is misalloc.
GobiQMICore.cpp: In member function 'virtual std::vector, std::basic_string > > cGobiQMICore::GetAvailableDevices()':
GobiQMICore.cpp:319:39: error: 'read' was not declared in this scope
GobiQMICore.cpp:334:21: error: 'close' was not declared in this scope
the code:
/*===========================================================================
METHOD:
GetAvailableQDLPorts (Public Method)
DESCRIPTION:
Return the set of available Gobi QDL ports
RETURN VALUE:
std::vector <sDeviceID>
===========================================================================*/
std::vector <std::string> cGobiQDLCore::GetAvailableQDLPorts()
{
std::vector <std::string> devices;
std::string path = "/sys/bus/usb/devices/";
std::vector <std::string> files;
DepthSearch( path,
2,
"ttyUSB",
files );
int fileNum = files.size();
for (int i = 0; i < fileNum; i++)
{
// Example "/sys/bus/usb/devices/8-1/8-1:1.1/ttyUSB0"
std::string nodePath = files[i];
int lastSlash = nodePath.find_last_of( "/" );
// This is what we want to return if everything else matches
std::string deviceNode = nodePath.substr( lastSlash + 1 );
// Move down one directory to the interface level
std::string curPath = nodePath.substr( 0, lastSlash );
// Read bInterfaceNumber
int handle = open( (curPath + "/bInterfaceNumber").c_str(),
O_RDONLY );
if (handle == -1)
{
continue;
}
char buff[4];
memset( buff, 0, 4 );
bool bFound = false;
int ret = read( handle, buff, 2 );
if (ret == 2)
{
// Interface 1 or 0
ret = strncmp( buff, "01", 2 );
if (ret == 0)
{
bFound = true;
}
ret = strncmp( buff, "00", 2 );
if (ret == 0)
{
bFound = true;
}
}
close( handle );
if (bFound == false)
{
continue;
}
// Move down one directory to the device level
curPath = curPath.substr( 0, curPath.find_last_of( "/" ) );
// Read idVendor
handle = open( (curPath + "/idVendor").c_str(), O_RDONLY );
if (handle == -1)
{
continue;
}
bFound = false;
ret = read( handle, buff, 4 );
if (ret == 4)
{
ret = strncmp( buff, "05c6", 4 );
if (ret == 0)
{
bFound = true;
}
}
close( handle );
if (bFound == false)
{
continue;
}
// Read idProduct
handle = open( (curPath + "/idProduct").c_str(), O_RDONLY );
if (handle == -1)
{
continue;
}
bFound = false;
ret = read( handle, buff, 4 );
if (ret == 4)
{
ret = strncmp( buff, "920c", 4 );
if (ret == 0)
{
bFound = true;
}
}
close( handle );
if (bFound == false)
{
continue;
}
// Success!
devices.push_back( deviceNode );
}
return devices;
}
T
I know it does not need to be included in case of C, but what about C++?
I think that you should always include the headers you need. I guess your compiler is doing the job for you, if you use the "-Wall" parameter with gcc, you should get a warning.
Under Linux, to know what header you need to include just type man function. Sometime you might get the bash man page, for open you need to indicate the section man 2 read and in the synopsis, you have the required headers. To get those man pages, you also need to install the manpages-dev on Debian based distribution.
To answer to your question, I also had that kind of issue when I was writing C++ programs using a namespace. If you are inside a namespace, try calling this function like that ::read(...)