In D programming, how can I iterate through all files in a folder?
Is there a D counterpart to python's glob.iglob?
http://dlang.org/phobos/std_file.html#dirEntries
So like
import std.file;
foreach(string filename; dirEntries("folder_name", "*.txt", SpanMode.shallow) {
// do something with filename
}
See the docs for more info. The second string, the *.txt filter, is optional, if you leave it out, you see all files.
The SpanMode can be shallow to skip going into subfolders or something like SpanMode.depth to descend into them.
Take a look at std.file.dirEntries. It will allow you to iterate over all of the files in a directory either shallowly (so it doesn't iterate any subdirectories), with breadth-first search, or with depth-first search. And you can tell it whether you want it to follow symlinks or not. It also supports wildcard strings using std.path.globMatch. A basic example would be something like
foreach(DirEntry de; dirEntries(myDirectory, SpanMode.shallow))
{
...
}
However, because dirEntries returns a range of DirEntrys, it can be used in the various range-based functions in Phobos and not just with foreach.
Related
I just noticed that std::filesystem::recursive_directory_iterator uses different path separateors (i.e. / vs \) depending on whether it's on Windows or Linux, is there a way to make it return paths with '/' to make it consistent across systems?
This is how I am getting the paths:
for(auto& path: fs::recursive_directory_iterator(dir_path))
{
// Skip directories in the enumeration.
if(fs::is_directory(path)) continue;
string path_str = path.path().string();
}
What I mean is, the contents of path_str will be different between the two OSs (because the separators will be different), I would like them to be the same. I could just replace them on the final string, but that uses more cycles than if I can instruct the stl to use '/' for everything isntead.
So, your problem has nothing to do with recursive_directory_iterator, which iterates on directory_entry objects, not paths. Your confusion probably stems from the fact that directory entries are implicitly convertible to paths, so you can use them as such.
Your problem is really about path::string(), which, as the documentation states, uses the native format (i.e. with a platform dependent separator). You would get the same problem regardless of how you get your path.
If you want to get / as the directory separator, use path::generic_string() instead to get the path in generic format.
for(auto& dir_entry: fs::recursive_directory_iterator(dir_path))
{
if(dir_entry.is_directory()) continue;
string path_str = dir_entry.path().generic_string();
}
How can I list only files, but not directories using list.files (not recursively)? It has an include.dirs argument, but this is ignored when not being used recursively.
I had been thinking something like
list.files(path=myDir, pattern="[^/]$")
but that doesn't seem to work, or a few variations on it. Is there a regex that I can plug in here or a function. I know I can do list.dirs and take a setdiff, but this is already slow enough, I want this to be quicker.
PS: currently on linux, but need something that works cross-platform.
PPS: file.info is really slow, so I think that is also not going to work.
PPPS: It doesn't need to be list.files, that is just the function I had thought should do it.
Consider this regex pattern that matches any file containing letters or numbers and contains the dot extension (to leave out subdirectories but unfortunately files without extensions):
# WITH ANCHORING
files <- list.files(path, pattern=("[a-zA-Z0-9]*[.][a-zA-Z0-9]*$"))
# MATCHING LETTER AND/OR NUMBER FILES WITH EXTENSION
files = list.files(myDir, pattern=("[a-zA-Z0-9]*[.]"))
# WILDCARD FILE MATCHING WITH EXTENSION
files = list.files(myDir, pattern=("*[.]"))
Some other regex variations to catch files with periods (note these also get directories with periods and miss files with no extensions)
list.files(pattern="\\..+$")
list.files(pattern="\\.[[:alnum:]]+$")
And using system2 with ls seems to work pretty well (thanks #42- as well from comments),
system2("ls", args=c("-al", "|", "grep", "^-"))
should get only regular files (including ones without extensions), or
system2("ls", args=c("--classify"))
should return files with directories having a "/" appended so they can be determined.
For an alternative open-source solution, consider the Python solution that allows you to condition if item is a directory and using os.path.join() is agnostic to any OS platform.
import os
files = [f for f in os.listdir(myDir) if os.path.isfile(os.path.join(myDir, f))]
I need a nice column for Centrify tool which include all the log files under the different folders, for example;
/oradata1/oracle/admin/A/scripts/rman_logs/*.log
/oracle/oracle/admin/B/scripts/rman_logs/*.log
/oradata2/admin/C/scripts/logs/*.log
I used this but after the * character user can see all logs;
/ora(data(1|2)|cle)/oracle|admin/admin/*/scripts/rman_logs
/ora(data(1|2)|cle)/oracle|admin/admin/*/scripts/rman_logs
Which expression I must use.
If I understandy our question correctly, you want only .log files. You can use a positive lookahead to assert that it is indeed a log file (contains .log at the end of filename), and match the filename whatever it is (.*).
Then it's really easy. (?=.*\.log(?:$|\s)).* Of course, you can also add specific folders if you wish to restrict the matches, but the positive lookahead will still do its work. I.e. (?=.*\.log(?:$|\s)).*/scripts/.*
EDIT: As your comment, you only need those folders, so you just specify their filepaths in alternations and add [^.\s\/]*\.log at the end. So:
(?:\/oradata1\/oracle\/admin\/A\/scripts\/rman_logs\/|\/oracle\/oracle\/admin\/B\/scripts\/rman_logs\/|\/oradata2\/admin\/C\/scripts\/logs\/)[^\s.\/]*\.log You may shorten the regex by trying to combine filepath elements, but, imo, not necessary as you might as well specify each filepath individually, if they don't overlap too much.
I have found a global expression.
this is not a good way but it works and save me from lots of job. The main files are under the ....../scripts/rman_logs/ for all servers so I use this way.
I can produce these lines and can be a command group for users so this works good
tail /////scripts/rman_logs/*.log
tail ////scripts/rman_logs/.log
Thanks for your helps.
Is there any way to open files which have common name beginning without specifying their full names and amount? They should be open one by one, not all at once.
For example, I have files:
certainstr_123.txt, certainstr_5329764.txt, certainstr_1323852.txt.
Or, maybe, it can be more easily done in some another language?
Thank you.
C++ doesn't define a standard way for listing files in that way.
The best cross platform approach is to use a library such as the boost filesystem module. I don't think boost::filesystem has wildcard search, you have to filter files yourself but it isn't difficult.
You could use regular expressions, like in the other answer (it's the perfect-fit solution).
Probably it could be enough to check file extension (i->path().extension()) and filename starting with "certainstr_" (boost::starts_with or std::string::substr).
If you choose C++11 standard regex library make sure to have a recent libstdc++.
There are a lot of system specific functions. E.g. see:
How can I get the list of files in a directory using C or C++?
How do I get a list of files in a directory in C++?
for some Unix/Windows examples.
You could also try something like (Windows):
std::system("dir /b certainstr_*.txt > list.txt");
or (Unix):
std::system("ls -m1 certainstr_*.txt > list.txt");
parsing the list.txt output file (of course this is a hack).
Anyway, depending on your needs, Python-based (or script-based) solutions could be simpler (see also How to list all files of a directory?):
[glob.glob('certainstr_*.txt')][3]
or also:
files = [f for f in os.listdir('.') if re.match(r'certainstr_\d+.txt', f)]
This is the Python equivalent of https://stackoverflow.com/a/26585425/3235496
I suggest just using a regular expression. Pseudo code:
boost::regex reg("^certainstr_\\d+.txt$");
for(recursive_directory_iterator it("."); it != recursive_directory_iterator(); ++it)
{
if(boost::regex_search(it->string(), reg))
{
cout << *it << endl;
}
}
I'm doing some work involving some automated file moving, and these files contain relative paths that must be maintained. Unfortunately, I'm finding the facilities offered by System.IO.Path, System.String, and Powershell's operators to be a little ill-equipped to handle my work gracefully.
One function that would be very useful to me is the notion of a subtraction of paths, that would work in theory like subtracting vectors. Conceptually, A - B gets you a path from B to A. In the application to paths, D:\A\B\C\D - D:\A\B\ = \C\D. Likewise, D:\A\B\ - D:\A\B\C\D = \..\.. in this case. I can accept, for now, that this only makes sense when one path is wholly contained in the other.
This seems to consist of two steps: 1) determine containment of one path in the other. 2) remove the contained path from the containing path. 3) Optionally, replace folder names with the parent .. symbol based on the sidedness of the operation.
As I am concerned with NTFS, I need both containment and replacement operations to be case-insensitive. For containment, I can use select-string since it is case-insensitive, and allows the -simple switch which allows me to use a path without hacking it apart to escape them for regex.
Removing the string from the other is a little more annoying though. System.IO.Path has nothing for this, System.String's pertinent methods are all case-sensitive, and powershell's operators all require massaging so that the regex will match things.
All this seems like more work than it should be--are there any tools I'm missing that would better handle this?
Determine containment - convert your paths to absolute paths (if not already). You can use Resolve-Path for this. Then you can use $path1.StartsWith($path2, 'OrdinalIgnoreCase') to test for containment.
Remove contained path - $path1.Substring($path2.length)
Replace parent folder names with ... - although I don't have the regex off the top of my head, I'm pretty sure you could do this with a regular expression search/replace using PowerShell's -replace operator
filedirectorypath, on CodePlex, may offer what you need
It's not a PowerShell specific API, but that's no reason not to use it from PowerShell.
Benefits of the NDepend.Helpers.FilePathDirectory over the .NET Framework class System.IO.Path include:
Strongly typed File/Directory path.
Relative / absolute path conversion.
Path normalization API
Path validity check API
Path comparison API
Path browsing API.
Path rebasing API
List of path operations (TryGetCommonRootDirectory, GetListOfUniqueDirsAndUniqueFileNames, list equality…)