Python glob.glob to work with optional subdirectories? - regex

Is there a way to make glob.glob (not glob2) search for files in optional subdirectories in Python 2.7?
I want to search for files ending with "_stats.txt", in these two paths:
/starting_path/Data/Intensities/BaseCalls/Primary_Analysis_Results/results/FASTQ_1mm_currentDate/Project_1/trimming_currentDate/cutadapt_S1_stats.txt
/starting_path/Data/Intensities/BaseCalls/Primary_Analysis_Results/results/FASTQ_1mm_currentDate/trimming_currentDate/cutadapt_S1_stats.txt
As you can see, the "Project_1" subdirectory doesn't always exist in file paths. For the moment I have tried with the following code:
stats_paths=glob.glob("/starting_path/Data/Intensities/BaseCalls/Primary_Analysis_Results/results/FASTQ_*/**/trimming_*/*_stats.txt")
but it only works when the "Project_1" subdirectory exists. When it is not in the path I get an empty list.
Thanks in advance!

Related

How can I ignore some directories in Doxygen? [duplicate]

I want Doxygen to ignore, bypass, not search the following directories of my project:
*/.svn/*
*/docs/*
*/Properties/*
*/bin/*
According to the Doxygen FAQ:
How can I exclude all test directories from my directory tree?
Simply put an exclude pattern like this in the configuration file:
EXCLUDE_PATTERNS = */test/*
So, my Doxygen file looks like this:
# If the value of the INPUT tag contains directories, you can use the
# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude
# certain files from those directories. Note that the wildcards are matched
# against the file with absolute path, so to exclude all test directories
# for example use the pattern */test/*
EXCLUDE_PATTERNS = */.svn/* \
*/docs/* \
*/published/* \
*/bin/* \
*/obj/*
The output from Doxygen is:
Searching for include files...
Searching for example files...
Searching for images...
Searching for files in directory c:/Test_Fixtures/pc_application/docs
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/tmp
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/tmp/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/tmp/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/.svn/tmp/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/tmp
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/tmp/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/tmp/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/.svn/tmp/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/tmp
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/tmp/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/tmp/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/html/search/.svn/tmp/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/text-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/tmp
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/tmp/prop-base
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/tmp/props
Searching for files in directory c:/Test_Fixtures/pc_application/docs/rtf/.svn/tmp/text-base
Which clearly shows that the EXCLUDE_PATTERNS is not working.
I also have the EXCLUDE option set as:
# The EXCLUDE tag can be used to specify files and/or directories that should
# excluded from the INPUT source files. This way you can easily exclude a
# subdirectory from a directory tree whose root is specified with the INPUT tag.
EXCLUDE = ../.svn \
../docs \
../published \
../docs/.svn \
../docs/.svn
Which is not working either.
So, the USD$ 100000 question is: How do I make Doxygen exclude, bypass, ignore, stay clear of specified directories or folders (preferably something that works)?
I am using the Doxygen Wizard on Windows 7 on 64-bit CPU platform, and Doxygen 1.7.3.
Note that the wildcards are matched against the file with absolute path
So, just use absolute paths for your exclusions ;)
PS: BTW, I have struggled too many times with that. This small mention on the doxyfile's comment seems to go unnoticed too often.
Oops, I missed the detail that you had already tried that. Maybe it's an issue with the multi-line value: try inlining all the paths, using just a space as separator. That (plus absolute patterns) is enough on the few systems I have been using doxygen lately.
I have done some deeper testing, and also taken another look at the doxyfile's documentation.
The correct syntax is using space for separation. If you really want to go multi-line, the supported and documented syntax would be:
EXCLUDE_PATTERNS = */.svn/*
EXCLUDE_PATTERNS += */docs/*
EXCLUDE_PATTERNS += */published/*
# and so on
Also, take a closer look at how exclude patterns work: the directory itself is included, then everything within it will be tested against the exclude patterns and (since it will always match), be excluded on a file-per-file basis.
So take a closer look at your output: the Searching for files in directory lines are supposed to be there (doxygen will search the directory, but find nothing on it because everything is being excluded); are you getting Parsing code for file or Generating docs for for any of the contents on those directories? If you don't get any of those, this means that everything is working fine (directories are searched, but nothing on them is included). If the files are indeed being included, give the space separation or the += syntax a try. I see nothing on the docs even hinting that your \ syntax could work (of course, I may have overlooked something).
How do I make Doxygen exclude, bypass, ignore, stay clear of specified directories or folders (preferably something that works)?
Use the INPUT configuration option!
INPUT = src other_folder README.md
As mentioned, exclude patterns will still consider the files in the directories, and not parse them based on it matching the regex. Specifying folders and files to be considered will achieve the desired result of not even searching the folders in the first place, which can dramatically improve generation time.
Note that this needs the EXCLUDE directive to be empty, or the searching will happen.
Precisions from David C.'s comment:
More precisely, adding a directory to the EXCLUDE directive causes everything in that directory to be searched. You can exclude specific files (that would otherwise by hit by the INPUT directive) without blowing up the search process.
Note that I can see a similar phenomenon when I use doxygen. However, that happens when the tool searches for example files and images:
Searching for example files...
Searching for files in directory .../wpkg/mainline/documentation
Searching for files in directory .../wpkg/mainline/documentation/.svn
Searching for files in directory .../wpkg/mainline/documentation/.svn/prop-base
[...]
Searching for images...
Searching for files in directory .../wpkg/mainline/documentation
Searching for files in directory .../wpkg/mainline/documentation/.svn
Searching for files in directory .../wpkg/mainline/documentation/.svn/prop-base
[...]
As you have the the EXCLUDE_PATTERNS setup to ignore the .svn sub-directories. I suppose that's a bug in doxygen which should check for those exclusions when scanning for examples and images.
Also it looks like it could be that it prints all the directories on the screen, but properly ignores them as before using them it checks the exclusion patterns. But that is just a guess; although it seems someone says that's the way it works here:
http://doxygen.10944.n7.nabble.com/EXCLUDE-DIRECTORY-PATTERN-td2185.html

How to use the exclude_files regex in cpplint?

I am using cpplint to check my sourcode agains the google style guide.
Cpplint's help says:
cpplint.py supports per-directory configurations specified in CPPLINT.cfg
files. CPPLINT.cfg file can contain a number of key=value pairs.
Currently the following options are supported:
"exclude_files" allows to specify a regular expression to be matched against
a file name. If the expression matches, the file is skipped and not run
through liner.
Example file:
filter=-build/include_order,+build/include_alpha
exclude_files=.*\.cc
The above example disables build/include_order warning and enables
build/include_alpha as well as excludes all .cc from being
processed by linter, in the current directory (where the .cfg
file is located) and all sub-directories.
How I use cpplint:
I use cpplint by this command to check all files in my source folder:
cpplint src/*.c
Well there is one special file foo.cc which must not be checked. So I tried to create a CPPLIN.cfg to use the exclude_files property. My file looks like this:
set noparent
filter=-build/include_dir
exclude_files=foo.cc
Nevertheless foo.cc is still checked.
What I have already tried to do:
I tried exclude_files=/.*\.cc/. This should exclude all files ending with *.cc. Nevertheless all files are still checked.
I tried to remove my filter from the file. This caused more errors than before. So I am now sure that my CPPLINT.cfg file is found by cpplint.
Question:
How to use the exclude_files regex in cpplint correctly?
Turns out apparently that the doc is wrong: exclude_files only excludes files in the same directory as CPPLINT.cfg, not in subdirectories. See https://github.com/google/styleguide/issues/220
So the solution would be to create src/CPPLINT.cfg and put exclude_files=.*\.cc in it.

Git - Ignore directories based on their contents

Using .gitignore, is there a way to ignore a directory if it contains a certain file (or directory)?
This would be something like look-ahead assertions, though my use case is a little different: I want to ignore Mercurial repos in my project, to keep from accidentally committing them as part of the project. That is, I want to ignore all directories containing .hg, not just .hg itself.
I can work around this using the answer from this question, adding each directory name to .gitignore, but I'd like to make it more general if I can.
There is no way to do it beside adding all of them to your .gitignore file.
What you can do it to write a scipt which append all the desired paths to your .gitignore.
The content of .gitignore is alist of paths so git can be configured based upon content.
Each line in a gitignore file specifies a pattern

Why doesn't Xcode6 see included header files

I'm trying to build opencv2 as a universal framework. I am systematically removing the files/folders that I do not need. But I am running into this issue where the include files are not found. See the image below:
The following image clearly shows that the file is indeed there.
One of the contractors working with us said he had put the include files into the same directory as the source files and rename them according to their file structure but using "." instead of "/" as shown below:
But that means that I must go through all of the files that include files and change the include statement to use "." instead of "/". REALLY?
Is this true? Or do we have a configuration set wrong?
You need to setup search paths for your target in Build Settings->Search Paths->Header search paths.

How to hgignore all files of a particular extension except in one directory and its subdirectories?

I would like to use the .hgignore file of Mercurial to ignore all files with file extension .tex, except those .tex files in one particular directory and whatever subdirectory of this directory.
I presume syntax: regexp will be required for this.
A brief explanation of the particular regular expression used, would also be very welcome, so that we can all learn a bit here.
Let's say you want to exclude the directory named exclude. The following regex would then match all files that end in .tex unless exclude/ comes somewhere before that:
^(?!.*\bexclude/).*\.tex$