Is there a jest equivalent, or a way of doing it in bash, to the git regex approach for including files that match one regex (all files in the /tests folder), but then exclude any that match a second (any test files that include API in the name)? e.g.
git ls-files -- 'tests' ':!:*API*'
I would like to do it as single line command because I want to be able to run all the tests in the /tests directory, but in two mutually exclusive sets using a single yarn test script from my package.json file.
EDIT: Expanding on Brenda J. Butler's answer:
yarn jest `find tests -type f ! -name \*API\* | grep -v .DS_`
Realising that jest can take either a regex OR a file list,
using backticks to do a bash substitution, and
additionally, as this is on MacOS, using a grep -v to also pull out the .DS_Store from the file list.
bash:
# finding files under tests whose names don't contain API
$ find tests -type f ! -name \*API\*
# finding files under tests whose names contain API
$ find tests -type f -name \*API\*
Related
I'm trying to remove all of the folder meta files from a unity project in the git repo my team is using. Other members don't delete the meta file associated to the folder they deleted/emptied and it's propagating to everyone else. It's a minor annoyance that shouldn't need to be seen so I've added this to the .gitignore:
*.meta
!*.*.meta
and now need to remove only the folder metas. I'd rather remove the metas now than wait for them to appear and have git remove them later. I'm using git bash on Windows and have tried the following commands to find just the folder metas:
find . -name '*.meta' > test.txt #returns folders and files
find . -regex '.*\.meta' > test.txt #again folders and files
find . -regex '\.[^\.]{0,}\.meta' > test.txt #nothing
find . -regex '\.[^.]{0,}\.meta' > test.txt #nothing
find . -regex '\.{2}' > test.txt #nothing
find . -regex '(\..*){2}' > test.txt #nothing
I know regex is interpreted differently per program/language but the following will produce the results I want in Notepad++ and I'm not sure how to translate it for git or git bash:
^.*/[^.]{0,}\.meta$
by capturing the lines (file paths from root of repo) that end with a /<foldername>.meta since I realized some folders contained a '.' in their name.
Once this is figured out I need to go line by line and git rm the files.
NOTE
I can also run:
^.*/.*?\..*?\.meta$\n
and replace with nothing to delete all of the file metas from the folders and files result, and use that result to get all of the folder metas, but I'd also like to know how to avoid needing Notepad++ as an extra step.
To confine the results only to indexed files use git ls-files, the swiss-army knife of index-aware file listing. git update-index is the core-command index munger,
git ls-files -i -x '*.meta' -x '!*.*.meta' | git update-index --force-remove --stdin
which will remove the files from your index but leave them in the work tree.
It's easier to express with two conditions just like in .gitignore. Match *.meta but exclude *.*.meta:
find . -name '*.meta' ! -name '*.*.meta'
Use -exec to run the command of your choice on the matched files. {} is a placeholder for the file names and ';' signifies the end of the -exec command (weird syntax but it's useful if you append other things after the -exec ... ';').
find . -name '*.meta' ! -name '*.*.meta' -exec git rm {} ';'
I have a list of file in my project:
For example:
1. src/index.1.js
2. src/screens/index.1.js
3. src/screens/index.2.js
I want to ignore all the files having the numeric number.
I have tried using **/*.1.* , **/*.2.*. Is there a way to ignore all the file with numeric value?
You can use a range. For your example:
**/*.[0-9].js
Would match a js file in any directory that ends with .(number).js
Git uses glob pattern to match ignored files. Use the following to ignore all such above-mentioned files (with multi-digit numbers also).
**/*.[0-9]*.js
Why don't you run the following find command after eventually adapting the \.js part if you do not want to take into account only the .js files:
find . -type f -regextype sed -regex '.*\/.*\.[0-9]\+\.js'
./src/screens/index.2.js
./src/screens/index.123.js
./src/index.1.js
when you find all the files you are interested in, change your find command into:
find . -type f -regextype sed -regex '.*\/.*\.[0-9]\+\.js' -exec git checkout {} \;
to checkout those files.
Let's consider I have files located in different subfolders and I would like to search, test and replace something into these files.
I would like to do it in three steps:
Search of a specific pattern (with or without regexp)
Test to replace it with something (with or without regexp)
Apply the changes only to the concerned files
My current solution is to define some aliases in my .bashrc in order to easily use grep and sed:
alias findsrc='find . -name "*.[ch]" -or -name "*.asm" -or -name "*.inc"'
alias grepsrc='findsrc | xargs grep -n --color '
alias sedsrc='findsrc | xargs sed '
Then I use
grepsrc <pattern> to search my pattern
(no solution found yet)
sedsrc -i 's/<pattern>/replace/g'
Unfortunately this solution does not satisfy me. The first issue is that sed touch all the files even of no changes. Then, the need to use aliases does not look very clean to me.
Ideally I would like have a workflow similar to this one:
Register a new context:
$ fetch register 'mysrcs' --recurse *.h *.c *.asm *.inc
Context list:
$ fetch context
1. mysrcs --recurse *.h *.c *.asm *.inc
Extracted from ~/.fetchrc
Find something:
$ fetch files mysrcs /0x[a-f0-9]{3}/
./foo.c:235 Yeah 0x245
./bar.h:2 Oh yeah 0x2ac hex
Test a replacement:
$ fetch test mysrcs /0x[a-f0-9]{3}/0xabc/
./foo.c:235 Yeah 0xabc
./bar.h:2 Oh yeah 0xabc hex
Apply the replacement:
$ fetch subst --backup mysrcs /0x[a-f0-9]{3}/0xabc/
./foo.c:235 Yeah 0xabc
./bar.h:2 Oh yeah 0xabc hex
Backup number: 242
Restore in case of mistake:
$ fetch restore 242
This kind of tools look pretty standard to me. Everybody needs to search and replace. What alternative can I use that is standard in Linux?
#!/bin/ksh
# Call the batch with the 2 (search than replace) pattern value as argument
# assuming the 2 pattern are "sed" compliant regex
SearchStr="$1"
ReplaceStr="$2"
# Assuming it start the search from current folder and take any file
# if more filter needed, use a find before with a pipe
grep -l -r "$SearchStr" . | while read ThisFile
do
sed -i -e "s/${SearchStr}/${ReplaceStr}/g" ${ThisFile}
done
should be a base script to adapt to your need
I often have to perform such maintenance tasks. I use a mix of find, grep, sed, and awk.
And instead of aliases, I use functions.
For example:
# i. and ii.
function grepsrc {
find . -name "*.[ch]" -or -name "*.asm" -or -name "*.inc" -exec grep -Hn "$1"
}
# iii.
function sedsrc {
grepsrc "$1" | awk -F: '{print $1}' | uniq | while read f; do
sed -i s/"$1"/"$2"/g $f
done
}
Usage example:
sedsrc "foo[bB]ar*" "polop"
for F in $(grep -Rl <pattern>) ; do sed 's/search/replace/' "$F" | sponge "$F" ; done
grep with the -l argument just lists files that match
We then use an iterator to just run those files which match through sed
We use the sponge program from the moreutils package to write the processed stream back to the same file
This is simple and requires no additional shell functions or complex scripts.
If you want to make it safe as well... check the folder into a Git repository. That's what version control is for.
Yes there is a tool doing exactely that you are looking for. This is Git. Why do you want to manage the backup of your files in case of mistakes when specialized tools can do that job for you?
You split your request in 3 subquestions:
How quickly search into a subset of my files?
How to apply a substitution temporarly, then go back to the original state?
How to substitute into your subset of files?
We first need to do some jobs in your workspace. You need to init a Git repository then add all your files into this repository:
$ cd my_project
$ git init
$ git add **/*.h **/*.c **/*.inc
$ git commit -m "My initial state"
Now, you can quickly get the list of your files with:
$ git ls-files
To do a replacement, you can either use sed, perl or awk. Here the example using sed:
$ git ls-files | xargs sed -i -e 's/search/replace/'
If you are not happy with this change, you can roll-back anytime with:
$ git checkout HEAD
This allows you to test your change and step-back anytime you want to.
Now, we did not simplified the commands yet. So I suggest to add an alias to your Git configuration file, usually located here ~/.gitconfig. Add this:
[alias]
sed = ! git grep -z --full-name -l '.' | xargs -0 sed -i -e
So now you can just type:
$ git sed s/a/b/
It's magic...
In a bash script I would like to parse the names of all subdirectories and find all subdirectories that have a matching string, but I do not want subdirectories of a match. I am interested in automating construction of my $PATH and $PYTHONPATH variables based on directory structure.
Here's an example:
Let's say I want to go through my ~/dev and ~/bin folders and find all subdirectories with bin/ which holds programs that I will want to run at the shell. I can get a list with
$ ls -lR $HOME/bin $HOME/dev |grep "\/" | grep "bin:"
/Users/dat5h/bin:
/Users/dat5h/bin/project/bin:
...
These can all be appended to $PATH and have all available scripts ready to run.
BUT, let's say I was searching for directories with python modules and packages to add to $PYTHONPATH. I could conceivably look for all directories that start with /py-. So, I try:
$ ls -lR $HOME/bin $HOME/dev |grep "\/" | grep "/py-"
/Users/dat5h/bin/py-test:
/Users/dat5h/bin/py-test/test-package:
/Users/dat5h/bin/py-test/test-package/nested-test:
...
My thinking is that I would not want to put package directories and subdirectories into the path. I'm pretty sure that would be strange, but I am actually new to python so suggestions would be helpful. How would I go about constructing a test case to only get directories with py-* but non of the subsequent subdirectories?
I tried:
$ ls -lR $HOME/bin $HOME/dev |grep "\/" | egrep "/py-.*[^/]:"
But this doesn't get the job done either. Maybe a better regex? Any help would be greatly appreciated!
SOLUTION
The solution I ended up satisfied with was the find suggested below with a cutoms regex:
find $HOME/bin $HOME/dev -type d -regex ".*\/py\(\w\|-\w\)*"
This will find all subdirectories of ~/bin and ~/dev that are some variant of "pySOMETHING", "py-SOMETHING", "pySOME_THING_ELSE", or "py-SOME_THING_ELSE" but does not grab any subdirectories of those unless they also match this string. This ensures that I can have some simple naming convention for all of my directories with python modules/packages and import them this way without accidentally being able to import nested packages without the hierarchy.
Does this:
find -type d -regex ".*py.*bin.*"
give you some start?
I'd like to do some code refactoring in vim. I have found the following gem to apply transformations to all buffers.
:dobuf %s/match/replace/gc
My code is layed out with the root directory having a directory for the dependencies and a build directory. I want to load all .cc , .h and .proto files from ./src ./include and ./tests. But not from the dependencies and build directories, into background/hidden buffers. I want to do this to do the refactor using the command above.
If someone knows of a cleaner way to perform the use case, please show it.
Note: I know you can string together find and sed to do this from the shell, however I prefer doing it in vim , if at all possible. The /gc prefix in the pattern I presented above serves the role of confirming replacements on each match, I need this functionality as often I don't want to replace certain matches, the find and sedsolution is too restrictive and finicky when attempting my use-case, it is also easy to destroy files when doing in-place replacements.
For reference using sed and find:
List candidate replacements:
find src include tests -name *.h -or -name *.cc -or -name *.proto|
xargs sed -n 's/ListServices/list_services/p'
Perform replacements:
`find src include tests -name *.h -or -name *.cc -or -name *.proto|
xargs sed -i 's/ListServices/list_services`'
You can use :argadd to add the files you need to vim's argument list. This will load them as inactive buffers (you can see them afterwards with an :ls. In your case, it might look like this:
argadd src/**/*.cc
argadd src/**/*.h
argadd src/**/*.proto
And so on, for the include and tests directories. You might want to make a command for that or experiment with glob patterns to make it a bit simpler. Afterwards, your command should work, although I'd recommend running it with :argdo instead:
argdo %s/match/replace/gc
This will only execute it for the buffers you explicitly specified, not for any of the other ones you might have opened at the time. Check :help argadd and :help argdo for more information.