All
I have following shell command working as per expectation on shell but not working when invoked inside perl
Shell command:
grep -P -s -irl --include \*.v "\s+hello\s?[(].*" <PATH>
working fine
Inside Perl:
$inst_search = `grep -P -s -irl --include \*.v "\s+$inst\s?[(].*" #plt_dirs`;
not working
I am suspecting i am missing something with regexp inside grep..please correct me !
Thanks,
Vivek
Perl will escape special shell characters when calling exec/system/qx (or backticks) with a string.
Try using the exec or system functions, but passing a list, e.g.
system('grep', '-P', '-s', '-irl', '--include', '\*.v', '"\s+hello\s?[(].*"', #plt_dirs);
You may also want to look at a module that does some of the error handling for you, like IPC::System::Simple.
Try this one:
$inst_search = qx#grep -P -s -irl --include \*.v "\s+$inst\s?[(].*" #plt_dirs#;
Or use any other non-alphanumeric character instead of "#"for quoting.
Related
I am exporting all AWS ElasticBeanstalk environment variables and piping the output to serve as command line arguments using xargs.
export $(/opt/elasticbeanstalk/bin/get-config --output YAML environment | sed -r 's/: /=/' | xargs)
Most strings in the YAML output from get-config are not quoted, so the above command snippet breaks when it encounters any environment variable with a space in its value, because the export command uses spaces to delimit the beginning of a new key-value pair.
For example, say I have an environment variable called TEST_VARIABLE with a value of THIS STRING. The above command fails with an error:
-bash: export: `THIS STRING': not a valid identifier
In essence, my question is, how do I get /opt/elasticbeanstalk/bin/get-config --output YAML environment to quote all strings?
I'll be using below env.yaml file as example (I don't use AWS BS so I don't know if there would be a heavy syntax difference), next time please provide an edited example:
env.yaml
env1: this the 1st
env2: this the 2nd
In any case, piping to xargs will be hard for quotes to be kept (as they'd end up being interpreted by a shell, but then you'd need to re-quote them).
You should instead try to produce equivalent (several) export lines, to be consumed by the running shell, something on the lines of source <( output with several export x="..." lines) (valid syntax for bash, zsh, and maybe others).
Pasting below two possibilities:
using only sed
Below solution works (I chose single quotes), on the assumption that there's no single-quoted value.
$ sed -E "s/(.+): (.+)/export \1='\2'/" env.yaml
export env1='this the 1st'
export env2='this the 2nd'
$ source <(sed -E "s/(.+): (.+)/export \1='\2'/" env.yaml)
$ env|egrep ^env
env1=this the 1st
env2=this the 2nd
using yq
Use https://github.com/kislyuk/yq for the needed quoting then sed for the : substitution:
$ yq '.. style="single"' env.yaml|sed -e 's/^/export /' -e 's/: /=/'
export env1='this the 1st'
export env2='this the 2nd'
$ source <(yq '.. style="single"' env.yaml|sed -e 's/^/export /' -e 's/: /=/')
$ env|egrep ^env
env1=this the 1st
env2=this the 2nd
I am currently running the gsutil rsync cloud build command:
gcr.io/cloud-builders/gsutil
-m rsync -r -c -d -x "\.gitignore" . gs://mybucket/
I am using the -x "\.gitignore" argument here to try and not copy over the .gitignore file, as mentioned here:
https://cloud.google.com/storage/docs/gsutil/commands/rsync
However, when looking in the bucket and the logs, it still says:
2021-04-23T13:29:37.870382893Z Step #1: Copying file://./.gitignore [Content-Type=application/octet-stream]...
So rsync is still copying over the file despite the -x "\.gitignore" argument.
According to the docs -x is a Python regexp, so //./.gitignore should be captured by \.gitignore
Does anyone know why this isn't working and why the file is still being copied?
See the rsync.py source code:
if cls.exclude_pattern.match(str_to_check):
In Python, re.match only returns a match if it occurs at the start of string.
So, in order to find a match anywhere using the -x parameter, you need to prepend the pattern you need to find with .* or with (?s).*:
gcr.io/cloud-builders/gsutil
-m rsync -r -c -d -x ".*\.gitignore" . gs://mybucket/
Note that to make sure .gitignore appears at the end of string, you need to append $, -x ".*\.gitignore$".
I am trying to put together a simple script that does a regex on a git clone output and captions the directory that the default clone cmd is cloning too and then captures this into a shell for or something that can then be piped into xargs?
E.g:
git clone git#github.com:thorchain/instaswap-sdk.git |& grep "\'\S*\'" | xargs cd
In this example grep is not the right tool as its output is the matching line and not the match. However, for the life of me, I can't seem to find a simple regex matching tool?
Thanks in adv
Luke
Turns out the missing bit was the grep -o that I needed. What I was trying to do was built a bash function the git cloned and cd into the repos directory.
End solution here:
function gitc() {
cd `git clone "$#" |& grep -o "\'\S*\'" | tr -d "'"`
}
So I am writing a few scripts for migrating SVN to GIT, we have a bunch of "old" branches in SVN that still exist but don't need to be moved to GIT. (Branches which happened to have already been merged to trunk).
After a bit of google-fu I've come up with the following:
$(git for-each-ref --format='%(refname:short)' --merged origin/trunk | grep '(?!origin\/trunk)origin\/.*')
To be passed to
git branch -D --remote _previouscommandgoeshere_
If I run just git for-each-ref --format='%(refname:short)' --merged origin/trunk I get the following output:
origin/IR1091
origin/IR1102
origin/IR1105
...
origin/IR932
origin/Software
origin/trunk
origin/trunk#6792
origin/trunk#6850
When I add the grep command I get 0 values.
However, https://regexr.com/3ot1t has thaught me that my regexp is doing exactly what I want to do. Remove all branches except for the trunk branch.
What is wrong with the regexp/grep? (note I am not a linux/grep guru. This is all done in bash that comes with windows git)
The regexp is right, but grep by default does not support PCRE expression constructs like Negative look-ahead (?!. You need to enable the -P flag to enable the PCRE library, without that it just supports the Basic Regular Expression engine
.. | grep -oP '(?!origin\/trunk)origin\/.*'
Or use a perl regex match on the command line for which no flags need to be set up
.. | perl -ne 'print if /(?!origin\/trunk)origin\/.*/'
grep -P 'origin/(?!trunk)'
just this, can match what your wanted
I am new to Bamboo Atlassian environment. I have a question regarding implement Bash script under plan/branch on Bamboo.
I am trying to run a regex inside script stage but I am getting an error:
/tmp/SW-2636-ScriptBuildTask-4921335221935380637.sh: [[: not found
My code:
if [[ ${bamboo.planRepository.branchName} =~ [0-9]+\.[0-9]+R ]]; then
do Blah Blah
else
do something else
fi
I have also tried with singe [] instead of [[ ]] but didn't get.
I ran this script independently as a Bash script and its running fine. Unable to understand how to add regex inside if condition over Bamboo.
Any suggestion/example would be helpful
This will make a trick:
if [ "$(ps -p "$$" -o comm=)" != "bash" ]; then
bash "$0" "$#"
exit "$?"
fi
Add this script at the top of your script to make Bamboo Bash compatible.