I am having trouble understanding what this command does and more importantly the significance of both the $s in the command. I started by making a file foo and filled it with words like SHELL, WELL, etc. I then tried the command, however it did not print anything, so I thought that maybe something was wrong with my terminal. See my terminal below:
ifsmy12#cloudshell:~/3300/tests/t2 (cs3300-301722)$ cat foo
SHELL
HELL
WELL
SHELL
SWELL
DWELL
FAREWELL
ifsmy12#cloudshell:~/3300/tests/t2 (cs3300-301722)$ grep "$SHELL$" foo
ifsmy12#cloudshell:~/3300/tests/t2 (cs3300-301722)$
I then decided to play around with the command by first trying it without the first $ and then doing the same thing except without the second $. My results are below:
ifsmy12#cloudshell:~/3300/tests/t2 (cs3300-301722)$ grep "$SHELL" foo
ifsmy12#cloudshell:~/3300/tests/t2 (cs3300-301722)$ grep "SHELL$" foo
SHELL
SHELL
I got some results but I am still confused as to what each $ does and why the command does not work with both in place. Can someone explain?
$SHELL expands to the path of your currently executing shell. $ without text after it does not expand and stays unaltered.
If executed in the Bash shell, your final command would look like:
grep '/bin/bash$' foo
$ as part of a regular-expression – which is what a grep pattern is – means "end of line".
You can verify it by prepending your command with echo, or enabling command tracing in your shell: set -x
All that said, using $ unescaped and expecting it to not expand is bad practice (in other words: waiting for surprising stuff to happen). Better to be explicit where you want shell expansion to be performed and where not:
grep "$SHELL"'$' foo
grep "$SHELL\$" foo
Related
This seems like a relatively basic question, but I can't find it anywhere after an hour of searching. Many (there are a lot!) of the similar questions do not seem to hit the point.
I am writing a script ("vims") to use vim in a sed-like mode (so I can call normal vim commands on a stream input without actually opening vim), so I need to pass each argument to vim with a "-c" flag prepended to it. There are also many characters that need to be escaped (I need to pass regex expressions), so some of the usual methods on SO do not work.
Basically, when I write:
cat myfile.txt | vims ':%g/foo/exe "norm yyPImyfile: \<esc>\$dF,"' ':3p'
which are two command-line vim arguments to run on stdout,
I need these two single-quoted arguments to be passed exactly the way they are to my function vims(), which then tags each of them with a -c flag, so they are interpreted as commands in vim.
Here's what I've tried so far:
vims() {
vim - -nes -u NONE -c '$1' -c ':q!' | tail -n +2
}
This seems to work perfectly for a single command. No characters get escaped, and the "-c" flag is there.
Then, using the oft-duplicated question-answer, the "$#" trick, I tried:
vims() {
vim - -nes -u NONE $(for arg in "$#"; do echo -n " -c $arg "; done) -c ':q!' | tail -n +2
}
This seems to break the spaces within each string I pass it, so does not work. I also tried a few variations of the printf command, as suggested in other questions, but this has weird interactions with the vim command sequences. I've tried many other different backslash-quote-combinations in a perpetual edit-test loop, but have always found a quirk in my method.
What is the command sequence I am missing?
Add all the arguments to an array one at a time, then pass the entire array to vim with proper quoting to ensure whitespace is correctly preserved.
vims() {
local args=()
while (($# > 0)); do
args+=(-c "$1")
shift
done
vim - -nes -u NONE "${args[#]}" -c ':q!' | tail -n +2
}
As a rule of thumb, if you find yourself trying to escape things, add backslashes, use printf, etc., you are likely going down the wrong path. Careful use of quoting and arrays will cover most scenarios.
I think I have uncovered an error in grep. If I run this grep statement against a db log on the command line it runs fine.
grep "Query Executed in [[:digit:]]\{5\}.\?" db.log
I get this result:
Query Executed in 19699.188 ms;"select distinct * from /xyztable.....
when I run it in a script
LONG_QUERY=`grep "Query Executed in [[:digit:]]\{5\}.\?" db.log`
the asterisk in the result is replaced with a list of all files in the current directory.
echo $LONG_QUERY
Result:
Query Executed in 19699.188 ms; "select distinct <list of files in
current directory> from /xyztable.....
Has anyone seen this behavior?
This is not an error in grep. This is an error in your understanding of how scripts are interpreted.
If I write in a script:
echo *
I will get a list of filenames, because an unquoted, unescaped, asterisk is interpreted by the shell (not grep, but /bin/bash or /bin/sh or whatever shell you use) as a request to substitute filenames matching the pattern '*', which is to say all of them.
If I write in a script:
echo "*"
I will get a single '*', because it was in a quoted string.
If I write:
STAR="*"
echo $STAR
I will get filenames again, because I quoted the star while assigning it to a variable, but then when I substituted the variable into the command it became unquoted.
If I write:
STAR="*"
echo "$STAR"
I will get a single star, because double-quotes allow variable interpolation.
You are using backquotes - that is, ` characters - around a command. That captures the output of the command into a variable.
I would suggest that if you are going to be echoing the results of the command, and little else, you should just redirect the results into a file. (After all, what are you going to do when your LONG_QUERY contains 10,000 lines of output because your log file got really full?)
Barring that, at the very least do echo "$LONG_QUERY" (in double quotes).
I want to extract the output of a command run through shell script in a variable but I am not able to do it. I am using grep command for the same. Please help me in getting the desired output in a variable.
x=$(pwd)
pw=$(grep '\(.*\)/bin' $x)
echo "extracted is:"
echo $pw
The output of the pwd command is /opt/abc/bin/ and I want only /root/abc part of it. Thanks in advance.
Use dirname to get the path and not the last segment of the path.
You can use:
x=$(pwd)
pw=`dirname $x`
echo $pw
Or simply:
pw=`dirname $(pwd)`
echo $pw
All of what you're doing can be done in a single echo:
echo "${PWD%/*}"
$PWD variable represents current directory and %/* removes last / and part after last /.
For your case it will output: /root/abc
The second (and any subsequent) argument to grep is the name of a file to search, not a string to perform matching against.
Furthermore, grep prints the matching line or (with -o) the matching string, not whatever the parentheses captured. For that, you want a different tool.
Minimally fixing your code would be
x=$(pwd)
pw=$(printf '%s\n' "$x" | sed 's%\(.*\)/bin.*%\1%')
(If you only care about Bash, not other shells, you could do sed ... <<<"$x" without the explicit pipe; the syntax is also somewhat more satisfying.)
But of course, the shell has basic string manipulation functions built in.
pw=${x%/bin*}
I am trying to replace the bottom one of these 2 lines with sed in a file.
<rule>out_prefix=orderid ^1\\d\+ updatemtnotif/</rule>\n\
<rule>out_prefix=orderid ^2\\d\+ updatemtnotif/</rule>\n\
And the following command seems to do that when executed as a command at the bash prompt
sed -i 's#out_prefix=orderid ^2\\\\d\\+ updatemtnotif/#out_prefix=orderid ^2\\\\d\\+ updatemtnotif_fr/#g' /opt/temp/rules.txt
however, when I try to execute the same command remotely over ssh using here documents, the command fails to modify the file.
I think this is probably an escaping issue, but I have had no luck trying to modify the command in numerous ways. Can any one tell me what should I do to get it working over ssh? Thanks in advance!
to clarify,
input: <rule>out_prefix=orderid ^2\\d\+ updatemtnotif/</rule>\n\
output: <rule>out_prefix=orderid ^2\\d\+ updatemtnotif_fr/</rule>\n\
You can use it with ssh and heredoc like this:
ssh -t -t user#localhost<<'EOF'
sed 's~out_prefix=orderid ^2\\\\d\\+ updatemtnotif/~out_prefix=orderid ^2\\\\d\\+ updatemtnotif_fr/~' ~/path/to/file
exit
EOF
PS: It is important to quote the 'EOF' as shown.
I managed to fix it. had to escape the backslashes in the command I used inside the shell script.
's#out_prefix=orderid ^2\\\\\\\\d\\\\+ updatemtnotif/#out_prefix=orderid ^2\\\\\\\\d\\\\+ updatemtnotif_fr/#g' /opt/temp/rules.txt
That's a whole lot of backslashes but it did the trick.
How could I make a separate sed script (let's call it script.sed) that would display only the changed lines without having to use the -n option while executing it? (Sorry for my English)
I have a file called data2.txt with digits and I need to change the lines ending with ".5" and print those changed lines out in the console.
I know how to do it with a single command (sed -n 's/.5$//gp' data2.txt), however our university professor requires us to do the same using sed -f script.sed data2.txt command.
Any ideas?
The following should work for your sed script:
s/.5$//gp
d
The -n option will suppress automatic printing of the line, the other way to do that is to use the d command. From man page:
d Delete pattern space. Start next cycle.
This works because the automatic printing of the line happens at the end of a cycle, and using the d command means you never reach the end of a cycle so no lines are printed automatically.
This might work for you (GNU sed):
#n
s/.5$//p
Save this to a file and run as:
sed -f file.sed file.txt