I have a perl inline replace command in a shell script that isn't working for some reason. With below command, its replacing all ":" with replacement string in the perl command.
When I manually ssh to the box and running the perl command works as expected.
ssh host "cd /x/somedirectory && perl -pi -e 's#\${somehost}:\${someport}#10.20.30.40:8443#g' config/app.properties"
Edit1
Please note that $somehost and $someport are NOT shell variables. I'm looking for a literal text replacement.
What am I doing wrong? I tried using different delimiters, escaping { and } etc but still no luck.
To create a single-quoted shell literal from a string, escape ' by replacing them with '\''.
To create a double-quoted shell literal from a string, escape \, " and $ by prefixing them with \.
The Perl command you want is
s#\${somehost}:\${someport}#10.20.30.40:8443#g
So the remote shell command you want is
perl -pi -e 's#\${somehost}:\${someport}#10.20.30.40:8443#g' config/app.properties
So the local shell command you want is
ssh host 'perl -pi -e '\''s#\${somehost}:\${someport}#10.20.30.40:8443#g'\'' config/app.properties'
or
ssh host "perl -pi -e 's#\\\${somehost}:\\\${someport}#10.20.30.40:8443#g' config/app.properties"
[Removed cd /x/somedirectory && to keep things simple. Just add it back in.]
The shell on the remote machine will perform another level of escaping. To receive the sequence \$ on the remote machine you have to send the sequence \\\$:
ssh host "cd /x/somedirectory && perl -pi -e
's#\\\${somehost}:\\\${someport}#10.20.30.40:8443#g' config/app.properties"
So, you can write Perl...
#!/usr/bin/perl
use Net::OpenSSH;
my $ssh = Net::OpenSSH->new($host);
my $old = quotemeta("${somehost}:${someport}");
$ssh->system('cd', $some_directory, \\'&&',
'perl', '-pi', '-e',
"s|$old|10.20.30.40:8443|g",
'config/app.properties');
Try this ssh command with here-doc and avoid all escaping:
ssh -t -t host<<'EOF'
cd /x/somedirectory &&
perl -i -pe 's#\${somehost}:\${someport}#10.20.30.40:8443#g' config/app.properties
exit
EOF
Related
I used following command and tried numerous varieties (including sed):
perl -pi -e s/multi on/172\.31\.1\.1\/32 dev eth0\\ndefault via 172\.31\.1\.1 dev eth0/g /etc/sysconfig/network-scripts/route-eth0
error is: "Substitution pattern not terminated at -e line 1.
When i try sed instead, it complains about unterminated "s" command
Aim is to replace:
multi on
by:
172.31.1.1/32 dev eth0
default via 172.31.1.1 dev eth0
(without using any | > " ' $ characters in the command)
If you don't want to quote the command (why not?), you need to escape all the spaces.
perl -pi -e s/multi\ on/172.31.1.1\\/32\ dev\ eth0\\ndefault\ via\ 172.31.1.1\ dev\ eth0/ /etc/sysconfig/network-scripts/route-eth0
OTOH, you don't need to escape . in the replacement string, it only has special meaning in regular expressions.
You also don't need the g modifier if multi on can only appear once on a line.
I want to know the best way to compress the current working directory so that only the last directory's full name is visible. Let me give an example:
$ echo $PWD
/Users/mac/workshop/project1/src
I want to be able to do bash regex replacement operations on it such that I can get ~/w/p/src
I can obtain the first part of getting the leading ~ by doing ${PWD/#$HOME/\~}
$ echo ${PWD/#$HOME/\~}
~/workshop/project1/src
What other regex operations can I do (is it possible to chain the regex operators?) so that I get the following
$ echo ${PWD/#$HOME/\~} ...
~/w/p/src
Note that I need to do only using bash i.e. no sed, awk, grep etc.
The intention for this is so that, I can set the PROMPT value based on bash i.e.
in my .bashrc, I want to:
export PROMPT=${PWD/#$HOME/\~}...
Do-able in just bash, but not as simple as you'd like:
$ squashPWD() {
local pwd parts part
IFS=/ read -ra parts <<< "${PWD/#$HOME/\~}"
for part in "${parts[#]:0:${#parts[#]}-1}"; do
pwd+="${part:0:1}/"
done
echo "$pwd${parts[-1]}"
}
$ pwd
/home/jackman/tmp/adir/foo
$ squashPWD
~/t/a/foo
$ cd /usr/local/share/doc/fish/
$ squashPWD
/u/l/s/d/fish
If you don't need bash:
squashPWD() { perl -pe 's/^$ENV{HOME}/~/; s{([^/])[^/]*(?=/)}{$1}g' <<<"$PWD"; }
Either way, your prompt can be something like:
PS1='\u#\h:$(squashPWD) \$ '
It doesn't need to be all bash, you can use a function in your bashrc or bash_profile and thus use sed or awk. You can put something like this in your bashrc:
short() {
local short_path=$(echo "$PWD" | sed -E 's!/(.)[^/]*!/\1!g')
local last_dir=${PWD##*/}
echo "${short_path::-1}${last_dir}" # remove last character (1st character of last directory, and just append the last directory)
}
PS1='$(short) '
Keep in mind I don't think I replace your $HOME directory with ~, but you know how to do that :)
So I have the following string in my config.fish, and init.vim:
Fish: eval sh ~/.config/fish/colors/base16-monokai.dark.sh
Vim: colorscheme base16-monokai
Vim: let g:airline_theme='base16_monokai'
And I have the following shell script:
#!/bin/sh
theme=$1
background=$2
if [ -z '$theme' ]; then
echo "Please provide a theme name."
else
if [ -z '$background' ]; then
$background = 'dark'
fi
base16-builder -s $theme -t vim -b $background > ~/.config/nvim/colors/base16-$theme.vim &&
base16-builder -s $theme -t shell -b $background > ~/.config/fish/colors/base16-$theme.$background.sh &&
base16-builder -s $theme -t vim-airline -b $background > ~/.vim/plugged/vim-airline-themes/autoload/airline/themes/base16_$theme.vim
sed -i -e 's/foo/eval sh ~/.config/fish/colors/base16-$theme.$background.sh/g' ~/Developer/dotfiles/config.fish
sed -i -e 's/foo/colorscheme base16-$theme/g' ~/Developer/dotfiles/init.vim
sed -i -e 's/foo/let g:airline_theme='base16_$theme'/g' ~/Developer/dotfiles/init.vim
fi
Basically the idea is the script will generate whichever theme is passed through using this builder.
I have tried referring this documentation but I am not very skilled at regex so if anybody could give me a hand I would appreciate it.
What I need to happen is once the script is generated sed will look for the above strings and replace theme with the newly generated theme ones.
Try this :
sed -i "s|\(eval sh ~/\.config/fish/colors/base16-\)\([^.]*\)\.\([^.]*\)\\(.*\)|\1$theme.$background\4|
" ~/Developer/dotfiles/config.fish
sed -i "s/\(base16\)\([-_]\)\([a-zA-Z]*\)/\1\2$theme/g" ~/Developer/dotfiles/init.vim
Assuming in the second sed command that the theme is an alphanumeric string. If not, you can complete the character range : [a-zA-Z] with additional characters (eg [a-zA-Z0-9]).
You can replace something in sed using this syntax: sed "s#regex#replacement#g". Because you have /s and 's in your strings, it's easiest not to need to escape them.
There are some characters that need to be escaped to make the regexes. . and $ need to be escaped with a \. The $ in the replacement string needs to be escaped too.
If you want to capture a certain part from match, it's easiest to use char classes. For example, eval sh ~/\.config/fish/colors/base16-([^.]+)\.dark\.sh would be the regex to use if you want your replacement to be airline_theme='$1_base16_\$theme'. In that case, the $1 in the replacement is the thing captured in the regex.
[^.]+ will capture everything up to the next .
I hope this helps you to better understand regexes! This should be detailed enough to show you how to write your own.
You need to use double quotes for parameter expansion not single quotes.
You need to escape the single quotes: 'hello'\''world'
I will make one line for you and leave it as an exercise to fix the other lines
sed -i -e 's~\(let g:airline_theme='\''\)[^'\'']*\('\'\)'~base16_'"$theme"~' ~/Developer/dotfiles/init.vim
The first character after the s in the sed expression string is used as the pattern separator, so by putting / first you have specified / as the separator.
Additionally using the single quote tells the shell not to expand any variables, you are going to want to use double quotes instead.
try something like
sed -i -e "s#foo#eval sh ~/.config/fish/colors/base16-$theme.$background.sh#g" ~/Developer/dotfiles/config.fish
as you've now commented that you needed to find the previous theme string instead of foo
sed -i -e "s#eval sh \~/\.config/fish/colors/base16-.*?\..*?\.sh#eval sh ~/.config/fish/colors/base16-$theme.$background.sh#g" ~/Developer/dotfiles/config.fish
Im using a shell script to get a file using wget and search it for a pattern. My shell script is as follows:
#Execute commands one by one
while read line
do
STARTTIME=$(($(date +%s%N)/1000000))
line2=$(wget -q --post-data "$line" -O PHPFiles/test.php http://localhost:1234/XSS/XSS2/test.php)
ENDTIME=$(($(date +%s%N)/1000000))
GAP=$(($ENDTIME-$STARTTIME))
DIFF=$(($DIFF+($ENDTIME-$STARTTIME)))
echo "Time Taken "$GAP
finalSearchLine1="${line/&name2=/ }"
finalSearchLine2="${finalSearchLine1/name=/}"
echo "$finalSearchLine2"
if grep -q -F "$finalSeachLine2" -a PHPFiles/test.php;
then
echo found
success=$((success+1))
else
echo not found
failure=$((failure+1))
fi
rm PHPFiles/test.php
done < $1
echo "***************"
echo "Success "$success
echo "Failure "$failure
echo "Total Time "$DIFF
echo "Average Time "$((DIFF/(success+failure)))
However, I'm having trouble with the grep command. Sometimes, the data $finalSearchLine2 contains quotes such as:
<script >alert("XSS"); </script>
This seem to cause trouble with the grep command. For the if statement, I always seem to get the result as found even when there is no matching pattern in the $finalSearchLine2 variable. I dont know if its possible to use escape strings within the variable for grep. Can anyone suggest a possible solution for this?
Grep needs double quotes to be escaped like this \"
So as a first solution you could try:
temp_variable=$(sed 's/"/\\"/g' <<< $temp)
if grep -q -F "$temp_variable" -a /PHPFiles/test.php;
So you first escape the double quotes with sed and you store the result in temp_variable. Then you use temp_variable in grep.
I want to write a script that modifies a variable in a .properties file. The user enters the new value which is in turn written into the file.
read -p "Input Variable?" newVar
sed -r 's/^\s*myvar=.*/myvar=${newVar}/' ./config.properties
Unfortunately problems arise when the user inputs special characters. In my use case it is very likely that a "/" character is typed. So my guess is that I have to parse ${newVar} for all slashes and escape them? But how? Is there a better way?
have a look at bash printf
%q quote the argument in a way that can be reused as shell input
Example:
$ printf "%q" "input with special characters // \\ / \ $ # #"
input\ with\ special\ characters\ //\ \\\ /\ \\\ \$\ #\ #
Avoiding shell quoting is a good general principle.
#! /usr/bin/env perl
use strict;
use warnings;
die "Usage: $0 properties-file ..\n" unless #ARGV;
print "New value for myvar?\n";
chomp(my $new = <STDIN>);
$^I = ".bak";
while (<>) {
s/^(\s*myvar\s*=\s*).*$/$1$new/;
print;
}
Substitution with s/// as above will be familiar to sed users.
The code above uses Perl's in-place editing facility (enabled most commonly with the -i switch but above with the special $^I variable) to modify files named on the command line and create backups with the .bak extension.
Example usage:
$ cat foo.properties
theirvar=123
myvar=FIXME
$ ./prog foo.properties
New value for myvar?
foo\bar
$ cat foo.properties
theirvar=123
myvar=foo\bar
$ cat foo.properties.bak
theirvar=123
myvar=FIXME
Edit: oops, we are only qoting the value, not the regex. So this is what you need
You are better off using perl instead of sed for this if it is available.
read -p "Input Variable?" newVar
perl -i -p -e 'BEGIN{$val=shift;}' \
-e 's/^\s*myvar=.*/myvar=$val/' \
"$newVar" ./config.properties
Edit2: Sorry, still does not handle \ characters in newVar. Guess one of the other solutions is better. As stated before, dealing with shell escaping is your issue.
You are better off using a tool that understands variables -- Perl, maybe AWK -- trying to quote a random string so that you avoid all unintended interactions with sed command parsing is asking for trouble.
Also, you won't get your variable interpolated when using single quotes, and even with -r, sed does not grok Perl regex syntax -- -r only gets you to the egrep version of regexes, so \s doesn't do what you want.
Anyway, ignoring my own advice, here's how we'd do it in the old days before we had those better tools:
read -p "Input Variable?" newVar
sed "/^ *myvar=/c\\
myvar=`echo \"$newVar\" | sed 's/\\\\/\\\\\\\\/'`" ./config.properties
If you don't think your users will figure out how to input literal backslashes at your prompt, you can simplify this to:
read -p "Input Variable?" newVar
sed "/^ *myvar=/c\\
myvar=$newVar" ./config.properties