Hello I currently have a file in this format:
This is a test file
location = /home/files
My question is how do I replace this line location = /home/files with this location = /home/documents/test_files?
When there's a "/" in the original string, or replacement string, we muse use "\" to escape "/". Alternatively you can use any character as substitution delimiter, such as "#"
$ cat sample.csv
This is a test file
location = /home/files
$ sed 's#/home/files#/home/documents/test_files#' sample.csv
This is a test file
location = /home/documents/test_files
Just choose a different delimiter:
sed 's-location = /home/files-location = /home/documents/test_files-' test.txt
sed 's~location = /home/files~location = /home/documents/test_files~' test.txt
Or escape the /:
sed 's/location = \/home\/files/location = \/home\/documents\/test_files/' test.txt
Related
I have read the following:
How Do I Use Variables In A Sed Command
How can I use variables when doing a sed?
Sed replace variable in double quotes
I have learned that I can use sed "s/STRING/$var1/g" to replace a string with the contents of a variable. However, I'm having a hard time finding out how to replace with a variable that contains double quotes, brackets and exclamation marks.
Then, hoping to escape the quotes, I tried piping my result though sed 's/\"/\\\"/g' which gave me another error sed: -e expression #1, char 7: unknown command: E'. I was hoping to escape the problematic characters and then do the variable replacement: sed "s/STRING/$var1/g". But I couldn't get that far either.
I figured you guys might know a better way to replace a string with a variable that contains quotes.
File1.txt:
Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
Variable:
var1=$(cat file1.txt)
Example:
echo "STRING" | sed "s/STRING/$var1/g"
Desired output:
Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
using awk
$ echo "STRING" | awk -v var="$var1" '{ gsub(/STRING/,var,$0); print $0}'
Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
-v var="$var1": To use shell variable in awk
gsub(/STRING/,var,$0) : To globally substitute all occurances of "STRING" in whole record $0 with var
Special case : "If your var has & in it " say at the beginning of the line then it will create problems with gsub as & has a special meaning and refers to the matched text instead.
To deal with this situation we've to escape & as follows :
$ echo "STRING" | awk -v var="$var1" '{ gsub(/&/,"\\\\&",var); gsub(/STRING/,var,$0);print $0}'
&Example test here
<tag>"Hello! [world]" This line sucks!</tag>
End example file
The problem isn't the quotes. You're missing the "s" command, leading sed to treat /STRING/ as a line address, and the value of $var1 as a command to execute on matching lines. Also, $var1 has unescaped newlines and a / character that'll cause trouble in the substitution. So add the "s", and escape the relevant characters in $var1:
var1escaped="$(echo "$var1" | sed 's#[\/&]#\\&#; $ !s/$/\\/')"
echo "STRING" | sed "s/STRING/$var1escaped/"
...but realistically, #batMan's answer (using awk) is probably a better solution.
Here is one awk command that gets text-to-be-replaces from a file that may consist of all kind of special characters such as & or \ etc:
awk -v pat="STRING" 'ARGV[1] == FILENAME {
# read replacement text from first file in arguments
a = (a == "" ? "" : a RS) $0
next
}
{
# now run a loop using index function and use substr to get the replacements
s = ""
while( p = index($0, pat) ) {
s = s substr($0, 1, p-1) a
$0 = substr($0, p+length(pat))
}
$0 = s $0
} 1' File1.txt <(echo "STRING")
To be able to handle all kind of special characters properly this command avoids any regex based functions. We use plain text based functions such as index, substr etc.
I'm trying to implement the following substitution
sed -i 's/$config['default_host'] = '';/$config['default_host'] = 'localhost';/' /etc/roundcube/config.inc.php
but it's not working.
What i want to do to is replace $config['default_host'] = ''; with $config['default_host'] = 'localhost'; inside the file /etc/roundcube/config.inc.php
Any ideas?
You should escape the special characters, because sed consider $ as a end of the character in a line
sed "s/\$config\['default_host'\] = '';/\$config['default_host'] = 'localhost';/" fileName
Using Grouping concept
sed "s/\(\$config\['default_host'\] = \)'';/\1'localhost';/" fileName
Output:
$config['default_host'] = 'localhost';
I want to replace
$rcmail_config['default_host'] = '';
in /var/lib/roundcube/config/main.inc.php
with
$rcmail_config['default_host'] = 'localhost';
I've tried:
sed -i "s/$rcmail_config['default_host'] = '';/$rcmail_config['default_host'] = 'localhost';/g" /var/lib/roundcube/config/main.inc.php
and
sed -i s/$rcmail_config['default_host'] = '';/$rcmail_config['default_host'] = 'localhost';/g /var/lib/roundcube/config/main.inc.php
But it does not work.
What could I try next?
You need to escape the $ and [ symbols and also you don't need to repeat the same string in the replacement part. Instead of this, you may use capturing groups.
sed -i "s/\(\$rcmail_config\['default_host'\] = \)'';/\1'localhost';/g" file
I have separate files include path string for each like ;
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#yahoo.com/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user55#ccc.com/
what i want to trim lines like;
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo/
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail/
path = /aaa/bbb/ccc.com/user/dddd/.user#abc/
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc.com/
I am almost be able to achieve with below (all strings are in separate files but at the 15th line)
sed -r '15s!#[^/]+(/[^/]+/[^.#]+#[^.]+).*$!\1/!g' $file
however, i have an issue with dot part that cuts it as ;
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc/
instead, it should have been;
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc/
Thanks in advance,
Using a pattern with three capture groups should do what you need. The first group will capture the portion behind the initial # (as a group we omit from the replacement), the second group will include the /dddd/ portion, and the third being the complete user#somewhere with a prepended .
's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g'
Depending on your version of bash you could use it like this:
sed -i.bak -r 's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g' $file
↳ (GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
sed -i bak -E 's!(#.+\..+)(/.+/)(.+#.+)!\2.\3!g' $file
↳ GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin12)
result:
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo.com/
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/.user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc.com/
It's a little unclear if you want to keep the full extension on the end of the last match; if not sed is probably not the best choice because it can't do look-ahead, look-behind assertions, nor toggle greedy in any straight-forward manner. In the case that's a deal breaker, you could use this pattern on one of many other avenues:
(#.+\..+)(/.+/)(.+#.+?)(\..*/)
result:
path = /aaa/bbb/ccc.com/user/dddd/.user#yahoo
path = /aaa/bbb/ccc.com/user/dddd/.user#hotmail
path = /aaa/bbb/ccc.com/user/dddd/.user#abc
path = /aaa/bbb/ccc.com/user/dddd/.user55#ccc
You would have to use two matches:
sed -E 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g'
Regex: (.*?\..*?)\/(.*?)#\1
Replacement: \1\/\2
Flags: g (Global)
Result:
path = /aaa/bbb/ccc.com/user/dddd/user#yahoo.com/
path = /aaa/bbb/ccc.com/user/dddd/user#hotmail.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/user#abc.xxx.co.uk/
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc.com/
sed -E 's/(\w+#\w+)[\w\.]*/\1/g'
Regex: (\w+#\w+)[\w\.]*
Replacement: \1
Flags: g (Global)
Result:
path = /aaa/bbb/ccc.com/user/dddd/user#yahoo/
path = /aaa/bbb/ccc.com/user/dddd/user#hotmail/
path = /aaa/bbb/ccc.com/user/dddd/user#abc/
path = /aaa/bbb/ccc.com/user/dddd/user55#ccc/
If the -E switch is not available on your version of sed, then you might have to use perl.
Example:
perl -pe 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g' -i filename.ext
If I try this in bash, I get the following result:
root#home [~]# echo "path = /aaa/bbb/ccc.com/user#ccc.com/dddd/user55/" | sed -E 's/(.*?\..*?)\/(.*?)#\1/\1\/\2/g'
path = /aaa/bbb/ccc.com/user/dddd/user55/
root#home [~]# echo "path = /aaa/bbb/ccc.com/user/dddd/user55/" | sed -E 's/(\w+#\w+)[\w\.]*/\1/g'
path = /aaa/bbb/ccc.com/user/dddd/user55/
I'd like to know if I can remove a \n (newline) only if the current line has one ore more keywords from a list; for instance, I want to remove the \n if it contains the words hello or world.
Example:
this is an original
file with lines
containing words like hello
and world
this is the end of the file
And the result would be:
this is an original
file with lines
containing words like hello and world this is the end of the file
I'd like to use sed, or awk and, if needed, grep, wc or whatever commands work for this purpose. I want to be able to do this on a lot of files.
Using awk you can do:
awk '/hello|world/{printf "%s ", $0; next} 1' file
this is an original
file with lines
containing words like hello and world this is the end of the file
here is simple one using sed
sed -r ':a;$!{N;ba};s/((hello|world)[^\n]*)\n/\1 /g' file
Explanation
:a;$!{N;ba} read whole file into pattern, like this: this is an original\nfile with lines\ncontaining words like hell\
o\nand world\nthis is the end of the file$
s/((hello|world)[^\n]*)\n/\1 /g search the key words hello or world and remove the next \n,
g command in sed substitute stands to apply the replacement to all matches to the regexp, not just the first.
A non-regex approach:
awk '
BEGIN {
# define the word list
w["hello"]
w["world"]
}
{
printf "%s", $0
for (i=1; i<=NF; i++)
if ($i in w) {
printf " "
next
}
print ""
}
'
or a perl one-liner
perl -pe 'BEGIN {#w = qw(hello world)} s/\n/ / if grep {$_ ~~ #w} split'
To edit the file in-place, do:
awk '...' filename > tmpfile && mv tmpfile filename
perl -i -pe '...' filename
This might work for you (GNU sed):
sed -r ':a;/^.*(hello|world).*\'\''/M{$bb;N;ba};:b;s/\n/ /g' file
This checks if the last line, of a possible multi-line, contains the required string(s) and if so reads another line until end-of-file or such that the last line does not contain the/those string(s). Newlines are removed and the line printed.
$ awk '{ORS=(/hello|world/?FS:RS)}1' file
this is an original
file with lines
containing words like hello and world this is the end of the file
sed -n '
:beg
/hello/ b keep
/world/ b keep
H;s/.*//;x;s/\n/ /g;p;b
: keep
H;s/.*//
$ b beg
' YourFile
a bit harder due to check on current line that may include a previous hello or world already
principle:
on every pattern match, keep the string in hold buffer
other wise, load hold buffer and remove \n (use of swap and empty the current line due to limited buffer operation available) and print the content
Add a special case of pattern in last line (normaly hold so not printed otherwise)