This question already has answers here:
Bash Search File for Pattern, Replace Pattern With Code that Includes Git Branch Name
(1 answer)
Replace a string in shell script using a variable
(12 answers)
Closed 6 years ago.
I have a file file.txt with this content: Hi {YOU}, it's {ME}
I would like to dynamically create a new file file1.txt like this
YOU=John
ME=Leonardo
cat ./file.txt | sed 'SED_COMMAND_HERE' > file1.txt
which content would be: Hi John, it's Leonardo
The sed command I tried so far is like this s#{\([A-Z]*\)}#'"$\1"'#g but the "substitution" part doesn't work correctly, it prints out Hi $YOU, it's $ME
The sed utility can do multiple things to each input line:
$ sed -e "s/{YOU}/$YOU/" -e "s/{ME}/$ME/" inputfile.txt >outputfile.txt
This assumes that {YOU} and {ME} occurs only once each on the line, otherwise, just add g ("s/{YOU}/$YOU/g" etc.)
You can use awk with 2 files.
$> cat file.txt
Hi {YOU}, it's {ME}
$> cat repl.txt
YOU=John
ME=Leonardo
$> awk -F= 'FNR==NR{a["{" $1 "}"]=$2; next} {for (i in a) gsub(i,a[i])}1' repl.txt file.txt
Hi John, it's Leonardo
First awk command goes through replacement file and stores each key-value in an array a be wrapping keys with { and }.
In second iteration we just replace each key by value in actual file.
Update:
To do this without creating repl.txt you can use `process substitution**:
awk -F= 'FNR==NR{a["{" $1 "}"]=$2; next} {
for (i in a) gsub(i,a[i])} 1' <(( set -o posix ; set ) | grep -E '^(YOU|ME)=') file.txt
Related
This question already has answers here:
bash: shortest way to get n-th column of output
(8 answers)
Closed 1 year ago.
I need to extract substrings from a file into a new file. Mac or Linux.
The data is between the 4th and 5th "|" symbol.
HD|262339|9400530374||K7UKD|A|HA|12/15/2009|03/13/2020
The actual columnar position varies, sometimes by a lot, but the data is always between the 4th and 5th pipe symbol.
Sample data is as above, expected output would be K7UKD.
I've tried various hacks at a regex:
grep "/\|(\w+)\|/" input.txt > output.txt
Converting my comment to answer so that solution is easy to find for future visitors.
There are 2 ways to get it:
Any awk version:
awk -F'|' '{print $5}' file
K7UKD
or using gnu-awk:
awk -v RS='|' 'NR == 5' file
Here is a bash solution using read:
IFS='|' read -ra arr <<< 'HD|262339|9400530374||K7UKD|A|HA|12/15/2009|03/13/2020' &&
echo "${arr[4]}"
K7UKD
Or using cut:
cut -d'|' -f5 file
Or using sed:
sed -E 's/^([^|]*\|){3}\|([^|]*).*/\2/' file
This question already has answers here:
Escape a string for a sed replace pattern
(17 answers)
Closed 4 years ago.
I cannot expand this variable in sed. I've tried everything I can think of.
I am trying to put the md5sum of file1 in line 10 of file2
I can take $x out of the regex and put some text and it works. It just will not accept the variable. printf the variable is fine.
#!/bin/bash
x=$(md5sum /etc/file1)
printf "$x \n"
sed -i 10"s/.*/$x/g" /usr/bin/file2
You may use this command that uses ~ as regex delimiter instead of / since output of md5sum contains /:
sed -i "10s~.*~$x~" /usr/bin/file2
After I reduced the variable from the md5sum output which includes the filename and directory by running $x thru:
x=$(echo $x | head -n1 | awk '{print $1;}')
Leaving only the MD5 it worked and quit erroring.
I'm trying to add a 'chr' string in the lines where is not there. This operation is necessary only in the lines that have not '##'.
At first I use grep + sed commands, as following, but I want to run the command overwriting the original file.
grep -v "^#" 5b110660bf55f80059c0ef52.vcf | grep -v 'chr' | sed 's/^/chr/g'
So, to run the command in file I write this:
sed -i -E '/^#.*$|^chr.*$/ s/^/chr/' 5b110660bf55f80059c0ef52.vcf
This is the content of the vcf file.
##FORMAT=<ID=DP4,Number=4,Type=Integer,Description="#ref plus strand,#ref minus strand, #alt plus strand, #alt minus strand">
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT 24430-0009S21_GM17-12140
1 955597 95692 G T 1382 PASS VARTYPE=1;BGN=0.00134309;ARL=150;DER=53;DEA=55;QR=40;QA=39;PBP=1091;PBM=300;TYPE=SNP;DBXREF=dbSNP:rs115173026,g1000:0.2825,esp5400:0.2755,ExAC:0.2290,clinvar:rs115173026,CLNSIG:2,CLNREVSTAT:mult,CLNSIGLAB:Benign;SGVEP=AGRN|+|NM_198576|1|c.45G>T|p.:(p.Pro15Pro)|synonymous GT:DP:AD:DP4 0/1:125:64,61:50,14,48,13
chr1 957898 82729935 G T 1214 off_target VARTYPE=1;BGN=0.00113362;ARL=149;DER=50;DEA=55;QR=38;QA=40;PBP=245;PBM=978;NVF=0.53;TYPE=SNP;DBXREF=dbSNP:rs2799064,g1000:0.3285;SGVEP=AGRN|+|NM_198576|2|c.463+56G>T|.|intronic GT:DP:AD:DP4 0/1:98:47,51:9,38,10,41
If I understand what is your expected result, try:
sed -ri '/^(#|chr)/! s/^/chr/' file
Your question isn't clear and you didn't provide the expected output so we can't test a potential solution but if all you want is to add chr to the start of lines where it's not already present and which don't start with # then that's just:
awk '!/^(#|chr)/{$0="chr" $0} 1' file
To overwrite the original file using GNU awk would be:
awk -i inplace '!/^(#|chr)/{$0="chr" $0} 1' file
and with any awk:
awk '!/^(#|chr)/{$0="chr" $0} 1' file > tmp && mv tmp file
This can be done with a single sed invocation. The script itself is something like the following.
If you have an input of format
$ echo -e '#\n#\n123chr456\n789chr123\nabc'
#
#
123chr456
789chr123
abc
then to prepend chr to non-commented chrless lines is done as
$ echo -e '#\n#\n123chr456\n789chr123\nabc' | sed '/^#/ {p
d
}
/chr/ {p
d
}
s/^/chr/'
which prints
#
#
123chr456
789chr123
chrabc
(Note the multiline sed script.)
Now you only need to run this script on a file in-place (-i in modern sed versions.)
This question already has answers here:
How to use variables in a command in sed?
(4 answers)
Closed 8 years ago.
Would like to ask because I'm having an issue with sed command in unix scripting.
#!/bin/sh
cnt=2
sed '1 c\
$cnt' test.txt
I want to replace the first line of text file test.txt with the value of variable cnt which is 2. How can I pass the variable on the above sed command? The sed command treats $cnt as string.
Variables are not expanded in single quotes. They are expanded in double quotes, though.
sed "1 c\\
$cnt" test.txt
Note that sed doesn't update the input file by default, it outputs the changed version instead. If your implementation of sed supports it, use the -i switch to modify the input file. Otherwise, you'd have to redirect the output to a new file and rename it back to the original name:
sed "..." text.txt > text.new
mv text.new text.txt
Change to:
#!/bin/sh
cnt=2
sed "1 c\
$cnt" test.txt
For variable interpolation to happen, u need to put it within double quotes rather than single quotes.
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 8 years ago.
Having some difficulty in getting a sed | grep pipe to work when using vars as numbers.
In the string below the '3,5p' works fine, but when substituting the numbers for vars I get the error
sed: -e expression #1, char 4: extra characters after command
working=$(sed -n '3,5p' ${myFile} | grep -n "string" |cut -f1 -d: )
notWorking=$(sed -n '${LINESTART},${LINEEND}p' ${myFile} | grep -n "string" |cut -f1 -d: )
I would also be interested in any advice how I could change command so the line number returned is replaced with $string2 in the file myFile
thanks
Art
You need the variables to be expanded by sed. For that, you have to enclose the expression within double quotes:
sed -n "${LINESTART},${LINEEND}p" ${myFile}
^ ^
instead of
sed -n '${LINESTART},${LINEEND}p' ${myFile}
As you are checking for the line number in $myFile where string is found, it line is in between $LINESTART and $LINEEND, you can do:
awk 'NR>=start && NR<=end && /string/ {print NR}' start=$LINESTART end=$LINEEND ${myFile}
Suppose you want to replace a string just if it appears in specific lines. You can use this:
sed -i.bak "$LINESTART,$LINEEND s/FIND/REPLACE/' file
-i.bak makes a backup of the file and does an in-place edit: file will contain the modified file, while file.bak will be the backup.
Test
$ cat a
hello
this is
something
i want changed
end
but this is not to be changed
$ sed -i.bak '3,5 s/changed/NEW/' a
$ cat a
hello
this is
something
i want NEW <---- "changed" got replaced
end
but this is not to be changed <---- this "changed" did not