I have a question which I really hope to find the answer.
These is a file with several lines that are the same, but I just want to replace the 1st occurence after a match.
I want to change the first Iam2 to Iwas2 after [blabla] line.
file.txt:
Iam
Iam1
Iam2
Iam21
[blabla]
Iam3
Iam2
Iam3
Iam2
I tried:
$ cat file.txt | sed '1,/[blabla]/a \' | sed '0,/Iam2/s//Iwas2/'
and also
$ cat file.txt | sed '/[blabla]/a \' | sed '0,/Iam2/s//Iwas3/'
What am I doing wrong?
One way assuming infile with the data of the question:
sed -e '
## From "[blabla]" until last line try to substitute "Iam2". If the
## substitution succeeds go to label "a".
/\[blabla\]/,$ {
s/Iam\(2\)/Iwas\1/;
ta
}
## The substitution didn't succeed, so begin next cycle.
b
## Label "a".
:a
## Substitution succeed, so I don't want to repeat previous process. Now
## read each line and print it without modification until end of file.
N;
P;
s/^[^\n]*\n//;
ba
' infile
That yields:
Iam
Iam1
Iam2
Iam21
[blabla]
Iam3
Iwas2
Iam3
Iam2
awk 'f==1 && /Iam2/{$0="Iwas2";f=0}/blabla/{f=1}1' file
This might work for you (GNU sed):
sed '/\[blabla\]/,/Iam2/!b;/Iam2/!b;s//Iwas2/;:a;n;$!ba' file
Related
I got this text in file.txt:
Osmun.Prez#mail.com:c7lB2m6b#3.a.a:tt_webid_v2=6990226111024612869; tt_webid=6990226111024612869; tt_csrf_token=VD5Nb_TQFH4RKhoJeSe2nzLB; R6kq3TV7=AHkh4PB6AQAA3LIS90nWf2ss0Q7ZTCQjUat4axctvhQY68DdUEz92RwpmVSX|1|0|e9d6917c2fe555827dcf5ee916ba9778079ab2a9; ttwid=1%7CAFodeNF0iZM2fyy-ZeiZ6HTpZoG_MSx6SmXHgGVQ-V4%7C1627538859%7C59ca1e4a56f9f537b55e655a6dabff88e44eb48502b164ed6b4199f5a5263cb0; passport_csrf_token_default=6f7653c3ce946a6ce5444723fb0c509b; passport_csrf_token=6f7653c3ce946a6ce5444723fb0c509b; sid_guard=0483b7d37f4e4bd20ab3046e29724798%7C1627538893%7C5184000%7CMon%2C+27-Sep-2021+06%3A08%3A13+GMT; uid_tt=27b52febe6222486b9f6b6a90ef4ffeace5ea25c09d29a1583be5a1ecf760996; uid_tt_ss=27b52febe6222486b9f6b6a90ef4ffeace5ea25c09d29a1583be5a1ecf760996; sid_tt=0483b7d37f4e4bd20ab3046e29724798; sessionid=0483b7d37f4e4bd20ab3046e29724798; sessionid_ss=0483b7d37f4e4bd20ab3046e29724798; store-idc=maliva; store-country-code=us; odin_tt=294845c8f7711db177f7c549a9f44edb1555031b27a2a485df809cd92c4e544ac0772bf462df5b7a100f6e488c45303cd62df3b6b950f0842520cd887850137b035d990f29cc8b752765e594560c977f; cmpl_token=AgQQAPNSF-RMpbE89z5HYF0_-2PcrxjXf4fZYP5_ZA
How can I delete everything from the string inside ( first & only instance ) from :tt_ to _ZA in file.txt keeping only Osmun.Prez#mail.com:c7lB2m6b#3.a.a using bash linux?
Thank you
Something like:
sed -i "s/:tt_.*//" file.txt
if you want to edit the file in place. If not, remove the -i switch.
The sed command means: replace (s), in each line of file.txt, all the chars (.*) starting by the pattern :tt_ with an empty string (//).
Or the command:
sed -i "s/:tt_.*_ZA//" file.txt
which is more adherent to what you ask for, but returns the same output.
Use pattern substitution:
i=$(cat file.txt)
echo "${i/:tt*_ZA}"
Assuming the general requirement is to remove everything after the 2nd : ...
Sample data:
$ cat file.txt
Osmun.Prez#mail.com:c7lB2m6b#3.a.a:tt_webid_v ... to end of line
some.one#home.com:B52_m6b#9_az.more.stuff:delete from here ... to end of line
One sed idea:
$ sed -En 's/^([^:]*:[^:]*).*$/\1/p' file.txt
Osmun.Prez#mail.com:c7lB2m6b#3.a.a
some.one#home.com:B52_m6b#9_az.more.stuff
Using awk
awk 'BEGIN{FS=OFS=":"}{print $1,$2}'
Using : as the delimiter, it is easy to extract the columns before :tt
This deletes all chars from ":tt_" to the last "_ZA", inclusive, in file.txt
Mac_3.2.57$cat file.txt | sed 's/\(\)[:]tt.*_ZA\(.*\)/\1\2/'
Osmun.Prez#mail.com:c7lB2m6b#3.a.a
Mac_3.2.57$
Or if it is always the first 2 values which are separated by colon (as per you example)
cat file.txt | cut -f1,2 -d’:’
I need something similar to : sed: Replace part of a line
I have an IPaddres that as the pattern, I need to replace string before it.
Sample :
#stuff SSIPaddress
Needs to be
stuff SIPaddress
Ideas or at least how to turn the one from the link to work for me :)
Based on your example :
sed 's/^#stuff SS\(\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}\)\(.*\)/stuff S\1\3/' <<< "#stuff SS1.2.3.4"
outputs :
stuff S1.2.3.4
CodeGnome, you're right, but let me show what I tried:
sed: Replace part of a line -this replaces after a pattern.
I tried to reverse the string to work for me ( replace after pattern, right )
echo “`grep "SS" file | sed -re '/([0-9]{1,3}\.){4}/p'
| grep -v drama`” | rev | sed -i …|rev
Problem with this idea is, it prints a duplicate of the matched line back in the file && cant double pipe sed -i without a file at the end ( err sed: no input file)
The winning code, after a lot of sed madness
sed -ri 'drama/! { /SS/ s/SS/S/;s/#// } ' file
due to
#*.* SSdrama:123
#*.* SSIP
" Do not match drama, in a line with SS pattern, for all else ( only one ocurrence in my sample) sed replace SS with S and # with '';Note that I didn't have to match the IP at the end at all.
I've got files repeatedly containing the string \n\n} and I need to replace such string with \n} (removing one of the two newlines).
Since such files are dynamically generated through a bash script, I need to embed replacing code inside the script.
I tried with the following commands, but it doesn't work:
cat file.tex | sed -e 's/\n\n}/\n}/g' # it doesn't work!
cat file.tex | perl -p00e 's/\n\n}/\n}/g' # it doesn't work!
cat file.tex | awk -v RS="" '{gsub (/\n\n}/, "\nb")}1' # it does work, but not for large files
You didn't provide any sample input and expected output so it's a guess but maybe this is what you're looking for:
$ cat file
a
b
c
}
d
$ awk '/^$/{f=1;next} f{if(!/^}/)print "";f=0} 1' file
a
b
c
}
d
a way with sed:
sed -i -n ':a;N;$!ba;s/\n\n}/\n}/g;p' file.tex
details:
:a # defines the label "a"
N # append the next line to the pattern space
$!ba # if it is not the last line, go to label a
s/\n\n}/\n}/g # replace all \n\n} with \n}
p # print
The i parameter will change the file in place.
The n parameter prevents to automatically print the lines.
This Perl command will do as you ask
perl -i -0777 -pe's/\n(?=\n})//g' file.tex
This should work:
cat file.tex | sed -e 's/\\n\\n}/\\n}/g'
if \n\n} is written as raw string.
Or if it's new line:
cat file.tex | sed -e ':a;N;$!ba;s/\n\n}/\n}/g'
Another method:
if the first \n is any new line:
text=$(< file.tex)
text=${text//$'\n\n}'/$'\n}'}
printf "%s\n" "$text" #> file
If the first \n is an empty line:
text=$(< file.tex)
text=${text//$'\n\n\n}'/$'\n\n}'}
printf "%s\n" "$text" #> file
Nix-style line filters process the file line-by-line. Thus, you have to do something extra to process an expression which spans lines.
As mentioned by others, '\n\n' is simply an empty line and matches the regular expression /^$/. Perhaps the most efficient thing to do is to save each empty line until you know whether or not the next one will contain a close bracket at the beginning of the line.
cat file.tex | perl -ne 'if ( $b ) { print $b unless m/^\}/; undef $b; } if ( m/^$/ ) { $b=$_; } else { print; } END { print $b if $b; }'
And to clean it all up we add an END block, to process the case that the last line in the file is blank (and we want to keep it).
If you have access to node you can use rexreplace
npm install -g regreplace
and then run
rexreplace '\n\n\}' '\n\}' myfile.txt
Of if you have more files in a dir data you can do
rexreplace '\n\n\}' '\n\}' data/*.txt
Given a file like this:
a
b
a
b
I'd like to be able to use sed to replace just the last line that contains an instance of "a" in the file. So if I wanted to replace it with "c", then the output should look like:
a
b
c
b
Note that I need this to work irrespective of how many matches it might encounter, or the details of exactly what the desired pattern or file contents might be. Thanks in advance.
Not quite sed only:
tac file | sed '/a/ {s//c/; :loop; n; b loop}' | tac
testing
% printf "%s\n" a b a b a b | tac | sed '/a/ {s//c/; :loop; n; b loop}' | tac
a
b
a
b
c
b
Reverse the file, then for the first match, make the substitution and then unconditionally slurp up the rest of the file. Then re-reverse the file.
Note, an empty regex (here as s//c/) means re-use the previous regex (/a/)
I'm not a huge sed fan, beyond very simple programs. I would use awk:
tac file | awk '/a/ && !seen {sub(/a/, "c"); seen=1} 1' | tac
Many good answers here; here's a conceptually simple two-pass sed solution assisted by tail that is POSIX-compliant and doesn't read the whole file into memory, similar to Eran Ben-Natan's approach:
sed "$(sed -n '/a/ =' file | tail -n 1)"' s/a/c/' file
sed -n '/a/=' file outputs the numbers of the lines (function =) matching regex a, and tail -n 1 extracts the output's last line, i.e. the number of the line in file file containing the last occurrence of the regex.
Placing command substitution $(sed -n '/a/=' file | tail -n 1) directly before ' s/a/c' results in an outer sed script such as 3 s/a/c/ (with the sample input), which performs the desired substitution only on the last on which the regex occurred.
If the pattern is not found in the input file, the whole command is an effective no-op.
Another approach:
sed "`grep -n '^a$' a | cut -d \: -f 1 | tail -1`s/a/c/" a
The advantage of this approach is that you run sequentially on the file twice, and not read it to memory. This can be meaningful in large files.
This might work for you (GNU sed):
sed -r '/^PATTERN/!b;:a;$!{N;/^(.*)\n(PATTERN.*)/{h;s//\1/p;g;s//\2/};ba};s/^PATTERN/REPLACEMENT/' file
or another way:
sed '/^PATTERN/{x;/./p;x;h;$ba;d};x;/./{x;H;$ba;d};x;b;:a;x;/./{s/^PATTERN/REPLACEMENT/p;d};x' file
or if you like:
sed -r ':a;$!{N;ba};s/^(.*\n?)PATTERN/\1REPLACEMENT/' file
On reflection, this solution may replace the first two:
sed '/a/,$!b;/a/{x;/./p;x;h};/a/!H;$!d;x;s/^a$/c/M' file
If the regexp is no where to found in the file, the file will pass through unchanged. Once the regex matches, all lines will be stored in the hold space and will be printed when one or both conditions are met. If a subsequent regex is encountered, the contents of the hold space is printed and the latest regex replaces it. At the end of file the first line of the hold space will hold the last matching regex and this can be replaced.
Another one:
tr '\n' ' ' | sed 's/\(.*\)a/\1c/' | tr ' ' '\n'
in action:
$ printf "%s\n" a b a b a b | tr '\n' ' ' | sed 's/\(.*\)a/\1c/' | tr ' ' '\n'
a
b
a
b
c
b
A two-pass solution for when buffering the entire input is intolerable:
sed "$(sed -n /a/= file | sed -n '$s/$/ s,a,c,/p' )" file
(the earlier version of this hit a bug with history expansion encountered on a redhat bash-4.1 install, this way avoids a $!d that was being mistakenly expanded.)
A one-pass solution that buffers as little as possible:
sed '/a/!{1h;1!H};/a/{x;1!p};$!d;g;s/a/c/'
Simplest:
tac | sed '0,/a/ s/a/c/' | tac
Here is all done in one single awk
awk 'FNR==NR {if ($0~/a/) f=NR;next} FNR==f {$0="c"} 1' file file
a
b
c
b
This reads the file twice. First run to find last a, second run to change it.
tac infile.txt | sed "s/a/c/; ta ; b ; :a ; N ; ba" | tac
The first tac reverses the lines of infile.txt, the sed expression (see https://stackoverflow.com/a/9149155/2467140) replaces the first match of 'a' with 'c' and prints the remaining lines, and the last tac reverses the lines back to their original order.
Here is a way with only using awk:
awk '{a[NR]=$1}END{x=NR;cnt=1;while(x>0){a[x]=((a[x]=="a"&&--cnt==0)?"c <===":a[x]);x--};for(i=1;i<=NR;i++)print a[i]}' file
$ cat f
a
b
a
b
f
s
f
e
a
v
$ awk '{a[NR]=$1}END{x=NR;cnt=1;while(x>0){a[x]=((a[x]=="a"&&--cnt==0)?"c <===":a[x]);x--};for(i=1;i<=NR;i++)print a[i]}' f
a
b
a
b
f
s
f
e
c <===
v
It can also be done in perl:
perl -e '#a=reverse<>;END{for(#a){if(/a/){s/a/c/;last}}print reverse #a}' temp > your_new_file
Tested:
> cat temp
a
b
c
a
b
> perl -e '#a=reverse<>;END{for(#a){if(/a/){s/a/c/;last}}print reverse #a}' temp
a
b
c
c
b
>
Here's another option:
sed -e '$ a a' -e '$ d' file
The first command appends an a and the second deletes the last line. From the sed(1) man page:
$ Match the last line.
d Delete pattern space. Start next cycle.
a text Append text, which has each embedded newline preceded by a backslash.
Here's the command:
sed '$s/.*/a/' filename.txt
And here it is in action:
> echo "a
> b
> a
> b" > /tmp/file.txt
> sed '$s/.*/a/' /tmp/file.txt
a
b
a
a
awk-only solution:
awk '/a/{printf "%s", all; all=$0"\n"; next}{all=all $0"\n"} END {sub(/^[^\n]*/,"c",all); printf "%s", all}' file
Explanation:
When a line matches a, all lines between the previous a up to (not including) current a (i.e. the content stored in the variable all) is printed
When a line doesn't match a, it gets appended to the variable all.
The last line matching a would not be able to get its all content printed, so you manually print it out in the END block. Before that though, you can substitute the line matching a with whatever you desire.
Given:
$ cat file
a
b
a
b
You can use POSIX grep to count the matches:
$ grep -c '^a' file
2
Then feed that number into awk to print a replacement:
$ awk -v last=$(grep -c '^a' file) '/^a/ && ++cnt==last{ print "c"; next } 1' file
a
b
c
b
I need advice on how to change the file formatted following way
file1:
A 504688
B jobnameA
A 504690
B jobnameB
A 504691
B jobnameC
...
into file2:
A B
504688 jobnameA
504690 jobnameB
504691 jobnameC
...
One solution I could think of is:
cat file1 | perl -0777 -p -e 's/\s+B/\t/' | awk '{print $2"\t"$3}'.
But I am wondering if there is more efficient way or already known practice that does this job.
perl -nawe 'print "#F[1 .. $#F]", $F[0] eq "A" ? "\t" : "\n"' < /tmp/ab
Look up the options in perlrun.
Another useful one to add is -l (append newline to print), but not in this case.
Assuming your input file is tab separated:
echo $'A\tB'
cut -f2 filename | paste - -
Should be pretty quick because this is exactly what cut and paste were written to do.
awk '/^A/{num=$2}/^B/{print num,$2}' file
Or, alternately,
awk '{num=$2;getline;print num,$2}' file
Here is an sed solution:
sed -e 'N' -e 's/A\s*\(.*\)\nB\s*\(.*\)/\1\t\2/' file
This version will also print the header at the top:
sed '1{h;s/.*/A\tB/p;g};N;s/A\s*\(.*\)\nB\s*\(.*\)/\1\t\2/' file
Or an alternative:
sed -n '/^A\s*/{s///;h};/^B\s*/{s///;H;g;s/\n/\t/p}' file
If your sed does not support semicolons as a command separator for the alternative:
sed -n '
/^A\s*/{ # if the line starts with "A"
s/// # remove the "A" and the whitespace
h # copy the remainder into the hold space
} # end if
/^B\s*/{ # if the line starts with "B"
s/// # remove the "B" and the whitespace
H # append pattern space to hold space
g # copy hold space to pattern space
s/\n/\t/p # replace newline with tab and print
}' file
This version will also print the header at the top:
sed -n '/^A\s*/{s///;h;1s/.*/A\tB/p};/^B\s*/{s///;H;g;s/\n/\t/p}' file
This will work with any header text, not just fixed A and B >>
awk '{a=$1;b=$2;getline;if(c!=1){print a,$1;c=1};print b,$2}' file1 >file2
...and it will print also header row
If you need \t separator, then use:
awk '{a=$1;b=$2;getline;if(c!=1){print a"\t"$1;c=1};print b"\t"$2}' file1 >file2
This might work for you:
sed -e '1i\A\tB' -e 'N;s/A\s*\(\S*\).*\nB\s*\(\S*\).*/\1\t\2/' file