With awk or sed how could I replace each line after a pattern match?
the pattern would always start as S:\ and something else,
I need the entire line from S:\~ to the end to appear on the next lines before a blank line.
I have an input like this:
S:\dir1\subfolder1longsubf
abcdefg
1234567
permissions
S:\dir2\verylongsub
some random string
some random string
S:\dir3
some random string
some random string
S:\dir4\sub2\sub3
some random string
some random string
some random string
and I need an ouput like this:
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir2\verylongsub
S:\dir2\verylongsub
S:\dir2\verylongsub
S:\dir3
S:\dir3
S:\dir3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
I would use awk:
awk '/^S/ {line=$0} {print NF?line:""}' file
This stores the line starting with S. Then, it prints either this stored value or an empty line if the line is empty.
Test
$ awk '/^S/ {line=$0} {print NF?line:""}' file
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir1\subfolder1longsubf
S:\dir2\verylongsub
S:\dir2\verylongsub
S:\dir2\verylongsub
S:\dir3
S:\dir3
S:\dir3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
S:\dir4\sub2\sub3
With GNU sed:
sed '/^S:/{h};/^[^S]/{g}' file
Related
I have following data
https://link1.com
asndiaiusdias Rye ioajsidsauihduiashd
link1.com/image.jpg
$89.99
https://link2.com
8iqiwudhuiqhwdqwuidhuiqhwi Rye iqwdihqwuidhuiqwhduihqwi
https://link2.com/image.jpg
$22.99
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
https://link4.com
iashduhuasdi rye huiqwheui
https://link4.com/image.jpg
$232.99
My goal is to in case-sensitive match "Rye"(also rye or RYe or rYe) and delete 1 line before the match and 3 lines after match
so result should be:
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
You can use sed,grep,awk no need to use only sed, just need to work
You may use this awk with an empty RS:
awk -v RS= '$3 !~ /^[rR][yY][eE]$/' file
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
$ awk -v RS= 'tolower($3) != "rye"' file
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
or if you can have multiple blocks of text output and want them each separated by a blank line:
$ awk -v RS= -v ORS='\n\n' 'tolower($3) != "rye"' file
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
every other answer is assuming that 1 line before and 3 after actually means paragraphs:
$ perl -00 -ne 'print if !/\Wrye\W/i' input.txt
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
-00 enables paragraph mode
-n doesn't print records by default
'print if !/\Wrye\W/i - prints a paragraph unless it matches
however if 1 line before and 3 after needs to be taken literally:
$ perl -0777 -pe 's/.*\n.*\Wrye\W.*\n(.*\n){3}//ig' input.txt
https://link3.com
8iqiwudhuiqhwdqwuidhuiqhwi SOMETHING ELSE iqwdihqwuidhuiqwhduihqwi
https://link3.com/image.jpg
$42.99
-0777 read the entire file
-p print
.*\n - match a line including the end of line (note that without /s . doesn't match \n)
Note: somebody has raised the dos compatibility issue in a comment. The "." matches any character except newline, which includes \r, thus .*\n covers also dos line endings.
Alternatively, you can use Perl for a job like this:
$ perl -i -pe 'BEGIN{undef $/;} s/.*?\n.*rye.*?\n(^.*?\n){3}///mig' input.txt
$ sed -e "/${exclude}/I,+2d" -i /path/to/file
then I easily managed deleting before line
I need to search and replace the value for CustomerEMailID in xml file with the sed search and replace sed s/// . I was able to get the value for CustomerEMailID, how do I replace it with emailID using sed?
//Print the value
sed -n 's/.*CustomerEMailID="\([^"]*\).*/\1/p' xmltoconvert.xml
//xml file:
<Order
CustomerEMailID="XXXX"
Try this:
sed 's/\(.*CustomerEMailID="\)[^"]*\(.*\)/\1emailID\2/' xmltoconvert.xml
It will replace the value you want and print the whole file to terminal.
Input file:
<Order
CustomerEMailID="XXXX">
Output file:
<Order
CustomerEMailID="emailID">
With awk you could use the following:
awk -v email="emailID" 'BEGIN{FS=OFS="\""}{for(i=1;i<=NF;i++) if($i ~ /CustomerEMailID=/) $(i+1)=email}1' file
It first sets the field separators FS and OFS to ".
Then it will look for the index of parameter matching the pattern /CustomerEMailID=/ and replace the next parameter to the string stored in the awk variable email.
I have a csv file like that :
0;test1;description;toto
1;test2;description;tata
2;test3;desc
ription;tutu
3;test4;description;tete
In shell, I would like to replace all the line that doesn't start with a number.
In this exemple I want to replace \nription by ription
I don't find the correct expression with sed, grep... :(
I want this result :
0;test1;description;toto
1;test2;description;tata
2;test3;description;tutu
3;test4;description;tete
Thanks a lot
EDIT 1 :
I have try something like this :
LC_ALL=C tr '(\n)[0-9]' ' ' < hotels.csv > test.csv
Or this :
sed ':a;N;$!ba;s/\r\n?![0-รง-9]/ /g' hotels.csv
But i think my regex is wrong and it doesn't work :(
With awk this seems feasible:
awk -F ';' '{if (NR>1 && match($1,/^[0-9]+$/)) printf("\n"); printf("%s",$0);} END{printf("\n")}' infile.csv
What it does:
from the second line: check if first field is a number and print a newline
in any line: print the entire line ($0) without trailing newline
Output is sent to STDOUT, input comes from infile.csv
EDIT: Sorry, i missed to copy the match(...)
Using grep -P
grep -P "^\d" file.csv
Use grep to match lines that begin with a digit.
due to peculiarities of sed's pattern space processing, you will have to use something like this ..
Note: ~ must be a char not present in your text
$cat file
0;test1;description;toto
1;test2;description;tata
2;test3;desc
ription;tutu
3;test4;description;tete
$ sed 'N;s/\n/~/' file | sed -r 's/~([0-9])/\n\1/g;s/~//g'
0;test1;description;toto
1;test2;description;tata
2;test3;description;tutu
3;test4;description;tete
PS: if your input file has Windows line endings you will have to use \r\n instead of \n
awk '{sub(/3;desc/,"3;description;tutu")}NR == 4 {next}1' file
0;test1;description;toto
1;test2;description;tata
2;test3;description;tutu
3;test4;description;tete
I've got the following sed replacement, which replaces an entire line with different text, if a certain string is found in the line.
sed "s/.*FOUNDSTRING.*/replacement \"text\" for line"/ test.txt
This works fine - but, for example I want to add a new line after 'for'. My initial thought was to try this:
sed "s/.*FOUNDSTRING.*/replacement \"text\" for \n line"/ test.txt
But this ends out replacing with the following:
replacement "text" for n line
Desired outcome:
replacement "text" for
line
It can be painful to work with newlines in sed. There are also some differences in the behaviour depending on which version you're using. For simplicity and portability, I'd recommend using awk:
awk '{print /FOUNDSTRING/ ? "replacement \"text\" for\nline" : $0}' file
This prints the replacement string, newline included, if the pattern matches, otherwise prints the line $0.
Testing it out:
$ cat file
blah
blah FOUNDSTRING blah
blah
$ awk '{print /FOUNDSTRING/ ? "replacement \"text\" for\nline" : $0}' file
blah
replacement "text" for
line
blah
Of course there is more than one way to skin the cat, as shown in the comments:
awk '/FOUNDSTRING/ { $0 = "replacement \"text\" for \n line" } 1' file
This replaces the line $0 with the new string when the pattern matches and uses the common shorthand 1 to print each line.
How could I use sed to find all lines that don't have exactly 35 occurrences of the "|" character?
If I can't use sed, what could I use?
I'd use awk as it's more readable and doesn't involve nasty regular expression syntax:
awk -F'|' 'NF != 36' filename
Or grep:
grep -v '^\([^|]*|\)\{35\}[^|]*$' filename
But if you want to use sed:
sed '/^\([^|]*|\)\{35\}[^|]*$/d' filename
Here's something you could try:
perl -ne 'print unless (split(/|/, $_)==36);' your_input_file
Splits each line at | and counts the number of resulting parts. If there are 36, you've got 35 | and the line is not printed. Otherwise, the line is printed.