I tried to do the following command in bash:
ls -1 | sed s/\(.*\)/"\1"/
which is add double quotes around each output of ls, but the result shows
sed: 1: "s/(.*)/\1/": \1 not defined in the RE
after I add single quotes around the regular expression, I got the right result. the right one is:
ls -1 | sed 's/\(.*\)/"\1"/'
theocratically I do not need the outer quotes right? any one has the same experience?
Single quotes are used to disable shell parsing of various sequences including backslash escapes. If you don't use them, your sequences like \( are passed to sed as (. You may check that by adding echo to the beginning of your command.
Sending the command to echo will show you what sed sees
$ echo sed s/\(.*\)/"\1"/
sed
Hmm, the sed script disappeared altogether. The exposed "*" is forcing the shell to try to match files. Let's disable that:
$ set -f
$ echo sed s/\(.*\)/"\1"/
sed s/(.*)/\1/
The shell ate the quotes and the backslashes. Quoting the sed script:
$ echo sed 's/\(.*\)/"\1"/'
sed s/\(.*\)/"\1"/
That gives the right result, sed will see the script you want to give it. How can we do that without quotes
$ echo sed s/\\\(.\*\\\)/\"\\1\"/
sed s/\(.*\)/"\1"/
And that's ugly.
I have a text file with a line that reads:
<div id="page_footer"><div><? print('Any phrase's characters can go here!'); ?></div></div>
And I'm wanting to use sed or awk to extract the substring above between the single quotes so it just prints ...
Any phrase's characters can go here!
I want the phrase to be delimited as I have above, starting after the single quote and ending at the single-quote immediately followed by a parenthesis and then semicolon. The following sed command with a capture group doesn't seem to be working for me. Suggestions?
sed '/^<div id="page_footer"><div><? print(\'\(.\+\)\');/ s//\1/p' /home/foobar/testfile.txt
Incorrect would be using cut like
grep "page_footer" /home/foobar/testfile.txt | cut -d "'" -f2
It will go wrong with single quotes inside the string. Counting the number of single quotes first will change this from a simple to an over-complicated solution.
A solution with sed is better: remove everything until the first single quote and everything after the last one. A single quote in the string becomes messy when you first close the sed parameter with a single quote, escape the single quote and open a sed string again:
grep page_footer /home/foobar/testfile.txt | sed -e 's/[^'\'']*//' -e 's/[^'\'']*$//'
And this is not the full solution, you want to remove the first/last quotes as well:
grep page_footer /home/foobar/testfile.txt | sed -e 's/[^'\'']*'\''//' -e 's/'\''[^'\'']*$//'
Writing the sed parameters in double-quoted strings and using the . wildcard for matching the single quote will make the line shorter:
grep page_footer /home/foobar/testfile.txt | sed -e "s/^[^\']*.//" -e "s/.[^\']*$//"
Using advanced grep (such as in Linux), this might be what you are looking for
grep -Po "(?<=').*?(?='\);)"
I have a sql file and I want to separate every statement in a new line. To do this I need to replace "),(" with "\n". I tried the following but I doesn't work:
sed ’s/\),\(/\n/’ tables.sql
Thanks
Escaped parenthesis in sed are capture groups. The syntax would be:
$ echo $'(a),(b)' | sed 's/),(/\\n/g'
(a\nb)
I am trying to remove single quotes and double quotes from a file. Can I do it in a single sed command?
I am trying :
sed 's/\"//g;s/\'//g' txt file
but get this error
`' ' is unmatched.
Please help.
Another possibility would be to use tr:
tr -d \'\" file
You cannot escape a single quote inside a pair of singe quotes in shell. Escaping double quotes is allowed though. Following sed command should work:
sed "s/['\"]//g" file
Try this one instead :
sed -e 's|["'\'']||g' txt
To remove single quotes, simply use double quotes for the regex in sed:
sed -e "s/'//g" ./new_file.csv
You can use commands below
sed "s/'/ /g" file.txt > newfile.txt
sed 's/\"//g' newfile.txt > Required_file.txt
Required_file.txt is the final output.
I solved it (in Centos 7) by removing surrounding quotes all together like:
sed -i s/\'//g file;sed -i s/\"//g file
Well, here's what I've came to.
First, I found out with ord() what are codes for single and double quotes characters, and then used $(..) syntax to pass it into unquoted sed expression. I used XX and yy instead of empty strings. Obviously it is faaar from optimal, i.e. they perhaps should be combined into one expression, I encourage you to experiment with it.
There are several techniques to avoid quoting problems, you can also put sed expression into separate file, to avoid it to be interpreted by shell. The ord() / chr() trick is also useful when trying to deal with single unreadable characters in output, e.g. UTF strings on non-UTF console.
dtpwmbp:~ pwadas$ echo '"' | perl -pe 'print ord($_) . "\n";'
34
"
dtpwmbp:~ pwadas$ echo "'" | perl -pe 'print ord($_) . "\n";'
39
'
dtpwmbp:~ pwadas$ echo \'\"
'"
dtpwmbp:~ pwadas$ echo \'\" | sed -e s/$(perl -e 'print chr(34) . "\n"')/XX/g | sed -e s/$(perl -e 'print chr(39) . "\n"')/yy/g
yyXX
dtpwmbp:~ pwadas$
EDIT (note that this time, both characters are replaced with the same string "yy").There might be some shell utilities for "translating" characters to character codes and opposite, i.e. it should be possible to do this without using perl or other language interpreter.
dtpwmbp:~ pwadas$ echo \'\" | sed -e s/[`perl -e 'print chr(34) . chr(39)'`]/yy/g
yyyy
dtpwmbp:~ pwadas$
and here's yet another way in shell, perhaps even simpler
dtpwmbp:~ pwadas$ X="'"; Y='"' ; echo $X$Y; echo $X$Y | sed -e "s/$X/aa/g;s/$Y/bb/g"
'"
aabb
dtpwmbp:~ pwadas$
I've got a CSV file that looks like:
1,3,"3,5",4,"5,5"
Now I want to change all the "," not within quotes to ";" with sed, so it looks like this:
1;3;"3,5";5;"5,5"
But I can't find a pattern that works.
If you are expecting only numbers then the following expression will work
sed -e 's/,/;/g' -e 's/\("[0-9][0-9]*\);\([0-9][0-9]*"\)/\1,\2/g'
e.g.
$ echo '1,3,"3,5",4,"5,5"' | sed -e 's/,/;/g' -e 's/\("[0-9][0-9]*\);\([0-9][0-9]*"\)/\1,\2/g'
1;3;"3,5";4;"5,5"
You can't just replace the [0-9][0-9]* with .* to retain any , in that is delimted by quotes, .* is too greedy and matches too much. So you have to use [a-z0-9]*
$ echo '1,3,"3,5",4,"5,5",",6","4,",7,"a,b",c' | sed -e 's/,/;/g' -e 's/\("[a-z0-9]*\);\([a-z0-9]*"\)/\1,\2/g'
1;3;"3,5";4;"5,5";",6";"4,";7;"a,b";c
It also has the advantage over the first solution of being simple to understand. We just replace every , by ; and then correct every ; in quotes back to a ,
You could try something like this:
echo '1,3,"3,5",4,"5,5"' | sed -r 's|("[^"]*),([^"]*")|\1\x1\2|g;s|,|;|g;s|\x1|,|g'
which replaces all commas within quotes with \x1 char, then replaces all commas left with semicolons, and then replaces \x1 chars back to commas. This might work, given the file is correctly formed, there're initially no \x1 chars in it and there're no situations where there is a double quote inside double quotes, like "a\"b".
Using gawk
gawk '{$1=$1}1' FPAT="([^,]+)|(\"[^\"]+\")" OFS=';' filename
Test:
[jaypal:~/Temp] cat filename
1,3,"3,5",4,"5,5"
[jaypal:~/Temp] gawk '{$1=$1}1' FPAT='([^,]+)|(\"[^\"]+\")' OFS=';' filename
1;3;"3,5";4;"5,5"
This might work for you:
echo '1,3,"3,5",4,"5,5"' |
sed 's/\("[^",]*\),\([^"]*"\)/\1\n\2/g;y/,/;/;s/\n/,/g'
1;3;"3,5";4;"5,5"
Here's alternative solution which is longer but more flexible:
echo '1,3,"3,5",4,"5,5"' |
sed 's/^/\n/;:a;s/\n\([^,"]\|"[^"]*"\)/\1\n/;ta;s/\n,/;\n/;ta;s/\n//'
1;3;"3,5";4;"5,5"