I have a file with different word in each line. My goal is to replace ",ca.," specific characters to ",ca~". but when i am trying to replece throuh sed commad, all comma separated values replacing with '~'. Please help.
cat file1.csv | tr ",ca.," ",ca~" > file2.csv
sed -i s/',ca.,'/',ca.'/g file3.csv
perl -i -pe 's/,ca.,/,ca~/g' file4.csv
sed -Ei 's/,ca.,|,ca,/,ca~/g' file5.csv
Related
this is my original string:
NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1
I want to only add back slash to all the spaces before ':'
so, this is what I finally want:
NetworkManager/system\ connections/Wired\ 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1
I need to do this in bash, so, sed, awk, grep are all ok for me.
I have tried following sed, but none of them work
echo NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1 | sed 's/ .*\(:.*$\)/\\ .*\1/g'
echo NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1 | sed 's/\( \).*\(:.*$\)/\\ \1.*\2/g'
echo NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1 | sed 's/ .*\(:.*$\)/\\ \1/g'
echo NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1 | sed 's/\( \).*\(:.*$\)/\\ \1\2/g'
thanks for answering my question.
I am still quite newbie to stackoverflow, I don't know how to control the format in comment.
so, I just edit my original question
my real story is:
when I do grep or use cscope to search keyword, for example "address1" under /etc folder.
the result would be like:
./NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1
if I use vim to open file under cursor, suppose my vim cursor is now at word "NetworkManager",
then vim will understand it as
"./NetworkManager/system"
that's why I want to add "\" before space, so the search result would be more vim friendly:)
I did try to change cscope's source code, but very difficult to fully achieve this. so have to do a post replacement:(
If you only want to do the replacements if there is a : present in the string, you can check if there are at least 2 columns, setting the (output)field separator to a colon.
Data:
cat file michaelvandam#Michaels-MacBook-Pro
NetworkManager/system connections/Wired 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1
NetworkManager/system connections/Wired 1.nmconnection 14 address1=10.1.10.71/24,10.1.10.1%
Example in awk:
awk 'BEGIN {FS=OFS=":"}{if(NF>1)gsub(" ","\\ ",$1)}1' file
Output
NetworkManager/system\ connections/Wired\ 1.nmconnection:14 address1=10.1.10.71/24,10.1.10.1
NetworkManager/system connections/Wired 1.nmconnection 14 address1=10.1.10.71/24,10.1.10.1
This could be simply done in awk program, with your shown samples, please try following.
awk 'BEGIN{FS=OFS=":"} {gsub(/ /,"\\\\&",$1)} 1' Input_file
Explanation: Simple explanation would be, setting field separator and output field separator as : for this program. Then in main program using gsub(Global substitution) function of awk. Where substituting space with \ in 1st field only(as per OP's remarks it should be done before :) and printing line then.
An idea for a perl one liner in bash to use \G and \K (similar #CarySwoveland's comment).
perl -pe 's/\G[^ :]*\K /\\ /g' myfile
See this demo at tio.run or a pattern demo at regex101.
This might work for you (GNU sed):
sed -E ':a;s/^([^: ]*) /\1\n/;ta;s/\n/\\ /g' file
Replace spaces before : by newlines then replace newlines by \ 's.
Alternative using the hold space:
sed -E 's/:/\n:/;h;s/ /\\ /g;G;s/\n.*\n//' file
Split the line on the first :.
Amend the front section, remove the middle and append the unadulterated back section.
My answer is ugly and I think RavinderSingh13's answer is THE ONE, but I already took the time to write mine and it works (It's written step by step, but it's a one line command):
I got inspired by HatLess answer:
first get the text before the : with cut (I put the string in a file to make it easy to read, but this works on echo):
cut -d':' -f1 infile
Then replace spaces using sed:
cut -d':' -f1 infile | sed 's/\([a-z]\) /\1\\ /g'
Then echo the output with no new line:
echo -n "$(cut -d':' -f1 infile | sed -e 's/\([a-z]\) /\1\\ /g')"
Add the missing : and what comes after it:
echo -n "$(cut -d':' -f1 infile | sed -e 's/\([a-z]\) /\1\\ /g')" | cat - <(echo -n :) | cat - <(cut -d':' -f2 infile)
I have a text file that has lines of different lengths. I need to make these uniform so that the PLSQL Developers text import function reads them correctly. Lines that are 89 characters long need to be padded with 4 spaces on the end. For some reason the -i argument to sed isn't accepted either.
The file can be found here
I have tried a number of different regex commands found from various sources through Google but none of them are working, either because the 'function cannot be parsed' or it doesn't add the spaces needed.
The code that I wrote that worked using Notepad++ was
Find: (^.{89})($)
Replace: \1 \2
I've tried a number of unix sed commands such as
sed -e "s/(^.{89})($)/\1 \2/" file.txt
sed -e "s/(^.{89})($)/\1\s\s\s\s\2/" file.txt
sed -e "s/(^.{89})($)/\1\ \ \ \ \2/" file.txt
sed -e "s/\(^.\{89\}\)\($\)/\1\ \ \ \ \2" file.txt
sed -e 's/\(^.\{89\}\)\($\)/\1[[:space:]]\2/g' file.txt
sed -e 's/\(^.\{89\}\)\($\)/\1[[:space:]]\{4\}\2/g' file.txt
sed -e 's/(^.{89})($)/\1[[:space:]]{4}\2/g' file.txt
The main issue here is that you are using BRE POSIX syntax and unescaped ( / ) are treated as literal parentheses and unescaped {/} are treated as literal braces.
This should work:
sed 's/^.\{89\}$/& /' file.txt > newfile.txt
Here:
^ - matches the start of a line
.\{89\} - matches any 89 chars
$ - asserts the end of line position.
The & in the replacement refers to the whole match.
If you need to use -i option, see sed edit file in place.
A software tools kludge for HP-UX, based on its manuals:
sed 's/.*/ /' file | paste -d ' ' file - | cut -c 1-93
How it works:
Use sed to create a dummy stream of the same number of lines as file, but with four spaces on each line.
Use paste to append the dummy stream to what's in file. (Because of the -d ' ' it adds five spaces, but it doesn't much matter.)
Use cut to chop off anything over 93 bytes.
If HP-UX sed is even less like GNU sed than I've supposed, it could be replaced with some equivalent like:
yes ' ' | head -n $(wc -l < file) | paste -d ' ' file - | cut -c 1-93
Try this:
awk '{ <br>
diff = 89 - length($0); <br>
for(i=1;i<=diff; i++) <br>
{ <br>
$0 = $0 "a" <br>
} <br>
print $0 <br>
} <br>
' FileName.txt
I've read a lot of questions about how to replace spaces from a file but I have the following problem:
I have a file like so:
<foo>"crazy foo"</foo> <bar>dull-bar</bar>
and I'm trying to remove spaces between > < and only those ones so the file would be like:
`<foo>"crazy foo"</foo><bar>dull-bar</bar>`
So far I've tried to remove then by using sed and tr. Sed is not working by any chance and using tr '> <' '><' outputs:
<foo>"crazy foo"</foo><<bar>dull-bar</bar>
sed -i -e "s/> *</></g" YourFile
-i means YourFile is modified. Remove this option to test your command and display the result in shell output.
* matches n spaces.
The g at the end of sed expression means "Replace all the occurrences".
You could try something like this
echo "<foo>"crazy foo"</foo> <bar>dull-bar</bar>" | sed 's/>[[:space:]]*</></g '
awk -F"\"" '{print $3}' file.txt | sed 's/ //g'
I have a file with strings similar to this:
abcd u'current_count': u'2', u'total_count': u'3', u'order_id': u'90'
I have to find current_count and total_count for each line of file. I am trying below command but its not working. Please help.
grep current_count file | sed "s/.*\('current_count': u'\d+'\).*/\1/"
It is outputting the whole line but I want something like this:
'current_count': u'3', 'total_count': u'3'
It's printing the whole line because the pattern in the s command doesn't match, so no substitution happens.
sed regexes don't support \d for digits, or x+ for xx*. GNU sed has a -r option to enable extended-regex support so + will be a meta-character, but \d still doesn't work. GNU sed also allows \+ as a meta-character in basic regex mode, but that's not POSIX standard.
So anyway, this will work:
echo -e "foo\nabcd u'current_count': u'2', u'total_count': u'3', u'order_id': u'90'" |
sed -nr "s/.*('current_count': u'[0-9]+').*/\1/p"
# output: 'current_count': u'2'
Notice that I skip the grep by using sed -n s///p. I could also have used /current_count/ as an address:
sed -r -e '/current_count/!d' -e "s/.*('current_count': u'[0-9]+').*/\1/"
Or with just grep printing only the matching part of the pattern, instead of the whole line:
grep -E -o "'current_count': u'[[:digit:]]+'
(or egrep instead of grep -E). I forget if grep -o is POSIX-required behaviour.
For me this looks like some sort of serialized Python data. Basically I would try to find out the origin of that data and parse it properly.
However, while being hackish, sed can also being used here:
sed "s/.*current_count': [a-z]'\([0-9]\+\).*/\1/" input.txt
sed "s/.*total_count': [a-z]'\([0-9]\+\).*/\1/" input.txt
I want to get rid of all invalid characters; example hexadecimal value 0x1A from an XML file using sed.
What is the regex and the command line?
EDIT
Added Perl tag hoping to get more responses. I prefer a one-liner solution.
EDIT
These are the valid XML characters
x9 | xA | xD | [x20-xD7FF] | [xE000-xFFFD] | [x10000-x10FFFF]
Assuming UTF-8 XML documents:
perl -CSDA -pe'
s/[^\x9\xA\xD\x20-\x{D7FF}\x{E000}-\x{FFFD}\x{10000}-\x{10FFFF}]+//g;
' file.xml > file_fixed.xml
If you want to encode the bad bytes instead,
perl -CSDA -pe'
s/([^\x9\xA\xD\x20-\x{D7FF}\x{E000}-\x{FFFD}\x{10000}-\x{10FFFF}])/
"&#".ord($1).";"
/xeg;
' file.xml > file_fixed.xml
You can call it a few different ways:
perl -CSDA -pe'...' file.xml > file_fixed.xml
perl -CSDA -i~ -pe'...' file.xml # Inplace with backup
perl -CSDA -i -pe'...' file.xml # Inplace without backup
The tr command would be simpler. So, try something like:
cat <filename> | tr -d '\032' > <newfilename>
Note that ascii character '0x1a' has the octal value '032', so we use that instead with tr. Not sure if tr likes hex.
Try:
perl -pi -e 's/[^\x9\xA\xD\x20-\x{d7ff}\x{e000}-\x{fffd}\x{10000}-\x{10ffff}]//g' file.xml
There is actually a way to do this with sed, like so:
cat input_file | LANG=C sed -E \
-e 's/.*/& /g' \
-e 's/(('\
'[\x9\xa\xd\x20-\x7f]|'\
'[\xc0-\xdf][\x80-\xbf]|'\
'[\xe0-\xec][\x80-\xbf][\x80-\xbf]|'\
'[\xed][\x80-\x9f][\x80-\xbf]|'\
'[\xee-\xef][\x80-\xbf][\x80-\xbf]|'\
'[\xf0][\x80-\x8f][\x80-\xbf][\x80-\xbf]'\
')*)./\1?/g' \
-e 's/(.*)\?/\1/g' \
-e 's|]]>|]]>]]<![CDATA[>|g' > output_file
This works in four steps:
Add a single whitespace character to the end of every line.
Replace every sequence of legal characters followed by any character
with the same sequence of legal characters followed by a question mark
character (instead of the any).
Note that in a line of only legal characters, the '.' matches the last
character in the line, which is why we added a space in step 1.
Remove the last character in the line, which we expect to be a question mark.
Replace the string ']]>' with ']]>]]'.
The LANG=C env variable is set to prevent sed from doing charset conversion itself - it should treat every character as 8-bit ascii.