I need to insert a command (as string) to an existing file after a certain match. The existing string is a long make command and I only need to modify it by inserting another string at specific location. I tried using sed but it either adds a new line before/after the matching string or replaces it. I'd like to know if at least it is possible to accomplish what I want with sed or should I be using something else? Could you please provide me with some hints?
Example:
The file contains two make commands and I am only interested in the second one without bbnote.
oe_runmake_call() {
bbnote make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="arm-poky-linux-gnueabi-gcc" "$#"
make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="my_command_here arm-poky-linux-gnueabi-gcc" --sysroot=/some/path "$#"
}
Thanks in advance!
Here's the code:
http://hastebin.com/tigatoquje.go
You could do something like this using Sed:
sed -r 's:(^\s+make.+ CC=\"):\1your_command_here :g' file.log >outfile.log
or with sed in-place edit:
sed -ir 's:(^\s+make.+ CC=\"):\1your_command_here :g' file.log
Without sed regex option:
sed 's:\(^\s\+make.\+ CC=\"\):\1your_command_here :g' file.log > outfile.log
Outputs:
oe_runmake_call() {
bbnote make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="arm-poky-linux-gnueabi-gcc" "$#"
make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="your_command_here arm-poky-linux-gnueabi-gcc" --sysroot=/some/path "$#"
}
How:
sed -r 's:(^\s+make.+ CC=\"):\1your_command_here :g'
-r = regex option
^make(CC=\") = starts with make and set a capture group on CC="
\1your_command_here = \1 reference capture group then add command text
You could use perl.
Replace YOUR_COMMAND with what you want added. This assumes your file is in file.txt:
perl -i.bak -pl -e '/^make/ and s/(CC=".*")/$1 YOUR_COMMAND /' file.txt
Related
I have a bunch of txt-files containing stuff like this:
text_i_need_to_remove{text_i_need_to_retain}
text_i need_to_remove{text_i_need_to_retain}
...
How do I remove text before curly braces (and curly braces themselves) and retain just only text_i_need_to_retain?
Deleting everything upto { or } at end of line
:%s/.*{\|}$//g
From bash shell, you can use text processing tools like sed and awk. Assume file is named ip.txt
1) With sed, which is pretty similar to regex we used inside vim. The -i flag allows to make change in place, i.e it modifies the input file itself.
$ sed -i 's/.*{\|}$//g' ip.txt
2) With awk, one can again use substitution or in this case, split the line on curly brackets and use only the second column.
$ awk -F'{|}' '{print $2}' ip.txt > tmp && mv tmp ip.txt
If you have GNU awk, there is -i inplace option for in place editing
$ gawk -i inplace -F'{|}' '{print $2}' ip.txt
To make changed to all files in current directory, use
sed -i 's/.*{\|}$//g' *
Or if they have common extension, say .txt, use
sed -i 's/.*{\|}$//g' *.txt
:%s/^.*{\(.*\)}$/\1/ or in bash, sed 's/^.*{\(.*\)}$/\1/ foo.txt
\(.*\) is a control group which feeds into \1 and looks like a lumbering zombie.
you can use this in vim;
:%s/^.*{// | %s/}$//
you can also use this script; first run this, if everythink is ok, uncomment sed with -i option as below;
#!/bin/bash
for item in $(ls /dir/where/my/files/are)
do
sed -i 's/^.*{//;s/}$//' /dir/where/my/files/are/$item
done
sed -i ; inplace replace
or
Only use as below;
sed -i 's/^.*{//;s/}$//' /dir/where/my/files/are/*
Perl can be used to do the substitution on all files:
perl -i -pe 's/.*{|}$//g' *.txt
I have a bunch of files with filenames composed of underscore and dots, here is one example:
META_ALL_whrAdjBMI_GLOBAL_August2016.bed.nodup.sortedbed.roadmap.sort.fgwas.gz.r0-ADRL.GLND.FET-EnhA.out.params
I want to remove the part that contains .bed.nodup.sortedbed.roadmap.sort.fgwas.gz. so the expected filename output would be META_ALL_whrAdjBMI_GLOBAL_August2016.r0-ADRL.GLND.FET-EnhA.out.params
I am using these sed commands but neither one works:
stringZ=META_ALL_whrAdjBMI_GLOBAL_August2016.bed.nodup.sortedbed.roadmap.sort.fgwas.gz.r0-ADRL.GLND.FET-EnhA.out.params
echo $stringZ | sed -e 's/\([[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.\)//g'
echo $stringZ | sed -e 's/\[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.[[:lower:]]\.//g'
Any solution is sed or awk would help a lot
Don't use external utilities and regexes for such a simple task! Use parameter expansions instead.
stringZ=META_ALL_whrAdjBMI_GLOBAL_August2016.bed.nodup.sortedbed.roadmap.sort.fgwas.gz.r0-ADRL.GLND.FET-EnhA.out.params
echo "${stringZ/.bed.nodup.sortedbed.roadmap.sort.fgwas.gz}"
To perform the renaming of all the files containing .bed.nodup.sortedbed.roadmap.sort.fgwas.gz, use this:
shopt -s nullglob
substring=.bed.nodup.sortedbed.roadmap.sort.fgwas.gz
for file in *"$substring"*; do
echo mv -- "$file" "${file/"$substring"}"
done
Note. I left echo in front of mv so that nothing is going to be renamed; the commands will only be displayed on your terminal. Remove echo if you're satisfied with what you see.
Your regex doesn't really feel too much more general than the fixed pattern would be, but if you want to make it work, you need to allow for more than one lower case character between each dot. Right now you're looking for exactly one, but you can fix it with \+ after each [[:lower:]] like
printf '%s' "$stringZ" | sed -e 's/\([[:lower:]]\+\.[[:lower:]]\+\.[[:lower:]]\+\.[[:lower:]]\+\.[[:lower:]]\+\.[[:lower:]]\+\.[[:lower:]]\+\.\)//g'
which with
stringZ="META_ALL_whrAdjBMI_GLOBAL_August2016.bed.nodup.sortedbed.roadmap.sort.fgwas.gz.r0-ADRL.GLND.FET-EnhA.out.params"
give me the output
META_ALL_whrAdjBMI_GLOBAL_August2016.r0-ADRL.GLND.FET-EnhA.out.params
Try this:
#!/bin/bash
for line in $(ls -1 META*);
do
f2=$(echo $line | sed 's/.bed.nodup.sortedbed.roadmap.sort.fgwas.gz//')
mv $line $f2
done
I am using sed to edit some sql script. I want to copy all the lines from the first "CREATE" pattern until the first "ALTER" pattern. The issue I am having is that sed copies all lines between each set of CREATE and ALTER instead of only the first occurrence (more than once).
sed -n -e '/CREATE/,/ALTER/w createTables.sql' $filename
Perl to the rescue:
perl -ne 'print if /CREATE/ .. /ALTER/ && close ARGV' -- "$filename" > createTables.sql
It closes the input when the ALTER is matched, i.e. it doesn't read any further.
Using sed
sed -n '/CREATE/,/ALTER/{p;/ALTER/q}' file > createTables.sql
or alternatively(note the newline)
sed -n '/CREATE/,/ALTER/{w createTables.sql
/ALTER/q}' file
So I have the following string in my config.fish, and init.vim:
Fish: eval sh ~/.config/fish/colors/base16-monokai.dark.sh
Vim: colorscheme base16-monokai
Vim: let g:airline_theme='base16_monokai'
And I have the following shell script:
#!/bin/sh
theme=$1
background=$2
if [ -z '$theme' ]; then
echo "Please provide a theme name."
else
if [ -z '$background' ]; then
$background = 'dark'
fi
base16-builder -s $theme -t vim -b $background > ~/.config/nvim/colors/base16-$theme.vim &&
base16-builder -s $theme -t shell -b $background > ~/.config/fish/colors/base16-$theme.$background.sh &&
base16-builder -s $theme -t vim-airline -b $background > ~/.vim/plugged/vim-airline-themes/autoload/airline/themes/base16_$theme.vim
sed -i -e 's/foo/eval sh ~/.config/fish/colors/base16-$theme.$background.sh/g' ~/Developer/dotfiles/config.fish
sed -i -e 's/foo/colorscheme base16-$theme/g' ~/Developer/dotfiles/init.vim
sed -i -e 's/foo/let g:airline_theme='base16_$theme'/g' ~/Developer/dotfiles/init.vim
fi
Basically the idea is the script will generate whichever theme is passed through using this builder.
I have tried referring this documentation but I am not very skilled at regex so if anybody could give me a hand I would appreciate it.
What I need to happen is once the script is generated sed will look for the above strings and replace theme with the newly generated theme ones.
Try this :
sed -i "s|\(eval sh ~/\.config/fish/colors/base16-\)\([^.]*\)\.\([^.]*\)\\(.*\)|\1$theme.$background\4|
" ~/Developer/dotfiles/config.fish
sed -i "s/\(base16\)\([-_]\)\([a-zA-Z]*\)/\1\2$theme/g" ~/Developer/dotfiles/init.vim
Assuming in the second sed command that the theme is an alphanumeric string. If not, you can complete the character range : [a-zA-Z] with additional characters (eg [a-zA-Z0-9]).
You can replace something in sed using this syntax: sed "s#regex#replacement#g". Because you have /s and 's in your strings, it's easiest not to need to escape them.
There are some characters that need to be escaped to make the regexes. . and $ need to be escaped with a \. The $ in the replacement string needs to be escaped too.
If you want to capture a certain part from match, it's easiest to use char classes. For example, eval sh ~/\.config/fish/colors/base16-([^.]+)\.dark\.sh would be the regex to use if you want your replacement to be airline_theme='$1_base16_\$theme'. In that case, the $1 in the replacement is the thing captured in the regex.
[^.]+ will capture everything up to the next .
I hope this helps you to better understand regexes! This should be detailed enough to show you how to write your own.
You need to use double quotes for parameter expansion not single quotes.
You need to escape the single quotes: 'hello'\''world'
I will make one line for you and leave it as an exercise to fix the other lines
sed -i -e 's~\(let g:airline_theme='\''\)[^'\'']*\('\'\)'~base16_'"$theme"~' ~/Developer/dotfiles/init.vim
The first character after the s in the sed expression string is used as the pattern separator, so by putting / first you have specified / as the separator.
Additionally using the single quote tells the shell not to expand any variables, you are going to want to use double quotes instead.
try something like
sed -i -e "s#foo#eval sh ~/.config/fish/colors/base16-$theme.$background.sh#g" ~/Developer/dotfiles/config.fish
as you've now commented that you needed to find the previous theme string instead of foo
sed -i -e "s#eval sh \~/\.config/fish/colors/base16-.*?\..*?\.sh#eval sh ~/.config/fish/colors/base16-$theme.$background.sh#g" ~/Developer/dotfiles/config.fish
Im trying to iterate over each file and folder inside a directory and extract part of the file name into a variable, but I can't make sed work correctly. I either get all of the file name or none of it.
This version of the script should capture the entire file name:
#!/bin/bash
for f in *
do
substring=`echo $f | sed -E -n 's/(.*)/\1/'`
echo "sub: $substring"
done
But instead I get nothing:
sub:
sub:
sub:
sub:
...
This version should give me just the first character in the filename:
#!/bin/bash
for f in *
do
substring=`echo $f | sed -E 's/^([a-zA-Z])/\1/'`
echo "sub: $substring"
done
But instead I get the whole file name:
sub: Adlm
sub: Applications
sub: Applications (Parallels)
sub: Desktop
...
I've tried numerous iterations of it and what it basically boils down to is that if I use -n I get nothing and if I don't I get the whole file name.
Can someone show me how to get just the first character?
Or, my overall goal is to be able to extract a substring and store it into a variable, if anybody has a better approach to it, that would be appreciated as well.
Thanks in advance.
If you want to modify a shell parameter, you probably want to use a parameter expansion.
for f in *; do
# This version should expand to the whole parameter
echo "$f"
# This version should expand to the first character in the filename
echo "${f::1}"
done
Parameter expansions are not as powerful as sed, but they are built in to the shell (no launching a separate process or subshell necessary) and there are expansions for:
Substrings (as above)
Replacing and substituting characters
Altering the case of strings (bash 4+)
and more.
This version of the script should capture the entire file name:
sed -E -n 's/(.*)/\1/'
But instead I get nothing.
You used -n so naturally it won't yield anything. Perhaps you should remove -n or add p:
sed -E -n 's/(.*)/\1/p'
This version should give me just the first character in the filename:
sed -E 's/^([a-zA-Z])/\1/'
But instead I get the whole file name,
You didn't replace anything there. Perhaps what you wanted was
sed -E 's/^([a-zA-Z]).*/\1/'
Also I suggest quoting your arguments well:
substring=`echo "$f" | sed ...'`
Finally the simpler method is to use substring expansion if you're using Bash as suggested by kojiro.
You forget to add .* after the capturing group in sed,
$ for i in *; do substring=`echo $i | sed -E 's/^(.).*$/\1/'`; echo "sub: $substring"; done
It's better to use . instead of [a-zA-Z] because it may fail if the first character starts with any special character.
I prefer awk to sed. It seems to be easier for me to understand.
#!/bin/bash
#set -x
for f in *
do
substring=`echo $f | awk '{print substr($1,1,1)}'`
echo "sub: $substring"
done