Can anyone please help me write a shell script in linux which would replace the hostname in a particular file.
eg : I have multiple files which have certain ip addresses.
http://10.160.228.12:8001/soa-infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
http://VQAIAAPPDEV:8001/soa-infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
Basically what I would want to replace is the string between "http://" and ":8001" with any required string.
Can someone help me with this please.
Some More info:-
I want to do this iteratively across many folders. So basically it will search all the files in each folder and perform the necessary changes.
You could use sed. Saying:
sed -r 's|(http://)([^:]*)(:8001)|\1something\3|g' filename
would replace is the string between "http://" and ":8001" with something.
If you want to make the change to the file in-place, use the -i option:
sed -i -r 's|(http://)([^:]*)(:8001)|\1something\3|g' filename
Use sed command from Linux shell
sed -i 's%OldHost%NewHost%g' /yourfolder/yourfile
Tried with "for"
# cat replace.txt
http://10.160.228.12:8001/soa- infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
http://VQAIAAPPDEV:8001/soa-infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
# for i in `cat replace.txt | awk -F: '{print $2}' | sed 's/^\/\///g' | sed '/^$/d'` ; do sed -i "s/$i/Your_hostname/" replace.txt ; done
# cat replace.txt
http://Your_hostname:8001/soa- infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
http://Your_hostname:8001/soa-infra/services/default/AIAAsyncErrorHandlingBPELProcess/client?WSDL
Its working for me...!
Related
I have a bunch of txt-files containing stuff like this:
text_i_need_to_remove{text_i_need_to_retain}
text_i need_to_remove{text_i_need_to_retain}
...
How do I remove text before curly braces (and curly braces themselves) and retain just only text_i_need_to_retain?
Deleting everything upto { or } at end of line
:%s/.*{\|}$//g
From bash shell, you can use text processing tools like sed and awk. Assume file is named ip.txt
1) With sed, which is pretty similar to regex we used inside vim. The -i flag allows to make change in place, i.e it modifies the input file itself.
$ sed -i 's/.*{\|}$//g' ip.txt
2) With awk, one can again use substitution or in this case, split the line on curly brackets and use only the second column.
$ awk -F'{|}' '{print $2}' ip.txt > tmp && mv tmp ip.txt
If you have GNU awk, there is -i inplace option for in place editing
$ gawk -i inplace -F'{|}' '{print $2}' ip.txt
To make changed to all files in current directory, use
sed -i 's/.*{\|}$//g' *
Or if they have common extension, say .txt, use
sed -i 's/.*{\|}$//g' *.txt
:%s/^.*{\(.*\)}$/\1/ or in bash, sed 's/^.*{\(.*\)}$/\1/ foo.txt
\(.*\) is a control group which feeds into \1 and looks like a lumbering zombie.
you can use this in vim;
:%s/^.*{// | %s/}$//
you can also use this script; first run this, if everythink is ok, uncomment sed with -i option as below;
#!/bin/bash
for item in $(ls /dir/where/my/files/are)
do
sed -i 's/^.*{//;s/}$//' /dir/where/my/files/are/$item
done
sed -i ; inplace replace
or
Only use as below;
sed -i 's/^.*{//;s/}$//' /dir/where/my/files/are/*
Perl can be used to do the substitution on all files:
perl -i -pe 's/.*{|}$//g' *.txt
I need to insert a command (as string) to an existing file after a certain match. The existing string is a long make command and I only need to modify it by inserting another string at specific location. I tried using sed but it either adds a new line before/after the matching string or replaces it. I'd like to know if at least it is possible to accomplish what I want with sed or should I be using something else? Could you please provide me with some hints?
Example:
The file contains two make commands and I am only interested in the second one without bbnote.
oe_runmake_call() {
bbnote make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="arm-poky-linux-gnueabi-gcc" "$#"
make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="my_command_here arm-poky-linux-gnueabi-gcc" --sysroot=/some/path "$#"
}
Thanks in advance!
Here's the code:
http://hastebin.com/tigatoquje.go
You could do something like this using Sed:
sed -r 's:(^\s+make.+ CC=\"):\1your_command_here :g' file.log >outfile.log
or with sed in-place edit:
sed -ir 's:(^\s+make.+ CC=\"):\1your_command_here :g' file.log
Without sed regex option:
sed 's:\(^\s\+make.\+ CC=\"\):\1your_command_here :g' file.log > outfile.log
Outputs:
oe_runmake_call() {
bbnote make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="arm-poky-linux-gnueabi-gcc" "$#"
make -j 8 CROSS_COMPILE=arm-poky-linux-gnueabi- CC="your_command_here arm-poky-linux-gnueabi-gcc" --sysroot=/some/path "$#"
}
How:
sed -r 's:(^\s+make.+ CC=\"):\1your_command_here :g'
-r = regex option
^make(CC=\") = starts with make and set a capture group on CC="
\1your_command_here = \1 reference capture group then add command text
You could use perl.
Replace YOUR_COMMAND with what you want added. This assumes your file is in file.txt:
perl -i.bak -pl -e '/^make/ and s/(CC=".*")/$1 YOUR_COMMAND /' file.txt
My script gets this string for example:
/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
let's say I don't know how long the string until the /importance.
I want a new variable that will keep only the /importance/lib1/lib2/lib3/file from the full string.
I tried to use sed 's/.*importance//' but it's giving me the path without the importance....
Here is the command in my code:
find <main_path> -name file | sed 's/.*importance//
I am not familiar with the regex, so I need your help please :)
Sorry my friends I have just wrong about my question,
I don't need the output /importance/lib1/lib2/lib3/file but /importance/lib1/lib2/lib3 with no /file in the output.
Can you help me?
I would use awk:
$ echo "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file" | awk -F"/importance/" '{print FS$2}'
importance/lib1/lib2/lib3/file
Which is the same as:
$ awk -F"/importance/" '{print FS$2}' <<< "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
importance/lib1/lib2/lib3/file
That is, we set the field separator to /importance/, so that the first field is what comes before it and the 2nd one is what comes after. To print /importance/ itself, we use FS!
All together, and to save it into a variable, use:
var=$(find <main_path> -name file | awk -F"/importance/" '{print FS$2}')
Update
I don't need the output /importance/lib1/lib2/lib3/file but
/importance/lib1/lib2/lib3 with no /file in the output.
Then you can use something like dirname to get the path without the name itself:
$ dirname $(awk -F"/importance/" '{print FS$2}' <<< "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file")
/importance/lib1/lib2/lib3
Instead of substituting all until importance with nothing, replace with /importance:
~$ echo $var
/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
~$ sed 's:.*importance:/importance:' <<< $var
/importance/lib1/lib2/lib3/file
As noted by #lurker, if importance can be in some dir, you could add /s to be safe:
~$ sed 's:.*/importance/:/importance/:' <<< "/dir1/dirimportance/importancedir/..../importance/lib1/lib2/lib3/file"
/importance/lib1/lib2/lib3/file
With GNU sed:
echo '/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file' | sed -E 's#.*(/importance.*)#\1#'
Output:
/importance/lib1/lib2/lib3/file
pure bash
kent$ a="/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
kent$ echo ${a/*\/importance/\/importance}
/importance/lib1/lib2/lib3/file
external tool: grep
kent$ grep -o '/importance/.*' <<<$a
/importance/lib1/lib2/lib3/file
I tried to use sed 's/.*importance//' but it's giving me the path without the importance....
You were very close. All you had to do was substitute back in importance:
sed 's/.*importance/importance/'
However, I would use Bash's built in pattern expansion. It's much more efficient and faster.
The pattern expansion ${foo##pattern} says to take the shell variable ${foo} and remove the largest matching glob pattern from the left side of the shell variable:
file_name="/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
file_name=${file_name##*importance}
Removeing the /file at the end as you ask:
echo '<path>' | sed -r 's#.*(/importance.*)/[^/]*#\1#'
Input /dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
Returns: /importance/lib1/lib2/lib3
See this "Match groups" tutorial.
Im trying to iterate over each file and folder inside a directory and extract part of the file name into a variable, but I can't make sed work correctly. I either get all of the file name or none of it.
This version of the script should capture the entire file name:
#!/bin/bash
for f in *
do
substring=`echo $f | sed -E -n 's/(.*)/\1/'`
echo "sub: $substring"
done
But instead I get nothing:
sub:
sub:
sub:
sub:
...
This version should give me just the first character in the filename:
#!/bin/bash
for f in *
do
substring=`echo $f | sed -E 's/^([a-zA-Z])/\1/'`
echo "sub: $substring"
done
But instead I get the whole file name:
sub: Adlm
sub: Applications
sub: Applications (Parallels)
sub: Desktop
...
I've tried numerous iterations of it and what it basically boils down to is that if I use -n I get nothing and if I don't I get the whole file name.
Can someone show me how to get just the first character?
Or, my overall goal is to be able to extract a substring and store it into a variable, if anybody has a better approach to it, that would be appreciated as well.
Thanks in advance.
If you want to modify a shell parameter, you probably want to use a parameter expansion.
for f in *; do
# This version should expand to the whole parameter
echo "$f"
# This version should expand to the first character in the filename
echo "${f::1}"
done
Parameter expansions are not as powerful as sed, but they are built in to the shell (no launching a separate process or subshell necessary) and there are expansions for:
Substrings (as above)
Replacing and substituting characters
Altering the case of strings (bash 4+)
and more.
This version of the script should capture the entire file name:
sed -E -n 's/(.*)/\1/'
But instead I get nothing.
You used -n so naturally it won't yield anything. Perhaps you should remove -n or add p:
sed -E -n 's/(.*)/\1/p'
This version should give me just the first character in the filename:
sed -E 's/^([a-zA-Z])/\1/'
But instead I get the whole file name,
You didn't replace anything there. Perhaps what you wanted was
sed -E 's/^([a-zA-Z]).*/\1/'
Also I suggest quoting your arguments well:
substring=`echo "$f" | sed ...'`
Finally the simpler method is to use substring expansion if you're using Bash as suggested by kojiro.
You forget to add .* after the capturing group in sed,
$ for i in *; do substring=`echo $i | sed -E 's/^(.).*$/\1/'`; echo "sub: $substring"; done
It's better to use . instead of [a-zA-Z] because it may fail if the first character starts with any special character.
I prefer awk to sed. It seems to be easier for me to understand.
#!/bin/bash
#set -x
for f in *
do
substring=`echo $f | awk '{print substr($1,1,1)}'`
echo "sub: $substring"
done
Howdie do,
I'm writing a script that will remove duplicate IP's from two files. For example,
grep -rw "123.234.567" /home/test/ips/
/home/test/ips/codingte:123.234.567
/home/test/ips/codingt2:123.234.567
Ok, so that IP is in two different files and so I need to remove the IP from the second file.
The grep gives me the file path and the IP address. My thinking: store the file path in a variable with awk and then use find to go to that file and use sed to remove the duplicate IP, so I changed my grep statement to:
grep -rw "123.234.567" . | awk -F ':' '{print $1}'
which returns:
./codingte
./codingt2
I originally tried to use the fully pathname in the find command, but that didn't work either
find -name /var/cpanel/dips/codingte -exec sed '/123.234.567/d' {} \;
So, I just did a CD in the directory and changed the find command to:
find -name 'codingt2' -exec sed '/123.234.567/d' {} \;
Which runs, but doesn't delete the IP address:
cat codingt2
123.234.567
Now, I know the issue is with the dots in the IP address. They need to be escaped, but I'm not sure how to do this. I've been reading for hours on escaping the regex, but I'm not sure how to do this with sed
Any help would be appreciated. I'm just trying to learn more about regex and using them with other linux tools such as awk and find.
I haven't written the full script yet. I'm trying to break it into pieces and then bring it together in the script.
So you know what the output should look like:
codingte
123.234.567
codingt2
The second file would just have the IP removed
cat FILE1.txt | while read IP ; do sed -i "/^${IP}$/d" FILE2.txt ; done
The command does the following:
There are two files: FILE1.txt and FILE2.txt
It will remove in FILE2.txt lines (in your case, IP addresses) found in FILE1.txt
You want grep -l which only print the filenames containing a match:
grep -lrw "123.234.567" /home/test/ips/
would print
/home/test/ips/codingte
/home/test/ips/codingt2
So, to skip the first file and work on the rest:
grep -l ... | sed 1d | while IFS= read -r filename; do
whatever with "$filename"
done
I think you're just missing the -i argument to sed to edit the files in place.
echo foo > test
find -name test -exec sed -i 's/foo/bar/' {} \;
seems to do the trick.