Rewrite URL using sed while maintaining filename

Rewrite URL using sed while maintaining filename - regex

I would like to find all instances of a URL in a file and replace them with a different link structure.
An example would be convert http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png to /images/Security_Panda.png.
I am able to identify the link using a regular expression such as:
^(http:)|([/|.|\w|\s])*\.(?:jpg|gif|png)
but need to rewrite using sed so that the file name is maintained. I understand that I will need to use s/${PATTERN}/${REPLACEMENT}/g.
Tried: sed -i 's#(http:)|([/|.|\w|\s])*\.(?:jpg|gif|png)#/dir/$1#g' test without success? Thoughts on how to improve the approach?

In basic sed, you need to escape the () symbols like \(..\) to mean a capturing group.
sed 's~http://[.a-zA-Z0-9_/-]*\/\(\w\+\.\(jpg\|gif\|png\)\)~/images/\1~g' file
Example:
$ echo 'http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png' | sed 's~http://[.a-zA-Z0-9_/-]*\/\(\w\+\.\(jpg\|gif\|png\)\)~/images/\1~g'
/images/Security_Panda.png

You can use:
sed 's~^.*/\([^/]\{1,\}\)$~/images/\1~' file
/images/Security_Panda.png
Testing:
s='http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png'
sed 's~^.*/\([^/]\{1,\}\)$~/images/\1~' <<< "$s"
/images/Security_Panda.png

Easier way if you change your idea.
#!/usr/bin/env bash
URL="http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png"
echo "/image/${URL##*/}"

Another way
command line
sed 's#^http:.*/\(.*\).$#/images/\1#g'
Example
echo "http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png "|sed 's#^http:.*/\(.*\).$#/images/\1#g'
results
/images/Security_Panda.png

An awk version:
awk -F\/ '/(jpg|gif|png) *$/ {print "/images/"$NF}' file
/images/Security_Panda.png

Related

Swap columns in bash using SED without using loop

I'm new to Sed, I'm trying to learn some pattern using Sed.
I got a filenamne.txt that has the following entry:
ppp/jjj qqq/kkk rrr/lll
My goal is to swap the word before the slash and the word after the slash in each of the three word1/word2 columns:
jjj/ppp kkk/qqq lll/rrr
I tried using sed –re ‘s!(.*)(/)(.*)!\1\2\!’ filename.txt, but it didn't work. Any idea how can I go about it?

$ echo "ppp/jjj qqq/kkk rrr/lll" | sed -e 's/$/ /' -e 's!\([^/]*\)/\([^ ]*\) !\2/\1 !g'
jjj/ppp kkk/qqq lll/rrr

Use replacement in perl command-line is a lot more straight-forward :-
perl -pe 's/(\w+)\/(\w+)/$2\/$1/g' file
jjj/ppp kkk/qqq lll/rrr

$ sed 's#\([^ ]*\)/\([^ ]*\)#\2/\1#g' file
jjj/ppp kkk/qqq lll/rrr

BASH: replacing PERL with SED for in-place substitution

Would like to replace this statement with perl:
perl -pe "s|(?<=://).+?(?=/)|$2:80|"
with
sed -e "s|<regex>|$2:80|"
Since sed has a much less powerful regex engine (for example it does not support look-arounds) the task boils down to writing a sed compatible regex to match only a domain name in a fully qualitied URL. Examples:
http://php2-mindaugasb.c9.io/Testing/JS/displayName.js
http://php2-mindaugasb.c9.io?a=Testing.js
http://www.google.com?a=Testing.js
Should become:
http://$2:80/Testing/JS/displayName.js
http://$2:80?a=Testing.js
http://$2:80?a=Testing.js
A solution like this would be ok:
sed -e "s|<regex>|http://$2:80|"
Thanks :)

Use the below sed command.
$ sed "s~//[^/?]\+\([?/]\)~//\$2:80\1~g" file
http://$2:80/Testing/JS/displayName.js
http://$2:80?a=Testing.js
http://$2:80?a=Testing.js
You must need to escape the $ at the replacement part.

sed 's|http://[^/?]*|http://$2:80|' file
Output:
http://$2:80/Testing/JS/displayName.js
http://$2:80?a=Testing.js
http://$2:80?a=Testing.js

How to cut a string from a string

My script gets this string for example:
/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
let's say I don't know how long the string until the /importance.
I want a new variable that will keep only the /importance/lib1/lib2/lib3/file from the full string.
I tried to use sed 's/.*importance//' but it's giving me the path without the importance....
Here is the command in my code:
find <main_path> -name file | sed 's/.*importance//
I am not familiar with the regex, so I need your help please :)
Sorry my friends I have just wrong about my question,
I don't need the output /importance/lib1/lib2/lib3/file but /importance/lib1/lib2/lib3 with no /file in the output.
Can you help me?

I would use awk:
$ echo "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file" | awk -F"/importance/" '{print FS$2}'
importance/lib1/lib2/lib3/file
Which is the same as:
$ awk -F"/importance/" '{print FS$2}' <<< "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
importance/lib1/lib2/lib3/file
That is, we set the field separator to /importance/, so that the first field is what comes before it and the 2nd one is what comes after. To print /importance/ itself, we use FS!
All together, and to save it into a variable, use:
var=$(find <main_path> -name file | awk -F"/importance/" '{print FS$2}')
Update
I don't need the output /importance/lib1/lib2/lib3/file but
/importance/lib1/lib2/lib3 with no /file in the output.
Then you can use something like dirname to get the path without the name itself:
$ dirname $(awk -F"/importance/" '{print FS$2}' <<< "/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file")
/importance/lib1/lib2/lib3

Instead of substituting all until importance with nothing, replace with /importance:
~$ echo $var
/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
~$ sed 's:.*importance:/importance:' <<< $var
/importance/lib1/lib2/lib3/file
As noted by #lurker, if importance can be in some dir, you could add /s to be safe:
~$ sed 's:.*/importance/:/importance/:' <<< "/dir1/dirimportance/importancedir/..../importance/lib1/lib2/lib3/file"
/importance/lib1/lib2/lib3/file

With GNU sed:
echo '/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file' | sed -E 's#.*(/importance.*)#\1#'
Output:
/importance/lib1/lib2/lib3/file

pure bash
kent$ a="/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
kent$ echo ${a/*\/importance/\/importance}
/importance/lib1/lib2/lib3/file
external tool: grep
kent$ grep -o '/importance/.*' <<<$a
/importance/lib1/lib2/lib3/file

I tried to use sed 's/.*importance//' but it's giving me the path without the importance....
You were very close. All you had to do was substitute back in importance:
sed 's/.*importance/importance/'
However, I would use Bash's built in pattern expansion. It's much more efficient and faster.
The pattern expansion ${foo##pattern} says to take the shell variable ${foo} and remove the largest matching glob pattern from the left side of the shell variable:
file_name="/dir1/dir2/dir3.../importance/lib1/lib2/lib3/file"
file_name=${file_name##*importance}

Removeing the /file at the end as you ask:
echo '<path>' | sed -r 's#.*(/importance.*)/[^/]*#\1#'
Input /dir1/dir2/dir3.../importance/lib1/lib2/lib3/file
Returns: /importance/lib1/lib2/lib3
See this "Match groups" tutorial.

Sed replace domain in URL

I have these strings http://sub.domain.com/myuri/default.aspx, https://sub.domain.com/myuri/default.aspx and https://domain.com
Is it possible to use sed to replace only the domain part?
For example, this URL:
http://sub.domain.com/myuri/default.aspx
Would become:
http://anotherdomain.com/myuri/default.aspx
Please note that the protocol may differ between https and http.
I did search but could not find something similar.

You will need non-greedy pattern that sed can't offer, use perl instead:
perl -pe '/(http|https):\/\/(.*?)(\/|$)/ && s/$2/anotherdomain/g'
Edit:
awk also does the job well and it's even simpler actually:
awk -F/ 'gsub($3,"anotherdomain",$0)' <<< "$urls"
Example:
#!/bin/bash
urls=$(cat << 'EOF'
https://sub.domain.com/myuri/default.aspx
http://sub.domain.com/myuri/default.aspx
http://blabla
EOF
)
perl -pe '/(http|https):\/\/(.*?)(\/|$)/ && s/$2/anotherdomain/g' <<< "$urls"
Output:
bash test.sh
https://anotherdomain/myuri/default.aspx
http://anotherdomain/myuri/default.aspx
http://anotherdomain

If I follow your question, then yes sed 's/sub\.domain\.com/anotherdomain\.com/1' -
echo "http://sub.domain.com/myuri/default.aspx" | \
sed 's/sub\.domain\.com/anotherdomain\.com/1'
Output is
http://anotherdomain.com/myuri/default.aspx
And with,
echo "https://sub.domain.com/myuri/default.aspx" | \
sed 's/sub\.domain\.com/anotherdomain\.com/1'
Output is
https://anotherdomain.com/myuri/default.aspx

You can use sed like this:
sed -r 's|(https?://)[^/]+([[^:blank:]]*)|\1anotherdomain.com\2|g' file
http://anotherdomain.comn.com/myuri/default.aspx
https://anotherdomain.comn.com/myuri/default.aspx
https://anotherdomain.comn.com
PS: Use sed -E on OSX.

Based on #hek2mgl's solution:
SERVER=www.example.com
sed "s=\(https\?://\)[^/]\+=\1$SERVER=" \
<<< 'https://anotherdomain.com/myuri/default.aspx'
It will output:
https://www.example.com/myuri/default.aspx
Modifications from hek2mgl's sed line:
a little shorter (no need to catch the part after domain name to paste it as is in replacement)
deals with both http:// and https:// syntax

You can use sed:
SERVER=www.example.com
sed "s~https\?://\([^/]\+\)\(.*\)~http://$SERVER\2~" <<< "http://newsub.domain.com/myuri/default

Regex with sed to parse archive name

I'd like to parse different kinds of Java archive with the sed command line tool.
Archives can have the followin extensions:
.jar, .war, .ear, .esb
What I'd like to get is the name without the extension, e.g. for Foobar.jar I'd like to get Foobar.
This seems fairly simple, but I cannot come up with a solution that works and is also robust.
I tried something along the lines of sed s/\.+(jar|war|ear|esb)$//, but could not make it work.

You were nearly there:
sed -E 's/\.+(jar|war|ear|esb)$//' file
Just needed to add the -E flag to sed to interpret the expression. And of course, respect the sed 's/something/new/' syntax.
Test
$ cat a
aaa.jar
bb.war
hello.ear
buuu.esb
hello.txt
$ sed -E 's/\.+(jar|war|ear|esb)$//' a
aaa
bb
hello
buuu
hello.txt

Using sed:
s='Foobar.jar'
sed -r 's/\.(jar|war|ear|esb)$//' <<< "$s"
Foobar
OR better do it in BASH itself:
echo "${s/.[jwe]ar/}"
Foobar

You need to escape the | and the () and also add ' if you do not add option like -r or -E
echo "test.jar" | sed 's/\.\(jar\|war\|ear\|esb\)$//'
test
* is also not needed, sine you normal have only one .

On traditionnal UNIX (tested with AIX/KSH)
File='Foobar.jar'
echo ${File%.*}
from a list having only your kind of file
YourList | sed 's/\....$//'
form a list of all kind of file
YouList | sed -n 's/\.[jew]ar$/p
t
s/\.esb$//p'

We Keep Coding

c++ django amazon-web-services regex python-2.7 google-cloud-platform list unit-testing opengl ember.js

Rewrite URL using sed while maintaining filename - regex

You can use: sed 's~^./\([^/]\{1,\}\)$~/images/\1~' file /images/Security_Panda.png Testing: s='http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png' sed 's~^./\([^/]\{1,\}\)$~/images/\1~' <<< "$s" /images/Security_Panda.png

Easier way if you change your idea. #!/usr/bin/env bash URL="http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png" echo "/image/${URL##*/}"

Another way command line sed 's#^http:./\(.\).$#/images/\1#g' Example echo "http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png "|sed 's#^http:./\(.\).$#/images/\1#g' results /images/Security_Panda.png

An awk version: awk -F\/ '/(jpg|gif|png) *$/ {print "/images/"$NF}' file /images/Security_Panda.png

Related

Swap columns in bash using SED without using loop

BASH: replacing PERL with SED for in-place substitution

How to cut a string from a string

Sed replace domain in URL

Regex with sed to parse archive name

Categories

Resources

We Keep Coding

c++ django amazon-web-services regex python-2.7 google-cloud-platform list unit-testing opengl ember.js

Rewrite URL using sed while maintaining filename - regex

You can use: sed 's~^.*/\([^/]\{1,\}\)$~/images/\1~' file /images/Security_Panda.png Testing: s='http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png' sed 's~^.*/\([^/]\{1,\}\)$~/images/\1~' <<< "$s" /images/Security_Panda.png

Easier way if you change your idea. #!/usr/bin/env bash URL="http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png" echo "/image/${URL##*/}"

Another way command line sed 's#^http:.*/\(.*\).$#/images/\1#g' Example echo "http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png "|sed 's#^http:.*/\(.*\).$#/images/\1#g' results /images/Security_Panda.png

An awk version: awk -F\/ '/(jpg|gif|png) *$/ {print "/images/"$NF}' file /images/Security_Panda.png

Related

Swap columns in bash using SED without using loop

BASH: replacing PERL with SED for in-place substitution

How to cut a string from a string

Sed replace domain in URL

Regex with sed to parse archive name

Categories

Resources

You can use: sed 's~^./\([^/]\{1,\}\)$~/images/\1~' file /images/Security_Panda.png Testing: s='http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png' sed 's~^./\([^/]\{1,\}\)$~/images/\1~' <<< "$s" /images/Security_Panda.png

Another way command line sed 's#^http:./\(.\).$#/images/\1#g' Example echo "http://www.domain.com/wp-content/uploads/2013/03/Security_Panda.png "|sed 's#^http:./\(.\).$#/images/\1#g' results /images/Security_Panda.png