I have a bunch of crawl errors on my site to pages that don't exist (and never existed). It was created from some bad code that generated JSON-LD schema.org for a search page. Basically, I have thousands of 404s that look like this
http://www.domain.com/search/%7Bsearch_term%7D%2Fpage%2F2%2Fpage%2F3%2Fpage%2F6%2Fpage%2F2%2Fpage%2F3%2Fpage%2F6%2Fpage%2F6%2Fpage%2F2%2Fpage%2F2%2Fpage%2F3%2Fpage%2F2%2Fpage%2F6%2Fpage%2F3%2Fpage%2F2%2Fpage%2F6%2Fpage%2F3%2Fpage%2F2%2Fpage%2F7%2Fpage%2F2%2Fpage%2F2%2Fpage%2F8%2Fpage%2F3
http://www.domain.com/search/%7Bsearch_term%7D%2Fpage%2F2%2Fpage%2F6%2Fpage%2F2%2Fpage%2F3%2Fpage%2F6%2Fpage%2F2%2Fpage%2F6%2Fpage%2F6%2Fpage%2F6%2Fpage%2F2%2Fpage%2F3%2Fpage%2F3%2Fpage%2F3%2Fpage%2F2%2Fpage%2F2%2Fpage%2F3%2Fpage%2F7%2Fpage%2F2%2Fpage%2F7%2Fpage%2F2%2Fpage%2F8%2Fpage%2F3
I am terrible with regex, and could use some help on figuring out how to resolve this. As a short term solution, I just want to redirect URL requests with /search/{search_term}/ in the URL to just the /search page.
Any tips on what I should be doing? This is what I have been messing around with which is obviously wrong. Sorry if this is terribly easy question, but I've been trying different things I find online, and have just beating my head without success.
RewriteRule ^/search/%7Bsearch_term%7D$ /search [R,L=301]
RewriteRule ^/search/\{search_term\}$ /search [R,L=301]
You're pretty close. You can use this rule:
RewriteRule ^search/\{search_term\} /search [R,L=302,NC]
Or if search_term is also a dynamic string then use:
RewriteRule ^search/\{[^}]+\} /search [R,L=302,NC]
EDIT: You will need this directive in your Apache or vhost config:
AllowEncodedSlashes On
otherwise Apache rejects requests with these special characters without giving you any chance to handle them in mod_rewrite.
Just off the top of my head, this should do it:
RewriteRule ^search\/(.+) http://www.yourdomain.com/search [R=302]
Explanation of the syntax above:
^ indicates the start of the match
\ is the escape character, so \/ means escape the forward slash (probably unnecessary, but does no harm)
() is a capture group
. means any character
+ means one or more of
So the whole regex means:
starting at the current position in the folder hierarchy, match
search/
followed by one or more characters.
N.B. Importantly, it is the R flag which indicates the type of redirect, so you need R=302, (not L=302, which doesn't exist)
Related
I'm not used to regex and figure I've lost too many hours trying to resolve this, so thought I'd ask for help. I am trying to prettify the html extension.
My site will use URLs that have variable parameters. For example:
mysite.com/article/this-is-an-entry
mysite.com/article/this-is-an-entirely-different-entry
All will use .html as the extension.
In the htaccess file, I have tried
RewriteRule ^(article\/[a-z].*)$ $1.html [NC,L]
as well as slight variations of this, but cannot get this right. Thanks in advance for any assistance.
Firstly, let's look at the regex you have:
^(article/[a-z].*)$
This matches exactly the string "article/", followed by at least one letter (case insensitive due to the NC flag), followed by zero or more of anything. It's quite broad, but should match the examples you gave.
One way to test that it's matching is to add the R=temp flag to the rule, which tells Apache to redirect the browser to the new URL (I recommend using "=temp" to stop the browser caching the redirect and making later testing harder). You can then observe (e.g. in your browser's F12 debug console) the original request and the redirected one.
RewriteRule ^(article/[a-z].*)$ $1.html [NC,L,R=temp]
However, as CBroe points out, your rule will match again on the target URL, so you need to prevent that. A simple way would be to use the END flag instead of L:
Using the [END] flag terminates not only the current round of rewrite processing (like [L]) but also prevents any subsequent rewrite processing from occurring in per-directory (htaccess) context.
So:
RewriteRule ^(article/[a-z].*)$ $1.html [NC,END]
Alternatively, you can make your pattern stricter, such as changing the . ("anything") to [^.] ("anything other than a dot"):
^(article/[a-z][^.]*)$
To be even more specific, you can add a RewriteCond with an extra pattern to not apply the rule to, such as "anything ending .html".
While I'm good with many things scripting and programming, the one thing I never really got into is regex.
I'm tweaking a third party script that uses the url structure of site.com/username/p/seo-friendly-link. I want it to be site.com/username/seo-friendly-link instead
its .htaccess file has
RewriteRule ^([A-Za-z0-9_-]+)/p/(.*)$ ./dir/pages/post.php?user=$1&pid=$2
RewriteRule ^p/(.*)$ ./dir/pages/post.php?pid=$1
when I remove /p as
RewriteRule ^([A-Za-z0-9_-]+)/(.*)$ ./dir/pages/post.php?user=$1&pid=$2
I see it the error in the apache logs as
[error] [client ::1] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
I've tried adding the L to the end of the unmodified rule and it still gives the same error.
I want to learn regex, but I feel like this particular example isn't that noob friendly and it'd instead led to more confusion.
can someone explain what those two lines do, how to get what I need or why it's not as simple as it seems?
Not an expert in .htaccess configuration rules, but as far the regex is concerned the problem seems to be that ^([A-Za-z0-9_-]+)/(.*)$ covers too many possible matches because of the last (.*) group which literally matches anything.
In other words, ^([A-Za-z0-9_-]+)/(.*)$ matches both username/p/seo-friendly-link and username/seo-friendly-link. It will also match the target part dir/pages/post.php?user=$1&pid=$2, probably leading to the infinite redirects.
To restrict the regex, you should exclude the / from the second group. So instead of (.*), use [^/]* where [^/] defines a character class that matches any character except /:
RewriteRule ^([A-Za-z0-9_-]+)/([^/]*)$ ./dir/pages/post.php?user=$1&pid=$2
Change the first line to
RewriteRule ^([A-Za-z0-9_-]+)/p/(.*)$ ./dir/pages/post.php?user=$1&pid=$2 [L,NC,R]
This makes the url redirect and asserts it is the last rule to be checked.
We have changed e-commerce vendors and I need to perserve some of the SEO we've done.
I want to do a 301 for a set of pages to a single new URL
I have a set of pages that end all with the same tickets.htm. So, for example, I have a pages like /blank_tickets.htm and /concert_tickets.htm and the list goes on.
So I tried this:
RewriteRule ^/(.*)tickets.htm$ /t/tickets/types/standard
I tried variations of this no leading / no $, etc.
I'm sure I'm missing something simple but my Google-fu is not returning a relevant example.
Thanks!
You didn't escape the dot. Use \. in your regular expression at the .htm part.
Edit: also use the [R=301] flag, or simply [R] to produce a HTTP 301 header.
I am working on some SEO for my site, and using urls like category/this-cat.html and category/this-cat-p2.html to redirect to index.php?mode=viewCat&id=this-cat and index.php?mode=viewCat&id=this-cat&start=10 respectively.
The problem is that my RewriteRule needs a conditional to check if the -p2 part is present in the URL, and then return either 0 or the digit after p.
Current rule:
RewriteRule ^category/(.*?)\.html$ index.php?mode=viewCat&title=$1
I would have thought that the correct syntax for this would have been:
RewriteRule ^category/(.*?)(?(-p)(.*?)|0)\.html$ index.php?mode=viewCat&title=$1&start=$2
however, this causes the server to return a 500 error.
Even after having read tutorials and worked with them for the past week, I still have little grasp on them. Can anyone explain how to make a conditional like this work?
You're close. I believe what you are trying to do is not capture the optional -p# grouping, but want the digit if it exists. The non-capture flag for a group is a ?: prefix.
RewriteRule ^category/(.*?)(?:-p(\d+))?\.html$ index.php?mode=viewCat&title=$1&start=$2
Note: I used \d (digit) as it's better to be specific about what you are matching. Also start will have a digit or nothing. Your server-side code is better suited to handle the rest of the logic you described.
I'm revealing my embarrassing ignorance of REGEX-fu here, but: I currently have a website where a load of the articles' URLs are written as "article_name", whilst the newer ones are written as "article-name".
I want to move all of them to using dashes, so is there a regular expression I could use to rewrite the older URLs to their newer equivalents?
Thanking you in advance!
First you must achieve consistency in the existing URLs. Basically, you have to normalize all existing names to always use dashes. Ok, you've done that.
We're starting with the following assumption:
The URL is roughly of the form:
http://example.com/articles/what-ever/really-doesnt_matter/faulty_article_name
where only URLs under /articles should be rewritten, and only the /faulty_article_name part needs to be sanitized.
Greatly updated, with something that actually works
For Apache:
RewriteEngine On
RewriteRule ^(/?articles/.*/[^/]*?)_([^/]*?_[^/]*)$ $1-$2 [N]
RewriteRule ^(/?articles/.*/[^/]*?)_([^/_]*)$ $1-$2 [R=301]
That's generally inspired by GApple's answer.
The first /? ensures that this code will run on both vhost confs and .htaccess files. The latter does not expect a leading slash.
I then add the articles/ part to ensure that the rules only apply for URLs within /articles.
Then, while we have at least two underscores in the URL, we keep looping through the rules. When we end up with only one remaining underscore, the second rule kicks in, replaces it with a dash, and does a permanent redirect.
Phew.
Try this:
RewriteRule ^([^_]*)_([^_]*_.*) $1-$2 [N]
RewriteRule ^([^_]*)_([^_]*)$ /$1-$2 [L,R=301]
The first rule replaces one underscore at a time until there are one or less left. The last rule will then replace the last underscrore and do an external redirect.
A potential different approach to think about:
I'm assuming that your "old format" and your "new format" will be in different directories for this idea, if they aren't you might want to consider making the new format have a different directory name.
For instance:
http://site.com/articles/2008/12/31/new_years_celebration
http://site.com/article/2008/12/31/new-years-celebration
In which case you could use mod_rewrite to detect anything in the "old directory" and redirect it to a "redirector.php".
Although on second thought, your mod_rewrite could look for something like this:
RedirectRule /articles/(.*_.*) /redirector.php?article=$1
Matching anything with a _ and sending it through the redirector.
Inside of redirector.php you can get the $_SERVER['REQUEST_URI'] and use tools like preg_replace and even database queries to find the correct url to redirect them to - as well as study the number of hits to old urls.
How will mod rewrite know what the actual url is supposed to be? You can rewrite all articles to use the underscore or the dash, but there is no way for mod_rewrite to tell if new location exists.
For example,
/I_Like_Bees is stored as /path/i_like_bees
/I-like-flowers is stored as /path/i-like-flowers
You want i-like-bees to rewrite to i_like_bees.
If you rewrite underscores to dashes, i_like_bees wouldn't be found
if you rewrite dashes to underscores i-like-flowers wouldn't be found
If you stored all your articles consistently you could easily make a rewrite rule work. Instead you probably have to write a script to check the directories existence and do a 301 redirect to the correct place.
Here's a method: http://yoast.com/apache-rewrite-dash-underscore/
Basically it separates the url into tokens on either side of the underscore, and rewrites the tokens again with the underscore replaced. The problem is it only replaces a single underscore at a time; it will redirect to a closer but not quite correct url, which will again redirect to a even closer, but possibly still not correct url...
It suggests fixing the multiple redirects by having several rewrite conditions & rules with successively more underscores and tokens, but this would require as many conditions and rules as you have underscores in your longest title.
Make sure to add any qualifiers if you can however, as the rule may replace paths you don't want changed (eg., image files) as is.