Via Apache, is it possible to exclude access to my site if a particular page request www.mysite.com/this_page/ is made? The referrer IP address changes.
I realize that I can "catch" this request in my code and give back an error page, but I'd prefer for the request to never make it to my actual code. I can also just remove the page, but then my site would return its "page not found" page. So, still a page.
My host is Webfaction and I use Django. So it's Apache.
The reason is that I am seeing some weird activity to this particular page. For the curious, I am getting requests for this page about 20 times a day in bursts of 1-3 requests in 2 seconds. The IP addresses are Ukrainian. The referrers are mostly various Ukrainian websites, which seem real enough, but they do not contain an actual link to my webpage.
I don't think blocking a whole country is a good idea, but here's how to do it.
First, get a list of CIDRs of the country in question. For most countries, it will be a large list. One place to get that would be http://www.find-ip-address.org/ip-country/. Then, merely put those into an .htaccess or Apache config file:
<Location /this_page>
order allow, deny
deny from 5.1.0.0/19
deny from 5.22.156.0/22
deny from 5.34.176.0/21
deny from 5.53.112.0/21
deny from 5.56.24.0/21
deny from 5.57.64.0/21
deny from 5.58.0.0/16
deny from 5.83.16.0/21
deny from 5.104.32.0/19
deny from 5.105.0.0/16
#...
</Location>
(Leading whitespace is optional, .htaccess files don't need Location tags if you're happy blocking the whole directory.)
The above config keys on the client IP rather than the IP of the host of the referring link. That part is much harder to do, though perhaps keying on the Ukrainian TLD would be sufficient:
RewriteEngine on
RewriteCond %{HTTP_REFERER} ^https?://[^:/?#]*\.ua[:/?#]
RewriteRule ^/this_page - [F,L]
(Note that %{HTTP_REFERER} has no double Rs. The spec was misspelled, my config above was not.)
If you're getting slammed by high volume, you might want to consider dropping the connections at the firewall level rather than Apache. Again, that would be by client rather than by referrer.
Related
I'm trying to set up Cloudfront->Application Elastic Load Balancer->Auto Scaling->EC2 AWS stack.
Everything works until it scales to more than 1 EC2 instance, which then causes a redirect loop with the error message "Too many redirects".
Here are the related settings:
I've enabled an ACM SSL certificate and attached it to the CloudFront distribution.
DNS pointed to CloudFront domain name.
Cloudfront 'Origin Protocol Policy' = HTTP Only
ELB Listener 1 = HTTP:80 redirects to HTTPS:443
ELB Listener 2 = HTTPS:443 forwards to the target group of 2 EC2 instances
.htaccess
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^app\.php(?:/(.*)|$) %{ENV:BASE}/$1 [R=301,L]
RedirectMatch 302 ^/$ /app.php/
Please help me solve this redirect loop and explain why the current settings are not working.
Any time you spend on this is highly appreciated.
You appear to be using both mod_rewrite and RedirectMatch to perform two different redirects:
This appears to redirect any request starting with app.php to the base website URL:
RewriteRule ^app\.php(?:/(.*)|$) %{ENV:BASE}/$1 [R=301,L]
This appears to be redirecting any request to / coming in to /app.php/:
RedirectMatch 302 ^/$ /app.php/
These rules seem to be in direct conflict with one another. If you try to request either the root website path /, or /app.php you are going to get into a redirect loop.
This condition tells Apache to track redirects internally in order to prevent a redirect loop:
RewriteCond %{ENV:REDIRECT_STATUS} ^$
However that only works as long as you have one server. When you have multiple load-balanced servers they can't track if a redirect has been issued by another server in the pool.
I suggest taking a look at these redirect rules and only using one of them depending on what your specific needs are.
I was able to resolve this, temporarily, with Mark B's answer.
"You didn't include any info about logging in and user sessions in your question. For the short term, I would enable sticky sessions on the load balancer. Long term I would look into a distributed session store." – Mark B
I have a website www.example.com and it is hosted on elastic-beanstalk. I am using the name.com DNS servers. I have followed the steps in the following blogs to set up https and URL settings:
https://colintoh.com/blog/map-custom-domain-to-elastic-beanstalk-application
https://medium.com/#jbesw/tutorial-adding-https-to-a-custom-domain-on-elastic-beanstalk-29a5617b8842
i.e
Create a CNAME pointing www.example.com to the beanstalk
Add a URL redirect for #.example.com to https://www.example.com
After this, the links www.example.com works, and http://example.com gets redirected to www.example.com.
But for a page inside the site, like www.example.com/about, just typing in http://example.com/about does not work and does not get redirected to www.example.com/about.
Most blogs suggest moving to AWS Route 53. Is that the only option?
The issue, as you've found out, is that DNS-level redirects don't work on a page-specific level. At least, not without some extra magic happening in the background (which some registrars implement.)
Even if that setup did work, you'd still have some SEO issues to deal with. For example, you want the example.com > www.example.com redirect to (In any case I know of) to be a 301 redirect. This let's search engines like Google know "Use only the www version of this page please." Otherwise, you effectively have two pages floating around out there either of which (or both) could be indexed and considered duplicate content of one another.
Using the Route 53 servers is certainly an option but no one you have to use. The issue is that you need to do this on a server-level—not a DNS level.
On a server level, you can specify more complex and granular redirection rules such as "send any non-www, non-https traffic to the www, https version of the page and indicate this is a permanent preference (301)` that redirect (on an Apache server) would look like this:
RewriteEngine On
RewriteCond %{HTTPS} !=on [OR]
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^(.*)$ https://www.{HTTP_HOST}%{REQUEST_URI} [R=301,L,NE]
Quick Reference: NC means case-insensitive matching. R specifies the type of redirect (301 here), NE specifies to not escape characters like # or ? which are used in many URL schemes. For a full list of flags used during Apache RewriteRules, read this webpage.
There are different ways to achieve this for Apache, NGINX, and Windows Server. Amazon has a reference article detailing some of the implementation approaches for this. Copying the details of the article here is beyond the scope of your question IMO.
So, to answer your question: Route 53 isn't your only option. You can absolutely use whatever registrar or DNS host you'd like. The issue is that you need to re-think your approach entirely and focus on server-level rules rather than DNS-level rules. I'm no expert and find it annoying to do it this way, so hopefully, someone will jump in with a more insightful approach.
We have an issue where spammers are subscribing to all our feeds.
I already disabled all feeds via a function, as well as using a plugin.
Each day they still appear in our logs with thousands of entries.
For example:
/leaderboard/feed/index.php
/leaderboard/feed/
/some-category/feed/
Is there any way to block access to all visitors querying /feed/ or anything containing the word "feed" in the URL via .htaccess, and deny them access to the site?
Yes rewrite rules can do that:
RewriteEngine On
RewriteRule ^feed/ - [NC,F]
This will give forbidden error for every URI that starts with /feed/
Following the advice on google's pagespeed addon, I moved my static content (images, css, javascript files) to a subdomain of my site on which there should be no cookies, thus saving some space in the request headers for each request for these files. However, on looking at the results I find that cookies are still being created for this sub domain. I have google adsense on my site and it is writing cookies to the root domain *.example.com rather than only to www.example.com
Is there any way to force adsense to write cookies only to the exact domain used for displaying the adds and not to the root domain?
There appears to be no way to force adsense to create cookies for only a specific subdomain. The generally used solution is to simply serve your content from an entirely different domain. SO uses sstatic.net, Yahoo uses yimg.com, Youtube uses ytimg.com.
I'd recommend just getting a new domain name. It's only $10 a year or so (depending on your TLD). The ROI of that is, what, 10 minutes developer time?
There can be a proxy script -- a landing page -- where all the traffic for *.example.com lands and then request is dispatched to the intended subdomain. Alternatively, you can place that script in the each subdomains' root folder to redirect the adsense cookie to wwww.yourdomain.com. In Linux environment, it can be handled using .htaccess file, like:
# place this in your subdomain's root
RewriteEngine on
RewriteCond %{HTTP_COOKIE} ^.*SOME_ID=(\d+).*$ [QSA]
RewriteRule .* ../cache/$1 [QSA,L]
Where SOME_ID is a recurring name in each generated adsense cookie. <= This is to identify/ensure that the cookie under consideration is the adsense cookie.
I'm sure this is possible, but its beyond my meager abilities with .htaccess files.
We have an internal php app that we use, we have basic security internally, but dont need to worry too much. I would like to make it available online for use when staff are out and about. I would like to have additional security based on htaccess or htpassword files.
Is it possible to write a htaccess file that does the following
If user is accessing from office.mydomain.com it means they are internal (office.mydomain.com resolves to an internal ip like 192.168.22.22) so allow unimpeded access
If the user is accessing from outside it will be external.myoffice.com - if this is the case as an added bit of security I would like to use .htaccess and a password file to get the user to enter an apache password.
Can anyone tell me how to write this with .htaccess file?
Update: Thanks for all the answers, I have posted what worked for me as an answer to help others.
You can use
RewriteCond %{REMOTE_ADDR} !^192\.168\.
to specify the condition of an external IP, or use
RewriteCond %{REMOTE_ADDR} ^192\.168\.
for the condition of a local IP.
You will just have to integrate these into your existing htaccess rules in a sensible way.
I think this does do what you want;
http://codesanity.net/2009/11/conditional-htpasswd-multienvironment-setups/
http://tomschlick.com/2009/11/08/conditional-htpasswd-multi-environments/
https://tomschlick.com/2009/11/08/conditional-htpasswd-multi-environments
Correct address for the resource as of 2022/01/15.
https://tomschlick.com/conditional-htpasswd-multi-environments/
Here you go
order deny,allow
allow from 192.168.22.0/255.255.255.0
deny from all
You can use a subnet mask to make sure the visitors are from the same network. If you need to address another network, just use those IP's (as the server sees them)
To Complete this answer the following Works.
#allows everything if its on a certain host
SetEnvIf HOST "^www.mysite.com" external_url
SetEnvIf HOST "^localhost" local_url
Order Deny,Allow
AuthName "Restricted Area"
AuthType Basic
AuthUserFile path/to/your/.htpasswd
AuthGroupFile /
Require valid-user
#Allow valid-user
Deny from all
Allow from env=external_url
Allow from env=local_url
Satisfy any
This pops up a Restricted Area login box if you visit via the www.mysite.com but displays nothing if you are coming locally.