There are some bots that just keep hitting my site over and over again.
http://proxy.parser.by/proxy.php (Referrer)
I don't want to even reply to anyone that is requesting a .php, or .htm, or .html file.
What is the best way of not responding to such requests?
Update: (I don't want to incur the cost of responding)
This is probably best done at the server level, before the request even gets to Django. For example, in Apache, you can use mod_rewrite for access control. This rule rejects all requests with paths ending with .php, .htm, or .html:
RewriteRule \.(php|html?)$ - [F]
The Apache documentation explains how to block requests by user agent, by referer, by orginating IP address, and so on.
Drop in some middleware which will detect any request you don't want to handle and have it return an HTTP 403 (Forbidden).
Related
I am trying to make use of functional url in case of mono lambda function, I have created a functional url with no security.
URL was created successfully, but Not able to hit that url using postman. So I use chrome web browser to hit my url(Get request). But the problem was whenever I hit the url, My function gets executed twice.
If anyone have faced same issue, Please assist.
There are two possibilities I can think off-
Chrome/browser sending another request for favicon.png
If you have configuration on server side that enforce HTTP to HTTPS conversion of the request, like re-direct to enforce SSL connection. In that case as well, browser send one request HTTP and redirect request to HTTPS. e.g when you hit- http://example.org, if it enforce the https, then again browser send another request to https://example.org.
You need to check possibilities here using network trouble shooting. Hope this will help you!
Can anyone help me correct my regex?
My hosting provides IP geolocation and I want to redirect US clients to another sub-directory of my website.
The website runs on a Nginx server and the hosting provides an interface where one can add redirection rules. The interface consists of Domain, Redirect from, Redirect To, Traffic from (country) and HTTP status code 301 or 302.
Example:
for all non-US clients website.com/blog/article/really-good-book
only for US clients website.com/us/blog/article/really-good-book
I currently have:
Redirect from ^/(?!us/)(.*)$
Redirect to /us/$1
This currently redirects me to website.com/us/index.php and nothing else. So the redirect is applied, it only appends index.php instead of blog/article/really-good-book.
Thank you for your help.
Eventually I found my own solution:
Redirect from ^(?!(/us|/index.php|/wp-admin|/wp-login.php))(.*)$
Redirect to /us/$1
I also added a rule for my wordpress backend, otherwise it would always redirect me back to website.com/us while trying to access website.com/us/wp-admin.
I am trying to switch our site from HTTP to HTTPS. In some scenarios, we need the site to use HTTP and at other times, HTTPS. I inted to use CGI to determine whether the request is HTTP or HTTPS.
As far as I can tell, the JSON requests must match the original protocol request. If you request, HTTP:// example.org you must call JSON with HTTP:// example.org /file.JSON. If you request, HTTPS:// example.org/ you must call JSON with HTTPS:// example.org/file.JSON.
Normally, I would use CGI variables to tell me whether the request is HTTP or HTTPS. I can test for CGI.HTTPS to see if it is on or off. I can check CGI.SERVER_PORT too see if it is 80 or 443. I can check CGI.SERVER_PORT_SECURE to see if it is 0 or 1.
When I view our web site in every browser, I can dump the CGI variables and get what I expect 100% of the time.
When a few other people in our office and outside our office make the same request, they get CGI variable values that suggest their request is NOT secure. CGI.HTTPS will show off. CGI.SERVER_PORT will show 40. CGI.SERVER_PORT_SECURE will show 0. Every other indicator will show that the site is secure in every browser, but the CGI variable values say it's not secure.
The site behaves flawlessly 100% for everyone for dev and stage. Only in live, which is behind a load balancer, does this issue exist (for some people).
Is this a load balancer issue? Is this certificate settings issue? Why are my CGI variables lying to me? How can I work around this issue?
With HA Proxy 1.5 I need to rewrite URL from
http://main.domain.com/my-foo
to
http://othersite.com:8081/other-bar
Here is what I tried:
frontend ft_def
bind :80
mode http
acl has_special_uri path_beg /my-foo
use_backend def if has_special_uri
default_backend def
backend def
mode http
option forwardfor
reqirep ^([^\ ]*\ )/my-foo(.*) \1/other-bar\2
server myserver othersite.com:8081
This works:
URL
http://main.domain.com/my-foo/home.html
becomes
http://othersite.com:8081/other-bar/home.html
and in the browser the initial URL http://main.domain.com/my-foo/home.html appears.
It is exactly what I need: it is completely transparent for the user.
But redirect does not work: when I click on a link on the page the URL is then
http://main.domain.com/other-bar/page2.html
I would like to get http://main.domain.com/my-foo/page2.html instead appearing in the browser.
Is it possible with HA Proxy? I tried many configurations without success.
Thanks!
If you're talking about links in HTML (as opposed to, say, Location: headers for redirects)... HAProxy 1.5 won't be able to modify those.
Presumably, based on what you describe, the page /other-bar/page1.html is internally linking to <a href="/other-bar/page2.html"> when it really should link to <a href="page2.html">. You'd need relative links in order for something like this to work transparently... otherwise, component "X" in your chain will have to be able to modify links on the fly in the response body, but only links, of course, since you wouldn't want to blindly regex-replace the page contents as a whole... and HAProxy 1.5 doesn't munge response bodies, so it can't fulfill the role of component "X."
HAProxy 1.6 might be able to do this, with Lua, but that's a maybe... and if it can be made to do it, it isn't likely to be at the level of performance you'd typically expect from HAProxy, because scrubbing the html in Lua is probably going to be a relatively expensive proposition.
Normally any URL changes should be handled by the web app itself, e.g. there is a :path option under :url configuration in Phoenix.Endpoint for the Phoenix framework: https://hexdocs.pm/phoenix/Phoenix.Endpoint.html, which was specifically designed for this purpose. Then the browser will still request the URL appears in the HTML file but HAProxy will always rewrite it.
Hi I have a dynamic application that has a search form. I'm trying to use CloudFront with a load balancer. When you do a search the application creates some URL parameters and reloads the page. For some reason cloudfront is doing a 302 redirect back to the origin. Does anybody know how to fix this?
I have the following configuration.
It's clear that you're going to want to turn on Forward Query Strings. Without it, cloudfront will treat the following urls as the same:
http://www.example.com
http://www.example.com?q=search_term
http://www.example.com?q=search_term&option=true
It's however not clear to me that this would result in 302 to the original url (btw. origin means something very specific when dealing with cloudfront/other cdns - it's often short for origin server which is the source behind the cdn)
The 302 redirect is much more likely to your Viewer Protocol Policy, which you have set http requests to redirect to http. You should double check that your application isn't reloading the http version of the page.