I have a website that I built using Django. Using the settings.py file, I send myself error messages that are generated from the site, partly so that I can see if I made any errors.
From time to time I get rather strange errors, and they seem to mostly be around about the same area of the site (where I wrote a little tutorial trying to explain how I set up a Django Blog Engine).
The errors I'm getting all appear like something I could have done in a typo.
For example, these two errors are very close together. I never had an 'x' or 'post' as a variable on those pages.
'/blog_engine/page/step-10-sub-templates/{{+x.get_absolute_url+}}/'
'/blog_engine/page/step-10-sub-templates/{{+post.get_absolute_url+}}/'
The user agent is:
'HTTP_USER_AGENT': 'Mozilla/5.0 (compatible; Purebot/1.1; +http://www.puritysearch.net/)',
Which I take it is a scraper bot, but I can't figure out what they would be able to get with this kind of attack.
At the risk of sounding stupid, what should I do? Is it a hack attempt or are they simply trying to copy my site?
Edit: I'll follow the advice already given, but I'm really curios as to why someone would run a script like this. Are they just trying to copy. It isn't hitting admin pages or even any of the forms. It would seem like harmless (aside from potential plagiarism) attempts to dig in and find content?
From your USER_AGENT info it looks like this is a web spider from puritysearch.net.
I suggest you do is put a CAPTCHA code in you website. Program it to trigger when something tries to access 10 pages in 10 seconds (mostly no humans would do this or figure out a proper criteria to trigger your CAPTCHA).
Also, maintain robots.txt file which most crawlers honor. Mention your rules in robots.txt. You can say the crawlers to keep off certain busy sections of your site etc.
If the problem persists, you might want to contact that particular site's system admin & try to figure out what's going on.
This way you will not be completely blocking crawlers (which are needed for your website to become popular) and at the same time you are making sure that your users get fast experience on your site.
Project HoneyPot has this bot listed as a malicious one http://www.projecthoneypot.org/ip_174.133.177.66 (check the comments there) and what you should probably do is ban that IP and/or Agent.
Related
Apologies if this question is not asked correctly but I actually can't pinpoint the problem. Hence the lack of code in the question itself. I'm struggling with this error so please go easy on me....
I was developing a meteor app locally without too much difficulty. It seems to run well on localhost. But then I decided to deploy it and have been running into some problems as outlined below. I have deployed the app "successfully" using both mup/DO and modulus.io (with compose hosting the DB in both instances) and whole thing sort of works, but...
You can register fine but when you try to log in the button doesn't work...press it again and you login but the usename and password are in the URL... how does this happen?
When you login you can begin by creating a supplier, then create products for that supplier. Only thing is the suppliers are saved to the DB and they are in the product count but they are nowhere to be seen in the middle section page.
Once you create a new supplier, the add new products for the existing suppliers is no longer accessible.
I am so confounded that I don't know what code to put up so I'm giving access to my codebase - it's on github here and this is the modulus site here. Go ahead and register and you can see for yourself.
You will easily be able to see the errors in the console when you start trying to log into the site so there is no point in posting them here.
Many, many thanks in advance guys.
I have reviewed your code and I don't think that the errors occur due to the deployment.
Here's a list with a few suggestions that should help you to fix your code:
In your /client/helpers/config.js file, you try to configure the behaviour of {{> loginButtons}}. That does not make any sense, since you do not have the accounts-ui package installed.
The /client/templates/includes/header.html file references with pathFor to homepage. This route is currently not available in your /lib/router.js.
Users are able to access the /loggedIn path even if they are not logged in. Furthermore, you always redirect users to this path if the submit form event in the register template occurs. This means, they can easily bypass the registration just by clicking on the submit button.
Watch your console logs. There are a lot of template helper exceptions.
Unfortunately, I could not check the login bug you described, because I received an exception when invoking the submit event. I recommend to use a rather defensive programming approach, you should at least check if the variable's value is not undefined and if it is, then you should handle those situations accordingly.
For example, in your /client/templates/includes/login.js file, you have the following code:
var userId = Meteor.userId();
var supplier = Suppliers.findOne({userId: userId});
var supplierId = supplier._id;
This will raise an exception if supplier is undefined.
All in all, you should rethink your release planning and deployment, since your app is far from working. Furthermore, please try to break your issue into chunks next time and provide a clear problem statement, because your question won't be useful to other readers without it.
We have been noticing a lot of 404 errors being thrown in our Coldfusion CFIDE server monitor, and it took us a while to find out that things like missing favicons are causing these errors.
We use a custom 404 template page, which contains some logic to it (more than just basic HTML). So, whenever a 404 for a favicon occurs, these pages are generated and returned to the user.
Since these favicons are requested by default on many browsers (if there is no in the header that specifies one, it looks in the site root or something like that), it's throwing massive amounts of 404s on our server, which costs processing time as well as bandwidth for delivery. Our server runs fine most of the time, but when it does get some heavy usage, we can sometimes have major performance problems.
I know that this is a performance issue, but would it be enough of one to warrant trying to fix this? If so, is there a way with the Coldfusion server (or our underlying Windows Server 2003 running IIS) to filter what files actually throw a CF 404 error? Ideally, for files like these favicons, CSS, and Javascript (since a visitor never really "sees" the output of these), we would simply want to return an HTTP 404 response with no content, as it is unneeded...
Yes, missing favicons do affect performance, as does any missing content when you have a custom 404 page (esp. if the 404 page is handled by a content management system). This is because every time the file (image, video, page, etc) is requested by the browser, it causes load on your server. Let's say the favicon is missing on every page of your website. If your 404 page is part of your content management system, this doubles the load on your server (basically requesting 2 pages every time instead of 1). If your 404 page is different, but still has logic, this increases the load, but only as much as the file requires (less logic = less load, and vice-versa).
I would suggest fixing this issue, but not necessarily by killing custom 404 pages for certain extensions. In my opinion, it would be better (for you, and your visitors) if you simply added a favicon file to all of your sites. Not only would this solve your 404 issue, but it would help your visitors to recognize your website quicker when bookmarking pages, or adding the site as an App Tab (which would apply, even if your site isn't available to the public, as backend sites are a great use case for App Tabs). Aside from your server's performance, having to download the 404 page causes for network performance as well, both on your server's end as well as the end-user. The 404 page may also not be cached, and even if it is, probably not for as long as an existing favicon would be, which causes the request to happen far more often than it would if you simply created a favicon.
If you don't want to take the time (or don't have a need) to do advanced branding (such as creating a custom logo for the favicon), a basic image with a letter on it (e.g. "K") will do. Favicons are extremely useful for the public, any staff, and even yourself, so I would say it's definitely worth your time to at least do a basic favicon.
Only you can really judge if its enough of a problem to warrant fixing... how many times per second are the 404's generated, what additional load do they put on the server, etc.
Regarding a fix... why don't you just deploy a favicon? It would probably be quicker than worrying about the problem.
As someone else touched on, I think the easiest and most sensible way of dealing with this problem is not to try to deal with it at the 404-handling side of things, but rather just make sure the 404 doesn't occur in the first place. If this is all happening because of missing favicons... fix it by not having missing favicons! If you don't have the resources or desire to brand one appropriately, just use a generic one. It's better to treat the actual problem than a symptom of the problem.
I'm developing a web application. It's months away from completion but I would like to build a landing page to show to potential customers to explain things and gauge their interest--basically collecting their email address and if they feel like it additional information like names + addresses.
Because I'm already using Django to build my site I thought I might use another Django App to serve as this landing page. The features I need are
to display a fairly static page and potentially a series of pages,
collect emails (and additional customer data)
track their actions--e.g., they got through the first two pages but didnt fill out the final page.
Is there any pre-existing Django app that provides any of these features?
If there is not a Django app, then does anyone know of another, faster/better way than building my own app? Perhaps a pre-existing web service that you can skin and make look like your own? Maybe there's the perfect system but it's PHP?--I'm open for whatever.
Option 1: Google Sites
You can set it up very very quickly. Though your monitoring wouldn't be as detailed as you're asking for.. Still, easy and fasssst!
Option 2: bbclone
Something else that may be helpful is to set up some PHP based site (wordpress or something) and use bbclone for tracking stuff on it. I've found bbclone to be pretty intense with the reporting what everyone does - though it's been a while since I used it.
Option 3: Django Flatpages
The flatpages Django contrib app is pretty handy for making static flat pages. I'd probably just embed a Google Docs Form to collect email addresses (as that's super fast and lets you get back to real work). But this suggestion would still leave you needing to figure out how to get the level of detail you want on the stats end.
Perhaps consider Google Analytics anyway?
Regardless, I suggest you use Google Analytics with everything. That'll work with anything you do really, and for all I know, perhaps you can find a way to get the stats you're really looking for out of it.
A client I'm working for has mysteriously ended up with some malicious scripting going on on their site. I'm a little baffled however because the site is static and not dynamically generated - no PHP, Rails, etc. At the bottom of the page though, somebody opened a new tag and a script. When I opened the file on the webserver and stripped the malicious stuff and re-uploaded, it was still there. How is this possible? And more importantly, how can I combat this?
EDIT:
To make it weirder, I just noticed the script only shows up in the source if the page is accessed directly as 'domain.com/index.html' but not as just 'domain.com'.
EDIT2:
At any rate, I found some php file (x76x09.php) sitting on the web server that must have been updating the html file despite my attempts to strip it of the script. I'm currently in the clear but I do have to do some work to make sure rogue files don't just appear again and cause problems. If anyone has any suggestions on this feel free to leave a comment, otherwise thanks for the help everyone! It was very much appreciated!
No it's not possible unless someone has access to your files. So in your case someone has access to your files.
Edit: It's best if you ask in serverfault.com regarding what to do in case the server is compromised, but:
change your shell passwords
have a look at /var/log/messages for login attempts
finger root
have a look at last modification time of those files
There is also a high propability that the files where altered via http by using a vulnerability of a software component you use together with the static files.
To the point about the site not having pages executing on the server, XSS is absolutely still possible using a DOM based attack. Usually this will relate to JavaScript execution outputting content to the page. Just last week WhiteHat Security had an XSS vulnerability identified on a purely “static” page.
It may well be that the attack vector relates to file level access but I suggest it’s also worthwhile taking a look at what’s going on JS wise.
You should probably talk to your hosting company about this. Also, check that your file permissions aren't more lenient than they should be for your particular environment.
That's happened to me before - this happens if they get your ftp details. So, whoever did it, obviously got ahold of your ftp details somehow.
Best thing to do is change your password and contact your webhosting company to figure out a better solution.
Unfortunately, FTP isn't the most secure...
I've been working through an issue with my django project. The issue is I've got one project, which will retrieve data for users of different clients. I need to know 'from where' a viewer is coming from (request.path [my original solution]). I've been looking at a number of different options which sound close to what I want to do, but I'm not sure what the best option is, not having done this before.
My first option was to add a url in the urls.py with a 'tag' or 'keyword' then look for that tag/keyword in the request.path, which I'd add as a session key. Then go onto get the data.
Something else I started looking at was the sites framework. After reading through the documentation, I'm still confused how sites actually works, so I'm not sure if this is the right option.
Another solution talked about using middleware, this came up in connection with the research into using the sites framework.
And then yet another talked about doing this in apache.
Could some one help point me in the right direction?
Cheers,
T
If you need to know from which URL came your user to your currrent page you should check the REFERER http header, available in request.META.get('HTTP_REFERER').
See http://docs.djangoproject.com/en/1.2/ref/request-response/#ref-request-response for more informations.
Be careful though, the referer meta is not mandatory and could be missing due to private browsing or direct access to the page from the URL bar.
It's not completely clear from your question, but if you're asking for the URL that the user was on before coming to the current page, you probably want request.META['HTTP_REFERRER'].
Edit after comment
That would be a very bad idea. Global variables are not safe given that you potentially have multiple requests being processed at the same time. The referrer is already available from the request, which can be accessed in all views and templates, so I don't know what else a middleware would give you.