So I have built a site on my workplace network using django and deploying it with wamp. I don't have a direct connection from my workplace to the internet so I can't show any actual code (not without taking some time to print and rewrite everything).
Now ever since I deployed the site I have been facing one of the worst and weirdes bugs I have ever encountered. It happens only on IE and not even all the time.
what happens is I created a modelform (I added a few lines to make it nicer, but nothing complicated that I couldn't handle on the validation part), and just put it in a template as is. I have added a csrf token and everything seems to be in order.
That is until you click on sending the data. That's when he sends - for some reason, a completely empty request. I don't mean he sends an empty form - he sends a request without any data at all. First I thought it had something to do with the jquery UI I have (just for adding dates), or the csrf going weird but ommiting them didn't change anything (as in, when you omit the csrf token and middlewere it acts as if you sent an empty form, but when it's enabled it acts as if the csrf caught an exeption - because an empty request means no value for the csrf hidden field).
I actually used wireshark and caught the error and ... that's it. I'm no closer to solving it. I tried everything, from changing the page to quirks mode, to rewriting all the html to writing several different template versions just to see what happens.
And It's driving me insane becuase this doesn't even happen all the time. I know I mentioned that already but I just can't make any sense of it. And before you ask - no, it's not an option to use a different webbrowser because all the web-apps my company uses are fit to work with IE and most people here are not technological enough to use multiple browsers.
one more important thing that might help - the bug almost always happens the first time you visit the page (or when you open it in a new tab). If, say, you already went back and forth a few times in the same tab, it usually works fine. we're using IE8, on windows xp computers. Any help would be extremely appreciated, thanks!
Related
Right now, after a user makes a purchase, I'm sending them to their order page where I include some code for conversion tracking (G Analytics, Adwords, Facebook) -- trouble is, especially with Google, if the user reloads the page the conversion is tracked twice (or more).
I've been trying to think of a way to prevent this -- setting up a temporary receipt page for example (that way if the user revisits their order page they're not firing the conversion tags over and over), but I'm not quite sure how to do this -- still, even if it's reloaded, it'll still mess up the conversions.
Any thoughts on how to accomplish this? I thought about using a hidden field to detect a reload, but still - not quite sure how to accomplish that.
So this is what I did. It's not fullproof though. It'll only create the conversion code if the page is loaded within a certain time of the order.
<% if Time.now < #order.created_at + 5.seconds %>
I am blocking a huge number of bots, except the ones from search engines, and then only allowing 2seconds of session management.
However, spam bots are still able to by-pass these measure and create a huge number of requests which is 'killing' the server.
I have read other articles on this site but none seem to directly answer this issue.
A bot probably behaves faster than a human. You could time how long it takes them to fill out the form. Anything less than a second or two is a bot.
A bot probably doesn't have JavaScript turned on. You could use that to your advantage.
You could hide a link via css (or not give it any text) that takes the bot to a bot.cfm page, which could then set a session value.
There are some open source projects but I can't remember the names of them off the top of my head.
CF10 has a new validation function.
Ben Nadel has written some useful posts in his blog regarding spiders/bots.
http://www.bennadel.com/blog/1083-ColdFusion-Session-Management-And-Spiders-Bots.htm
http://www.bennadel.com/blog/154-ColdFusion-Session-Management-Revisited-User-vs-Spider-III.htm
For forms, I use <cfimage> to create a captcha image. I have found that stuffing the captcha phrase in a session variable can cause problems (I can't remember what the problems were though). So, I now use <cfencrypt> to include an encrypted phrase in the form itself. The action page decrypts the phrase and compares it to what the user put in the captcha form field.
I've found CFSPAMProtect to be very useful at blocking automated form fillers.
It bases its SPAM/HAM test on an aggregate score of a number metrics including time on page, mouse movement (via JS) as well as the classic hidden form fields that shouldn't be filled in (but are filled in by dumb robots).
You can assign your own weightings and monitor the SPAM catch via email to allow you to tailor things.
It can work on its own or link to some third party SPAM tools such as Akismet.
So far I've found that it's good enough on it's own.
It's a custom tag and easy to implement in existing forms too which is nice.
Give it a go...
A coworker of mine was working on preventing duplicate form submissions (on a successful submission, we are emailed and a DB entry is created). Submissions are not successful if there are errors on the form.
I was reviewing his code, and I noted that he only used P/R/G on a successful submission. The form submission would still be duplicated on a refresh/revisit if it were not successful. I noted this in my review, but he argued that he didn't see a benefit to redirect every time because the unsuccessful submission is ignored. I noted that if a user refreshes the page or revisits it in history, they will still get a "CONFIRM FORM RESUBMISSION" warning in major browsers.
Is this enough of a reason to redirect after each POST? Whether or not it is, is there any other reason to redirect after a POST even if the submission is idempotent (due to errors in the form)? Is there ever a reason that you should not redirect after a form submission?
The main reason is due to every modern browser's obnoxious dialog box that opens when you hit the "Back" or "Reload" button (and people do - often, whether you want them to or not) after a POST operation, warning you that a POST re-submit is about to take place. I certainly understand why they chose to do this, but it does mean that as programmers we go to great lengths to ensure that a user doesn't have to ever see the message.
So, I disagree with those who say there is no reason to redirect after an unsuccessful post. Theoretically there shouldn't be one, yes, but due to user interface issues with browsers, yes, absolutely you need to.
There is no reason to redirect after an unsuccessful POST. Repeating it won't do any harm and it makes your life easier since you do not need to store all form values in the session for re-filling the form.
Besides that, during development it makes thing much easier especially when using a file upload field since you can just make the form submission always fail and then hit F5 until all your validation code etc works fine.
I have implemented a custom middleware that check for certain fields in a user's profile. The fields are not required at sign up (in order to make it quick and easy for a user to sign up), however, I would prefer that they fill them out.
My middleware checks if they are set. If not, it notifies the user via a message. I ran into two problems.
Whenever I submit a posted form, because no template displays the messages, the middleware would add the message a second time since the middleware was called when the message was posted and after the redirect it was called again.
I solved this problem by iterating through the messages in my middleware and checking if the message I am about to add is already in there. If yes, it doesn't add it again.
When a user fixes the problem by updating their profile, on the very next page load, the messages are still there. After that though, everything works. At the beginning of my middleware, I actually put a check that returns None if the request was posted (I would have thought this would solve both problems, but it didn't solve either).
Any idea how to solve the 2nd issue? Is there a better way to solve my first one?
Thanks.
Edit:
Is there a way to clear the messages in a view? I've tried iterating through them (without storage.used=False) and they are still there. I would expect that this would solve both my problems.
Yoy can use sticky messages of https://github.com/AliLozano/django-messages-extends that is one storage that only keeps messages in request instead save in session
So, are you using django.contrib.messages to store permanent notices? It's intended for showing one-time notifications, where the user sees the message once and then it goes away. The type of things it's meant for are messages like, "Form edited successfully."
As far as getting rid of messages in a user's message stack are concerned: any time you use a RequestContext (discussed here) to render a template, all messages will be flushed (whether they are actually shown on the page or not).
I'm not totally sure this is the answer you're actually after, and I'm a bit confused by your question. However, I'm somewhat sure you're using messages outside its intended purpose, which is likely why you're running into trouble.
You could use process_reponse() method of Middleware to add the message. By that time, you would know whether to show the message to user or not depending on whether his profile now has the field filled in.
Consider using django.contrib.messages. May be you do not want to show "fill in the XYZ field" message on all requests, but only on few pages, say whenever user logs in or views his profile page.
Just place logic that creates messages to context processor instead of middleware. Modify request and return empty dict.
I'm building a website using Ubuntu, Apache, and Django. I'd like to block people from filling out and submitting a particular form on my site more than once. I know it's pretty much impossible to block a determined user from changing his IP address, deleting his cookies, and so on; all I'm looking for is something that will deter the casual user from re-submitting.
It seems to me that blocking multiple form submissions from the same IP address is the best way to achieve what I'm looking for. However, I'm unsure how I should do this, and whether I should be doing this from Apache or from Django. Any tips?
Edit: I'm looking to prevent intentional re-submission, not just unintentional double submission. e.g. I have a survey that I want to discourage people from voting multiple times on.
If your main concern is to prevent someone writes a script and automatically submit the form many times, you may want to use CAPTCHA with your form.
Several whole countries are NAT'ed, and some (most?) large multinational corporations too, many with several hundred thousand users each. Blocking anything by IP is a bad idea.
Go for a cookie instead, which is as good as it's going to get. You could also make the user login in, in which case you'd know if the form was submitted repeatedly for that login.
I would use the session id, and store form submissions in a table with session id, timestamp, and optionally some sort of form identifier. Then, when a form is submitted, you could check the table to make sure that it had not happened within a certain period of time.
Filtering on IP address and/or cookies are both easy to get around, but they will prevent the casual user from accidentally submitting the same stuff multiple times due to browser hick-ups, impatience and so on.
If you want something better than that you could implement login, but of course that prevents a lot of users from responding.
Add to the form a monotonically increasing id number in a hidden field.
As each form is submitted, record the id in a "used" list/map (or mark it used, or whatever, implementation detail).
If you get the same id a second time (if it's already in your used map) inform the user they double-submitted.
While nothing is fool proof, I would suggest something like this: When a user loads the page with your form on, a cookie is set and the value of the cookie is appended with a fixed secret string and the md5 value of this is written to a hidden field on the form. Ensure that a new value is generated each time the user access the form.
When the user submits the form, you check that the cookie value and form value match, that the cookie the user was given has not been used to submit the form before and that the referrer id match the URL of the form. Optionally you make sure that there has been no attempts to post from that IP in the last 2 minutes (fast enough that it wont matter to most people, but slow enough to slow down bots).
To fix this the user has to make a script that loads the page, store the cookies and submit the correct values. This is much more difficult than if the user could just submit the form.
Added Based on edit: I would block the users in the Django framework. This allows you to present a much better error message to the user and you only block them from that form.
This is a question of authentication and authorisation, which are related but not the same. In order to manage authorisation you must first authenticate (reliably identify) the user.
If you want to make this resist intentional misuse then you are going to end up with not only usernames and passwords but demands for information that personally identifies your users, along the lines of the stuff a bank asks for when you want to open an account. The bleeding hearts and lefties will snivel endlessly about invasion of privacy but in fact you are doing exactly the same as a bank and for exactly the same reasons.
It's a lot of work and may be affected by law. Do you really want to do it?
The following methods are all relatively simple, both to implement and to hack around. Anyone with Firebug and a little knowledge won't even blink.
The following JavaScript uses Mootools, and I haven't checked it to be bug free. I understand that JQ syntax is almost identical, and raw JS is similar enough, so the point should be clear.
1) If the form is being submitted via AJAX, you can check before submitting (sorry if I'm just stating the obvious).
var sent = 0;
$('myForm').addEvent('submit', function(){
if(!sent) this.send();
})
This is really simple, and surprisingly effective until they reload the page.
2) Add a JavaScript cookie. Again, with Mootools:
$('myForm').addEvent('submit', function(){
if(Cookie.read('submitted')){ alert('once only'); return false;}
else{ Cookie.write('submitted', 1); return true; }
})
This will work even if the user reloads the page.
3) Add a Python session cookie. I am not familiar with Python, but if it is like PHP, this will have no advantage over method 2. In either case, the user can delete the cookie with FireCookie or WebDeveloper Toolbar (or their equiv's on other browsers) and reload the page.
4) Add a Flash cookie (use Flex). This is ideal - Flash cookies are stored in a different location, are not obvious, and are very difficult to remove. The only downside is that you need to create and embed a tiny swf.
5) Store a value in a hidden field, and check for the value.
A hash can be added to the internal links to insure that the value remains filled in even if the page is navigated away from.
6) Other games can be played incrementing a URL (or a custom URL using htaccess) for each visitor.
An swf cookie is the best idea of the above, though it can be combined with the others.