I'm building a website using Ubuntu, Apache, and Django. I'd like to block people from filling out and submitting a particular form on my site more than once. I know it's pretty much impossible to block a determined user from changing his IP address, deleting his cookies, and so on; all I'm looking for is something that will deter the casual user from re-submitting.
It seems to me that blocking multiple form submissions from the same IP address is the best way to achieve what I'm looking for. However, I'm unsure how I should do this, and whether I should be doing this from Apache or from Django. Any tips?
Edit: I'm looking to prevent intentional re-submission, not just unintentional double submission. e.g. I have a survey that I want to discourage people from voting multiple times on.
If your main concern is to prevent someone writes a script and automatically submit the form many times, you may want to use CAPTCHA with your form.
Several whole countries are NAT'ed, and some (most?) large multinational corporations too, many with several hundred thousand users each. Blocking anything by IP is a bad idea.
Go for a cookie instead, which is as good as it's going to get. You could also make the user login in, in which case you'd know if the form was submitted repeatedly for that login.
I would use the session id, and store form submissions in a table with session id, timestamp, and optionally some sort of form identifier. Then, when a form is submitted, you could check the table to make sure that it had not happened within a certain period of time.
Filtering on IP address and/or cookies are both easy to get around, but they will prevent the casual user from accidentally submitting the same stuff multiple times due to browser hick-ups, impatience and so on.
If you want something better than that you could implement login, but of course that prevents a lot of users from responding.
Add to the form a monotonically increasing id number in a hidden field.
As each form is submitted, record the id in a "used" list/map (or mark it used, or whatever, implementation detail).
If you get the same id a second time (if it's already in your used map) inform the user they double-submitted.
While nothing is fool proof, I would suggest something like this: When a user loads the page with your form on, a cookie is set and the value of the cookie is appended with a fixed secret string and the md5 value of this is written to a hidden field on the form. Ensure that a new value is generated each time the user access the form.
When the user submits the form, you check that the cookie value and form value match, that the cookie the user was given has not been used to submit the form before and that the referrer id match the URL of the form. Optionally you make sure that there has been no attempts to post from that IP in the last 2 minutes (fast enough that it wont matter to most people, but slow enough to slow down bots).
To fix this the user has to make a script that loads the page, store the cookies and submit the correct values. This is much more difficult than if the user could just submit the form.
Added Based on edit: I would block the users in the Django framework. This allows you to present a much better error message to the user and you only block them from that form.
This is a question of authentication and authorisation, which are related but not the same. In order to manage authorisation you must first authenticate (reliably identify) the user.
If you want to make this resist intentional misuse then you are going to end up with not only usernames and passwords but demands for information that personally identifies your users, along the lines of the stuff a bank asks for when you want to open an account. The bleeding hearts and lefties will snivel endlessly about invasion of privacy but in fact you are doing exactly the same as a bank and for exactly the same reasons.
It's a lot of work and may be affected by law. Do you really want to do it?
The following methods are all relatively simple, both to implement and to hack around. Anyone with Firebug and a little knowledge won't even blink.
The following JavaScript uses Mootools, and I haven't checked it to be bug free. I understand that JQ syntax is almost identical, and raw JS is similar enough, so the point should be clear.
1) If the form is being submitted via AJAX, you can check before submitting (sorry if I'm just stating the obvious).
var sent = 0;
$('myForm').addEvent('submit', function(){
if(!sent) this.send();
})
This is really simple, and surprisingly effective until they reload the page.
2) Add a JavaScript cookie. Again, with Mootools:
$('myForm').addEvent('submit', function(){
if(Cookie.read('submitted')){ alert('once only'); return false;}
else{ Cookie.write('submitted', 1); return true; }
})
This will work even if the user reloads the page.
3) Add a Python session cookie. I am not familiar with Python, but if it is like PHP, this will have no advantage over method 2. In either case, the user can delete the cookie with FireCookie or WebDeveloper Toolbar (or their equiv's on other browsers) and reload the page.
4) Add a Flash cookie (use Flex). This is ideal - Flash cookies are stored in a different location, are not obvious, and are very difficult to remove. The only downside is that you need to create and embed a tiny swf.
5) Store a value in a hidden field, and check for the value.
A hash can be added to the internal links to insure that the value remains filled in even if the page is navigated away from.
6) Other games can be played incrementing a URL (or a custom URL using htaccess) for each visitor.
An swf cookie is the best idea of the above, though it can be combined with the others.
Related
I want to make sure that my visitors (not authenticated users), are unable to visit a particular view without coming directly from a "previous view". I've kind of had to manually create a form preview and confirmation state. It's the step between submission and preview, and preview and confirm I'd like to "secure".
form submission-view -> preview-view -> confirm-view.
Is there some way that I can create a unique hash, POST it, and check if it's correct, or somewhat generate a cookie, session — or anything else that feels clever?
I'm a Django beginner (programming beginner in general) and any snippets' or pointing me in a right direction would be very much appreciated!
Thanks.
There are at least two ways you can accomplish this that I can think of:
One would be to include a hidden field in your form or querystring value that contains your hash/unique that you want to pick up in the next view. If it's not there, or incorrect, redirect.
Another would be to check the referring url from the request.META to see if they've come from the view you want them to come in on first, and save a session value from the form submission to carry through the rest of the views. If it's not there, redirect. If the referring URL isn't what you expect, redirect.
Whether you use a cookie, session, querystring parameter or hidden form post, it's all doing the same thing - validating a value exists. Whatever method works best, is what makes the most sense for you as the developer and most likely maintainer of said app.
I am blocking a huge number of bots, except the ones from search engines, and then only allowing 2seconds of session management.
However, spam bots are still able to by-pass these measure and create a huge number of requests which is 'killing' the server.
I have read other articles on this site but none seem to directly answer this issue.
A bot probably behaves faster than a human. You could time how long it takes them to fill out the form. Anything less than a second or two is a bot.
A bot probably doesn't have JavaScript turned on. You could use that to your advantage.
You could hide a link via css (or not give it any text) that takes the bot to a bot.cfm page, which could then set a session value.
There are some open source projects but I can't remember the names of them off the top of my head.
CF10 has a new validation function.
Ben Nadel has written some useful posts in his blog regarding spiders/bots.
http://www.bennadel.com/blog/1083-ColdFusion-Session-Management-And-Spiders-Bots.htm
http://www.bennadel.com/blog/154-ColdFusion-Session-Management-Revisited-User-vs-Spider-III.htm
For forms, I use <cfimage> to create a captcha image. I have found that stuffing the captcha phrase in a session variable can cause problems (I can't remember what the problems were though). So, I now use <cfencrypt> to include an encrypted phrase in the form itself. The action page decrypts the phrase and compares it to what the user put in the captcha form field.
I've found CFSPAMProtect to be very useful at blocking automated form fillers.
It bases its SPAM/HAM test on an aggregate score of a number metrics including time on page, mouse movement (via JS) as well as the classic hidden form fields that shouldn't be filled in (but are filled in by dumb robots).
You can assign your own weightings and monitor the SPAM catch via email to allow you to tailor things.
It can work on its own or link to some third party SPAM tools such as Akismet.
So far I've found that it's good enough on it's own.
It's a custom tag and easy to implement in existing forms too which is nice.
Give it a go...
Im actually working in a django project and I'm not sure about the best format of the URL to access into one particular object page.
I was thinking about these alternatives:
1) Using the autoincremental ID => .com/object/15
This is the simplest and well known way of do that. "id_object" is the autoincremental ID generated by the database engine while saving the object. The problem I find in this way is that the URLs are simple iterable. So we can make an simple script and visit all the pages by incrementing the ID in the URL. Maybe a security problem.
2) Using a <hash_id> => .com/object/c30204225d8311e185c3002219f52617
The "hash_id" should be some alphanumeric string value, generated for example with uuid functions. Its a good idea because it is not iterable. But generate "random" uniques IDs may cause some problems.
3) Using a Slug => .com/object/some-slug-generated-with-the-object
Django comes with a "slug" field for models, and it can be used to identify an object in the URL. The problem I find in this case is that the slug may change in the time, generating broken URLs. If some search engine like Google had indexed this broken URL, users may be guided to "not found" pages and our page rank can decrease. Freezing the Slug can be a solution. I mean, save the slug only on "Add" action, and not in the "Update" one. But the slug can now represent something old or incorrect.
All the options have advantages and disadvantages. May be using some combination of them can some the problems.
What do you think about that?
I think the best option is this:
.com/object/AUTOINCREMENT_ID/SLUG_FIELD
Why?
First reason: the AUTOINCREMENT_ID is simple for the users to identify an object. For example, in an ecommerce site, If the user want to visit several times the page (becouse he's not sure of buying the product) he will recognize the URL.
Second reason: The slug field will prevent the problem of someone iterating over the webpage and will make the URL more clear to people.
This .com/object/10/ford-munstang-2010 is clearer than .com/object/c30204225d8311e185c3002219f52617
IDs are not strictly "iterable". Things get deleted, added back, etc. Over time, there's very rarely a straight linear progression of IDs from 1-1000. From a security perspective, it doesn't really matter. If views need to be protected for some reason, you use logins and only show what each user is allowed to see to each user.
There's upsides and downsides with every approach, but I find slugs to be the best option overall. They're descriptive, they help users know where there at and at a glance enable them to tell where they're going when they click a URL. And, the downsides (404s if slugs change) can be mitigated by 1) don't change slugs, ever 2) set up proper redirects when a slug does need to change for some reason. Django even has a redirects framework baked-in to make that even easier.
The idea of combine an id and a slug is just crazy from where I'm sitting. You still rely on either the id or the slug part of the URL, so it's inherently no different that using one or the other exclusively. Or, you rely on both and compound your problems and introduce additional points of failure. Using both simply provides no meaningful benefit and seems like nothing more than a great way to introduce headaches.
Nobody talked about the UUID field (django model field reference page) which can be a good implementation of the "hash id". I think you can have an url like:
.com/object/UUID/slug
It prevents from showing an order in the URL if this order is not relevant.
Other alternatives could be:
.com/object/yyyy-mm-dd/ID/slug
.com/object/kind/ID/slug
depending of the relevant information you want to have in the url
I am running ColdFusion MX, so I don't have the possibility of using the built-in cfimage Captcha functionality in my application, before form submitting.
But the problem is without captcha the bots submit the forms.
What will be best way to prevent automatic submitting?
Captchas don't have to be images!
Try one of the following solutions:
Most bots don't understand CSS. Create two submit buttons, the first with a value that will be rejected by the server, the second with a value that will be accepted by the server. Hide the first one using CSS.
Ask the user to answer a simple math problem. This will require you to create the math problem and store the expected solution somewhere (like the user's session), then compare the user's submitted answer with the stored answer. For extra protection, you can create simple addition, subtraction and multiplication questions. Avoid division, remainders are a pain for some users.
Bots read the names of form elements, and tend to ignore text labels. Try creating a checkbox named "optout" (like a newsletter), checked by default. Next to the checkbox, ask the user to uncheck the checkbox if they are a human. The opposite technique also works (unchecked checkbox that you ask the user to check).
All of these solutions can be done without third party code or API calls.
That being said, reCAPTCHA is pretty good and easy to integrate into almost any environment.
Take a look at cfformprotect - it will work with CFMX 6 and all later engines.
It aims to be fully accessible - and invisible to most users - with an assortment of methods to stop bots and spammers.
Also you might want to look at a CF wrapper for reCaptcha, which is compatible with CFMX 7.
A technique I used with a different technology was to use image buttons. Your POST handler gets the x,y co-ordinates where the images were clicked. I found the bots (which are just generating post requests) were passing 0,0 and by dropping those requests on the floor I brought the spam posts down to less than the real ones. Sorry that I don't know how to do that in CF but I hope the technique is useful to you.
Its always a good idea to do data validation on the server side before processing no matter which solution you use.
This post may help: http://www.bennadel.com/blog/405-Fully-Accessible-Spam-Form-Submission-Blocking-Using-ColdFusion-And-X-HTML-Version-III-.htm
How about using calculation method? Just like 8 + 5 = ?
OR
how about using ColdFusion.Ajax.submitForm?
I would like to have a system where my users can invite their friends. We prefer not to use a URL shortener when sending the invite link but it is also important that the link be relatively short. I am thinking the best way to accomplish this is just give each user a "profile username" like "tonyamoyal12" and let them request a new unique one if they want.
When my users send out invites, it will send out a URL like http://mydomain/invite/profile_username and essentially if the invitee logs in at that URL, the inviter gets credit. Can anyone think of drawbacks to this approach? Most invite URL's have hashes to verify the integrity of the invite but I think my approach works fine.
UPDATE
The profile username is that of the INVITER not INVITEE. So a user signs up on the INVITER'S profile page and therefore the inviter gets a "point" for having someone sign up on his page.
Thanks!
In these types of systems, you don't usually assign any user data (i.e., user names) before the invitee has actually signed up, and it may be a bit of a pain to get that kind of URL working depending on the framework you're using.
The process is normally:
Invite a user, which sends them an e-mail.
Invitee clicks through a link in the e-mail to go to the site's main registration page.
Invitee registers with a valid user name of their choosing, and based on some unique random key (included in the clickthrough link), you can do your business logic with the two users involved (add to friends list, or whatever).
The drawback to generating your own user names is that they're more likely to be guessed than a random number, because you'll likely use English words in them. If you generate and assign random user names (i.e., "s243k2ldk8sdl"), the invitee is not going to be pleased since they have to do extra work to change the user name, or somehow remember that name.
EDIT, since I didn't understand the question very well.
I think the scheme is fine, except I would just use the invitor's user name in the URL and not allow them to change it (why allow it?). The only issue is if there is some kind of limit put on the number of invites (or maybe there is a reward for each invite), where you'd want to secure each clickthrough with some kind of unique hash value only valid for the invitor's URL.
EDIT 2
Since the users in the system do not have user names assigned, you could go either way. Allowing "user name" assignment on a first-come, first-served basis would be fine, as this would let everyone share their URL more easily with friends since it's memorable and can simply be typed in. However, that goes out the window if a unique key is required to sign up... in which case, it's going to be simpler to just not implement the user name thing and direct everyone to a single registration page of some kind.
Why not just create your own bespoke URL shortening?
If the reason you are avoiding URL-shortening is that you don't want to depend on external companies then that could be a good solution for you.
You can't independently track the invites. At some point you may want to know how many invites went out from a user vs. how many were accepted. With this single URL system you can't track that information.
Bots can easily be written to spam such a system. (Perhaps solved with captcha on resulting pages)
well if the website is large you will get name conflicts, and you will be dependent on the inviter putting in the invitees name which they could do poorly.
If you want to do it that way then you will have to deal with name clashes.
Also it is possible that someone could come along and decide to randomly type in names to see if they get it hit. Say I wanted to be nosey and spy on a friend to see if they are sending out invites to other friends.
EDIT: ahh ok. well if they are just clicking a link to go to the inviter then thats not a problem. That seems perfectly normal and there is no secret about exposed usernames.
You could create a unique hash for each invite and keep an association of hashes with user names. This would require a bit of storage overhead, but you could have expiration of invites to help combat that.
So http://mydomain/invite/RgetSqtu would be an example link, with a DB table that stores RgetSqtu/profile until it is used.
You would probably want to provide a helpful error page if the hash could not be found, like the following:
We are sorry but the invite you entered could not be found. This could be caused by the invite being typed incorrectly, being used already, or being too old (invites expire after 3 days).
I'd suggest passing the inviter's username in the querystring, and have that querystring fill either an editable or non-editable textbox on the new user registration page. That way you still have just one registration page, the URL is short, and users get credit for referring friends.
http://mydomain/invite/register.html?inviter=invitersUsername
leads to
First Name: _________
Last Name: _________
Referred By: invitersUsername
If I understand the setting correctly an existing user creates invites by giving emails of their friends to which your system will send a mail with in that the [inviteUrl]/[inveter'sUserName].
So in the case I send a mail to invite you, the url would be:
www.yourThang.com/invite/borisCallens
Every time somebody visits this I (the user borisCallens) gets a point.
What would stop me from visiting this url a gazillion times and thus win the invite-your-friend game?