I'm trying to build a simple API which should do the following:
A user makes a request on /getContent endpoint, with the geographical coordinates for this user. By content, it means audio files.
Every time they send a request, we should just get a random object from a Model and return the URL field from it, to be consumed by the front end. For this, it can be a random one, it doesn't matter much which one.
Also, we should keep tracking about the requests each user makes. This way, we can check how many requests the user has made, and when they were made.
Every 5 requests or so, the idea is to send the user a customized content based on their location. My idea is to store this content in another model, since it would have many more fields in comparison from the standard content.
Basically, at every request, I'd check if it's time to send a special content. If not, just send the random one. Otherwise, I'd check if the time is appropriate, and the user is within a valid location based on the special content's data in the model. If this validation passes, we send the URL of the special content, otherwise, we just send the random one.
I'm having a hard time figuring out the best way to design this. My initial idea is to have two different models:
Model 1: Standard content. It has some fields to its meta data, such as duration, title and other stuff like this.
Model 2: Custom content. Besides the meta data, it should contain the geographical data, and the datetime range. This will allow the checking to be made if the content should be played or not.
Now it's the part I'm pretty much clueless. How to make it all work together?
QUESTIONS
Maybe storing every single request data from every user, and checking this data might not be very effective. It would require some writing at every request instead of just reading.
Since I'd be using two different models, how can I make the decision to happen in the view? I mean, the final output would be the same, an URL. But I'd have to make the decision process to happen in the view on which model to use.
I appreciate the help!
Related
I have flow where users can create (model) forms. If form is valid, object gets saved and flow continues, but selection is in multiple pages. I need to keep current state of created object, which cannot be saved before it's completely valid.
One thing I can do is to always pass things around those views in the ModelForm to make sure, that user never loses data, but I also wanna make sure, that if he leaves the flow and comes back, he doesn't lose data, that he already entered previously.
That's why I decided I wanna save all the fields to session.
Is this correct approach?
How would you do this?
Where would you put this session logic?
What's best way of getting the fields from incomplete form to be saved?
Edit:
Please don't give me advice on how to use session, I am talking more about high level logic and architecture than specific implementation.
I should describe my flow a bit more. Model has 3 fields.
normal dropdown (foreign key referencing another model)
textfield
another foreign key, but this time not done by select, but it's own separate page with lots of filters to help user pick the right (foreign) model
Flow is not linear, because user can start in different parts of page.
Sometimes user can go to page, where he has first 2 fields + button "Browse", which takes you to selection page for 3rd field. Then after he selects field there, he comes back.
But sometimes he selects first this field and then comes to screen with 2 remaining fields, where he needs to fill those.
django-formtools offers a great way to do this using Form wizard.
The form wizard application splits forms across multiple Web pages. It
maintains state in one of the backends so that the full server-side
processing can be delayed until the submission of the final form.
More info here https://django-formtools.readthedocs.io/en/latest/wizard.html
to save in session:
request.session["variable_name"] = "value"
to get from session request.session["variable_name"]. sure you can use request.session.get("..") in both too
I am using Django's native Authorization/Authentication model to manage logins for my WebApp. This creates instances of the User model.
I would like to write a simple class-based-APIView that can tell me if a specific email is already used (IE: Is there already a user with the given email in my database?). The first time this API is called, it should get the matching User object from the DB. But subsequent times it is called, it should return it from the Memcache (if and only if, the underlying row in the database is unchanged). How can I do that??
Should I inherit from generic.APIView? Why or why not? What would the view look like? In particular I want to understand how to properly do the memcaching and cache-coherency checking. Furthermore, how would this memcaching scheme work if I had another API that modified the User object?
Thanks. I was unable to find detailed idiot-proof manual on using memcaching properly in Django.
Caching is perhaps the simplest part of django - so I'll leave that discussion to the last. The bigger problem is figuring out when your model changed.
You can decide what constitutes an update. For example, you might consider that only when a particular field is updated, then the cache is updated. Your cache update process should be limited to the writing/updating code or view. If you go about this method, then I would recommend django-model-utils and its StatusField - you can add this logic in save() method by overriding it; or implement it at the code that is updating models.
You can also do a simpler approach, that is, no matter what is updated - as long as save() is called, expire the cache and repopulate it.
The rest of the code is very simple.
Attempt to fetch the item from the cache, if the item doesn't exist (called a cache miss), then you populate the cache by fetching from the database. Otherwise, you'll get the item from the cache and then you save yourself a database hit.
The cache interface is very simple, you set('somekey', 'somevalue') you can optionally tell it when to expire the item. Then you try to get('somekey'), if this returns None, then its a cache miss (perhaps the item expired), and you have to fetch it and populate the cache. Otherwise, you'll get the cached object.
I have an app where user submitted data needs to go through a verification process before it shows up on the site. At the moment this means they cannot edit the item without removing it from the site (so our admins can check it's okay).
I'd like to write another model where I can store revisions. Basically three fields where I store the date submitted, a boolean saying if the user is ready for that revision to be considered and a third where I store all the changes (as a pickled/JSON dict).
The problem I have at the moment is I don't want to bombard the admins with a complete listing each time. I only want them to see the changed fields. This means I need a way of generating a list of which fields have changed when the user submits the edit ModelForm so I only save this data in the revision.
There are probably several ways of doing this but my post-pub-quiz brain is slightly numb and can't think of the best way. How would you do it?
In terms of where this would go, I'd probably write it as an abstract ModelForm-inheriting class that other forms use. I'd override save() to stop it writing the data directly back to database (I'd want to redirect it through this fancy new revisions model).
Come to think of it, is there an app that already does this generically?
My form validators are pretty good, and if a form passes is_valid, all data should be ok to insert in the db. Should I still validate something on the db model? What else could there be validated on the db side? Because right now, except maybe for uniqueness ( which I can't do from my FormModel ), I can't think of anything else.
EDIT:
I did some work with Rails earlier, and there you would validate a form on the client side, using JS, and on the server side using model validations. I saw in django you can validate on the client side, using JS, and on the server side you have 2 validation checks: forms and models. This is what confused me.
All data should be validated in the database if possible whether you validate from the front end or not. The first validation should be the datatype, for instance using a date datatype will ensure that no nondates can ever get into your database. If you have relationships between tables these absolutely must be enforced at the database level. If the data must be unique, it is irresponsible to not put a unique index on it. If you have a distinct set of values that are the only ones allowed, then put them in a lookup table and add a forign key constraint to that table.
The reason why it is CRITICAL to do validations in the database itself is that the user interface will not be the only thing that interacts with the database (even if you think it will be). Other applications may do so, people will need to make data changes through imports or at a query window (to fix/change large amounts of data such as when client a buys client B and you need to convert all the data to client A). Also if you change the application interface you might lose the some of the critical data integrity checks in the rewrite. Data integrity is one of the most critical factors in database design and maintenance. If you can't count on data integrity, you have no data. I have never seen a database that lets this stuff be handled by the application that didn't lose data integrity over time. Remember the database will far outlast the current application. People will still be looking at this data for years to come. The application typically doesn't consider reporting which is where the data integrity problems tend to come to light. You don't want to have to explain why you have 10,000,000 in orders that you can't identify who they were shipped to, for instance.
If your data has a constraint that's always valid, you should force it in the model/database level (and optionally at the form level). Your DB can be input in multiple ways besides just a form where validation was checked. E.g., someone can go to the django shell to save models directly or someone could create/edit a model in the admin interface or some later designer creates a new form somehow, that doesn't validate correctly.
Granted this is only required if there are additional constraints on the data. Django automatically will validate for things like fields storing proper values, if you are using the correct field types. E.g., IntegerField validates to ensure it contains an integer, EmailField checks that its entered in the form of a valid email address, django.contrib.localflavor.us.models.PhoneNumberField is a US phone number, etc. Note, this only happens if your models have the proper fields (e.g., if you use CharFields for email addresses no validation can be performed.
But there may be other links between data structures, where you should write your own validation. E.g., if all custom orders requiring special instructions (and non-custom orders only sometimes have special instructions), you should check to enforce all custom orders have something in the special instructions field (and maybe have some minimum length).
EDIT: In response to your edit, the reason for three potential validations in django is straightforward -- different validations at different points for different reasons.
Client side (javascript/jquery) validation can't be trusted at all, and should only be given as a convenience for users almost as an afterthought (if you want a spiffy smooth interface). AFAIK, django doesn't have JS validation unless you use an external package like django-ajax-forms or something, but you don't trust that the validation is correct.
Second, there's a difference between form and model validation. One model may have multiple forms for different purposes. For example, you may have a blog with a Comment Model and allow two types of users to comment: signed in users, or anonymous users. The form for anonymous users may require giving a name/email before they comment, while the form for logged in users doesn't need those fields. The signed in user form, when processed in a view may automatically add the correct name and email addresses of the signed in user to the comment model before being saved.
In contrast, model validation always applies and will always be true at the database level, regardless of how they tried saving the data. If you want to make sure some condition always applies make sure it is at the DB level. (And you don't have to write put that validation in at the form level).
I'm building a website using Ubuntu, Apache, and Django. I'd like to block people from filling out and submitting a particular form on my site more than once. I know it's pretty much impossible to block a determined user from changing his IP address, deleting his cookies, and so on; all I'm looking for is something that will deter the casual user from re-submitting.
It seems to me that blocking multiple form submissions from the same IP address is the best way to achieve what I'm looking for. However, I'm unsure how I should do this, and whether I should be doing this from Apache or from Django. Any tips?
Edit: I'm looking to prevent intentional re-submission, not just unintentional double submission. e.g. I have a survey that I want to discourage people from voting multiple times on.
If your main concern is to prevent someone writes a script and automatically submit the form many times, you may want to use CAPTCHA with your form.
Several whole countries are NAT'ed, and some (most?) large multinational corporations too, many with several hundred thousand users each. Blocking anything by IP is a bad idea.
Go for a cookie instead, which is as good as it's going to get. You could also make the user login in, in which case you'd know if the form was submitted repeatedly for that login.
I would use the session id, and store form submissions in a table with session id, timestamp, and optionally some sort of form identifier. Then, when a form is submitted, you could check the table to make sure that it had not happened within a certain period of time.
Filtering on IP address and/or cookies are both easy to get around, but they will prevent the casual user from accidentally submitting the same stuff multiple times due to browser hick-ups, impatience and so on.
If you want something better than that you could implement login, but of course that prevents a lot of users from responding.
Add to the form a monotonically increasing id number in a hidden field.
As each form is submitted, record the id in a "used" list/map (or mark it used, or whatever, implementation detail).
If you get the same id a second time (if it's already in your used map) inform the user they double-submitted.
While nothing is fool proof, I would suggest something like this: When a user loads the page with your form on, a cookie is set and the value of the cookie is appended with a fixed secret string and the md5 value of this is written to a hidden field on the form. Ensure that a new value is generated each time the user access the form.
When the user submits the form, you check that the cookie value and form value match, that the cookie the user was given has not been used to submit the form before and that the referrer id match the URL of the form. Optionally you make sure that there has been no attempts to post from that IP in the last 2 minutes (fast enough that it wont matter to most people, but slow enough to slow down bots).
To fix this the user has to make a script that loads the page, store the cookies and submit the correct values. This is much more difficult than if the user could just submit the form.
Added Based on edit: I would block the users in the Django framework. This allows you to present a much better error message to the user and you only block them from that form.
This is a question of authentication and authorisation, which are related but not the same. In order to manage authorisation you must first authenticate (reliably identify) the user.
If you want to make this resist intentional misuse then you are going to end up with not only usernames and passwords but demands for information that personally identifies your users, along the lines of the stuff a bank asks for when you want to open an account. The bleeding hearts and lefties will snivel endlessly about invasion of privacy but in fact you are doing exactly the same as a bank and for exactly the same reasons.
It's a lot of work and may be affected by law. Do you really want to do it?
The following methods are all relatively simple, both to implement and to hack around. Anyone with Firebug and a little knowledge won't even blink.
The following JavaScript uses Mootools, and I haven't checked it to be bug free. I understand that JQ syntax is almost identical, and raw JS is similar enough, so the point should be clear.
1) If the form is being submitted via AJAX, you can check before submitting (sorry if I'm just stating the obvious).
var sent = 0;
$('myForm').addEvent('submit', function(){
if(!sent) this.send();
})
This is really simple, and surprisingly effective until they reload the page.
2) Add a JavaScript cookie. Again, with Mootools:
$('myForm').addEvent('submit', function(){
if(Cookie.read('submitted')){ alert('once only'); return false;}
else{ Cookie.write('submitted', 1); return true; }
})
This will work even if the user reloads the page.
3) Add a Python session cookie. I am not familiar with Python, but if it is like PHP, this will have no advantage over method 2. In either case, the user can delete the cookie with FireCookie or WebDeveloper Toolbar (or their equiv's on other browsers) and reload the page.
4) Add a Flash cookie (use Flex). This is ideal - Flash cookies are stored in a different location, are not obvious, and are very difficult to remove. The only downside is that you need to create and embed a tiny swf.
5) Store a value in a hidden field, and check for the value.
A hash can be added to the internal links to insure that the value remains filled in even if the page is navigated away from.
6) Other games can be played incrementing a URL (or a custom URL using htaccess) for each visitor.
An swf cookie is the best idea of the above, though it can be combined with the others.