I have a field in a model that I want users to feel like they can write an arbitrary amount of text in. Django provides a CharField and a TextField in the models. I assume that the difference is that one of them is a char(max_length) and the other is a varchar internally.
I am tempted to use the TextField, but since it doesn't respect max_length, I am somewhat wary of someone dumping loads of data into it and DOSing my server. How should I deal with this?
Thanks!
Fields in model only represent the way data is stored in database.
You can very easily enforce maximum length in form which will validate users' input. Like this.
class InputForm(forms.Form):
text = forms.CharField(max_length=16384, widget=forms.TextArea)
...
This will make sure the maximum length user can successfully enter is 16k.
Related
In my project few models has many fields like more than 25. Like i have a model name PeriodOfStay. and it fields are like
date_of_entry
i94_number
port_of_entry
city ....etc (please check the image for all field)
also it has many boolean fields . in one form user can multiple options.
most of the fields are optional.
so i am confused should i put all the fields in one model. Is it best practice. I don't want split one model to more and use OneToOne Relation cause in that case i need to break up many models cause most of the models in my project are like this also i need to send all data at once in a single request.
I just need to save data and show data to user. in some case i need to search by some field . Like in this form i need to search by i94_number.
Is using JsonField is ok for this problem cause i need to search & filter in some case.
I appreciate any help. Advance Thanks For Help.
Regarding your question about when to use one-to-one relations:
https://dba.stackexchange.com/a/15405
I think a JsonField is not what you want if you want to search & filter based on some fields. Keeping it in normal fields will make it faster
Some related resource that might be interesting: https://en.wikipedia.org/wiki/Database_normalization
I am working on a project where users can roll dice pools. A dice pool is, for example, a throw with 3 red dice, 2 blue and 1 green. A pool is composed of several dice rolls and modifiers.
I have 3 models connected this way:
class DicePool(models.Model):
# some relevant fields
class DiceRoll(models.Model):
pool = models.ForeignKey(DicePool, on_delete=models.CASCADE)
# plus a few more information fields with the type of die used, result, etc
class Modifier(models.Model):
pool = models.ForeignKey(DicePool, on_delete=models.CASCADE)
# plus about 4 more information fields
Now, when I load the DicePool history, I need to prefetch both the DiceRoll and Modifier.
I am now considering replacing the model Modifier with a textfield containing some JSON in DicePool. Just to reduce the number of database queries.
Is it common to use a json textfield instead of a database relationship?
Or am I thinking this wrong and it's completely normal to do additional queries to prefetch_related everytime I load my pools?
I personally find using a ForeignKey cleaner and it would let me do db-wise changes to data if needed. But my code is making too many db queries and I am trying to see where I can improve it.
FYI: I am using MySQL
Is it common to use a JSON text field instead of a database relationship?
I don't think so. Also, I don't believe it's advisable because (especially using MySQL that doesn't support things like JSONField) you'll end up with a text that you'd then need to parse somehow to a dict and then look up the things you want.
Personally (and I would assume that most people) would stick to FK relationships. Also, by doing prefetch_related or select_related you're already avoiding unnecessary queries.
I'm facing a dilemma, I'm creating a new product and I would not like to mess up the way I organise the informations in my database.
I have these two choices for my models, the first one would be to use foreign keys to link my them together.
Class Page(models.Model):
data = JsonField()
Class Image(models.Model):
page = models.ForeignKey(Page)
data = JsonField()
Class Video(models.Model):
page = models.ForeignKey(Page)
data = JsonField()
etc...
The second is to keep everything in Page's JSONField:
Class Page(models.Model):
data = JsonField() # videos and pictures, etc... are stored here
Is one better than the other and why? This would be a huge help on the way I would organize my databases in the futur.
I thought maybe the second option could be slower since everytime something changes all the json would be overridden, but does it make a huge difference or is what I am saying false?
A JSONField obfuscates the underlying data, making it difficult to write readable code and fully use Django's built-in ORM, validations and other niceties (ModelForms for example). While it gives flexibility to save anything you want to the db (e.g. no need to migrate the db when adding new fields), it takes away the clarity of explicit fields and makes it easy to introduce errors later on.
For example, if you start saving a new key in your data and then try to access that key in your code, older objects won't have it and you might find your app crashing depending on which object you're accessing. That can't happen if you use a separate field.
I would always try to avoid it unless there's no other way.
Typically I use a JSONField in two cases:
To save a response from 3rd party APIs (e.g. as an audit trail)
To save references to archived objects (e.g. when the live products in my db change but I still have orders referencing the product).
If you use PostgreSQL, as a relational database, it's optimised to be super-performant on JOINs so using ForeignKeys is actually a good thing. Use select_related and prefetch_related in your code to optimise the number of queries made, but the queries themselves will scale well even for millions of entries.
I'm working on a Django app for keeping track of collections (coins, cards, gems, stamps, cars, whatever). You can have multiple collections, each collection can have sets (Pirates cards, Cardinals cards, etc.) and then of course the individual items in each collection/set. Each item can contain multiple pictures, a name, and description, but here's where I'm unsure how to proceed. Each collection will need it's own set of values, or fields, that the user will need to determine (condition, dimensions in the appropriate units, coin thickness, model number, etc). How can I make custom fields such that the user can name the field and choose the input type (text, numbers, dropdown w/choices) and those fields will show up to be entered on each item within that collection?
This would be called an Entity-Attribute-Value (EAV) model and it is quite tricky to implement in the way you want. You have to anticipate all sorts of issues with user input, how to validate field types, what happens when the user wants to change fields, etc. I would start by reading the issues raised in that question and think about ways that you could modify your schema to avoid letting users define their own metadata at runtime. Are there some fields that could be common to all collections (like condition, dimensions, model number)? How tolerant do you want to be of data type issues, and will users be allowed to change field types after creation?
The more thought you put into implementation, the more issues you can avoid down the road.
Summary:
Looking for a good way to save data to Django models for which the associated forms are generated dynamically.
Detail:
I've been puzzling over the best approach for creating dynamic Django forms backed by models. For example, I'd like to create an interface where a user can create an HTML form, customize the types of fields in that form dynamically (Number, String, Dropdown Box, Date, etc.), and then display that form to other users so those users can submit data which is saved to a database. I'm not sure how to make an efficient approach to persist the data.
www.formsite.com and www.mailchimp.com have some form-building tools that are nice examples of what I am trying to do. Jacob Kaplan-Moss has an excellent tutorial on how to create the forms dynamically, but the tutorial doesn't get into how to persist the data.
As a dummy example, one (perhaps bad?) approach might be to create some models like below, where there is a database table for the SurveyQuestions (storing the customizable names and datatypes of each field) and one for the SurveyQuestionResponses (each record storing an individual response for a SurveyQuestion on a particular Survey).
However, it seems like this approach might result in really complex and slow queries. For example, if a Survey has 10 questions and you would like to display 10 user responses to that survey, there would be queries to select all 10 SurveyQuestions and then for each survey responder, there would be a query to select each of the SurveyQuestionResponses. It seems like the number of queries needed could add up really fast!
class Survey(models.Model):
# some fields here.
pass
class SurveyQuestion(models.Model):
""" Defines the headings and field
types for a given Survey.
"""
survey = models.ForeignKey(Survey)
field_name = models.CharField(
max_length=255,
help_text='Enter the name for this field heading')
field_type = models.IntegerField(
choices=choices.FIELD_TYPES,
help_text='Enter the data type for this field')
display_order = models.IntegerField(default=0)
class SurveyQuestionResponse(models.Model):
survey_field = models.ForeignKey(SurveyQuestion)
response value = models.TextArea(blank=True, null=True)
Is there a better approach to persisting data based on dynamic forms? Should I be somehow converting a form respondent's response to some sort of pickled format and store it to a TextField (Instead of having 10 SurveyQuestionResponse records there would be one record with all the response values pickled together)? I'm not too familiar with NoSQL options, but would a NoSQL approach work best for this type of thing? Is there some sort of rendering or caching that would make sense to do?
I keep encountering situations where saving data from dynamic forms like this would be very useful. I am wondering what other people's approaches are. Any advice is much appreciated. Thanks for reading this admittedly long question.
Joe
For a relational database an Entity-attribute-value model(EAV) could be used to achieve a dynamic, or open schema. Relational databases are not really suited for this type of schema, and this generally results in very slow queries over time. NoSQL has its own set of issues but I think that it would be best suited to your requirements. If you decide to take this route you can take a look at MongoDB. I have not used it extensively, but it seems most similar to relational database than the other NoSQL database out there, and its python interface seems pretty similar to django's ORM. By the was I remember finding a nice EAV example for Django. Though I don't remember where at the moment.