I have 5 different models....similar structure but seperated due to lots of record so easier management and scalability.
I'm not sure if this is possible with django, but i have a div on the website which is called latest....i want to grab the latest 3 records and show it in this div....however having 5 different models its difficult....each have a timestamp field. Is it possible to query something like show latest 3 records in total, but check 5 of these models and display?
Usually if it was in one model I could have just easily said show latest x....but separated models makes its complicated. So i don't want to grab latest 3 records from each model....rather 3 in total but just consider from 5 different models and show latest 3 (either through filtering of id of timestamp field)
Please kindly let me know if there is a solution.
There is two different solution.
At first you can merge models and make a general model with common columns and each of 5 models connect to general model with ForeignKey. So you can make query on general model and access specific columns with related objects.
Second way is usingdjango contenttypes framework
Related
I am working on a project where users can roll dice pools. A dice pool is, for example, a throw with 3 red dice, 2 blue and 1 green. A pool is composed of several dice rolls and modifiers.
I have 3 models connected this way:
class DicePool(models.Model):
# some relevant fields
class DiceRoll(models.Model):
pool = models.ForeignKey(DicePool, on_delete=models.CASCADE)
# plus a few more information fields with the type of die used, result, etc
class Modifier(models.Model):
pool = models.ForeignKey(DicePool, on_delete=models.CASCADE)
# plus about 4 more information fields
Now, when I load the DicePool history, I need to prefetch both the DiceRoll and Modifier.
I am now considering replacing the model Modifier with a textfield containing some JSON in DicePool. Just to reduce the number of database queries.
Is it common to use a json textfield instead of a database relationship?
Or am I thinking this wrong and it's completely normal to do additional queries to prefetch_related everytime I load my pools?
I personally find using a ForeignKey cleaner and it would let me do db-wise changes to data if needed. But my code is making too many db queries and I am trying to see where I can improve it.
FYI: I am using MySQL
Is it common to use a JSON text field instead of a database relationship?
I don't think so. Also, I don't believe it's advisable because (especially using MySQL that doesn't support things like JSONField) you'll end up with a text that you'd then need to parse somehow to a dict and then look up the things you want.
Personally (and I would assume that most people) would stick to FK relationships. Also, by doing prefetch_related or select_related you're already avoiding unnecessary queries.
I need to determine the best approach to determine the structure of my Django app models at runtime based on the structure of an uploaded CSV file, which will then be held constant once the models are created in Django.
I have come across several questions relating to dynamically creating/altering Django models at run-time. The consensus was that this is bad practice and one should know before hand what the fields are.
I am creating a site where a user can upload a time-series based csv file with many columns representing sensor channels. The user must then be able to select a field to plot the corresponding data of that field. The data will be approximately 1 Billion rows.
Essentially, I am seeking to code in the following steps, but information is scarce and I have never done a job like this before:
User selects a CSV (or DAT) file.
The app then loads only the header row in (these files are > 4GB).
The header row is split by ",".
I use the results from 3 to create a table for each channel (columns), with the name of the field the same as the individual header entry for that specific channel.
I then load the corresponding data into the respective tables and I ahve my models for my app that will then not be changed again.
Another option I am considering is creating model with 10 fields, as I know there will never be more than 10 channels. Then reading my CSV into the table when a user loads a file, and just having those fields empty.
Has anyone had experience with similar applications?
That are allot of records, never worked with so many. For performance the fixed fields idea sounds best. If you use PostgreSQL you could look at the JSON field but don't know the impact on so many rows.
For flexible models you could use the EAV pattern but this works only for small data sets in my experience.
I have a model class in my django project:
*user_id
*amount
*net_balance
*created_on
I have a list of user_ids(let's say 3). I need to get the last row for each user_id and then do some operation and create a new row for each user id. How do this efficiently. I can certainly do 6 transactions (if there are 3 items in list of userids).
If you want the most recent entry then
YourModel.objects.filter(user=user_id).latest('created_on')
If I understand your question correctly then you need to get all the user_ids (presumably you have a separate User model?) and then loop through them - for each user getting the most recent entry and then create the new row.
You need 1 select (at least) for all the records you interested and 1 insert query for each record returned.
The select query can be generated by ORM abilities (aggregation) or you can use raw SQL if you fill comfortable. If you use PostgreSQL, you can use distinct ability (I recommended) as:
Model.objects.order_by('user_id', '-created_on').distinct('user_id')
or you can use aggregation abilities as:
Model.objects.filter(user_id__in=[1,2,3]).values('user_id', 'created_on').annotate(last_row=Max('created_on')).filter(created_on=F('last_row'))
The correct answer depends on your Django version and database. But there are lots of good features in Django to achieve this kind of stuffs.
I have 4 tables in my database. The image below shows the rows and columns with the name of the table enclosed in a red box. 4 tables total. Am I going about the relationship design correctly? This is a test project and I am strongly assuming that I will use a JOIN to get the entire set of data on one table. I want to start this very correctly.
A beginner question but is it normal that the publisher table, for example, has 4 rows with Nintendo?
I am using Django 1.7 along with PostgreSQL 9.3. I aim to keep simple with room to grow.
Basically you've got the relations back-to-front here...
You have game_id (i.e. a ForeignKey relation) on each of publisher, developer and platform models... but that means each of those entities can only be related to a single game. I'm pretty sure that's not what you want.
You need it the other way around... instead put three foreign keys onto the game model, one each for publisher, developer and platform.
A ForeignKey is what's called a many-to-one relation. In this example I think what you want is for 'many' games to be related to 'one' publisher. Same for developer and platform.
is it normal that the publisher table, for example, has 4 rows with Nintendo?
No, that's is an example of why you have it backwards. You should only have a single row for each publisher.
yes you are correct in saying that something is wrong.
First of all those screen shots are hard to follow, for this simple example they could work but that is not the right tool, pick up pen and paper and sketch some relational diagrams and think about what are the entities involved in the schema and what are their relations, for example you know you have publishers, and they can publish games, so in this restricted example you have 2 entities, game and publisher, and a relation publish among them (in this case you can place a fk on game if you have a single publisher for a game, or create an intermediary relation for a many to many case). The same point can be made for platform and games, why are you placing an fk to game there, what will happen if the game with id 2 will be published for nintendo 64 ? You are making the exact same mistake in all the entities.
Pick up any book about database design basics, maybe it will help in reasoning about your context and future problems.
In django I have three models:
SimpleProduct
ConfigurableProduct Instead of showing several variations of SimpleProducts, the user will see one product with options like color.
GroupProduct - Several SimpleProducts that are sold together.
First I'm creating all the SimpleProducts, then I create ConfigurableProducts from several products that are variations on the same product and last GroupProducts which are combiniations of several SimpleProducts.
When a user navigate to a category I need to show him all the three types. If a SimpleProduct is part of a ConfigurableProduct I don't want to show it twice.
How do I make the query? Do I have to create three several queries?
How do I use pagination on three models at the same time?
Can I somehow use inheritance?
Thanks
I think this question is tough to answer without understanding your business logic a little more clearly. Here are my assumptions:
Configurable options are ad hoc, i.e., you sell balls in red, blue, and yellow, shirts in small, medium, and large, etc. There is no way to represent these options abstractly because they don't transcend categories. (If they did, your database design is all wrong. If everything had custom color options, you would just make that a column in your database table.)
Each configuration option has a pre-existing business identity at your company. There's some sku associated with red balls or something like that. For whatever reason, it is necessary to have a database row for each possible configuration option. (If it isn't, then again, you're doing it all wrong.)
If this is the case, my simplest recommendation would be to have some base class that all products inherit from with a field: representative_product_id. The idea is that for every product, there is a representative version that gets shown on the category page, or anywhere else in your catalog. In your database, this will look like:
Name id representative_id
red_ball 1 1
blue_ball 2 1
green_ball 3 1
small_shirt 4 4
medium_shirt 5 4
large_shirt 6 4
unique_thing 7 7
As for django queries, I would use F objects if you have version 1.1 or later. Just:
SimpleProduct.objects.filter(representative_id=F('id'))
That will return a queryset whose representative ids match their own ids.
At this point, someone will clamor for data integrity. The main condition is that representative_id must in all cases point to an object whose representative_id matches its id. There are ways to enforce this directly, such as with a pre_save validator or something like that. You could also do effectively the same thing by factoring out a ProductType table that contains a representative_id column. I.e.:
Products
Name id product_type
_________________________________
red_ball 1 ball
blue_ball 2 ball
green_ball 3 ball
small_shirt 4 shirt
medium_shirt 5 shirt
large_shirt 6 shirt
unique_thing 7 thing
Types
Name representative_id
_______________________________
ball 1
shit 4
thing 7
This doesn't replace the need to enforce integrity with some validator, but it makes it a little more abstract.
Go with Django's multi-table inheritance, with a base class you won't instanciate directly. The base class still has a manager you can run queries against, and that will contain the base attributes of any subclass instance.
To tackle your question about configurable products that must not be displayed redundantly, I think you have two options:
Make configurable products a multiple choice of ConfigurableProductChoice (unrelated to SimpleProduct). Have the ConfigurableProductChoice extend the ConfigurableProduct. That way you'll have a single ConfigurableProduct in your results and no redundancy.
Make configurable products be associated to various options, and design a rule to compute the price from what options are selected. A simple addition would be fine. Your product IDs will need to encode what options are selected. You still have no redundancy, because you didn't involve SimpleProduct.