Django ORM get Latest version of each row (Inner Join) - django

I have a table (short version ofcourse) like the following
--------------------------------------------------------
|id | Simulation_Name | Simulation_Version | sim_key |
--------------------------------------------------------
| 1 | MySim1 | 1 | 10 |
| 2 | MySim1 | 2 | 10 |
| 3 | MySim2 | 1 | 11 |
| 4 | MySim2 | 2 | 11 |
--------------------------------------------------------
On the Front End, I just want to display the latest version for each simulation. I'd appreciate any help on how to do this using Django ORM. I know there is a Max field but that only works with dates..
Currently, I'm using Raw SQL with an inner join to achieve this.
Any pointers on ORM? Thanks in advance
This is how I do it right now,
SELECT *
FROM
simulation_table AS A INNER JOIN
(SELECT id, max(simulation_version) as max_version
from
simulation_table
group by sim_key) AS B
ON
A.sim_key= B.sim_keyAND A.simulation_version= B.simulation_version

Max should work with integers:
from django.db.models import Max
simulations = Simulation.objects.values('name').annotate(max_version=Max('version'))

Related

Sum where version is highest by another variable (no max version in the whole data)

I'm struggling having this measure to work.
I would like to have a measure that will sum the Value only for the max version of each house.
So following this example table:
|---------------------|------------------|------------------|
| House_Id | Version_Id | Value |
|---------------------|------------------|------------------|
| 1 | 1 | 1000 |
|---------------------|------------------|------------------|
| 1 | 2 | 2000 |
|---------------------|------------------|------------------|
| 2 | 1 | 3000 |
|---------------------|------------------|------------------|
| 3 | 1 | 5000 |
|---------------------|------------------|------------------|
The result of this measure should be: 10.000 because the house_id 1 version 1 is ignored as there's another version higher.
By House_id the result should be:
|---------------------|------------------|
| House_Id | Value |
|---------------------|------------------|
| 1 | 2000 |
|---------------------|------------------|
| 1 | 3000 |
|---------------------|------------------|
| 2 | 5000 |
|---------------------|------------------|
Can anyone help me?
EDIT:
Given the correct answer #RADO gave, now I want to further enhance this measure:
Now, my main Data table in reality has more columns.
What if I want to add this measure to a table visual that splits the measure by another column from (or related to) the Data table.
For example (simplified data table):
|---------------------|------------------|------------------|------------------|
| House_Id | Version_Id | Color_Id | Value |
|---------------------|------------------|------------------|------------------|
| 1 | 1 | 1 (Green) | 1000 |
|---------------------|------------------|------------------|------------------|
| 1 | 2 | 2 (Red) | 2000 |
|---------------------|------------------|------------------|------------------|
| 2 | 1 | 1 (Green) | 3000 |
|---------------------|------------------|------------------|------------------|
| 3 | 1 | 1 (Green) | 5000 |
|---------------------|------------------|------------------|------------------|
There's a Color_Id in the main table that is connected to a Color table.
Then I add a visual table with ColorName (from the ColorTable) and the measure (ColorId 1 is Green, 2 is Red).
With the given answer the result is wrong when filtered by ColorName. Although the Total row is indeed correct:
|---------------------|------------------|
| ColorName | Value |
|---------------------|------------------|
| Green | 9000 |
|---------------------|------------------|
| Red | 2000 |
|---------------------|------------------|
| Total | 10000 |
|---------------------|------------------|
This result is wrong per ColorName as 9000 + 2000 is 11000 and not 10000.
The measure should ignore the rows with an old version. In the example before this is the row for House_Id 1 and Color_Id Green because the version is old (there's a newer version for that House_Id).
So:
How can I address this situation?
What If I want to filter by another column from (or related to) the Data table such as Location_Id? It is posible to define the measure in such a way that could work for any given number splits for columns in the main Data table?
I use "Data" as a name of your table.
Sum of Latest Values =
VAR Latest_Versions =
SUMMARIZE ( Data, Data[House_id], "Latest_Version", MAX ( Data[Version_Id] ) )
VAR Latest_Values =
TREATAS ( Latest_Versions, Data[House_id], Data[Version_Id] )
VAR Result =
CALCULATE ( SUM ( Data[Value] ), Latest_Values )
RETURN Result
Measure output:
How it works:
We calculate a virtual table of house_ids and their max versions, and store it in a variable "Latest_Versions"
We use the table from the first step to filter data for the latest versions only, and establish proper data lineage
(https://www.sqlbi.com/articles/understanding-data-lineage-in-dax/)
We calculate the sum of latest values by filtering data for the latest values only.
You can learn more about this pattern here:
https://www.sqlbi.com/articles/propagate-filters-using-treatas-in-dax/

PowerBI Sort Columns in Matrix Visual

I have a Matrix visual in Microsoft PowerBI with Australian 'States' as rows and 'Months Ago' as columns.
By default the Matrix shows my columns from 0 months ago to 12. I would like it to show from 12 months ago on the left to 0 months ago on the right.
+-------------------+-----------------------------+-------+
| | Months Ago | |
+-------------------+-----------------------------+-------+
| State | 0 | 1 | 2 | 3 | 4 | 5 | Total |
+-------------------+----+----+----+----+----+----+-------+
| Queensland | 10 | 10 | 10 | 10 | 10 | 10 | 60 |
+-------------------+----+----+----+----+----+----+-------+
| New South Wales | | | | | | | |
+-------------------+----+----+----+----+----+----+-------+
| Victoria | | | | | | | |
+-------------------+----+----+----+----+----+----+-------+
| South Australia | | | | | | | |
+-------------------+----+----+----+----+----+----+-------+
| Western Australia | | | | | | | |
+-------------------+----+----+----+----+----+----+-------+
Currently I am only given the option to sort by the value type fields (ie revenue etc).
Is there any option to sort/order the Column Headers?
I don't think there is an option for you to sort column headers directly.
However, you can change the default sort order for the Months Ago column so that it will be reflected in general.
You can add a custom column MonthSrt = 12 - [Months Ago] in query editor:
(It won't work in DAX because of a known issue)
Then you can select the Months Ago column and sort it by MonthSrt:
The custom sort will be applied when you use the Months Ago column in visuals:
You can also make groups (1 to 1 items) al give them al logical number:
The order will change automaticly in the matrix
The following solution worked for me to display the dates in descending order in a matrix:
how to sort column dates in descending order of matrix in power bi

How to run raw query with a model with dynamic fields in Django 1.9?

I have a complex result that requires writing raw sql queries.
See https://stackoverflow.com/a/38548462/80353
The expected result is a table showing several columns.
The first column header is simply Product and the other column headers are store names.
The values are simply the product names and the aggregated sales values of the product in these stores.
Which stores will be shown is entirely dynamic. Maximum should be 9 stores.
The same in text format:
Store table
------------------------------
| id | code | address |
|-----|------|---------------|
| 1 | S1 | Kings Row |
| 2 | S2 | Queens Street |
| 3 | S3 | Jacks Place |
| 4 | S4 | Diamonds Alley|
| 5 | S5 | Hearts Road |
------------------------------
Product table
------------------------------
| id | code | name |
|-----|------|---------------|
| 1 | P1 | Saucer 12 |
| 2 | P2 | Plate 15 |
| 3 | P3 | Saucer 13 |
| 4 | P4 | Saucer 14 |
| 5 | P5 | Plate 16 |
| and many more .... |
|1000 |P1000 | Bowl 25 |
|----------------------------|
Sales table
----------------------------------------
| id | product_id | store_id | amount |
|-----|------------|----------|--------|
| 1 | 1 | 1 |7.05 |
| 2 | 1 | 2 |9.00 |
| 3 | 2 | 3 |1.00 |
| 4 | 2 | 3 |1.00 |
| 5 | 2 | 5 |1.00 |
| and many more .... |
| 1000| 20 | 4 |1.00 |
|--------------------------------------|
The relationships are:
Sales belongs to Store
Sales belongs to Product
Store has many Sales
Product has many Sales
What I want to achieve
I want to display by pagination in the following manner:
Given the stores S1-S3:
-------------------------
| product | S1 | S2 | S3 |
|---------|----|----|----|
|Saucer 12|7.05|9 | 0 |
|Plate 15 |0 |0 | 2 |
| and many more .... |
|------------------------|
For more details of the schema, check the question in How to get back aggregate values across 2 dimensions using Python Cubes?
My question
The schema is not super important to my question which is:
Since I am going to write a complex raw query, is there a way to map the query result to a model where the fields are dynamic?
I found documentation about how to execute raw queries in Django and how to execute raw queries to existing models with fixed fields and matching table.
My question is is it possible to do that for a model that has no matching table and dynamic fields?
If so, how?
Or if I choose to use materialised view in postgresql, how do I match it with a model class?

How do I write an update join postgres query as queryset in Django 1.9?

Situation
I have the following query in Postgres 9.5.
update warehouse_shelf
set package_count = counter.result
from (
select count(id) as result, shelf_id
from warehouse_package
group by shelf_id
) counter
where counter.shelf_id = warehouse_shelf.id;
These are my tables.
warehouse_shelf
+----+------+---------------+
| ID | NAME | PACKAGE_COUNT |
+----+------+---------------+
| 1 | S1 | 3 |
| 2 | S2 | 1 |
| 3 | S3 | 0 |
+----+------+---------------+
warehouse_package
+----+------+---------------+
| ID | NAME | SHELF_ID |
+----+------+---------------+
| 1 | P1 | 1 |
| 2 | P2 | 1 |
| 3 | P3 | 1 |
| 4 | P4 | 2 |
+----+------+---------------+
Question
How do I execute the query above whenever I make a change to an individual package (e.g. save, delete, create, update etc) via django model?
I want to execute using django queryset if possible and avoid executing it as raw query.
Considering you have models and their foreign key relations:
from django.db.models import Count
shelves = WhShelf.objects.all()
for shelf in shelves:
count = WhPackage.objects.filter(shelf_id=shelf.id).aggregate(Count('shelf'))
shelf.update(package_count=count[shelf__count'])
Alterantively, you can run a single query:
WhShelf.objects.annotate(package_count=WhPackage.objects.
filter(shelf_id=shelf.id).aggregate(Count('shelf'))['shelf__count'])

Update Autonumber field after DELETE query

INTRODUCTION AND RELEVANT INFORMATION:
I have MS ACCESS 2007 database that I edit using ADO and C++.
PROBLEM:
My problem is that primary key also represents an ordinal number of the record, and after deletion, it should be properly updated. Primary key is of autonumber type.
Here is an example of what I am talking about:
| #PK | Other data ... |
| 1 | ... |
| 2 | ... |
| 3 | ... |
| 4 | ... |
| 5 | ... |
Now if I delete the 3rd record I get the following problem:
| #PK | Other data ... |
| 1 | ... |
| 2 | ... |
| 4 | ... |
| 5 | ... |
but I should get the following result:
| #PK | Other data ... |
| 1 | ... |
| 2 | ... |
| 3 | ... | // edited to reflect the change ( previous value was 4 )
| 4 | ... | // edited to reflect the change ( previous value was 5 )
If I delete last record and then insert new one I get this result:
| #PK | Other data ... |
| 1 | ... |
| 2 | ... |
| 3 | ... |
| 4 | ... |
| 6 | ... | // this should be 5
QUESTIONS:
Is there a way for me to programmatically update the autonumber field after I perform the DELETE query ?
EDIT:
Since I am aware this is a bad practice, I would prefer adding new field that should be ordinal number so my table can look like this:
| #PK | Ordinal | Other data ... |
| 1 | 1 | ... |
| 2 | 2 | ... |
| 4 | 3 | ... |
| 5 | 4 | ... |
but I would prefer it to update itself automatically. If this is not possible, I would prefer to update the field with SQL query after I perform the deletion.
Thank you.
Best regards.
It is possible, but not the right way. Primary keys are used for relationships, so if you change the values, you need to update all related tables. Even if you currently don't have any related tables, you still should consider adding a separate field for the order, otherwise you may face the same problem in the future when you want to add related tables.
EDIT To answer your question:
Is there a way to add another field that would represent ordinal number and will automatically increment after inserting new record?
If you set it to autonumber, it will automatically increment, but you will not be able to modify it. You can set it to number and when you insert, you use SELECT MAX(oredinal) + 1 FROM mytable to increment it.
For MS Access use
ALter Table Customer alter column CustomerID Counter(1,1)
For Sql Server
DBCC CHECKIDENT (orders, RESEED, 0)
This will set the value of the next ID to be 1, you can use above command.
Ref URL# http://www.howtogeek.com/howto/database/reset-identity-column-value-in-sql-server/
I have decided to add a new field in my table that will hold the ordinal number of the record.
If we assume the field is named OrdinalNumber then the following solution worked for me:
// when inserting record, I just had to add COUNT( PK ) + 1
INSERT INTO MyTable ( OrdinalNumber , ... ) SELECT COUNT( PK ) + 1 , ...
from MyTable ;
// when deleting, I had to perform following two queries :
DELETE from MyTable where PK = ? ;
// decrement all the successors ordinal number by one
UPDATE MyTable set OrdinalNumber = ( OrdinalNumber - 1 ) where ( PK > ? );
Everything seem to work well. I wish there was an easier way though...
Thanks everyone for helping. I have upvoted all the answers.