I would like to get the list of all available groups with an additional Boolean field showing if a particular user is member of that group.
#This give me all groups related to the user "myuser"
mygroup = myuser.groups.all()
The result is a list of id, groupname related to the user "myuser".
How can I have a list of all groups (including those not part of mygroup) with evidence of membership for "myuser" on additional Boolean field (user_membership)?
id, groupname, user_membership(true/false)
You could use a conditional expression:
Group.objects.annotate(user_membership=Case(
When(user=myuser, then=Value(True)),
default=Value(False),
output_field=BooleanField(),
))
This would return all groups with an additional user_membership field containing True or False values.
Related
Given the following models:
class Customer(models.Model):
pass
class User(models.Model):
email = models.EmailFIeld(blank=True, default="")
customer = models.ForeignKey(Customer, ...)
I want to enforce the following:
IF user has email
IF user has no customer
email must be globally unique (i.e. unique in the entire unfiltered users table)
IF user has customer
email must be unique within the user's customer
I attempted to implement this with two UniqueConstraints:
UniqueConstraint(
name="customer_scoped_unique_email",
fields=["customer", "email"],
condition=(
Q(customer__isnull=False)
& ~Q(email=None)
),
),
UniqueConstraint(
name="unscoped_unique_email",
fields=["email"],
condition=(
Q(customer=None)
& ~Q(email=None)
),
),
Testing has revealed that this still allows a user without a customer to be created with an email identical to an existing user (with a customer). My understanding is that this is because UniqueConstraint.condition determines both when the unique constraint should be triggered and what other records are included in the uniqueness check.
Is there any way to achieve my desired logic in the database, ideally in a Django ORM-supported way, and ideally with a UniqueConstraint or CheckConstraint? This must occur in the database. It's obviously possible in Python, but I want the extra reliability of a database constraint.
Is there any way to achieve my desired logic in the database ...
Yes, you can use triggers (see SQL in the Triggers section below).
... ideally in a Django ORM-supported way ...
Not within Django ORM, but for PostgreSQL, you could use django-pgtrigger to define it on models.
... and ideally with a UniqueConstraint or CheckConstraint?
This is not supported at the database level, since partial indexes only contain the rows based on WHERE.
Partial indexes
UniqueConstraint.condition has the same database restrictions as Index.condition.
PostgreSQL: https://www.postgresql.org/docs/8.0/indexes-partial.html
A partial index is an index built over a subset of a table; the subset is defined by a conditional expression (called the predicate of the partial index). The index contains entries for only those table rows that satisfy the predicate.
SQLite: https://www.sqlite.org/partialindex.html
A partial index is an index over a subset of the rows of a table.
... In partial indexes, only some subset of the rows in the table have corresponding index entries.
Only rows of the table for which the WHERE clause evaluates to true are included in the index.
Triggers
Before insert or update on user table, check unscoped unique email.
PostgreSQL trigger:
CREATE OR REPLACE FUNCTION unscoped_unique_email() RETURNS TRIGGER AS $unscoped_unique_email$
DECLARE
is_used_email bool;
BEGIN
IF NEW.email IS NOT NULL AND NEW.customer_id IS NULL THEN
SELECT TRUE INTO is_used_email FROM user WHERE email = NEW.email AND (id != NEW.id OR NEW.id IS NULL);
IF is_used_email IS NOT NULL THEN
RAISE EXCEPTION 'duplicate key value violates unique constraint "unscoped_unique_email"'
USING DETAIL = format('Key (email)=(%s) already exists.', NEW.email);
END IF;
END IF;
RETURN NEW;
END;
$unscoped_unique_email$ LANGUAGE plpgsql;
CREATE TRIGGER unscoped_unique_email BEFORE INSERT OR UPDATE ON user
FOR EACH ROW EXECUTE PROCEDURE unscoped_unique_email();
I am playing around with django ORM
import django
django.setup()
from django.contrib.auth.models import User, Group
from django.db.models import Count
# All users
print(User.objects.all().count())
# --> 742
# Should be: All users which are in a group.
# But the result is different. I don't understand this.
print(User.objects.filter(groups__in=Group.objects.all()).count())
# --> 1731
# All users which are in a group.
# distinct needed
print(User.objects.filter(groups__in=Group.objects.all()).distinct().count())
# --> 543
# All users which are in a group. Without distinct, annotate seems to do this.
print(User.objects.filter(groups__in=Group.objects.all()).annotate(Count('pk')).count())
# --> 543
# All users which are in no group
print(User.objects.filter(groups__isnull=True).count())
# --> 199
# 199 + 543 = 742 (nice)
I don't understand the second query which returns 1731.
I know that I can use distinct().
Nevertheless 1731 looks like a bug to me.
What is the intention why below query is not distinct/unique?
User.objects.filter(groups__in=Group.objects.all())
Raw MySQL query looks like this:
SELECT user.id, group.id FROM user LEFT JOIN group ON user.group_id = group.id
The result will contain all possible combinations of users and groups and I guess some users belong to more than one group.
You are trying to fetch all users from all groups, but a user can present in multiple groups that's why distinct is required. if you want users ina specific group instead of doing an all try a filter query.
I assume that User.groups is a ForeignKey or some other relationship that associates each User with zero to many Group instances.
So the query which confuses you:
User.objects.filter(groups__in=Group.objects.all())
That query can be described as:
Access the Group model manager (Group.objects).
Make a QuerySet:
Return all Group instances (Group.objects.all()).
Access the User model manager (User.objects).
Make a Queryset:
Join to the Group model, on the User.groups foreign key.
Return every (User + Group) row which has an associated Group.
That is not “all users which are in a group”; instead, it is “All user–group pairs where the group exists”.
By querying on each of the multiple-value User.groups field, you are implying that the query must contain a join from User to Group rows.
Instead, you want:
Access the User model manager (User.objects).
Make a QuerySet:
Return all rows which have groups not empty.
User.objects.filter(groups__isnull=False)
Note that this – “All users which have a non-empty set of associated groups” – is the inverse of another example query you have (“All users which are in no group”).
Since groups is a ManyToManyField the query translated into INNER JOIN statement.
If you print the following you will see the query generated by the QuerySet:
>>> print(User.objects.filter(groups__in=Group.objects.all()).query)
SELECT `auth_user`.`id`, .... , `auth_user`.`date_joined` FROM `auth_user` INNER JOIN `auth_user_groups` ON (`auth_user`.`id` = `auth_user_groups`.`user_id`) WHERE `auth_user_groups`.`group_id` IN (SELECT `auth_group`.`id` FROM `auth_group`)
As you would see the query joins auth_user and auth_user_groups tables.
Where auth_user_groups is the ManyToManyField table not the table for Group model. Thus a user will come more than once.
You would want to use annotate get users having grous, in my case the numbers are following:
$ ./manage.py shell
>>>
>>> from django.contrib.auth.models import User, Group
>>> from django.db.models import Count
>>>
# All users
>>> print(User.objects.all().count())
556
>>>
# All users which are not in a group.
>>> print(User.objects.annotate(group_count=Count('groups')).filter(group_count=0).count())
44
>>>
# All users which are in a group.
>>> print(User.objects.annotate(group_count=Count('groups')).filter(group_count__gt=0).count())
512
>>>
Annotate is similar to distinct in behaviour. It creates a group by query. You can see and inspect the query as following.
>>> print(User.objects.annotate(group_count=Count('groups')).filter(group_count__gt=0).query)
SELECT `auth_user`.`id`, `auth_user`.`password`, `auth_user`.`last_login`, `auth_user`.`is_superuser`, `auth_user`.`username`, `auth_user`.`first_name`, `auth_user`.`last_name`, `auth_user`.`email`, `auth_user`.`is_staff`, `auth_user`.`is_active`, `auth_user`.`date_joined`, COUNT(`auth_user_groups`.`group_id`) AS `group_count` FROM `auth_user` LEFT OUTER JOIN `auth_user_groups` ON (`auth_user`.`id` = `auth_user_groups`.`user_id`) GROUP BY `auth_user`.`id` HAVING COUNT(`auth_user_groups`.`group_id`) > 0 ORDER BY NULL
When you run a 'DISTINCT' query against a database you end up with a listing of each distinct row in the data results. The reason that you have more 'DISTINCT' rows in your Django result is there is a combinatoric cross multiplication going on, creating extra results.
Other answers have mentioned all of this, but since you're asking the why:
The ORM, in this join, would probably allow you to pull fields attached to the group from the query. So if you wanted, say, all these users and all the groups and the group contact for some kind of massive weird mail merge, you could get them.
The post-processing brought on by DISTINCT is narrowing your results down according to the fields you have pulled rather than the rows in the query. If you were to use the PyCharm debugger or something, you might find that the groups aren't as easy to access using various ORM syntax when you have the distinct as when you don't.
I was wondering if there is a variable like :APP_USER, but instead of the username it should return the user group, so I can check if the current user is administrator.
The APEX_UTIL package contains some functions that can be useful for this too:
CURRENT_USER_IN_GROUP This function returns a Boolean result based on
whether the current user is a member of the specified group. You can
use the group name or group ID to identify the group.
GET_GROUPS_USER_BELONGS_TO This function returns a comma then a space
separated list of group names to which the named user is a member.
I found the solution myself:
SELECT group_name
FROM wwv_flow_group_users
WHERE user_id = (SELECT user_id
FROM wwv_flow_users
WHERE user_name ='MRITTMAN')
This lists all group names the user "MRITTMAN" is assigned to.
I want to execute a simple query like:
select *,count('id') from menu_permission group by menu_id
In Django format I have tried:
MenuPermission.objects.all().values('menu_id').annotate(Count('id))
It selects only menu_id. The executed query is:
SELECT `menu_permission`.`menu_id`, COUNT(`menu_permission`.`id`) AS `id__count` FROM `menu_permission` GROUP BY `menu_permission`.`menu_id`
But I need other fields also. If I try:
MenuPermission.objects.all().values('id','menu_id').annotate(Count('id))
It adds 'id' in group by condition.
GROUP BY `menu_permission`.`id`
As a result I am not getting the expected result. How I can get all all fields in the output but group by a single one?
You can try subqueries to do what you need.
In my case I have two tables: Item and Transaction where item_id links to Item
First, I prepare Transaction subquery with group by item_id where I sum all amount fields and mark item_id as pk for outer query.
per_item_total=Transaction.objects.values('item_id').annotate(total=Sum('amount')).filter(item_id=OuterRef('pk'))
Then I select all rows from item plus subquery result as total filed.
items_with_total=Item.objects.annotate(total=Subquery(per_item_total.values('total')))
This produces the following SQL:
SELECT `item`.`id`, {all other item fields},
(SELECT SUM(U0.`amount`) AS `total` FROM `transaction` U0
WHERE U0.`item_id` = `item`.`id` GROUP BY U0.`item_id` ORDER BY NULL) AS `total` FROM `item`
You are trying to achieve this SQL:
select *, count('id') from menu_permission group by menu_id
But normally SQL requires that when a group by clause is used you only include those column names in the select that you are grouping by. This is not a django matter, but that's how SQL group by works.
The rows are grouped by those columns so those columns can be included in select and other columns can be aggregated if you want them to into a value. You can't include other columns directly as they may have more than one value (since the rows are grouped).
For example if you have a column called "permission_code", you could ask for an array of the values in the "permission_code" column when the rows are grouped by menu_id.
Depending on the SQL flavor you are using, this could be in PostgreSQL something like this:
select menu_id, array_agg(permission_code), count(id) from menu_permissions group by menu_id
Similary django queryset can be constructed for this.
Hopefully this helps, but if needed please share more about what you need to do and what your data models are.
The only way currently that it works as expected is to hve your query based on the model you want the GROUP BY to be based on.
In your case it looks like you have a Menu model (menu_id field foreign key) so doing this would give you what you want and will allow getting other aggregate information from your MenuPermission model but will only group by the Menu.id field:
Menu.objects.annotate(perm_count=Count('menupermission__id')).values('perm_count')
Of course there is no need for the "annotate" intermediate step if all you want is that single count.
query = MenuPermission.objects.values('menu_id').annotate(menu_id_count=Count('menu_id'))
You can check your SQL query by print(query.query)
This solution doesn't work, all fields end up in the group by clause, leaving it here because it may still be useful to someone.
model_fields = queryset.model._meta.get_fields()
queryset = queryset.values('menu_id') \
.annotate(
count=Count('id'),
**{field.name: F(field.name) for field in model_fields}
)
What i'm doing is getting the list of fields of our model, and set up a dictionary with the field name as key and an F instance with the field name as a parameter.
When unpacked (the **) it gets interpreted as named arguments passed into the annotate function.
For example, if we had a "name" field on our model, this annotate call would end up being equal to this:
queryset = queryset.values('menu_id') \
.annotate(
count=Count('id'),
name=F("name")
)
you can use the following code:
MenuPermission.objects.values('menu_id').annotate(Count('id)).values('field1', 'field2', 'field3'...)
I want to somehow join a group with all my permissions.
I want to query ALL permissions, and for each permission query a boolean indicator if the group has it.
So assume this
group = Group.objects.get(pk=1) # specific group
Permission.objects.all().annotate( group has it ??)
group.permissions.all()
won't help since I want to query all permissions.
UPDATE:
Clear explanation:
Assume my Permission table is (pk values): 1, 2 ,3 - total three rows.
Group table: one group with pk=1.
Group-permission (many-to-many) table: group with pk 1, has permission 1,2 (so two rows)
I want to display all the permissions, with an indicator near them whether the group has it.
So in our case I should get :
1 True
2 True
3 False
Cause the group don't have permission with pk=3.
I think the below query would work for you
from django.db.models import Case, When, BooleanField
group_name = 'your group name'
Permission.objects.annotate(
has_perm=Max(Case(
When(group__name=group_name, then=1),
default=0,
output_field=BooleanField()
))
).values_list('name', 'has_perm').order_by('has_perm')
Conditional expressions were introduced in Django 1.8, and aggregation (annotate) is also well documented.