what's the benefits of KeyProperty ? if i can connect two entity using user_id and id
class User(ndb.Model):
name = ndb.StringProperty()
class Attend(ndb.Model):
user_id = ndb.IntegerProperty()
date = ndb.DateProperty()
User(id=1,name="xyz)
Attend(user_id = 1, date = "xyz")
There are a couple of minor benefits of storing a KeyProperty over an ID:
It is easier to do x.user_key.get() than ndb.Key(Attend, x.user_id).get()
The key is fully specified so there is no doubt about the entity it refers to but for the ID you also need to know the entity type.
These are both really minor so you can do either one. I sometimes store keys and sometimes store IDs based on other factors.
Related
Assuming the following example model:
# models.py
class event(models.Model):
location = models.CharField(max_length=10)
type = models.CharField(max_length=10)
date = models.DateTimeField()
attendance = models.IntegerField()
I want to get the attendance number for the latest date of each event location and type combination, using Django ORM. According to the Django Aggregation documentation, we can achieve something close to this, using values preceding the annotation.
... the original results are grouped according to the unique combinations of the fields specified in the values() clause. An annotation is then provided for each unique group; the annotation is computed over all members of the group.
So using the example model, we can write:
event.objects.values('location', 'type').annotate(latest_date=Max('date'))
which does indeed group events by location and type, but does not return the attendance field, which is the desired behavior.
Another approach I tried was to use distinct i.e.:
event.objects.distinct('location', 'type').annotate(latest_date=Max('date'))
but I get an error
NotImplementedError: annotate() + distinct(fields) is not implemented.
I found some answers which rely on database specific features of Django, but I would like to find a solution which is agnostic to the underlying relational database.
Alright, I think this one might actually work for you. It is based upon an assumption, which I think is correct.
When you create your model object, they should all be unique. It seems highly unlikely that that you would have two events on the same date, in the same location of the same type. So with that assumption, let's begin: (as a formatting note, class Names tend to start with capital letters to differentiate between classes and variables or instances.)
# First you get your desired events with your criteria.
results = Event.objects.values('location', 'type').annotate(latest_date=Max('date'))
# Make an empty 'list' to store the values you want.
results_list = []
# Then iterate through your 'results' looking up objects
# you want and populating the list.
for r in results:
result = Event.objects.get(location=r['location'], type=r['type'], date=r['latest_date'])
results_list.append(result)
# Now you have a list of objects that you can do whatever you want with.
You might have to look up the exact output of the Max(Date), but this should get you on the right path.
I have three models, let's take an imaginary example:
class Entity(models.Model):
name = models.CharField()
class EntityAssociation(models.Model):
buddy1 = models.ForeignKey(Entity, related_name='+')
buddy2 = models.ForeignKey(Entity, related_name='+')
class EntityPhoto(models.Model):
entity = models.ForeignKey(Entity, null=True)
association = models.ForeignKey(EntityAssociation, null=True)
title = ...
We have some people (Entity), that can share personal photos of themselves. We also have some relations between entities (represented by EntityAssociation) that can also share photos of them together.
For a single entity, I can retrieve all the photo associated to an entity, either directly or through an association, doing so:
obj = Entity.objects.last()
EntityPhoto.objects.filter(
Q(entity=obj) | Q(association__buddy1=obj) | Q(association__buddy2=obj)
)
What I want is being able to prefetch all the photos of a set of entities selected. A typical use-case would be:
for entity in Entity.objects.all().prefetch(???):
print(entity.name, 'has', len(entity.photos_prefetched), 'photos')
print([x.title for x in entity.photos_prefetched])
And this should be returning all the photos. A solution with three queries (Entity listing, prefetch through entity, prefetch through association attr ; two would be perfect) would satisfy me but the more important is to be able to iterate through a single list, on each entity
I tried to look the internal code of Prefetch but it looks like a prefetch is tied to a lookup, plus I don't know how to make the Q query in this case (what should be the right operand in Q(entity__in=...)?)
Notice: The point here is not about refactoring the database structure (EntityAssociation is used for a plenty of other things so it can't be reduced to a M2M of EntityPhoto for example.) but optimizing this specific use-case, if possible.
I'm currently playing a bit with Prefetch. In my opinion it's in a pretty good shape and quite powerful.
My approach to your problem would probably be something like:
entities = Entity.objects.all().prefetch(Prefetch(
'entity_photo__set',
EntityPhoto.objects.filter(
Q(entity=obj) | Q(association__buddy1=obj) | Q(association__buddy2=obj
)
to_attr="photos_prefetched",
))
You can then access those photos with for entity in entities: entity.photos_prefetched.
If this is not working, the problem might be that you're referencing back (Q(entity=obj)) to the actual entity. Not sure if this is properly possible. I'm having trouble with references back to the object, might be a bug in Django.
I have a database schema similar to this:
class User(models.Model):
… (Some fields irrelevant for this query)
class UserNotifiy(models.Model):
user = models.ForeignKey(User)
target = models.ForeignKey(<Some other Model>)
notification_level = models.SmallPositivIntegerField(choices=(1,2,3))
Now I want to query for all Users that have a UserNotify object for a specific target and at least a specific notification level (e.g. 2).
If I do something like this:
User.objects.filter(usernotify__target=desired_target,
usernotify__notification_level__gte=2)
I get all Users that have a UserNotify object for the specified target and at least one UserNotify object with a notification_level greater or equal to 2. These two UserNotify objects, however, do not have to be identical.
I am aware that I can do something like this:
user_ids = UserNotify.objects.filter(target=desired_target,
notification_level__gte=2).values_list('user_id', flat=True)
users = User.objects.filter(id__in=user_ids).distinct()
But this seems a step too much for me and I believe it executes two queries.
Is there a way to solve my problem with a single query?
Actually I don't see how you can run the first query, given that usernotify is not a valid field name for User.
You should start from UserNotify as you did in your second example:
UserNotify.objects.filter(
target=desired_target,
notification_level__gte=2
).select_related('user').values('user').distinct()
I've been looking for this behaviour but I've never found a better way than the one you describe (creating a query for user ids and inject it in a User query). Note this is not bad since if your database support subqueries, your code should fire only one request composed by a query and a subquery.
However, if you just need a particular field from the User objects (for example first_name), you may try
qs = (UserNotify.objects
.filter(target=desired_target, notification_level__gte=2)
.values_list('user_id', 'user__first_name')
.order_by('user_id')
.distinct('user_id')
)
I am not sure if I understood your question, but:
class User(models.Model):
… (Some fields irrelevant for this query)
class UserNotifiy(models.Model):
user = models.ForeignKey(User, related_name="notifications")
target = models.ForeignKey(<Some other Model>)
notification_level = models.SmallPositivIntegerField(choices=(1,2,3))
Then
users = User.objects.select_related('notifications').filter(notifications__target=desired_target,
notifications__notification_level__gte=2).distinct('id')
for user in users:
notifications = [x for x in user.notifications.all()]
I don't have my vagrant box handy now, but I believe this should work.
I'm currently creating a structure where I have employees which belong to a company.
Within this company I need to be able to create several groups. Ranks if you will. You could assign less permissions to lower ranks and more permissions to higher ranks.
I want to go for object level permissions and I noticed the django-guardian project gave me exactly what I needed. It works with the native User and Group objects so I'm now trying to find a way to implement the native group object in a company object.
Problems I face is that name in group is unique. So if 2 companies add the same group, errors will occur.
I found an implementation that works in a way but seems quite 'hacky' to me. In my company I declared a group variable that references Group:
class Company(models.Model):
...
groups = models.ManyToManyField(Group, through='CompanyRole')
CompanyRole basically houses the group name and a reference to company and group
class CompanyRole(models.Model):
group = models.ForeignKey(Group)
company = models.ForeignKey(Company)
real_name = models.CharField(max_length=60, verbose_name=_('Real name'))
objects = CompanyGroupManager()
I created a custom manager with a convenient method to add a new 'company group'
class CompanyGroupManager(models.Manager):
def create_group(self, company, group_name):
un_group_name = str(company.id) + '#' + group_name
group = Group.objects.create(name=un_group_name)
company_group = self.model(
real_name=group_name,
company=company,
group=group
)
company_group.save(using=self._db)
return company_group
Here's the part I don't really feel confortable about. In order to change the problem with the unique name on the Group model I used a combination of the company id, a hash sign and the actual group name to avoid clashes.
Now my question is: are there better methods in my scenario, am I missing something or is this a good way of accomplishing what I need?
Unfortunately there is no way of getting around the unique requirement, that is because this field is used as the id:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.Field.unique
Your options are the following:
1) Mocking the model.
You would basically just create a new Group model that doesn't have the unique requirement. The downside here is that you'd need to use it everywhere, so if this requires updating 3rd party apps, it might not be worth it.
2) make the name you unique. (As you did)
Make sure that you document your convention well, so that all future coders will know what they are looking at.Something like "company name"#"group name" could make more intuitive sense than an id. If the a hash might appear in either then use a more certain delimiter ("__" is a relatively common way of connecting related concepts in django, I might go for this).
I would recommend that you add the following to make it easy for you to access the name.
def get_name(self):
# Explain how you get the group name from your uniqueified name
return self.name.split('#')[1]
Group.add_to_class('get_name', get_name)
When you access your group's name in your app, just do:
my_group.get_name()
You might also want to put the generating the uniqueified name into an overridden version of the save(). This would give you nicer split between model and view...
I am quite new to google app engine. I know google datastore is not sql, but I am trying to get many to many relationship behaviour in it. As you can see below, I have Gif entities and Tag entities. I want my application to search Gif entities by related tag. Here is what I have done;
class Gif(ndb.Model):
author = ndb.UserProperty()
link = ndb.StringProperty(indexed=False)
class Tag(ndb.Model):
name = ndb.StringProperty()
class TagGifPair(ndb.Model):
tag_id = ndb.IntegerProperty()
gif_id = ndb.IntegerProperty()
#classmethod
def search_gif_by_tag(cls, tag_name)
query = cls.query(name=tag_name)
# I am stuck here ...
Is this a correct start to do this? If so, how can I finish it. If not, how to do it?
You can use repeated properties https://developers.google.com/appengine/docs/python/ndb/properties#repeated the sample in the link uses tags with entity as sample but for your exact use case will be like:
class Gif(ndb.Model):
author = ndb.UserProperty()
link = ndb.StringProperty(indexed=False)
# you store array of tag keys here you can also just make this
# StringProperty(repeated=True)
tag = ndb.KeyProperty(repeated=True)
#classmethod
def get_by_tag(cls, tag_name):
# a query to a repeated property works the same as if it was a single value
return cls.query(cls.tag == ndb.Key(Tag, tag_name)).fetch()
# we will put the tag_name as its key.id()
# you only really need this if you wanna keep records of your tags
# you can simply keep the tags as string too
class Tag(ndb.Model):
gif_count = ndb.IntegerProperty(indexed=False)
Maybe you want to use list? I would do something like this if you only need to search gif by tags. I'm using db since I'm not familiar with ndb.
class Gif(db.Model):
author = db.UserProperty()
link = db.StringProperty(indexed=False)
tags = db.StringListProperty(indexed=True)
Query like this
Gif.all().filter('tags =', tag).fetch(1000)
There's different ways of doing many-to-many relationships. Using ListProperties is one way. The limitation to keep in mind if using ListProperties is that there's a limit to the number of indexes per entity, and a limit to the total entity size. This means that there's a limit to the number of entities in the list (depending on whether you hit the index count or entity size first). See the bottom of this page: https://developers.google.com/appengine/docs/python/datastore/overview
If you believe the number of references will work within this limit, this is a good way to go. Considering that you're not going to have thousands of admins for a Page, this is probably the right way.
The other way is to have an intermediate entity that has reference properties to both sides of your many-to-many. This method will let you scale much higher, but because of all the extra entity writes and reads, this is much more expensive.