Take the example in document
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50), nullable=False)
addresses = db.relationship('Address', lazy='dynamic',
backref=db.backref('person', lazy='select'))
We can only set the lazy parameter to relationship when we build the models. But I found no document show how to change it after instantiated it.
In most of my situation like
user = User.query.first()
addresses = user.addresses.filter(is_del=0).all()
which is One-to-Many or
wallet = user.wallet
which is One-to-One Model, I just set the lazy to dynamic or select to let the model only get data what I need.
However recently I want to export the data in database using Admin Front-end page.
user_list = UserModel.query.all()
for x in user_list:
item = {
"ID": x.id if x.id else ''
}
if 'basic' in fields or is_all_field == 1:
ex_item = {
"Email": x.base.account,
"Full Name": x.base.full_name,
"Display Name": x.display_name if x.display_name else '',
"Phone Number": x.base.phone_number if x.base.phone_number else '',
"Status": status_map(x.is_sharer,x.is_block,x.status_id)
"Register Time": x.create_date.strftime('%Y-%m-%d %H:%M:%S') if x.create_date else ''
}
item = {**item, **ex_item}
if ......
........
If I continue to use select and dynamic as lazy. It will be very very slow cause every loop the parent query will access database every time when it use subquery.
I test the speed between select and joined using single field like "Full Name": x.base.full_name to export all user data. Select got 53s and joined got 0.02s.
Is there a way to change the lazy parameter based on not changing the original Model?
According to the documentation you can use options to define the type of loading you want. I believe it will override your default value defined in your relationship
Useful links
Joined Loads
Select Loads
Lazy Load
So, basically, if you are using lazy=' select', lazy load, and want to switch to joinedload to optimize your query, use the following:
from sqlalchemy.orm import joinedload # import the joinedload
user_list = db.session.query(UserModel).options(joinedload("person")) # running a joinedload which will give you access to all attributes in parent and child class
for x in user_list:
# now use x.account directly
Related
New to Django and I wanted to append a key:value pair to each Job object in a Queryset/list before passing it to a template. Researching on here it says you can't just append a field, but rather need to create a new dict or could possibly add attributes or something, but messing around I got it to work great, but logically it seems it shouldn't work, and I worry I may run into problems or data inconsistency somewhere.
My "jobs" view gets all Jobs currently not assigned to a User, filters out Jobs based on the current signed in Users listed skillset(a custom field for my User model), and then the functionatily I'm adding is to check the current Users schedule, and add a value "conflict":True if the schedule conflicts so I can highlight it red when rendering the list of jobs to the screen.
views.py (abbreviated):
def jobs (request):
//get all unassigned jobs, and create a list of only jobs where the user has matching skills
available = Job.objects.filter(assigned_to__isnull=True).order_by('start_time').all()
available = [job for job in available if request.user.is_skilled(job)]
//go through each job, make it a dict, and add "conflict":True to it if scheudle confict
for job in available:
if not request.user.is_available(job):
job = job.__dict__
job["conflict"] = True
//run custom serializer() to format each field and return list of jobs
return JsonResponse({'available': [job.serialize() for job in available]})
The part that doesn't make sense to me is job.__dict__ should be converting the job object to a dict, so I actually expected to get an error when I attempt to run job.serialize(). Infact, there is another view where I attempt the same thing with a single Job object(not a list or Queryset) and it gives error Dict does not contain method serialize(). And if I don't first convert to dict I get an error TypeError: 'Job' object does not support item assignment. But somehow when working with a list, the conversion doesn't occur, but my Job object now has a new field "conflict" that I can check with job.conflict in JS, templates, and in the serialize() model method:
models.py (abbreviated):
class Job (models.Model):
title = models.CharField(max_length=50)
description = models.CharField(max_length=500)
start_time = models.DateTimeField()
end_time = models.DateTimeField()
skillsets = models.ManyToManyField(Skillset, related_name="jobs")
def serialize(self):
return {
'id': self.id,
'title': self.title,
'description': self.description,
'start_time': self.start_time.strftime("%b %d %Y, %I:%M %p"),
'end_time': self.end_time.strftime("%b %d %Y, %I:%M %p"),
'skillsets': [skill.title for skill in self.skillsets.all()],
'conflict': self.conflict if 'conflict' in self.__dict__ else False,
}
So question is, why does available contain a list of only Job objects after I should have converted some of them to Dict? I do this again on the User's object later to highlight users who are available for a particular Job, and same thing, even though I supposedly convert to Dict and add a field, the final output is a list of User objects with an added field available. I was expecting assignable to be a list of dicts, not a list of user objects:
Also in views.py:
def job_details (request, job_id):
//get a single Job, and list of users who are specialist, then filter out users who don't have required skills
job = Job.objects.get(pk=job_id)
users = User.objects.filter(user_type=User.SPECIALIST)
users = list(filter(lambda usr: usr.is_skilled(job), users))
//for each user, check their schedule against this job, and add "available":True/False as determined
for usr in users:
if (usr.is_available(job)):
usr = usr.__dict__
usr["available"] = True
else:
usr = usr.__dict__
usr["available"] = False
//return job and users to be rendered on screen, colored based on their availability
return render(request, "jobs/job_details.html", {
'job': job.serialize(),
'assignable': users,
'conflict': True if request.user.user_type == User.SPECIALIST and not request.user.is_available(job) else False})
Is there a logical explaination? Is it because I'm doing it in a for loop that the conversion doesn't stick, but the field gets added to the User/Job objects, like is it converting back after it exits the for loop?
To address your immediate question - the statement usr = usr.__dict__ done when iterating over the list simply assigns a variable usr. It does not change the element in the original list. To transform the list of model instances to dicts, you can use map function, or list comprehension. For example:
users = list(map(lambda usr: {**usr.__dict__, "available": usr.is_available(job)}, users))
BTW, probably slightly better approach would be to set "available" attribute right on the model instance, and then access it in serialize method via getattr: getattr(self, "available", False).
Or, you can take it a step further and add available as a python property to model class. Then you will be able to document its meaning and usage:
class User(models.Model):
...
#property
def available(self):
"""Document the meaning..."""
return getattr(self, "_available", False)
#available.setter
def available(self, value):
self._available = value
I am trying to implement filtering in one of my data APIs built on Django(non-rel, mongodb-engine) + Tastypie(non-rel). I want to implement filtering on a sub-document inside my main document. The main documents in mongo looks like
{"user_name": "abc", "email": "abc#domain.com", "safety" :{"intra": True, "inter": False, "external": ['a', 'b']}}
The sub-document contains 2 boolean fields and list field. In mongo I can easily query on sub-documents and their fields however I am not able to implement it at the API level.The django model looks like this
from djangotoolbox.fields import DictField, EmbeddedModelField
class UserSafety(models.Model):
user_name = models.CharField()
email = models.EmailField()
safety = DictField() <------- What to use here!
The corresponding Tasty-pie resource looks like
from tastypie_nonrel.fields import DictField, EmbeddedModelField
class UserSafetyResource(ModelResource):
user_name = field.CharField(attribute='user_name')
email = fields.CharField(attribute='email')
safety = DictField(attribute='safety') <----What to use here!
# This part i want to add
class Meta:
filtering = ['safety.intra': ALL, 'safety.inter': ALL]
Eventually I would like it to work this like in an HTTP request
GET http://<server>/my_endpoint/usersafety?safety.intra=true
Any ideas how this can be achieved? May be using either the embedded type fields?
I have a class called ToggleProperty. I use it to store information about whether a use toggled some properties on an object. Examples of properties are "like", "bookmark" and "follow".
class ToggleProperty(models.Model):
# "like", "bookmark", "follow" etc
property_type = CharField()
# The user who toggled the property
user = ForeignKey(User)
# The object upon which the user is toggling the property, e.g. "user likes image"
object_id = models.TextField()
content_type = models.ForeignKey(ContentType)
content_object = generic.GenericForeignKey('content_type', 'object_id')
Now, I'd like to get a list of users that are followed by a certain other user, let's call him Tom.
I can't just query on ToggleProperty, because that would give me ToggleProperties, not Users!
So I do this:
# First get the ContentType for user, we'll need it
user_ct = ContentType.objects.get_for_model(User)
# Now get the users that Tom follows
followed_by_tom = [
user_ct.get_object_for_this_type(id = x.object_id) for x in
ToggleProperty.objects.filter(
property_type = "follow",
user = tom,
content_type = ContentType.objects.get_for_model(User))
]
The problem with this is that it hits the database in my view, and I don't like that.
If this wasn't ugly enough, hear me out. I'm actually interested in the images uploaded by the users that Tom follows, so I can show Tom all the images by the people he follows.
So to the code above, I add this:
images = Image.objects.filter(user__in = followed_by_tom)
This ends up performing over 400 queries, and taking over a second to process. There has to be a better way, could you please show me the path?
This piece:
followed_by_tom = [
user_ct.get_object_for_this_type(id = x.object_id) for x in
ToggleProperty.objects.filter(
property_type = "follow",
user = tom,
content_type = ContentType.objects.get_for_model(User))
]
gets all of the User instances which are followed in N queries when it could be done in a single query. In fact you don't need the instances themselves only the ids to get the Images with the IN query. So you can remove the extra queries in the loop via:
followed_by_tom = ToggleProperty.objects.filter(
property_type="follow",
user=tom,
content_type=ContentType.objects.get_for_model(User)
).values_list('object_id', flat=True)
images = Image.objects.filter(user__in=followed_by_tom)
Since followed_by_tom is never evaluated the ORM should execute this as a single query with a sub-select.
I'm trying to query an ndb Model with a computed property, but it's returning an empty list. This answer suggests that I should be able to query computed properties and so do the docs. What am I doing wrong?
from django.template import defaultfilters
class Video(models.SfxModel):
title = ndb.StringProperty()
slug = ndb.ComputedProperty(
lambda self: str(defaultfilters.slugify(self.title)) )
In Interactive Console
from app.lib.videos import Video
slug = Video.query().get().slug
print slug
# => "some-dasherized-string"
print Video.query(Video.slug == slug).fetch()
# => []
the 'issue' you are having is the eventual consistency given for non ancestor queries.
what you are seeing is completely normal for the high replication datastore.
when you put an entity and query for it right after it could be that its not replicated over all datacenters so it could not be found.
if you want this to work you have to use entity groups by adding a parent to an entity.
this can be an entity key or a constructed key that does not belong to any stored entity.
this works:
class Video(ndb.Model):
title = ndb.StringProperty()
slug = ndb.ComputedProperty(lambda self: self.title.replace(' ', '-'))
v = Video(parent = ndb.Key(Video, 'xxx'), title = 'foo bar')
v.put()
print Video.query(Video.slug == v.slug, ancestor = ndb.Key(Video, 'xxx')).get()
Is there an easy way to fetch the ManyToMany objects from a query that returns more than one object? The way I am doing it now doesn't feel as sexy as I would like it to. Here is how I am doing it now in my view:
contacts = Contact.objects.all()
# Use Custom Manager Method to Fetch Each Contacts Phone Numbers
contacts = PhoneNumber.objects.inject(contacts)
My Models:
class PhoneNumber(models.Model):
number = models.CharField()
type = models.CharField()
# My Custom Manager
objects = PhoneNumberManager()
class Contact(models.Model):
name = models.CharField()
numbers = models.ManyToManyField(PhoneNumber, through='ContactPhoneNumbers')
class ContactPhoneNumbers(models.Model):
number = models.ForeignKey(PhoneNumber)
contact = models.ForeignKey(Contact)
ext = models.CharField()
My Custom Manager:
class PhoneNumberManager(models.Manager):
def inject(self, contacts):
contact_ids = ','.join([str(item.id) for item in contacts])
cursor = connection.cursor()
cursor.execute("""
SELECT l.contact_id, l.ext, p.number, p.type
FROM svcontact_contactphonenumbers l, svcontact_phonenumber p
WHERE p.id = l.number_id AND l.contact_id IN(%s)
""" % contact_ids)
result = {}
for row in cursor.fetchall():
id = str(row[0])
if not id in result:
result[id] = []
result[id].append({
'ext': row[1],
'number': row[2],
'type': row[3]
})
for contact in contacts:
id = str(contact.id)
if id in result:
contact.phonenumbers = result[id]
return contacts
There are a couple things you can do to find sexiness here :-)
Django does not have any OOTB way to inject the properties of the through table into your Contact instance. A M2M table with extra data is a SQL concept, so Django wouldn't try to fight the relations, nor guess what should happen in the event of namespace collision, etc... . In fact, I'd go so far as to say that you probably do not want to inject arbitrary model properties onto your Contact object... if you find yourself needing to do that, then it's probably a sign you should revise your model definition.
Instead, Django provides convenient ways to access the relation seamlessly, both in queries and for data retrieval, all the while preserving the integrity of the entities. In this case, you'll find that your Contact object offers a contactphonenumbers_set property that you can use to access the through data:
>>> c = Contact.objects.get(id=1)
>>> c.contactphonenumbers_set.all()
# Would produce a list of ContactPhoneNumbers objects for that contact
This means, in your case, to iterate of all contact phone numbers (for example) you would:
for contact in Contact.objects.all():
for phone in contact.contactphonenumbers_set.all():
print phone.number.number, phone.number.type, phone.ext
If you really, really, really want to do the injection for some reason, you'll see you can do that using the 3-line code sample immediately above: just change the print statements into assignment statements.
On a separate note, just for future reference, you could have written your inject function without SQL statements. In Django, the through table is itself a model, so you can query it directly:
def inject(self, contacts):
contact_phone_numbers = ContactPhoneNumbers.objects.\
filter(contact__in=contacts)
# And then do the result construction...
# - use contact_phone_number.number.phone to get the phone and ext
# - use contact_phone_number.contact to get the contact instance