I have a couple of related models, that look a bit like this:
class Book(models.Model):
title = models.TextField()
class Author(models.Model):
"""
>>> b = Book(title='some title')
>>> b.save()
>>> a = Author(name='some name')
>>> a.save()
>>> a.books.add(b)
>>> b in a.books.all()
True
"""
name = models.TextField()
books = models.ManyToManyField(Book)
This version is a simplification of my production app, but the same test in production fails - a.books.all() returns the empty list, even after I do an a.books.add(b).
I've looked in the database (sqlite), and a new entry has definitely been created in the joining table, book_author. I've also tried calling a transaction.commit(), and a connection.close() to try and refresh the view of the DB. No joy. Any pointers towards what sorts of things might be causing the strange behaviour in production would be gratefully received.
One thought I had was that it had something to do with a third related model, for which I've manually specified a through table, something like this:
class Genre(models.Model):
desc = models.TextField()
books = models.ManyToManyField(Book, through='BookGenres')
class BookGenres(models.Model):
book = models.ForeignKey(Book)
genre = models.ForeignKey(Genre)
However, adding this to the test app doesn't break things... What else should I look for?
[edit 11/5] more weird behaviour, following on from advice from Daniel the comments (thanks for trying! :-)
More strange behaviour:
>>>a.books.all()
[]
>>>a.books.filter(pk=b.id)
[]
>>>a.books.filter(pk=b.id).count()
1
>>>len(a.books.filter(pk=b.id))
0
As I say, my "real" models are more complex, and I haven't been able to replicate this behaviour in simplified tests, but any ideas for what to look at would be gratefully appreciated.
I'm not sure that the in operator necessarily works in querysets. Don't forget that Django model instances don't have identity, so two objects loaded from the db in two separate operations can have different internal IDs even if they have the same pk.
I would explicitly query the item you're expecting :
>>> a.books.add(b)
>>> a.books.filter(pk=b.pk).count()
1
I'd also add that I don't see the point of this test. Django's model operations are very well covered by its own test suite - you should reserve your unit/doctests for your own business logic.
Related
I'm getting rather tired of paging through lots of irrelevant little fiddly properties while looking for the actual database structure of my models. Would it be a bad thing to use proxy models universally just to keep my code better organized / more readable? I.e.
class Foo_Base( models.Model):
title = models.CharField( ...)
# other DB fields. As little as possible anything else.
class Bar_Base( models.Model):
foo = models.ForeignKey( Foo_Base, ... )
etc. not many more lines than there are columns in the DB tables. Then at the bottom or elsewhere,
class Foo( Foo_Base):
class Meta:
proxy=True
#property
def some_pseudo_field(self):
# compute something based on the DB fields in Foo_Base
return result
#property
# etc. pages of etc.
The fact that makemigrations and migrate tracks proxy models makes me slightly concerned, although this usage seems to be exactly what the Django documentation says they are for (wrapping extra functionality around the same database table).
Or is there another way to organize my code that accomplishes the same (keeping fundamental stuff and fiddly little support bits apart).
[Edit] am offering up something that seems to work as a self-answer below. I'd still very much like to hear from anybody who knows for a fact that this is OK, given the deep Django magic on its declarative field declarations.
(About the only thing I dislike about Python, is that it does not have include functionality for reading in a heap of code from another file! )
I think I may have found an answer: use a plugin class inheriting from object,
as is commonplace for class-based Views.
I'd still very much like to hear from anybody who knows for a fact that this is OK, given the deep Django magic on its declarative field declarations.
Minimal proof of concept:
class PenMethods1( object):
#property
def namey(self):
return format_html('<div class="namey">{name}</div>', name=self.name )
class PenTest1(PenMethods1, models.Model):
name = models.CharField( max_length=16, blank=True )
def __repr__(self):
return f'<Pentest1 object id={self.id} name={self.name}>'
Initial migration was OK. Then I added
pnum = models.ForeignKey( 'Pennum', models.SET_NULL, null=True)
(Pennum was something already lying around in my playpen) and ran makemigrations and migrate. Again OK and basic functioning checks out...
>>> from playpen.models import PenTest1, Pennum
>>> n = Pennum.objects.last()
>>> n
<Pennum object id=3 name="t3" num=14 >
>>> p = PenTest1.objects.get(name='bar')
>>> p
<Pentest1 object id=2 name=bar>
>>> p.namey
'<div class="namey">bar</div>'
>>> p.pnum=n
>>> p.save()
>>> n=Pennum.objects.last()
>>> list(n.pentest1_set.all())
[<Pentest1 object id=2 name=bar>]
>>>
I have searched everywhere but I cannot find the solution for this problem, which in a first look should be trivial.
Please consider the Djano Model:
class UserModel(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
institution=models.CharField(max_length=128)
class TeamModel(models.Model):
creator= models.ForeignKey(UserModel,related_name='creator')
members= models.ManyToManyField(UserModel,related_name='teams'))
name=models.CharField(max_length=64)`
Those models are fixed, I cannot change them. I can of course include new fields.
To retrieve the teams associated with a user, I can simply do:
queryset= instance_UserModel.teams.all()
It works perfectly! Now I want to order the teams by the number of members.
queryset= queryset.annotate(num_members=Count('members')).order_by('num_members')
It does not work, because num_members value is always "1"!
print queryset[0].num_members
print queryset[1].num_members
returns always "1". What is happening???!!!
Thank you in advance,
CBar
This was not tested
class TeamModel(models.Model):
creator= models.ForeignKey(UserModel,related_name='creator')
members= models.ManyToManyField(UserModel,related_name='teams'))
name=models.CharField(max_length=64)`
views.py
import operator
teams = TeamModel.objects.all()
team_member_count = {}
for team in teams:
team_member_count[team.name] = team.members.all.count()
sorted_teams_by_member_count = sorted(team_member_count.items(), key=operator.itemgetter(1))
I found a solution, but I don't know how efficient it is regarding database operations. And I still do not understand why my previous code does not work.
qUserTeams=instance_UserModel.teams.all()
qAllTeams=TeamModel.objects.all().annotate(num_members=Count('members')).order_by('num_members')
queryset= qAllTeams & qUserTeams
Instead of dynamically altering a models file by adding fields, very bad i've been told, i'm suppose to maintain a type of flexibility by having variable field list names(i think).
Thus, when an attribute is added to the database, this attribute can be accessed without the models file being altered.
I cant figure out how to create variable field list names in my models class though.
I'm having trouble sifting through reading materials to find a solution to my problem, and trial and era is 15hrs and counting.
Could some one point me in the right direction.
New Edit
Heres what im trying to achieve.
When an attribute is added, i add it to the table like this.
c = 'newattributename'
conn = mdb.connect('localhost', 'jamie', '########', 'website')
cursor = conn.cursor()
cursor.execute("alter table mysite_weightsprofile add column %s integer not null; SET #rank=0; UPDATE mysite_weightsprofile SET %s = #rank:=#rank+1 order by %s DESC;" % (c, c, a))
cursor.close()
conn.close()
Now, in my models class i have
class WeightsProfile(models.Model):
1attributes = models.IntegerField()
2attributes = models.IntegerField()
3attributes = models.IntegerField()
class UserProfile(WeightsProfile):
user = models.ForeignKey(User, unique=True)
aattributes = models.CharField()
battributes = models.CharField()
cattributes = models.CharField()
Now all i want to do is get access to the new attribute that was added in the table but not added to in the models file.
Does sberry2A have the right answer. I hope it is, it seems the simplest.
I might not be following what you are asking, but assuming you have some model, like Person, which will start out having some defined fields, but may have several more added in the future...
class Person(models.Model):
fname = models.CharField(max_length=255)
lname = models.CharField(max_length=255)
age = models.IntegerField()
# more fields to come
Then you could use a PersonAttribute model...
class PersonAttribute(models.Model):
name = models.CharField(max_length=32)
value = models.CharField(max_length=255)
Then you could just add a ManyToMany relationship field to your Person...
attributes = models.ManyToManyField(PersonAttribute)
Or something similar.
I don't really understand what it is you're trying to do, but South is a good system for handling changes to models. It makes migrations, so that it understands the changes you've made and knows how to change them in the database in a way that you can use for both development sites and production.
I don't understand what you're after either, JT, but I really doubt South (see #Dougal) is going to help you if what you want boils down to "Look at the relevant DB table to know what fields the model should have at read time. But not write time.". South is brilliant for evolving schemas/models, but not at runtime, and not inconsistently across rows/instances of models. And hacking models at runtime is definitely a world of hurt.
Indeed, Django's ORM isn't built for dynamic fields (at least for now) - it was built to abstract writing SQL and speed up dev against an RDBMS, not schemaless/NoSQL stuff.
Speaking of which, if someone landed me with a spec that effectively said "We don't know what fields the model will have to store" I'd suggest we try MongoDB for that data (alongside Postgres for trad relational data), probably via MongoEngine
I am trying to add an additional custom field to a django model. I have been having quite a hard time figuring out how to do the following, and I will be awarding a 150pt bounty for the first fully correct answer when it becomes available (after it is available -- see as a reference Improving Python/django view code).
I have the following model, with a custom def that returns a video count for each user --
class UserProfile(models.Model):
user = models.ForeignKey(User, unique=True)
positions = models.ManyToManyField('Position', through ='PositionTimestamp', blank=True)
def count(self):
from django.db import connection
cursor = connection.cursor()
cursor.execute(
"""SELECT (
SELECT COUNT(*)
FROM videos_video v
WHERE v.uploaded_by_id = p.id
OR EXISTS (
SELECT NULL
FROM videos_videocredit c
WHERE c.video_id = v.id
AND c.profile_id = p.id
)
) AS Total_credits
FROM userprofile_userprofile p
WHERE p.id = %d"""%(int(self.pk))
)
return int(cursor.fetchone()[0])
I want to be able to order by the count, i.e., UserProfile.objects.order_by('count'). Of course, I can't do that, which is why I'm asking this question.
Previously, I tried adding a custom model Manager, but the problem with that was I also need to be able to filter by various criteria of the UserProfile model: Specifically, I need to be able to do: UserProfile.objects.filter(positions=x).order_by('count'). In addition, I need to stay in the ORM (cannot have a raw sql output) and I do not want to put the filtering logic into the SQL, because there are various filters, and would require several statements.
How exactly would I do this? Thank you.
My reaction is that you're trying to take a bigger bite than you can chew. Break it into bite size pieces by giving yourself more primitives to work with.
You want to create these two pieces separately so you can call on them:
Does this user get credit for this video? return boolean
For how many videos does this user get credit? return int
Then use a combination of #property, model managers, querysets, and methods that make it easiest to express what you need.
For example you might attach the "credit" to the video model taking a user parameter, or the user model taking a video parameter, or a "credit" manager on users which adds a count of videos for which they have credit.
It's not trivial, but shouldn't be too tricky if you work for it.
"couldn't you use something like the "extra" queryset modifier?"
see the docs
I didn't put this in an answer at first because I wasn't sure it would actually work or if it was what you needed - it was more like a nudge in the (hopefully) right direction.
in the docs on that page there is an example
query
Blog.objects.extra(
select={
'entry_count': 'SELECT COUNT(*) FROM blog_entry WHERE blog_entry.blog_id = blog_blog.id'
},
)
resulting sql
SELECT blog_blog.*, (SELECT COUNT(*) FROM blog_entry WHERE blog_entry.blog_id = blog_blog.id) AS entry_count
FROM blog_blog;
Perhaps doing something like that and accessing the user id which you currently have as p.id as appname_userprofile.id
note:
Im just winging it so try to play around a bit.
perhaps use the shell to output the query as sql and see what you are getting.
models:
class Positions(models.Model):
x = models.IntegerField()
class Meta:
db_table = 'xtest_positions'
class UserProfile(models.Model):
user = models.ForeignKey(User, unique=True)
positions = models.ManyToManyField(Positions)
class Meta:
db_table = 'xtest_users'
class Video(models.Model):
usr = models.ForeignKey(UserProfile)
views = models.IntegerField()
class Meta:
db_table = 'xtest_video'
result:
test = UserProfile.objects.annotate(video_views=Sum('video__views')).order_by('video_views')
for t in test:
print t.video_views
doc: https://docs.djangoproject.com/en/dev/topics/db/aggregation/
This is either what you want, or I've completely misunderstood!.. Anywhoo... Hope it helps!
If you have some models:
class Teacher(models.Model):
name = models.CharField(max_length=50)
class Student(models.Model):
age = models.PositiveIntegerField()
teacher = models.ForeignKey(Teacher, related_name='students')
and you use it like this:
>>> student = Student.objects.get(pk=1)
>>> student.teacher.name # This hits the database
'Some Teacher'
>>> student.teacher.name # This doesn't (``teacher`` is cached on the object)
'Some Teacher'
That's awesome. Django caches the related object so that you can use it again without having to abuse your database.
But, if you use it like this:
>>> teacher = Teacher.objects.get(pk=1)
>>> for student in teacher.students.all(): # This hits the database
... print(student.age)
...
8
6
>>> for student in teacher.students.all(): # This does too (obviously)
... print(student.age)
...
8
6
There's no caching or efficient access to related objects this direction.
My question is thus: Is there a built-in (or non-problematic way) to backward access related objects in an efficient way (a cached way), like you can in the student.teacher example above?
The reason I want this is because I have a model with multiple methods that need access to the same related objects over and over, so a page that should have 12 queries ends up with about 30.
There isn't any built-in way. I've written about this issue on my blog, with a technique to optimise accessing reverse relationships.
orokusaki,
You just need to cache the queryset as a python variable
students = teacher.students.all()
And then just use students in your for loops.
Below is a link to Django's own documentation about this specific issue :-)
http://docs.djangoproject.com/en/1.1/topics/db/optimization/#understand-cached-attributes
Try this perhaps?
teacher = Teacher.objects.select_related().get(pk=1)
http://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.QuerySet.select_related
I have never used select_related in concert with a .all() on its result, so I'm not sure if it will yield DB savings or not.