I try to do query in sqlalchemy to get with self-referential relationship which is filtered on parent and also child level.
category_country = Table('category_country', Base.metadata,
Column('category_id', Integer, ForeignKey('category.id'), primary_key=True),
Column('country_id', Integer, ForeignKey('country.id'), primary_key=True)
)
class Category(Base):
__tablename__ = "category"
id = Column(Integer, primary_key=True, autoincrement=True)
parent_id = Column(Integer, ForeignKey('category.id'))
subcategories = relationship("Category", backref=backref('parent', remote_side=id))
countries = relationship(Country, secondary = category_country, backref='categories')
class Country(Base):
__tablename__ = "country"
id = Column(Integer, primary_key=True)
query
category = s.query(Category).join(Category.countries).options(contains_eager(Category.countries)).filter(Country.id == 1).filter(Category.id == category_id).join(Category.countries, aliased=True).join(Category.subcategories, aliased=True).options(contains_eager(Category.countries)).filter(Country.id == 1).first()
but it doesn't work. I need to find children which are from country 1 and its parent is category_id and country is also 1
I don't completely get your model/code on my first read, but the way I would tackle this is by splitting the self-referential join into a subquery() statement like this:
filter_by_country = (db.session.query(...)
.filter(...)
.subquery())
final_results = (db.session.query(...)
.join(filter_by_country,
db.and_(Category.id == filter_by_country.c.id, ..., ...))
.options(...)
.filter(...)
.etc(...).first())
I've found this sort of a pattern can help simplify these type of queries. Hope this helps.
Related
I have the classes Shift and Staff that are related in three ways. A Shift object can have a staff1, staff2, and a list of standby staff that is managed with a secondary table. Here are the models.
class Shift(db.Model):
__tablename__ = 'shifts'
shift_id = db.Column(db.Integer, primary_key=True)
staff1_id = db.Column(db.Integer, db.ForeignKey('staff.staff_id'))
staff1 = db.relationship('Staff', foreign_keys=[staff1_id])
staff2_id = db.Column(db.Integer, db.ForeignKey('staff.staff_id'))
staff2 = db.relationship('Staff', foreign_keys=[staff2_id])
standby = db.relationship('Staff', secondary='standby_staff')
class Staff(db.Model):
__tablename__ = 'staff'
staff_id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20))
standby_staff = db.Table('standby_staff', db.metadata,
db.Column('shift_id', db.Integer, db.ForeignKey('shifts.shift_id')),
db.Column('staff_id', db.Integer, db.ForeignKey('staff.staff_id')))
I want to be able to query for all the shifts that a staff member is associated with, either as staff1, staff2, or on standby. I wrote the following SQLAlchemy query:
shifts = Shift.query.filter((Shift.staff1 == staff) |
(Shift.staff2 == staff) |
(Shift.standby.contains(staff))).all()
The SQL query produced is:
SELECT shifts.shift_id AS shifts_shift_id, shifts.staff1_id AS shifts_staff1_id, shifts.staff2_id AS shifts_staff2_id
FROM shifts, standby_staff AS standby_staff_1
WHERE ? = shifts.staff1_id OR ? = shifts.staff2_id OR shifts.shift_id = standby_staff_1.shift_id AND ? = standby_staff_1.staff_id
Here's the problem. If the staff member is not present in the secondary table (never scheduled as standby staff), then the query returns zero results. As soon as the staff member is in the secondary table, all the expected results are returned.
What's going on here? Shouldn't the fact that the conditions are being linked with OR not necessitate the staff member being in standby? What am I doing wrong?
---EDIT---
To be sure this problem wasn't being caused by logical operators being evaluated in an unexpected order, I added self_group() to the end of the SQLAlchemy query so that the SQL query would contain parentheses around the last two conditions:
def query(staff_id):
staff = Staff.query.get(staff_id)
shifts = Shift.query.filter((Shift.staff1 == staff) |
(Shift.staff2 == staff) |
(Shift.standby.contains(staff)).self_group())
SELECT shifts.shift_id AS shifts_shift_id, shifts.staff1_id AS shifts_staff1_id, shifts.staff2_id AS shifts_staff2_id
FROM shifts, standby_staff AS standby_staff_1
WHERE ? = shifts.staff1_id OR ? = shifts.staff2_id OR (shifts.shift_id = standby_staff_1.shift_id AND ? = standby_staff_1.staff_id)
Let's say I have two models:
class Testmodel1():
amount = models.IntegerField(null=True)
contact = models.ForeignKey(Testmodel2)
entry_time = models.DateTimeField()
stage = choicesfiled
class Testmodel2():
name = models.CharField()
mobile_no = models.CharField()
I want to find out the object of Testmodel1 for contact > 3 which is created in the last 24 hours last = arrow.utcnow().shift(hours=-24).date().
I am applying a query:
n1=Testmodel1.objects.filter(entry_time__gte=last, stage=1).annotate(t_count=Count('contact')).filter(t_count__gt=3)
But it seems it's not working. Because I am getting an empty queryset.
Any help would be appreciated.
Only a partial answer. Sorry! Your code looks fine to me, so I'm just trying to find a solution by approaching it from a different direction.
Here's how I structure (sort of) similar code on one of my projects.
from datetime import timedelta, date
....
base_date = date.today()
start_date = base_date + timedelta(days=30)
end_date = base_date
possible_holidays = Holiday.objects.filter(
start_date__lte=start_date, end_date__gte=end_date)
From there, could you just do something like:
if possible_holidays.contact_set.count() > 3:
pass
Does that work?
The problem is your Many-to-One relationship is inverted. This relationship is a parent-child relationship, where a parent can have multiple children, but a children can only have one parent. In database this relationship is stored as a child's ForeignKey field that points to the child's parent.
In your case Testmodel1 is a parent and Testmodel2 is a child (Testmodel1 can have multiple contacts represented by Testmodel2) This means that ForeignKey field should belong to Testmodel2, not Testmodel1.
class Testmodel1():
amount = models.IntegerField(null=True)
entry_time = models.DateTimeField()
stage = choicesfiled
class Testmodel2():
name = models.CharField()
mobile_no = models.ForeignKey()
parent = models.ForeignKey(Testmodel1,
related_name='contacts',
)
With this model structure you can reference Testmodel1's contacts as testmodel1.contacts.all(). Your query then should look like this:
n1 = (Testmodel1.objects
.filter(entry_time__gte=last, stage=1)
.annotate(t_count=Count('contacts'))
.filter(t_count__gt=3)
)
docs reference
I have defined models:
tags = db.Table('tags',
db.Column('tag_id', db.Integer, db.ForeignKey('tag.id')),
db.Column('photo_id', db.Integer, db.ForeignKey('photo.id')),
)
class Tag(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(32), unique=True)
class Photo(db.Model):
id = db.Column(db.Integer, primary_key=True)
sha1sum = db.Column(db.LargeBinary(20), unique=True)
...
tags = db.relationship('Tag', secondary=tags,
backref=db.backref('photos', lazy='dynamic'))
In flask controller/view I get array of input tags for example ['summer', 'selfie', ...].
Questions:
Efficient query for photos which contain all of the requested tags?
How can it be extended for search with incomplete tags such as ['summ', 'elfi', ...]?
Might not be the most efficient (if you have many tags to search), but very readable way to compose the query:
input_tags = ['selfie', 'summer']
q = db.session.query(Photo)
for tag in input_tags:
q = q.filter(Photo.tags.any(Tag.name == tag))
For incomplete use startswith(..) instead of ==:
input_tags = ['sum', 'fu']
q = db.session.query(Photo)
for tag in input_tags:
q = q.filter(Photo.tags.any(Tag.name.startswith(tag)))
I'm having an issue with turbogears admin controller throwing an error when I try to edit the User, ShoppingItem or ShoppingList items (code below). The error coming up is AttributeError: 'function' object has no attribute 'primary_key'. The Local Variables in frame always come back as the same:
mapper
<Mapper at 0x3719810; ShoppingList>
fields
['id']
self
<sprox.sa.provider.SAORMProvider instance at 0x03E537B0>
value
<bound method OrderedProperties.items of <sqlalchemy.util._collections.OrderedProperties object at 0x037199F0>>
entity
<class 'insertmealhere.model.shoppinglist.ShoppingList'>
field_name
'items'
I'm having trouble figuring out what is different between this and the other many-to-many relationships that are configured elsewhere in the code and are not throwing this error. I'm running Turbogears 2.2 on Python 2.7.8 currently on a windows 8.1 system. Any help is greatly appreciated.
list_item_table = Table("list_item_table", metadata,
Column('item_id', Integer, ForeignKey('shopping_item.id', onupdate="CASCADE", ondelete="CASCADE"), primary_key=True),
Column('list_id', Integer, ForeignKey('shopping_list.id', onupdate="CASCADE", ondelete='CASCADE'), primary_key=True))
class ShoppingItem(DeclarativeBase):
__tablename__ = "shopping_item"
id = Column(Integer, primary_key=True)
name = Column(String(50))
quantity = Column(String(5))
measure = Column(String(10))
# less important optional parameters that will be useful for users
brand = Column(String(50))
list_id = Column(Integer, ForeignKey('shopping_list.id'))
shopping_list = relation("ShoppingList", secondary=list_item_table, backref="items")
def get_owner_id(self):
return self.list.user_id
#classmethod
def delete_list(cls, id, user_id):
item = DBSession.query(cls).filter_by(id=id).one() # get the item from the given ID
if item.get_owner_id() == user_id: # owned by current user
DBSession.delete(item) # delete from shopping list
return True
flash(_("You do not have authorization to perform that action."))
return False
class ShoppingList(DeclarativeBase):
__tablename__ = 'shopping_list'
id = Column(Integer, primary_key=True)
date = Column(Date, index=True, nullable=False)
static = Column(Boolean, nullable=False, default=False)
# static is true if the items from the meal plan have been imported into the shopping list. Once done you can edit
# the items in the shopping list, remove items, etc. Until the shopping list is made static it is impossible to edit
# the items that are imported from the schedule as they do not exist in the shopping list! (and we do not want to
# edit them in the recipe!
user_id = Column(Integer, ForeignKey('tg_user.user_id'))
user = relation("User", backref="shopping_lists")
date_user_list = Index('date_user_list', 'date', 'user_id')
Maybe it's the list_id = Column(Integer, ForeignKey('shopping_list.id')) in the ShoppingItem model class that's confusing SQLAlchemy?
I have a problem. I have to do this query:
#app.route('/api/subscriptions/<string:id>', methods=('DELETE',))
#decorators.login_required
def delete_subscription(id):
dbsession = DBSession()
session = Session()
favorit = (dbsession.query(StudentsFavorites)
.filter(Exams.number == str(id))
.filter(StudentsFavorites.exam_id)
.filter(Students.id == StudentsFavorites.student_id)
.filter(Students.id == str(session.get_user_id()))
.delete() )
dbsession.flush()
return jsonify(error=False)
But when I do this query I get this exception:
OperationalError: (OperationalError) no such column: exams.number u'DELETE FROM students_favorites WHERE exams.number = ? AND students_favorites.exam_id AND students.id = students_favorites.student_id AND students.id = ?' ('123123123', 'a24213')
The tables are very big and got lots of information, so i can't post all of it. But this query works:
#app.route('/api/subscriptions/<string:id>', methods=('PUT',))
#decorators.login_required
def add_subscription(id):
dbsession = DBSession()
session = Session()
examID = (dbsession.query(Exams.id)
.filter(Exams.number == id).first()
)
favorit=StudentsFavorites(student_id=session.get_user_id(), exam_id=examID.id)
dbsession.add(favorit)
dbsession.flush()
return jsonify(error=False)
Short view to the table:
table: Exams
rows: id, number (number is the id i put into the function)
table: StudentsFavorites
rows: student_id, exams_id
table: Students
rows: id
I really didn't understand, why he didn't find the number row in the exception.
EDIT:
Database StudentsFavorites:
class StudentsFavorites(Base):
"""N:M resolve model for the exams to the semester.
"""
__tablename__ = "students_favorites"
student_id = Column(Unicode(255), ForeignKey("students.id"), primary_key=True, autoincrement=False)
exam_id = Column(Integer, ForeignKey("exams.id"), primary_key=True, autoincrement=False)
created_at = Column(DateTime, nullable = False, default = datetime.now)
student = relationship("Students", uselist = False, lazy="joined")
exam = relationship("Exams", uselist=False, lazy="joined")
Something like this? I tried this:
(dbsession.query(StudentsFavorites)
.filter(StudentsFavorites.exam.id == str(id))
.filter(StudentsFavorites.student.id == str(session.get_user_id()))
.delete()
)
But got the error, that id didn't exist in exams / student
You have two cases of the same problem. Your query has information for StudentFavorites which means it knows about StudentFavorites.student_id and StudentFaovrites.exams_id. It doesn't know anything about Students.id, Exames.id and Exames.number. In order for you to query a StudentFavorites object and have it know about those other values you're going to have to perform a sql join.
Join's can be a bit of a pain in the ass to get working in sqlalchemy (well... in regular sql as well). Since I don't know what your table schema is I can't talk about that but the view should look something like this.
#app.route('/api/subscriptions/<string:id>', methods=('DELETE',))
#decorators.login_required
def delete_subscription(id):
dbsession = DBSession()
session = Session()
favorit = (dbsession.query(StudentsFavorites)
.join(Exames)
.join(students)
.filter(Exams.number == str(id))
.filter(StudentsFavorites.exam_id)
.filter(Students.id == StudentsFavorites.student_id)
.filter(Students.id == str(session.get_user_id()))
.delete() )
dbsession.flush()
return jsonify(error=False)
Alternatively, you can look into setting up Foreign key relationships in your table statements if you use the ORM to create your tables
The reason your second example works is because you're specifying a query over an exam table and only using values found in that table.
Response to Edit:
Right now your table relationships aren't set up correctly. Specifically the sections: Many To Many and Deleting Rows from the Many to Many Table
This example code is explained in much more (and better) detail in the posted link but the basic idea is that you have a associate_table (in your case StudentFavorites) contains foreign keys which have a relationship which is specified in one or more of your other tables. I personally advise that you go with the table example and not the object example.
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('left.id')),
Column('right_id', Integer, ForeignKey('right.id'))
)
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary=association_table,
backref="parents")
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)