The contains operation in SQLAlchemy only accepts one Model object instead of a list of objects. If I want to create a filter that accepts containing any of a group of objects, is there a more SQL-style way than creating multiple filters using contains and combining them with union?
For example, see the following code:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(100))
son = db.relationship('Son', backref = 'parent', lazy = 'dynamic')
def __repr__(self):
return '<"%s">' % self.name
class Son(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(1000))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
def __repr__(self):
return '<"%s">' % self.name
db.create_all()
son1 = Son(name = '1 a 1')
son2 = Son(name = '1 b 1')
son3 = Son(name = '1 c 1')
son4 = Son(name = '2 a 2')
son5 = Son(name = '2 b 2')
user0 = User(name = 'w1')
user1 = User(name = 'w2')
user2 = User(name = 'w3')
user0.son.append(son1)
user0.son.append(son2)
user1.son.append(son3)
user1.son.append(son4)
user2.son.append(son5)
db.session.add(son1)
db.session.add(son2)
db.session.add(son3)
db.session.add(son4)
db.session.add(son5)
db.session.add(user0)
db.session.add(user1)
db.session.add(user2)
db.session.commit()
son_query = Son.query.filter(Son.name.ilike('%a%'))
son_query_all = son_query.all()
print son_query.all()
user_query = User.query.filter(User.son.contains(son_query_all[0])).union(*[User.query.filter(User.son.contains(query)) for query in son_query_all[1:]])
print user_query.all()
The example firstly creates two models: User and Son, and then creates 3 User instances and 5 Son instances. user0 contains son1 and son2, user1 contains son3 and son4, and user2 contains son5. Note that the name of son1 and son4 are both like %a%. Now I want to select all User instances containing Son instances whose name likes %a%.
The current method is to select all Son instances in son_query_all, and then selects User instances containing individual desired Son instances, and then combines the selecting result using union. Is there a more SQL-style way for SQLAlchemy to select the same? For example, is there anything like contains_any so that the last query can be changed into something like
user_query = User.query.filter(User.son.contains_any(son_query_all))
Note that of course I can define a custom contains_any function for the same purpose using the union and contains operation. My question is whether there is a more efficient way than simply union all contains-ed?
The correct way to solve this kind of filtering is to use JOIN. Basically you join Son table with User table and filter by joined entity's field.
In SQLAlchemy you can express it like this:
User.query.join(Son).filter(Son.name.ilike('%a%'))
And it will produce following SQL:
SELECT * FROM user
JOIN son ON user.id = son.user_id
WHERE lower(son.name) LIKE lower('%a%')
Related
In my application, I am using SQLAlchemy (via MySQL) with Flask, and in my current situation I need to create entries dynamically. For example, while creating an instance from table A, my script will see that we need to create a related entry B, so after creating B, it will attempt to assign B to one of the relationship attributes of the instance for A.
Here are some example models:
a_b = db.Table('a_b',
db.Column('a_id', db.Integer,
db.ForeignKey('a.id',
onupdate="CASCADE",
ondelete="CASCADE"),
primary_key=True),
db.Column('b_id', db.Integer,
db.ForeignKey('b.id',
onupdate="CASCADE",
ondelete="CASCADE"),
primary_key=True))
class ModelA(db.Model):
__tablename__ = 'a'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
model_bs = db.relationship('ModelB', secondary=a_b,
back_populates='model_as')
class ModelB(db.Model):
__tablename__ = 'b'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
model_as = db.relationship('ModelA', secondary=a_b,
back_populates='model_bs')
The problem I'm running into is an SQL duplicate entry exception, after running something like this:
from base import db
from base.database.models.models import ModelA, ModelB
def creation_test():
with db.session.no_autoflush:
# Start creating A
a = ModelA()
a.name = 'A Example'
# See that we need B, so create B
b = ModelB()
b.name = 'B Example'
b.model_as = []
db.session.add(b)
db.session.flush((b,))
# Use B in the relationship from A
a.model_bs = [b]
db.session.add(a)
db.session.flush((a,))
db.session.commit()
creation_test()
It will spit out an exception like:
sqlalchemy.exc.IntegrityError: (_mysql_exceptions.IntegrityError) (1062, "Duplicate entry '1-1' for key 'PRIMARY'") [SQL: u'INSERT INTO a_b (a_id, b_id) VALUES (%s, %s)'] [parameters: (1L, 1L)]
Is there something wrong with how the entries are being added to the session and/or how they are being flushed? Is it even possible to instantiate entries like this?
I appreciate any feedback.
After updating my version of SQLAlchemy and testing a few things, it seemed like I was able to solve this by removing the flushes in between creating each instance, and instead just perform a single flush before committing the session:
from base import db
from base.database.models.models import ModelA, ModelB
def creation_test():
with db.session.no_autoflush:
# Start creating A
a = ModelA()
a.name = 'A Example'
# See that we need B, so create B
b = ModelB()
b.name = 'B Example'
b.model_as = []
db.session.add(b)
# Use B in the relationship from A
a.model_bs = [b]
db.session.add(a)
db.session.flush()
db.session.commit()
creation_test()
I am using the following model:
class Topping(models.Model):
name = models.CharField(max_length=30)
class Pizza(models.Model):
name = models.CharField(max_length=50)
toppings = models.ManyToManyField(Topping)
def __str__(self): # __unicode__ on Python 2
return "%s (%s)" % (self.name, ", ".join(topping.name
for topping in self.toppings.all()))
And now I want only the elements for vegetarian menu, filtered by tomatoes
pizza_item = Pizza.objects.filter(toppings__name='tomatoes')
My select is:
SELECT `pizza`.`id`, `pizza`.`name`
FROM `pizza`
INNER JOIN `pizza_toppings` ON (
`pizza`.`id` = `pizza_toppings`.`pizza_id` )
INNER JOIN `web_topping` ON (
`pizza_toppings`.`topping_id` = `topping`.`id` )
WHERE `topping`.`name` = azucar
but i want get:
SELECT `pizza`.`id`, `pizza`.`name`, `topping`.`name`
FROM `pizza`
INNER JOIN `pizza_toppings` ON (
`pizza`.`id` = `pizza_toppings`.`pizza_id` )
INNER JOIN `web_topping` ON (
`pizza_toppings`.`topping_id` = `topping`.`id` )
WHERE `topping`.`name` = azucar
This last query works fine in mysql db. And works using pizza.objects.raw but i want get using django ORM
Is a select with topping.name i try it using prefetch_select('toppings'). but i cant get the same select.
Have you tried using the values method for Queryset ?
Something like :
pizza_item = Pizza.objects.filter(toppings__name='tomatoes').values("id", "name", "toppings__name")
I am not sure if that's doable. Because when you use Pizza.objects... you are limited to the fields that are in the Pizza model. Since the Pizza model does not contain toppings' name field. You cannot retrieve it. You can only retrieve toppings' id field:
pizza_item = Pizza.objects.filter(toppings__name='tomatoes').values('id', 'name', 'toppings')
Which will provide "toppings"."topping_id" in SELECT.
Also, since you have specified that toppings__name='tomatoes, all of toppings' name will be tomatoes in this queryset, so what is the point of having topping.name in your result?
How I can make a query with two models using this
model.objects.raw(...)
and into the sql query has the INNER JOIN with the another model(table) this is possible
model.objects.raw('
SELECT establecimiento.nombre, categoria.titulo
FROM establecimiento INNER JOIN
categoria ON establecimiento.categoria = categoria.id')
I need print the establecimiento's name with his categoria's name
class Establecimiento(models.Model):
nombre = models.CharField(max_length = 140)
categoria = models.ForeignKey(Categoria)
ciudad = models.ForeignKey(Ciudad)
def __unicode__(self):
return self.nombre
class Categoria(models.Model):
titulo = models.CharField(max_length = 140)
Fetching objects from the ORM automatically does any joins required and will return objects (instances of the models) which you can use to follow relationships.
If you simply fetch all your Establecimiento objects, you can access the related Categoria objects, like this:
all_objects = Establecimiento.objects.all()
for obj in all_objects:
print('Number: {} Category: {}'.format(obj.nombre, obj.categoria.titulo))
Or, if you want to fetch only those two specific properties, use values, like this:
all_objects = Establecimiento.objects.values('nombre','ciudad__titulo')
for obj in all_objects:
print('Number: {} Category: {}'.fromat(obj['nombre'],obj['ciudad__titulo']))
I have a problem. I have to do this query:
#app.route('/api/subscriptions/<string:id>', methods=('DELETE',))
#decorators.login_required
def delete_subscription(id):
dbsession = DBSession()
session = Session()
favorit = (dbsession.query(StudentsFavorites)
.filter(Exams.number == str(id))
.filter(StudentsFavorites.exam_id)
.filter(Students.id == StudentsFavorites.student_id)
.filter(Students.id == str(session.get_user_id()))
.delete() )
dbsession.flush()
return jsonify(error=False)
But when I do this query I get this exception:
OperationalError: (OperationalError) no such column: exams.number u'DELETE FROM students_favorites WHERE exams.number = ? AND students_favorites.exam_id AND students.id = students_favorites.student_id AND students.id = ?' ('123123123', 'a24213')
The tables are very big and got lots of information, so i can't post all of it. But this query works:
#app.route('/api/subscriptions/<string:id>', methods=('PUT',))
#decorators.login_required
def add_subscription(id):
dbsession = DBSession()
session = Session()
examID = (dbsession.query(Exams.id)
.filter(Exams.number == id).first()
)
favorit=StudentsFavorites(student_id=session.get_user_id(), exam_id=examID.id)
dbsession.add(favorit)
dbsession.flush()
return jsonify(error=False)
Short view to the table:
table: Exams
rows: id, number (number is the id i put into the function)
table: StudentsFavorites
rows: student_id, exams_id
table: Students
rows: id
I really didn't understand, why he didn't find the number row in the exception.
EDIT:
Database StudentsFavorites:
class StudentsFavorites(Base):
"""N:M resolve model for the exams to the semester.
"""
__tablename__ = "students_favorites"
student_id = Column(Unicode(255), ForeignKey("students.id"), primary_key=True, autoincrement=False)
exam_id = Column(Integer, ForeignKey("exams.id"), primary_key=True, autoincrement=False)
created_at = Column(DateTime, nullable = False, default = datetime.now)
student = relationship("Students", uselist = False, lazy="joined")
exam = relationship("Exams", uselist=False, lazy="joined")
Something like this? I tried this:
(dbsession.query(StudentsFavorites)
.filter(StudentsFavorites.exam.id == str(id))
.filter(StudentsFavorites.student.id == str(session.get_user_id()))
.delete()
)
But got the error, that id didn't exist in exams / student
You have two cases of the same problem. Your query has information for StudentFavorites which means it knows about StudentFavorites.student_id and StudentFaovrites.exams_id. It doesn't know anything about Students.id, Exames.id and Exames.number. In order for you to query a StudentFavorites object and have it know about those other values you're going to have to perform a sql join.
Join's can be a bit of a pain in the ass to get working in sqlalchemy (well... in regular sql as well). Since I don't know what your table schema is I can't talk about that but the view should look something like this.
#app.route('/api/subscriptions/<string:id>', methods=('DELETE',))
#decorators.login_required
def delete_subscription(id):
dbsession = DBSession()
session = Session()
favorit = (dbsession.query(StudentsFavorites)
.join(Exames)
.join(students)
.filter(Exams.number == str(id))
.filter(StudentsFavorites.exam_id)
.filter(Students.id == StudentsFavorites.student_id)
.filter(Students.id == str(session.get_user_id()))
.delete() )
dbsession.flush()
return jsonify(error=False)
Alternatively, you can look into setting up Foreign key relationships in your table statements if you use the ORM to create your tables
The reason your second example works is because you're specifying a query over an exam table and only using values found in that table.
Response to Edit:
Right now your table relationships aren't set up correctly. Specifically the sections: Many To Many and Deleting Rows from the Many to Many Table
This example code is explained in much more (and better) detail in the posted link but the basic idea is that you have a associate_table (in your case StudentFavorites) contains foreign keys which have a relationship which is specified in one or more of your other tables. I personally advise that you go with the table example and not the object example.
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('left.id')),
Column('right_id', Integer, ForeignKey('right.id'))
)
class Parent(Base):
__tablename__ = 'left'
id = Column(Integer, primary_key=True)
children = relationship("Child",
secondary=association_table,
backref="parents")
class Child(Base):
__tablename__ = 'right'
id = Column(Integer, primary_key=True)
I have following models (among others), I' trying to inplement tree structure using closure tables:
class Part(models.Model):
name = models.TextField()
quantity = models.IntegerField()
manufacturer = models.ForeignKey(Manufacturer)
def __str__(self):
return "Part {}: {}, {}".format(self.id, self.name, self.quantity)
def path(self):
ancestors = Part.objects.filter(closures_d__descendant__id__exact=self.id)
print(len(ancestors))
return "{}".format(reduce(lambda a, b: a+" > "+b, map(lambda x: x.name, ancestors), ""))
class Meta:
db_table = 'parts'
class Closure(models.Model):
ancestor = models.ForeignKey(Part, related_name='closures_a')
descendant = models.ForeignKey(Part, related_name='closures_d')
class Meta:
db_table = 'closures'
The problem is path function does not work as expected, the query that fetches
ancestors from database returns only one object (the part that called it).
I checked generated SQL queries, and it appears that closures are ignored (output from connection.queries):
[{'time': '0.001', 'sql': 'SELECT "parts"."id", "parts"."name", "parts"."quantity", "parts"."manufacturer_id" FROM "parts" WHERE "parts"."id" = 101 '}]
How to do the join properly? (I mean Python/Django way, I can do this in raw SQL)
The query should look like this (101 is example id, part exists in db and has few ancestors, query below returns correct results):
SELECT * from parts p JOIN closures c ON p.id=c.ancestor_id WHERE c.descendant_id=101
OK, I got it right, query for ancestors should say:
ancestors = Part.objects.filter(closures_a__descendant_id__exact=self.id)
instead of:
ancestors = Part.objects.filter(closures_d__descendant__id__exact=self.id)
Query returns correct results, but displayed SQL still looks wrong (no JOIN),
I have no idea why, I got generated query using this code (path function from class Part from question):
def path(self):
ancestors = Part.objects.filter(closures_a__descendant_id__exact=self.id)
from django.db import connection
print(connection.queries)
return "{}".format(reduce(lambda a, b: a+" > "+b, map(lambda x: x.name, ancestors), ""))