Sqlalchemy many to many relationship - python-2.7

I have two tables in many to many relationship:
class Association(db.Model):
__tablename__ = 'association'
club_id = db.Column(db.Integer, db.ForeignKey('clubs.id'), primary_key=True)
student_id = db.Column(db.Integer, db.ForeignKey('students.id'), primary_key=True)
joined_date = db.Column(db.String)
assoc_student = db.relationship("Student")
class Club(db.Model):
__tablename__ = 'clubs'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String)
location = db.Column(db.String)
club_assoc = db.relationship("Association")
class Student(db.Model):
__tablename__ = 'students'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
age = db.Column(db.String)
gender = db.Column(db.String)
Questions:
1) What is the difference between these two queries?
students = db.session.query(Association).filter_by(club_id='1')
students = Association.query.filter_by(club_id='1')
They seem to give the same result!
2) I'm trying to get a list of students with certain age but this following query doesn't work:
db.session.query(Association).filter_by(Association.club_id=='1', Association.assoc_student.age=='15')
But I get this error:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Association.assoc_student has an attribute 'age'
That's why I'm using this one:
db.session.query(Student).join(Association).filter(Association.club_id=='1', Student.age=='15')
Is there a better way to do this without "join"? Maybe with using "backref"!?

1) What is the difference between these two queries?
They do almost the same thing. Former is the way to query objects provided with SQLAlchemy (library Flask uses to access database).
Latter is the convenient way to query models added by Flask-SQLAlchemy library. It makes your queries more readable + extends query with few useful methods. Take a look at source of the flask_sqlalchemy.BaseQuery class to see them: get_or_404(), first_or_404() and paginate().
Usually you want to use latter method to query objects.
2) I'm trying to get a list of students with certain age but this following query doesn't work.
There are two things here:
Be aware about the difference between filter() and filter_by() methods. In your example you try to use filter_by() with SQL expressions instead of kwargs, which is incorrect.
When you're using filter() you can't specify columns over a relationships (like Association.assoc_student.age). The only allowed format is ModelName.column_name. That's why it fails.
Is there a better way?
Your second approach is absolutely correct and fine to use. I don't think there is a better way to do it. Alternatively you can use code below to avoid importing db (if you define query in another file):
Student.query.join(Association).filter(Association.club_id == '1', Student.age == '15')

Related

Flask how to query with a filter that checks whether a value is present in a list?

I'm making a bug tracker structured such that each user can have multiple projects, and each project can record multiple bugs. I'd like users to be able to see how many new bugs were reported across all their projects since their last login.
I structured it such that there's a Users model, a Projects model, and Bugs model. I've included the three models with their relevant columns below:
class Users(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key = True, nullable = False)
# ...
last_login = db.Column(db.DateTime, nullable = False)
# AS PARENT
owned_projects = db.relationship('Projects', backref="project_owner")
class Projects(db.Model):
id = db.Column(db.Integer, primary_key = True, nullable = False)
# ...
# AS CHILD
owner = db.Column(db.Integer, db.ForeignKey('users.id'))
# AS PARENT
bugs = db.relationship('Bugs', backref = 'containing_project')
class Bugs(db.Model):
id = db.Column(db.Integer, primary_key = True)
# ...
status = db.Column(db.String(20))
report_date = db.Column(db.DateTime, index = True, default = dt.datetime.now())
# AS CHILD
project = db.Column(db.Integer, db.ForeignKey('projects.id'))
To filter a Bugs query for bugs reported since the user's last login, I can do a filter like Bugs.query.filter(Bugs.report_date > current_user.last_login). This shows ALL bugs from ALL users, but I'm having trouble constructing a query that filters it down to only bugs in projects owned by the user.
.filter(Bugs.containing_project in current_user.owned_projects) returns "<flask_sqlalchemy.BaseQuery object at 0x04E06988>", but I have no idea how to work with that. I read about .contains but that goes in the wrong direction, current_user.owned_projects.contains(Bugs.project) does not work.
I also tried .filter(Bugs.containing_project.owner == current_user.id), but got an error "AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Bugs.containing_project has an attribute 'owner'".
The only option I can think of now is to iterate through every single bug to find the ones that belong to a project owned by the user, but that would be nightmarish in terms of performance. Surely there's a .in method or something similar?
Please advise on how I can achieve this, thanks in advance!
This code doesn't work?
.filter(Bugs.containing_project.any(Projects.id.in_([ proj['id'] for proj in current_user.owned_projects])
I refered this site.
SQLAlchemy: filter by membership in at least one many-to-many related table
And, if my answer doesn't work well, this site may help you.
SQLAlchemy how to filter by children in many to many

flask sqlalchemy one-to-many filter, count & grouping

I am currently trying to build a query which
give me for a one-to-many sqlalchemy query in flask both my result filters grouped and then says how many individual entries there are for it
Following is my database model to illustrate the question:
class cellphone(db.Model):
__tablename__ = 'cellphone'
id = db.Column(db.Integer, primary_key = True)
number = db.Column(db.String(30), unique=True)
sms = db.relationship('sms_accounting', backref="cellphone", lazy='dynamic')
class sms_accounting(db.Model):
__tablename__ = 'sms_accounting'
id = db.Column(db.Integer, primary_key = True)
timestamp = db.Column(db.DateTime, index=True, default=datetime.utcnow)
cellphone_id = db.Column(db.Integer, db.ForeignKey('cellphone.id'))
What I want to do now is find out how many SMS were sent within X days per number.
Filtering and grouping I managed to do, but to calculate the sum per device correctly is not possible.
def sms_count():
search_peroid='90' #time to fetch events in days
period_start = datetime.utcnow() - timedelta(hours=int(search_peroid))
phone_count = sms_accounting.query.filter(sms_accounting.timestamp.between(period_start, \
datetime.utcnow() )).group_by(sms_accounting.cellphone_id).all()
I found some examples for func.count, but unfortunately none of them works. This already starts with the usage,
AttributeError: BaseQuery object has no attribute 'func'
even though it was imported especially.
from sqlalchemy.sql.functions import func
Forgive me if I am wrong.
As an option, you could try executing an SQL Query through Flask.
db.session.execute('select number, count(sms_accounting.id) from cellphone join sms_accounting on sms_accounting.cellphone_id = cellphone.id');
You can easily add the time based filter using where.
Regarding the AttributeError, are you sure you are using the 'func' method properly? The correct usage can be found on this unrelated answer at https://stackoverflow.com/a/4086229/4854064. It might be that you accidentally called func as a method of the query object.

set the insert order of a many to many sqlalchemy flask app sqlite db

My Goal
I want to record the order of manys upon insert of data to my table (e.g. Clump-see tables below). The orderinglist module is really great, but how do i apply it to the intermediary table (named clump_syntaxs) between my many-to-many? anyone done this before and have a good example?
problem re-stated
How do i apply ordering upon insert to my many to many. Everything I try using the intermediary table-clump_syntaxs table crashes (sorry for the weird names!).
The following code (reduced for brevity) works! except that it only allows for a syntax to have a unique position (instead of a position for every Clump instance), and I am guessing I need the position variable to be on the clump_syntaxs table.all tables are sqlite
my intermediary table
from sqlalchemy.ext.orderinglist import ordering_list
clump_syntaxs = db.Table('clump_syntaxs',
db.Column('syntax_id', db.Integer, db.ForeignKey('syntax.id')),
db.Column('clump_id', db.Integer, db.ForeignKey('clump.id')),
)
add a clump and order syntax tables
class Clump(db.Model):
id = db.Column(db.Integer, primary_key=True)
syntaxs = db.relationship('Syntax', secondary=clump_syntaxs,
backref=db.backref('clumps', lazy='dynamic'),order_by="Syntax.position",
collection_class=ordering_list('position'))
class Syntax(db.Model):
id = db.Column(db.Integer, primary_key=True)
jobs = db.relationship('Jobs',lazy='dynamic', backref='jobhistory')
position = db.Column(db.Integer)
#Jobs table not included
Yes, you should move position field to the intermediary table ClumpSyntax, and take advantage of association_proxy() in Clump table.
import sqlalchemy.ext.associationproxy import association_proxy
class Syntax(db.Model):
id = db.Column(db.Integer, primary_key=True)
jobs = db.relationship('Jobs',lazy='dynamic', backref='jobhistory')
#position = db.Column(db.Integer) # moved to ClumpSyntax
#Jobs table not included
class ClumpSyntax(db.Model):
syntax_id = db.Column('syntax_id', db.Integer, db.ForeignKey('syntax.id'))
syntax = relationship(Syntax)
clump_id = db.Column('clump_id', db.Integer, db.ForeignKey('clump.id'))
position = db.Column(db.Integer)
# this constructor is very necessary !
def __init__(self, syntax =None):
self.syntax = syntax
class Clump(db.Model):
id = db.Column(db.Integer, primary_key=True)
_syntaxs = db.relationship('ClumpSyntax',
order_by=[ClumpSyntax.position],
collection_class=ordering_list('position'))
syntaxs = association_proxy('_syntaxs','syntax')
My similar request was satisfied quite well by this way, based on the article this and this. You can test it by code like below:
session= some_code_to_get_db_session()
syn1= Syntax()
syn2= Syntax()
syn3= Syntax()
session.add(syn1)
session.add(syn2)
session.add(syn3)
clump= Clump()
session.add(clump)
clump.syntaxs.append(syn1)
clump.syntaxs.append(syn2)
clump.syntaxs.append(syn3)
session.commit()
session.query(ClumpSyntax).count() # print out 3
session.query(Syntax).count() # print out 3

Django queryset - Adding HAVING constraint

I have been using Django for a couple of years now but I am struggling today with adding a HAVING constraint to a GROUP BY.
My queryset is the following:
crm_models.Contact.objects\
.filter(dealercontact__dealer__pk__in=(265,),
dealercontact__activity='gardening',
date_data_collected__gte=datetime.date(2012,10,1),
date_data_collected__lt=datetime.date(2013,10,1))\
.annotate(nb_rels=Count('dealercontact'))
which gives me the following MySQL query:
SELECT *
FROM `contact`
LEFT OUTER JOIN `dealer_contact` ON (`contact`.`id_contact` = `dealer_contact`.`id_contact`)
WHERE (`dealer_contact`.`active` = True
AND `dealer_contact`.`activity` = 'gardening'
AND `contact`.`date_data_collected` >= '2012-10-01'
AND `contact`.`date_data_collected` < '2013-10-01'
AND `dealer_contact`.`id_dealer` IN (265))
GROUP BY `contact`.`id_contact`
ORDER BY NULL;
I would get exactly what I need with this HAVING constraint:
HAVING SUM(IF(`dealer_contact`.`type`='customer', 1, 0)) = 0
How can I get this fixed with a Django Queryset? I need a queryset in this instance.
Here I am using annotate only in order to get the GROUP BY on contact.id_contact.
Edit: My goal is to get the Contacts who have no "customer" relation in dealercontact but have "ref" relation(s) (according to the WHERE clause of course).
Models
class Contact(models.Model):
id_contact = models.AutoField(primary_key=True)
title = models.CharField(max_length=255L, blank=True, choices=choices_custom_sort(TITLE_CHOICES))
last_name = models.CharField(max_length=255L, blank=True)
first_name = models.CharField(max_length=255L, blank=True)
[...]
date_data_collected = models.DateField(null=True, db_index=True)
class Dealer(models.Model):
id_dealer = models.AutoField(primary_key=True)
address1 = models.CharField(max_length=45L, blank=True)
[...]
class DealerContact(Auditable):
id_dealer_contact = models.AutoField(primary_key=True)
contact = models.ForeignKey(Contact, db_column='id_contact')
dealer = models.ForeignKey(Dealer, db_column='id_dealer')
activity = models.CharField(max_length=32, choices=choices_custom_sort(ACTIVITIES), db_index=True)
type = models.CharField(max_length=32, choices=choices_custom_sort(DEALER_CONTACT_TYPE), db_index=True)
I figured this out by adding two binary fields in DealerContact: is_ref and is_customer.
If type='ref' then is_ref=1 and is_customer=0.
Else if type='customer' then is_ref=0 and is_customer=1.
Thus, I am now able to use annotate(nb_customers=Sum('is_customer')) and then use filter(nb_customers=0).
The final queryset consists in:
Contact.objects.filter(dealercontact__dealer__pk__in=(265,),
dealercontact__activity='gardening',
date_data_collected__gte=datetime.date(2012,10,1),
date_data_collected__lt=datetime.date(2013,10,1))\
.annotate(nb_customers=Sum('dealercontact__is_customer'))\
.filter(nb_customers=0)
Actually there is a way you can add your own custom HAVING and GROUP BY clauses if you need.
Just use my example with caution - if Django ORM code/paths will change in future Django versions, you will have to update your code too.
Image you have Book and Edition models, where for each book there can be multiple editions and you want to select first US edition date within Book queryset.
Adding custom HAVING and GROUP BY clauses in Django 1.5+:
from django.db.models import Min
from django.db.models.sql.where import ExtraWhere, AND
qs = Book.objects.all()
# Standard annotate
qs = qs.annotate(first_edition_date=Min("edition__date"))
# Custom HAVING clause, to limit annotation by US country only
qs.query.having.add(ExtraWhere(['"app_edition"."country"=%s'], ["US"]), AND)
# Custom GROUP BY clause will be needed too
qs.query.group_by.append(("app_edition", "country"))
ExtraWhere can contain not just fields, but any raw sql conditions and functions too.
Are you not using raw query just because you want orm object? Using Contact.objects.raw() generate instances similar filter. Refer to https://docs.djangoproject.com/en/dev/topics/db/sql/ for more help.
My goal is to get the Contacts who have no "customer" relation in
dealercontact but have "ref" relation(s) (according to the WHERE
clause of course).
This simple query fulfills this requirement:
Contact.objects.filter(dealercontact__type="ref").exclude(dealercontact__type="customer")
Is this enough, or do you need it to do something more?
UPDATE: if your requirement is
Contacts that have a "ref" relations, but do not have "customer"
relations with the same dealer
you can do this:
from django.db.models import Q
Contact.objects.filter(Q(dealercontact__type="ref") & ~Q(dealercontact__type="customer"))

Insert data into specific table (sqlalchemy)

It's pretty easy to insert data into a database by using sqlalchemy.
address.name = 'Joe'
address.age = 26
session.add(address)
But actually I have three tables - how can I specify the table I want to insert my data in?
I solved the problem by using the sql expression language to add the new row:
engine.execute(table_addresses.insert(), name='Joe', age=20)
engine.execute(table_contacts.insert(), email='joe#something.com', cellnumber='267534320')
Normaly I use the ORM but in my case it is more convinient.
You will probably want to look at the sqlalchemy ORM tutorial.
To get you started though, I suspect you are going to want to set up something like the following.
from sqlalchemy import Column, Integer, String
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
name = Column(String)
age = Column(Integer)
def __init__(self, name, age):
self.name = name
self.age = age
Now since it appears you already know how to get a session, you could just do this
address = Address('Joe', 26)
session.add(address)
session.commit() # This will do the actual database insertion
Although it is possible that you could probably just try to do it on the fly, it probably wouldn't make your life easier, as you'd still have to tell it what objects relate to integers, strings, etc, to map to the database table.