I have one MySQL server with multiple schemas and it looks something like this:
-app_metadata [database]
users [table]
app_data [table]
-site1 [database]
tableA [table]
tableB [table]
There is multiple sites (site1, site2.. site32) and there are new sites being added periodically but the structure of all sites are the same, they all have table A and table B that that can be represented by identical models in flask-SQLalchemy.
Retrieving information depending on what site the user is interested in is quite simple with a query (not actual code but the idea is there):
user_selection = site3
query = f'SELECT * FROM `{user_selection}`.tableA;'
I'm not that familiar with flask-sqlalchemy and I have a hard time figuring out how can I translate this so that the URI is dynamically switch (based on the site selection).
I can do bindings for the app_metadata database as follows (I assume, not really the issue):
class User(db.Model):
__bind_key__ = 'app_metadata'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
The problem is how do I connect to the second database (pretty much on the go) based on the users choice and change the bind key inside the table classes.
class TableA(db.Model):
__bind_key__ = *users_choice*
id = db.Column(db.Integer, primary_key=True)
....
class TableB(db.Model):
__bind_key__ = *users_choice*
id = db.Column(db.Integer, primary_key=True)
....
I can't really change the structure of the database but obviously I can play with the code.
Related
I have a catalog and the price is a one to many relationship, so that I can track the price of a catalog item over time. The model looks like:
class CatalogItem(db.Model)
id = db.Column(db.Integer(), primary_key=True)
name = db.Column(db.String(250))
price = db.relationship("Price", back_populates="catalogitem")
class Price(db.Model):
id = db.Column(db.Integer(), primary_key=True)
price = db.Column(db.Float())
timestamp = db.Column(db.DateTime(), server_default=func.now(), nullable=False)
catalogitem_id = db.Column(db.Integer(), db.ForeignKey("catalogitem.id"))
catalogitem = db.relationship("CatalogItem", back_populates="material_einheitspreis_verkauf")
And this is my View. At least I managed to only show the price.
class CatalogItemView(ModelView):
inline_models = [(Price, dict(form_columns=["id","price"]))]
There are two issues:
When I render a catalog item and set price as inline model, I can do that just fine, but the default behavior would allow me to add multiple prices. What I would actually like to do is to have just a price field. I'm looking for a way to limit the form to just one entity (and also leaving away the button "add price".
When editing a catalogitem, it shouldn't edit the price, but actually create a new relationship -> basically when I edit the price it will create a new Price entity.
For 1 I have not idea on how to achieve this. For 2 I guess I could maybe do this by adding some kind of additional form field outside of the model and then create the "magic" with some kind of listeners.
Any other ideas?
I'm making a bug tracker structured such that each user can have multiple projects, and each project can record multiple bugs. I'd like users to be able to see how many new bugs were reported across all their projects since their last login.
I structured it such that there's a Users model, a Projects model, and Bugs model. I've included the three models with their relevant columns below:
class Users(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key = True, nullable = False)
# ...
last_login = db.Column(db.DateTime, nullable = False)
# AS PARENT
owned_projects = db.relationship('Projects', backref="project_owner")
class Projects(db.Model):
id = db.Column(db.Integer, primary_key = True, nullable = False)
# ...
# AS CHILD
owner = db.Column(db.Integer, db.ForeignKey('users.id'))
# AS PARENT
bugs = db.relationship('Bugs', backref = 'containing_project')
class Bugs(db.Model):
id = db.Column(db.Integer, primary_key = True)
# ...
status = db.Column(db.String(20))
report_date = db.Column(db.DateTime, index = True, default = dt.datetime.now())
# AS CHILD
project = db.Column(db.Integer, db.ForeignKey('projects.id'))
To filter a Bugs query for bugs reported since the user's last login, I can do a filter like Bugs.query.filter(Bugs.report_date > current_user.last_login). This shows ALL bugs from ALL users, but I'm having trouble constructing a query that filters it down to only bugs in projects owned by the user.
.filter(Bugs.containing_project in current_user.owned_projects) returns "<flask_sqlalchemy.BaseQuery object at 0x04E06988>", but I have no idea how to work with that. I read about .contains but that goes in the wrong direction, current_user.owned_projects.contains(Bugs.project) does not work.
I also tried .filter(Bugs.containing_project.owner == current_user.id), but got an error "AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Bugs.containing_project has an attribute 'owner'".
The only option I can think of now is to iterate through every single bug to find the ones that belong to a project owned by the user, but that would be nightmarish in terms of performance. Surely there's a .in method or something similar?
Please advise on how I can achieve this, thanks in advance!
This code doesn't work?
.filter(Bugs.containing_project.any(Projects.id.in_([ proj['id'] for proj in current_user.owned_projects])
I refered this site.
SQLAlchemy: filter by membership in at least one many-to-many related table
And, if my answer doesn't work well, this site may help you.
SQLAlchemy how to filter by children in many to many
I am currently trying to build a query which
give me for a one-to-many sqlalchemy query in flask both my result filters grouped and then says how many individual entries there are for it
Following is my database model to illustrate the question:
class cellphone(db.Model):
__tablename__ = 'cellphone'
id = db.Column(db.Integer, primary_key = True)
number = db.Column(db.String(30), unique=True)
sms = db.relationship('sms_accounting', backref="cellphone", lazy='dynamic')
class sms_accounting(db.Model):
__tablename__ = 'sms_accounting'
id = db.Column(db.Integer, primary_key = True)
timestamp = db.Column(db.DateTime, index=True, default=datetime.utcnow)
cellphone_id = db.Column(db.Integer, db.ForeignKey('cellphone.id'))
What I want to do now is find out how many SMS were sent within X days per number.
Filtering and grouping I managed to do, but to calculate the sum per device correctly is not possible.
def sms_count():
search_peroid='90' #time to fetch events in days
period_start = datetime.utcnow() - timedelta(hours=int(search_peroid))
phone_count = sms_accounting.query.filter(sms_accounting.timestamp.between(period_start, \
datetime.utcnow() )).group_by(sms_accounting.cellphone_id).all()
I found some examples for func.count, but unfortunately none of them works. This already starts with the usage,
AttributeError: BaseQuery object has no attribute 'func'
even though it was imported especially.
from sqlalchemy.sql.functions import func
Forgive me if I am wrong.
As an option, you could try executing an SQL Query through Flask.
db.session.execute('select number, count(sms_accounting.id) from cellphone join sms_accounting on sms_accounting.cellphone_id = cellphone.id');
You can easily add the time based filter using where.
Regarding the AttributeError, are you sure you are using the 'func' method properly? The correct usage can be found on this unrelated answer at https://stackoverflow.com/a/4086229/4854064. It might be that you accidentally called func as a method of the query object.
I have Django project with two database models: Device and DeviceTest.
Every device in system should walk through some test stages from manufacturing to sale. And therefore many DeviceTest objects are connected to Device object through foreign key relationship:
class Device(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
name = models.CharField(max_length=255)
class DeviceTest(models.Model):
device = models.ForeignKey(Device)
created_at = models.DateTimeField(auto_now_add=True)
status = models.CharField(max_length=255)
tester = models.CharField(max_length=255)
action = models.CharField(max_length=255)
In my project there 2 kind of pages:
1) page with all tests for individual device
2) page with all devices with their latest status and action
Now I'm trying to optimize 2) page. To get latest test data I use this code:
status_list = []
last_update_list = []
last_action_list = []
for dev in device_list:
try:
latest_test = DeviceTest.objects.filter(device_id=dev.pk).latest('created_at')
status_list.append(latest_test.status)
last_update_list.append(latest_test.created_at)
last_action_list.append(latest_test.action)
except ObjectDoesNotExist:
status_list.append("Not checked")
last_update_list.append("Not checked")
last_action_list.append("Not checked")
For now in my database ~600 devices and ~4000 tests. And this is the main bottleneck in page loading.
What are the ways to speed up this calculation?
I came up with idea of adding extra field to Device model: foreign key to its last DeviceTest. In this scenario there wouldn't be any complicated requests to database at all.
And now I have a few questions:
Is it a good practice to add redundant field to model?
Is it possible to write migration rule to fill this redundant field to all current Devices?
And the most important, what are other choices to speed up my calculations?
id_list = [dev.id for dev in device_list]
devtests = DeviceTest.objects.filter(
device_id__in=id_list).order_by('-created_at').distinct('device')
That should give you, in one database call, in devtests only the latest entries for each device_id by create_at value.
Then do your loop and take the values from the list, instead of calling the database on each iteration.
However, it could also be a good idea to denormalize the database, like you suggested. Using "redundant fields" can definitely be good practice. You can automate the denormalization in the save() method or by listening to a post_save() signal from the related model.
Edit
First a correction: should be .distinct('device') (not created_at)
A list comprehension to fetch only the id values from the device_list. Equivalent to Device.objects.filter(...).values_list('id', flat=True)
id_list = [dev.id for dev in device_list]
Using the list of ids, we fetch all related DeviceTest objects
devtests = DeviceTest.objects.filter(device_id__in=id_list)
and order them by created_at but with the newest first -created_at. That also means, for every Device, the newest related DeviceTest will be first.
.order_by('-created_at')
Finally, for every device we only select the first related value we find (that would be the newest, because we sorted the values that way).
.distinct('device')
Additionally, you could also combine the device id and DeviceTest lookups
devtests = DeviceTest.objects.filter(device_in=Device.objects.filter(...))
then Django would create the SQL for it to do the JOIN in the database, so you don't need to load and loop the id list in Python.
I have two tables in many to many relationship:
class Association(db.Model):
__tablename__ = 'association'
club_id = db.Column(db.Integer, db.ForeignKey('clubs.id'), primary_key=True)
student_id = db.Column(db.Integer, db.ForeignKey('students.id'), primary_key=True)
joined_date = db.Column(db.String)
assoc_student = db.relationship("Student")
class Club(db.Model):
__tablename__ = 'clubs'
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String)
location = db.Column(db.String)
club_assoc = db.relationship("Association")
class Student(db.Model):
__tablename__ = 'students'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
age = db.Column(db.String)
gender = db.Column(db.String)
Questions:
1) What is the difference between these two queries?
students = db.session.query(Association).filter_by(club_id='1')
students = Association.query.filter_by(club_id='1')
They seem to give the same result!
2) I'm trying to get a list of students with certain age but this following query doesn't work:
db.session.query(Association).filter_by(Association.club_id=='1', Association.assoc_student.age=='15')
But I get this error:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Association.assoc_student has an attribute 'age'
That's why I'm using this one:
db.session.query(Student).join(Association).filter(Association.club_id=='1', Student.age=='15')
Is there a better way to do this without "join"? Maybe with using "backref"!?
1) What is the difference between these two queries?
They do almost the same thing. Former is the way to query objects provided with SQLAlchemy (library Flask uses to access database).
Latter is the convenient way to query models added by Flask-SQLAlchemy library. It makes your queries more readable + extends query with few useful methods. Take a look at source of the flask_sqlalchemy.BaseQuery class to see them: get_or_404(), first_or_404() and paginate().
Usually you want to use latter method to query objects.
2) I'm trying to get a list of students with certain age but this following query doesn't work.
There are two things here:
Be aware about the difference between filter() and filter_by() methods. In your example you try to use filter_by() with SQL expressions instead of kwargs, which is incorrect.
When you're using filter() you can't specify columns over a relationships (like Association.assoc_student.age). The only allowed format is ModelName.column_name. That's why it fails.
Is there a better way?
Your second approach is absolutely correct and fine to use. I don't think there is a better way to do it. Alternatively you can use code below to avoid importing db (if you define query in another file):
Student.query.join(Association).filter(Association.club_id == '1', Student.age == '15')