Sqlalchemy duplicate entry when creating entries dynamically - flask

In my application, I am using SQLAlchemy (via MySQL) with Flask, and in my current situation I need to create entries dynamically. For example, while creating an instance from table A, my script will see that we need to create a related entry B, so after creating B, it will attempt to assign B to one of the relationship attributes of the instance for A.
Here are some example models:
a_b = db.Table('a_b',
db.Column('a_id', db.Integer,
db.ForeignKey('a.id',
onupdate="CASCADE",
ondelete="CASCADE"),
primary_key=True),
db.Column('b_id', db.Integer,
db.ForeignKey('b.id',
onupdate="CASCADE",
ondelete="CASCADE"),
primary_key=True))
class ModelA(db.Model):
__tablename__ = 'a'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
model_bs = db.relationship('ModelB', secondary=a_b,
back_populates='model_as')
class ModelB(db.Model):
__tablename__ = 'b'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
model_as = db.relationship('ModelA', secondary=a_b,
back_populates='model_bs')
The problem I'm running into is an SQL duplicate entry exception, after running something like this:
from base import db
from base.database.models.models import ModelA, ModelB
def creation_test():
with db.session.no_autoflush:
# Start creating A
a = ModelA()
a.name = 'A Example'
# See that we need B, so create B
b = ModelB()
b.name = 'B Example'
b.model_as = []
db.session.add(b)
db.session.flush((b,))
# Use B in the relationship from A
a.model_bs = [b]
db.session.add(a)
db.session.flush((a,))
db.session.commit()
creation_test()
It will spit out an exception like:
sqlalchemy.exc.IntegrityError: (_mysql_exceptions.IntegrityError) (1062, "Duplicate entry '1-1' for key 'PRIMARY'") [SQL: u'INSERT INTO a_b (a_id, b_id) VALUES (%s, %s)'] [parameters: (1L, 1L)]
Is there something wrong with how the entries are being added to the session and/or how they are being flushed? Is it even possible to instantiate entries like this?
I appreciate any feedback.

After updating my version of SQLAlchemy and testing a few things, it seemed like I was able to solve this by removing the flushes in between creating each instance, and instead just perform a single flush before committing the session:
from base import db
from base.database.models.models import ModelA, ModelB
def creation_test():
with db.session.no_autoflush:
# Start creating A
a = ModelA()
a.name = 'A Example'
# See that we need B, so create B
b = ModelB()
b.name = 'B Example'
b.model_as = []
db.session.add(b)
# Use B in the relationship from A
a.model_bs = [b]
db.session.add(a)
db.session.flush()
db.session.commit()
creation_test()

Related

Have a custom primary key for users.id in flask-security

I want to have a custom key for the field id, for example, id_user, I've tried the following
class UserModel(db.model, UserMixin)
...
#property
def id(self):
return self.id_user
But couldn't make it work. When I try to login it sends me this message:
{
"message": "You don't have the permission to access the requested resource. It is either read-protected or not readable by the server."
}
I ended up with a nasty solution. I cloned the UserModel object, added a duplicated field for id, with the custom key I needed and told Flask-Security to use that object as the UserModel This is the function code I used:
def clone_model(model):
data = model
attr = getattr(model, "id_user")
setattr(data, "id", attr)
return data
cUserModel = clone_model(UserModel)
user_datastore = SQLAlchemyUserDatastore(db, cUserModel, Roles)
security = Security(app, user_datastore)
Hope someone find it useful
For anybody who is using Flask-Security-Too looking to change the default id column, here is my solution.
Define the User and Role Tables as usual and just define the id column depending on your preference.
class Role(db.Model, FsRoleMixin):
__tablename__ = 'role'
id = Column(String, primary_key=True)
class Users(db.Model, FsUserMixin):
__tablename__ = 'users'
id = Column(String, primary_key=True)
For me, I need to use String primary_key Column. For that to I needed to change the many-to-many relationship table (roles_users). Below is the code snippet for the same.
# Initialize the db object
db = SQLAlchemy()
# import this Class
from flask_security.models.fsqla_v2 import FsModels
# call this after creating the db object
FsModels.db = db
FsModels.user_table_name = 'users' # If you want a different table name than the default
FsModels.role_table_name = 'role'
# Create the relationship table as per your preferences
FsModels.roles_users = db.Table(
"roles_users",
Column("user_id", String, ForeignKey("users.id")),
Column("role_id", String, ForeignKey("role.id")),
)

SQLAlchemy many-to-many relationship updating association object with extra column

In my application, a 'set' can have a number of 'products' associated with it. Products listed against a set must have quantities defined. For this many-to-many relationship I have followed the SQLAlchemy documentation to use an association table with an additional column (quantity).
I am trying to create a form where the user can assign products and quantities against a given set. Both the sets and products already exist in the database. The data from the form are:
set.id
product.id
quantity
This works to create a new association (e.g. set 1 is 'linked' to product 3 with quantity=XYZ) but I get an integrity error when I try to update an existing record.
I can manually add a relationship/record (dummy data) or within the Flask view function as follows:
s = Set.query.get(2)
p = Product.query.get(3)
a = Set_Product_Association(set=s, product=p, quantity=23)
db.session.add(a)
db.session.commit()
Updating the record (different quantity) manually as follows works:
s.products[0].quantity = 43
db.session.add(s)
db.session.commit()
However when I use the code from the first block instead (with the aim to update the quantity field for a given, existing set and product ID), i.e.:
a = Set_Product_Association(set=s, product=p, quantity=43)
I get an integrity error
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: set_product_association.set_id, set_product_association.product_id [SQL: 'INSERT INTO set_product_association (set_id, product_id, quantity) VALUES (?, ?, ?)'] [parameters: (2, 3, 43)]
I assume this is to tell me that I'm trying to append a new record rather than updating the existing one.
How should I approach this? The 'manual' method works but relies on working out the correct index in the list (i.e. for the correct product.id).
Curiously, if I use form.popluate_obj(set) in my Flask view function to process the form data as described in my question here, I can update fields but not create new 'associations'. Unfortunately, I don't know what goes on behind the scenes there....
My models are defined like so:
class Set_Product_Association(db.Model):
__tablename__ = 'set_product_association'
set_id = db.Column(db.Integer, db.ForeignKey('sets.id'), primary_key=True)
product_id = db.Column(db.Integer, db.ForeignKey('products.id'), primary_key=True)
quantity = db.Column(db.Integer)
product = db.relationship("Product", back_populates="sets")
set = db.relationship("Set", back_populates="products")
class Set(db.Model):
__tablename__ = 'sets'
id = db.Column(db.Integer, primary_key=True)
products = db.relationship("Set_Product_Association",
back_populates="set")
class Product(db.Model):
__tablename__= 'products'
id = db.Column(db.Integer, primary_key=True)
part_number = db.Column(db.String(100), unique=True, nullable=False)
sets = db.relationship("Set_Product_Association",
back_populates="product")
Edit:
I've also tried reversing the operation as suggested here:
s = Set.query.get(2)
a = Set_Product_Association()
a.quantity = 43
a.product = Product.query.get(3)
a.set = s
db.session.commit()
But I still get an error:
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: set_product_association.set_id, set_product_association.product_id [SQL: 'INSERT INTO set_product_association (set_id, product_id, quantity) VALUES (?, ?, ?)'] [parameters: (2, 3, 43)]
You get an integrity error because you are trying to create a new object with the same primary keys.
This:
a = Set_Product_Association(set=s, product=p, quantity=43)
Does not update, but create.
If you want to update the actual row in the table, you need to update the existing one:
assoc = Set_Product_Association.query.filter_by(set=s, product=p).first()
assoc.quantity = 43
db.session.commit()
Also, from the documentation it is advised to not use a model but an actual table.

How to implement "contain any" in flask/sqlalchemy?

The contains operation in SQLAlchemy only accepts one Model object instead of a list of objects. If I want to create a filter that accepts containing any of a group of objects, is there a more SQL-style way than creating multiple filters using contains and combining them with union?
For example, see the following code:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(100))
son = db.relationship('Son', backref = 'parent', lazy = 'dynamic')
def __repr__(self):
return '<"%s">' % self.name
class Son(db.Model):
id = db.Column(db.Integer, primary_key = True)
name = db.Column(db.String(1000))
user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
def __repr__(self):
return '<"%s">' % self.name
db.create_all()
son1 = Son(name = '1 a 1')
son2 = Son(name = '1 b 1')
son3 = Son(name = '1 c 1')
son4 = Son(name = '2 a 2')
son5 = Son(name = '2 b 2')
user0 = User(name = 'w1')
user1 = User(name = 'w2')
user2 = User(name = 'w3')
user0.son.append(son1)
user0.son.append(son2)
user1.son.append(son3)
user1.son.append(son4)
user2.son.append(son5)
db.session.add(son1)
db.session.add(son2)
db.session.add(son3)
db.session.add(son4)
db.session.add(son5)
db.session.add(user0)
db.session.add(user1)
db.session.add(user2)
db.session.commit()
son_query = Son.query.filter(Son.name.ilike('%a%'))
son_query_all = son_query.all()
print son_query.all()
user_query = User.query.filter(User.son.contains(son_query_all[0])).union(*[User.query.filter(User.son.contains(query)) for query in son_query_all[1:]])
print user_query.all()
The example firstly creates two models: User and Son, and then creates 3 User instances and 5 Son instances. user0 contains son1 and son2, user1 contains son3 and son4, and user2 contains son5. Note that the name of son1 and son4 are both like %a%. Now I want to select all User instances containing Son instances whose name likes %a%.
The current method is to select all Son instances in son_query_all, and then selects User instances containing individual desired Son instances, and then combines the selecting result using union. Is there a more SQL-style way for SQLAlchemy to select the same? For example, is there anything like contains_any so that the last query can be changed into something like
user_query = User.query.filter(User.son.contains_any(son_query_all))
Note that of course I can define a custom contains_any function for the same purpose using the union and contains operation. My question is whether there is a more efficient way than simply union all contains-ed?
The correct way to solve this kind of filtering is to use JOIN. Basically you join Son table with User table and filter by joined entity's field.
In SQLAlchemy you can express it like this:
User.query.join(Son).filter(Son.name.ilike('%a%'))
And it will produce following SQL:
SELECT * FROM user
JOIN son ON user.id = son.user_id
WHERE lower(son.name) LIKE lower('%a%')

Turbogears2 AdminController throwing an error with a many-to-many relationship

I'm having an issue with turbogears admin controller throwing an error when I try to edit the User, ShoppingItem or ShoppingList items (code below). The error coming up is AttributeError: 'function' object has no attribute 'primary_key'. The Local Variables in frame always come back as the same:
mapper
<Mapper at 0x3719810; ShoppingList>
fields
['id']
self
<sprox.sa.provider.SAORMProvider instance at 0x03E537B0>
value
<bound method OrderedProperties.items of <sqlalchemy.util._collections.OrderedProperties object at 0x037199F0>>
entity
<class 'insertmealhere.model.shoppinglist.ShoppingList'>
field_name
'items'
I'm having trouble figuring out what is different between this and the other many-to-many relationships that are configured elsewhere in the code and are not throwing this error. I'm running Turbogears 2.2 on Python 2.7.8 currently on a windows 8.1 system. Any help is greatly appreciated.
list_item_table = Table("list_item_table", metadata,
Column('item_id', Integer, ForeignKey('shopping_item.id', onupdate="CASCADE", ondelete="CASCADE"), primary_key=True),
Column('list_id', Integer, ForeignKey('shopping_list.id', onupdate="CASCADE", ondelete='CASCADE'), primary_key=True))
class ShoppingItem(DeclarativeBase):
__tablename__ = "shopping_item"
id = Column(Integer, primary_key=True)
name = Column(String(50))
quantity = Column(String(5))
measure = Column(String(10))
# less important optional parameters that will be useful for users
brand = Column(String(50))
list_id = Column(Integer, ForeignKey('shopping_list.id'))
shopping_list = relation("ShoppingList", secondary=list_item_table, backref="items")
def get_owner_id(self):
return self.list.user_id
#classmethod
def delete_list(cls, id, user_id):
item = DBSession.query(cls).filter_by(id=id).one() # get the item from the given ID
if item.get_owner_id() == user_id: # owned by current user
DBSession.delete(item) # delete from shopping list
return True
flash(_("You do not have authorization to perform that action."))
return False
class ShoppingList(DeclarativeBase):
__tablename__ = 'shopping_list'
id = Column(Integer, primary_key=True)
date = Column(Date, index=True, nullable=False)
static = Column(Boolean, nullable=False, default=False)
# static is true if the items from the meal plan have been imported into the shopping list. Once done you can edit
# the items in the shopping list, remove items, etc. Until the shopping list is made static it is impossible to edit
# the items that are imported from the schedule as they do not exist in the shopping list! (and we do not want to
# edit them in the recipe!
user_id = Column(Integer, ForeignKey('tg_user.user_id'))
user = relation("User", backref="shopping_lists")
date_user_list = Index('date_user_list', 'date', 'user_id')
Maybe it's the list_id = Column(Integer, ForeignKey('shopping_list.id')) in the ShoppingItem model class that's confusing SQLAlchemy?

set the insert order of a many to many sqlalchemy flask app sqlite db

My Goal
I want to record the order of manys upon insert of data to my table (e.g. Clump-see tables below). The orderinglist module is really great, but how do i apply it to the intermediary table (named clump_syntaxs) between my many-to-many? anyone done this before and have a good example?
problem re-stated
How do i apply ordering upon insert to my many to many. Everything I try using the intermediary table-clump_syntaxs table crashes (sorry for the weird names!).
The following code (reduced for brevity) works! except that it only allows for a syntax to have a unique position (instead of a position for every Clump instance), and I am guessing I need the position variable to be on the clump_syntaxs table.all tables are sqlite
my intermediary table
from sqlalchemy.ext.orderinglist import ordering_list
clump_syntaxs = db.Table('clump_syntaxs',
db.Column('syntax_id', db.Integer, db.ForeignKey('syntax.id')),
db.Column('clump_id', db.Integer, db.ForeignKey('clump.id')),
)
add a clump and order syntax tables
class Clump(db.Model):
id = db.Column(db.Integer, primary_key=True)
syntaxs = db.relationship('Syntax', secondary=clump_syntaxs,
backref=db.backref('clumps', lazy='dynamic'),order_by="Syntax.position",
collection_class=ordering_list('position'))
class Syntax(db.Model):
id = db.Column(db.Integer, primary_key=True)
jobs = db.relationship('Jobs',lazy='dynamic', backref='jobhistory')
position = db.Column(db.Integer)
#Jobs table not included
Yes, you should move position field to the intermediary table ClumpSyntax, and take advantage of association_proxy() in Clump table.
import sqlalchemy.ext.associationproxy import association_proxy
class Syntax(db.Model):
id = db.Column(db.Integer, primary_key=True)
jobs = db.relationship('Jobs',lazy='dynamic', backref='jobhistory')
#position = db.Column(db.Integer) # moved to ClumpSyntax
#Jobs table not included
class ClumpSyntax(db.Model):
syntax_id = db.Column('syntax_id', db.Integer, db.ForeignKey('syntax.id'))
syntax = relationship(Syntax)
clump_id = db.Column('clump_id', db.Integer, db.ForeignKey('clump.id'))
position = db.Column(db.Integer)
# this constructor is very necessary !
def __init__(self, syntax =None):
self.syntax = syntax
class Clump(db.Model):
id = db.Column(db.Integer, primary_key=True)
_syntaxs = db.relationship('ClumpSyntax',
order_by=[ClumpSyntax.position],
collection_class=ordering_list('position'))
syntaxs = association_proxy('_syntaxs','syntax')
My similar request was satisfied quite well by this way, based on the article this and this. You can test it by code like below:
session= some_code_to_get_db_session()
syn1= Syntax()
syn2= Syntax()
syn3= Syntax()
session.add(syn1)
session.add(syn2)
session.add(syn3)
clump= Clump()
session.add(clump)
clump.syntaxs.append(syn1)
clump.syntaxs.append(syn2)
clump.syntaxs.append(syn3)
session.commit()
session.query(ClumpSyntax).count() # print out 3
session.query(Syntax).count() # print out 3