My question is how do i join unrelated resources that have a similar variable
Both resources have an 8 length VARCHAR variable, both named code.
Due to how to data is constructed I cannot make any assumption that would lead to this being a foreign key relation but I do however need to join this two tables together if they have similar code values, how do I join these resources together to be displayed in tastypie/django?
class CodeDescription(models.Model):
code = models.CharField(db_column='Code', max_length=10)
description = models.CharField(db_column='Description', max_length=255)
class TechnicalDif(models.Model):
code = models.CharField(db_column='Code', max_length=10)
As you can see these tables hold the same sort of value but CodeDescription holds the details of what the code means, but doesn't necessarily have the definition for all the codes, so a foreignkey relation cannot be applied, How would i join these two tables for display using them as a tastypie resource?
The only way I think you can do this is programatically in the dehydrate method of the relevant TastyPie ResourceClass. So for each loaded object you could do a query and set an appropriate value in the bundle.
Alternatively, I suppose you could create a join table, that joins related CodeDescriptions to TechnicalDifs, and populate it programatically with a query that you run over the tables intermittently (e.g. when something changes).
Related
I'm facing a dilemma, I'm creating a new product and I would not like to mess up the way I organise the informations in my database.
I have these two choices for my models, the first one would be to use foreign keys to link my them together.
Class Page(models.Model):
data = JsonField()
Class Image(models.Model):
page = models.ForeignKey(Page)
data = JsonField()
Class Video(models.Model):
page = models.ForeignKey(Page)
data = JsonField()
etc...
The second is to keep everything in Page's JSONField:
Class Page(models.Model):
data = JsonField() # videos and pictures, etc... are stored here
Is one better than the other and why? This would be a huge help on the way I would organize my databases in the futur.
I thought maybe the second option could be slower since everytime something changes all the json would be overridden, but does it make a huge difference or is what I am saying false?
A JSONField obfuscates the underlying data, making it difficult to write readable code and fully use Django's built-in ORM, validations and other niceties (ModelForms for example). While it gives flexibility to save anything you want to the db (e.g. no need to migrate the db when adding new fields), it takes away the clarity of explicit fields and makes it easy to introduce errors later on.
For example, if you start saving a new key in your data and then try to access that key in your code, older objects won't have it and you might find your app crashing depending on which object you're accessing. That can't happen if you use a separate field.
I would always try to avoid it unless there's no other way.
Typically I use a JSONField in two cases:
To save a response from 3rd party APIs (e.g. as an audit trail)
To save references to archived objects (e.g. when the live products in my db change but I still have orders referencing the product).
If you use PostgreSQL, as a relational database, it's optimised to be super-performant on JOINs so using ForeignKeys is actually a good thing. Use select_related and prefetch_related in your code to optimise the number of queries made, but the queries themselves will scale well even for millions of entries.
I'm building a Flask app, which, at startup, should read some number of tsv files, each of which has the same schema, put them in tables (one for each file), and then users will specify which table/file they want to query, and some number of keys.
I'm not sure how to do this, but the best way seems to be to specify one schema and then, once the app starts, read the files and dynamically create tables for each file. I can't find anywhere in the SQLalchemy docs any mention of how to use the same schema multiple times. Perhaps I need to extend my schema class, but i'm not sure how to do this at startup.
Thanks in advance!
-- EDIT --
It looks like this answers half of my question:
Flask-SQLAlchemy. Create several tables with all fields identical
So my question now is: Can you do the above in Flask, and can you do it as the app starts?
You can take 2 approaches.
Sub-classing - You create a base Mixin for schema and subclass it for each concrete tables. This approach is useful, if you expect that in future the schema for different tables might diverge. If a new field needs to be added in only one table you can add it only in sub-class. (variables db, Model etc is used from flask sqlalchemy quickstart)
class BaseMixin(object):
name = db.Column(String(80), unique=True)
field2 = db.Column ...
class SubClass1(BaseMixin, db.Model)
pass
class Subclass2(BaseMixin, db.Model)
additional_field_for_subclass2 = db.Column(...
pass
Common table for all - If you are confident that the schema will remain the same for all tables. I would suggest you create one table for all you data, with a additional field data_source which will indicate where the row/data came from.
class CommonTable(db.Model):
data_source = db.Column(String(100) ..)
field1 = ...
field2 = ...
I'm currently creating an equipment management database and need to allow equipment to be associated with other equipment.
Thanks to this stackoverflow question I currently have something akin to the following (vastly simplified):
class Equipment(models.Model):
equipment_title = models.CharField()
relates_to = models.ForeignKey('self')
However, to relate a dynamic number of equipment to other equipment I think I need something like a one-to-many field that doesn't exist natively within Django, e.g. a filter housing may be associated with many filter units, and several filter housings may be associated with a machine tool.
How can I get around this? I'm not sure that it's the right place for a many-to-many field...
A ForeignKey is a one-to-many relationship, defined on the "many" side. Since your relationship is pointing to self anyway, it already does what you want.
Some of my models are only unique in a combination of keys. I don't want to use an auto-numbering id as the identifier as subsets of the data will be exported to other systems (such as spreadsheets), modified and then used to update the master database.
Here's an example:
class Statement(models.Model):
supplier = models.ForeignKey(Supplier)
total = models.DecimalField("statement total", max_digits=10, decimal_places=2)
statement_date = models.DateField("statement date")
....
class Invoice(models.Model):
supplier = models.ForeignKey(Supplier)
amount = models.DecimalField("invoice total", max_digits=10, decimal_places=2)
invoice_date = models.DateField("date of invoice")
statement = models.ForeignKey(Statement, blank=True, null=True)
....
Invoice records are only unique for a combination of supplier, amount and invoice_date
I'm wondering if I should create a slug for Invoice based on supplier, amount and invoice_date so that it is easy to identify the correct record.
An example of the problem of having multiple related fields to identify the right record is django-csvimport which assumes there is only one related field and will not discriminate on two when building the foreign key links.
Yet the slug seems a clumsy option and needs some kind of management to rebuild the slugs after adding records in bulk.
I'm thinking this must be a common problem and maybe there's a best practice design pattern out there somewhere.
I am using PostgreSQL in case anyone has a database solution. Although I'd prefer to avoid that if possible, I can see that it might be the way to build my slug if that's the way to go, perhaps with trigger functions. That just feels a bit like hidden functionality though, and may cause a headache for setting up on a different server.
UPDATE - after reading initial replies
My application requires that data may be exported, modified remotely, and merged back into the master database after review and approval. Hidden autonumber keys don't easily survive that consistently. The relation invoices[2417] is part of statements[265] is not persistent if the statement table was emptied and reloaded from a CSV.
If I use the numeric autonumber pk then any process that is updating the database would need to refresh the related key numbers or by using the multiple WITH clause.
If I create a slug that is based on my 3 keys but easy to reproduce then I can use it as the key - albeit clumsily. I'm thinking of a slug along the lines:
u'%s %s %s' % (self.supplier,
self.statement_date.strftime("%Y-%m-%d"),
self.total)
This seems quite clumsy and not very DRY as I expect I may have to recreate the slug elsewhere duplicating the algorithm (maybe in an Excel formula, or an Access query)
I thought there must be a better way I'm missing but it looks like yuvi's reply means there should be, and there will be, but not yet :-(
What you're talking about it a multi-column primary key, otherwise known as "composite" or "compound" keys. Support in django for composite keys today is still in the works, you can read about it here:
Currently Django models only support a single column in this set,
denying many designs where the natural primary key of a table is
multiple columns [...] Current state is that the issue is
accepted/assigned and being worked on [...]
The link also mentions a partial implementation which is django-compositekeys. It's only partial and will cause you trouble with navigating between relationships:
support for composite keys is missing in ForeignKey and
RelatedManager. As a consequence, it isn't possible to navigate
relationships from models that have a composite primary key.
So currently it isn't entirely supported, but will be in the future. Regarding your own project, you can make of that what you will, though my own suggestion is to stick with the fully supported default of a hidden auto-incremented field that you don't even need to think about (and use unique_together to enforce the uniqness of the described fields instead of making them your primary keys).
I hope this helps!
No.
Model needs to have one field that is primary_key = True. By default this is the (hidden) autofield which stores object Id. But you can set primary_key to True at any other field.
I've done this in cases, Where i'm creating django project upon tables which were previously created manually or through some other frameworks/systems.
In reality - you can use whatever means you can think of, for joining objects together in queries. As long as query returns bunch of data that can be associated with models you have - it does not really matter which field you are using for joins. Just keep in mind, that the solution you use should be as effective as possible.
Alan
I'm having some problems with the following...
I have a table with phone numbers. I want to use the same table for both users and companies. A user can have several phone numbers and a company too. So i want a One to many unidirectional relationship with two different join tables. One linking phone numbers to users, the other linking phone numbers to companies.
This is solution following the doctrine2 manual chapter 5.9 found here: (click)
My users entity holds this code:
/** #ORM\ManyToMany(targetEntity="Application\Entity\PhoneNumber")
* #ORM\JoinTable(name="user_phone_number_linker",
* joinColumns={#ORM\JoinColumn(name="user_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\JoinColumn(name="phone_number_id", referencedColumnName="id")}
* )
*/
protected $phone_numbers;
I use a unidirectional one to many because the thing is I can't make a bidirectional one because if I refer back to the user I cannot use the same phone number entity class for the company. Now it all works fine, but when I delete a phone number I get the following error:
An exception occurred while executing 'DELETE FROM phone_number WHERE id = ?' with params {"1":1}:
SQLSTATE[23000]: Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails (database/user_phone_number_linker, CONSTRAINT user_phone_number_linker_ibfk_11 FOREIGN KEY (phone_number_id) REFERENCES phone_number (id))
If I set the ON DELETE CASCADE value manually in the database it works fine, but this is not the idea of using doctrine2 and I think I should be able to solve it within the code without going to my phpMyAdmin panel. Somehow the cascading from the phone number towards the join table should be initiated on deletion, but without making a reference back to the join table from the phone_number entity.
Hope someone smart can help me solve this.
EDIT
In the meantime I learned a lot more about Doctrine2 and reviewing my old question made me realize that this is not a correct way to store several phoneNumbers in one table in the first place. To be able to store user phone numbers and company phone numbers in the same table I should use table inheritance with a discriminator column. The column should hold some user/company discriminator.
Because of this column the doctrine ORM will "know" if that phoneNumber is actually a user or a company phone-number. I need to make two different entity definitions following the single table inheritance mapping principles from the doctrine 2 specs.
One class UserPhoneNumber will have a many-to-one relationship with User the other called CompanyPhoneNumber a one-to-many relationship with Company. I don't necessarily need a join column, the user_id or company_id columns can be in the phone-number table. In the User class the Company association is omitted and in the Company class the User association is omitted (database should allow null values for those columns).
If I do use a join table it is according to the one-to-many unidirectional with join table description in the Doctrine2 specs
READ MORE
Otherwise you can also read more on associations and cascade issues here on this elaborate Doctrine2 in depth website.
As you said, your relation is unidirectional. You've defined a relation from Users to PhoneNumbers. The cascade delete will work when you delete a User, it will remove all rows in user_phone_number_linker because that's the relation you've defined.
If you want to do it the other way, you've got to create a relation from PhoneNumbers to Users. Doctrine needs it to work for you. But you have the problem that the entity is shared by two other entities, Users and Companies.
Keep in mind that entities are objects, not tables. So you could try to create two entities to the same table, one named PhoneNumberUsers and the other PhoneNumberCompanies. This way you'll be able to create the needed relation to do the cascade delete. I haven't tested by myself, but I think it could work.
By the way, you can remove the oncascade parameter on the Users' entity join table. I've the same scenario as you with users and roles, and I haven't used it. I think it's only needed when you want to cascade from entity to entity. I'm not sure about that, but that's what I've been experiencing until now.
My bad,
The phone number user relationship is regarded a Many-To-Many relationship, so if want to remover the phone number I should not only remove the phone number itself, but I have to explicitly remove the phone number from the user as well. So in the Controller like this:
// Remove the phone number user connection from the database
$user->removePhoneNumber($phone_number);
// Remove the phone number from the database
$em->remove($phone_number);
I just thought the unique restriction which makes the relationship to a unidirectional One-To-Many would be enough to make doctrine take care of it. That was not correct.