How would you model these database relationships? (User, Competence, Interest) - django

I'm working with a Django system and the thing is pretty simple actually, but one thing got me bothered and I thought that must be a better way to do it.
Basically it's an internal tool for searching the developers the company has available to allocate them in projects based on their skills and interests.
So let's say Employee John can add a Topic React to his list of competence and select his skills levels from 1 to 5 stars, meaning 1 = total beginner and 5 = expert. He can also select his level of interest from 1 to 5. So you can have a list like below
The way this is designed at DB is like this:
Table user with all relevant user information
Table knowledge_topic with columns [id, label], where label is just the name of the language/tech
Table knowledge_level with columns[id, knowledge_level, interest_level, topic_id, user_id] where topic_id makes the relationship to knowledge_topic table and user_id to user table
The competence list endpoint can give a result like this for example:
[
{
"id": 2,
"topic": {
"id": 8,
"label": "Docker"
},
"interest_level": 4,
"knowledge_level": 2,
"user": 2
},
{
"id": 5,
"topic": {
"id": 9,
"label": "Flask"
},
"interest_level": 1,
"knowledge_level": 2,
"user": 2
},
{
"id": 1,
"topic": {
"id": 1,
"label": "React"
},
"interest_level": 4,
"knowledge_level": 4,
"user": 2
},
{
"id": 16,
"topic": {
"id": 3,
"label": "SCSS"
},
"interest_level": 2,
"knowledge_level": 1,
"user": 1
},
{
"id": 136,
"topic": {
"id": 2,
"label": "Django"
},
"interest_level": 1,
"knowledge_level": 1,
"user": 3
},
{
"id": 137,
"topic": {
"id": 9,
"label": "Flask"
},
"interest_level": 1,
"knowledge_level": 1,
"user": 3
},
{
"id": 138,
"topic": {
"id": 3,
"label": "SCSS"
},
"interest_level": 1,
"knowledge_level": 1,
"user": 3
}
]
As you can see, the "uniqueness" here is tied to the combination of user + knowledge. So every time I have an user add a new competence, I'll add an extra row to knowledge_competence table, which to me feels unnecessary.
Is there a better way of doing this or am I overthinking it? I was thinking perhaps creating a table interest with fields [id, level] and competence also with [id, level] and perhaps a "label" field in both tables just to have a text description what the levels mean. Then I can make a relationship chain of user -> topic -> competence and user -> topic -> interest. That way both tables have always 5 rows (the levels of competence and interest), topics always restricted to the universe of languages all employees have in that company and users to all employees. There'll be barely any data to add in DB, only when there is really a new employee or topic. Would this be a good design?

I'd build a table for "devs", and a table for "skills".
Then create a table "dev_skill", to associate devs to skills, with extra data : "interest" and "experience"
There is an exemple on how to do that with django : Django's ManyToMany Relationship with Additional Fields

Related

DRF: Set ChoiceField default programmatically from DB query

I asked a similar question here about setting the default value of a TextChoices field on a model. Now I'm taking it to the next level.
I've moved the choices settings from the model to the serializer ChoiceField. The site is Django/DRF/django_tenants backend with Vue frontend. I want the front/backends to be using the same set of choices and, just as important, the same default values. So I created an OptionGroup table. Here are two sample rows from that table (some options omitted):
{
"results": {
"id": 1,
"name": "payment_method",
"defined_in": "option_table",
"default_option": "5",
"options": [
{
"id": 2,
"name": "Bank Transfer",
"value": "2"
},
{
"id": 5,
"name": "Credit Card",
"value": "5"
},
{
"id": 8,
"name": "PayPal",
"value": "8"
}
]
}
}
{
"results": {
"id": 3,
"name": "invoice_type",
"defined_in": "class",
"app_label": "invoices",
"choices_class": "InvoiceType",
"default_option": "standard",
"options": [
{
"name": "Standard",
"value": "standard"
},
{
"name": "Retainer",
"value": "retainer"
},
{
"order": 1,
"name": "Estimate",
"value": "estimate"
}
]
}
}
The first one (defined_in = "option_table") has its options defined in a related Option model. The second one has its options defined in a subclass of TextChoices in the backend code. In addition to keeping the front/backend in sync for options and default values, this will also allow users to set which option for each select list field they want as the default value. Note the default_option field above.
I created a class, extended from the ChoiceField class (from DRF), and added logic to the __init__() method to get the default value from the OptionGroup model and set it for the serializer field being defined.
class CustomChoiceField(serializers.ChoiceField):
def __init__(self, choices, **kwargs):
option_group = kwargs.pop('option_group', None)
if option_group and 'default' not in kwargs:
group = OptionGroup.objects.filter(name__iexact=option_group)
if group and group.default_option:
kwargs['default'] = group.default_option
super().__init__(choices, **kwargs)
And this is how it is being used:
type = CustomChoiceField(
choices=choices.InvoiceType.choices,
option_group='invoice_type',
)
The problem is that the active schema at that point is "public" instead of the actual tenant schema. It seems the __init__() for my custom class is being executed too early in the request process - before django_tenants has connected to the proper schema.
Does anyone have a suggestion as to how this can be accomplished, or is what I'm trying to do just not possible?

DynamoDB LIKE '%' (contains) search over an array of objects using a key from the object, NodeJS

I am trying to use a "LIKE" search on DynamoDB where I have an array of objects using nodejs.
Looking through the documentation and other related posts I have seen this can be done using the CONTAINS parameter.
My question is - Can I run a scan or query over all of my items in DynamoDB where a value in my object is LIKE "Test 2".
Here is my DynamoDB Table
This is how it looks as JSON:
{
"items": [
{
"description": "Test 1 Description",
"id": "86f550e3-3dee-4fea-84e9-30df174f27ea",
"image": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/86f550e3-3dee-4fea-84e9-30df174f27ea.jpg",
"live": 1,
"status": "new",
"title": "Test 1 Title"
},
{
"description": "Test 2 Description",
"id": "e17dbb45-63da-4567-941c-bb7e31476f6a",
"image": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/e17dbb45-63da-4567-941c-bb7e31476f6a.jpg",
"live": 1,
"status": "new",
"title": "Test 2 Title"
},
{
"description": "Test 3 Description",
"id": "14ad228f-0939-4ed4-aa7b-66ceef862301",
"image": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/14ad228f-0939-4ed4-aa7b-66ceef862301.jpg",
"live": 1,
"status": "new",
"title": "Test 3 Title"
}
],
"userId": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
}
I am trying to perform a scan / query which will look over ALL users (every row) and look at ALL items and return ALL instances where description is LIKE "Test 2".
I have tried variations of scans as per the below:
{
"TableName": "my-table",
"ConsistentRead": false,
"ExpressionAttributeNames": {
"#items": "items",
},
"FilterExpression": "contains (#items, :itemVal)",
"ExpressionAttributeValues": {
":itemVal":
{
"M": {
"description": {
"S": "Test 2 Description"
},
"id": {
"S": "e17dbb45-63da-4567-941c-bb7e31476f6a"
},
"image": {
"S": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/e17dbb45-63da-4567-941c-bb7e31476f6a.jpg"
},
"live": {
"N": "1"
},
"status": {
"S": "new"
},
"title": {
"S": "Test 2 Title"
}
}
}
}
}
The above scan works but as you can see I am passing in the whole object as an ExpressionAttributeValues, what I want to do is just pass in the description for example something like the below (which doesnt work and returns no items found).
{
"TableName": "my-table",
"ConsistentRead": false,
"ExpressionAttributeNames": {
"#items": "items.description",
},
"FilterExpression": "contains (#items, :itemVal)",
"ExpressionAttributeValues": {
":itemVal":
{
"S": "Test 2"
}
}
}
Alternatively, would it be better to create a separate table where all the items are added and they are linked via the userId? I was always under the impression there should be one table per application but in this instance I think if I had all the item data at the top level, scanning it would be a lot safer and faster.
So with nearly 200 views since posting and no responses I have come up with a solution that does not immediately solve the initial problem (I honestly do not think it can be solved) but have come up with an alternative approach.
Firstly I do not want two tables as this seems overkill, and I do not want the aws costs associated with two tables.
This has lead me to restructure the primary keys with prefixes which I can search over using the "BEGINS_WITH" dynamodb selector query.
Users will be added as U_{USER_ID} and items will be added as I_{USER_ID}_{ITEM_ID}, this way I only have one table to manage and pay for and this allows me to run BEGINS_WITH "U_" to get a list of users or "I_" to get a list of items.
I will then flatten the item data as strings so I can run "contains" searches on any of the item data. This also allows me to run a "contains {USER_ID}" search on the primary keys for items so I can get a list of items for a particular user.
Hope this helps anyone who might come up against the same issue.

How to apply filter in loopback while including another model via "hasMany" relation?

I am using loopback 3. I have two models project and project members.
Project has "hasMany" relation with project members.
So far, I use http://localhost:3000/api/v1/Projectsfilter[include]=projectMember which gives me result like below :-
{
"projectName": "project 1 ",
"clientNames": {},
"projectShortCode": "string",
"projectMember": [
{
"projectId": 1,
"userId": 1,
"id": 1
},
"projectName": "project 2",
"clientNames": {},
"projectShortCode": "string",
"projectMember": [
{
"projectId": 1,
"userId": 2,
"id": 2
}
}
How do I apply filter on api that I get only those project in result which has userId = 1 ?
I'm afraid you can't filter projects by the related model property.
But what you could do after the api call is to filter your array, eg. you can call something like this:
api.makeRequest(projectsURL).filter(project => project.userId === 1);
Here you can find more info about that issue:
https://github.com/strongloop/loopback/issues/1754
Loopback Filter Based On Related Model Properties

Django JSONField and searching through the list of dictionaries using ILIKE

Is it possible to search against one key value in the list of dictionaries using ILIKE (icontains) operator? My json field looks like this:
object = MyModel()
object.json_data = [
{
"type": 1,
"results": [
{
"score": 1,
"comment": "Some text comment 1",
},
{
"score": 2,
"comment": "Some text comment 2",
},
{
"score": 3,
"comment": "Some text comment 3",
}
]
},
{
"type": 2,
"results": [
{
"score": 4,
"comment": "Some text comment 4",
},
{
"score": 5,
"comment": "Some text comment 5",
},
{
"score": 6,
"comment": "Some text comment 6",
}
]
}
]
object.save()
And now, how to write the query to search in a "comment" key?
MyModel.objects.filter(json_data__??__results__??__comment__icontains="text comment")
I'm using Django 1.9.
Thanks!
this works for me (note the [])
query = User.objects.filter(data__campaigns__contains=[{'key': 'value'}])
You should be able to search simply by chaining it, django style:
MyModel.objects.filter(json_data__results__contains={"comment":"text comment"})
check out the documentation for JSON field in Django 1.9:
https://docs.djangoproject.com/es/1.9/ref/contrib/postgres/fields/#querying-jsonfield
which includes contains lookup:
https://docs.djangoproject.com/es/1.9/ref/contrib/postgres/fields/#std:fieldlookup-hstorefield.contains
If this doesn't work for case-insensitive, then I would see what query it produces, and simply rework it with extra where:
MyModel.objects.extra(where=["json_data->>'results'->'comment' ILIKE %s"], params=["%text comment%"])
or you can use the specific symbols for json as stated in postgres documentation, like <#
http://www.postgresql.org/docs/9.5/static/functions-json.html

Ember - Only update fields returned in response JSON

we would like to add lazy loading functionality to our ember project, but it looks like ember will always override fields not returned by the response JSON with NULL. First I get a list of users:
GET https://example.com/users
{
"users": [
{
"id": 1,
"name": 'user1',
"email": 'email#user1.com',
"images": [],
"posts": []
},
{
"id": 2,
"name": 'user2',
"email": 'email#user2.com',
"images": [],
"posts": []
},
{
"id": 3,
"name": 'user3',
"email": 'email#user3.com',
"images": [],
"posts": []
},
]
}
This provides a minimal set of user information, with two empty hasMany relations "images" and "posts".
Now, if somebody want to see the users posts or images he would click a button which triggers the lazy loading:
GET https://example.com/userImages/1
{
"user": {
"id": 1,
"images": [1,2,3,4]
},
"images": [
{
"id": 1,
"name": "image1",
"path" "path/to/img1/"
},
{
"id": 2,
"name": "image2",
"path" "path/to/img2/"
},
{
"id": 3,
"name": "image3",
"path" "path/to/img3/"
},
{
"id": 4,
"name": "image4",
"path" "path/to/img4/"
}
]
}
To reduce traffic to a minimum, only the newly loaded information is included. After the adapter has deserialzed and pushed the new data to the store, all fields from User1 which are not included in the payload (name, email) are set to NULL in the ember store (tested with store.pushPayload('model', payload)).
Is there a possibility to update only incoming data? Or is there a common best practice to handle such a case?
Thanks in advance for your help
EDIT:
One possibility would be to extend the ember-data "_load()" function with the block
for (var key in record._data) {
var property = record._data[key];
if( typeof(data[key]) == 'object' && data[key] == null ) {
data[key] = property;
}
}
But this is the worst possible solution I can imagine.
I think what you want is the store's update method. It's like push (or pushPayload), except that it only updates the data that you give it.
Your property returns a promise and that promise returns whatever came back from the server.
foobar: function() {
return this.store.find('foobar');
}
When the promise resolves, you have two versions of the data, the one already rendered in the client (dataOld) and the one that just returned from the backend (dataNew). To update the client without removing what hasn't change, you have to merge the old and the new. Something along the lines of:
foobar: function() {
var dataOld = this.get('foobar');
return this.store.find('foobar').then(function(dataNew) {
return Ember.$.merge(dataOld, dataNew);
});
}