How can I include nested relation for all potential layers in Loopback 4? - loopbackjs

I'm trying to make a REST API where all the employees in the company are returned in a hierarchical order(Employees that don't have a manager are at the base tree while the people they manage are right under them, and so on.
I'm including the employees[] relation and an identical nested relation in the filter while calling the data from the repository. Currently, the call for employees only happens on the first two layers, and the employees[] relation isn't shown on the third layer so there is no way to see the layers after that.
Here is the part of the code that calls the relations for the first two layers:
filter = {
include: [
{
relation: 'employees',
scope: {
include: [{relation: 'employees'}],
},
},
],
where: {managerId: -1},
};
Is there a way declare this filter so it scales into all layers my hierarchy might have? Or do I have to add as many new scopes as I think I need?

You might not be able to achieve this in a single filter clause. Better fetch underlying employees recursively for each layer.
EDIT
Actually, we can indeed do the recursion in the filter clause itself if we know how many max layers there can be. It will also break the recursion when there are no more child layers. Then your filter clause would look like
let filter = {
include: {
relation: 'employees'
},
where: {managerId: -1},
};
let childLayer = {
relation : 'employees'
};
let previousLayer = filter.include;
for(let i=0; i<=10; i++){ //do this for 10 layers
previousLayer.scope = {
include : JSON.parse(JSON.stringify(childLayer)) //to avoid reference carry forwards
};
previousLayer = previousLayer.scope.include;
}
//do the query here using filter object

Related

Is there a way how to address nested properties in AWS DynamoDB for purpose of documentClient.query() call?

I am currently testing how to design a query from AWS.DynamoDB.DocumentClient query() call that takes params: DocumentClient.QueryInput, which is used for retrieving data collection from a table in DynamoDB.
Query seems to be simple and working fine while working with indexes of type String or Number only. What I am not able to make is an query, that will use a valid index and filter upon an attribute that is nested (see my data structure please).
I am using FilterExpression, where can be defined logic for filtering - and that seems to be working fine in all cases except cases when trying to do filtering on nested attribute.
Current parameters, I am feeding query with
parameters {
TableName: 'myTable',
ProjectionExpression: 'HashKey, RangeKey, Artist ,#SpecialStatus, Message, Track, Statistics'
ExpressionAttributeNames: { '#SpecialStatus': 'Status' },
IndexName: 'Artist-index',
KeyConditionExpression: 'Artist = :ArtistName',
ExpressionAttributeValues: {
':ArtistName': 'BlindGuadian',
':Track': 'Mirror Mirror'
},
FilterExpression: 'Track = :Track'
}
Data structure in DynamoDB's table:
{
'Artist' : 'Blind Guardian',
..
'Track': 'Mirror Mirror',
'Statistics' : [
{
'Sales': 42,
'WrittenBy' : 'Kursch'
}
]
}
Lets assume we want to filter out all entries from DB, by using Artist in KeyConditionExpression. We can achieve this by feeding Artist with :ArtistName. Now the question, how to retrieve records that I can filter upon WritenBy, which is nested in Statistics?
To best of my knowledge, we are not able to use any other type but String, Number or Binary for purpose of making secondary indexes. I've been experimenting with Secondary Indexes and Sorting Keys as well but without luck.
I've tried documentClient.scan(), same story. Still no luck with accessing nested attributes in List (FilterExpression just won't accept it).
I am aware of possibility to filter result on "application" side, once the records are retrieved (by Artists for instance) but I am interested to filter it out in FilterExpression
If I understand your problem correctly, you'd like to create a query that filters on the value of a complex attribute (in this case, a list of objects).
You can filter on the contents of a list by indexing into the list:
var params = {
TableName: "myTable",
FilterExpression: "Statistics[0].WrittenBy = :writtenBy",
ExpressionAttributeValues: {
":writtenBy": 'Kursch'
}
};
Of course, if you don't know the specific index, this wont really help you.
Alternatively, you could use the CONTAINS function to test if the object exists in your list. The CONTAINS function will require all the attributes in the object to match the condition. In this case, you'd need to provide Sales and WrittenBy, which probably doesn't solve your problem here.
The shape of your data is making your access pattern difficult to implement, but that is often the case with DDB. You are asking DDB to support a query of a list of objects, where the object has a specific attribute with a specific value. As you've seen, this is quote tricky to do. As you know, getting the data model to correctly support your access patterns is critical to your success with DDB. It can also be difficult to get right!
A couple of ideas that would make your access pattern easier to implement:
Move WrittenBy out of the complex attribute and put it alongside the other top-level attributes. This would allow you to use a simple FilterExpression on the WrittenBy attribute.
If the WrittenBy attribute must stay within the Statistics list, make it stand alone (e.g. [{writtenBy: Kursch}, {Sales: 42},...]). This way, you'd be able to use the CONTAINS keyword in your search.
Create a secondary index with the WrittenBy field in either the PK or SK (whichever makes sense for your data model and access patterns).

How can you filter a Django query's joined tables then iterate the joined tables in one query?

I have table Parent, and a table Child with a foreign key to table Parent.
I want to run a query for all Parents with a child called Eric, and report Eric's age.
I run:
parents = Parents.objects.filter(child__name='Eric')
I then iterate over the queryset:
for parent in parents:
print(f'Parent name {parent.name} child Eric age {parent.child.age}')
Clearly this doesn't work - I need to access child through the foreign key object manager, so I try:
for parent in parents:
print(f'Parent name {parent.name}')
for child in parent.child_set.all():
print(f'Child Eric age {parent.child.age}')
Django returns all children's ages, not just children named Eric.
I can repeat the filter conditions:
parents = Parents.objects.filter(child__name='Eric')
for parent in parents:
print(f'Parent name {parent.name}')
for child in parent.child_set.filter(name='Eric'):
print(f'Child Eric age {child.age}')
But this means duplicate code (so risks future inconsistency when another dev makes a change to one not the other), and runs a second query on the database.
Is there a way of getting the matching records and iterating over them? Been Djangoing for years and can't believe I can't do this!
PS. I know that I can do Child.objects.filter(name='Eric').select_related('parent'). But what I would really like to do involves a second child table. So add to the above example a table Address with a foreign key to Parent. I want to get parents with children named Eric and addresses in Timbuktu and iterate over the all Timbuktu addresses and all little Erics. This is why I don't want to use Child's object manager.
This is the best I could come up with - three queries, repeating each filter.
children = Children.objects.filter(name='Eric')
addresses = Address.objects.filter(town='Timbuktu')
parents=(
Parent.objects
.filter(child__name='Eric', address__town='Timbuktu')
.prefetch_related(Prefetch('child_set', children))
.prefetch_related(Prefetch('address_set', addresses))
)
The .values function gives you direct access to the recordset returned (thank you #Iain Shelvington):
parents_queryset_dicts = Parent.objects
.filter(child__name='Eric', address__town='Timbuktu')
.values('id', 'name', 'child__id', 'address__id', 'child__age', 'address__whatever')
.order_by('id', 'child__id', 'address__id')
Note though that this retrieves a Cartesian product of children and addresses, so our gain in reduced query count is slightly offset by double-sized result sets and de-duplication below. So I am starting to think two queries using Child.objects and Address.objects is superior - slightly slower but simpler code.
In my actual use case I have multiple, multi-table chains of foreign key joins, so am splitting the query to prevent the Cartesian join, but still making use of the .values() approach to get filtered, nested tables.
If you then want a hierarchical structure, eg, for sending as JSON to the client, to produce:
parents = {
parent_id: {
'name': name,
'children': {
child_id: {
'age': child_age
},
'addresses': {
address_id: {
'whatever': address_whatever
},
},
},
}
Run something like:
prev_parent_id = prev_child_id = prev_address_id = None
parents = {}
for parent in parents_queryset_dicts:
if parent['id'] != prev_parent_id:
parents[parent['id']] = {'name': parent['name'], children: {}, addresses: {}}
prev_parent_id = parent['id']
if parent['child__id'] != prev_child_id:
parents[parent['id']]['children'][parent['child__id']] = {'age': parent['child__age']}
prev_child_id = parent['child__id']
if parent['address__id'] != prev_address_id:
parents[parent['id']]['addresses'][parent['address__id']] = {'whatever': parent['address__whatever']}
prev_address_id = parent['address__id']
This is dense code, and you no longer get access to any fields not explicitly extracted and copied in, including any nested ~_set querysets, and de-duplication of the Cartesian product is not obvious to later developers. You can grab the queryset, keep it, then extract the .values, so you have both from the same, single, database query. But often the three query repeated filters is a bit cleaner, if a couple database queries less efficient:
children = Children.objects.filter(name='Eric')
addresses = Address.objects.filter(town='Timbuktu')
parents_queryset = (
Parent.objects
.filter(child__name='Eric', address__town='Timbuktu')
.prefetch_related(Prefetch('child_set', children))
.prefetch_related(Prefetch('address_set', addresses))
)
parents = {}
for parent in parents_queryset:
parents[parent.id] = {'name': parent['name'], children: {}, addresses: {}}
for child in parent.child_set: # this is implicitly filtered
parents[parent.id]['children'][child.id] = {'age': child.age}
for address in parent.address_set: # also implicitly filtered
parents[parent.id]['addresses'][address.id] = {'whatever': address.whatever}
One last approach, which someone briefly posted then deleted - I'd love to know why - is using annotate and F() objects. I have not experimented with this, the SQL generated looks fine though and it seems to run a single query and not require repeating filters:
from django.db.models import F
parents = (
Parent.objects.filter(child__name='Eric')
.annotate(child_age=F('child__age'))
)
Pros and cons seem identical to .values() above, although .values() seems slightly more basic Django (so easier to read) and you don't have to duplicate field names (eg, with the obfuscation above of child_age=child__age). Advantages might be convenience of . accessors instead of ['field'], you keep hold of the lazy nested recordsets, etc. - although if you're counting the queries you probably want things to fall over if you issue an accidental query per row.

How to limit records of relations with include-filter in loopback?

I want to query records from a specific model via REST-Api from a LoopBack-application. Also i want to include related objects via the include-filter.
This works fine, but returns ALL related objects. Is it possible to limit them and also to order them by a field of related objects?
Models:
- DEPARTMENT
Fields:
- id
- name
- ...
Relations_ -> hasMany: Messages
Relations_ -> hasMany: Members
- MESSAGE
Fields:
- id
- senderId
- body
- ...
- MEMBER
Fields:
- id
- email
- ...
Queries:
What i want to achieve is to query all Departments with all their members, but only the last message ordered by a specific field (created-timestamp).
The first approach could be the plain query-string variant of a GET-Request:
http://loopback-server:3000/api/departments?filter[include]=members&filter[include]=messages
This will return all departments with all messages and all members. However, i would like to limit the number of returned messages to the last one (or last 5 or whatever, sorted by a specific field of MESSAGE-model.
I also tried the jsonfied query syntax:
http://loopback-server:3000/api/departments?filter={"include":{"relation": "messages","limit":1}}
Unfortunately the "limit"-parameter is not used here for the relation of messages.
The following variant will return only first department, means the limit-param is applied to the departments-model, not the relation model.
http://loopback-server:3000/api/departments?filter={"include":{"relation": "messages"},"limit":1}
Then i discovered the scope-parameter
and tried this:
http://loopback-server:3000/api/departments?filter={"include":{"relation": "messages","scope":{"limit":1, "skip":0}}}
This gives a really strange result. This ommits all messages related to departments, instead of one specific record returning one message (it has over 10), what i would expect. Removing the scope-parameter shows that the departments indeed has many messages each.
(I know that the parameters of an URL with all these special characters like {",:"} need to be url-encoded. I leave it clean here for better readability)
My question:
How to achieve that query with a single request?
It's not possible to query relationships by their properties (yet). As for the limit, your last approach with the scope should be modified a little:
"scope":{{"include":{"relation": "messages","limit":1, "skip":0}}}
Here you can read about queries on relations by their properties:
https://github.com/strongloop/loopback/issues/517
I dunno what version you are in, but for Loopback 3
you can do this..
include: {
{
relation: 'Messages', // include the messages object
scope: { // this is where you do a normal filter
where: {<whatevercondition>},
order: "<fieldname> <ASC/DESC>",
limit:1,
include:{
//yes, you can include 3rd level relation as well.
}
}
},
{
relation: 'Members', // include the Members object
scope: { // further filter the related model
order: "<fieldname> <ASC/DESC>",
limit: <whateverlimityoument>
}
}
}
try this code:
`${urlApi}/user/?filter[limit]=${records_per_page}&filter[skip]=${(currentPage -1) *
records_per_page}`
Limit for inclusion scope works correctly when you have only one parent record.
If you want to select N parent records and include 1 related record in each of them, try my workaround: Limit for included records in Loopback4

Sort 'Days' Items in Sitecore according to days of week

Is there any way to sort the following items according to the days of week.
In C# I can do something like this:
string [] initialArray = {"Friday", "Monday", ... } ;
string [] sortedArray = initialArray.OrderBy(s => Enum.Parse(typeof(DayOfWeek), s)).ToArray() ;
But I don't know how can I achieve this kind of functionality with Sitecore.
If what you really care is to display the days sorted on the front-end regardless of how they are organised in the Content Editor then simply sort them in code before you display them, e.g.
using System.Linq;
var openingHours = Sitecore.Context.Item.Children
.OrderBy(s => Enum.Parse(typeof(DayOfWeek), s.DisplayName));
If you want to sort them in the Content Editor then you need to create a custom sorter. Sitecore Climber has provided links, but for this specific example you can use:
using Sitecore.Data.Comparers;
using Sitecore.Data.Items;
public class DayOfWeekComparer : Comparer
{
protected override int DoCompare(Item item1, Item item2)
{
var x = (int)Enum.Parse(typeof(DayOfWeek), item1.DisplayName);
var y = (int)Enum.Parse(typeof(DayOfWeek), item2.DisplayName);
return x.CompareTo(y);
}
}
Then in the core database create an item of item type /sitecore/templates/System/Child Sorting under /sitecore/system/Settings/Subitems Sorting and set the type to your class.
You should set the Subitem Sorting on the Standard values of a template. In this instance it looks like you have a simple Folder template, so you would need to create a more specific template for your Opening Hours folder. Even so, the user can still decide to re-order the items OR change the default sort for that folder. The only guaranteed way to force the output is by sorting before you render, i.e. the first bit of code.
please check next articles:
http://firebreaksice.com/how-to-sort-sitecore-items-in-the-content-editor/
http://sitecore.alexiasoft.nl/2009/08/04/sorting-sitecore-items/
http://sitecoreblog.blogspot.in/2010/11/change-default-subitems-sort-order.html

converting linq query to icollection

I need to take the results of a query:
var query = from m in db.SoilSamplingSubJobs where m.order_id == id select m;
and prepare as an ICollection so that I can have something like
ICollection<SoilSamplingSubJob> subjobs
at the moment I create a list, which isnt appropriate to my needs:
query.ToList();
what do I do - is it query.ToIcollection() ?
List is an ICollection. You can change you query.ToList() code to the following.
query.ToList() as ICollection<SoilSamplingSubJob>;
You question sounds like this query is returned as a function result. If this is the case remember that the Linq to SQL objects are connected by default so you will need to manage where your database context get opened and closed. Alternatively you can create DTOs (Data Transfer Objects) that hold the data you want to use in the rest of your program. These objects can fit into your object hierarchy any way you want.
You can also create these DTOs as part of the query.
var query = from m in db.SoilSamplingSubJobs where m.order_id == id
select new SubJobDTO {
OrderNumber = m.order_id
};
return query.ToList() as ICollection<SubJobDTO>;
Since, query implements IEnumerable, you can pass it to the constructor of the collection of your choice. ICollection is an interface, implemented by several classes (including List<T>, which ToList returns) with different performance characteristics.
With slightly different syntax, you can do something like this as well that can cut down on what goes into the ICollection. This type of approach is great for Angular and MVC when you have a huge table but only want to load some props into the cshtml page:
ICollection<SoilSamplingSubJob> samples = dbContext.GetQuery().Where(m => m.order_id == id)
.AsEnumerable().Select(s => new
{
Id = s.order_id,
CustomProperty = s.some_thing
}).ToList() as ICollection<SoilSamplingSubJob>