Cube.js time series excluding weekends - cube.js

I am using mongo db BI connector and trying to render date series charts using cube.js. Is there a way to apply time dimension with a date range and considering only business days?
Below are my sample cube schema and collection
cube(`Entries`, {
sql: `SELECT * FROM db.entries`,
measures: {
count: {
type: `count`
}
},
dimensions: {
result: {
sql: `result`,
type: `string`
},
date: {
sql: `date`,
type: `time`
},
},
});
Query :
{
"measures": [
"Entries.count"
],
"timeDimensions": [
{
"dimension": "Entries.date",
"granularity": "day"
}
]
}

You can do this by introducing weekdays segment:
cube(`Entries`, {
sql: `SELECT * FROM db.entries`,
measures: {
count: {
type: `count`
}
},
dimensions: {
result: {
sql: `result`,
type: `string`
},
date: {
sql: `date`,
type: `time`
},
},
segments: {
weekdays: {
sql: `DAYOFWEEK(${date}) <> 1 AND DAYOFWEEK(${date}) <> 7`
}
}
});
And use it in query like:
{
"measures": [
"Entries.count"
],
"timeDimensions": [
{
"dimension": "Entries.date",
"granularity": "day"
}
],
"segments": ["Entries.weekdays"]
}
To hide zero weekend values pass fillMissingDates: false when preparing the data:
resultSet.chartPivot({ fillMissingDates: false })
More info about segments here: https://cube.dev/docs/segments

Related

How to filter nested list of list of objects with JSON field in django

I have a Model named Ad and it has a JSONfield named location_details.
while fetching the list of objects from Ad Model, I am applying filter for the location_details field
http://127.0.0.1:8000/ad-list/?limit=10&offset=0&ordering=-start_date&location_category=bus_station%2Chospital
While creating an ad, I compute the nearby places and store it as a JSON field as shown below
[ { name: "transit", label: "Transit", sub_categories: [ { name: "bus_station", label: "Bus Stop", values: [{name: 'A'}, {name: 'B'}], }, { name: "airport", label: "Airport", values: [], }, { name: "train_station", label: "Train Station", values: [], }, ], }, { name: "essentials", label: "Essentials", sub_categories: [ { name: "hospital", label: "Hospital", values: [{name: 'te'}], }, { name: "school", label: "Schools", values: [{name: 'B'}], }, { name: "atm", label: "ATMs", values: [], }, { name: "gas_station", label: "Gas Stations", values: [{name: 'C'}], }, { name: "university", label: "Universities", values: [], }, ], }, { name: "utility", label: "Utility", sub_categories: [ { name: "movie_theater", label: "Movie Theater", values: [], }, { name: "shopping_mall", label: "Shopping Mall", values: [], }, ], }, ];
I want to filter the ad list with multiple location_categories with the above JSON object, only if the location category has some data in it.
I don't know how to start with it, this is how my code looks like now
location_category = django_filters.CharFilter(method="filter_location_category")
def filter_location_category(self, queryset, name, value):
return queryset.filter(location_details__sub_categories__name__in=value.split(","))
For example, with the below request
http://127.0.0.1:8000/ad-list/?limit=10&offset=0&ordering=-start_date&location_category=bus_station%2Chospital
I want to get all the ads where bus_station and hospital have some data in it (inside the values key) and want to ignore it if the values list is empty in the JSON.
Any help would be highly appreciated. Thank you so much for your attention and participation.

DynamoDB unmarshall array of dynamodb records? (SDK v3 javascript)

I can't find a way to unmarshal an array of DynamoDB records? I'm getting multiple items back from my query in DynamoDB, and I can't use the unmarshal function that comes from #aws-sdk/util-dynamodb on an array of records. When I iterate through the array of records and unmarshal each one, that works, but this is not the best solution since I'm getting back a lot of data. How can I get DynamoDB to return multiple records without iterating through the array of records to unmarshall each one?
Code:
const client = new DynamoDBClient({ region: 'us-east-1' });
const currentDateUTC = moment().utc().format('YYYY-MM-DD');
const command = new QueryCommand({
TableName: 'Device-Data',
IndexName: 'timestamp-index',
KeyConditionExpression: '#timestamp = :timestamp',
ExpressionAttributeNames: { '#timestamp': 'timestamp' },
ExpressionAttributeValues: { ':timestamp': { S: currentDateUTC } },
});
const data = await client.send(command)
Response:
[
{
deviceId: {
S: "test1",
},
userId: {
S: "318ecd46-bead-440f-83eb-60345aaa1c40",
},
timestamp: {
S: "2022-06-12",
},
frequency: {
S: "Monthly",
},
},
{
deviceId: {
S: "test2",
},
userId: {
S: "318ecd46-bead-440f-83eb-60345aaa1c40",
},
timestamp: {
S: "2022-06-12",
},
frequency: {
S: "Monthly",
},
},
]
Desired Response:
[
{
deviceId: "test1",
userId: "318ecd46-bead-440f-83eb-60345aaa1c40",
timestamp: "2022-06-12",
frequency: "Monthly",
},
{
deviceId: "test2",
userId: "318ecd46-bead-440f-83eb-60345aaa1c40",
timestamp: "2022-06-12",
frequency: "Monthly",
},
]

Django pymongo search, sort, limit on inner array and count them

I am learning Django with MongoDb and have a collection to search into. Here is a sample document in the collection:
{
"_id": { "$oid": "62615907568ddfca4fef3a25" },
"word": 'entropy,
"count": 4,
"occurrence": [
{"book": "62615907568ddfca4fef3a23", "year": 1942, "sentence": 0 },
{"book": "62615907568ddfca4fef3a23", "year": 1942, "sentence": 5 },
{"book": "62615907568ddfca4fef3a75", "year": 1928, "sentence": 0 },
{"book": "62615907568ddfca4fef3a90", "year": 1959, "sentence": 8 }
]
}
I want to retrieve the array elements of 'occurrence' field of a specific document (word):
Sorted by year
Within an year range
Limited with offset
count the total results (without limit/offset)
What I have done so far is:
offset= 0
limit= 10
search= {'$and': [{"_id": word_id_obj, "occurrence": {"$elemMatch": {"year": {"$gte": 1940, "$lte": 1960}}}}]}
word_docs= wordcollec.aggregate([
{"$match": search},
{"$project":
{"occurrence":
{"$slice": ['$occurrence', offset, limit]}
}
},
{"$sort": {'occurrence.year': -1}}
])
# Count total results
recnt= wordcollec.aggregate([{"$match": search}, {'$group' : {"_id": "$_id", "sno" : {"$sum" : {"$size": "$occurrence"}}}}])
The year range, count are not working and I am unable to sort them. How can I rewrite my query for this?
Thanks in advance.
Use $unwind then $sort
db.collection.aggregate([
{
$match: {
_id: { "$oid": "62615907568ddfca4fef3a25" },
occurrence: { $elemMatch: { year: { $gte: 1940, $lte: 1960 } } }
}
},
{
$set: { totalWithoutLimitOrOffset: { $size: "$occurrence" } }
},
{
$unwind: "$occurrence"
},
{
$match: { "occurrence.year": { $gte: 1940, $lte: 1960 } }
},
{
$sort: { "occurrence.year": -1 }
},
{
$skip: 1
},
{
$limit: 2
},
{
$group: {
_id: "$_id",
word: { $first: "$word" },
count: { $first: "$count" },
totalWithoutLimitOrOffset: { $first: "$totalWithoutLimitOrOffset" },
occurrence: { $push: "$occurrence" }
}
}
])
mongoplayground

Remove special characters in mongo aggregate while query run

I would like to get all the datas from mongo while i need to replace special characters, but not rewrite the data while the query run:
data in db:
[{
number: "12/34",
type: "string"
},
{
number: "56-78",
type: "string"
},
{
number: "910*2",
type: "string"
}]
the number what I would like to query is: 1234, 5678
the related output is:
[{
number: "12/34",
type: "string"
},
{
number: "56-78",
type: "string"
}]
what I try is to add field temporary without special characters, but I cannot remove all of them because addfield doesn't handle regex. i try to do this with reduce but it's not work for me.
Anybody can help me?
With MongoDB v4.2+, you can use $regexFindAll to locate all digits. Use $reduce and $concat to reconstruct a string with special characters removed/sanitized. Perform the search on the sanitized string.
db.collection.aggregate([
{
"$addFields": {
"sanitized": {
"$regexFindAll": {
"input": "$number",
"regex": "\\d"
}
}
}
},
{
"$addFields": {
"sanitized": {
"$reduce": {
"input": "$sanitized.match",
"initialValue": "",
"in": {
"$concat": [
"$$value",
"$$this"
]
}
}
}
}
},
{
"$match": {
$expr: {
"$in": [
"$sanitized",
[
"1234",
"5678"
]
]
}
}
},
{
"$project": {
sanitized: false
}
}
])
Here is the Mongo playground for your reference.

Using $slice with $regex together on subDocument array in mongodb

Hi I am stuck on a problem in mongo. Let me first show you an example of the data. This is a record from my table.
{
"_id": "3691149613248",
"following": [{
"content_id": "2584833593728",
"date": "2015-08-20 12:46:55"
}, {
"content_id": "3693447751360",
"date": "2015-09-11 12:17:55"
}, {
"content_id": "2582396936896",
"date": "2015-09-12 07:04:02"
}, {
"content_id": "3697346507456",
"date": "2015-09-14 09:56:03"
}, {
"content_id": "3697755500800",
"date": "2015-09-16 10:05:51"
}, {
"content_id": "2589701320192",
"date": "2015-09-16 10:51:19"
}, {
"content_id": "2585723555136",
"date": "2015-09-16 11:40:26"
}, {
"content_id": "3695996668352",
"date": "2015-09-16 12:50:25"
}, {
"content_id": "3694290368512",
"date": "2015-09-16 12:50:33"
}, {
"content_id": "3691210127552",
"date": "2015-09-16 13:02:57"
}, {
"content_id": "3694134958464",
"date": "2015-09-16 13:06:17"
}, {
"content_id": "3697315148736",
"date": "2015-09-17 06:58:35"
}, {
"content_id": "3692104837824",
"date": "2015-09-17 12:19:12"
}, {
"content_id": "3693400309376",
"date": "2015-09-22 05:43:04"
}]
}
I want to fetch following array with condition that only specific records to fetch i.e. content_ids with prefix 369 and fetch number of content_id specified in limit and offset.
I am using $slice for fetching records for given limit & offset for following array. But how to filter content_id along with $slice.
My current query:
db.collectionName.find({
_id: "3691149613248"
}, {
"following": {
"$slice": [0, 10]
}
});
This is fetching following array with content_id that is specified in limit & offset. But it is fetching all content_id including prefix 258& 369 but I only need content_id with prefix 369 using mongo query.
Can any one help??
You can use combination of $unwind and $match with mongo aggregation to get expected output like:
db.collection.aggregate({
$match: {
"_id": "3691149613248" // you can skip this condition if not required
}
}, {
$unwind: "$following"
}, {
$match: {
"following.content_id": {
$regex: /^369/
}
}
}, {
$group: {
_id: "$_id",
"following": {
$push: "$following"
}
}
})
If you want to apply skip and limit to above query then you can easily use it like:
db.collection.aggregate({
$match: {
"_id": "3691149613248" //use this if you want to filter out by _id
}
}, {
$unwind: "$following"
}, {
$match: {
"following.content_id": {
$regex: /^369/
}
}
}, {
$skip: 4 // you can set offset here
}, {
$limit: 3 // you can set limit here
}, {
$group: {
_id: "$_id",
"following": {
$push: "$following"
}
}
})
EDIT :
If you are using php version less than 5.4 then query will be as:
$search = "369";
$criteria = array(array("$match" => array("_id" => "3691149613248")),
array("$unwind" => "$following"),
array("$match" => array("following.content_id" => array("$regex" => new MongoRegex("/^$search/")))),
array("$skip" => 4), array("$limit" => 3),
array("$group" => array("_id" => "$_id", "following" => array("$push" => "$following"))));
$collection - > aggregate($criteria);
If you are using PHP version greater than 5.3 then just replace { and } braces with [ and ] respectively.