Use $regex inside $expr in mongodb aggregation - regex

My doc looks as follows
doc = {
name: 'abc',
age:20
}
and my query looks like
{ $expr: {$and:[{ $gt:[ "$age", 10 ] },
{ $regex:["$name",'ab']}
]
}
} }
But it's not working and I get an error
Unrecognized expression '$regex'
How can I make it work?
My original query looks like this
db.orders.aggregate([{
$match: {}},
{$lookup: {
from: "orders",
let: {
"customer_details": "$customerDetails"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{ $or: [
{
$eq: ["$customerDetails.parentMobile","$$customer_details.parentMobile"]
},
{$eq: ["$customerDetails.studentMobile","$$customer_details.parentMobile"]
},
{$eq: ["$customerDetails.studentMobile","$$customer_details.parentMobile"]
},
{$eq: ["$customerDetails.studentMobile","$$customer_details.studentMobile"]
}
]
},
{$eq: ["$customerDetails.zipCode","$$customer_details.zipCode"]},
{$eq: ["$customerDetails.address","$$customer_details.address"]}
]
}
}
}],
as: "oldOrder"
}
}])
I want to use regex for matching address.
Any help will be greatly appreciated. Thanks in advance.

If your mongoDB version is 4.2, then you can use $regexMatch
try this
db.collection.find({
$expr: {
$and: [
{
$gt: [
"$age",
10
]
},
{
$regexMatch: {
input: "$name",
regex: "ab"
}
}
]
}
})
check this Mongo Playground

$regex is a query operator you cannot use inside $expr because it only supports aggregation pipeline operators.
{
"$expr": { "$gt": ["$age", 10] } ,
"name": { "$regex": "ab" }
}
If you have mongodb 4.2, you can use $regexMatch
{ "$expr": {
"$and": [
{ "$gt": ["$age", 10] },
{
"$regexMatch": {
"input": "$name",
"regex": "ab", //Your text search here
"options": "i",
}
}
]
}}

Related

Unrecognized expression ‘$regex’

Not able to get desired output. Getting “Unrecognized expression ‘$regex’” error
[
{
'$lookup': {
'from': 'profiles',
'let': {
'userId': '$userId',
},
'pipeline': [
{
'$match': {
$expr: {
$and: [
{ '$eq': ['$uniqueId', '$$mcontactId'] },
{
$or: [{ 'birthDate': { '$regex': '$$today' } },
{ 'spouseBirthdate': { '$regex': '$$today' } },
{ 'weddingAnniversary': { '$regex': '$$today' } },
],
},
],
},
},
},
{
'$project': {
'userId': 1,
'uniqueId': 1,
'mobileNumber': 1,
'whatsApp': 1,
'emailId': 1,
'lastName': 1,
'firstName': 1,
'address': 1,
'signature': 1,
},
},
],
'as': 'profile',
},
},
]
$regex is a query operator, you are trying to use it within an $expr which uses the "aggregation" language as oppose to the "query" language normally used within a $match stage.
Apart from that you have some other issue's in your pipeline, for example you only define $userId as a variable for the $lookup stage but in it you're trying to use $$today and $$mcontactId which are not defined anywhere.
Regardless once you sort out those issue's you have two options:
if the regex match is not related to the input variables just use $regex outside the $expr, like so:
{
'$match': {
$and: [
{
$expr: {
'$eq': [
'$uniqueId',
'$$userId',
],
},
},
{
$or: [
{
'birthDate': {
'$regex': '03-05',
},
},
],
},
],
},
},
Mongo Playground
if the regex does not to use an input variable from the $lookup then you need to use an aggregation operator, like $regexMatch to do the match within the $expr

How to project all nested documents from an array of objects?

I have a document like this:
{
"_id": "5ffc130e9fb31b26162e0bad",
"results": [
{
"customer": {
"display_name": "Manno Dispensary - first",
"ext_acct_id": "267"
}
},
{
"customer": {
"display_name": "Manno Dispensary - second",
"ext_acct_id": "262"
}
},
{
"customer": {
"display_name": "Kako Dispensary - first",
"ext_acct_id": "261"
}
},
{
"customer": {
"display_name": "Kako Dispensary - second",
"ext_acct_id": "263"
}
}
]
}
I want to write a MongoDB query which does a regex search on "customer.display_name" and returns all those documents in results that satisfies this criteria.
I have written this query till now, and it returns me the desired output, but the problem is, it is only retuning one document inside results, Am I missing anything in this?
my desired output:
{
"_id": "5ffc130e9fb31b26162e0bad",
"results": [
{
"customer": {
"display_name": "Manno Dispensary - first",
"ext_acct_id": "267"
}
},
{
"customer": {
"display_name": "Manno Dispensary - second",
"ext_acct_id": "262"
}
}
]
}
What I am actually getting:
{
"_id": "5ffc130e9fb31b26162e0bad",
"results": [
{
"customer": {
"display_name": "Manno Dispensary - first",
"ext_acct_id": "267"
}
}
]
}
this is the query which I have written to fetch all customers which contain "Manno" in their customer name.
My collection name is Order(mongoose), search="Manno"
Order.find({
results: {
$elemMatch : {
"customer.display_name": {$regex: search}
}
}
},{
results: {
$elemMatch : {
"customer.display_name": {$regex: search}
}
}
});
The $elemMatch and $ will return only single matching document/object form array, try $filter and $regexMatch,
$filter to iterate loop of results array
$regexMatch to check regular expression condition, it will return true or false
Order.find({
results: {
$elemMatch: {
"customer.display_name": { $regex: "Manno" }
}
}
},
{
results: {
$filter: {
input: "$results",
cond: {
$regexMatch: {
input: "$$this.customer.display_name",
regex: "Manno"
}
}
}
}
})
Playground

MongoDB Aggregation regex match object id

I have a collection;
"users": [
{
"_id": ObjectId("5c4185be19da7e815cb18f59"),
"name": "User1"
},
{
"_id": ObjectId("5c4185be19da7e815cb18f5a"),
"name": "User2"
} ]
I need to search users collection by regex.
db.results.aggregate([{
"$match": {
"name": {
"$regex": "user",
"$options": "si"
}
}
}
])
this works for searching against user field. I tried with the below code to search against id. But it didn't work for me.
db.results.aggregate([{
"$match": {
"_id": {
"$regex": "18f5a",
"$options": "si"
}
}
}
])
Thanks in advance.
The _id field is ObjectId type by default hence you can't regex match it.
If you're using Mongo version 4.0+ you can use toString.
db.results.aggregate([
{
$addFields: {
_id: {$toString: "$_id"}
}
},
{
"$match": {
"_id": {
"$regex": "18f5a",
"$options": "si"
}
}
}
])

How can I exclude results from elasticsearch based on the contents of a field?

I'm using elasticsearch on AWS to store logs from Cloudfront. I have created a simple query that will give me all entries from the past 24h, sorted from new to old:
{
"from": 0,
"size": 1000,
"query": {
"bool": {
"must": [
{ "match": { "site_name": "some-site" } }
],
"filter": [
{
"range": {
"timestamp": {
"lt": "now",
"gte": "now-1d"
}
}
}
]
}
},
"sort": [
{ "timestamp": { "order": "desc" } }
]
}
Now, there a are certain sources (based on the user agent) for which I would like to exclude results. So my question boils down to this:
How can I filter out entries from the results when a certain field contains a certain string? Or:
query.filter.where('cs_user_agent').does.not.contain('Some string')
(This is not real code, obviously.)
I have tried to make sense of the Elasticsearch documentation, but I couldn't find a good example of how to achieve this.
I hope this makes sense. Thanks in advance!
Okay, I figured it out. What I've done is use a Bool Query in combination with a wildcard:
{
"from": 0,
"size": 1000,
"query": {
"bool": {
"must": [
{ "match": { "site_name": "some-site" } }
],
"filter": [
{
"range": {
"timestamp": {
"lt": "now",
"gte": "now-1d"
}
}
}
],
"must_not": [
{ "wildcard": { "cs_user_agent": "some string*" } }
]
}
},
"sort": [
{ "timestamp": { "order": "desc" } }
]
}
This basically matches any user agent string containing "some string", and then filters it out (because of the "must_not").
I hope this helps others who run into this problem.
nod.js client version:
const { from, size, value, tagsIdExclude } = req.body;
const { body } = await elasticWrapper.client.search({
index: ElasticIndexs.Tags,
body: {
from: from,
size: size,
query: {
bool: {
must: {
wildcard: {
name: {
value: `*${value}*`,
boost: 1.0,
rewrite: 'constant_score',
},
},
},
filter: {
bool: {
must_not: [
{
terms: {
id: tagsIdExclude ? tagsIdExclude : [],
},
},
],
},
},
},
},
},
});

is it possible to write regular expression in $cond in MongoDB

I need to use $cond to combine differenet column, and one $cond I need to write is as following:
create_widget: {
$sum:{
$cond:[{$and: [ {$eq: ['$Method', 'POST']},
{Url:{$regex: /.*\/widgets$/}} ]}, 1, 0]
}
}
and this code is not right, it seems, regular expression can not be put here.Is there any other way to do this? I want to match Url and regular expression and put the code under $cond.
A sample data looks as
{"BrandId":"a","SessionId":"a1","Method":"POST","Url":"/sample/widgets"}
{"BrandId":"a","SessionId":"a2","Method":"POST","Url":"/sample/blog"}
{"BrandId":"b","SessionId":"b1","Method":"PUT","Url":"/sample/widgets"}
The whole code I wrote is as following:
db.tmpAll.aggregate([
{$group: {
_id: {BrandId:'$BrandId'},
SessionId: {$addToSet: '$SessionId'},
create_widget: {
$sum:{
$cond:[{$and: [ {$eq: ['$Method', 'POST']},
{} ]}, 1, 0]
}
}
}},
{$group: {
_id: '$_id.BrandId',
distinct_session: {$sum: {$size: '$SessionId'}},
create_widget: {$sum: '$create_widget'}
}}
]);
The expected result of sample code is
{ "_id" : "a", "distinct_session" : 2, "create_widget" : 1 }
{ "_id" : "b", "distinct_session" : 1, "create_widget" : 0 }
For MongoDB 4.2 and newer production releases, and in the 4.1.11 and newer development versions, use $regexMatch which is a syntactic sugar on top of $regexFind which can be used for regex matching and capturing.
db.tmpAll.aggregate([
{ "$group": {
"_id": {
"BrandId": "$BrandId",
"SessionId": "$SessionId"
},
"widget_count": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": ["$Method", "POST"] },
{ "$regexMatch": {
"input": "$Url",
"regex": /widget/
} }
]
}, 1, 0
]
}
},
"session_count": { "$sum": 1 }
} },
{ "$group": {
"_id": "$_id.BrandId",
"create_widget": { "$sum": "$widget_count" },
"distinct_session": { "$sum": "$session_count" }
} }
]);
There is an open JIRA issue for this SERVER-8892 - Use $regex as the expression in a $cond. However, as a workaround, For older MongoDB versions which do not have the above features, use the following workaround in your aggregation pipeline.
It uses the $substr operator in the $project operator stage to extract the part of the URL and acts as a workaround for the regex. :
db.tmpAll.aggregate([
{ "$group": {
"_id": {
"BrandId": "$BrandId",
"SessionId": "$SessionId"
},
"widget_count": {
"$sum": {
"$cond": [
{
"$and": [
{ "$eq": ["$Method", "POST"] },
{ "$eq": [ { "$substr": [ "$Url", 8, -1 ] }, "widget"] }
]
}, 1, 0
]
}
},
"session_count": { "$sum": 1 }
} },
{ "$group": {
"_id": "$_id.BrandId",
"create_widget": { "$sum": "$widget_count" },
"distinct_session": { "$sum": "$session_count" }
} }
]);
Output
/* 1 */
{
"result" : [
{
"_id" : "a",
"create_widget" : 1,
"distinct_session" : 2
},
{
"_id" : "b",
"create_widget" : 0,
"distinct_session" : 1
}
],
"ok" : 1
}