Want to search all matching pattern from Mongo DB nested fields with dynamic keys.
DB Structure:
_id: 'dsdsdsadadad',
results: {
tables: {
jvm: {
data: [
{
Prediction: 1,
Jvm: 'service_name',
Status: 'OK'
},
{
second: 'New second set'
}
}
}
}
Tried By $,
db.col_name.find('results.tables.jvm.data.$.Jvm': {'$regexp': 'service.*'})
By using $i
db.col_name.find('results.tables.jvm.data.$i.Jvm': {'$regexp': 'service.*'})
By giving particular key 0 also,
db.col_name.find('results.tables.jvm.data.0.Jvm': {'$regexp': 'service.*'})
No results!
Expected O/P:
The above doc and where all Jvm starts with service* keyword
Thanks,
You should directly use the dot notation to query an array of nested objects:
db.collection.find({ "results.tables.jvm.data.Jvm": { $regex: "service.*" } })
MongoDB will try to find every document that contains at least one nested document under data having Jvm field matching your regex.
MongoDB Playground
Related
I am storing regex expressions in MongoDB and would like to use them for queries.
Documents structure:
{
"name": "foo",
"regex_expression": "^[^# ]+#(bar\\.com)$"
}
I tried the following query but it doesn't work.
db.collection.find({$expr: {$regex: ["foo#bar.com", "$regex_expression]}})
Is it possible to do this kind of query?
You need the $regexMatch operator.
db.collection.find({
$expr: {
$regexMatch: {
input: "foo#bar.com",
regex: "$regex_expression"
}
}
})
Demo # Mongo Playground
I just started working with elastic search. By started working I mean I have to query an already running elastic database. Is there a good documentation of the regex they follow. I know about the one on their official site, but its not very helpful.
The more specific problem is that I want to query for lines of the sort:
10:02:37:623421|0098-TSOT {TRANSITION} {ID} {1619245525} {securityID} {} {fromStatus} {NOT_PRESENT} {toStatus} {WAITING}
or
01:01:36:832516|0058-CT {ADD} {0} {3137TTDR7} {23} {COM} {New} {0} {0} {52} {1}
and more of a similar structure. I don't want a generalized regex. If possible, could someone give me a regex expression for each of these that would run with elastic?
I noticed that it matches if the regexp matches with a substring too when I ran with:
query = {"query":
{"regexp":
{
"message": "[0-9]{2}"
}
},
"sort":
[
{"#timestamp":"asc"}
]
}
But it wont match anything if I use:
query = {"query":
{"regexp":
{
"message": "[0-9]{2}:.*"
}
},
"sort":
[
{"#timestamp":"asc"}
]
}
I want to write regex that are more specific and that are different for the two examples given near the top.
turns out my message is present in the tokenized form instead of the raw form, and : is one of the default delimiters of the tokenizer, in elastic. And as a reason, I can't use regexp query on the whole message because it matches it with each token individually.
I have a couple of indexes in my Elasticsearch DB as follows
Index_2019_01
Index_2019_02
Index_2019_03
Index_2019_04
.
.
Index_2019_12
Suppose I want to search only on the first 3 Indexes.
I mean a regular expression like this:
select count(*) from Index_2019_0[1-3] where LanguageId="English"
What is the correct way to do that in Elasticsearch?
How can I query several indexes with certain names?
This can be achieved via multi-index search, which is a built-in capability of Elasticsearch. To achieve described behavior one should try a query like this:
POST /index_2019_01,index_2019_02/_search
{
"query": {
"match": {
"LanguageID": "English"
}
}
}
Or, using URI search:
curl 'http://<host>:<port>/index_2019_01,index_2019_02/_search?q=LanguageID:English'
More details are available here. Note that Elasticsearch requires index names to be lowercase.
Can I use a regex to specify index name pattern?
In short, no. It is possible to use index name in queries using a special "virtual" field _index but its use is limited. For instance, one cannot use a regexp against index name:
The _index is exposed as a virtual field — it is not added to the
Lucene index as a real field. This means that you can use the _index
field in a term or terms query (or any query that is rewritten to a
term query, such as the match, query_string or simple_query_string
query), but it does not support prefix, wildcard, regexp, or fuzzy
queries.
For instance, the query from above can be rewritten as:
POST /_search
{
"query": {
"bool": {
"must": [
{
"terms": {
"_index": [
"index_2019_01",
"index_2019_02"
]
}
},
{
"match": {
"LanguageID": "English"
}
}
]
}
}
}
Which employs a bool and a terms queries.
Hope that helps!
Why use POST when you are not adding any additional data to it.
I advise using GET for your case. Secondly, If the Index have similar names like in your case, you should be using an index pattern like in the query below,
GET /index_2019_*/_search
{
"query": {
"match": {
"LanguageID": "English"
}
}
}
OR in a URL
curl -XGET "http://<host>:<port>/index_2019_*/_search" -H 'Content-Type: application/json' -d'{"query": {"match":{"LanguageID": "English"}}}'
While searching for indices using a regex is not possible you might be able to use date math to take you a bit further.
You can look at the docs here
As an example, lets say you wish the last 3 months from those indices
that means that if we have
index_2019_01
index_2019_02
index_2019_03
index_2019_04
And today is 2019/04/20, we could use the following query to get 04,03 and 02
GET /<index-{now/M-0M{yyyy_MM}}>,<index-{now/M-1M{yyyy_MM}}>,<index-{now/M-2M{yyyy_MM}}>
I used M-0M for the first one so the query construction loop doesn't need a special case for the first index
Look at the docs regarding URL encoding this query and how to have literal braces in the index name, if a client is used the URL encoding is done for you (at least in the python client)
I have the following problem:
I am using NodeJS with Express and MongoDB to query my database.
I have a document in the collection of "domains" containing the field "domain".
For example:
{
"domain" : "mydomain.com, www.mydomain.com, beta.mydomain.com, *.beta.mydomain.com",
"APIKeys" : [ "Public" : 111111 ]
}
Or another document:
{
"domain" : "example.com, *.example.com",
"APIKeys" : [ "Public" : 222222 ]
}
I would like to query the database and return the result if extractHostname(req.get('Referrer')) matches any of the domains in the field.
var collection = 'domains';
var query = { $and: [ { 'APIKeys.Public' : req.query.APIKey }, {'domain' : extractHostname(req.get('Referrer')) } ] };
var projection = { '_id' : 1 , 'playerPref' : 1 };
For example: extractHostname(req.get('Referrer')) = beta.mydomain.com it should return true, since it matches the regex of beta.mydomain.com.
'test.beta.mydomain.com' should return true since it matches the regex of *.beta.mydomain.com.
'test.www.mydomain.com' should return false.
'www.mydomain.com.maliciousdomain.com' should return false.
Any idea how I can make a query like this to check if the Referrer is in matching conditions?
The problem I am facing is that any of the strings in the field need to match the query, and not the other way around. Keeping in mind the wildcard in the field as opposed in the search string. (It is like a reserve regex?)
Kind regards,
Hugo
After seeing your updated requirements I've created a pattern that will match any of your various domain types. Here it is
(?(?=.* .*)(([^ \n>]*)(?:.*))|([^\w\W]))
It will always match mydomain.com, with an optional www., optional beta. and optional wildcard.
I have a mongo collection User which contains data like:-
{
id : 1,
name : "gaurav",
skills : "C++ HTML CSS"
}
when I am searching for all users that have C++ skill in it with the following query I am getting correct results as expected
db.user.find({skills:{contains:"C++"}});
But when I am searching all the unique names from the user using the same condition I m not getting any desired result
db.user.distinct('name',{skills:{contains:"C++"}});
Can anyone help me with what I am doing wrong?
The "contains" is not a valid keyword for MongoDB queries. You need $regex which submits a general "regular expression" statement matching the pcre specifications:
db.user.distinct( "name", { "skills": { "$regex": "C\+\+" } })
If using JavaScript as you language then this is also safe:
db.user.distinct( "name", { "skills": /C\+\+/ })
To determine if the string "C++" occurred somewhere within the string value of the field being tested. The + character is reserved in "regex" operations and therefore you need to escape it with a \ char as the standard escaping mechanism.
On your data this is the result:
db.user.distinct( "name", { "skills": { "$regex": "C\+\+" } })
[ "gaurav" ]
Try to use REGEX like below query
db.user.distinct("name",{"skills":{"$regex":"C++.*"}})