Elasticsearch query with wildcard and match conditions - regex

I have this index:
{
"mappings": {
"records" : {
"properties" : {
"suggest" : {
"type" : "completion",
"contexts": [
{
"name": "year",
"type": "category",
"path": "year"
}
]
}
}
}
}
}
I put some records:
POST http://localhost:9200/example/records
{
"suggest": {
"input": "foo123" ,
"contexts": {
"year": "1999"
}
}
}
POST http://localhost:9200/example/records
{
"suggest": {
"input": "some123" ,
"contexts": {
"year": "1999"
}
}
}
POST http://localhost:9200/example/records
{
"suggest": {
"input": "thing123" ,
"contexts": {
"year": "2000"
}
}
}
Now I would do this query (sql like):
SELECT * FROM example WHERE SUGGEST LIKE %123% AND YEAR=1999
How can I do in Elastic Search?
I type:
POST http://localhost:9200/example/records/_search?pretty
{
"query": {
"bool": {
"must": [
{ "wildcard" : { "suggest" : "*123*" } }
],
"filter":[
{ "term" : { "year" : "1999" } }
]
}
}
}
I have returned this response with blank results:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
I am expecting to have returned this records:
foo123, year 1999
some123, year 1999
How can I do?

You need to use bool query with must if you care about score:
{
"query": {
"bool": {
"must": [
{ "wildcard" : { "name" : "*foo*" } },
{ "term" : { "year" : "1999" } }
]
  }
}
}
or with filter if you just want to filter values and possibly cache the filter:
{
"query": {
"filter": {
"must": [
{ "wildcard" : { "name" : "*foo*" } },
{ "term" : { "year" : "1999" } }
]
  }
}
}

Related

Query with id, nested array and range in Elastic Search (Open Search AWS)

I have a ES document like below :
{
"_id" : "test#domain.com",
"age" : 12,
"hobbiles" : ["Singing", "Dancing"]
},
{
"_id" : "test1#domain.com",
"age" : 7,
"hobbiles" : ["Coding", "Chess"]
}
I am storing email as id, age and hobbiles, hobbies is nested type, age is long I want to query with id, age and hobbiles, something like below :
Select * FROM tbl where _id IN ('val1', 'val2') AND age > 5 AND hobbiles should match with Chess or Dancing
How can I do in Elastic Search ? I am using OpenSearch 1.3 (latest) : AWS
I will suspect that field hobbiles is keyword, then the query suggested:
PUT test
{
"mappings": {
"properties": {
"age": {
"type": "long"
},
"hobbiles": {
"type": "keyword"
}
}
}
}
POST test/_doc/test#domain.com
{
"age": 12,
"hobbiles": [
"Singing",
"Dancing"
]
}
POST test/_doc/test1#domain.com
{
"age": 7,
"hobbiles": [
"Coding",
"Chess"
]
}
GET test/_search
{
"query": {
"bool": {
"filter": [
{
"terms": {
"_id": [
"test1#domain.com",
"test#domain.com"
]
}
}
],
"must": [
{
"range": {
"age": {
"gt": 5
}
}
},
{
"terms": {
"hobbiles": [
"Coding",
"Chess"
]
}
}
]
}
}
}

Appending list of object when querying elasticsearch

Hi I am trying to get a query based on the following documents.
document 1
{
"date" : "2021-05-19",
"items" : [
{
"keyA" : "ValueA"
}
]
}
document 2
{
"date" : "2021-05-19",
"items" : [
{
"keyB" : "ValueB"
}
]
}
Output I expect
{
"date" : "2021-05-19",
"items" : [
{
"keyA" : "ValueA"
},
{
"keyB" : "ValueB"
}
]
}
I don't want to update this structure in ES. Only when I query, I want the result to be in this format.
Is it possible? or should I handle it after receiving the result?
You can use a combination of terms and top_hits aggregation
{
"size": 0,
"aggs": {
"dateagg": {
"terms": {
"field": "date"
},
"aggs": {
"top_keys": {
"top_hits": {
"_source": {
"includes": [
"items.*"
]
}
}
}
}
}
}
}
Search Result will be
"aggregations": {
"dateagg": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 1621382400000,
"key_as_string": "2021-05-19T00:00:00.000Z", // note this
"doc_count": 2,
"top_keys": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "67598919",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"items": [
{
"keyA": "ValueA" // note this
}
]
}
},
{
"_index": "67598919",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"items": [
{
"keyB": "ValueB" // note this
}
]
}
}
]
}
}
}
]
}
}

How to write this query in elasticsearch

I have SQL query:
WHERE A = 1 AND B = 2 AND C REGEXP (eee|fff|ggg)
How can I write this query in elasticsearch ?
Not tested but something like this QUERY-DSL you need for your case with filter and regexp,
GET /_search
{
"size": 10,
"query": {
"filter" : {
"bool" : {
"must" : [
{ "term" : {"A" : 1}},
{ "term" : {"B" : 2}}
]
}
},
"regexp": {
"C": {
"value": "eee|fff|ggg"
}
}
}
}
OR
GET /_search
{
"size": 10,
"query": {
"filter": {
"bool": {
"must": [
{
"term": {
"A": 1
}
},
{
"term": {
"B": 2
}
},
{
"regexp": {
"C": {
"value": "eee|fff|ggg"
}
}
}
]
}
}
}
}

Request body variable from JSON not accepting integer values

My sample JSON file for postman runner:
[ { "name": "runner", "hitler_id": "4006abc", "year": "2017", "boolean": "false", "expected": 717962 } ]
Pre request script:
var member = data.name; var booking = data.boolean; var fyyear = data.year; var sid = data.hitler_id;
console.log(data.name); console.log(data.boolean); console.log(data.year); console.log(data.hitler_id);
Body with parameters:
{ "size": 0, "query": { "bool": { "filter": [ { "terms": { "name": [ "{{name}}" ] } }, { "terms": { "salesman_id": [ "{{sid}}" ] } }, { "terms": { "fyyear": [ "{{fyyear}}" ] } }, { "terms": { "boolean": [ "{{boolean}}" ] } } ] } }, "aggs": { "year": { "terms": { "field": "year" }, "aggs": { "value": { "sum": { "field": "value" } } } } } }
For only string variables are accepted - name and boolean fields are working and the value is populated
for the other two, the variable values are not passed.
The variables are not used in your request body that way.
Either you have to store them in environment oder global variables via
pm.globals.set("variable_key", variable_value)
pm.environment.set("variable_key", "variable_value");
or just skip the pre-request script if you just want to use your data and reference the fields directly in your body:
{
"size": 0,
"query": {
"bool": {
"filter": [
{
"terms": {
"name": [
"{{name}}"
]
}
},
{
"terms": {
"salesman_id": [
"{{hitler_id}}"
]
}
},
{
"terms": {
"fyyear": [
{{year}}
]
}
},
{
"terms": {
"boolean": [
{{boolean}}
]
}
}
]
}
},
"aggs": {
"year": {
"terms": {
"field": "year"
},
"aggs": {
"value": {
"sum": {
"field": "value"
}
}
}
}
}
}
However take care you're storing the values in your data file. You stored the bool and the year as strings". But they should be represented as you already did for the "expected" var.

failed to parse search source, expected field name but got [START_OBJECT]

My Search Body:
{
"query":{
"filtered":{
"filter":{
"bool":{
"should":[
{
"term":{
"categories":2
}
},
{
"term":{
"categories":5
}
}
]
}
}
},
"bool":{
"should":[
{
"match":{
"name":"perferendis"
}
},
{
"match":{
"brand":"in"
}
}
]
}
},
"filter":{
"and":{
"filters":[
{
"bool":{
"must_not":{
"term":{
"condition":1
}
}
}
},
{
"range":{
"highest_sales_rank":{
"gte":96
}
}
},
{
"range":{
"review_rating":{
"gte":1
}
}
},
{
"range":{
"review_count":{
"gte":12
}
}
},
{
"range":{
"upper_price":{
"gte":68
}
}
},
{
"bool":{
"must_not":{
"term":{
"updated_at":0
}
}
}
}
]
}
},
"sort":{
"updated_at":"asc"
},
"size":10,
"from":40
}
However if I take out the filtered part the query succeeds
"filtered":{
"filter":{
"bool":{
"should":[
{
"term":{
"categories":2
}
},
{
"term":{
"categories":5
}
}
]
}
}
},
I previously used this format:
"filter":{
"bool":{
"should":[
{
"match":{
"categories":"16310211"
}
},
{
"match":{
"categories":"493964"
}
}
]
}
},
but it only works with elastic search 2, and as AWS only supports 1.5.6 I am not able to use this format, related to my previous question Narrowing search result to multiple categories
the Query DSL had changes among version 1.x and 2.x, do you need change your query, i did an example.
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"categories": 2
}
},
{
"term": {
"categories": 5
}
},
{
"bool": {
"should": [
{
"match": {
"name": "perferendis"
}
},
{
"match": {
"brand": "in"
}
}
]
}
}
],
"must": [
{
"range": {
"highest_sales_rank": {
"gte": 96
}
}
},
{
"range": {
"review_rating": {
"gte": 1
}
}
},
{
"range": {
"review_count": {
"gte": 12
}
}
},
{
"range": {
"upper_price": {
"gte": 68
}
}
}
],
"must_not": [
{
"term": {
"condition":1
}
},
{
"term":{
"updated_at":0
}
}
]
}
}
}
},
"sort":{
"updated_at":"asc"
},
"size":10,
"from":40
}
And i removed your AND filter, AND filters does not cache in a good way.
Feel free to ask some questions.