I am very new to elassandra and facing an issue while trying to update an index that i created before. I initially created the index as below
curl -XPUT 'http://xxx.xx.xx.xxx:xxxx/yesanoa?pretty' -H 'Content-Type:
application/json' -d'
{
"settings": {
"index.mapping.ignore_malformed": true,
"analysis": {
"normalizer": {
"yesanoa_norm": {
"type": "custom",
"char_filter": [],
"filter": ["lowercase", "asciifolding"]
}
}
}
},
"mappings": {
"movies": {"discover" : ".*","properties" : {"title" : {
"type" : "text", "normalizer": "yesanoa_norm", "index" :
"analyzed"},"site_review" : {
"type" : "text", "normalizer": "yesanoa_norm", "index" :
"analyzed"}}}}}'
And I'm trying to update this as below,
curl -XPUT 'http://xxx.xx.xx.xxx:xxxx/yesanoa/genre' -d'{
"mappings": {
"genre": {"discover" : ".*","properties" : {"title" : {
"type" : "text", "normalizer": "yesanoa_norm", "index" :
"analyzed"}}}}}'
I'm getting the following error
"No handler found for uri [/yesanoa/genre] and method [PUT][root#dbcasyn elassandra-5.5.0.4]"
I tried rebuilding the index using nodetool option. But nothing worked.
Please help me guys on this.
When you have an existing index and want to add a new type (until Elaticsearch/Elassandra v5), you use the following syntax :
PUT anIndex/_mapping/aType
{
"properties": {
"aProperty": {
"type": "text"
}
}
}
See the elasticsearch Put Mapping documentation
EDIT: By the way, the mapping update does not support the discovery option. You have do it explicitly.
Related
How can I change mapping or my input to resolve these error, using elasticsearch on AWS,
Mapping:
{
"index_patterns": ["*-students-log"],
"mappings": {
"properties": {
"Data": {
"type": "object",
"properties": {
"PASSED": {
"type": "object"
}
}
},
"insertion_timestamp": {
"type": "date",
"format": "epoch_second"
}
}
}
}
My data :
curl -XPOST -H 'Content-Type: application/json' https://******.us-east-1.es.amazonaws.com/index_name/_doc/1 -d '{"Data": {"PASSED": ["Vivek"]},"insertion_timestamp": 1591962493}'
Error I got :
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"object mapping for [Data.PASSED] tried to parse field [null] as object, but found a concrete value"}],"type":"mapper_parsing_exception","reason":"object mapping for [Data.PASSED] tried to parse field [null] as object, but found a concrete value"},"status":400}
What is the missing or wrong piece in the above data? Is there any other datatype I should use for array of string?
Any help would be appreciated...
JSON arrays are not considered JSON objects when ingested into Elasticsearch.
The docs state the following regarding arrays:
There is no dedicated array datatype. Any field can contain zero or
more values by default, however, all values in the array must be of
the same datatype.
So, instead of declaring the whole array as an object, speficy the array entries' data type (text) directly:
PUT abc-students-log
{
"mappings": {
"properties": {
"Data": {
"type": "object",
"properties": {
"PASSED": {
"type": "text"
}
}
},
"insertion_timestamp": {
"type": "date",
"format": "epoch_second"
}
}
}
}
POST abc-students-log/_doc
{
"Data": {
"PASSED": [
"Vivek"
]
},
"insertion_timestamp": 1591962493
}
I'm working on AWS Elastic Search. I've come across one situation in my project where in my reports i have to search keywords like "corona virus".
But result should come with containing keywords like "Corona virus" and "corona" and "virus" and "coronavirus".
Please guide me how i should build my query DSL.
Note: Working on PHP language.
Appreciate your help.
//Amit
You need to use shingle token filter
A token filter of type shingle that constructs shingles (token
n-grams) from a token stream. In other words, it creates combinations
of tokens as a single token. For example, the sentence "please divide
this sentence into shingles" might be tokenized into shingles "please
divide", "divide this", "this sentence", "sentence into", and "into
shingles".
Mapping
PUT index91
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"filter": [
"lowercase",
"shingle_filter"
]
}
},
"filter": {
"shingle_filter": {
"type": "shingle",
"min_shingle_size": 2,
"max_shingle_size": 3,
"output_unigrams": true,
"token_separator": ""
}
}
}
},
"mappings": {
"properties": {
"title": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
Data:
POST index91/_doc
{
"title":"corona virus"
}
Query:
GET index91/_search
{
"query": {
"match": {
"title": "coronavirus"
}
}
}
Result:
"hits" : [
{
"_index" : "index91",
"_type" : "_doc",
"_id" : "gNmUZHEBrJsHVOidaoU_",
"_score" : 0.9438393,
"_source" : {
"title" : "corona virus"
}
}
It will also work for "corona", "corona virus","virus"
When trying to apply MethodResponse template I am failing to see any difference in final response. My goal is to successfully apply schema with minItems and maxItems for array property.
Example response from lambda method:
{
"_id": "5d5110f52e8b560af82dec69",
"index": 0,
"friends": [
{
"id": 0,
"name": "Mcconnell Pugh"
},
{
"id": 1,
"name": "Peggy Caldwell"
},
{
"id": 2,
"name": "Jocelyn Mccarthy"
}
]
}
Schema I have tried to apply in MethodResponse:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title" : "Empty Schema",
"type" : "object",
"properties" : {
"friends" : {
"type" : "array",
"minItems" : 1,
"maxItems" : 2,
"items" : {
"type" : "object",
"properties" : {
"name" : {
"type" : "string"
},
"id": {
"type" : "integer"
}
}
}
},
"index" : {
"type" : "string"
}
}
}
I would expect to see only two "friends" in final response, not all of them.
After long research and a lot of AWS documentations I have found that:
It support JSON Schema 4, however not all features are supported -> related docs
Method Response basically does not apply validation. In my understanding, it just bring use if you want to export your API to Swagger to have well described specs -> related docs last Paragraph is important
So final answer to my question would be - you just cannot use Method Response as a filter, that's not the purpose of it.
(Here's my Model scheme:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "QuestionsModel",
"type": "array",
"items": {
"type": "object",
"properties": {
"section_name": { "type": "string" },
"options" : {
"type" : "array",
"items" : {
"type" : "array",
"items" : {
"type" : "string"
}
}
}
}
Here's the Mapping template:
#set($inputRoot = $input.path('$'))
[
#foreach($question in $inputRoot) {
"section_name" : "$question.section_name.S",
"options" : [
#foreach($items in $question.options.L) {
[
#foreach($item in $items.L) {
"$item.S"
}#if($foreach.hasNext),#end
#end
]
}#if($foreach.hasNext),#end
#end
]
}#if($foreach.hasNext),#end
#end
]
Although this syntax correctly maps the data it results in "options" being an empty array.
Without the "options" specified then my iOS app receives valid JSON. But when I try various syntaxes for "options" then I either get invalid JSON or an "Internal Service Error" and CloudWatch isn't much better offering Unable to transform response.
The options valid is populated with this content: {L=[{"L":[{"S":"1"},{"S":"Dr"}]},{"L":[{"S":"2"},{"S":"Mr"}]},{"L":[{"S":"3"},{"S":"Ms"}]},{"L":[{"S":"4"},{"S":"Mrs"}]},{"L":[{"S":"5"},{"S":"Prof."}]}]} which is provided by a Lambda function.
I can only conclude, at this point, that API Gateway VTL doesn't support nested arrays.
AWS iOS SDK for Modelling doesn't support array of arrays.
You have to define a dictionary in between any nested arrays.
So instead of array/object/array/array you slip in an extra "awshack" object: array/object/array/awshack-object/array
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "QuestionsModel",
"type": "array",
"items": {
"type": "object",
"properties": {
"section_name": { "type": "string" },
"options" : { "type" : "array",
"items" : {
"type" : "object",
"properties" : {
"awshack" : {
"type" : "array",
"items" : { "type" : "string" }
}
}
}
}
}
}
}
In the mapping template the "awshack" is slipped in outside the innermost loop.
#foreach($items in $question.options.L)
{"awshack" :
[#foreach($item in $items.L)
"$item.S"#if($foreach.hasNext),#end
#end
#if($foreach.hasNext),#end
]}#if($foreach.hasNext),#end
#end
Amazon confirms this limitation.
I'm trying to define dynamic templates in Elastic Search to automatically set analysers for currently undefined properties for translations.
E.g. The following does exactly what i want, which is to set lang.en.title to use the english analyzer:
PUT /cl
{
"mappings" : {
"titles" : {
"properties" : {
"id" : {
"type" : "integer",
"index" : "not_analyzed"
},
"lang" : {
"type" : "object",
"properties" : {
"en" : {
"type" : "object",
"properties" : {
"title" : {
"type" : "string",
"index" : "analyzed",
"analyzer" : "english"
}
}
}
}
}
}
}
}
}
Which stems lang.en.title as expected e.g.
GET /cl/_analyze?field=lang.en.title&text=knocked
{
"tokens": [
{
"token": "knock",
"start_offset": 0,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
}
]
}
But what i'm trying to do is set all future string properties of lang.en to use the english analyser using a dynamic template which i can't seem to get working...
PUT /cl
{
"mappings" : {
"titles" : {
"dynamic_templates" : [{
"lang_en" : {
"path_match" : "lang.en.*",
"mapping" : {
"type" : "string",
"index" : "analyzed",
"analyzer" : "english"
}
}
}],`enter code here`
"properties" : {
"id" : {
"type" : "integer",
"index" : "not_analyzed"
},
"lang" : {
"type" : "object"
}
}
}
}
}
The english analyser isn't being applied as lang.en.title isn't stemmed as desired -
GET /cl/_analyze?field=lang.en.title&text=knocked
{
"tokens": [
{
"token": "knocked",
"start_offset": 0,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
}
]
}
What am i missing? :)
Your dynamic template is defined correctly. The issue is that you will need to index a document with the lang.en.title field in it before the dynamic template will apply the appropriate mapping. I ran the same dynamic mapping that you have defined above in your question locally and got the same result as you.
However, then I added a single document to the index.
POST /cl/titles/1
{
"lang.en.title": "Knocked out"
}
After adding the document, I ran the analyzer again and I got the expected output:
GET /cl/_analyze?field=lang.en.title&text=knocked
{
"tokens": [
{
"token": "knock",
"start_offset": 0,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
}
]
}
The index needs to have a document inserted so that it can execute the defined mapping template for the inserted fields. Once that field exists in the index and has the dynamic mapping applied, _analyze API calls will execute as expected.