Unable to send data to AWS elastic search instance using logstash - amazon-web-services

Question

I am trying to send data to AWS elastic search end point using logstash that is installed on my local machine.
The logstash conf file looks like this
input {
file {
path => "/path/log.txt"
}
}
output {
amazon_es {
hosts => ["https://search-abclostashtrial-5jdfc43oqql7qsrhfgbvwewku.us-east-2.es.amazonaws.com"]
action => "index"
region => "us-east-2"
index => "trial"
ssl => true
}
}
The Elastic search Access policy looks like this
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-2:0415721453395:domain/abclostashtrial/*"
}
]
}
I am using logstash-output-amazon_es plugin to send the query like
sudo bin/logstash -f /path/logstash/abc.conf
And I get the following error log.
[ERROR] 2019-04-30 20:05:52.900 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[INFO ] 2019-04-30 20:05:53.165 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2019-04-30 20:05:58.037 [LogStash::Runner] runner - Logstash shut down.
What am I missing here ?

Answer 1

One option to start with is to create an AccessKey that has rights to write to Elasticsearch, and configure that in the output. Example:
amazon_es {
hosts => ["vpc-xxxxxxxxx-es-yyyyyy4pywmwigwi47em.us-east-1.es.amazonaws.com"]
region => "us-east-1"
aws_access_key_id => 'AKIxxxxxxxxxxx'
aws_secret_access_key => '11xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
index => "production-logindex-%{+YYYY.MM.dd}"
}