Not allowed to reindex Elasticsearch? - amazon-web-services

Question

I've set my AWS Elasticsearch instance so that anyone can do anything (create, delete, search, etc.) to it.
These are my permissions (replace $myARN with my Elasticsearch ARN):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "$myARN"
}
]
}
When I PUT a new index:
PUT http://my-elasticsearch-domain.us-west-2.es.amazonaws.com/index-name
Or I DELETE an index:
DELETE http://my-elasticsearch-domain.us-west-2.es.amazonaws.com/index-name
I get this:
{
"acknowledged": true
}
Which means I can create and delete indexes but when I try to POST a reindex I get:
{
"Message": "Your request: '/_reindex' is not allowed."
}
Do I have to sign this request? Why should I have to sign this request but not creating or deleting indexes?

Answer 1

The reason is simply because the Amazon Elasticsearch Service is a kind of restricted environment where you don't have access to the full range of services and endpoints provided by a barebone install of Elasticsearch.
You can check the list of endpoints that you're allowed to use on the Amazon Elasticsearch Service and _reindex is not part of that list.
UPDATE
There's another way to achieve what you want, though. By leveraging Logstash, you can source the data from ES, apply any transformation you wish and sink it back to ES.
input {
elasticsearch {
hosts => ["my-elasticsearch-domain.us-west-2.es.amazonaws.com:80"]
index => "index-name"
docinfo => true
}
}
filter {
mutate {
remove_field => [ "#version", "#timestamp" ]
}
# add other transformations here
}
output {
elasticsearch {
hosts => ["my-elasticsearch-domain.us-west-2.es.amazonaws.com:80"]
manage_template => false
index => "%{[#metadata][_index]}"
document_type => "%{[#metadata][_type]}"
document_id => "%{[#metadata][_id]}"
}
}

Answer 2

Reindex feature will not be available in previous versions 1.5 and 2.3. So currently if you use the versions 1.5 or 2.3, it would be good for you to move on to the latest ES version so that you will get better indexing performance and other features which are not supported in previous versions.
Also have a look into the below link to know more the APIs which are supported in different versions of AWS Elasticsearch. If you look into the 5.1 section you can the “_reindex” is listed there.
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-supported-es-operations.html#es_version_5_1

Answer 3

I was able to do this using the following tool
taskrabbit/elasticsearch-dump
After installing it, you can run this on the command line:
elasticdump \
--input=http://es.com:9200/api/search \
--input-index=my_index \
--output=http://es.com:9200/api/search \
--output-index=my_index \
--type=mapping
NOTE: I did have to use the --awsChain option to find my credentials.