I know that AWS Elastic Search doesn't support the transport-client (only access via http/80) so is there another way of leveraging AWS Elastic Search using Spring Data?
AWS now supports ES 2.x so getting more interesting to use.
Any help or progress on this topic?
rgds
Colin
Related
Can we use source database endpoints which are hosted on other servers than AWS server for using AWS DMS? If yes, can you help me by sharing any tutorials.
I have seen this image in AWS official documentation.
Thankyou!
Yes, you can use DMS to migrate an on-prem (or diff cloud) db to AWS and vice versa.
In same documentation from where you take snapshot for e.g you have part "Migrating an On-Premises Oracle Database to Amazon Aurora MySQL".
If you check your screenshoot you can notice "The source or target database must be on an AWS service"
You have a very nice explanation in this blog article:
https://medium.com/workfall/how-to-do-database-migration-using-aws-database-migration-service-dms-from-on-premise-ec2-to-rds-d46b9144d3cc
We are deploying multi-region (and possibly multi-cloud in the future).
Our ElasticSearch endpoint must thus be public.
I know I can add an IP-based policy on the AWS Elastic Search to essentially whitelist all endpoints which should be allowed to write their logs to the AWS ES service.
Looking for a "saner" alternative, I came across:
https://discuss.elastic.co/t/how-to-connect-beats-to-aws-elasticsearch-with-authentication/83465
and specially
https://forums.aws.amazon.com/thread.jspa?threadID=294252
the latter specifically saying:
Filebeat doesn't support IAM authentication so using it with this AWS
Elasticsearch service typically doesn't work.
However, I found this:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-aws.html#aws-credentials-options
which is the filebeat aws module, which seems to suggest that it actually may support it.
Couldn't find any official documentation nor blog which confirms that I could use filebeat on remote machines to send (authenticated? signed?) logs to a public AWS ElasticSearch endpoint, allowing me to keep the policy open without having to put a whitelist (or maybe I need both).
We were doing most of our cloud processing (and still do) using AWS. However, we also now have some credits on GCP and would like to use and want to explore interoperability between the cloud providers.
In particular, I was wondering if it is possible to use AWS S3 from within GCP. I am not talking about migrating the data but whether there is some API which will allow AWS S3 to work seamlessly from within GCP. We have a lot of data and databases that are hosted on AWS S3 and would prefer to keep everything there as it still does the bulk of our compute.
I guess one way would be to transfer the AWS keys to the GCP VM and then use the boto3 library to download content from AWS S3 but I was wondering if GCP, by itself, provides some other tools for this.
From an AWS perspective, an application running on GCP should appear logically as an on-premises computing environment. This means that you should be able to leverage the services of AWS that can be invoked from an on-premises solution. The GCP environment will have Internet connectivity to AWS which should be a pretty decent link.
In addition, there is a migration service which will move S3 storage to GCP GCS ... but this is distinct from what you were asking.
See also:
Getting started with Amazon S3
Storage Transfer Service Overview
Is there any procedure or documentation supported by Microsoft in order to migrate AWS Elasticsearch to Azure Elasticsearch? Do anyone knows the process to do so?
I don't know any information by Microsoft about it, but our AWS Elasticsearch Service supports manual snapshots, that you can store on S3 and can then use to restore a cluster elsewhere.
The people from Alibaba Cloud have a step-by-step post of how to migrate from AWS to their cloud, so you can take that as a starting point.
Is there any easy way to find the consolidated AWS Services which has tags?
I am currently looking each service in AWS documentation and finding out, which is tedious.
For example - services like ec2,s3,lambda,dynamodb - has tagging feature, so that resources can be identified by tags(assuming they are tagged).
Services like sns,sqs - do not have tagging option.
Need to identify a list of AWS services which do not support tagging.
Check: Supported Resources for Tag Editor Tagging (Console)
for services and resources that can be tagged using AWS Console.