I'm using elastic4s in my scala project to communicate with ElasticSearch. For development I started a local node and everything is working fine. For production, I would like to use Amazon Elasticsearch Service. I've configured that service and allowed access to it via the ip of my ec2 instance. I can verify that it works by ssh-ing into ec2 and doing:
curl search-blabla-blabla.us-east-1.es.amazonaws.com/_cluster/health
However, I am having trouble connecting elastic4s to that ES instance. I'm trying:
ElasticClient.remote("search-blabla-blabla.us-east-1.es.amazonaws.com", 9300)
which results in:
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
Reading the docs, it appears that elastic4s can only connect via TCP [1] and Amazon Elasticsearch Service does not support TCP[2]: The service supports HTTP on port 80, but does not support TCP transport.
Could someone confirm that elastic4s and Amazon ES really don't work together? Because that would mean I have to re-write all of my ES code.
Amazon's Elasticsearch service does not support TCP, just the HTTP protocol. Elastic4s uses TCP only, as does the Elasticsearch Java API (Elastic4s being a wrapper around the Java client).
If you want to use HTTP you'll need to write the queries by hand and use a regular HTTP client, or use an Elasticsearch client that supports the HTTP protocol.
Update:
As of version 5.2, elastic4s supports HTTP as well as TCP. This is currently experimental. https://github.com/sksamuel/elastic4s#quick-start
Related
I'm trying to deploy an UDP-based application on kubernetes, but I'm having troubles finding a suitable cloud provider that has an UDP loadbalancer with IP-based sticky sessions.
I have tried using DigitalOcean Kubernetes Service (DOKS) but they don't support UDP loadbalancers.
EKS (AWS' kubernetes service) provides UDP support with NLB for example, but they don't seem to have sticky sessions on that type of loadbalancer, only on the classic LB.
Is there another cloud provider (I'm thinking of GCE or Azure) that provides my required functionalities out of the box?
I'm asking this here to know if anyone else has had the same problem and maybe has already tried various solutions, and has already found the perfect fit.
I know in Nginx Ingress Controller (which I know works with AWS and NLB with UDP support as you stated) can expose UDP services and supports sticky sessions. I have not done this in AWS or any other cloud provider, but I have with similar use cases on bare-metal.
As #jordanm posted, the answer was to apply the stickiness parameter through the ec2 console.
I have an existing application that I was trying to deploy it to google cloud run, but the application can't connect to an external NATS services. It seems that cloud run only supports http, websockets, and gRPC for outbound connections and traffic. It doesn't seem that an application can make a pure TCP/IP connection with binary data from a cloud run application.
Just looking for someone to confirm this. I have not been able to find any documentations that explicitly states the inbound and outbound limitations of cloud run.
Note: my app can connect to REDIS though.
Thank you for your help.
There's no outbound protocol limitation.
In terms of inbound, Cloud Run only supports HTTP/1 and HTTP/2 (which includes gRPC) over TLS (HTTPS). That said, server-streaming is not supported (e.g. websockets or streaming gRPC calls).
Therefore, you cannot serve with arbitrary TCP wire protocols on Cloud Run.
Yes according with this document, Cloud Run just support HTTP, WebSockets and gRPC by the moment, also here is pointed the http requests not supported, I suggest you create a feature request, if the petition earns enough people popularity this feature can be added to Cloud Run
I have an existing LAMP project on AWS (Elastic-beanstalk). I now what to set up web sockets on AWS too. According to AWS documentation, the way to do that is via AWS API Gateway. I don't know how this all works but there's documentation I found for setting up WebSockets.
Does the Gateway API connect to another service? If so, what service is this? What am I missing?
I mostly just want to make a Web Socket service to look incoming messages to the MySQL database on my Elastic-beanstalk project. I am totally confused about how to do this. Can anyone advise me about what steps I need to take?
Just because api gateway supports web sockets, doesn't mean you need to use it. ALBs do as well and are a more exact fit for elastic beanstalk.
Does an Application Load Balancer support WebSockets?
AWS doesn't support PHP (Ratchet) Web Sockets
I have to connect AWS Elasticsearch with Alexa. According to a file, I have to put
client = new ElasticSearchClient("your.elastic.url", 9300, "your.cluster.name");
What is the elastic URL and 9300 for my AWS hosted ES Cluster?
As mentioned in the comment, AWS ES does not offer the possibility to connect via TCP.
TCP transport
The service supports HTTP on port 80, but does not support TCP
transport.
developerguide aes-supported-resources
But you can try this how-to-use-java-high-level-rest-client-with-spring-boot-to-talk-to-aws-elasticsearch.
Another thing, it totally depends upon the underlying library that is https://github.com/unterstein/elastic-alexa you can give it try with https with out port.
As per your question you can get AWS ES cluster details and endpoint here.
ES -> Dashboard -> select domain
in the Top, you see cluster name and below in endpoint.
I developed our websocket project on wildfly. When we test it on localhost or within our local network, everything work fine. But when I deployed it on AWS, websocket don't work any longer. We can access other html pages. But when we conenct to "ws://ip/project location ", chrome just says hand shake error. I have experienced the same web socket problem on jelastic hosting too. My question is
Why it is happening like this?
Is websocket protocol not stable enough?
Is there any suitable hosting for websocket projects in java?
So far balancers don't forward websocket headers. To make WS working you must have a public IP address and no other services in front of your application.
I suggest you try deploying to the cloud provider : Heroku - their sample app code using node.js and websockets will get you up and running quickly. A locally running websocket app which uses a specific port - say 8888 will run fine on heroku with :
var port = process.env.PORT || 8888;
as heroku internally will deploy your app with a run-time generated port visible via PORT .
If you are using node.js with websockets I suggest using the einaros ws implementation
var WebSocketServer = require("ws").Server;
which seamlessly handles the notion of ws port -vs- the http port
Currently ELB doesn't support Websocket in HTTP mode. To be able to handle Websocket you need to configure the ELB in tcp mode (the payload of the tcp connection will be send directly to the server, so the ELB doesn't impact the http and ws flow). With this set up you won't be able to see the caller ip.
Without the ELB Websocket works perfectly (AWS only sees ip traffic and the OS only tcp one), we haven't change any thing for a plain old http server in order to use WS (except the WS handling code in the web server).
To know if you are using ELB look at the bill, AWS can provide you a lot of very interesting services, for a fee.