I have installed elasticsearch in one instance and kibana in another instance.
Both the services are running and I can connect elasticsearch using curl and its instance public ip with port 9200
version: 7.9.2 both
Assume: Public ips
elasticsearch - x.x.x.x
kibana - y.y.y.y
Issue:
Cant connect kibana instance with its curl and public ip with port 5601
Error: Failed to connect to y.y.y.y port 5601: connection refused
Query:
Correct config for elasticsearch.yml and kibana.yml
` kibana.yml:
port: 5601
server.host: "y.y.y.y"
elasticsearch.hosts: ["http://x.x.x.x:9200"]
elasticsearch.yml:
network.host: 0.0.0.0
http.port: 9200 `
It is extremely likely you have not configured the correct security group rules on the kibana instance to permit you to access the service. You need an ingress rule permitting tcp to port 5601 from whatever your ingress range is.
Likewise, it is extremely likely you have not granted access to elasticsearch (x.x.x.x:9200) from y.y.y.y
Check your security group rules.
Also, please ensure your elasticsearch public ip does not permit access from 0.0.0.0 - publically accessible elasticsearch clusters are a prime target for naughty people.
Related
My EC2 instance has the following security rules:
Unfortunately, if I browse its public IP address via HTTPS, I get "Unable to reach the site", while if I browse it via HTTP it works as it should.
SOLVED - I had to set Apache to listen on port 443.
I have Elasticseasrch and Kibana installed on EC2 instance where I am able to access Elasticsearch using on this url http://public-ip/9200. But I am unable to access Kibana using http://public-ip/5601.
I have configured kibana.yml and added certain fields.
server.port: 5601
server.host: 0.0.0.0
elasticsearch.url: 0.0.0.0:9200
On doing wget http://localhost:5601 I am getting below output:
--2022-06-10 11:23:37-- http://localhost:5601/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:5601... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83731 (82K) [text/html]
Saving to: ‘index.html’
What am I doing wrong?
Server Host set to 0.0.0.0 means it should be accessible from outside localhost but double check that the listener is actually listening for external connections on that port using netstat -nltpu. The server is also accessible on it's public IP on port 9200 so try the following:
EC2 Security Group should inbound TCP traffic on that port 5601 from your IP address.
Network ACLs should allow inbound/outbound TCP traffic on port 5601.
OS firewall ( e.g. ufw or firewalld ) should allow traffic on that port. You can run iptables -L -nxv to check the firewall rules.
Try connecting to that port from a different EC2 instance in the same VPC. It is possible that what ever internet connection you are using may have a firewall blocking connections on that port. This is common with corporate firewalls.
If these fail, next you want to check if the packets are reaching your EC2 instance so you can run a packet capture on that port using tcpdump -ni any port 5601 and check if you have any packets coming in/out on that port.
if you don't see any packets on tcpdump, use VPC Flow Logs to see if packets are coming in/out that port.
Considering the kibana port (5601 ) is open via security groups
I could able to resolve the issue by updating config server.host:localhost to server.host:0.0.0.0
and elasticsearch.hosts: ["http://localhost:9200"] (in my case kibana and ES both are running on the same machine) in kibana.yml
https://discuss.elastic.co/t/kibana-url-gives-connection-refused-from-outside-machine/122067/8
I have my Elasticsearch cluster in a VPC, I'd like to access this EC cluster from my local Macbook.
I have set up a bastion host that uses the same VPC and the same security group, and I was able to ssh into this bastion host from my Macbook.
But somehow, my code just cannot connect to my ES cluster through this bastion host, here's my command to run port forwarding:
ssh -i ~/Downloads/keypairs/20210402-02.pem ubuntu#ec2-123-456.us-west-2.compute.amazonaws.com -N -L 9200:vpc-es-domain-20210331-abc123def.us-west-2.es.amazonaws.com:443
Here's my timeout exception when accessing the ES cluster in the VPC:
java.net.ConnectException: Timeout connecting to [vpc-es-domain-20210331-abc123def.us-west-2.es.amazonaws.com/10.0.47.182:443]
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:823) ~[elasticsearch-rest-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248) ~[elasticsearch-rest-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235) ~[elasticsearch-rest-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514) ~[elasticsearch-rest-high-level-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1484) ~[elasticsearch-rest-high-level-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454) ~[elasticsearch-rest-high-level-client-7.6.1.jar:7.6.1]
at org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:497) ~[elasticsearch-rest-high-level-client-7.6.1.jar:7.6.1]
Here are the rules of my SG:
Inbound:
All TCP TCP 0 - 65535 0.0.0.0/0
All traffic All All sg-abc123 / default
SSH TCP 22 0.0.0.0/0
Outbound:
All traffic All All 0.0.0.0/0
When I've ssh'ed into my bastion host, and run curl vpc-es-domain-20210331-abc123def.us-west-2.es.amazonaws.com, I got this response:
{
"name" : "abc123",
"cluster_name" : "abc123097:es-domain-beta-20210331",
"cluster_uuid" : "abc123def",
"version" : {
"number" : "7.8.0",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "unknown",
"build_date" : "2021-01-15T06:15:47.944536Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
I'm suspecting my command to do port forwarding is not correct? But after research, this looks the most legit option to me.
Any insight would be greatly appreciated!
The code running on your local computer is trying to connect directly to the Elasticsearch server without going through the SSH tunnel. The SSH command is opening a tunnel from your local port 9200 to the remote server. The local software trying to connect to Elasticsearch should be connecting to localhost:9200 not vpc-es-domain-20210331-abc123def.us-west-2.es.amazonaws.com/10.0.47.182:443.
The endpoint vpc-es-domain-20210331-abc123def.us-west-2.es.amazonaws.com/10.0.47.182:443 doesn't look valid anyway. It has a hostname and an IP address in there.
You mentioned in the comments:
"I actually launched my bastion host using the same SG as my ES"
However just placing two resources in the same security group does nothing unless that security group also has a rule specifically allowing traffic between the resources within it. Security groups do not have this rule by default, except for the default security group in the default VPC that is created automatically when you first create your AWS account.
So please make sure that the security group has a rule that will allow the bastion host to connect to the Elasticsearch server over port 443.
I'm having trouble connecting my EC2 instance via ssh. Currently my session times out when I try to connect.
I have a security group with the following settings
Inbound:
Type: All traffic
Protocol: All
Port Range: All
Source: 0.0.0.0/0
Outbound:
Type: All traffic
Protocol: All
Port Range: All
Destination: 0.0.0.0/0
I followed the instructions on saving the private key and converting it to use with putty. When I put the public dns into putty, I am unable to connect. I verified the host name resolves by an online DNS checker.
On the client side, I launch putty and put the following information in:
Host name (or IP address): ec2-user#<Public DNS value>
Port: 22
Connection Type: ssh
In the connection->ssh->Auth->Private Key File for Authentication I point it to my private key from AWS after it has been transformed to a ppk.
Is there anything else I need to setup to be able to connect to the EC2 instance?
It turned out to be an issue with the account and not a technical issue. For whatever reason my account was set to isolated mode by Amazon. The AWS tech support verified that all of the settings were correct.
I have an EC2 instance on Amazon (AWS). The instance is behind a ELB (Elastic Load Balancer). I want to allow HTTPS connections to reach the EC2 instance.
Is it necessary to have the load balancer configured for HTTPS, ie, to check the certificates etc, or can this just be done traditionally within the EC2 instance and virtual host SSL configuration ?
The reason I'm asking is because I have allowed traffic via ELB -> EC2 for port 80 and 443, but only port 80 reaches the instance.
EDIT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00021s latency).
Not shown: 996 closed ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
443/tcp open https
3306/tcp open mysql
EDIT 2
Here is my other stack overflow questions explaining the bigger problem I have, hence why I opened this question. HTTPS only works on localhost
Check whether any application is running on port 443.
Use this command to check:
nmap -sT -O localhost
EDIT
Add the certificate files on the server and then upload them to IAM using the command:
aws iam upload-server-certificate --server-certificate-name my-server-cert
--certificate-body file://my-certificate.pem --private-key file://my-private-key.pem
--certificate-chain file://my-certificate-chain.pem
For more info check this:
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/ssl-server-cert.html