I deployed an ES service 5.1 over AWS. The access policy is restricted to a unique IP. This public IP belongs to a EC2 instance (red hat).
At the EC2 instance, I installed kibana 5.1.2 using yum and I configured the kibana.yml as follows:
server.host: "ec2-x-x-x-x.us-west-2.compute.amazonaws.com"
server.name: "my-kibana"
elasticsearch.url: "http://my-es-domain-xxxxxxxxxxxx.us-west-2.es.amazonaws.com"
The rest config remains untouched.
With the EC2 instance I can do cURL requests over my ES with no problem (I had created an index, add some documents and search). But Kibana is giving me this error:
ui settings Elasticsearch plugin is red
plugin:kibana#5.1.2 Ready
plugin:elasticsearch#5.1.2 Request Timeout after 3000ms
plugin:console#5.1.2 Ready
plugin:timelion#5.1.2 Ready
So I can't use my Kibana client. Can you tell what's missing?
Thanks.
When you configure kibana with
elasticsearch.url: "http://my-es-domain-xxxxxxxxxxxx.us-west-2.es.amazonaws.com"
it always look for es running on 9200 port by default.
If you are running es on 80 port then explicitly mention that in configuration
like this:
elasticsearch.url: "http://my-es-domain-xxxxxxxxxxxx.us-west-2.es.amazonaws.com:80"
You can read more about this setting here
elasticsearch.url:
Default: "http://localhost:9200" The URL of the Elasticsearch instance
to use for all your queries.
I had this problem locally, too. Elasticsearch and Kibana on local Debian 8 system. Although telnet/curl for localhost:9200 worked, I had to enable it in iptables
iptables -A INPUT -p tcp --dport 9200 -j ACCEPT
Maybe this helps someone else, too.
Related
I configured ELK stack in ec2 Ubuntu,18.04 LTS instance-aws. I configured elasticsearch, logstash & finally kibana in Ubuntu instance and started those with no issues. My question is, once I started kibana, have to view the kibana dashboard in my desktop(windows 7 machine) browser. But, always getting "This site can’t be reached" message. Please find the below commands for detail.
Completed the installation & configuration of elasticsearch, logstash in Ubuntu,18.04 LTS instance-aws without errors.
Now for kibana I installed using,
sudo apt-get install kibana
removed the comments of below lines from "/etc/kibana/kibana.yml"
server.port: 5601
server.host: "localhost"
elasticsearch.url: "http://localhost:9200"
then started the service using,
sudo service kibana start
then hit the url the http://YOUR_ELASTIC_IP:5601 in my windows browser and given the 'aws ubuntu instance - IPv4 Public IP' for 'YOUR_ELASTIC_IP'.
Anyone can help me to view the kibana from my windows machine which is installed in ec2 ubuntu instance...
Might be a security group issue, make sure that port(5601) is open for your ip address
I need to put sonatype nexus3 up on AWS. Following an old tutorial for nexus 2, I was led to try this on EC2. What I'm currently trying is an instance with a security group that allows inbound requests from anywhere on ports 80,8080,22,4000,443, and 8081. I'm using a Amazon Linux AMI 2016.09.0 (HVM), SSD Volume Type instance. I install docker using the instructions from here http://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html#install_docker. I then simply use the official docker image from here https://hub.docker.com/r/sonatype/nexus3/ with the following command.
docker run -d -p 8081:8081 --name nexus sonatype/nexus3
Using docker ps I can confirm that this seems to be running. When I try to connect to the provided public DNS url ending with amazonaws.com on port 8081, I simply get connection refused. Same thing on port 80 or any of the other ports and the same thing when I add /nexus to the end of the URL.
Attempting the quick test that documentation for this image suggests:
>curl -u admin:admin123 http://localhost:8081/service/metrics/ping
curl: (56) Recv failure: Connection reset by peer
Using the exact same docker command on my local machine (OS X) I am able to access nexus on localhost. Why can't I get this working?
The issue appears to have been with Sonatype's official image. This image which works the exact same way, works perfectly with the exact same process.
I currently have an EC2 instance that I am using to host my mongodb sever on from port 27017. Previously I had just setup the security group to just use my home IP address to authenticate a TCP connection to port 27017, however I no longer have a static IP. I now have one that changes everyday that I cannot control. Is there a way to create a mongo URI like mongolabs has
mongodb://<username><passs>#<my EC2 IP>:27017/db
that I can use to connect from PyMongo.
There are many, many guides available by searching that describe how to enable MongoDB authentication.
Alternatively, you could create a small script that uses the AWS CLI to update the security group with your current IP address. The script could be run when needed or set to run automatically your computer starts or you log in.
Install AWS CLI on your machine. You should have proper IAM permissions to update the security group. Then you can use below bash script to update your security group with your current IP address.
#!/bin/bash
ip = 'curl -s http://whatismijnip.nl |cut -d " " -f 5'
sleep 5
aws ec2 authorize-security-group-ingress --group-name MySecurityGroup --protocol tcp --port 22 --cidr $ip/24
I am trying to access my ElasticSearch on a running EC2 instance from outside the Cloud. I currently have SSH/HTTP/HTTPS open to the public for inbound traffic as well as all open for outbound traffic. I set up a public IP for my EC2 instance as well.
By default ElasticSearch is on port 9200. I'm not sure if I configured my elasticsearch.yml file correctly but it basically has the default configuration I only changed the cluster.name to something else.
When I type in my public IP with port 9200 into my local browser or locally do a telnet {public-ip} 9200, there is no response. When I SSH into my EC2 instance. I can perform a curl localhost:9200 and I get the correct response from elasticsearch
How can I connect to my ElasticSearch running on my EC2 instance from outside the cloud?
I added a Custom Rule for my security group for inbound traffic that includes port 9200 and is open to 0.0.0.0/0 and I still cannot access this EC2 instance
Potential issues to check are wrong binding and instance operating system firewall.
Check where elasticsearch is binding, as if it is binding to 127.0.0.1 you won't be able to reach it from the outside.
Check binding by running in one shell on the elasticsearch ec2:
sudo netstat -lptun | grep 9200
If it shows 127.0.0.1:9200 then there is a misconfiguration if otherwise shows
*:9200 or :9200 then it is correct.
If it shows 127.0.0.1 then you should modify elasticsearch parameter network.bind_host as described in:https://www.elastic.co/guide/en/elasticsearch/reference/1.4/modules-network.html
Additionally http/HTTPS and ssh are usually allowed by default operating system firewall, whereas elasticsearch 9200 is not. This is usually the case for rhel and centos. You can temporarily disable iptables and check if it works.
To disable iptables run:
sudo iptables -F
If after disabling iptables the connection works you should configure iptables to allow connection on 9200.
I hope this helps.
G.
It is mess around Security Groups
You can add or remove rules for a security group (also referred to as
authorizing or revoking inbound or outbound access).
You shuld use the SG while launching your instance whith bounded 9200
Establish an SSH tunnel from your desktop to EC2.. then simply use your browser.. follow steps as given in https://www.jeremydaly.com/access-aws-vpc-based-elasticsearch-cluster-locally/
I'm able to connect to an ElastiCache Redis instance in a VPC from EC2 instances. But I would like to know if there is a way to connect to an ElastiCache Redis node outside of Amazon EC2 instances, such as from my local dev setup or VPS instances provided by other vendors.
Currently when trying from my local set up:
redis-cli -h my-node-endpoint -p 6379
I only get a timeout after some time.
SSH port forwarding should do the trick. Try running this from you client.
ssh -f -N -L 6379:<your redis node endpoint>:6379 <your EC2 node that you use to connect to redis>
Then from your client
redis-cli -h 127.0.0.1 -p 6379
It works for me.
Please note that default port for redis is 6379 not 6739. And also make sure you allow the security group of the EC2 node that you are using to connect to your redis instance into your Cache security group.
Also, AWS now supports accessing your cluster more info here
Update 2018
The previous answer was accurate when written, however it is now possible with some configuration to access redis cache from outside using the directions according to Accessing ElastiCache Resources from Outside AWS
Old Answer
No, you can't without resorting to 'tricks' such as a tunnel, which maybe OK for testing but will kill any real benefit of using a super-fast cache with the added latency/overhead.
The Old FAQ under How is using Amazon ElastiCache inside a VPC different from using it outside?:
An Amazon ElastiCache Cluster, inside or outside a VPC, is never allowed to be accessed from the Internet
However, this language has been removed in the current faq
These answers are out of date.
You can access elastic-cache outside of AWS by following these steps:
Create a NAT instance in the same VPC as your cache cluster but in a
public subnet.
Create security group rules for the cache cluster and
NAT instance.
Validate the rules.
Add an iptables rule to the NAT
instance.
Confirm that the trusted client is able to connect to the
cluster.
Save the iptables configuration.
For a more detailed description see the aws guide:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/accessing-elasticache.html#access-from-outside-aws
Not so old question, I ran to the same issue myself and solved it:
Sometimes, for developing reasons you need to access from outside (to avoid multi-deployments just for a simple bug-fix maybe?)
Amazon have published a new guide that uses the EC2 as proxies for the outside world:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/accessing-elasticache.html#access-from-outside-aws
Good luck!
BTW if anyone wants a windows EC2 solution, try these at the DOS prompt (on said windows EC2 machine):
To Add port-forwarding
C:\Users\Administrator>netsh interface portproxy add v4tov4 listenport=6379 listenaddress=10.xxx.64.xxx connectport=6379 connectaddress=xxx.xxxxxx.ng.0001.use1.cache.amazonaws.com
To list port-forwarded ports
C:\Users\Administrator>netsh interface portproxy show all
Listen on ipv4: Connect to ipv4:
Address Port Address Port
10.xxx.128.xxx 6379 xxx.xxxxx.ng.0001.use1.cache.amazonaws.com 6379
To remove port-forwarding
C:\Users\Administrator>netsh interface portproxy delete v4tov4 listenport=6379 listenaddress=10.xxx.128.xxx
We are using HAProxy as a reserved proxy server.
Your system outside AWS ---> Internet --> HAProxy with public IP --> Amazon Redis (Elasticache)
Notice that there is another good reason to do that (at that time)
As we use node.js client, which don't support Amazon DNS fail over, the client driver don't support dns look up again.
If the redis fail, the client driver will keep connect to the old master, which is slave after failed over.
By using HAProxy, it solved that problem.
Now using the latest ioredis driver, it support amazon dns failover.
This is a solid node script that will do all the dirty work for you. Tested and verified it worked.
https://www.npmjs.com/package/uzys-elasticache-tunnel
How to use
Usage: uzys-elasticache-tunnel [options] [command]
Commands:
start [filename] start tunneling with configuration file (default: config.json)
stop stop tunneling
status show tunneling status
Options:
-h, --help output usage information
-V, --version output the version number
Usage Example
start - uzys-elasticache-tunnel start ./config.json
stop - uzys-elasticache-tunnel stop
status - uzys-elasticache-tunnel status
Its is not possible to directly access the classic-cluster from a VPC instance. The workaround would be configuring NAT on the classic instance.
NAT need to have a simple tcp proxy
YourIP=1.2.3.4
YourPort=80
TargetIP=2.3.4.5
TargetPort=22
iptables -t nat -A PREROUTING --dst $YourIP -p tcp --dport $YourPort -j DNAT \
--to-destination $TargetIP:$TargetPort
iptables -t nat -A POSTROUTING -p tcp --dst $TargetIP --dport $TargetPort -j SNAT \
--to-source $YourIP
iptables -t nat -A OUTPUT --dst $YourIP -p tcp --dport $YourPort -j DNAT \
--to-destination $TargetIP:$TargetPort
I resolved using this amazon docs it says you ll have to install stunnel in your another ec2 machine.
https://aws.amazon.com/premiumsupport/knowledge-center/elasticache-connect-redis-node/