DataPusher is not working with my CKAN 2.8 install. I have DataPusher and CKAN on the same VPS (an Amazon EC2 instance). I cannot curl /api/3/action/resource_show from within the instance, but I can from outside it at the same IP address I can access the CKAN web gui from. I am using the default port settings/followed the official CKAN documentation for setting up CKAN and DataPusher/DataStore.
Upon checking the error logs (specifically datapusher.error.log in /var/log/apache2) the latest message is:
ConnectionError: HTTPConnectionPool(host='{ckan.site_url value, in this case the public IP of the instance}', port=80): Max retries exceeded with url: /api/3/action/resource_show (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f3bb0151490>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I had a similar issue but I used a different approach to solve it.
The system looks up DNS names in the /etc/hosts file before it goes to the external DNS server. I simply pointed my hostname (from the URL) to the local IP address like so:
172.16.22.2 ckan.installation.url
This way, the server connects to itself when it needs to reach ckan.installation.url and users connect to ckan.installation.url (public facing IP) when they need to access the site.
Ultimately the issue is that with an AWS EC2 VPS, your Ubuntu instance is not aware of its public-facing IP address, which is probably what you're using to reach the CKAN web gui hosted on said VPS.
Ideally the CKAN API can be hit internally but I have been unable to do so with localhost/127.0.0.1 in place of the VPS's external/public-facing IP address. The issue with setting the ckan site_url to localhost is that is what you will be directed to from the CKAN web gui when attempting to use DataPusher (e.g. manually initiating upload of a resource to the DataStore). Your computer obviously won't know localhost refers to the CKAN dev server... So in short, the ckan site_url value must be something accessible by both DataPusher and people/devices on the public Internet (assuming you want your CKAN instance to be publicly-accessible).
The solution here is to open port 80 to the public IP address of the AWS EC2 instance in the inbound rules of the instance's security group. In other words, you are letting the instance hit itself at port 80. Seem inefficient, but I don't have an alternative at the moment. It's better than nothing!
Related
I have configured a Hashicorp Vault server on a EC2 instance. When trying to use postman to test transit secret engine API I keep getting a error connection refused on postman, I went full ape mode and opened all ports on the security group inbound rule and it didn't work, I attached an elastic IP to the instance and didnt work either, im just trying with a simple GET and I just keep getting the same connectionrefused error.
When I use cUrl on the ssh connected session i have no issues though. The specified hosted adress is 127.0.0.1:8200, in postman I replaced that localhost with the public adress of the instance that i obviously censored in the screencap, in the headers theres the token needed to access vault, for simplicity I was just using the root token.
Postman screecap if it helps
#Emilio Marchant
I have faced similar issue (not with postman, but with telnet), Let's try to understand problem here.
The issue is with 127.0.0.1 IP. This is loopback IP and When you (or your computer) call an IP address, you are usually trying to contact another computer on the internet. However, if you call the IP address 127.0.0.1 then you are communicating with the localhost – in principle, with your own computer.
Reference link : https://www.ionos.com/digitalguide/server/know-how/localhost/
What you can try is below.
Start vault dev server with --dev-listen-address parameter.
Eg:
vault server -dev -dev-listen-address="123.456.789.1:8200"
in above command replace '123.456.789.1:8200' with '<your ec2 instance private IP : 8200'>
Next set VAULT_ADDR and VAULT_TOKEN parameter as below
export VAULT_ADDR='http://123.456.789.1:8200'
export VAULT_TOKEN='*****************'
Again replace 'http://123.456.789.1:8200' with 'http://[Your ec2 instance private IP]:8200'
For Vault_token : you should get a root token in console, when you start vault server , use that token
Now try to connect from postman or using curl command. It should work.
Reference question and solution :
How to connect to remote hashicorp vault server
The notable thing here is that the response is "connection refused". This error means that the connection is getting established and it found that there are no processes running on that port. This error means that there is no issue with firewall. A firewall will cause the connection to either drop (reject) or timeout (ignore), but won't give "Econnrefused".
The most likely issue is that the vault server process is not bound to the correct network interface. There must be a configuration in hashicorp-vault to setup the IP on which to bind. Most servers, by default, bind only on loopback address which is accessible only from 127.0.0.1. You need to bind it to "all" network interfaces by changing that to 0.0.0.0. I am not aware of the specific configuration option of hashicorp vault, but there has to be something to this effect.
Possible security issue:
Note that some servers expect you to run it behind a reverse proxy so that you can setup SSL (https) and other authentication if needed. Applications like vault servers should not be publicly accessible on http without SSL.
I'm attempting to install WSO2 GREG in a docker container using docker-toolbox (Kitematic and boot2docker) on my Macbook Pro.
The application is up and running just fine but it does some redirects during login that the networking layers in Kitematic->boot2docker seem to confuse.
Here are the details:
My guest systems IP is 192.168.99.100. Accessing the application via that IP works just fine.
When I attempt to login, the app redirects my browser to a URL using the containers IP (172.17.0.2) rather than the guest's IP (192.168.99.100). Unfortunately, the container IP is not accessible to my host computer and the redirect fails with a page not accessible.
I've updated the configuration file for the app to point to the Guest OS's IP in what I believe is the relevant configuration file (repository/conf/identity/sso-idp-config.xml).
I have updated the carbon.xml file to reference the Guest IP (192.168.99.100) but the application still seems to start on the containers IP.
[2016-01-28 18:43:36,737] INFO {org.wso2.carbon.ui.internal.CarbonUIServiceComponent} - Governance Center Publisher Default Context : https://172.17.0.2:9449/publisher
So I think there are two possible answers I'm looking for:
How can I configure GREG to startup and reference the guest IP rather than the container's IP?
How can I configure boot2docker or my local machine to forward requests on the containers IP as requests to the guest VM?
I'm not sure about GREG, but in IS and AM that happens if you are not defining a particular IP/Hostname in <HostName> and <MgtHostName> in repository/conf/carbon.xml (they are commented out by default)
I have set up a a micro EC2 instance on AWS. Currently, I am using the free tier in Oregon. There are two problems which I am facing.
When I try to SSH the instance using the public DNS, it says host does not exist but when I try conencting it using the public IP, it connects to it. What setting is needed to use the public DNS ?
I have opened the SSH client using the IP address. I want to set up my application which needs Node.js and MongoDB. I installed Node.js using this
Next I installed MongoDB using this
Then I connected to my instance using Filezilla and uploaded my code to it. I then start my node application which uses socket.io.
When I try to connect to socket.io server using web browser, I get a message which says connection refused "error 111". I have opened TCP port 80 in instance's security groups. In iptables, I have forwarded port 80 to 8080, but still it does not work. I have also checked that the firewall is disabled in ec2. Kindly help me to resolve this issue.
Did you check if all of the necessary ports are open on Amazon Security Policy?
What you can do is to allow all traffic on Amazon Security Policy for test and see if the connection goes well or not.
You might also check if you need access DB from outside. In that case, you also have to open the mongodb port and setup mongodb correctly as well.
Other tools that might useful to test firewall and connection issue will be tcpdump and syslog file
For the dns issue, did you try to nslookup on that name and see if the IP shown matches your server IP?
As Amazon gives a long DNS hostname for the server, I always use my own domain name. It's much easier.
example : ec2.domainname.com, which points to the Amazon IP address
Hope that help.
My problem is resolved now..
For the DNS issue, earlier I needed proxy to access internet, so I guess the DNS name was not getting resolved. When I tried using proxy free internet, I was able to ssh using public DNS.
And regarding connection to socket.io, I used port 8080 instead of 80 and used "sudo node main.js" to run my node file. Now I am able to connect to the socket.io server and MongoDB.
Another thing which I want to ask is that would running the node file with sudo rights create some security issue ?
Thanks for the answer! That also worked for me. I had the same problem trying to connect through sockets (http://myipaddress:3000) to a node.js server, i tried opening ports on the actual ec2 instance and disabling the firewall through SSH but nothing worked. Had to go to Security Groups on the ec2 console and open a new inbound tcp rule enabling that port
I have created a fresh EC2 instance, installed Apache2 and pointed my domain (hamidlab.com) to ip of this instance. When I browse my domain it shows default apache/ubuntu page, then I stopped apache2 service and try to access (hamidlab.com) it still shows apache/ubuntu default page, now when I try to access 1.hamidlab.com it says
Could Not Connect
Description: Could not connect to the requested server host.
and returns Header Status Code: 502 Connection refused
I tried with nginx server, still same caching issue.
Do AWS have any caching set ?
I am not using any other service than ec2.
I had a Micro Instance from which I created an AMI Image. I then upgraded to a Large Instance with this Image in tow and assigned an elastic IP Address. I changed my A Name to point to the new IP and, according to a reverse DNS lookup service, my DNS appears to have propagated correctly (cranku.com).
I created a virtual host for the domain name and restarted apache. And, yet, the domain is not responding to my requests. Could I be missing something here?
I am deploying Django with Mod Wsgi on Apache. I have moved MYSQL to a mounted EBS volume but that seems to be working here (and it worked on the instance from which I created the AMI). Restarting Apache works (/etc/apache/init.d/restart). Do I have to configure it in any other ways.
Any clues on how to proceed?
I can reach your ssh server on the machine, but attempts to reach the webserver here are failing too, in a manner that makes me think the packets are being DROPed rather than REJECTed. Have you authorized port 80?