I created a private endpoint in AWS API Gateway to make it private. Only resources in the private vpc can access to this endpoint. This is my design and it works very well. The problem is that it makes me a bit hard to connect it from my local computer.
As a workaround, I can launch a EC2 instance which in the same VPC and I can connect to this EC2 to access the endpoint. But it is not easy to do. I'd like to run postman from my local to connect to the API endpoint. I am looking for a better way to allow me to access it from my local.
Can anyone help me on that?
There are few ways. One one would be VPN, but I personally use proxy capability of postman. What I do has five stages.
1. Bastion host - Amazon Linux 2
with ssh port open in a public subnet.
2. ssh dynamic tunnel
ssh -D 1080 -C -q -N -oStrictHostKeyChecking=no -l ec2-user <public-ip-of-your-bastion> -v
The command will create SOCKS5 proxy from your local workstation to the bastion on port 1080.
3. HTTP -> SOCKS5 conversion
Since postman does not support SOCK5, a conversion is needed. I use http-proxy-to-socks:
hpts -s 127.0.0.1:1080 -p 8080
which will create HTTP proxy on port 8080 forwarded to SOCKS5 proxy on port 1080.
4. Setup HTTP proxy on postman
5. Query private API from postman
Query the private API from your local workstation using its https endpoint, just like if it was a public API.
Related
I have the current setup.
Account A exposes an elasticache(redis) service via private link. The elasticache servers sit behind a NLB.
Account B accesses the service with the DNS name created during the VPC endpoint creation.
When running the cli command from Account B:
redis-cli -c -h dns-name -p 6379
The inital connection is successful. However, then when I run get foo, if the slot for key foo is not on the node connected to currently, redis redirects to another node in the server as expected. However, this redirection involves a private IP address of Account A which Account B has no idea about. Thus resulting in a timeout. Is there any workaround for this?
Is there any workaround for this?
Yes, you would need a Redis client which supports NAT mapping, as the ioredis package for Node.js. AFAIK, redis-cli does not support that.
I have configured a Hashicorp Vault server on a EC2 instance. When trying to use postman to test transit secret engine API I keep getting a error connection refused on postman, I went full ape mode and opened all ports on the security group inbound rule and it didn't work, I attached an elastic IP to the instance and didnt work either, im just trying with a simple GET and I just keep getting the same connectionrefused error.
When I use cUrl on the ssh connected session i have no issues though. The specified hosted adress is 127.0.0.1:8200, in postman I replaced that localhost with the public adress of the instance that i obviously censored in the screencap, in the headers theres the token needed to access vault, for simplicity I was just using the root token.
Postman screecap if it helps
#Emilio Marchant
I have faced similar issue (not with postman, but with telnet), Let's try to understand problem here.
The issue is with 127.0.0.1 IP. This is loopback IP and When you (or your computer) call an IP address, you are usually trying to contact another computer on the internet. However, if you call the IP address 127.0.0.1 then you are communicating with the localhost – in principle, with your own computer.
Reference link : https://www.ionos.com/digitalguide/server/know-how/localhost/
What you can try is below.
Start vault dev server with --dev-listen-address parameter.
Eg:
vault server -dev -dev-listen-address="123.456.789.1:8200"
in above command replace '123.456.789.1:8200' with '<your ec2 instance private IP : 8200'>
Next set VAULT_ADDR and VAULT_TOKEN parameter as below
export VAULT_ADDR='http://123.456.789.1:8200'
export VAULT_TOKEN='*****************'
Again replace 'http://123.456.789.1:8200' with 'http://[Your ec2 instance private IP]:8200'
For Vault_token : you should get a root token in console, when you start vault server , use that token
Now try to connect from postman or using curl command. It should work.
Reference question and solution :
How to connect to remote hashicorp vault server
The notable thing here is that the response is "connection refused". This error means that the connection is getting established and it found that there are no processes running on that port. This error means that there is no issue with firewall. A firewall will cause the connection to either drop (reject) or timeout (ignore), but won't give "Econnrefused".
The most likely issue is that the vault server process is not bound to the correct network interface. There must be a configuration in hashicorp-vault to setup the IP on which to bind. Most servers, by default, bind only on loopback address which is accessible only from 127.0.0.1. You need to bind it to "all" network interfaces by changing that to 0.0.0.0. I am not aware of the specific configuration option of hashicorp vault, but there has to be something to this effect.
Possible security issue:
Note that some servers expect you to run it behind a reverse proxy so that you can setup SSL (https) and other authentication if needed. Applications like vault servers should not be publicly accessible on http without SSL.
We want to have a Test cloud virtual network that allows us to make an snmp-get over multiple virtual devices. To achieve this I am using GNS3. Now, we just deployed a GNS3 Server on EC2 (Ubuntu 18), but we can't ping nor snmp get any router outside the GNS3 server. We can ping these devices while we are in the GNS3 server, but this does not work from another server or my computer.
The GNS3 server already created and deployed.
The VPG, Site to site VPN, and VPC are already created, and the servers were added to this VPC.
After some weeks of research, our team found the solution, if anyone is having this same problem consider these important points in your AWS configuration:
Server A (GNS3) must be in a different Subnet than Server B (Test server that you want to ping from).
A Route Table must be created in AWS config pointing to the GNS3 ips.
Configure the NAT in Server A (In my case is an Ubuntu 18) using the following instructions:
Set up IP FORWARDing and Masquerading
iptables --table nat --append POSTROUTING --out-interface ens5 -j MASQUERADE
iptables --append FORWARD --in-interface virbr0 -j ACCEPT
Enables packet forwarding by kernel
echo 1 > /proc/sys/net/ipv4/ip_forward
Apply the configuration
service iptables restart
This will allow your virtual GNS3 devices in Server A to be reached from Server B (A more detailed explanation here). Additionally, you might want to test an SNMP-WALK from Server B to your virtual device in Server A (a router in my case).
If this does not work try debugging using flow logs in AWS and looking if server A is effectively receiving the requests.
I'm trying to put use an RDS proxy to pool and share connections established with an RDS database with a PostgreSQL engine. The problem is I'm able to open a connection to the DB, both through an ECS instance or in PgAdmin, however, I'm not able to connect through the proxy. To attempt a connection through the proxy in PgAdmin, I'm using the proxy endpoint as opposed to the DB endpoint but the connection attempt times out.
I've successfully created the proxy and associated with my DB, both proxy and DB status is available. I've followed the example proxy setup and the DB and the proxy are using the same VPC security group.
Any ideas?
It seems to me that you are connecting to the proxy from outside of AWS. If this is the case, then its not possible to do this directly:
Your RDS Proxy must be in the same VPC as the database. The proxy can't be publicly accessible, although the database can be.
Your may be able to connect to RDS since it publicly accessible. RDS proxy on the other hand, can only be access from within the same VPC, e.g., from an instance.
Therefor, the solution is to setup an instance in the same VPC as your RDS and proxy. The instance must be accessible using SSH.
On the instance, you can run pgadmin4, in docker:
docker run --rm -p 8080:80 \
-e 'PGADMIN_DEFAULT_EMAIL=user#domain.com' \
-e 'PGADMIN_DEFAULT_PASSWORD=Fz77T8clJqJ4XQrQunGA' \
-d dpage/pgadmin4
The command, after setting up the docker, will server pgadmin4 on port 8080 on the instance.
You can check on the instance if its working:
curl localhost:8080
which can give the following indicating that its working:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: /login?next=%2F. If not click the link
However, since your instance is only accessible through ssh (port 22), to access it from your local workstation, you have to establish an ssh tunnel:
ssh -i <private-key> -L 8080:localhost:8080 -N ubuntu#<public-instance-ip> -v
In the above, my instance was Ubuntu. For Amazon Linux 2, the user would be ec2-user.
The tunnel will forward port 8080 from the instance (i.e. pgadmin4) to your local workstation on port 8080.
Then you just point your browser to localhost:8080 and you should see the pgadmin4 welcome screen.
P.S. My RDS and proxy settings used for the verification:
I have enabled Private Google API access for a VPC and I use this HTTP proxy solution described to connect my offsite datacenter to the Google Cloud backend.
Using the solution, I have verified that the Google object storage api's work, by using gsutil to move files across the offsite network.
However I am unable to connect to mqtt.googleapis.com that is required for cloud IOT.
I think this is because the MQTT broker running at mqtt.googleapis.com cannot be accessed via a private network unless it is also proxied like the HTTP proxy solution described above.
Meanwhile actual gsutil IOT commands work fine because I presume they are running over the Google HTTP API.
To solve this I see we'd need any one of the below, unless someone has different way to do this?
Run an MQTT broker proxy in the private VPC and route MQTT packets to the mqtt.googleapis.com . Is there a suitable MQTT proxy broker that we can use in this case?
If we get a range of public IP's that the mqtt bridge (mqtt.googleapis.com) is running at then we can simply build the network routes for this one use case. Is this available?
Would it work via the HTTP protocol bridge in IoT Core? Is using HTTP instead of MQTT an option?
I managed to get this to work using NGINX as a reverse proxy and stream the TCP traffic directly to mqtt.googleapis.com. Here are the steps to achieve this
Install Nginx with the --with-stream configuration flag . This builds Nginx with the functionality of a TCP streaming proxy
My Nginx conf file contains the following to point to Google's broker. The Nginx server is running in an instance in the VPC
/etc/nginx/nginx.conf
stream {
upstream google_mqtt {
server mqtt.googleapis.com:8883;
}
server {
listen 8883;
proxy_pass google_mqtt;
}
}
The internal private VPC has a DNS Server that resolves mqtt.googleapis.com to the IP of the Nginx server