I'm trying to access Neptune cluster status endpoint from an SSH tunnel. I can hit it without issue on my bastion host, but when doing via ssh tunnel, I get:
https://localhost:8182/status
{"detailedMessage":"Bad request.","requestId":"random-request-id-appears-here","code":"BadRequestException"}
How can I do this? It seems like I need something with sigv4. I was hoping to see the response work after hitting this in my browser.
I've also tried awscurl after setting my env variables, I get:
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=8182): Max retries exceeded with url: /status (Caused by SSLError(SSLCertVerificationError("hostname 'localhost' doesn't match either of '*.id.us-east-2.neptune.amazonaws.com', '*.id.us-east-2.neptune.amazonaws.com', '*.cluster-custom-id.us-east-2.neptune.amazonaws.com', '*.cluster-ro-id.us-east-2.neptune.amazonaws.com'")))
When using SSH tunnel for accessing Neptune using localhost, one need to explicitly pass Neptune endpoint as host header for signing the request. Consider below example for awscurl:
awscurl -k --service neptune-db --access_key $ACCESS_KEY --secret_key $SECRET_KEY --region <neptune_instance_region> --session_token $SESSION_TOKEN --header 'host: <neptune-cluster-endpoint-withouthttp-withoutport>' https://localhost:8182/status
Without the explicit host header, request would be signed using "localhost" with an invalid signature.
Related
Iam trying to set up aws cluster. I installed kubectl and configured aws with credentials. When I tried to display pods or any kubectl commands, Iam getting this error
revaa#revaa-Lenovo-E41-25:~$ kubectl get pod
Unable to connect to the server: dial tcp 10.0.12.77:443: i/o timeout
How to resolve this
The ip used is a private, it cannot be accessed outside the aws. Change your cluster's api server endpoint access to public. It worked.
I am trying to bulk load RDF N-triples data from S3 bucket into Neptune loader. I have created a S3 bucket, IAM Role, Endpoint and Neptune cluster as per the following link https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load-data.html.
And I am trying to execute a CURL command locally from windows using command prompt to load the data,
curl -X POST -H "Content-Type: application/json" https://<clusterEndpoint>:<clusterPort>/loader -d "{\"source\":\"s3://<bucketName>\",\"format\":\"ntriples\",\"iamRoleArn\":\"arn:aws:iam::<account-id>:role/<role-name>\",\"region\":\""<region>\",\"failOnError\":\"FALSE\",\"parallelism\":\"MEDIUM\",\"updateSingleCardinalityProperties\":\"FALSE\",\"queueRequest\":\"TRUE\"}"
On executing the above Curl command I am getting the following error,
Failed to connect to <neptuneClusterEndoint> port <portNumber>: Timed out
Also, when I tried to check the cluster status using the command curl http://<neptuneCluster>:<portNumber>/status , I got the same time out error.
I am trying to make a Neptune Load using Curl command without creating an EC2 instance. May I know why I am getting the connection failed error? Is there a way to use curl command to make a Neptune load successfully?
The curl command needs to have access to the Neptune VPC. That could be via an EC2 bastion host over an SSH tunnel for example. As you are trying to avoid using EC2 you will need to setup an alternative way to access Neptune such as a load balancer. You could also use a Lambda function so long as the Lambda function has access to the VPC. There are many other ways you could decide to connect but as Neptune does not expose a public IP address you will need to configure a way for your curl command to access that VPC. Also be aware that if you have IAM Authentication enabled on the Neptune cluster that the request will have to be signed using SigV4 credentials.
If you have a Neptune Notebook configured, you can just use the %load command and it can also handle any SigV4 needs for you as well.
I created a private endpoint in AWS API Gateway to make it private. Only resources in the private vpc can access to this endpoint. This is my design and it works very well. The problem is that it makes me a bit hard to connect it from my local computer.
As a workaround, I can launch a EC2 instance which in the same VPC and I can connect to this EC2 to access the endpoint. But it is not easy to do. I'd like to run postman from my local to connect to the API endpoint. I am looking for a better way to allow me to access it from my local.
Can anyone help me on that?
There are few ways. One one would be VPN, but I personally use proxy capability of postman. What I do has five stages.
1. Bastion host - Amazon Linux 2
with ssh port open in a public subnet.
2. ssh dynamic tunnel
ssh -D 1080 -C -q -N -oStrictHostKeyChecking=no -l ec2-user <public-ip-of-your-bastion> -v
The command will create SOCKS5 proxy from your local workstation to the bastion on port 1080.
3. HTTP -> SOCKS5 conversion
Since postman does not support SOCK5, a conversion is needed. I use http-proxy-to-socks:
hpts -s 127.0.0.1:1080 -p 8080
which will create HTTP proxy on port 8080 forwarded to SOCKS5 proxy on port 1080.
4. Setup HTTP proxy on postman
5. Query private API from postman
Query the private API from your local workstation using its https endpoint, just like if it was a public API.
I am new to aws as well as server and it's configurations.
I have an app server which basically is an API server.
When I test running with my localhost I am able to access it.
curl --location --request GET 'http://localhost:5000/api/Supplier'
But not with my machine public or private or elastic IPs.
curl --location --request GET 'http://xxx-xx-xx-xx:5000/api/Supplier'
i get the following error
(7) Failed to connect to xxx-xx-xx-xx port 5000: Connection refused
Are there any additional things that are needed here?
New to AWS and AWS CLI, I have installed and configured the AWS CLI, and I am simply trying to list the buckets in S3, but I am behind a proxy.
How do I troubleshoot and resolve and the following error?
C:\Users\MyUserName\Desktop >aws s3 ls
HTTPSConnectionPool(host='s3.us-east-2.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', error(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond')))
The only troubleshooting step I have attempted is to Set the HTTP_PROXY and HTTPS_PROXY variables to my IP on port 80.
The key to using the AWS CLI behind a proxy is to configure two environment variables.
The IP address is the address of your proxy server, which is probably not your local IP. Consult with your network administrator to get the correct IP address and basic authentication parameters.
Chrome, IE, etc. support proxy servers, so you may already have these parameters setup in your browser. For Chrome go to settings and search for Open proxy settings. Similar technique for other browsers.
For Windows:
set HTTP_PROXY=http://a.b.c.d:n
set HTTPS_PROXY=http://w.x.y.z:m
Or for basic authentication:
set HTTP_PROXY=http://username:password#a.b.c.d:n
set HTTPS_PROXY=http://username:password#w.x.y.z:m
For Linux, macOS, or Unix:
export HTTP_PROXY=http://a.b.c.d:n
export HTTPS_PROXY=http://w.x.y.z:m
Or for basic authentication:
export HTTP_PROXY=http://username:password#a.b.c.d:n
export HTTPS_PROXY=http://username:password#w.x.y.z:m
Using an HTTP Proxy