EC2 instances connecting to lambda result in ConnectFailure - amazon-web-services

I'm trying to access lambda functions from a Windows VM I have created in EC2 for dev purposes but even a simple 'list functions' command fails to connect
I have tried using the AWS CLI through PowerShell, the dotnet sdk and the VS AWS Toolkit but each of these times out after a long waiting period. I can, however, list other services such as my databases and S3 buckets.
aws cli failure message
VS toolkit failure message
I have tried creating a new VM with the same results. I've disabled windows firewall altogether, allowed all traffic through the security group and have VPC endpoints for my subnet (ssm, ec2messages, lambda, ec2).
I have no trouble connecting to the lambda service through my own computer. On the VM, I have modified the .aws/credentials file to match the one on my computer for both the admin and current user but I still can't connect. This tells me that the problem isn't related to my access key credentials.
I'm reaching the end of the troubleshooting options I can think of so any help would be very much appreciated!
Update: using telnet, I cannot connect to lambda.ap-southeast-2 but I can connect to s3.ap-southeast-2 and lambda.ap-southeast-1. It seems lambda.ap-southeast-2 is being blocked somewhere but it isn't windows firewall because it's off and the same problem happens on Ubuntu VMs.
In the VPC Management Console, I haven't set up any firewalls under network or dns filewalls and my network ACL allows all traffic.

Related

Connecting to DynamoDB from an EC2 instance running in an ECS cluster with VPC

I have an EC2 instance running inside an ECS cluster with VPC.
On the instance, I need to run a ECS task that needs access to DynamoDB.
When I try running the same task using Fargate, I can use the assignPublicIp = 'ENABLED' option to allow my task to have access to other AWS services, and everything works fine.
However, the assignPublicIp option is not available for the EC2 launch type, and I cannot figure out how to allow my EC2 instance have access to other AWS services.
I read the AWS docs and followed guides like this one to setup VPC endpoint for DynamoDB.
I also made sure that there aren't any network access restrictions by making sure that inbound/outbound rules for my NACL and security group for the VPC are wide open (at least for the sake of testing).
Here is how the rules look like, for both NACL and my security group:
Finally, I used the VPC > Reachability Analyzer to check if AWS can detect any problems regarding the connection path between my EC2 instance and DynamoDB, but the analysis reported a Reachable status.
It basically told me that there was no issues regarding establishing a connection along the following path:
Network interface for my EC2 instance (source)
Security group for the VPC
NACL for the VPC
Route table for the VPC
which includes the following route added by the VPC endpoint for DynamoDB
Destination: pl-02cd2c6b (com.amazonaws.us-east-1.dynamodb, 3.218.182.0/24, 3.218.180.0/23, 52.94.0.0/22, 52.119.224.0/20)
Target: the endpoint ID (e.g., vpce-foobar)
VPC endpoint for DynamoDB (destination)
Despite AWS telling me that I have a "Reachable" status, I still think it might be a network reachability problem, because when I run the task, the script I am running gets stuck right after it makes a GetItem call to DynamoDB.
If it was a permission error or an invalid parameter issue, I would get an error immediately, but everything just "hangs" there, until the task eventually times out.
Any pointers on what I might be missing here, or other workarounds would be very appreciated.
Thank you.
EDIT 1 (2021/02/13):
So I went back to the AWS docs to see if I had missed anything in setting up the VPC endpoints. I originally had one setup for DynamoDB, but since I also need to use S3 in my service, I went ahead and setup a Gateway VPC Endpoint for S3 too (I also wanted to see if the issue I am having is a generic network problem, or specific to DynamoDB).
Then, I made some changes to my script to try to make a call to S3 (to get the bucket's location, for simplicity) as the very first thing to do. I knew that the call would end up timing out, so I wanted to trigger the error immediately upon starting my script execution.
I waited until my task would eventually fail because of the timeout, and this time I noticed something interesting.
Here is the error logs I got when the task failed:
The IP address that my task was trying to reach was 52.85.146.194:443.
And here are the IP addresses that I found in the managed prefix list for S3, which I found in the VPC console:
The IP address I got the timeout error from is not on the list. Could this be a hint to the cause of the issue? Or am I missing something and there is actually nothing wrong with that?

Cannot connect to AWS RDS

I am trying to create a AWS RDS Sql Server database and connect to it from a local machine using SSMS. Later I'll be connecting from a web application (locally, then hosted somewhere eventually.) I am currently failing to connect to my instance (the instance is configured and running.) The error I'm getting is the network/instance related (not login.) Tried telnet and I can't even hit it that way.
Looking on the web, there seems to be a setup for network connections but it talks about EC2, VPC and things I don't think I need (or do I?)
Tried (nothing worked so far): Using the IP instead of hostname, explicitly specifying the port (1433), changing user/password, crying.
Speaking of things I hope I don't need to configure, there's also IAM authentication - didn't touch that yet.
Any input is appreciated before I open a ticket with Amazon.
UPDATE:
My scenario: Scenario
Solution - add the Inbound Rule to default Security Group: Security Groups
When you work with RDS, you need to set inbound rules; otherwise, you are unable to connect to the database. This concept is covered in this AWS tutorial. In this AWS tutorial, the database is MySQL and the app is a Java web app. However, the same concepts apply with respect to inbound rules:
Creating the Amazon Relational Database Service item tracker
One tip -- when you set an inbound rule to let your development machine connect, you can select MyIP...
Also - when you host your app (for example Elastic Beanstalk), you need to set an inbound rule for that as well (as discussed in that tutorial)

Why are outbound SSH connections from Google CloudRun to EC2 instances unspeakably slow?

I have a Node API deployed to Google CloudRun and it is responsible for managing external servers (clean, new Amazon EC2 Linux VM's), including through SSH and SFTP. SSH and SFTP actually work eventually but the connections take 2-5 MINUTES to initiate. Sometimes they timeout with handshake timeout errors.
The same service running on my laptop, connecting to the same external servers, has no issues and the connections are as fast as any normal SSH connection.
The deployment on CloudRun is pretty standard. I'm running it with a service account that permits access to secrets, etc. Plenty of memory allocated.
I have a VPC Connector set up, and have routed all traffic through the VPC connector, as per the instructions here: https://cloud.google.com/run/docs/configuring/static-outbound-ip
I also tried setting UseDNS no in the /etc/ssh/sshd_config file on the EC2 as per some suggestions online re: slow SSH logins, but that has not make a difference.
I have rebuilt and redeployed the project a few dozen times and all tests are on brand new EC2 instances.
I am attempting these connections using open source wrappers on the Node ssh2 library, node-ssh and ssh2-sftp-client.
Ideas?
Cloud Run works only until you have a HTTP request active.
You proably don't have an active request during this on Cloud Run, as outside of the active request the CPU is throttled.
Best for this pipeline is Cloud Workflows and regular Compute Engine instances.
You can setup a Workflow to start a Compute Engine for this task, and stop once it finished doing the steps.
I am the author of article: Run shell commands and orchestrate Compute Engine VMs with Cloud Workflows it will guide you how to setup.
Executing the Workflow can be triggered by Cloud Scheduler or by HTTP ping.

AWS Systems Manager Session Manager Port Forwarding not connecting

I have an EC2 Windows 2019 Server instance in a VPC in the private subnet. I have a bastion instance in the public subnet, and I know the bastion instance works, because it is being used for internet access for my Lambda API. I have VPC endpoints to:
com.amazonaws.eu-west-2.ssm
com.amazonaws.eu-west-2.ec2messages
com.amazonaws.eu-west-2.ec2
I would like to connect with RDP to the EC2 instance to manage my MySQL RDS instance. I have been trying to get Systems Manager Session Manager Port Forwarding to work, following several guides.
I have tried every guide I could find, and everything seem correctly setup.
Is it required to have specific ports setup inbound / outbound on the bastion instance SG, or on the SG the EC2 I want to RDP to is in? I have not been able to see that anywhere.
I have run AWSSupport-TroubleshootRDP and everything pass with "Success" and from the output everything looks good.
When I run the port forwarding command on my machine it starts the session, but I never get "Connection accepted" and the RDP connection fails when I try that.
Can anyone point me to what else I can do to verify that I have the correct network configurations, and to test why I am not getting "Connection accepted", as I have seen I should be getting from the guides.
Thanks in advance.
To test the ability to run RDP via Session Manager Port Forwarding, I did the following:
Launched an Amazon EC2 instance running Microsoft Windows Server 2019 Base
Associated an IAM Role with AmazonSSMManagedInstanceCore permissions
Set the Security Group to no inbound connections (to confirm that connections were being made via Session Manager)
Confirmed that I could use Session Manager to connect to PowerShell on the instance (using the Session Manager console)
Ran the following command on my Mac:
aws ssm start-session --target i-xxx --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["3389"],"localPortNumber":["3389"]}'
Used Microsoft Remote Desktop (which connects via RDP) to connect to localhost
It prompted me for a password. I used Get Windows Password to decode and obtain the password.
It connected successfully
So, yes, you can use Session Manager Port Forwarding to establish an RDP connection with a Windows instance even if it is in a private subnet (which I simulated by removing all inbound rules on the security group).
If you are having further problems, it might be due to your VPC Endpoint configurations.

AWS unable to connect to Java springboot API endpoints

I am trying to run my springboot API on AWS however when i try to connect to the endpoint the error Site cannot be reached IP refused to connect. This my first time working with AWS.
I created a linux instance and connected to it using filezilla. Afterwards i added my jar to a folder which i created on the linux instance using filezilla. I started the springboot project and its running but the problem is that i cannt seem to connect to the endpoints. Am i missing something, how do i connect to my endpoints.
The other thing to note is that i enabled https on my API and added swagger also.
You need to enable relevant ports in the instances' Security Group.
Look at this to create a new Inbound rule for the specific port.
You can go to the aws console, (here I am assuming you have deployed to us-east-1 if its something else, go to the relevant region.
Open up the relevant security group, and then click edit Inbound roles.