I'm getting the following error when trying to test a target connection to a Redshift db using AWS DMS tool:
Test Endpoint failed: Application-Status: 1020912, Application-Message: Query setup error
The redshift cluster is in the same VPC, subnet group & security group as the replication instance. The security group allows all traffic inbound from itself. Is there a location I can get full logs for the endpoint test, or more details about this error?
Query setup error is not a network error - but rather relates to errors coming from SQL engine itself. In my case AWS support was able to look up the logs. In there was an error line about the database name being wrong.
Related
I am trying to configure MSK connect in AWS and the below is the configuration.
INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
[Worker-02003b81ffe0ee9c3] [2022-06-02 14:26:40,955] INFO [AdminClient clientId=adminclient-1] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
[Worker-02003b81ffe0ee9c3] org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1654180000954, tries=1, nextAllowedTryMs=1654180001055) timed out at 1654180000955 after 1 attempt(s)
As per https://aws.amazon.com/premiumsupport/knowledge-center/msk-connector-connect-errors/ I have opened all traffic for the MSK connector to be able to reach the msk cluster, yet I notice timeout errors.
The connector and the cluster are both in same subnets and uses same security group ID. I ma able to telnet to the broker from a VM in the same subnet.
Note: I have plaintext enabled and no authentication. I have also given proper IAM permission and role attached. This is verified.
Adding the solution in case if it helps someone.
https://docs.aws.amazon.com/msk/latest/developerguide/mkc-tutorial-setup.html
I had to create a vpc endpoint as mentioned in the above doc and also associate the subnet route tables that my kafka uses.
Additionally also make sure your SG's have correct inbound and outbound rules
I'm looking for whitelisting Ip addresses to secure an internet facing transfer server via terraform but unfortunately terraform AWS provider still doesn't support adding new security group to vpc endpoint via terraform aws transfer resource.
I tried to update server using aws cli command but getting an error "An error occurred (InvalidRequestException) when calling the UpdateServer operation: Changing Security Group is not supported"
Any suggestion?
Assuming they are using EndpointType=VPC, the UpdateServer command does not support updating SecurityGroups.
Attaching a Security Group can either be done at server creation time using CreateServer or use EC2's ModifyVPCEndpoint API to update Security Group once the server has been created.
Refer to the documentation here: EndpointDetails - AWS Transfer Family (Under SecurityGroupIds)
(Console) In order to modify the Security Group of an AWS Transfer server once created do the following:
Go to the VPC service
Go to "Endpoints"
Click on the Endpoint that has "transfer" on the Server Name field
Click on the "Security Groups" Tab
Click on "Edit Security Groups"
I'm getting an error in configuring a database connection in a Google Cloud Data Fusion Pipeline.
"Encountered SQL error while getting query schema: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server."
We can't connect outside of the company building as the company IP's are whitelisted in AWS security settings. I can query easily using mysql workbench inside the company so, I'm guessing I need to add some IPs to our AWS security groups to provide Data Fusion permissions? I can't find a guideline on this. Where can I find the ip's required to provide in AWS? (Assuming that might fix it)
I've added a mysql plugin artefact using 'mysql-connector-java-8.0.17.jar', which is referred to by plugin name 'mysql-connector-java'.
Do VPN between your GCP VPC and your AWS VPC where your RDS is residing
https://cloud.google.com/solutions/using-gcp-apis-from-an-external-network
https://cloud.google.com/solutions/automated-network-deployment-multicloud
Simple way
Create Haproxy with public IP
Data Fusion --> VM Haproxy Public IP --> AWS RDS Private IP
I have an AWS Lambda function for an API. I have a test function inside of the API that returns a "Success" string so I know that I can hit the API from my machine (Postman).
The Lambda API is inside of my VPC. I have public and private subnets. The API is living in the public subnets. I also have a RDS database that's living in the private subnets. I can connect to the database through a Bastion host so I know the database is connected and working.
When I try an API call that connects to the database, I can see an error in CloudWatch that says:
[Error] Amazon.Lambda.AspNetCoreServer.APIGatewayProxyFunction: Unknown error responding to request: MySqlException:
MySql.Data.MySqlClient.MySqlException (0x80004005): Unable to connect to any of the specified MySQL hosts. ---> System.InvalidOperationException: Sequence contains more than one matching element
There's a lot of references to that error message that talks about the Linq SingleOrDefault() method, but I don't have any references to that method.
I think that error has something to do with the security groups. I have an RDS security group that has an inbound rule on port 3306 from my internal VPC ip addresses.
The security group for my Lambda function should be ok since I can successfully hit the test route.
Finally, I tried connecting to the database through the API in debug on my machine and I get an error "The requested name is valid, but no data of the requested type was found"
Does anyone have any other ideas or places to look?
Can you check your this setting is enable or not for RDS Security group ?
Disclaimer : Allowing inbound traffic on DB from internet is not recommended and should always be avoided
My RDS database (that I was trying to connect to and read from using AWS Lambda) was also a MySQL database.
When I tested the Lambda function I was getting 'Unable to connect to any of the specified MySQL hosts' (after waiting -- like it was timing-out).
I reviewed the answer from #aviboy2006, which helped me. I tried all the Security Group rules shown in that answer. I found I was able to successfully connect-to and read from the RDS MySQL database when I defined both an Inbound and Outbound rule for the associated VPC Security Group as
Type: MYSQL/Aurora
Protocol: TCP
Port Range: 3306
Source: Custom: 0.0.0.0/0
Trying to Migrate RDS mysql to Redshift, When connecting the AWS RedShift Database in Target Connection it Throws the Error:
Test Endpoint failed: Application-Status: 1020912, Application-Message: IN/A, Application-Detailed-Message: N/A
Please help to resolve...
I have resolved my issue by adding few ingress/egress rules weren't defined on the security groups which are attached to DMS and Redshift.
Check this link for further information: AWS DMS endpoint connection to Redshift not working
For me, the private IP of redshift in the server name of the endpoint worked. I don't know why but it's working now.
AWS - Data Migration Service Endpoint Redshift error (Test failed)
The Leader private IP of redshift in the server name of the endpoint will work.
In your DMS (Data Migration Service) configuration where you enter the Server Name value, avoid the Redshift cluster DNS Name or the Public IP address. Instead, try using the private IP of the Redshift Leader node. It worked for me after I changed to the private IP of leader node.