My application as well as usage from awscurl fails to properly hit the reader endpoint of my neptune cluster. I have spawned a single read replica in addition to the primary. I try to hit the status endpoint with it and it fails (whereas the primary works)
awscurl https://endpoint:8182/status --service neptune-db -v
I use the above between primary (works) reader (doesn't work). Why would this be?
Adding an answer to summarize the discussion in the comments.
As a general rule, connection failures such as this one are caused by one or more networking or security settings. Things to check include:
The calling application has the appropriate role and policies in place to allow access.
The calling application has access to the VPC Neptune is running in.
The request is correctly signed in cases where IAM authentication is enabled.
Security groups have the required ports open
The Neptune service is not blocked by Service Control Policies(SCP) if AWS Organizations is being used.
Subnets are accessible as needed.
Transit Gateways are working as expected. As noted in the comments the Route Analyzer can be used to help diagnose issues.
Related
I have a setup with a MSK Cluster and an MSK Connect Connector (which is a source connector to a postgres db, but that shouldn't matter here).
During the first step of my setup, it set it all up without authentication. And it worked fine.
Now, as my next step, I introduced TLS and IAM based authentication. But I end up with a "org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata"
I have already tried all answers to AWS MSK with IAM - Timeout Exception.
What I did specifically:
I gave my Connect-Cluster the additional AWS-managed policy "AmazonMSKFullAccess", so it can perform all actions on MSK.
And then I added Outbound- and Inbound- Rules to the security group of my MSK cluster so Port 9098 can be reached from within this same security group.
After I updated the configuration of the MSK cluster so only IAM auth is supported, I specifically did not re-create my connector (assuming this was not necessary).
Does anyone have an idea what could be missing?
Best greetings
Andy
It was my bad. The old Connector was still set to unauthenticated access - I needed to create a new one with the authentication method "IAM".
Not sure what the right terms were to start this question but basically I have a downloaded UI tool that runs on 0.0.0.0:5000 on my AWS EC2 instance and my ec2 instance has a public ip address associated with it. So right now everyone in the world can access this tool by going to {ec2_public_ip}:5000.
I want to run some kinda script or add security group inbound rules that will require authorization prior to letting someone view the page. The application running on port 5000 is a downloaded tool not my own code so it wouldnt be possible to add authentication to the tool itself (Its KafkaMagic FYI).
The one security measure I was able to do so far was only allow specific IPs TCP connection to port 5000, which is a good start but not enough as there is no guarantee someone on that IP is authorized to view the tool. Is it possible to require an IAM role to access the IP? I do have a separate api with a login endpoint that could be useful if it was possible to run a script before forwarding the request, is that a possible/viable solution? Not sure what best practice is in this case, there might be a third option I have not considered.
ADD-ON EDIT
Additionally, I am using EC2 Instance Connect and if it is possible to require an active ssh connection before accessing the ec2 instances ip that would be a good solution as well.
EDIT FOLLOWING INITIAL DISCUSSION
Another approach that would work for me is if I had a small app running on a different port that could leverage our existing UI to log a user in. If a user authenticated through this app, would it be possible to display the ui from port 5000 to them then? In this case KafkaMagic would be on a private ip and there would be a different IP that the user would go through before seeing the tool
In short, the answer is no. If you want authorization (I think, you mean, authentication) to access an application running on the server - you need tools that run on the server. If your tool offers such capability - use it. It looks like Kafka Magic has such capability: https://www.kafkamagic.com/faq/#how-to-authenticate-kafka-client-by-consumer-group-id
But you can't use external tools, like AWS, that perform such authentication. Security group is like a firewall - it either allows or blocks access to the port.
You can easily create a script that uses the aws sdk or even just executes the aws CLI to view/add/remove an ip address of a security group. How you execute that script depends on your audience and what language you use.
For a small number of trusted users you could issue them an IAM user and API key with a policy that allows them to manage a single dynamic security group. Then provide a script they can run/shortcut to click that gets the current gateway ip and adds/removes it from the security group.
If you want to allow users via website a simple script behind some existing authentication is also possible with sdk/cli approach(depending on available server side scripting).
If users have SSH access - you could authorise the ip by calling the script/cli from bashrc or some other startup script.
In any case the IAM policy that grants permissions to modify the SG should be as restrictive as possible (basically dont use any *'s in the policy). You can add additional conditions like the source IP/range (ie in your VPC) or that MFA must be active for user etc to make this more secure (can be handled in either case via script). If your running on ec2 id suggest looking at IAM Instance Roles as an easy way to give your server access to credentials for your script (but you can create a user and deploy the key/secret to the server and manage it manually if you wanted).
I would also suggest creating a dedicated security group for dynamically managed access alongside existing SGs required for internal operation for safety. It would be a good idea to implement a lambda function on a schedule to flush the dynamic SG (even if you script de-authorising an IP it might not happen so its good to clean up safely/automatically).
Let's assume, I run a Cloud Run service of Google.
Let's also assume someone wants to really harm you and finds out all API routes or is able to send a lot of post-requests by spamming the site.
There is a Email notification, which will popup on certain limits you set up before.
Is there also a way to automatically cut the Cloud Run service, or set it temporarily offline? I couldn't find any good resource or solution to this.
There are several solution to remove from traffic Cloud Run service, in addition of authentication solution proposed by Dondi
Delete the Cloud Run service. It might seem overkill, but, because the service is stateless, you will lost nothing (except the revision history)
If you have your Cloud Run service behind a Load Balancer
You can remove the serverless NEG that route the traffic to it
You can add a Cloud Armor policy that filter the originator IP to exclude it from the traffic
You can set the ingress to internal, or internal and cloud load balancing.
You can deploy a dummy revision (a hello world container for example), and route 100% of the traffic to it (traffic splitting feature)
You can't really "turn off" a Cloud Run service as it's fully managed by Google. A Cloud Run instance automatically scales down to zero if there are no requests, but it will continue on serving traffic.
To emulate what you want to do, make sure that your service requires authentication then revoke access on the offending user (or all users). As mentioned in the docs:
Cloud Run (fully managed) does not offer a direct way to make a service stop serving traffic, but you can achieve a similar result by revoking the permission to invoke the service to identities that are invoking the service. Notably, if your service is "public", remove allUsers from the Cloud Run Invoker role (roles/run.invoker).
Update: Access to a resource is managed through an IAM policy. In order to control access programmatically, you have to get the IAM policy first, then revoke the role to a user or a service account. Here's the documentation that gives an overview.
I'm trying to set up a Site to Site connection between our on-premise server and our cloud infrastructure. In our premises we have a SonicWall firewall installed and, since SonicOS 6.5.1.0 it's now easy to put an AWS access key and AWS Secret Key and let the software configure everything via SDK.
The problem is that the tutorial on how to configure the firewall (p. 8) says:
The security policy used, either for a group to which the user belongs or attached to the user directly, must
include the following permissions:
• AmazonEC2FullAccess – For AWS Objects and AWS VPN
• CloudWatchLogsFullAccess – For AWS Logs
Since it's not ideal to give anyone the full access to Amazon EC2 do you know which features SonicWall actually needs so I can disable everything else and follow the principle of least privilege?
Without looking into the code for SonicWall itself, it is not going to be easy to know exactly which API calls it's going to make to EC2. If you are prepared to at least temporarily grant full EC2 access, you could use AWS CloudTrail to monitor exactly which API calls are being made by the IAM user associated with your on-premises server, and then update your specific policy to match those calls.
Alternatively, start with the full access IAM policy template and go through and deny any calls you think are completely unrelated to SonicWall's functionality.
If you trust SonicWall then probably the easiest thing to do is to just allow the full EC2 access it claims is required (or start there and gradually remove them until something breaks!)
This is not a duplicate of the question "Getting my AWS credentials using an API call" because I am asking specifically about what Amazon means in the example that they give.
I am looking here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
I see this bit:
Warning
If you use services that use instance metadata with IAM roles, ensure that you don't expose your credentials when the services make HTTP calls on your behalf. The types of services that could expose your credentials include HTTP proxies, HTML/CSS validator services, and XML processors that support XML inclusion.
The following command retrieves the security credentials for an IAM role named s3access.
$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access
Where does this IP address come from? What is 169.254.169.254? It can't be my server, since I don't have software running on port 80, nor would I grant Amazon an alias on my server.
But I did actually run the above, and it simply timed out. So the IP address 169.254.169.254 is not a service that Amazon is actively running. So what is it?
Does anyone understand this example that Amazon offers?
169.254 is within the link-local address space: https://en.wikipedia.org/wiki/Link-local_address
It's usually used for a lot of localhost/local-subnet use cases. Amazon happens to put their metadata service at 169.254.169.254 so that it can be queried from EC2 Instances.
curl http://169.254.169.254/latest/meta-data
Should always return something, the full http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access will only return something if you had an IAM role attached to your instance named s3access.
169.254.169.254 is the address of the AWS metadata service. You can query this address from an EC2 server to obtain information about the server. The metadata that can be obtained in this manner is documented here.
Are you saying that when you run that curl command from an EC2 server it is timing out?