We have an on-premise DataSync agent (VM image) running, and an EFS with mount target.
We want to grant the agent access to the mount target in order to run sync tasks. However, there does not seem to be any security group assignable to the agent that we could grant egress access to the mount target.
So, currently, we grant public egress access to the mount target. Is there any way to nail this down to the agent?
If the agent was running on an EC2 instance, the instance itself could have a security group assigned, but there does not appear to be any alternative when the agent is running on-premise.
Turns out, I had a misconception.
DataSync Locations have a security group assigned, which is used when running datasync tasks using that location. And that security group needs egress access in the EFS mount target's security group.
Related
My issue is basically what is said in this question, except it's about EFS, rather than EC2, and I can't solve my problem with Route 53, as it's suggested.
I have an EFS instance and I try to mount it locally on my Windows machine (over WSL running Ubuntu 22.04.1 LTS) like so:
sudo mount -t efs -o tls,accesspoint=fsap-08fa969084c23b344 fs-003f3467bf1e15b13:/ efs
This results in the following:
Failed to resolve "fs-003f3467bf1e15b13.efs.us-east-1.amazonaws.com" - check that your file system ID is correct, and ensure that the VPC has an EFS mount target for this file system ID.
See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail.
Attempting to lookup mount target ip address using botocore. Failed to import necessary dependency botocore, please install botocore first.
It seems that the issue arises from the fact that I'm not trying to access EFS from an EC2 instance in my VPC, but from the public internet, where the DNS fs-003f3467bf1e15b13.efs.us-east-1.amazonaws.com and the private IP 172.31.43.109 obviously can't be resolved.
Therefore, I want to assign a static Elastic IP to the network interface of EFS, so I can access it publicly, but I get the following error:
Failed to associate address with eni-0fa8cf69d68b7bb01: You do not have permission to access the specified resource.
I don't think that I "do not have permission" because I'm the owner of the account and I have the AdministratorAccess IAM policy.
Is there a way to make EFS publicly accessible or mount it in any other way on my own machine?
Therefore, I want to assign a static Elastic IP to the network interface of EFS, so I can access it publicly, but I get the following error:
That's not a supported configuration on AWS. You can't assign a public IP to EFS. You need to look into SSH tunneling, or a VPN connection into the VPC, in order to mount an EFS volume from outside the VPC.
My guess is that AWS doesn't allow me to make EFS publicly available because that might make their AWS Transfer Family product obsolete, since it seems to solve the same problem - using EFS outside the cloud.
That's a very cynical take on things. In actuality Amazon simply designed EFS to be a service that complemented their compute services (EC2, ECS, EKS, Fargate, and Lambda). They did not design it to be a global, public NFS mountable file system.
I am trying to launch fargate task from EFS . My EFS filesystem is mounted on Same subnet, VPC and security group where my ECS is mounted.
still I am facing the same issue "failed to invoke EFS utils commands to set up EFS volumes: stderr: Failed to resolve "fs-0b5a160420b31f547.efs.us-east-1.amazonaws.com" - check that your file system ID is correct, and ensure that the VPC has an EFS mount target for this file system ID"
In my security group of ECS and EFS 2049 is enabled on inbound and Outbound network.
I find the solution by my own I haven't done VPC DNS name enabled as it was set as disable i changed it to enabled and then ITWoRKS.
I am trying to create a Fargate container with a mounted EFS volume via access point, all being created through cloudformation. I see the EFS created in the portal however the ECS task is failing with:
Failed to resolve "fs-XXX.efs.eu-west-2.amazonaws.com" - check that your file system ID is correct
Before adding the accesspoint the mounting worked. I need the accesspoint since the container is using non-root user.
The VPC has DNS and hostname lookup enabled.
Here is the cloudformation template:
https://pastebin.com/CgtvV17B
the problem was missing EFS Mount Target https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-efs-mounttarget.html
I think the fargate tasks can't reach the EFS system, check that the EFS subnets are reachable from the Fargate ( deployed in the same subnets at least), and the route tables are well configured. The security group of the ECS and EFS are well configured ( check if your EFS authorize the TCP 2049).
Also check the version of the Fargate, I think its working with EFS just for the version > 1.4
Try to deploy an instance EC2 with the same configuration ( same VPC and subnet properties ) and check if it can reach the EFS.
I have an Amazon Workspace running inside the Private Subnet. In the same subnets i have launched my RDS. I have give access to the Workspace security group to access the RDS inside RDS security group.
Even after all this configuration, i'm getting timedout error whenver i'm trying to connect to RDS from the workspace.
You must allow outbound access for the specific port on the security group attached to the WorkSpace and inbound access from the WorkSpace on the RDS security group (you can reference the security group IDs instead of ip-addresses).
Take also a look on the AWS Reachability Analyzer to check if everything should work. (use the network interface of your WorkSpace)
https://docs.aws.amazon.com/vpc/latest/reachability/getting-started.html
I would like to create EFS in AWS and it is said in documentation, that I can attach it only to instances, which have the same security group as my VPC.
How to know security group of my VPC?
Suppose it is default and my instances have different security groups, created at different times by different wizards. How can it be, that instance is belong to VPC but has different security group, than that VPC?
Amazon Elastic File System(EFS) is a regional service. If you create an EFS in a particular region (eg: us-east-1) then you can create multiple EC2 instances in different availability zones in the same us-east-1 region to access the EFS to read and write data.
All the EC2 instances in a particular region (eg: us-east-1) must belong to a VPC and a subnet.(Unless you use EC2-Classic). A VPC maps to a region and A subnet maps to an availability zone. You can setup mount targets in the availability zones of your VPC, So that EC2 instances can connect to EFS via a mount target and share the same file system.
Have a look at the following image from AWS Documentation.
Now, how can we make sure that our EFS can only be accessed by certain set of EC2 instances and not all the instances from all the subnets?
This is where the security groups come in handy. We can assign security groups to the EFS mount points such that only EC2s that the given security group is attached can access EFS via the mount target. Any other EC2 instances that are in a different security group cannot access the EFS. This is the way we restrict access to EFS.
So, when you are mounting the EFS to an EC2 instance, we have to add the same security group of the EFS to the EC2 instance.
Both an Amazon EC2 instance and a mount target have associated security groups. These security groups act as a virtual firewall that controls the traffic between them. If you don't provide a security group when creating a mount target, Amazon EFS associates the default security group of the VPC with it.
Regardless, to enable traffic between an EC2 instance and a mount target (and thus the file system), you must configure the following rules in these security groups:
The security groups you associate with a mount target must allow inbound access for the TCP protocol on the NFS port from all EC2 instances on which you want to mount the file system.
Each EC2 instance that mounts the file system must have a security group that allows outbound access to the mount target on the NFS port.
Read more about EFS security groups here.
Hope this helps.