Is it possible to use an EFS in AWS for several instances located in different regions?
If not, is it possible to do somethink like that using AWS console?? Doesn't matter latency or throughput between EC2 instance and network volume.
EFS can be accessed through Direct Connect or VPN. Estabilsh VPN connection between regions and you can mount EFS with the IP address of the corresponding mount target.
Related
I have an EFS file system. I have created two mounts one for us-east-1a and another for us-east-1b and both are in the same VPC. Now I have a requirement to add a mount point and it's in a different VPC but in the same account. When I try to create the mount target I get the below error
aws efs create-mount-target --file-system-id fs-abcdef --subnet-id subnet-156fd195808k8l --security-groups sg-99b88u518a368dp
An error occurred (MountTargetConflict) when calling the CreateMountTarget operation: requested subnet for new mount target is not in the same VPC as existing mount targets
is there a way I can use the EFS in two different VPCs?
VPC peering OR Transit Gateway is enough in order for NFS client from different VPC to connect to EFS in separate VPC.
Only one mount target for a certain EFS is needed per AZ. The error shows that you already have mount target for the specific EFS.
To connect your NFS client you can follow the AWS provide documentation
I have a aws EC2 (EC2-A) and Amazon Managed Blockchain running in VPC (VPC-A)
This EC2-A instance has some files and certificates (required for executing transactions in the blockchain)
EC2-A has EBS storage which can be mounted on only one EC2 instance at one time.
Transactions can be only executed to the blockchain network from the EC2-A, since they're are in the same VPC-A.
I have an aws EKS (Kubernetes cluster) running in VPC-B.
How can I access the files and certificates of EC2-A from a pod in my k8s cluster. Also I have another pod which will be blockchain client executing transactions in the blockchain network, which is in VPC-A.
Both these VPC-A and VPC-B are in the same aws account.
Mount a folder/files on an EC2 instance to a pod running in EKS is not supported. For your use case, you can easily share folder/files using EFS if not S3. If you are only allow to do pod to EC2 communication, you need a way for these resources to reach each other either by public IP if not VPC peering. Then you can run sftp, scp... any kind of off the shelf file sharing software you knew best for file exchange.
You need to connect 2 VPCs with VPC Peering, then you can install NFS in your EC2, and write PV, PVC point to the NFS EC2.
Is there a way to present onprem storage to AWS EC2 instances without copying to S3 etc. We have a storage array in our DC, I want to carve up LUNs and present them to our apps running on EC2 instances in AWS, using storage gateway. Can someone suggest if this is possible at all? I don't want to pay for AWS EBS volumes, when I have plenty of storage available with me. Thanks.
You will not be able to do this with Storage Gateway - it serves as an interface between your local network and S3.
If you want to expose your local disk to EC2, you will need to run a file share (NFS or Samba), set up a VPN or Direct Connect between your data center and your AWS VPC, and then mount the exported volumes on your EC2 instance.
If you don't want to pay for any EBS volumes, you should look for instance storage based AMIs. Keep in mind that you cannot stop these instances - you can only terminate them.
According to latest AWS cloud architecture and infrastructure developments, AWS cloud resources can access on premise storages. Not only computing instances but also AWS managed SaaS can do it same. Standard protocols such as NFS, SMB and iSCSI available and you can mount AWS computing instances to remote storage through site-to-site VPN.
I have created an Amazon EFS filesystem in the Sydney Region and would like to mount it on an EC2 instance in the Singapore region.
Is this possible?
Answering as people may not look at the comment.
You can now connect to Amazon EFS file systems from EC2 instances in other AWS regions using an inter-region VPC peering connection, and from on-premises servers using an AWS VPN connection.
https://aws.amazon.com/about-aws/whats-new/2018/10/amazon-efs-now-supports-aws-vpn-and-inter-region-vpc-peering/
This answer is outdated. Please see other answer.
An Amazon EFS filesystem is only accessible in the VPC that was selected when it was created, or via Direct Connect when linked to that VPC.
Each Amazon VPC is only in one Region.
As per the Amazon EFS Limits documentation page:
An Amazon EFS file system can be mounted from on-premises datacenter servers using AWS Direct Connect. However, other VPC private connectivity mechanisms such as a VPN connection and VPC peering are not supported.
I was thinking about switching from AWS Elastic Block Storage to AWS Elastic Filesystem (mainly for the easy scalability, also shareable storage seems nice).
At the moment I have one debian EC2 instance with one EBS volume. What's the easiest way to transfer my data from EBS to EFS?
The fastest way to achieve this is mount that EFS file system to your EC2 instance with EBS and then transfer the data from your EBS to EFS.
Follow this guide for mounting the EFS to your EC2 instance. https://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html
EFS is good for sharing data between multiple EC2 instances, but you would still want to use EBS for the root drive (boot volume) of your instance.
You cannot boot from an EFS volume.
You mention that you have "one debian EC2 instance with one EBS volume". However, it is generally best to keep data separate from the boot volume (eg in a database, an S3 bucket or in EFS). This allows the instance to be recreated from an AMI in case of problems, without losing data.
If you wish to move/copy data to an EFS volume, just use normal filesystem commands (eg cp -r).
I think you can also use AWS DataSync to copy data from existing folder to EFS mounted folder.
(1) You need to set up an NFS service using the instance that you have your EBS attached to.
cf. https://linuxhint.com/install-and-configure-nfs-server-ubuntu-22-04/ for example and step-by-steps.
You can test that your NFS server works by using another instance, and mounting it there using /etc/fstab .. (I think the link above shows you how to do that).
You will need the IP address of your NFS server (for me, this is 10.0.33.5)
(2) You will need to deploy a DataSync Agent - this is a new instance. It need lots of ram (so, expensive) - eg m1.xlarge - especially if your EBS is big and has many thousands of files. look here for that https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.html#ec2-deploy-agent
You now have an DataSync Agent Instance (which should be on the same subnet and AZ as your NFS instance) showing on your EC2. You will need it's private IP number.. (for me, this is 10.0.33.111)
(3) You need to create an AWS Endpoint. (in VPC) You are going to add one with AWS Services - search for and choose 'datasync', and add that to the subnets that your NFS Server and Agent are on. Once that is created, you will need the IP address of the subnet / AZ that you are using. (For me this is 10.0.33.222)
(4) You will need to get your Agent Activation Key. ssh into an instance (like your nfs server) on the same subnet and then to get your key, using the url below with your region (mine is eu-west-1 ) and the two IP numbers you have recorded.. Do not use MY ones!!
curl "http://10.0.33.111/?gatewayType=SYNC&activationRegion=eu-west-1&privateLinkEndpoint=10.0.33.222&endpointType=PRIVATE_LINK&no_redirect"
If all is well you will get a long Activation Key string like XXXX-XXXX-XXXX-XXXX
(5) Now you need to add all this into your DataSync Agents list (it's an Amazon EC2 Hypervisor, using a VPC endpoint using "AWS PrivateLink". The endpoint should show automatically) and paste in your ID Activation Key from step 4 above. You should now see an active Agent in your Agents list (in DataSync).
(6) Now you can create a Location that uses that agent. Select NFS, and your Agent, Now put the ip address of your nfs server (from step 1, eg 10.0.33.5) and the mount path (the same as what you used in your /etc/exports file on the instance where you have attached your EBS eg /mnt/mydrive
(7) NOW you can create a DataSync task from your NFS to your EFS..