I was thinking about switching from AWS Elastic Block Storage to AWS Elastic Filesystem (mainly for the easy scalability, also shareable storage seems nice).
At the moment I have one debian EC2 instance with one EBS volume. What's the easiest way to transfer my data from EBS to EFS?
The fastest way to achieve this is mount that EFS file system to your EC2 instance with EBS and then transfer the data from your EBS to EFS.
Follow this guide for mounting the EFS to your EC2 instance. https://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html
EFS is good for sharing data between multiple EC2 instances, but you would still want to use EBS for the root drive (boot volume) of your instance.
You cannot boot from an EFS volume.
You mention that you have "one debian EC2 instance with one EBS volume". However, it is generally best to keep data separate from the boot volume (eg in a database, an S3 bucket or in EFS). This allows the instance to be recreated from an AMI in case of problems, without losing data.
If you wish to move/copy data to an EFS volume, just use normal filesystem commands (eg cp -r).
I think you can also use AWS DataSync to copy data from existing folder to EFS mounted folder.
(1) You need to set up an NFS service using the instance that you have your EBS attached to.
cf. https://linuxhint.com/install-and-configure-nfs-server-ubuntu-22-04/ for example and step-by-steps.
You can test that your NFS server works by using another instance, and mounting it there using /etc/fstab .. (I think the link above shows you how to do that).
You will need the IP address of your NFS server (for me, this is 10.0.33.5)
(2) You will need to deploy a DataSync Agent - this is a new instance. It need lots of ram (so, expensive) - eg m1.xlarge - especially if your EBS is big and has many thousands of files. look here for that https://docs.aws.amazon.com/datasync/latest/userguide/deploy-agents.html#ec2-deploy-agent
You now have an DataSync Agent Instance (which should be on the same subnet and AZ as your NFS instance) showing on your EC2. You will need it's private IP number.. (for me, this is 10.0.33.111)
(3) You need to create an AWS Endpoint. (in VPC) You are going to add one with AWS Services - search for and choose 'datasync', and add that to the subnets that your NFS Server and Agent are on. Once that is created, you will need the IP address of the subnet / AZ that you are using. (For me this is 10.0.33.222)
(4) You will need to get your Agent Activation Key. ssh into an instance (like your nfs server) on the same subnet and then to get your key, using the url below with your region (mine is eu-west-1 ) and the two IP numbers you have recorded.. Do not use MY ones!!
curl "http://10.0.33.111/?gatewayType=SYNC&activationRegion=eu-west-1&privateLinkEndpoint=10.0.33.222&endpointType=PRIVATE_LINK&no_redirect"
If all is well you will get a long Activation Key string like XXXX-XXXX-XXXX-XXXX
(5) Now you need to add all this into your DataSync Agents list (it's an Amazon EC2 Hypervisor, using a VPC endpoint using "AWS PrivateLink". The endpoint should show automatically) and paste in your ID Activation Key from step 4 above. You should now see an active Agent in your Agents list (in DataSync).
(6) Now you can create a Location that uses that agent. Select NFS, and your Agent, Now put the ip address of your nfs server (from step 1, eg 10.0.33.5) and the mount path (the same as what you used in your /etc/exports file on the instance where you have attached your EBS eg /mnt/mydrive
(7) NOW you can create a DataSync task from your NFS to your EFS..
Related
I am using a AWS console which show a running instance but when I check the storage in S3 there are no files in it. I have checked all the other storage options but unable to find any files. I need to check that which website is hosted on this instance. What can i do?
Moreover, I do not have .pem of the instance
I am working on a client website and we have a doubt that one of our website is hosted on AWS. We want to migrate that website from AWS to another host.
Please guide what is the best way to do this?
Thanks
Amazon EC2 uses Elastic Block Store (EBS) volumes for its file system, not S3. So if the web site is hosted by an EC2 instance, the files will most likely be stored on an EBS volume. To get access, you will probably need to log in to the instance using SSH or the AWS Session Manager. Alternatively, you can try and attach the volume to a different EC2 instance that you have access to.
Firstly S3 is for object storage, the instance runs off block storage using EBS and is where the file structure is located. Unless a job exists to manually copy content to S3 you will not find it there.
There are a few options you can perform to gain access:
You can try connecting using Sessions Manager assuming that the SSM Agent is running on the host.
You can try using instance connect if it is setup, this will allow you to use an SSH terminal by specifying a temporary pem.
You can create an AMI of the server and then launch a new instance from this AMI (the server will be running as if it is the server that you took the AMI from so be aware services will be running).
You can take a EBS snapshot of the server and attempt to launch a new volume from this. You would then mount it to a host.
Be aware that if this is a Windows host, connecting via regular RDP will require the Windows password of the host. If this is the case you will need to follow these instructions.
I am deploying Wordpress site using LAMP AMI on EBS-backed EC2. Wordpress database will be stored in EBS.
Amazon keeps on updating base configuration of EC2 for better performance & security.
If I have to upgrade my EC2 then my EBS will get deleted.
How can I ensure that my EC2 gets updated without affecting my EBS?
EBS Snapshot is not of any help as new EC2 already has a EBS attached.
Do I have to always migrate my Wordpress site using backup & do this upgradtion?
Amazon EC2 instances use EBS volumes as their disk volumes. They are populated from an Amazon Machine Image (AMI) when the instance is launched.
When AWS releases updated AMIs, any new instance launched from the AMI will contain the updated disk content. However, existing EC2 instances will not be changed (since they have a copy of the AMI at the time that the instance was launched).
You can also change the Instance Type of an EC2 instance, which gives it different hardware (CPU, Memory). This can be done by stopping the instance, changing the type and starting it again. This will not affect the contents of EBS volumes.
If you wish to keep your instance "current", simply run sudo yum update (Linux) or run the Windows Update utility to update your operating system and associated utilities rather than launching a whole new instance.
Is there a way to present onprem storage to AWS EC2 instances without copying to S3 etc. We have a storage array in our DC, I want to carve up LUNs and present them to our apps running on EC2 instances in AWS, using storage gateway. Can someone suggest if this is possible at all? I don't want to pay for AWS EBS volumes, when I have plenty of storage available with me. Thanks.
You will not be able to do this with Storage Gateway - it serves as an interface between your local network and S3.
If you want to expose your local disk to EC2, you will need to run a file share (NFS or Samba), set up a VPN or Direct Connect between your data center and your AWS VPC, and then mount the exported volumes on your EC2 instance.
If you don't want to pay for any EBS volumes, you should look for instance storage based AMIs. Keep in mind that you cannot stop these instances - you can only terminate them.
According to latest AWS cloud architecture and infrastructure developments, AWS cloud resources can access on premise storages. Not only computing instances but also AWS managed SaaS can do it same. Standard protocols such as NFS, SMB and iSCSI available and you can mount AWS computing instances to remote storage through site-to-site VPN.
I am looking to migrate an application from one vpc to another. Currently, the application's ec2 instances utilize a nfs which I created from another ec2 instance and ebs volumes. These application related instances (web and app) as well as the nfs instance reside in one VPC (Let's call it VPC "A").
My question is how would I go about transferring the ebs data from the existing nfs in VPC "A" into an efs in VPC "B"? Additionally, the rate of data transfer is not relevant, so the file system's performance mode is just defined as general purpose. Thanks!
Not sure if this answer was technically possible at the time the above answer was given, but a much easier way to achieve a solution providing that the asker did not to run both applications concurrently (ie. duplicate the data instead of just switch the VPC it was available from).
My company was switching to a new VPC, so I merely wanted to make the data available in the new VPC. Rather than copy it to EBS, unmount, remount, copy to EFS, I switched the mount points as described here:
http://docs.aws.amazon.com/efs/latest/ug/manage-fs-access-change-vpc.html
IMO this is much simpler if you don't need to actually duplicate the data.
For backing up the data, AWS has a workflow outlined:
http://docs.aws.amazon.com/efs/latest/ug/efs-backup.html
It sounds like your requirement is to transfer the contents of an Amazon EBS volume in one VPC to a different VPC. It is unclear whether your NFS server is hosted from EC2+EBS, or whether it is running from Amazon EFS. I'll assume it is EC2+EBS.
A few different options for you (pick one!):
Create an AMI: Create an AMI of the Amazon EC2 instance running the server. Then launch a new instance in VPC B using that AMI.
Move the disk: Turn off the EC2 instance. Detach the EBS volume, the attach it to a different instance running in VPC B. Configure as necessary.
Use an EBS Snapshot: Snapshot the EBS volume that holds the data, create a new EBS volume from the snapshot (in the desired AZ), attach the EBS volume to the EC2 instance in VPC B.
However, if your server is running on Amazon Elastic File System (EFS), then you would need to copy the data to an EBS volume, detach the volume and attach it to an instance in the VPC B, then copy the data from the EBS volume to a new EFS share in VPC B.
I have a requirement of setting up Apache Tomcat on Amazon EC2 instance.
I have heard that EC2 is ephemeral and anything we put in may not survive restarts. So if I add a Tomcat in EC2 and restart the instance will it be deleted or removed.
What are the ways to overcome this issue ?
Sorry I am a newbie in AWS.
If what you need is just Tomcat then you can use pre-configured Amazon Beanstalk Tomcat container which does exactly that.
However if you need to build custom EC2 you can install tomcat on EBS linked to EC2, or even better use Amazon EFS to share between multiple EC2.
Ephemeral disk is temporary storage that it is added to your instance and sized according to instance type. The larger the instance, the more temporary storage.
For some instances like c1.medium and m1.small they use instance storage automatically as SWAP as they have a limited amount of memory, while many others are automatically formatted and mounted at /mnt.
You can take snapshots of your EC2 instances while they are running. Snapshots allow you to create AMI's from your current machine state, which will contain everything in your ephemeral storage. When you launch a new instance based on that AMI, it will contain everything as it was in the snapshot.
An Amazon Machine Image (AMI) provides the information required to launch an instance.
Take note that there is a big difference between stop and terminate. If you stop an instance that is backed by EBS, the information on the root volume will still be in the same state when you start the machine again. If you terminate the machine without taking a snapshot, even if it is backed by EBS, the storage inside the ephemeral disk will be lost forever.
All AMIs are categorized as either backed by Amazon EBS or backed by instance store. The former means that the root device for an instance launched from the AMI is an Amazon EBS volume created from an Amazon EBS snapshot. The latter means that the root device for an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3. For more information, see Amazon EC2 Root Device Volume.
The above answers should provide a good idea about what you can and cannot do with ephemeral disks, but I advise all (myself included) to learn more about ephemeral disks and their primary use cases.
Here are some good use cases for ephemeral storage that I know of:
Temporary backup
Re-format the original instance storage and use part of it for SWAP
RAID 10 with 6 disks (4 EBS and 2 Ephemeral disks) to improve overall performance and provide HA
Application cache, logs, any other random data
You are save as long as you not store it on ephemeral partition which is a part of your instance check this to have more info
http://www.heitorlessa.com/working-with-amazon-aws-ec2-ephemeral-disks/
basically you need to mount elastic drive to your instance and install Tomcat and all your software there, ephemeral storage is for swap or caching
You only need storage to keep your information, EBS or S3. EC2 instances are virtual machines and will not lose your information if restarts with storage.
Get all information you need at https://aws.amazon.com/es/ec2/