I would like to create a mirror image of my existing production environment in another AWS region for disaster recovery.
I know I don't need to recreate resouces such as IAM roles as its a "Global" service. (correct me if I am wrong)
Do I need to recreate key pairs in another region?
How about Launch configurations and Route 53 Records sets?
Launch configurations you will have to replicate into the another region as the AMIs, Security Groups, subnets, etc will all be different. Some instance types are not available in all regions so you will have to check that as well.
Route53 is another global thing but you will probably have to fiddle with your records to take advantage of multi-region architecture. If you have the same setup in two different regions you will probably want to implement latency based or geo routing to send traffic to the closest region. Heres some info on that
As for keys they are per region. But I read somewhere that you could create an AMI from your instance, move that to a new region, and fire an instance off that and as long as you use the same key name your existing key will work but take that with a grain of salt as I haven't tried it nor seen it documented anywhere.
Heres the official AWS info for migrating
Related
I am in process of coming up with a multi-region high availability (active-active) architecture for my product. A simplified version of our stack is that we use Lambda to implement our micro services, which are exposed as APIs using API Gateway. These micro services integrate with downstream services or databases like DynamoDB, Aurora RDS. So, '
Route 53 >> Api gateway >> Lambda >> Downstream service/Database
'
I am trying to figure out what is the best mechanism to configure Route 53 such that it understands any of the services in the stack fails so that it routes the incoming requests to another region. Eg if Lambda service in region-1 fails, then it is easy because I would create Health Check records pointing to these Lambdas and once they are not reachable Route 53 will itself route to next requests to region-2.
However, if the downstream resource eg RDS that Lambda is dependent on fails, how will Route 53 know this so as to route the next requests to region-2?
Appreciate any pointers on this.
It depends a bit on your envision failover setup.
Let us assume you have two regions: region1,region2
Now you could have two failure scenarios:
Lambda fails in region1 => you failover to Lambda in region2
RDS fails in region1 => you failover to RDS in region2
In both cases you need to ask yourself: What I want to do. If for example, in case 1 you connect from Lambda in region2 to RDS in region1 then high region transfer costs may occur, so you may want to trigger in any case a fail-over of RDS to region2.
Note: Generally it is very advisable to not connect directly with Lambda to RDS, but use instead RDS proxy (to avoid hammering the database with requests, slowing it down etc.): https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html
Generally, with RDS these region failovers are much more complicated (can answer on that bit if needed). It is also not simply changing IP to another region, because usually you need to promote in the other region the database (cluster) as a designated node to allow write operations.
For the databases (DynamoDB, Aurora) you mentioned there is though a solution: Use Global Tables.
A simpler solution could be - depending on your application - to use DynamoDB Global Tables (see https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GlobalTables.html). However, clearly DynamoDB is not a relational database so it may not fit all cases. Nevertheless, DynamoDB works generally very good with Lambda and is also easier for cross-region replication. Note: if you encrypt your data using AWS KMS CMK (recommended) then you need to have this key also available in all regions where you plan to use Global tables (see https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html).
Another solution could be AWS RDS Aurora Global Tables (https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) - those are available in multiple region and failover is thus easier (cf. https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-disaster-recovery.html).
In the Aurora case you have to detect a region failure yourself (e.g. you could have a lambda in both regions that regularly tries to connect to the current active cluster for writing) and automatically promote the cluster in the new region as primary if it is not available in the original region.
Do not forget: You need to regularly test the failover otherwise it is almost ensured that it will not work when you need it.
Generally having databases cross-regions implies transfer costs and additional resource costs compared to a single region - not only during failover, but all the time data is written.
With this configuration, I recommend failing over the entire stack (to another Region), rather than failing over individual tiers (components) of the architecture. (This is what you seem to be saying in your question, but just making sure we are on the same page).
Your question comes down to how to configure the health check, and specifically how to implement shallow versus deep (checking dependencies like RDS) health checks.
There is an AWS Well-Architected lab that covers these concepts Implementing Health Checks and Managing Dependencies to improve Reliability.
I would like to know if I can change the region of my EC2 instance?
The region I use is Ohio, the reason I decided to change is that I was studying AWS and I learned about latency. I don't really know if I'm sure what latency is, but I understand that my server is located in Ohio, and if my clients don't reside in Ohio, the time to download from the website may not be as fast as the download time for a person who lives in Ohio.
That said, most of my users are located in Brazil, so I need to know if it is a good alternative to change the Ohio region by São Paulo, due to the fact that most customers are in Brazil, which would make their lives more easy.
If it is possible to make this change, is it a very complex thing that I should really leave as it is? I'm just a curious beginner, with the goal of learning more
Note:
I also want to point out, I use AWS RDS, S3 and CloudFront, should I change anything in these services?
You cannot change the EC2 instance's region. However, you can create an Amazon Machine Image (AMI) from the instance and then copy that to the desired region. Then launch the instance in the new region from the AMI you created.
Similarly with RDS. While you cannot change the region, you can create a snapshot to S3 and restore that snapshot in the new region.
I have approximately 40 Windows EC2 instances running at the moment. This number will start to grow substantially in the next few months. Each one is a t2.small Windows 2016 Server instance. Cost is starting to become an issue as the number increases. Each instance has its own Elastic IP address because when user Tom wants to access his machine he will use the DNS tom.mydomain.com.
tom.mydomain.com is registered in a Route53 hosted zone pointing to Elastic IP 22.33.44.55 which has been associated with Tom's EC2 instance.
Problem is that Tom only needs to use his machine 4 hours per day. When not using it he simply shuts the machine down. But... An Elastic IP that is pointing to a stopped instance costs almost as much per hour as a t1.micro instance in a running state.
So what I want to do is when Tom logs into AWS console and starts his EC2 instance, it will automatically register itself with Route53 against the DNS "tom.mydomain.com".
In short I want to do away with the need for Elastic IPs which are fast becoming a very substantial cost.
The tutorial Auto-Register EC2 Instance in AWS Route 53
looks like it does exactly what I want to do. The problem is the scripting is for Linux. I want to get it working for Windows. I have everything done down to step 6 in the tutorial but am stuck there. Any one get something similar to this working on Windows?
I would recommend:
Create a web-based front-end where your users can authenticate and request access to their Amazon EC2 instance
You could use Amazon Cognito for authentication and DynamoDB for data storage
Once the user authenticates, the service can:
Start their EC2 instance (if it was previously stopped)
Associate the random public IP address to the customer's domain name
Tell the user that the instance is now available
Users login to the instance and perform their work function
You then have some mechanism (I'm not sure what) that detects that they no longer need the instance, and then Stops the instance to save costs
The above process avoids assigning IAM credentials to your users. While IAM credentials are important for staff members who work on your AWS infrastructure, they should not be assigned to end-users of your service.
The process also avoids assigning IAM permissions to each EC2 instance. While the instances themselves could call Route 53 to update a record for their domain name, this requires an IAM Role to be assigned to the EC2 instance. If your users have access to the instance itself, this would potentially open a security hole where they could call Route 53 with incorrect data, such as assigning other users' domain names to their own instance.
It's worth mentioning that the above recommendations mirror the way that Amazon WorkSpaces operates — users authenticate, their instance is started and after a period of non-use the instance is stopped.
I will recommend use of cloudformation template. Cloudformation can create EC2 and then attach it to route53 url. So when tom like to use the EC2 instance, he have to run the stack in Cloudformation. Once he finished he have to go back to cloudformation and destroy the stack.
Yes Cloudformation would be a recommended approach. You can try cloudkast which is an online cloudformation template generator. It will make your task of creating cloudformation template very easy and effortless
I'm trying to view an AMI shared from one of my amazon accounts with another amazon account and it's not visible. I've followed all of the instructions here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sharingamis-explicit.html
I've been able to share an EBS Volume successfully, but not the AMI. Are there any undocumented issues or steps anyone has run in to which might keep me from viewing the shared AMI?
Alternatively - is there a way to build an AMI from the snapshot?
It's possible that one of your accounts is set to a different region than your other. At least, that's what I just ran into.
AMIs don't cross regions - an AMI created in region A will never show up in any other region. You'll see this if your two accounts are in different regions, or even if you switch regions in one of your accounts.
There are two things you can do:
Just change regions in your target account. The AMI should magically appear - at least, it did for me. Easy, but unsatisfying, if you really like some other region.
In your source account, copy the AMI into the region that you'd like to use it from. Here are the official docs, but it's pretty straightforward. From the console, right-click the AMI, select Copy AMI, choose your region, and press the Copy AMI button. Wait until copied, then set permissions on the newly-generated AMI.
You'll have to wait, but at least you won't have to go through the song-and-dance that you used to.
You need to be in the correct region as well on the left hand side of the filter below launch there is a drop down menu that defaults the AMI's to "Owned by Me". Since you are in the destination account you need to select "Private Images"
All Amazon AWS AMI's are public and visible to all accounts. Are you talking about an AMI that you (or someone else) explicitly created following these steps:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami.html
If so, the instructions for sharing those are here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-sharingamis.html
(I'm not sure the difference between your link and this link. They seem... the same)
EBS Snapshots, which are not AMIs, can be shared with other accounts. The instructions for sharing EBS Snapshots with other accounts are here:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-modifying-snapshot-permissions.html
I don't know of any direct way to create an AMI from a snapshot. I don't think there is one.
I am working on a project and am at a point where the POC is done and now want to move towards a real product. I am trying to understand the Amazon cloud offerings just to see if I need to be aware of them at development time. I have a bunch of questions that I cannot get answered from the Amazon site. Its probably because I am new to the whole web services thing and have never hosted a site before. I am hoping someone out here will explain this to me like I am a C programmer :)
I see amazon has a bunch of offerings -
EC2
Elastic Block Store
Simple DB
AuotScaling
Elastic Load Balancing
I understand EC2 is virtual server instances that I can use and these could come pre-loaded with what I want (say Apache + python). I have the following questions -
If I want a custom instance of something (like say a custom apache module I wrote for my project). Can I create a server instance using the exact modules and make it the default the next time I create a new instance or in Autoscaling?
Do I get an IP Address to access this? Can I set my own hostname to it? I mean do I get a DNS record? Or is it what Elastic IP is?
How do I access it from the outside? SSH? Remote Desktop? Or is it entirely up to how I configure the instance?
What do they mean by Inter-Region or Intra-Region data transfer? What is data transfer to begin with? Is it just people using my instance? So if I go live with it that will be the cost I have to pay for people using it?
What is the difference between AutoScaling and Elastic Load Balancing?
What is Elastic Block Store? Is it storage? If so do I have to worry about backups or do they take care of it?
About the Simple DB -
It looks like the interface to use this is different to my regular SQL calls. Am I correct?
If so the whole development needs to be tailored specifically for Amazon. Which kind of sucks. Is there a better alternative?
Do I get data backups or do I have to worry about it myself?
Will I be able to connect to the DB using regular tools to inspect the DB (during or afte development). Or do I get other tools made by Amazon for it?
What about security? The DB is obviously somewhere in the cloud farm away from the EC2 instance. My DB password is going over the wire and so is all my data totally unencrypted. Don't I have to worry about that? The question comes up only because I don't own any of the hardware.
I really hope some one points me in the right direction here.
Thanks for taking the time to read.
P
I just went through the question and here I tried to answer few of them,
1) AWS EC2 instances doesnt publish pre-configured instances, in fact its configured by the developers and made it publicly available to the users so that they can use it. One can any one of those instances or you can just opt for what ever OS you want which is raw and provision it accordingly and create a snap shot of it so that you can use it for autos caling.The snap shot becomes the base AMI in your case.
2) Every instance you boot will have a public DNS attach to it, you can use the public DNS to connect to that instance using ssh if your are a linux user or using putty if you are a windows users. Apart from that, you can also attach a elastic IP which comes with a cost will is like peanuts and attach it to the instance and access your instance through the elastic IP and you can either map the public DNS or elastic ip to map to a website by adding a A record or Cname respectively.
3)AWS owns databases in the different parts of the world. For example you deploy your application depending upon your customer base, if you target customers are based out of India, the nearest region available is Singapore which is called as ap-southeast-1 by AWS. Each region will have multiple availability zones, example ap-southeast-1a and ap-southeast-1b, which are two different databases and geographically part. Intre region means from ap-southeast-1a to ap-southeast-1b. Inter Region means, from ap-southeast-1 to us-east-1 which is Northern Virginia Data centre. AWS charges from in coming and out going bandwidth, trust me its nothing.
They chargge 1/8th of a cent per GB. Its a thing to even think about it.
4)Elastic Load balancer is cluster which divides the load equally to all your regions across availability zones (if you are running in multi AZ) ELB sits on top the AWS EC2 instances and monitors the instance health periodically and enables auto scaling
5) To help you understand what is autoscaling please go through this document http://aws.amazon.com/autoscaling/
6)Elastic Block store or EBS are like hard disk which is a persistent data storage which can be attached to your instance.Regarding back up yes dependents upon your use case. I do backups of EBS periodically.
7)Simple Db now renamed as dynamo DB is nosql DB, I hope you understand what is nosql db, its a non RDMS db systems. Please read some documentation to understand what is nosql db is.
8)If you have mysql or oracle db you can opt for RDS, please read the documents.
9)I personally feel you are newbie to the entire cloud eco system, you need to understand what exactly cloud does first.
10)You dont have to make large number of changes to development as such, just make sure it works fine in your local box, it can be deployed to cloud with out much ado.
11) You dont have to use any extra tool for that, change the database end point to RDS(if your use it) or else install mysql in your ec2 instance and connect to the local db which resides in the ec2 instance and connect to it,which is as simple as your development mode.
12)You dont have to worry about any security issues aws, it is secured. Dont follow the myths, I am have been using aws since 3 years running I dont even know remember how many applications, like(e-commerce,m-commerce,social media apps) I never faced any kind of security issues and also aws allows to set your security how ever you want.
Go ahead, happy coding. Contact me if you have any problem.
The answer above is a good summary on AWS. Just wanted to add
AWS offers full data center, so it depends what you are trying to achieve. For starters you will need,
EC2 - This is your server, it comes with instance storage, which will be lost on restart
EBS - Your mounted storage, the data is persisted across reboots
S3 - Provides storage (RESTful API's on top, the cost is usage based rather than "provisioned" as in EBS)
Databases - can start with Amazon RDS, which provides managed database services, you can chose between various available databases. You can also install your own database using EC2 + EBS, you will have to take care of managing the database yourself.
Elastic IP: Public facing IP address, you can point your DNS server to this.
One great tool to calculate the pricing,
http://calculator.s3.amazonaws.com/calc5.html
Some other services to take in account are:
VPC (Virtual Private Cloud). This is your own private network. You can define subnets, route tables and internet gateways there. I would strongly recommend to use VPC for any serious deployment of more than one instance.
Glacier - this will replace your tape library to storing backups.
Cloud Formation - great tool for deployment and automation of instances.