Let me preface this by saying that I am primarily a programmer, though I have a pretty good working knowledge of Linux and "standard" LAMP installations. I have been tasked with setting up a persistent LAMP environment in Amazon Web Services (AWS), which is a good deal more involved than what I'm used to in this regard.
Although AWS is very well documented, I have yet to find a clear, definitive "Best Practices" overview for setting up a persistent LAMP environment. I followed the official Amazon tutorial ( http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-LAMP.html ) to set up a LAMP server on our EC2 instance, but found out later that these instances are "temporary" and that I need an EBS to make anything persist. Interestingly, EBS (Elastic Block Storage) does not appear in my Management Console , though they offer pricing out on the public side ( https://aws.amazon.com/ebs/pricing/ ). Is it still called EBS?
Of course, that begs the question - what happens to the programs I installed (Apache, MySQL) along with their respective config files? Surely Amazon doesn't expect us to reconfigure our server from scratch every time it boots up?
What I have now
1x EC2 instance running Amazon Linux. I installed and configured Apache and MySQL following the "Install LAMP" tutorial posted by Amazon.
1x Route 53 Hosted Zones (for DNS routing)
1x Elastic IP attached to the EC2 server
Additionally, there appears to be one unencrypted 8GB volume attached to /dev/xvda, although I didn't set it up and nobody has access but myself - it seems to have been generated when I requisitioned the EC2 - no idea if it is persistent or not.
What I think I need
So, here is what I'm thinking I need to do. Please tell me if I'm way off - is there a more sane alternative?
1x EC2 instance running Amazon Linux and Apache
1x RDS for MySQL
1x Route 53 Hosted Zone
1x Elastic IP attached to the EC2 server
1x (EBS? S3? EFS?) for storing htdocs
1x Snapshot of the EC2 to save server configuration
Does that sound right? Is there a better way to do this? Thanks so much for any advice. Amazon docs seem to be very good at giving granular information, but not as great at addressing overall strategy concerns.
Web Application
It is recommended to have 2 EC2 instances under an Elastic Load Balancer with these 2 instances being in separate availability zones for high availability. Going further is better to monitor these instances for CPU and bandwidth - CloudWatch - and once you see they are above some threshold you could automatically add more instances to the ELB, this is the auto scaling. Of course like you said you will need the AMI (snapshot) with the server software to be ready to be launched. You also need to take down servers when the load is small - again automatically with the metrics, but you should never go down under 2 machines. And don't forget to update these images when you upgrade the software.
Route 53
Because you would use an ELB you don't need Elastic IPs anymore, your web servers could only have private IPs. And on the Route 53 you need to point your website to the name of the ELB - here are more details about it
Database
For the database part go for the RDS for MySQL including Multi-AZ deployment, so you will have the master and one stand by replica in different availability zones.
EBS (disks)
For the EBS part you will ned to use that and they come in 3 flavours: magnetic(slowest), General Purpose SSD (faster) and Provisioned IOPS (fastest). These are the disks you mount on your machines, web servers and databases. For the database you should go with Provisioned, it is much harder to change them later, while for the web server we use General Purpose. In the AWS Console you find them under EC2, section Elastic Block Store.
The 8G disk that appeared is the default when you create a Linux machine and it is a General Purpose SSD which is good enough for a web server, but I think you should go with a bigger one, 50G at least.
Related
I have use case query regarding scaling that I have a PHP Magento application running inside a Docker container on AWS Elastic Beanstalk backed by a AWS RDS (t2.micro) instance.
Now when I use Apache JMeter for load testing, I throw just 100 users in 10 seconds and my RDS instance goes down but Beanstalk instance remains fine. My question is how do I scale the RDS when an huge number of traffic comes?
Some said to me that you should use a at least medium instance for your RDS but how can we use the autoscale feature? Whether it exists in the case of RDS or not?
Some also said to use AWS RDS Read Replicas! I did not work with Read Replicas before but how do I load balance the traffic between different RDS read replicas, what are different ways and what if we need some low database operations? Will it scale in?
Any appropriate guidance will be appreciable. :)
I'm working on an eCommerce website (developed and set up by other develpers) that have the web server on a m4.2xlarge instance and the database on a t2.micro (EC2 and EBS instances).
The usual traffic is about 5,000 unique visitors/day but we are expecting a huge increment in the nex future because of some advertising on big newspapers and I would like to understand if I need to scale up the web server instance or the DB instance or both.
It is not clear to me if the two instances are sharing the same RAM and CPU or if they are completely separated.
Thank you !
Both the systems will have different CPU and RAM.
You should choose RDS instead of a normal EC2 instance for Database.
You can take a backup from older instance and load into your RDS instances. This will also help you maintaining high availability as well.
Sorry, I had a few basic questions. I'm planning to use an AWS EC2 instance.
1) Is an EC2 instance a single virtual machine image or is it a
single physical machine? Documentation from Amazon states that it is
a "virtual server", but I wanted to clarify if it is an image
embedded inside one server or if it is an single physical server
itself.
2) Is an Elastic Load Balancer a single EC2 instance that handles
all requests from all users and simply forwards the request to the
least loaded EC2 instances?
3) When Auto-Scaling is enabled for an EC2 instance, does it simply
exactly replicate the original EC2 instance when it needs to scale
up?
An EC2 instance is a VM that gets some percentage of the underlying physical host's RAM, CPU, disk, and network i/o. That percentage could theoretically be 100% for certain instance types, including bare-metal instances, but is typically some fraction depending on which instance type you choose.
ELB is a service, not a single EC2 instance. It will scale on your behalf. It routes by round robin for TCP, and routes on fewest outstanding requests for HTTP and HTTPS.
Auto Scaling is "scale out" (it adds new EC2 instances), not "scale up" (resizing an existing EC2 instance). It launches a new instance from a template called an AMI.
It is a virtual server, a VM, as stated in the documentation.
It's a little more complicated that that, based on the way AWS might scale the load balancer, or create a version in each availability zone, etc. It also provides more features such as auto-scaling integration, health checks, SSL termination. I suggest you read the documentation.
It uses a machine image that you specify when you create the auto-scaling group (when you create the Launch Configuration used by the Auto-scaling group to be more precise). A common practice is to configure a machine image that will download any updates and launch the latest version of your application on startup.
You might also be interested in Elastic Beanstalk which is a PaaS that manages much of the AWS infrastructure for you. There are also third-party PaaS offerings such as OpenShift and Heroku that also manage AWS resources for you.
From an AWS newbie. Before I throw money at Amazon for higher-performing servers for my ETL tool, am I getting the most out of my existing setup?
I have set up a SQL Server RDS instance (db.m1.small) to which I'm connecting from a RHEL-6 EC2 instance (m1.small). On the EC2 I have a Python ETL job to read source flat-file data and run inserts to the RDS over the Microsoft ODBC driver. This process it an order of magnitude slower than when everything is hosted on my 64-bit Core-i7 Windows laptop with a large SSD drive.
My two servers are not in a VPC but are talking to each other via the same Internet IPs that I connect to them with from my laptop; could this be at the root of my performance problem? Could performance be improved if they were talking over local IPs instead? I've looked at Amazon's documentation of VPCs but they tout security improvements only, never performance improvements, over EC2 security groups.
Thanks for any help, BH
Is it possible to have most of our server hardware outside of EC2, but with some kind of load balancer to divert traffic to EC2 when there's load that our servers can't handle, or as a backup incase these servers go down?
For example, have a physical server serving out our service (let's ignore database consistency for the moment), but there's a huge spike due to some coolness - can we spin up some EC2 instances and divert traffic off to it? This is much like Amazon's own auto scaling.
And also, if our server hardware dies for some reason (gremlins eat the power cables for example) - can we route all our traffic over to EC2 instances?
Thanks
Yes you can but you will have to code. AWS has Command Line Tools for doing EC2/Autoscaling/S3 stuff with simple commands in bash or other interfaces and SDKs, like Boto for Python etc.
You can find it here: http://aws.amazon.com/code/
Each Ec2 instance has a public network interface associated with it. Use a DNS CNAME record to "switch" your site traffic to the Ec2 instance. If you need to load-balance across multiple machines you can use round-robin DNS, or start a ELB and put any number of Ec2 instances behind it.
Ec2 infrastructure is extremely easy to scale. Deploying your application on top of an Ec2 is a whole other matter. It could be trivial -- or insanely complicated.