I've got development and production instances in EC2. I've been updating my app in Visual Studio 2019 and redeploying it to the dev instance, then creating an AMI of that instance and using that image to update the production instance(s).
Suddenly my app no longer updates when I deploy to the dev instance. The logs all show the update was applied, but when I look at the files on the server they have not changed for days. I suspect I may be using AMIs incorrectly, but I'm not sure what I'm doing wrong.
How do I get my updates to show again?
You are facing the issue because creating an AMI from running environment isn't the right approach since EB runs several scripts under the hood to attach instances to that particular environment.
Note: Custom AMIs are ideal only when you're installing a lot of dependencies or software that you want to be baked into your AMI so subsequent deployments go through quick. Here's the documentation that walks you through the steps, and here's the summary of the steps:
The best approach would be to launch a stand alone EC2 using an EB
AMI as base (ideally an AMI with HVM virtualization).
Connect to the instance with SSH or RDP.
Perform any customizations you want.
(Windows platforms) Run the EC2Config service Sysprep. For
information about EC2Config, see Configuring a Windows Instance Using
the EC2Config Service. Ensure that Sysprep is configured to generate
a random password that can be retrieved from the AWS Management
Console.
In the Amazon EC2 console, stop the EC2 instance. Then on the
Instance Actions menu, choose Create Image (EBS AMI).
Related
I have my app developed in Cloud9, and I would like to use the terminal with GIT commands to deploy my app to a EC2 instance. The objective is to make the app run in the EC2 instance.
Is it necessary to deploy the app to a EC2 instance, or is it already in a EC2 instance and I just have to open the URL of the app?
My cloud9 environment is in a region and my EC2 Instance is in another region (I'm telling this just in case it changes something in the process).
I already did it to Heroku, but I can't see how does it work to a EC2 instance.
Thank you very much !
If you want to deploy your app to an EC2 instance from the Cloud9 console, you have a couple of options. Please note, you only need to use one of these options, not all of them. I would generally recommend option #1 for your case based on your original question.
Use AWS amplify instead of EC2 (Amplify is Amazon's version of Heroku)
Use the AWS CDK client, and specifically, you'll want to look at Instance and either utilize userData or CodeBuild to build your app and deploy to EC2.
Use the AWS client to deploy Cloudformation templates (this is the lower-level version of option #2 and will require more boilerplate)
Connect your Git repository to AWS CodePipeline and run a CI/CD flow to deploy it to EC2 on every commit to the main/master branch (this is fairly complicated)
The most important thing to understand is that Cloud9 is simply an IDE (integrated development environment) that is deployed to an EC2 instance that is managed by AWS (not you). Cloud9 is not a tool for actually deploying code (you'll need to use one of the options I mentioned above for that).
I have the following doubts about using custom AMI with AWS EB.
Now I have:
a default platform, Python 3.6 + Amazon Linux 1.10.0, and in EB configurations > Instances > AMI I get an ID that I think is the default AMI provided by AWS to launch the platform (and if it is like that than it should get modified at every platform update).
some platform configurations done with .ebextensions files
my Flask app that I deploy from CLI (eb deploy)
So, in order to avoid the .ebextensions configurations time, I'd like to use a custom AMI that includes (1) + (2) and continue to deploy my Flask app like before.
So to build the AMI:
can I stop an EC2 instance of my running env and make an AMI from that one from EC2 console? If I do so, then the AMI would contain even my .ebextensions files and my app, is it a problem?
if the AMI shouldn't include .ebextensions files, then the only way to custom the platform before doing the AMI is to SSH?
after having built the AMI I put its ID in EB console > configurations > instances and then EB takes care of everything, like updating the AMI id in EC2 > autoscaling > launch options?
to do a platofrm update I have first to manually rebuild the AMI starting from the new platform and then update the AMI ID in EB configurations? So it's not possible to update the platform from EB console like I was used to do before and then to save the new AMI?
when I deploy my app it then shouldn't contain .ebextensions files?
if I create the AMI with my app included, then EB autoscaling would even save the time of deploying the app? (Of course in this case to deploy I would have to create a new AMI first).
Thanks for help.
can I stop an EC2 instance of my running env and make an AMI from that one from EC2 console? If I do so, then the AMI would contain even my .ebextensions files and my app, is it a problem?
You don't have to stop it. You can make AMI from running instance. Also your instance its in ASG, so stopping it is not a good idea.
if the AMI shouldn't include .ebextensions files, then the only way to custom the platform before doing the AMI is to SSH?
It shoudn't matter if you have pre-existing app on the ami. New deployment will install your app anyway.
after having built the AMI I put its ID in EB console > configurations > instances and then EB takes care of everything, like updating the AMI id in EC2 > autoscaling > launch options?
Yes,
to do a platofrm update I have first to manually rebuild the AMI starting from the new platform and then update the AMI ID in EB configurations? So it's not possible to update the platform from EB console like I was used to do before and then to save the new AMI?
Probably, have to repeat the process.
when I deploy my app it then shouldn't contain .ebextensions files?
It depends what they do. If they install software which is already on the custom ami, you can remove it.
if I create the AMI with my app included, then EB autoscaling would even save the time of deploying the app? (Of course in this case to deploy I would have to create a new AMI first).
The purpose of the custom ami is to save time on installing and configuring custom software that is normally not on the AWS amis. Its not to replace or elimiate the need of deploying your APP. You still need to do it, but can skip installing custom packages.
You can create a custom AMI from a running EC2 instance from the console, and from the CLI. Any AMI you create is a faithful copy of the instance, so if the instance has ebextensions, then the AMI will do also.
I think I understand that you want to create an AMI from instances being managed by ElasticBeanstalk? If that is so, then there are certain files that need to exist on the ElasticBeanstalk EC2 instance so that ElasticBeanstalk and Cloudformation can manage the environment. The .ebextensions are scripts are used to configure the environment, at least in my experience there are maintained in your repo. If your AMI has .ebextensions then they are most likely needed.
I don't think it is typical to use a custom AMI under ElasticBeanstalk: the whole point is to let AWS manage that layer for you. I would recommend that if you really need a custom AMI, you look at doing what you want to do directly in EC2 and forgo ElasticBeanstalk. ElasticBeanstalk is really only an abstracted 'friendly' interface to EC2 and other services (eg autoscaling and load balancer are actually EC2). Maybe even consider putting your application into a docker?
You can create Custom AMI of EC2 instance which is running for Elastic beanstalk. IF you are going with custom AMI then no need to use .ebextension files because either AMI should include all the changes which has already done when you deployed application along with ebextension file or do the necessary changes in server before creating AMI. But it is good to use default AMI which AWS provides while creation of Elastic Beanstalk and use .ebextension files to do required tasks during deployment.
I have a release of my project. I build a docker image and deploy it on an ec2 instance.
Later, when I have a new release, I would like update the docker on ec2 remotely (without accessing the machine, just executing some service).
Is there a way how to do it without ECS and ElasticBeanstalk?
If it's not possible can I somehow re-run the cfn-init script?
My Research
https://aws.amazon.com/blogs/aws/new-ec2-run-command-remote-instance-management-at-scale/
You can manage your instances remotely (i.e. make changes without manually SSHing into the instance and typing commands) by using any of the many system management services out there. AWS offers Simple Systems Manager (SSM) of which the Run Command you linked is part. AWS also offers the OpsWorks service which uses Chef. You also have other products like Ansible and SaltStack, and you can optionally integrate the use of those services with the AWS SSM service.
I know this has been partially answered in a bunch of places, but the answers are so.. all over the map, dated and not well explained. I'm looking the best practice as of February 2016.
The setup:
A PHP-based RESTful application service that lives in an EC2 instance. The EC2 instance uses S3 for uploaded user data (image files), and RDS MySql for its DB (these two points aren't particularly important.)
We develop in PHPStorm, and our source control is GitHub. When we deploy, we just use PHPStorm's built-in SFTP deployment to upload files directly to the EC2 instance (we have one instance for our Staging environment, and another for our Production environment). I deploy to Staging very often. Could be 20 times a day. I just click on a file in PHPStorm and say 'deploy to Staging', which does the SFTP transfer. Or, I might just click on the entire project and click 'deploy to Staging' - certain folders and files are excluded from the upload, which is part of PHPStorm's deployment configuration.
Recently, I put our EC2 instance behind a Load Balancer. I did this so that I can take advantage of Amazon's free SSL offering via the Certificate Manager, which does not support individual EC2 instances.
So, right now, there's a Load Balancer with only a single EC2 instance behind it. I maintain an Elastic IP pointing to the EC2 instance so that I can access it directly (see my current deployment method above).
Question:
I have not yet had the guts to create additional (clone) EC2 instances behind my Load Balancer, because I'm not sure how I should be deploying to them. A few ideas came to mind, but they're all pretty hacky.
Given the scenario above, what is currently the smoothest and best way to A) quickly deploy a codebase to a set of EC2 instances behind a Load Balancer, and B) actually 'clone' my current EC2 instance to create additional instances.
I haven't been able to really paint a clear picture of the above in my head yet, despite the fact that I've gone over a few (highly technical) suggestions.
Thanks!
You need to treat your EC2 instance as 100% dispensable. Meaning, that it can be terminated at any time and you should not care. A replacement EC2 instance would start and take over the work.
There are 3 ways to accomplish this:
Method 1: Each deployment creates a new AMI image.
When you deploy you app, you deploy it to a worker EC2 instance whose sole purpose is for "setup" of your app. Once the new version is deployed, you create a fresh AMI image from the EC2 instance and update your Auto Scaling launch configuration with the new AMI image. The old EC2 instances are terminated and replaced with the new code.
New EC2 instances have the recent code already on them so they're ready to be added to the load balancer.
Method 2: Each deployment is done to off-instance storage (like Amazon S3).
The EC2 instances will download the recent code from Amazon S3 and install it on boot.
So to put the new code in action, you terminate the old instances and new ones are launched to replace them which start using the new code.
This could be done in a rolling-update fashion, or as a blue/green deployment.
Method 3: Similar to method 2, but this time the instances have some smarts and can be signaled to download and install the code.
This way, you don't need to terminate instances: the existing instances are told to update from S3 and they do so on their own.
Some tools that may help include:
Chef
Ansible
CloudFormation
Update:
Methods 2 & 3 both start with a "basic" AMI which is configured to pull the webpage assets from S3. This AMI is not changed from version-to-version of your website.
For example, the AMI can have Apache and PHP already installed and on boot it pulls the .php website assets from S3 and puts them in /var/www/html.
CloudFormation works well for this. In addition, for method 3, you can use cfn-hup to wait for update signals. When signaled, it'll pull updated assets from S3.
Another possibility is using Elastic Beanstalk which could be used to manage all of this for you.
Update:
To have your AMI image pull from Git, try the following:
Setup an EC2 instance with everything installed that you need to have installed for your web app
Install Git and setup a local repo ready to Git pull.
Shutdown and create an AMI of your instance.
When you deploy, you do the following:
Git push to GitHub
Launch a new EC2 instance, based on your AMI image.
As part of the User Data (specified during the EC2 instance launch), specify something like the following:
#!/bin/sh
cd /git/app
git pull
; copy files from repo to web folder
composer install
When done like this, that user data acts as a script which will run on first boot.
I have configured an aws asg using ansible to provision new instances and then install the codedeploy agent via "user_data" script in a similar fashion as suggested in this question:
Can I use AWS code Deploy for pulling application code while autoscaling?
CodeDeploy works fine and I can install my application onto the asg once it has been created. When new instances are triggered in the ASG via one of my rules (e.g. high cpu usage), the codedeploy agent is installed correctly. The problem is, CodeDeploy does not install the application on these new instances. I suspect it is trying to run before the user_data script has finished. Has anyone else encountered this problem? Or know how to get CodeDeploy to automatically deploy the application to new instances which are spawned as part of the ASG?
AutoScaling tells CodeDeploy to start the deployment before the user data is started. To get around this CodeDeploy gives the instance up to an hour to start polling for commands for the first lifecycle event instead of 5 minutes.
Since you are having problems with automatic deployments but not manual ones and assuming that you didn't make any manual changes to your instances you forgot about, there is most likely a dependency specific to your deployment that's not available yet at the time the instance launches.
Try listing out all the things that your deployment needs to succeed and make sure that each of those is available before you install the host agent. If you can log onto the instance fast enough (before AutoScaling terminates the instance), you can try and grab the host agent logs and your application's logs to find out where the deployment is failing.
If you think the host agent is failing to install entirely, make sure you have Ruby2.0 installed. It should be there by default on AmazonLinux, but Ubuntu and RHEL need to have it installed as part of the user data before you can install the host agent. There is an installer log in /tmp that you can check for problems in the initial install (again you have to be quick to grab the log before the instance terminates).