I'm deploying my app via Elastic Beanstalk, which creates and Elastic load balancer and puts all my instances behind it (3 or more).
Is there a way to contact each of these instances directly? I want to trigger a specific command on each instances (git pull command to synchronize with the latest code in my remote repo).
I have the list of IP address and public DNS of the instances from PHP SDK but since the firewalls rules restricts the source of IP address to the elastic load balancer IP on port 80, I can't seem to access them directly.
Is there a way around it?
P.S. The SSH port seems to open for all traffic, but how can I create a trigger with that? I'm hoping to create a PHP script to automate this with a webhook on the remote repo.
I highly suggest you use the EB CLI with git integration for all deployments, no matter how small. It is great because you can map a git branch to an environment with eb use YOUR_ENV then when you run eb deploy with that branch checked out it will deploy to that environment.
There is a lot of work involved in ensuring multiple servers pull the correct code and everything is working as expected. What if a server is in the processes of spinning up but is not ready for SSH so your script skips it and it does not get the new code?
Also, what happens when a new server spins up but it is using the old application because that's what is in EB? You could have your kickstart do a git pull but then what happens when you are not ready to push, a new server starts and is alone with the new code?
I could probably find 5 more edge cases without breaking a sweat. Look into eb deploy, you will be happy you did.
You need to setup a CI (or make a simple web service) and create a webhook in your repository. Your CI need to get all instances under your Elastic Beanstalk environment and then call git pull via SSH.
Or, just create a cron job in your all instances via .ebxensions script.
I thought it's not a good practice in Elastic Beanstalk to run git pull in order to synchronize your app with your git repo. Because, it misused the Application Version semantic meaning. Sometimes, you can't determine which app version are in your instances from Application Version. It's better to create a new Application Version in Elastic Beanstalk to deploy a new app version.
If you host your repo in Github, you can take a look into CodeDeploy.
Related
What are the exact differences between EC2, Beanstalk and LightSail in AWS?
What are good real time scenarios in which I should use these services?
They are all based on EC2, the compute service from AWS allowing you to create EC2 instances (virtual machines in the cloud).
Lightsail is packaged in a similar way than Virtual Private Server, making it easy for anyone to start with their own server. It has a simplified management console and many options are tuned with default values that maximize availability and security.
Elastic Beanstalk is a service for application developers that provisions an EC2 instance and a load balancer automatically. It creates the EC2 instance, it installs an execution environment on these machines and will deploy your application for you (Elastic Beanstalk support Java, Node, Python, Docker and many others)
Behind the scenes, Elastic Beanstalk creates regular EC2 instances that you will see in your AWS Console.
And EC2 is the bare service that allows the other to be possible. If you choose to create an EC2 instance, you will have to choose your operating system, manage your ssh key, install your application runtime and configure security settings by yourself. You have full control of that virtual machine.
In simple terms:
EC2 - virtual host or an image. which you can use it to install apps and have a machine to do whatever you like.
Lightsail - is similar but more user friendly management option and good for small applications.
Beanstalk - an orchestration tool, which does all the work to create an EC2, install application, software and give you freedom from manual tasks in creating an environment.
More details at - https://stackshare.io/stackups/amazon-ec2-vs-amazon-lightsail-vs-aws-elastic-beanstalk
I don't know if my scenario is typical in any way, but here are the differences that were critical for me. I'm happier EC2 than EB:
EC2:
just a remote linux machine with shell (command line) access
tracable application-level errors, easy to see what is wrong with your application
you can use AWS web console panel or AWS command line tool to manage
you will need repeated steps if you want to reproduce same environment
some effort to get proper shell access (eg fix security rule to your IP only)
no load balancer provided by default
Elastic Beanstalk
a service that creates a EC2 instance with a programming language of your choice (eg Python, PHP, etc)
runs one application on that machine (for python - application.py)
upload applications as .zip file, extra effort needed to use your git source
need to get used to environment vs applications mental model
application level errors hidden deep in the server logs, logs downloaded in separate menu
can be managed by web console, but also needs another CLI tool in addition to AWS CLI (you end up installing two CLI tools)
provides load balancer and other server-level services, takes away the manual setup part
great for scaling stable appications, not so much for trial-and-see experimentaion
probably more expensive than just an EC2 instance
Amazon EC2 is a virtual host, in other words, it is a server where you can SSH configure your application, install dependencies and so on, like in your local machine. EC2 has a dozen of AMI (Amazon Machine Image: it is some kind of operating system of your EC2 server, for instance, you can have EC2 running on Linux based OS or in windows OS). To summarize, it is a great idea if you need a machine in your hands.
Amazon Lightsail is a simple tool that you can deploy and manage application with small management of servers. You can find it very practical if your application is small, For instance, it will perfectly fit your application if you use Wordpress or other CMS.
AWS Elastic Beanstalk is an orchestration tool. You can manage your application within that service, it is more elevated then AWS Light Sail.
If you still do not understand the differences, you can take a look at each service overview.
There is also an answer in Quora
I have spent only 10 mins on these technologies but here is my first take.
EC2 - a baremetal service. It gives you a server with an OS. That is it. There is nothing else installed on it. So if you need a webserver (nginx) or python, you'll need to do it yourself.
Beanstalk - helps you deploy your applications. Say you have a python/flask application which you want to run on a server. Traditionally you'll have to build the app, move the deployable package to another machine where a web server should be installed, then move the package into some directory in the web server. Beanstalk does all this for you automatically.
LightSail - I haven't tried it but it seem to be an even simpler option to create a server with pre-installed os/software.
In summary, these seem to make application deployment more easier by pre-configuring the server/EC2s with the required software packages and security policies (eg. port nos. etc.).
I am not an expert so I could be wrong.
I've been tasked with getting a new SSL installed on a website, the site is hosted on AWS EC2.
I've discovered that I need the key pair in order to connect to the server instance, however the client doesn't have contact with the former web master.
I don't have much familiarity with AWS so I'm somewhat at a loss of how to proceed. I'm guessing I would need the old key pair to access the server instance and install the SSL?
I see there's also the Certificate Manager section in AWS, but don't currently see an SSL in there. Will installing it here attach it to the website or do I need to access the server instance and install it there?
There is a documented process for updating the SSH keys on an EC2 instance. However, this will require some downtime, and must not be run on an instance-store-backed instance. If you're new to AWS then you might not be able to determine whether this is the case, so would be risky.
Instead, I think your best option is to bring up an Elastic Load Balancer to be the new front-end for the application: clients will connect to it, and it will in turn connect to the application instance. You can attach an ACM cert to the ELB, and shifting traffic should be a matter of changing the DNS entry (but, of course, test it out first!).
Moving forward, you should redeploy the application to a new EC2 instance, and then point the ELB at this instance. This may be easier said than done, because the old instance is probably manually configured. With luck you have the site in source control, and can do deploys in a test environment.
If not, and you're running on Linux, you'll need to make a snapshot of the live instance and attach it to a different instance to learn how it's configured. Start with the EC2 EBS docs and try it out in a test environment before touching production.
I'm not sure if there's any good way to recover the content from a Windows EC2 instance. And if you're not comfortable with doing ops, you should find someone who is.
I have a very specific requirement for which I'm trying to use Elastic Beanstalk where I do not require an internet facing web server. The actual requirement is a little involved but boils down to needing a continuously running EC2 server which every 15mins uses wget or curl to pull a fairly large file from a private URL (other options have been looked at but for reasons outside my control this is the only way) where the transfer takes about 360s thus precluding the use of a CloudWatch-triggered Lambda function.
Since I don't want the hassle of managing the EC2 instance myself, I was looking at Elastic Beanstalk as a nice managed solution. I have proven my solution works as required on Beanstalk (where I have configured a cron job in .ebextensions/ to invoke a small shell script to wget the file and then upload it to a S3 bucket). It doesn't need Node.js, PHP, Python, Ruby, Go, Docker or any of the other platform options offered by Beanstalk, but Beanstalk is perfect for my particular use case.
The only thing is, it seems I get an Apache web server running on the EC2 whether I like it or not. What are my options here for configuring Beanstalk so that no Apache process is started?
I'm new to aws and am having some issues with getting my mobile app back running again. Forgive me if this question seems vague.
For a school project we created a mobile app on aws and deployed using docker containers (another student managed these tasks). When trying to get my own key pair to ssh into my ec2 instance i detached the volume associated with my instance and reattached it after getting my own key pair. Now i can ssh into my instance but my front end cant talk to my web server.
So my question is, do i create a new application on elastic beanstalk to deploy my app? Even though when i run lsblk is shows a have a docker loop device and when i run docker images i see several that match the name of my application? or do i somehow get the container running again, docker run doesn't seem to be working.
No need, just upload a new update into Elastic Beanstalk. AWS will handle the rest.
FYI, Elastic Beanstalk - Single Docker Container update process (simple under the hood):
You upload the update into AWS.
AWS will put it on your S3.
Inside your EC2, there is an Elastic Beanstalk agent. It will check for a new update.
If there is an update, the agent will download the update file and extract it.
The agent will build a new Docker image.
If the build is success, it will generate a new config to tell Nginx (web proxy) the new web server container.
Nginx will be reloaded.
Your old docker container will be destroyed.
Don't change anything inside EC2 of Elastic Beanstalk, except you know what you do. Elastic Beanstalk is design for automate deployment and scaling. So, if you change something in your EC2 manually, it might be lost. Of course, you can modified your EC2 instance, but you need to automate it using .ebextensions or take an image.
Was wondering if someone can help with below amazon AWS question, seems a basic item but can't find any answers, getting very frustrated.
1) I have an EC2 instance running that has a third party process running in the background, and when called from command line it spits out a number.
2) I have a java web app that runs this command line and uses the output for the web gui etc..
But for the life of me, i cannot figure out how to deploy my java web app on the SAME existing EC2 that's running the process, every time i try to create an elastic beanstalk it creates a new EC2 instance.
How do i make the elastic beanstalk to run off the same existing EC2 i already have? I understand there are other workaround to pass the data remotely but this seems a fundamental requirement that is missing from AWS - that you cannot run your web app with backend/batch processes on the same EC2 instance?
Thank you
Elastic Beanstalk is basically a higher abstraction layer on EC2 and it's tightly coupled with it. That means at a minimum every time you deploy your application it will spin up an EC2 server.
The advantage is that you don't need to manage your EC2 instances, for example it will autoscale automatically depending on your traffic demand.
The disadvantage is that it theoretically doesn't allow you to tweak little things in the EC2 instance because you may mess up how the Elastic Beanstalk interprets your app. Also, I believe you cannot force your Elastic Beanstalk deployment to use a specific AMI.
If you want more flexibility in your app (which sounds like your do) I recommend do your own deployment for your application (No Elastic Beanstalk). That way you can run the your app and your jobs on the same EC2 Instance.
You can use custom AMI with Elastic Beanstalk.
AWS documentation has a guide on how to create and use a customized AMI: Using Custom AMIs
But then again, nobody's stopping you from running your background processes on the standard Elastic Beanstalk instance. I run background cron jobs and Flask application on one Elastic Beanstalk instance.
files:
"/tmp/cronjob-for-foobar" :
mode: "000777"
owner: ec2-user
group: ec2-user
content: |
# skip
# clean up files created by above cronjob
30 23 * * * rm $HOME/cron*.log
container_commands:
70-foobar-cronjobs:
command: crontab /tmp/cronjob-for-foobar
Obviously, you can have anything scheduled in cron, as long as you stay within your instance limits.