Can someone help me to understand the basics of spawning EC2 instances and deploying AMIs and how to configure them properly?
Current situation:
In my company we have 1 server and a few clients which run calculations and return the results when they are done. The system is written in Python but sometimes we run out of machine power so I am considering to support the clients with additional EC2 clients - on demand. The clients connect to the server via an internal IP which is set in a config file.
Question:
Am I assuming right that I just create an AMI where our Python client sits in autostart and once its started it connects to the public IP and picks up new tasks? Is that the entire magic or do I miss some really great features in this concept?
Question II
While spawning a new instance, can I start such instance with updated configuration or meta information or do I have to update my AMI before all the time I make a small change?
if you want to stick with just plain spawning EC2 instances, here are the answers to your questions:
Question I - This is one of the valid approaches and yes, if your Python client will be configured properly, it will 'just work'.
Question II - Yes, you can achieve that, which is very well explained here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html. There's also another way of having your configuration stored somewhere else, and just fetch it when the instance is starting.
Related
Just looking to be pointed in the right direction here!
I have an old "Playerworlds" MMO game that I'd like to set up a semi-permanent host onto an AWS instance.
The software is from 2008, and requires an EXE file to be started in order to start the environment.
In AWS, what type of setup should I use for this type of application hosting? Is there a solution with a UI like a Virtual Machine I can log into at all?
You can create Windows Server instances of EC2's easily enough. You will use RDP to log in and manipulate the machine. I'll admit I don't know if Windows Server 2012 will run the software - fire up an instance and give it a shot.
In GCP how can we check if the compute engines are not accessible on port 8080. Is there any API where we can check and validate this scenario
There are many ways of doing what you want - however there are a few factors that I don't know so this answer may sound a bit generic in a few places.
Scenario 1 - instances have to be accessible from the Internet
check if the firewall settings allow incoming traffic to your instances on port 8080 (you can use cloud console or gcloud).
if there isn't such a rule you have to create one - it's best to label your instances and create a proper rule.
now you can actually check if there's anything running on port 8080 - if this is a web app / API you can just use curl host.ip:port 2> errors.log.
You mentioned you have a lot of instances to check then some script would be handy - have a look at this SO answer how to create one that will read the addressess from the file.
If you want to do it like a pro use Ansible - here's a useful answer that will be helpful for this.
Scenario 2 - instances are not available from the Internet
you need to run the mentioned instance checking script from within your VPC your instances are in. Create a new VM for this purpose and run the script from there. If your instances are spread across many VPC's you need to create a VM in each of them and run the script.
And you can automate this with Ansible - even create/test instances & delete VM's. This may sound like an overkill but everything on how often you need to run those tests and on the number of VM's you need to test.
Also there's a question of testing if the ports are open from inside the insances.
if they are running Linux then dany L's suggestion is a good one. But since you have to repeat that many-many times Ansible may again be a good way to do this - have a look at another answer describing how to run a command on the target host.
if they are running Windows then it's more complicated but you can use netsh firewall command - and again - using Ansible is possible.
Is there a way to determine through a command line interface or other trick if an AWS EC2 instance is ready to receive ssh connections?
The running state seems not to be enough. Trying to connect in in the first minutes of the running state, the machine Status checks still shows initialising and ssh times out while trying to connect.
(I am using the awscli pip package.)
Running is similar to turning a computer on and finishing a bios check. As far as the hypervisor is concerned your instance is on.
The best way to know when your instance is ready, is to run a script at the end of startup (or when certain services are on) that will report its status to some other listener. Using that data, or event, you should know that your instance is ready to be connected to. This is purposely vague since there are so many different ways this can be accomplished.
You could also time the expected startup time, and try to connect after that and retry the connection if it fails. Still need a point at which you would stop trying as instances can fail to launch in some cases.
Lets say I have a website with 4 webapp server and 1 HAProxy before them to do load balancing. Now I want to update my webapp with new api/v2 and I start the rolling update. My webapp is doing HATEOAS so lets assume that 1 instance got updated and it sent a link like api/v2/dothis to a clinet.
Now the client made a request on this link and HAProxy directed it to 3rd server in the cluster which is still running the old webapp and doesn't know about api/v2.
How do people solve this problem in general, how do websites do rolling updates without disrupting the service.
Thanks in advance
You could use one of these options
Option a: Once you
updated instance 1, shut down all the other instances so all the
traffic goes to instance 1 (if this is even possible with the load
you might expect. You could do this at a time when your one instance
would be capable). Update instance 2 with the new webapp and bring it
online, continue with all the other instances.
Option b: Keep all the available resources in a place to where all
your servers can check whether the resources exists on another webapp
instance if they do not have it themselves (yet).
I feel that option a would be best, since you would not have to maintain another server/system for brokerage.
I'm still cheap.
I have a software development environment which is a bog-standard Ubuntu 11.04 plus a pile of updates from Canonical. I would like to set it up such that I can use an Amazon EC2 instance for the 2 hours per week when I need to do full system testing on a server "in the wild".
Is there a way to set up an Amazon EC2 server image (Ubuntu 11.04) so that whenever I fire it up, it starts, automatically downloads code updates (or conversely accepts git push updates), and then has me ready to fire up an instance of the application server. Is it also possible to tie that server to a URL (e.g ec2.1.mydomain.com) so that I can hit my web app with a browser?
Furthermore, is there a way that I can run a command line utility to fire up my instance when I'm ready to test, and then to shut it down when I'm done? Using this model, I would be able to allocate one or more development servers to each developer and only pay for them when they are being used.
Yes, yes and more yes. Here are some good things to google/hunt down on SO and SF
--ec2 command line tools,
--making your own AMI's from running instances (to save tedious and time consuming startup gumf),
--route53 APIs for doing DNS magic,
--ubunutu cloud-init for startup scripts,
--32bit micro instances are your friend for dev work as they fall in the free usage bracket
All of what James said is good. If you're looking for something requiring less technical know-how and research, I'd also consider:
juju (sudo apt-get install -y juju). This lets you start up a series of instances. Basic tutorial is here: https://juju.ubuntu.com/docs/user-tutorial.html