I am able to successfully provision linux ssh users via cloud-init upon initial startup using this guide: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-user-account-cloud-init-user-data/
The issue I am facing is that every time I need to add or remove users, I need to shut down the machine to modify to user-data script. How would I go about creating and removing users while reducing downtime?
If I manually add users (https://aws.amazon.com/premiumsupport/knowledge-center/new-user-accounts-linux-instance/), will those users be removed when the instance restarts and runs the startup script?
You could potentially use EC2 launch templates. Very easy to setup and easy to clone to different instance sizes and classes. Pretty slick how I use them with user data to control environment variables for dev, stage, prod use too!
Related
AWS console has the option to view/change user data where we can change the data when the instance is stopped. But what i am unable to understand is why is this option to edit provided if the script is not run when we stop and restart the instance.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
Because cloud-init is just one use for user data (and it came into existence after EC2).
Consider instead a pre-built AMI that is already configured to run a service. That service needs to be configured in some way, and user data is a great way to do that (especially prior to the introduction of AWS Systems Manager). For example, you could store a list of variable settings in user data, then have the service read those into its environment.
Using configuration in that way, it's very nice to be able to change user data and restart the service to pick up changes.
Let's say I have a webapp where users can click a button that starts a long running task (eg. 3 days long). The user can also select options, for example the things it wants that task to do.
No matter what, the task will be the same script that runs when the instance starts. However, I would like it to somehow take arguments from the button click to change the function of the startup script.
Is there a way to do this with AWS EC2 instances?
So, you're saying that you want to pass certain parameters to some software that will be launched on an EC2 instance.
There are many ways to do this:
When the instance is launched, you can pass User Data. This is commonly used to run a startup script, but it can also be used just to pass information to the instance that can be accessed via http://169.254.169.254/latest/user-data/. So, either pass the configuration directly or pass it as part of the startup script.
Store it in tags on the instance when the instance is launched. Once the software starts up, it can retrieve the tags associated with the instance (itself) and act appropriately.
Store the configuration in a database and have the software access the database to determine what it should do.
Store the configuration in Amazon S3 and have the software retrieve the configuration file.
Personally, I like the idea of Tags. It's very Cloud.
This behaviour isn't directly related to EC2, although EC2 could host an application that does these long-running parameterised tasks. Whether EC2 is a good choice also depends on how your tasks react to underlying failures: if the EC2 instance fails or restarts, what happens to your task?
Depending on your use case, some managed options might be:
AWS Step Functions
AWS Simple Workflow Service
AWS Batch
While cloud-init is the optimal way to create users in an EC2 host at the time it gets instantiated I would like to hear how the community usually manages a dynamic group of users. I am planning to use Ansible to achieve the user config management but I am unsure how we can call the Ansible playbook from cloud-init script. Any pointers will be of help.
As you said, cloud-init by nature runs once and only once per instance, at creation time. If you expect your list of users to change over time and you want to keep an instance up to date, cloud-init isn't going to cover that use case.
The user module in ansible can be a good way to manage system users throughout the lifecycle of an instance.
I can't speak for the entire community but one possibility would be to maintain a list of users in ansible and then run your playbook targeting the instance or instances when there is a change to the user list.
Rather than cloud-init, these ansible-playbook runs could be manually run by an operator, run at a regular time interval, or triggered to run automatically with a combination of version control and continuous delivery.
I seem to not understand one central point of AWS Autoscaling:
I created an AMI of an (Ubuntu) EC2 instance having my web-server installed, and I use this AMI as the launch-configuration for my Autoscaling group.
But when Autoscaling decides to Spin up a new instance, how am I supposed to launch my webserver on that instance. Am I supposed to write some start-up scripts or what is best practice to start a process of a newly spun-up instance from Autoscaling?
When I deploy an application (PostgreSQL, Elasticsearch, whatever) to an EC2 instance, I usually do it intending on being able to repeat the process. So my first step is to create an initial deployment script which will do as much of the install and setup process as possible, without needing to know the IP address, hostname, amount of memory, number of processors, etc. Basically, as much as I can without needing to know anything that can change from one instance to the next or upon shutdown / restart.
Once that is stable, I create an AMI of it.
I then create an initialization script which I use in the launch-configuration and have it execute that script on the previously created AMI.
That's for highly configured applications. If you're just going with default settings (e.g. IP address = 0.0.0.0), than yes, I would simply set 'sudo update-rc.d <> defaults 95 10', so that it runs on startup.
Then create the AMI. When you create a new instance from that AMI, the webserver should startup by default. If it doesn't, I would look at whether or not you really set the init.d script to do so.
Launching a new instance from an AMI should be no different than booting up a previously shutdown instance.
By the way, as a matter of practice when creating these scripts, I also do a few things to make things much cleaner for me:
1) create modules in separate bash scripts (e.g. creating user accounts, set environment variables, etc) for repeatability
2) Each deployment script starts by downloading and installing the AWS CLI
3) Every EC2 instance is launched with an IAM role that has S3 read access, IAM SSH describe rights, EC2 Address allocation/association, etc.
4) Load all of the scripts onto S3 and then have the deployment / initialization scripts download the necessary bash module scripts, chmod +x and execute them. It's as close to OOP I can get without overdoing it, but it creates really clean bash scripts. The top level launch / initialization scripts for the most part just download individual scripts from S3 and execute them.
5) I source all of the modules instead of simply executing them. That way bash shares variables.
6) Make the linux account creation part of the initialization script (not the AMI). Using the CLI, you can query for users, grep for their public SSH key requested from AWS, create their accounts and have it ready for them to login automagically.
This way, when you need to change something (i.e. change the version of an application, change a configuration, etc.) you simply modify the module script and if it changes the AMI, re-launch and re-AMI. Otherwise, if it just just changes the instance-specifically, than just launch the AMI with the new initialization script.
Hope that helps...
I am running a couple of ubuntu ec2 instances, I want to run an automation script which will pull the code from Github whenever a new instance is booted from the AMI. The thing is presently I am sshing to the server and run the command git pull origin master and it will ask for password key.
How do I automate this process? So after booting the new instance from a AMI this script should:
Run
Pull the code and also the submodule
Create couple of files and configure it
Please help me to achieve it.
Thanks
This will probably take some time and configuring, but this might set you on the right path.
First, setup your ssh keys, so that you can automatically pull from a repo, without a password. Outlined here: https://help.github.com/articles/generating-ssh-keys
Next, create a startup script to issue the 'pull' command from Github. Here: https://help.ubuntu.com/community/UbuntuBootupHowto
Then save your AMI, When you start a new EC2 AMI, the script should run, pulling in your Github changes.
Also to note, make sure gits remote path is using SSH, if it is HTTPS, it will ALWAYS ask for a password.
Your best best would be to utilize the fact the Ubuntu utilizes CloudInit within its canonical image.
Using CloudInit, you can pass scripts (i.e. shell scripts) to execute at various start up stages as EC2 user-data.
It would be very easy for your to make your GIT command line sequence execute from such a script. He is link to documentation, which includes examples.
https://help.ubuntu.com/community/CloudInit
Create a user-password access to your ubuntu instance. Replicate this particular instance if you need multiple. Now you are free of the key access. If you need to automate a process in that instance cron it or send the script via ssh to that instance and let the cron to find and run it.