Once you created an image using packer. How do you get the root/ packer/ (any user with sudo privilege)'s password?
packer.json
{
"builders": [
{
"type": "googlecompute",
"project_id": "project-id",
"source_image": "ubuntu-2004-focal-v20220615",
"ssh_username": "packer",
"zone": "us-west1-a"
}
],
}
After running:
packer build packer.json #a Ubuntu image is created.
Next step, how do I create a sudo user's account?
Related
We have a large number of EC2 instances, both Windows and Linux, and we have CloudHealth v 10.0.0.180 installed. I understand there are newer versions such as 10.0.0.220 but I can't find a definitive list of the versions and which one is the latest. I have an AWS custom doc that pushes CloudHealth v10.0.0.180 (see below) but if I update that doc to push 10.0.0.220 it says it succeeds but the version does not change. Below are the URLs I am using in the doc for both v 10.0.0.180 and 10.0.0.220. The full document code is below as well.
https://s3.amazonaws.com/remote-collector/agent/windows/18/CloudHealthAgent.exe\
https://s3.amazonaws.com/remote-collector/agent/windows/22/CloudHealthAgent.exe\
{
"description": "Download and Install CloudHealth Agents",
"schemaVersion": "2.2",
"mainSteps": [
{
"inputs": {
"runCommand": [
"Write-Output \"Installing CloudHealth Agent\"",
"$url = \"https://s3.amazonaws.com/remote-collector/agent/windows/22/CloudHealthAgent.exe\"",
"$output = \"C:\\CloudHealthAgent.exe\"",
"$start_time = Get-Date",
"Invoke-WebRequest -Uri $url -OutFile $output",
"C:\\CloudHealthAgent.exe /S /v\"/l* install.log /qn CLOUDNAME=aws CHTAPIKEY=6a4290cd-116d-46f5-b8f4-eb6c6ee4bf46\"",
"Write-Output \"Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)\""
]
},
"name": "CloudHealthAgentWindows",
"action": "aws:runPowerShellScript",
"precondition": {
"StringEquals": [
"platformType",
"Windows"
]
}
},
{
"inputs": {
"runCommand": [
"echo “Installing CloudHealth Agent”",
"sudo yum install wget -y",
"wget https://s3.amazonaws.com/remote-collector/agent/v22/install_cht_perfmon.sh",
"sudo sh install_cht_perfmon.sh 20 8fdf2776-eda0-441b-bca8-0566ded6daf1 aws;"
]
},
"name": "CloudHealthAgentLinux",
"action": "aws:runShellScript",
"precondition": {
"StringEquals": [
"platformType",
"Linux"
]
}
}
]
}
I just went through updating the agents myself. I had to uninstall the old agent, reboot the instance, and then I was able to successfully install the new (22) version.
Context
We have an automated AMI building process done by Packer in order to setup our instance images after code changes, and assign them to our Load Balancer launch configuration for faster autoscaling.
Problem
Recently the instances launched with the development environment AMI build started refusing the corresponding private key. After attaching an instance with that same AMI in a public subnet, I connected through EC2 connect and noticed that the public key was not present in the /home/ec2-user/.ssh/authorized_keys file, even though it was added at launch time through the launch configuration or even manually through the console.
The only one present being the Packed temporary SSH key created during the AMI packaging.
Additional info
Note that the key pair is mentionned in the instance details as though it was present but it is NOT, which was tricky to debug because we tend to trust what the console tells us.
What's even more puzzling is that the same AMI build for QA environment (which is exactly the same apart from some application variables) does include the EC2 SSH key correctly, and we are using the same key for DEV & QA environments.
We're using the same Packer version (1.5.1) than ever to avoid inconsistencies, so it most likely cannot come from that, but I suspect it does come from the build since it does not happen with other AMIs.
If someone has a clue about what's going on, I'd be glad to know.
Thanks !
Edit
Since it was requested in the comment, I can show you the "anonymized" packer template, for confidentiality reasons I can't show you the ansible tasks or other details. However, note that this template is IDENTICAL to the one used for QA (same source code) which does not gives the error.
"variables": {
"aws_region": "{{env `AWS_REGION`}}",
"inventory_file": "{{env `Environment`}}",
"instance_type": "{{env `INSTANCE_TYPE`}}"
},
"builders": [
{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "{{user `aws_region`}}",
"vpc_id": "vpc-xxxxxxxxxxxxxxx",
"subnet_id": "subnet-xxxxxxxxxxxxxxxxxx",
"security_group_id": "sg-xxxxxxxxxxxxxxxxxxxxx",
"source_ami_filter": {
"filters": {
"virtualization-type": "hvm",
"name": "amzn2-ami-hvm-2.0.????????.?-x86_64-gp2",
"root-device-type": "ebs",
"state": "available"
},
"owners": "amazon",
"most_recent": true
},
"instance_type": "t3a.{{ user `instance_type` }}",
"ssh_username": "ec2-user",
"ami_name": "APP {{user `inventory_file`}}",
"force_deregister": "true",
"force_delete_snapshot": "true",
"ami_users": [
"xxxxxxxxxxxxxxxxx",
"xxxxxxxxxxxxxxxxx",
"xxxxxxxxxxxxxxxxx"
]
}
],
"provisioners": [
{
"type": "shell",
"inline": [
"sudo amazon-linux-extras install ansible2",
"mkdir -p /tmp/libs"
]
},
{
"type": "file",
"source": "../frontend",
"destination": "/tmp"
},
{
"type": "file",
"source": "../backend/target/",
"destination": "/tmp"
},
{
"type": "ansible-local",
"playbook_file": "./app/instance-setup/playbook/setup-instance.yml",
"inventory_file": "./app/instance-setup/playbook/{{user `inventory_file`}}",
"playbook_dir": "./app/instance-setup/playbook",
"extra_arguments": [
"--extra-vars",
"ENV={{user `inventory_file`}}",
"--limit",
"{{user `inventory_file`}}",
"-v"
]
},
{
"type": "shell",
"inline": [
"sudo rm -rf /tmp/*"
]
}
]
}
In AWS to make a new AMI, I usually run commands manually to verify that they're working, and then I image that box to create an AMI. But there are alternatives like packer.io, what's a minimal working example for using this service to make a simple customized AMI?
https://github.com/devopracy/devopracy-base/blob/master/packer/base.json There's a packer file that looks very similar to what I use at work for a base image. It's not tested, but we can go into it a bit. The base image is my own base - all services are built using it as a source ami. That way I control my dependencies and ensure there's a consistent os under my services. You could simply add cookbooks from the chef supermarket to see how provisioning a service works with this file, or use this as a base. As a base you would make a similar, less detailed build for the service and call this as the source ami.
This first part declares the variables I use to pack. The variables are injected before the build from a bash file which I DON'T CHECK IN TO SOURCE CONTROL. I keep the bash script in my home directory and source it before calling packer build. Note there's a cookbook path for the chef provisioner. I use the base_dir as the location on my dev box or the build server. I use a bootstrap key to build; packer will make it's own key to ssh if you don't specify one, but it's nice to make a key on aws and then launch your builds with it. That makes debugging packer easier on the fly.
"variables": {
"aws_access_key_id": "{{env `AWS_ACCESS_KEY`}}",
"aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
"ssh_private_key_file": "{{env `SSH_PRIVATE_KEY_FILE`}}",
"cookbook_path": "{{env `CLOUD_DIR`}}/ops/devopracy-base/cookbooks",
"base_dir": "{{env `CLOUD_DIR`}}"
},
The next part of the file has the builder. I use amazon-ebs at work and off work too, it's simpler to create one file, and often the larger instance types are only available as ebs. In this file, I resize the volume so we have a bit more room to install stuffs. Note the source ami isn't specified here, I lookup the newest version here or there. Ubuntu has a handy site if you're using it, just google ec2 ubuntu locator. You need to put in a source image to build on.
"builders": [{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key_id`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "us-west-2",
"source_ami": "",
"instance_type": "t2.small",
"ssh_username": "fedora",
"ami_name": "fedora-base-{{isotime \"2006-01-02\"}}",
"ami_description": "fedora 21 devopracy base",
"security_group_ids": [ "" ],
"force_deregister": "true",
"ssh_keypair_name": "bootstrap",
"ssh_private_key_file": "{{user `ssh_private_key_file`}}",
"subnet_id": "",
"ami_users": [""],
"ami_block_device_mappings": [{
"delete_on_termination": "true",
"device_name": "/dev/sda1",
"volume_size": 30
}],
"launch_block_device_mappings": [{
"delete_on_termination": "true",
"device_name": "/dev/sda1",
"volume_size": 30
}],
"tags": {
"stage": "dev",
"os": "fedora 21",
"release": "latest",
"role": "base",
"version": "0.0.1",
"lock": "none"
}
}],
It's very useful to tag your images when you start doing automations on the cloud. These tags are how you'll handle your deploys and such. fedora is the default user for fedora, ubuntu for ubuntu, ec2-user for amazon linux, etc. You can look those up in docs for your distro.
Likewise, you need to add a security group to this file, and a subnet to launch in. Packer will use the defaults in aws if you don't specify those but if you're building on a buildserver or a non-default vpc, you must specify. Force deregister will get rid of an ami with the same name on a successful build - I name by date, so I can iterate on the builds daily and not pile up a bunch of images.
Finally, I use the chef provisioner. I have the cookbook in another repo, and the path to it on the buildserver is a variable at the top. Here we're looking at chef-zero for provisioning, which is technically not supported but works fine with the chef client provisioner and a custom command. Beside the chef run, I do some scripts of my own, and follow it up by running serverspec tests to make sure everything is hunky dory.
"provisioners": [
{
"type": "shell",
"inline": [
]
},
{
"type": "shell",
"script": "{{user `base_dir`}}/ops/devopracy-base/files/ext_disk.sh"
},
{
"type": "shell",
"inline": [
"sudo reboot",
"sleep 30",
"sudo resize2fs /dev/xvda1"
]
},
{
"type": "shell",
"inline": [
"sudo mkdir -p /etc/chef && sudo chmod 777 /etc/chef",
"sudo mkdir -p /tmp/packer-chef-client && sudo chmod 777 /tmp/packer-chef-client"
]
},
{
"type": "file",
"source": "{{user `cookbook_path`}}",
"destination": "/etc/chef/cookbooks"
},
{
"type": "chef-client",
"execute_command": "cd /etc/chef && sudo chef-client --local-mode -c /tmp/packer-chef-client/client.rb -j /tmp/packer-chef-client/first-boot.json",
"server_url": "http://localhost:8889",
"skip_clean_node": "true",
"skip_clean_client": "true",
"run_list": ["recipe[devopracy-base::default]"]
},
{
"type": "file",
"source": "{{user `base_dir`}}/ops/devopracy-base/test/spec",
"destination": "/home/fedora/spec/"
},
{
"type": "file",
"source": "{{user `base_dir`}}/ops/devopracy-base/test/Rakefile",
"destination": "/home/fedora/Rakefile"
},
{
"type": "shell",
"inline": ["/opt/chef/embedded/bin/rake spec"]
},
{
"type": "shell",
"inline": ["sudo chmod 600 /etc/chef"]
}
]
}
Naturally there's some goofy business in here around the chmoding of the chef dir, and it's not obviously secure - I run my builds in a private subnet. I hope this helps you get off the ground with packer, which is actually an amazing piece of software, and good fun! Ping me in the comments with any questions, or hit me up on github. All the devopracy stuff is a WIP, but those files will probably mature when I get more time to work on it :P
Good Luck!
While creating along my AWS CloudFormation template I hit the 16KB limitation of user data.... and then I found out I can put the script in S3 (with all my user data) and copy that file over as part of user data and run that script... but my question is how can I take the parameters that I am passing into CloudFormation like below and pass that into the file/script/userdata that I download from S3 that I will run? So how can I pass the parameters from CloudFormation into /root/usr.sh script?
Here is my user data:
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash -x\n\n",
"yum -y install tcsh lvm2 sysstat\n\n\n",
"# AWS CLI download and Installation\n",
"curl \"https://s3.amazonaws.com/aws-cli/awscli-bundle.zip\" -o \"/usr/awscli-bundle.zip\"\n",
"unzip /usr/awscli-bundle.zip -d /usr/awscmdline/\n",
"/usr/awscmdline/awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws\n",
"/usr/local/aws/bin/aws configure set region ",
{
"Ref": "AWS::Region"
},
"\n",
"/usr/local/bin/aws s3 cp s3://test123/usr.sh /root/usr.sh \n",
"chmod 744 /root/usr.sh \n",
"/root/usr.sh"
]
]
}
}
and here are the sample parameters:
"Parameters": {
"SelectInstanceType": {
"Description": "EC2 instance type",
"Type": "String",
"Default": "r3.8xlarge",
"AllowedValues": [
"r3.large",
"r3.xlarge",
"r3.2xlarge",
"r3.4xlarge",
"r3.8xlarge",
"c4.large",
"c4.xlarge",
"c4.2xlarge",
"c4.4xlarge",
"c4.8xlarge"
],
"ConstraintDescription": "Must be a valid EC2 instance type."
},
"Keyname": {
"Description": "Keypair to use to launch the instance",
"Type": "AWS::EC2::KeyPair::KeyName"
},
"IPAddress": {
"Description": "Private IP",
"Type": "String",
"Default": "10.10.10.X"
},
There's a few ways you could do it...
Configurations in a file
You could create a file with your configurations and then read the file from your script. For an example see: Setting environment variables with user-data
Set environment variables
As part of your User Data script, before you download and call a script, set environment variables (also in the above example file).
Pass parameters when executing your script
When downloading a script from Amazon S3 and the calling it, append parameters in the same way that your script is currently inserting AWS::Region. Your script will then need to read those parameters from the command-line.
Refer to parameters like this: { "Ref" : "InstanceTypeParameter" }
See: CloudFormation Parameters documentation
I want to build the ebs ami from source ami which restricted by LDAP.
So when I build with packer, I got ssh handshake failed error.
I try to ssh login to packer builder instance, but I got password prompt because of ldap restriction and can't login.
How can I build and provision with ssh login with packer?
{
"builders": [{
"type": "amazon-ebs",
"access_key": "{key}",
"secret_key": "{key}",
"region": "{region}",
"source_ami": "{myami}",
"instance_type": "m3.medium",
"ssh_username": "ec2-user",
"ssh_timeout": "10m",
"ami_name": "ami_from_packer {{timestamp}}",
"iam_instance_profile": "packer"
}]
}