I currently have an ec2 instance named and tagged "test-dev" and inside it at I have a test_batch.sh and hello_world.sh that I already prepared in the home dir of my user.
test_batch.sh
#!/bin/bash
echo "run hello world"
./hello_world.sh
hello_world.sh
#!/bin/bash
echo "hello world"
According to the documentation I can run the test_batch.sh via the following command.
aws ssm send-command \
--document-name "AWS-RunShellScript" \
--targets Key=tag:Name,Values=test-dev \
--parameters '{"commands":["cd ~/","bash test_batch.sh"]}' \
--output text
it says that it can't find the referenced file hello_world.sh so I assume the test_batch.sh is called properly.
Related
I am trying to set the current context of the cluster using shell script.
#!/usr/bin/env bash
#Set Parameters
echo ${cluster_arn}
sh "aws eks update-kubeconfig --name abc-eks-cluster --role-arn ${cluster_arn} --alias abc-eks-cluster"
export k8s_host="$(kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " ")"
However, the above command gives error:
sh: 0: Can't open aws eks update-kubeconfig --name abc-eks-cluster --role-arn arn:aws:iam::4399999999873:role/abc-eks-cluster-cluster-admin --alias abc-eks-cluster
Can someone suggest me how do what is the issue and how can I set the current context because the k8_host command yiedls another error that context is not set.
I have no idea why you are even involving sh as a subshell, when you're already in a shell script, but
Can someone suggest me ... what is the issue
You have provided the entire command to sh but failed to use the -c that informs it that the first argument is an inline shell snippet; otherwise, sh expects that the first non-option argument is a file which is why it says it cannot open a file with that horrifically long name
There are two outcomes to get out of this mess: use -c or stop trying to use a subshell
sh -c "aws eks update-kubeconfig --name abc-eks-cluster --role-arn ${cluster_arn} --alias abc-eks-cluster"
or:
aws eks update-kubeconfig --name abc-eks-cluster --role-arn ${cluster_arn} --alias abc-eks-cluster
I am writing a cloudformation script which is supposed to pick the stack name, template body and parameter file from a .txt file for deployment. I would not want the yaml and json file to be edited during a new deployment instead the .txt file should be edited
The code is below
aws cloudformation create-stack --stack-name $(<stack_name.txt) --template-body
file://$(<stack_template_file_name.txt) --parameters file://$(<stack_parameter_file_name.txt) capabilities "CAPABILITY_IAM" "CAPABILITY_NAMED_IAM" --region=us-west-2
note: stack_name.txt contains the name to be used the stack, stack_template_file_name.txt contains the name of the template.yml file, stack_parameter_file_name.txt contains the name of the parameter.json file
when I type the command directly in the cli, the stack is deployed but when I copy it into create.sh and run ./create.sh I get the error below
`' doesn't match a supported format.`
How can I fix this?
I just wanted to comment I found the same error from a different issue in the aws CLI.
' doesn't match a supported format.
When running:
aws s3 cp file.txt s3://bucket-name/
The issue was a malformed configuration in the
~/.aws config folder.
Re-running the
~/.aws configure command didn't seem to fix the issue
however by inspecting the files in the
~/.aws folder I was able to remove the random characters that were causing the issue.
In my case I was using Ubuntu Windows Linux Subsystem.
I was getting shell variables from properties file.
For e.g. My properties file abc.properties.
#VSRX
region=us-west-1
key_name=cft111
key_location=/var/cft111.pem
My Shell script abc.sh
#!/bin/bash
action=$1
source ./abc.properties
if [[ $action == "create" ]]; then
aws cloudformation deploy \
--template-file abc.yml \
--stack-name ${parent_stack_name} \
--capabilities CAPABILITY_NAMED_IAM \
--region ${region} \
--parameter-overrides KeyName=${key_name}
fi
I faced the similar issue and i tried echoing the command as well and i saw region parameter itself were invisible in echoed command.
To my resolution : it was only formatting issue and i fixed it with below two commands.
sed -i 's/\r$//' abc.sh
sed -i 's/\r$//' abc.properties
I found out that it had to do with the EC2 environment, the exact command worked in a Windows and Ubuntu system.
My CI pipeline will do two things
generate new lambda version and publish
Update an alias to point at that new version
This will be done via cli commands. My question is, how do I access the version number that been generated from the first command. It is returned and posted to the CLI. Can this be access easily via some nifty was command or will I have to parse it myself?
e.g.
version=$(aws lambda publish-version \
--function-name test_lambda --description "updated via cli" --region eu-west-1 \
--query Version \
--output text)
See Controlling Command Output from the AWS Command Line Interface page of AWS CLI User Guide, specifically How to Filter the Output with the --query Option and Text Output Format
This works but still curious if there is a better way.
version=$(aws lambda publish-version --function-name test_lambda --description "updated via cli" --region eu-west-1| jq '.Version')
NEW_LAMBDA_VERSION=$(aws lambda list-versions-by-function --function-name $LAMBDA_NAME_FOR_DEPLOY --no-paginate --query "max_by(Versions, &to_number(to_number(Version) || '0'))")
NEW_LAMBDA_VERSION=$(echo $NEW_LAMBDA_VERSION | jq -r .Version)
echo $NEW_LAMBDA_VERSION
In this case, I use on .gitlab-ci.yml.
I have a AWS EC2 instance That i need to manually access to the AWS console and make a daily image of the machine(AMI)
How i can make a daily AMI backup of the machine and delete old version (old then 7 days)?
Thank you!
Anything that you can do through the web console you can also do through the CLI.
In this particular case, I suspect a combination of aws ec2 create-image, aws ec2 describe-images, and aws ec2 deregister-image would let you do what you want.
AWS lambda would be a right solution to automate the backup of your ami and clean up. You can schedule the lambda function (basically a python code) to run periodically. This way you don't need to have your ec2 running all the time. An example here http://powerupcloud.azurewebsites.net/2016/10/15/serverless-automate-ami-creation-and-deletion-using-aws-lambda/
Below is a shell script I use that runs daily via cron. You can set the value of a variable prevday1 to set how long you want to keep your images. In your case you want 7 days to it would be
prevday1=$(date --date="7 days ago" +%Y-%m-%d)
Here is the full script:
#!/bin/bash
# prior to using this script you will need to install the aws cli on the local machine
# https://docs.aws.amazon.com/AmazonS3/latest/dev/setup-aws-cli.html
# AWS CLI - will need to configure this
# sudo apt-get -y install awscli
# example of current config - july 10, 2020
#aws configure
#aws configure set key ARIAW5YUMJT7PO2N7L *fake - user your own*
#aws configure secret X2If+xa/rFITQVMrgdQVpFLx1c7fwP604QkH/x *fake - user your own*
#aws configure set region us-east-2
#aws configure set format json
# backup EC2 instances nightly 4:30 am GMT
# 30 4 * * * . $HOME/.profile; /var/www/devopstools/shell-scripts/file_backup_scripts/ec2_backup.sh
script_dir="$(dirname "$0")"
# If you want live notifications about backups
#SLACK_API_URL="https://hooks.slack.com/services/T6VQ93KM/BT8REK5/hFYEDUCoO1Bw72wxxFSj7oY"
source "$script_dir/includes/helpers.sh"
prevday1=$(date --date="2 days ago" +%Y-%m-%d)
prevday2=$(date --date="3 days ago" +%Y-%m-%d)
today=$(date +"%Y-%m-%d")
instances=()
# add as many instances to backup as needed
instances+=("autobackup_impressto|i-0ed78a1f3583e1859")
for ((i = 0; i < ${#instances[#]}; i++)); do
instance=${instances[$i]}
instanceName="$(cut -d'|' -f1 <<<"$instance")"
instanceId="$(cut -d'|' -f2 <<<"$instance")"
prevImageName1="${instanceName}_${prevday1}_$instanceId"
prevImageName2="${instanceName}_${prevday2}_$instanceId"
newImageName="${instanceName}_${today}_$instanceId"
consoleout --green "Begin backing $instanceName [$instanceId]"
aws ec2 create-image \
--instance-id $instanceId \
--name "$newImageName" \
--description "$instanceName" \
--no-reboot
if [ $? -eq 0 ]; then
echo "$newImageName created."
echo ""
if [ ! -z "${SLACK_API_URL}" ]; then
curl -X POST -H 'Content-type: application/json' --data '{"text":":rotating_light: Backing up *'$newImageName'* to AMI. :rotating_light:"}' ${SLACK_API_URL} fi
echo -e "\e[92mBacking up ${newImageName} to AMI."
else
echo "$newImageName not created."
echo ""
fi
imageId=$(aws ec2 describe-images --filters "Name=name,Values=${prevImageName1}" --query 'Images[*].[ImageId]' --output text)
if [ ! -z "${imageId}" ]; then
echo "Deregistering ${prevImageName1} [${imageId}]"
echo ""
echo "aws ec2 deregister-image --image-id ${imageId}"
aws ec2 deregister-image --image-id ${imageId}
fi
imageId=$(aws ec2 describe-images --filters "Name=name,Values=${prevImageName2}" --query 'Images[*].[ImageId]' --output text)
if [ ! -z "${imageId}" ]; then
echo "Deregistering ${prevImageName2} [${imageId}]"
echo ""
echo "aws ec2 deregister-image --image-id ${imageId}"
aws ec2 deregister-image --image-id ${imagesId}
fi
consoleout --green "Completed backing $instanceName"
done
Also available here - https://impressto.net/automatic-aws-ec2-backups/
You can use https://github.com/alestic/ec2-consistent-snapshot and run it in a cron job. It supports various filesystems and has support for ensuring database snapshots are consistent. If you don't have a database in your instance, it will still ensure consistent snapshots by freezing the filesystem.
The command that I used to create a new (ubuntu) instance is this:
aws ec2 run-instances --image-id ami-XXXXXXXX --count 1 --instance-type
t2.micro --key-name abcdef --query 'Instances[0].InstanceId' --user-data file:///Users/<username>/<somedir>/UserData.sh --subnet-id subnet-XXXXXX --associate-public-ip-address
The UserData.sh contains this 3 lines including the newline:
#!/bin/bash
mkdir ~/latest
However, I do not see the "latest" dir when the system gets created and comes up. What am I doing wrong?
Disclaimer: I have already checked this: how to pass in the user-data when launching AWS instances using CLI. as well as other forums.
Also, is there any way to know if there is any warning messages etc which can give me some insights into what I am doing wrong? Is there any permission necessary at a AWS level ?
mkdir ~/latest
Whose home directory? Specify absolute path like mkdir /home/myuser/latest. Don't use C-Shell style notation.