My CI pipeline will do two things
generate new lambda version and publish
Update an alias to point at that new version
This will be done via cli commands. My question is, how do I access the version number that been generated from the first command. It is returned and posted to the CLI. Can this be access easily via some nifty was command or will I have to parse it myself?
e.g.
version=$(aws lambda publish-version \
--function-name test_lambda --description "updated via cli" --region eu-west-1 \
--query Version \
--output text)
See Controlling Command Output from the AWS Command Line Interface page of AWS CLI User Guide, specifically How to Filter the Output with the --query Option and Text Output Format
This works but still curious if there is a better way.
version=$(aws lambda publish-version --function-name test_lambda --description "updated via cli" --region eu-west-1| jq '.Version')
NEW_LAMBDA_VERSION=$(aws lambda list-versions-by-function --function-name $LAMBDA_NAME_FOR_DEPLOY --no-paginate --query "max_by(Versions, &to_number(to_number(Version) || '0'))")
NEW_LAMBDA_VERSION=$(echo $NEW_LAMBDA_VERSION | jq -r .Version)
echo $NEW_LAMBDA_VERSION
In this case, I use on .gitlab-ci.yml.
Related
I have around 200 lambda functions that I need to delete. Using the console I can only delete one at a time, which would be really painful. Does anyone know a cli command to bulk delete all the functions? Thanks!
I've just found an answer using an old script to delete IAM users I had:
aws lambda list-functions --region us-east-1 | jq -r '.Functions | .[] | .FunctionName' |
while read uname1; do
echo "Deleting $uname1";
aws lambda delete-function --region us-east-1 --function-name $uname1;
done
i am trying to fetch VPC details for all region.i tried to run my script without default profile which results in error "You must specify a region. You can also configure your region by running "aws configure" ,evnthough i have my own profile configured with all required details for it.
same script works fine after configuring default profile.
Question is does AWS CLI requires default profile as mandatory ?
My script
for region in `aws ec2 describe-regions --output text| cut -f4`
do
aws ec2 --profile sam --region $region --output text --query 'Vpcs[*].{VpcId:VpcId,CidrBlock:CidrBlock}'
describe-vpcs
done
cat .aws/config
[profile sam]
output = json
region = us-east-1
If you don’t have a default profile configured, you can define the target profile with the --profile option.
aws ec2 describe-regions --profile profile-name
Another way is to set the AWS_PROFILE environment variable. This way you don’t have to explicitly add the option for every AWS CLI command.
export AWS_PROFILE=profile-name
Seems a bug in your script. I tried the below and it worked for me.
for region in `aws ec2 describe-regions --output text| cut -f4`
do
aws ec2 describe-vpcs --profile <myProfile> --region $region --output text --query 'Vpcs[*].{VpcId:VpcId,CidrBlock:CidrBlock}'
done
found the issue , need to add --profile in my first line of code as well.It works fine now.
for region in `aws ec2 describe-regions --profile sam --output text| cut -f4
I am trying to set redrive policy for SQS using the AWS CLI Command below , but seeing an error related to redrive JSON. Can you please let me know how I can fix this?
redrive_policy="{\"RedrivePolicy\":{\"deadLetterTargetArn\":\"$dlq_arn\",\"maxReceiveCount\":\"15\"}}"
AWS CLI COMMAND
aws sqs set-queue-attributes --queue-url https://queue.amazonaws.com/12345678/test-queue --attributes $redrive_policy --region=us-east-1
Error Message
Parameter validation failed: Invalid type for parameter
Attributes.RedrivePolicy, value: OrderedDict([(u'deadLetterTargetArn',
u'arn:aws:sqs:us-east-1:12345678:dlq'), (u'maxReceiveCount', u'15')]),
type: , valid types:
Have you tried just creating the JSON in a separate file and passing it as an argument to your AWS CLI command? I find it's difficult to get all of the escaping correct when passing the JSON as a parameter. So you'd basically do it as the example shows in the AWS documentation:
https://docs.aws.amazon.com/cli/latest/reference/sqs/set-queue-attributes.html#examples
So first you'd create a new file called "set-queue-attributes.json" like so:
{
"DelaySeconds": "10",
"MaximumMessageSize": "131072",
"MessageRetentionPeriod": "259200",
"ReceiveMessageWaitTimeSeconds": "20",
"RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
"VisibilityTimeout": "60"
}
Then run the command like this:
aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue --attributes file://set-queue-attributes.json --region=us-east-1
if you want to run in the same command you can use this example:
aws sqs set-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue \
--attributes '{
"RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
"MessageRetentionPeriod": "259200",
"VisibilityTimeout": "90"
}'
Three Methods to achieve this:
Note: The solutions also work on any other AWS CLI commands that require a stringified JSON
1. Using the Command-line JSON processor jq (Recommended)
This method is recommended because of many reasons:
I've found jq a handy tool to use when working with AWS CLI as the need to stringify JSON comes up quite frequently.
Install for Ubuntu: sudo apt install jq
Basic Options:
jq -R: Returns the stringified JSON
jq -c: Eliminates spacing and newline characters
The benefit is that you can write JSON as JSON and Pipe the result into the jq -R command.
Method 1:
aws sqs set-queue-attributes \
--queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes RedrivePolicy=$(echo '{"maxReceiveCount":500,"deadLetterTargetArn":"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"}' | jq -R)
OR if you have a sqs-redrive-policy.json file:
Method 2:
In sqs-redrive-policy.json,
{
"maxReceiveCount": 500,
"deadLetterTargetArn": "arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"
}
Run in Command Line:
aws sqs set-queue-attributes \
--queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes RedrivePolicy=$(cat ~/path/to/file/sqs-redrive-policy.json | jq -c | jq -R)
As you can see the second benefit is that you can isolately modify only the --redrive-policy without having to touch any of the other attributes.
Common Confusion: A confusion is the name set-queue-attributes (it would be better named put-queue-attributes). as it doesn't overwrite all attributes but only overwrites the attributes mentioned with the command. So if you already set a Policy attribute earlier during create-queue, this will not overwrite the Policy to null. In other words, this is safe to use.
2. Using a stringified JSON
This is a pain to be honest, and I avoid this.
aws sqs set-queue-attributes \
--queue-url "https://sqs.us-east-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes '{
"RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\",\"maxReceiveCount\":\"500\"}",
}'
3. Use a filePathURL to the JSON file for attributes.json NOT sqs-redrive-policy.json
This is my last preference.
Reason:
This means setting all the attributes specified in the attributes.json file again at a single go.
Doesn't escape the pain of writing stringified JSON as text.
In attributes.json,
{
"RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\", \"maxReceiveCount\":\"5\"}"
}
Run in command line:
aws sqs set-queue-attributes \
--queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes file:///home/yourusername/path/to/file/attributes.json
I just want to get back a list of function names. Ideally I want to get all functions (just their name) starting with "some-prefix*". Can I do this with the cli?
Really want this as a cli command if possible (I want to avoid python or another sdk). I see there is a --cli-input-json arg, can I use that for filtering?
You can do that. Use the --query option. The CLI would look like this:
aws lambda list-functions --region us-east-1 --query 'Functions[].FunctionName' --output text
To get the list of functions whose name begin with some-prefix:
aws lambda list-functions --region us-east-1 --query 'Functions[?starts_with(FunctionName, `some-prefix`) == `true`].FunctionName' --output text
To get the complete JSON, the CLI would be:
aws lambda list-functions --region us-east-1
Details about the query parameter can be found here.
As the answer is already given by #krishna, but I was looking for a way to print all function name without specifying a prefix. So here you can get all lambda function name in particular region my default is us-west-2.
aws lambda list-functions --query 'Functions[*].[FunctionName]'
Or as I want them out in text format and space separated to use in my bash script so here you can get in text and single line space separated.
aws lambda list-functions --query 'Functions[*].[FunctionName]' --output text | tr '\r\n' ' '
I have come here for some help to clean up all lambda functions that I have created while following an AWS developer certification tutorial. If anyone is in the same boat, I have created a script to programmatically delete all lambda functions in my AWS account (NOT for production use)
#!/bin/bash
# STOP: DON'T USE/RUN THIS SCRIPT UNLESS YOU ARE 100% SURE WHAT YOU ARE DOING
# I am a learner and created 20+ lambda functions while following a tutorial
# I wrote this script to programatically cleanup functions at the end of course
# precondition: your aws cli is configured
# get all functions from aws account
functions=(`aws lambda list-functions --query 'Functions[*].[FunctionName]' --output text`)
for i in "${functions[#]}"
do
#delete functions 1-by-1
aws lambda delete-function --function-name "$i"
echo "deleted $i"
done
Incase, someone is looking for a similar query with string present in the lambda function name as a substring, try below
aws lambda list-functions --region us-east-1 --query 'FunctionName[?contains(FunctionName, 'containing-string'] == 'true'].[FunctionName]' --output text
Note - the '[]' brackets around '.FunctionName' will provide with each fucntionName on a new line.
You can easily get the list of all lambda functions in given region using below command:
aws lambda list-functions --region us-east-1 | jq -r .Functions[].FunctionName
Download the jq (Lightweight and flexible command-line JSON processor) from here:
https://stedolan.github.io/jq/download/
ec2-describe-instances --filter "instance-state-name=stopped"
This helps me list all stopped instances with all its details.
How should I modify the command that it only gives me the names of the stopped instances?
You are using old style commands. Use AWS CLI to get what you want.
aws ec2 describe-instances --filters "Name=instance-state-name,Values=stopped" --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]'
you can use aws cli combined with other tools like jq
aws ec2 describe-instances \
--filter Name=instance-state-name,Values=stopped \
--query 'Reservations[].Instances[].{ID: InstanceId,Hostname: PublicDnsName,Name: Tags[?Key==`Name`].Value }' \
| jq '.[] | .Name[]'
this will produce output in form:
"instance2"
"instance1"