MasterUserSecret field missing in describe-db-instances API - amazon-web-services

I have created the database with ManageMasterUserPassword=True. But I couldn't get the Secret Arn from the describe-db-instances command.
aws rds describe-db-instances --db-instance-identifier database-1 --query DBInstances[*].[MasterUsername,MasterUserSecret]
[
[
"postgres",
null
]
]
I have created the database even from console,
Still I am facing the same error.
But I can clearly see the database ARN in SecretsManager and Secrets ARN in database.

Make sure you have more recent awscli, the support for this feature was added in 2.9.10:
https://raw.githubusercontent.com/aws/aws-cli/v2/CHANGELOG.rst
2.9.10
...
* api-change:``rds``: Add support for managing master user password in AWS Secrets Manager for the DBInstance and DBCluster.
...
With this, the output (only when the feature is enabled on the db instance) will contain:
$ aws rds describe-db-instances --db-instance-identifier database-1 --region=us-east-1
...
"MasterUserSecret": {
"SecretArn": "arn:aws:secretsmanager:us-east-1:776665554444:secret:rds!db-88888888-82e1-4a59-8c35-888888888888-SyXcpL",
"SecretStatus": "active",
"KmsKeyId": "arn:aws:kms:us-east-1:888888888888:key/88888888-c6c4-43da-a4a4-888888888888"
},
You can get the actual values with (for example):
$ secret_arn=$(aws rds describe-db-instances --db-instance-identifier database-1 --region=us-east-1 --query DBInstances[*].[MasterUserSecret.SecretArn] --output text)
$ aws secretsmanager get-secret-value --secret-id ${secret_arn} --region us-east-1 --query SecretString --output text
{"username":"admin","password":"SVxxxxxxxxxxxxxxxxxxxxxxxY7gwkD"}

Related

AWS CLI query and filter for Step Functions

I would like to get Step Function ARN using AWS CLI by name with wildcard strig or get Step Function ARN by Step Function name
Here is an example:
aws stepfunctions list-state-machines --region us-east-1
I got this:
{
"stateMachines": [
{
"stateMachineArn": "arn:aws:states:us-east-1:012345678912:stateMachine:firstStepFunc",
"name": "firstStepFunc",
"type": "STANDARD",
"creationDate": "2022-12-01T14:43:09.577000+01:00"
}
]
}
I tried this one:
aws stepfunctions list-state-machines --query 'stateMachines[*].stateMachineArn' --region us-east-1 --output text
And get expected result:
arn:aws:states:us-east-1:012345678912:stateMachine:firstStepFunc
But if Step Functions will be more than one, it won't work.
I need something like that, but I have no idea how to write query in proper way:
aws stepfunctions list-state-machines --query 'stateMachines[*].stateMachineArn[?stateMachineArn==`*`]' --region us-east-1
aws stepfunctions list-state-machines --query 'stateMachines[*].stateMachineArn[?name==`*`]' --region us-east-1
Thanks in advance!
You could use contains functions for this, for example:
aws stepfunctions list-state-machines --query 'stateMachines[?contains(name,`dev`)]|[*].stateMachineArn' --region us-east-1 --output text
The expression above returns the ARN of all stepfunctions which have the dev keyword in their name. If you want to get only one (the first one, for example), you can do the following:
aws stepfunctions list-state-machines --query 'stateMachines[?contains(name,`dev`)]|[0].stateMachineArn' --region us-east-1 --output text

AWS cli secrets manager add key-value

I have a secret in AWS Secrets Manager created and have many Key-Value pairs added.
What I need is, to just append one more key-value pair in it using AWS CLI. I cannot find a command for that in documentation (or maybe overlooking something)
I tried this:
aws secretsmanager put-secret-value \
--secret-id $SECRET_NAME \
--region $REGION \
--secret-string '{"NEW_KEY":"NEW_VALUE"}'
But it removes all old key-value pairs from SecretsManager and leaves the only new one in it.
AWS CLI doesn't have that capability as of now. We need to use any external library/service to achieve this.
Below is an example using jq.
*Assume if your current secret value is {"key1": "value1"}
CURR_VAL=$(aws secretsmanager get-secret-value --secret-id $SECRET_NAME | jq -r ".SecretString")
# o/p: {"key1": "value1"}
NEW_VAL=$(echo $CURR_VAL | jq -c '. += {"key1": "value2"}')
# This will add or update the value of "key1"
# o/p: {"key1":"value2"}
aws secretsmanager put-secret-value --secret-id $SECRET_NAME --secret-string $NEW_VAL
This will update the secret value to {"key1": "value2"}

AWS SSM Agent - Using the aws cli, is there a way to list all the AWS instances that are missing the SSM agent?

I need to audit a large number of AWS accounts to determine which EC2 instances are missing the SSM agent. Then I need have all those instances and their tags outputted.
Running aws ssm describe-instance-information lists all the instances that have the agent installed and are running, but it doesn't list instances that are missing the agent or systems that might be turned off.
#!/bin/bash
for instance in $(aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --output text )
do
managed=$(aws ssm describe-instance-information --filters "Key=InstanceIds,Values=$instance" --query 'InstanceInformationList[*].[AssociationStatus]' --output text)
if [[ "$managed" != "Success" ]]; then
managed="Not Managed";
fi
aws ec2 describe-instances --instance-id $instance --output text --query 'Reservations[*].Instances[*].[InstanceId, Placement.AvailabilityZone, [Tags[?Key==`Name`].Value] [0][0], [Tags[?Key==`App`].Value] [0][0], [Tags[?Key==`Product`].Value] [0][0], [Tags[?Key==`Team`].Value] [0][0] ]'
echo "$managed"
done
Save and make the script executable, then run
script.sh > file.tsv
And finally import it into excel
This will print a list of all your instances with "success" printed beneath the ones which are managed.
for instance in $(aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --output text )
do;
managed=$(aws ssm describe-instance-information --filters "Key=InstanceIds,Values=$instance" --query 'InstanceInformationList[*].[AssociationStatus]' --output text)
echo "$instance $managed";
done
To add a simple but not well-formatted set of tags, replace the echo line with
if [[ "$managed" != "Success" ]]; then
managed="Fail";
fi
echo "$instance $managed"
aws --profile GC-Staging ec2 describe-instances --instance-id $instance --query 'Reservations[*].Instances[*].[Tags[*].Value]' --output text

How to list only the instance id and related tags

I using the CLI tools and I want to list only the instance ID and related tags of an instance. The command that I am using is ec2-describe instances. I have tried certain filters but nothing is working out.
Can anybody help ??
error :
./Instance_Audit.sh: line 24: $: command not found
./Instance_Audit.sh: line 25: syntax error near unexpected token `do'
./Instance_Audit.sh: line 25: ` do echo $ID ; echo $(aws ec2 describe-instances --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done'
Using the aws cli, I can produce a few different alternatives of the tag output:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[]]'
Result: i-xxxxx [{u'Value': 'key1value', u'Key': 'key1name'}, {u'Value': 'key2value', u'Key': 'key2name'}]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].*]'
Result: i-xxxxx [['key1value', 'key1name'], ['key2value', 'key2name']]
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,Tags[].Key,Tags[].Value]'
Result: i-xxxx ['key1name', 'key2name'] ['key1value' 'key2value']
Is any of those formats what you're looking for?
The AWS CLI command
aws ec2 describe-instances --region us-west-2 --query 'Reservations[].Instances[].[InstanceId,Tags[]]' --output text
produces these results:
i-789c55c3
Name Instance1
aws:cloudformation:logical-id Instance1
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
i-77c108cc
Name Instance2
aws:cloudformation:logical-id Instance2
aws:cloudformation:stack-id arn:aws:cloudformation:us-west-2:012345678901:stack/test10-test10-foobarfoobar/a6e545a0-af52-11e4-a0be-50d5020578e0
aws:cloudformation:stack-name test10-test10-foobarfoobar
If that is not in the format you are looking for, can you provide an example of they format you are expecting?
If you want to only display the instance id and the tags associated to that instance, you can use something like :
$ for ID in $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[].InstanceId" --output text); do echo $ID ; echo $(aws ec2 describe-instances --region eu-west-1 --query "Reservations[].Instances[?InstanceId==\`$ID\`].Tags[]") done
i-60****2a
[ [ { "Value": "SB", "Key": "Name" } ] ]
i-1a****56
[ [ { "Value": "Course and Schedule on Apache 2.4", "Key": "Name" } ] ]
i-88eff3cb
[ [ { "Value": "secret.com", "Key": "Name" } ] ]
i-e7****a5
[ [ { "Value": "2014.03 + Docker", "Key": "Name" } ] ]
I could not find a way to do that with only one AWS CLI call. Should someone come up with a shorter solution, I would love to hear from you.
If you want to filter to certain tag key/value only, you can edit the aws ec2 describe-instances to add a --filter option.
For example :
aws ec2 describe-instances --region eu-west-1 --filter "Name=tag:Name,Values=SB"
--Seb
simple answer:
aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].[InstanceId,Tags[*]]'
not so simple answer .. the same principle could be applied to both instances and vpc's
# how-to get all the tags per instance-id with aws ec2 for prd env
while read -r i ; do \
aws ec2 describe-tags --filters "Name=resource-id,Values=$i" \
--profile prd; done < <(aws ec2 describe-instances \
--query 'Reservations[*].Instances[*].InstanceId' --profile prd)
# how-to get all the tags per virtual-private-cloud-id
while read -r v; do aws ec2 describe-tags \
--filters "Name=resource-id,Values=$v" ; \
done < <(aws ec2 describe-vpcs \
--query 'Vpcs[*].VpcId'|perl -ne 's/\t/\n/g;print')

aws cli: How can I query list values?

The aws cli has a --query option, which allows you to select only some information.
For an example, I am interested in getting just the Security group name from ec2 describe-instances.
If I run:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,SecurityGroups]
my output looks like:
i-xxxxxxx m1.type [{u'GroupName': 'groupName', u'GroupId': 'sg-xxxxx'}]
I can also access elements of the list using an index:
aws ec2 describe-instances --output text --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,Tags[0].Value,Tags[0].Name]
Is it possible to query tags so that instead of Tag[0] I search for a Tag where the name is specified?
As of 1.3.0, you can now query that information like this:
--query 'Reservations[*].Instances[*].Tags[?Key==`<keyname>`].Value[]'
So where you have this:
"Tags" : [
{
"Value" : "webserver01",
"Key" : "InstanceName"
},
you'd want to do this:
aws ec2 describe-instances --query 'Reservations[*].Instances[*].Tags[?Key==`InstanceName`].Value[]'
What you probably want to use is the --filters option:
aws ec2 describe-instances --output text --filters "Name=tag-key, Values=SecurityGroups, Name=tag-value, Values=Foo" --region us-east-1
You can change the filters around to "query" for the exact field you're looking for.
checkout this slideshare from the Atlanta AWS meetup group's talk on the new AWS CLI for more examples
This way works for me: (this only works in version 1.3.0 and above)
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId, Tags[?Key==`Name`].Value[*]]'
select security_groups from aws.aws_ec2_instance;
> select security_groups from aws.aws_ec2_instance limit 1;
+---------------------------------------------------------------------------------------------------------------------------------+
| security_groups |
+---------------------------------------------------------------------------------------------------------------------------------+
| [{"GroupId":"sg-xxxx","GroupName":"xxxx"},{"GroupId":"sg-xxxxxx","GroupName":"xxxx"}] |
+---------------------------------------------------------------------------------------------------------------------------------+
This will just list out the security groups for your instances.
You can also use
select security_groups from aws.aws_ec2_instance where instance_id = 'i-xxxxxx';