While I try to create a new secrets for RDS using AWS CLI, I couldn't find the way where I could associate my secrets to RDS DB on creation itself.I have gone through the AWS CLI(https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/create-secret.html) and wasn't able to find a way for its association.
Do I need to do this step manually by logging into the console or could this be automated in some way?
Could anyone please help.
Thanks.
The association is only based on the form of the secret-string. For the RDS the forms are listed here.
For instance, for mysql the form of the secret-string is as follows:
{
"engine": "mysql",
"host": "<required: instance host name/resolvable DNS name>",
"username": "<required: username>",
"password": "<required: password>",
"dbname": "<optional: database name. If not specified, defaults to None>",
"port": "<optional: TCP port number. If not specified, defaults to 3306>"
}
Thus, to create the secret for mysql using CLI:
Create file called mydb.json (example):
{
"username": "admin",
"password": "asdf435325gfdg",
"engine": "mysql",
"host": "database-2.cba4sasaubqv.us-east-1.rds.amazonaws.com",
"port": 3306,
"dbInstanceIdentifier": "database-2"
}
Execute:
aws secretsmanager create-secret --name mysql-info --secret-string file://mydb.json
The more confusing CLI part begins when you want to enable an automatic secret rotations. I will just leave a link for that (it also has CLI info) if you are interested in this as well:
Enabling Rotation for an Amazon RDS Database Secret
Related
I am writing a small python function to know the aws region of our AWS EIPs. When I use the ipinfo services such as ipinfo.io, I do get these details:
~ ➤ curl ipinfo.io/18.138.84.13/json
{
"ip": "18.138.84.13",
"hostname": "ec2-18-138-84-13.ap-southeast-1.compute.amazonaws.com",
"city": "Singapore",
"region": "Singapore",
"country": "SG",
"loc": "1.2897,103.8501",
"org": "AS16509 Amazon.com, Inc.",
"postal": "048508",
"timezone": "Asia/Singapore",
"readme": "https://ipinfo.io/missingauth"
}%
~ ➤
From this information, I get all the details except the region_name. Of course, I can trim the hostname URL to grep the region name i.e. ap-southeast-1 but that is not optimal.
Is there any way in either boto3 or aws cli where I can hit the aws API with params like region=Singapore and in response I get ap-southeast-1.
Or if there is any aws api I can hit directly by giving input as EIP and it return me details including region_name. I need to use region_name further in my script for automating the job.
In AWS, IP addresses are assigned dynamically, they are not bounded by region, The eip pool will automatically you IP address which they have available. I would suggest to ignore the region base on ip address.
I have solved it by this:
hostname.rsplit('.', 4)[1]
This hostname is in the response of ipinfo.io/18.138.84.13/json
First, everything I am doing is from the CLI. I don't have permissions to use the web interface. I am trying to make a call to an existing Aurora Postrgres database using the AWS data api. I am following the directions on this page:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html
I am stuck on the section "Storing database credentials in AWS Secrets Manager".
I know how to create a secret (aws secretsmanager create-secret --name test2 --secret-string "{"Key":"test","Value":"12345"}") but I don't know what the --secret-string should be storing the database credentials.
All the documentation says is "Use Secrets Manager to create a secret that contains credentials for the Aurora DB cluster.", but it doesn't say what format the credentials should take.
When connecting to the database from my IDE I need to include the host, port, user, password, and database name. Do I need to include all of these in the secret-string?
"{"host":"my host","port":"12345","user":"my user","password":"my password","db_name":"my db name"}"
The SecretString templates for different databases are listed in Templates for Amazon RDS Databases.
For PostgreSQL the template is (I checked by manually creating secret in AWS console):
{
"username": "postgres",
"password": "adminpass",
"engine": "postgres",
"host": "<host-url>",
"port": 5432,
"dbClusterIdentifier": "<e.g. database-1>"
}
Alternatively, you can create the secret in AWS console, and inspect its structure. Then you can re-create the structure using AWS CLI.
I am starting to play with AWS.
I have created an EC2 instance using the AWS management console.
I would like to be able to create new, similar instances using the CLI so I've been looking at get-launch-template-data (which states "Retrieves the configuration data of the specified instance. You can use this data to create a launch template.") and expected the output of that to be valid input to create-launch-template.
I've viewed the AWS CLI documentation, and looked on StackOverflow but the only related issues I've found has been these ones:
Unable to create launchtemplate using awscli and
Amazon Launch Template - Updated AMI
I've been running:
aws ec2 get-launch-template-data --instance-id "i-xxx" --query "LaunchTemplateData" > MyLaunchData
aws ec2 create-launch-template --launch-template-name xxx --launch-template-data file://MyLaunchData
The error I get is:
An error occurred (InvalidInterfaceType.Malformed) when calling the CreateLaunchTemplate operation: '%s' is not a valid value for interface type. Enter a valid value and try again.
What I think is the relevant part of MyLaunchData is:
"NetworkInterfaces": [
{
"AssociatePublicIpAddress": true,
"DeleteOnTermination": true,
"Description": "",
"DeviceIndex": 0,
"Groups": [
"sg-xxx"
],
"InterfaceType": "interface",
"Ipv6Addresses": [],
"PrivateIpAddresses": [
{
"Primary": true,
"PrivateIpAddress": "xxx"
}
],
"SubnetId": "subnet-xxx"
}
],
Can someone point me in the right direction please?
(I've obviously replaced what I think is my data with xxx for privacy)
Many thanks
The InterfaceType is not allowed:
'%s' is not a valid value for interface type
I know that you are using the output from get-launch-template-data but there is no "InterfaceType": "interface".
AWS EC2 documentation
InterfaceType
Indicates the type of network interface. To create an Elastic Fabric Adapter (EFA),
specify efa. For more information, see Elastic Fabric Adapter in the Amazon Elastic
Compute Cloud User Guide.
Required: No
Type: String
Allowed Values: efa
To add to vo1umen's answer, to get around this issue you obviously need to remove InterfaceType from the NetworkInterfaces array. One way of doing this is to use jq:
jq -c 'del(.NetworkInterfaces[0]["InterfaceType"])' <<< $TemplateData
The -c is to keep the JSON flat formatted, you may not need to do that when storing in a file.
We're trying to enable Data API for our Aurora Serverless Cluster which has been set up to be in our private subnets in our VPC. We've tried enabling it via the console (i.e. Modify > Select Data API > Apply Immediately). There weren't any errors on after submitting, but when we get back to the "Modify" page, the "Data API" checkbox is still unselected.
We have also tried enabling Data API as described on the documentation https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html , but the response still shows us that the HttpEndpointEnabled is still false.
My guess is that this could be related to Private Subnets or Security Groups. Any feedback is appreciated!
$ aws rds modify-db-cluster --db-cluster-identifier our-database --enable-http-endpoint
{
"DBCluster": {
"Capacity": 8,
"MasterUsername": "admin",
"HttpEndpointEnabled": false,
"ReadReplicaIdentifiers": [],
"VpcSecurityGroups": [
{
"Status": "active",
"VpcSecurityGroupId": "sg-0a66b09ade97573f0"
}
],
The database was created with CloudFormation if that matters
Data API is available only for serverless db. Make sure you have selected "serverless" option while creating database.
I had the same problem when I was trying to create aurora-postgresql variant that doesn't have data api option. Then I selected aurora-mysql with serverless option and aws gave me option to enable http endpoint.
I have setup a few of the amazon AWS CLI tools (EC2, Auto Scaling, MOnitoring and ELB). The tools are setup correctly and work perfectly. My environment vars are all set, the relevant ones to this Q being:
export EC2_REGION=eu-west-1
export EC2_URL=https://ec2.$EC2_REGION.amazonaws.com
export AWS_ELB_URL=https://elasticloadbalancing.$EC2_REGION.amazonaws.com
When I run ec2-describe-instance-status i-XXXXXXXX for ANY of my instances, I get:
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
I KNOW the instance ID exists, I copied it out of the AWS web console, and it is in the eu-west-1 region, and my env vars are set to this region.
For the life of me I can't figure out why it will not find my instances. Is there anything glaringly obvious that I am doing incorrectly?
UPDATE: recreating x509 cert/pk solved this... for some reason.
I had the same problem. It was because I wasn't defining a region for my commands. I assumed it would list all instances across all regions but it defaults to us-west-1 and I don't have any instances there.
To describe my machines in Ireland I use the following:
ec2-describe-instances --region eu-west-1
NB: I'm defining my AWS access key and secret elsewhere.
To avoid this problem going forward, I've now set my region via an environment variable on my linux and windows machines: EC2_URL=https://ec2.eu-west-1.amazonaws.com
so that I don't have to be explicit on the command line.
Update May 2014 You can also set the region by adding the following lines to the ~/.aws/config file in your home folder (not tested on Windows). This is my preferred method now, especially on my VM's and containers:
[default]
region = eu-west-1
For more information see the offical docs here.
Update May 2021
Since I work across so many regions now I use Implicit and ephemeral environment variables to define my region for that command and NOT have a default in my .aws/config which can be dangerous. This also makes bash scripting easier as I can define it for the whole script/utility. It's a tiny bit more typing but far safer, more flexible and transparent e.g.:
AWS_DEFAULT_REGION=eu-central-1 aws ec2 describe-instances
# or for a script/utility
AWS_DEFAULT_REGION=us-east-1 ./tagInstances.sh
In my case, I had two sets of credentials in ~/.aws/credentials. To specify the credentials tag, use
aws ec2 describe-instances --instance-id <your-instance-id> --profile <your-profile-name> --region <your-region>
Weird issue - as usual when encountering something weird in software development, one should first question the assumptions:
I KNOW the instance ID exists, I copied it out of the AWS web console,
and it is in the eu-west-1 region, and my env vars are set to this
region.
So the instance ID stems from a different environment than the one you want to use it in - I would try to derive the instance ID via the same environment instead, i.e.:
ec2-describe-instances
I venture the guess that the list won't return the instances you are expecting. This would indicate that you are either using AWS credentials that belong to another account or that these credentials do not have the required Amazon EC2 read permissions assigned via IAM policies for example.
I had a similar issue and I write here the solution for anybody who can find it helpful.
I was stucked with this error during some hours.
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
Finally I found what was happening: I had my instance in a different region than the default region (US East (Northern Virginia)) and I had to update this information. By default the commands look only for instances in the default region!
It is explained in the docs, section (Optional): Set the region http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SettingUp_CommandLine.html
It's very simple problem. If you are getting this error
Client.InvalidInstanceID.NotFound: The instance ID 'i-XXXXXXXX' does not exist
Then follow the steps.
Check in which country region your instance is image here
Now enter root#Indian:~# aws configure
Enter
AWS Access Key ID [****************D7M2]:
AWS Secret Access Key [****************2h3r]:
Default region name [us-east-1]:
When asked for default region, Change the region to which the instances is residing. Eg: us-east-2. Then press Enter.
Acutally these are the list of available Region names "RegionNames"
"Regions": [
{
"RegionName": "ap-south-1",
"Endpoint": "ec2.ap-south-1.amazonaws.com"
},
{
"RegionName": "eu-west-2",
"Endpoint": "ec2.eu-west-2.amazonaws.com"
},
{
"RegionName": "eu-west-1",
"Endpoint": "ec2.eu-west-1.amazonaws.com"
},
{
"RegionName": "ap-northeast-2",
"Endpoint": "ec2.ap-northeast-2.amazonaws.com"
},
{
"RegionName": "ap-northeast-1",
"Endpoint": "ec2.ap-northeast-1.amazonaws.com"
},
{
"RegionName": "sa-east-1",
"Endpoint": "ec2.sa-east-1.amazonaws.com"
},
{
"RegionName": "ca-central-1",
"Endpoint": "ec2.ca-central-1.amazonaws.com"
},
{
"RegionName": "ap-southeast-1",
"Endpoint": "ec2.ap-southeast-1.amazonaws.com"
},
{
"RegionName": "ap-southeast-2",
"Endpoint": "ec2.ap-southeast-2.amazonaws.com"
},
{
"RegionName": "eu-central-1",
"Endpoint": "ec2.eu-central-1.amazonaws.com"
},
{
"RegionName": "us-east-1",
"Endpoint": "ec2.us-east-1.amazonaws.com"
},
{
"RegionName": "us-east-2",
"Endpoint": "ec2.us-east-2.amazonaws.com"
},
{
"RegionName": "us-west-1",
"Endpoint": "ec2.us-west-1.amazonaws.com"
},
{
"RegionName": "us-west-2",
"Endpoint": "ec2.us-west-2.amazonaws.com"
}
]
}
Default output format [None]:
Leave the output format blank and press Enter. Now You are Done
Now in the console just type
root#Indian-3543:~# aws ec2 describe-instance --instance-id i-06343434322t
MAKE HAPPY BE HAPPY
I got this fixed by changing EC2_URL from 'https://ec2.ap-southeast-1.amazonaws.com' to 'ec2.ap-southeast-1.amazonaws.com'