when I do this
"/users/akshay/bin/aws --version"
I get
"aws-cli/1.15.57 Python/2.7.10 Darwin/17.6.0 botocore/1.10.56"
why should I always use the path
"/users/akshay/bin/aws"
to refer to the aws cli? And why can't I just do
aws --version?
Related
I am trying to setup some build and deployment servers based on EC2 instances to deploy software to AWS via CloudFormation.
The current setup uses the AWS CLI to deploy CloudFormation templates, and authentication is handled using a credentials profile where the ~/.aws/config file has a profile with:
[profile x]
role_arn = x
credential_source = Ec2InstanceMetadata
region = x
The setup using the AWS CLI appears to be working fine, and can deploy CloudFormation templates, upload files to S3 etc.
I wanted to automate this further and use a configuration-based approach to allow for more flexibility in our deployments. To achieve this, I have written some Python code to parse a config file and use the Boto3 library (which the AWS CLI also uses) to replicate the functionality. However when I am trying to do similar things in Boto3 (like deploy CloudFormation and upload files to S3), I get the following error: Connection to sts.amazonaws.com timed out. Unfortunately I can't provide the full stack trace since it's on a separate network. I am running Python 3.7 and boto3-1.21-13, botocore-1.24.13.
I assume it might be because I need to setup a VPC endpoint for STS? However, I can't work out why and how the AWS CLI works fine, but Boto3 doesn't. Especially since AWS CLI uses Boto3 under the hood.
In addition, I have confirmed that I can retrieve instance metadata using curl from the EC2 instances.
To reproduce the error, this command fails for me:
python -c "import boto3;print(boto3.Session(profile_name='x').client('s3').list_objects('bucket')"
However this AWS cli command works:
aws --profile x s3 ls bucket
I guess I don't understand why the AWS CLI command works, when the boto3 command fails. Why does boto3 needs to call the sts.amazonaws.com endpoint, when the AWS CLI seemingly doesn't? What am I missing?
The aws cli and boto3 both use botocore, which is only a minor detail. Nevertheless, both the cli and boto3, when run in the same environment with the same access to the credentials, should indeed be able to reach the same endpoint.
This:
aws sts get-caller-identity --profile x
and:
python -c "import boto3;print(boto3.Session(profile_name='x').client('sts').get_caller_identity())"
are equivalent and should make the same api calls to the same endpoint.
As an aside, I find it is often best not to have your code concerned with session handling at all. It seems most simple to me for the code to expect the environment to handle that. So just export AWS_PROFILE and run the code. This prevents other user of the script from having to have the same profile and name it the same.
Yeah so it turns out I just needed to set/export AWS_STS_REGIONAL_ENDPOINTS='regional'.
After many hours of trawling the botocore and awscli source and logs, I found out that botocore sets it by default to 'legacy'.
Where as in v2 of the AWS CLI, they set it to 'regional'.
I am using the following command to create AWS Aurora Serverless instance
aws rds create-db-cluster --db-cluster-identifier test-cluster --database-name testdb --master-username test --master-user-password testtest --engine aurora --engine-mode serverless --region us-east-1
but I am getting the following error.
Unknown options: --engine-mode, serverless
Above command works great on my AWS account but its not working on my clients account. (I just have programmatic access to that account). I have double check the permissions and I have the similar permissions as of my own account.
Summary: AWS command to create serverless aurora cluster is working on one account but not on another account with similar permissions.
Account 1:
Account2:
The error message states that it does not know about the engine-mode argument. This is a clear indication that your AWS CLI version is out dated. Serverless was added as part of a recent (late 2018) release, so you need to update your client's AWS CLI to recognize these inputs.
I have figured it out. I was using awscli version 1.14 on my server and 1.16 on my laptop. I updated the awscli and now its working fine.
sudo pip install --upgrade awscli
I am following an AWS lambda tutorial https://docs.aws.amazon.com/lambda/latest/dg/vpc-rds-create-rds-mysql.html and the first step is to create an RDS instance via the CLI but I am getting the following error...
Invalid endpoint: https://rds.us-east-2.amazonaws.com
I've tried specifying a couple other parameters such as --availability-zone, -vpc-security-group-ids, etc but it still is saying the endpoint doesn't exist?
Make sure you have the latest version of the AWS CLI tool installed. It sounds like you have an older version before us-east-2 was added. You can check the version you have installed by running aws --version.
I was trying to use AWS secrets manager service for storing secret keys. I could store the key in the secrets manager, but while retrieving the key using the code they provide I'm getting this error.
UnknownServiceError: Unknown service: 'secretsmanager'. Valid service names are: acm, alexaforbusiness,...
I tried a search on the AWS CLI and the service is not on the list also.
aws: error: argument command: Invalid choice, valid choices are:
acm | alexaforbusiness
apigateway | application-autoscaling
appstream | appsync
.
.
.
Do I need some kind of aws cli update? May be because of the CLI version that I'm using.
Then, how can I update that. I tried
sudo apt-get install -y awscli
an it says
awscli is already the newest version (1.11.13-1ubuntu1~16.04.0).
secretsmanager is only available from aws cli version 1.15.8 and upwards
aws --version
aws-cli/1.15.10 Python/2.7.10 Darwin/17.5.0 botocore/1.10.10
You may need to manually install/update aws cli
I am working in the US-East-1 (N.Virginia) and have even configured the Default Region Name to us-east-1 using the command aws configure.
But I am not able to access Batch using CLI. Batch is not even listed as one of the Available Services in aws help.
Any ideas how to enable Batch in AWS Cli? I have administrative access in IAM console so permissions don't seem to be the issue.
The batch service is relatively new, so its commands only exist in fairly new versions of the aws CLI.
Commands for batch in the latest cli documentation: http://docs.aws.amazon.com/cli/latest/reference/batch/index.html?highlight=batch
If you are running Windows, simply download the updated installer. https://aws.amazon.com/cli/
If you are using OSX or Linux use pip. pip install --upgrade awscli