DynamoDBLocal is rejecting my credentials in spite of the documentation indicating that valid credentials are unnecessary:
The AWS SDKs for DynamoDB require that your application configuration specify an access key value and an AWS region value...these do not have to be valid AWS values in order to run locally.
In this case, I've set up my credentials ~/.aws/credentials as:
[default]
aws_access_key_id = BogusAwsAccessKeyId
aws_secret_access_key = BogusAwsSecretAccessKey
run DynamoDBLocal using:
java -Djava.library.path=./DynamoDBLoc_lib -jar DynamoDBLocal.jar
checked that it's working by hitting http://localhost:8000/shell/
then run my test Java app:
DefaultAWSCredentialsProviderChain credentialProvider = new DefaultAWSCredentialsProviderChain();
AWSCredentials awsCredentials = credentialProvider.getCredentials();
log.info("creds \"{}\", \"{}\"", awsCredentials.getAWSAccessKeyId(), awsCredentials.getAWSSecretKey());
AmazonDynamoDBClient client = new AmazonDynamoDBClient(credentialProvider);
client.withEndpoint("http://localhost:8000");
client.withRegion(Regions.US_WEST_2);
dynamoDB = new DynamoDB(client);
try {
TableCollection<ListTablesResult> tables = dynamoDB.listTables();
while (tables.iterator().hasNext()) { // <-- exception thrown here
log.info(tables.iterator().next().getTableName());
}
} catch (Exception e) {
log.error("", e);
}
which results in this output:
creds "BogusAwsAccessKeyId", "BogusAwsSecretAccessKey"
com.amazonaws.AmazonServiceException: The security token included in the request is invalid. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: UnrecognizedClientException
Any thoughts on why it is concerned with the validity of the credentials?
In your code, you are calling withRegion() after calling setEndpoint(). The call to withRegion() is setting the endpoint to DynamoDB's us-west-2 endpoint and that's why your authentication is failing (because it's actually going to the DynamoDB us-west-2 region). Remove the withRegion() line.
Related
I am new to AWS. I am trying to connect to AmazonS3 using IAM role. I am not using secret keys here. Changes have been done at AWS side regarding IAM.
I have to create AmazonS3 client object. I tried to create object by following code:
public AmazonS3 AmazonS3Provider() {
AmazonS3ClientBuilder.defaultClient();
}
But it is returning :
"status":500,"exception":"AmazonS3Exception","message":"Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied;
Next, I tried to connect AWS s3 using AWSSecretsManagerClient using the following code:
public AmazonS3 AmazonS3Provider() {
return (AmazonS3) AWSSecretsManagerClientBuilder.defaultClient();
}
But it was throwing error that AWSSecretsManagerClient object cannot be cast to AmazonS3.
Also, I tried to build client object using the following code:
AWSCredentials credentials;
try {
credentials = new ProfileCredentialsProvider().getCredentials();
} catch (Exception e) {
throw new AmazonClientException(
"Cannot load the credentials from the credential profiles file.", e);
}
return AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withRegion(s3EndPoint)
.build();
But is throwing AmazonClientException.
Can someone guide me how to build client object using IAM role.
IAM access key and secret key (AWS credential) path locations are stored in environment variable. I was able to create client object using WebIdentityTokenCredentialsProvider with the following code:
AmazonS3 client = AmazonS3ClientBuilder.standard()
.withCredentials(WebIdentityTokenCredentialsProvider.create())
.withRegion("Region-name")
.build();
I'm trying to use terraform to initiate connections with AWS to create infra.
If I run up aws configure sso, i can log in default to eu-west-2 and move around the estate
I then use terraform apply, with the aws part as follows:
provider "aws" {
region = "eu-west-2"
shared_credentials_file = "~/.aws/credentials"
profile = "450694575897_ProdPS-SuperUsers"
}
Terraform reports: Error: error using credentials to get account ID: error calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.
│ status code: 403, request id: 5b8be53d-253d-4c48-8568-ad78be14115f
The following vars are set:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
If I run
aws sts get-session-token --region=us-west-2
I get
An error occurred (InvalidClientTokenId) when calling the GetSessionToken operation: The security token included in the request is invalid.
I was having the same problem when i tried to deploy through terraform cloud.
You might be using an old key that is either deleted or inactive, to be sure:
1- Try to go to the security credentials on your account page: Click on your name in the top right corner -> My security credentials.
2- Check if the key you set in your credentials is deleted or still exists.
2.2- if it's deleted create a new key and use it.
3- If your key is still there, check if it is active.
I solved the issue doing the following:
$: aws configure
enter the access key:
enter the secret key:
select default region:
select default format[none/json]:
In your main.tf file add the profile shown as below
provider "aws" {
region = "eu-west-2"
profile="xxxuuzzz"
}
I have written some code to retrieve my secrets from the AWS Secrets Manager to be used for further processing of other components. In my development environment I configured my credentials using AWS CLI. Once the code was compiled I am able to run it from VS and also from the exe that is generated.
Here is the code to connect to the secrets manager and retrieve the secrets
public static string Get(string secretName)
{
var config = new AmazonSecretsManagerConfig { RegionEndpoint = RegionEndpoint.USWest2 };
IAmazonSecretsManager client = new AmazonSecretsManagerClient(config);
var request = new GetSecretValueRequest
{
SecretId = secretName
};
GetSecretValueResponse response = null;
try
{
response = Task.Run(async () => await client.GetSecretValueAsync(request)).Result;
}
catch (ResourceNotFoundException)
{
Console.WriteLine("The requested secret " + secretName + " was not found");
}
catch (InvalidRequestException e)
{
Console.WriteLine("The request was invalid due to: " + e.Message);
}
catch (InvalidParameterException e)
{
Console.WriteLine("The request had invalid params: " + e.Message);
}
return response?.SecretString;
}
This code pulls credentials from the AWS CLI but when I try to run this code in another PC, it gives an IAM security error as expected, because it cannot figure out what the credentials are to connect to the secret manager.
What would be the best approach to deploy such a configuration in production? Would I need to install and configure AWS CLI in each and every deployment?
If you're deploying the code in AWS you can use IAM Role, with a policy that allows getting secrets from Secret Manager, and attach this role in EC2 or ECS, etc
If you are in a corporate environment with existing authentication infrastructure in place, you probably want to look at identity federation solutions to use with AWS.
I have a kubernates pod which is trying to access the aws kms service using the aws java sdk to decrypt the password and I have the valid I am role attached to the pod but the request is failing with the below error,
{
"message": "Service Unavailable: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)),
SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), com.amazonaws.auth.profile.ProfileCredentialsProvider#219aa2a6: profile file cannot be null,
com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper#399fc8ea: Internal Server Error (Service: null; Status Code: 500; Error Code: null; Request ID: null)]"
}
below is the code part which supposes to invoke the kms
private final boolean kmsEnabled;
private final AWSKMS kmsClient;
public KmsKeyManager(#Value("${kms.enabled}") final boolean kmsEnabled,
#Value("${kms.endpoint}") final String kmwEndPoint,
#Value("${aws.region}") final String awsRegion) {
AwsClientBuilder.EndpointConfiguration endpointConfig = new AwsClientBuilder.EndpointConfiguration(kmwEndPoint, awsRegion);
kmsClient = AWSKMSClientBuilder.standard()
.withCredentials(new DefaultAWSCredentialsProviderChain())
.withEndpointConfiguration(endpointConfig)
.build();
this.kmsEnabled = kmsEnabled;
}
You may have to create a VPC endpoint since EKS pods may not have an external IP and thus will not be able to connect to KMS
The error message here contains the hint what is missing:
Unable to load AWS credentials from any provider in the chain …
See the docs how to provide the necessary credentials.
I am trying to use the AWS Java SDK to interact with S3 and Lambda.
Because our accounts are required to use multi-factor authentication, my ~/.aws/credentials file has two profiles, [default] and [auth]. The default has an aws_access_key_id and aws_secret_access_key I generated through the IAM management console - but this profile can't be used to do much of anything. The second profile, auth, is generated by a script that uses the MFA code to create a set of credentials that will last for 12 hours. This one has an aws_security_token in addition to the aws_access_key_id and aws_secret_access_key. (Potentially of note, this aws_security_token is NOT in my IAM management console -- but the profile works fine with the AWS CLI, and my coworkers confirm that this is true for them as well.)
I have tried various permutations of creating an S3 or Lambda client:
//private static AWSLambdaClient lambda = new AWSLambdaClient();
private static AWSLambda lambda = AWSLambdaClientBuilder.standard().withCredentials(new ProfileCredentialsProvider("auth")).withRegion(Regions.US_WEST_2).build();
//private static AmazonS3 s3client = new AmazonS3Client();
private static AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider("auth"));
In all cases it fails when I try to do anything with these clients. (Either s3client.listBuckets() or lambda.invoke(request).)
With the default profile, it fails in expected ways -- saying that the user doesn't have permissions for that operation.
With the "auth" profile, it fails in different ways for S3 and Lambda. S3 fails with "The AWS Access Key Id you provided does not exist in our records." Lambda fails with "The security token included in the request is invalid."
Full error messages below:
S3 with default profile:
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1A8FE9A40C6CF4FC), S3 Extended Request ID: M+IaUw223nj7ClIv3oB4sp7pOw2af8VMsrhLl/18oQB0rLMRFNjf8LFyXKj+AQNanGoUVgMpPNM=
S3 with auth profile:
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: E388FFE85FC71492), S3 Extended Request ID: f3RU7uEXK+2tt98fQYCkHMbsa7odsnsgTIdRJvx2xbvKUmDtT2xXWZ2Eqor1I1ODqEZ0xDe3Quc=
Lambda with default profile:
Caused by: com.amazonaws.services.lambda.model.AWSLambdaException: User: arn:aws:iam::181136347741:user/bkc is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:us-east-1:181136347741:function:sensor (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: e4670c43-61b6-11e7-8eeb-db8360af9fe6)
Lambda with auth profile:
Caused by: com.amazonaws.services.lambda.model.AWSLambdaException: The security token included in the request is invalid. (Service: AWSLambda; Status Code: 403; Error Code: UnrecognizedClientException; Request ID: 00a397f0-61b7-11e7-beb6-df232368bb9e)
So what am I doing wrong here? Is there something wrong with the way I'm creating the ProfileCredentialsProvider and passing that to the AWS clients? Is there something wrong with my auth profile, even though it works from the CLI? (Do we need to fix something in our script that creates the auth profile perhaps?)
For reference, here's the interesting parts of the script that generates the auth profile:
def get_mfa_tokens(serial, code):
'''Generate MFA access tokens using the given device serial and MFA code.'''
sts = boto3.client('sts')
return sts.get_session_token(
SerialNumber=serial,
TokenCode=code)
def add_auth_tokens(config, tokens):
'''Modify the given config object, adding certain token fields to it.'''
#auth section will be there most of the time, so LBYL.
if not config.has_section('auth'):
config.add_section('auth')
config.set('auth', '# expires {}'.format(tokens['Credentials']['Expiration']))
config.set('auth', 'aws_access_key_id', tokens['Credentials']['AccessKeyId'])
config.set('auth', 'aws_secret_access_key', tokens['Credentials']['SecretAccessKey'])
config.set('auth', 'aws_security_token', tokens['Credentials']['SessionToken'])