I have been tasked with converting a library from using the AWS Java SDK to the AWS C++ SDK. The library creates an Aead instance and allows an app that uses it to call Encrypt() and Decrypt() on strings. The library connects to AWS using an Access Key ID, Secret Access Key, and Temporary Session Token, read from the shared credentials file ~/.aws/credentials
For the Java library, the temp token is automatically updated once per minute, as described at https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
Refreshing IMDS credentials:
The AWS SDK for Java supports opt-in refreshing IMDS credentials in the background every 1 minute, regardless of the credential expiration time. This allows you to refresh credentials more frequently and reduces the chance that not reaching IMDS impacts the perceived AWS availability.
But there doesn't seem to be a similar provision in the C++ SDK. So I wondered if there is some AWS CLI command I could use to do the same, as a background task, so the C++ code (maybe slightly tweaked) could pick up a new temp token at regular intervals and not fail after the temp token has expired.
The only alternative, which seems a bit cumbersome, would be to run a supplementary Java app, using the AWS Java SDK, whose sole purpose was to update temp credentials for use by the C++ library.
Related
I am trying to access an Amazon S3 bucket programmatically through Java libraries. (to do basic cloud management from a third-party application). As a first step, I tried to print whether a bucket exists or not(3rd line)
AWSCredentials credentials=new BasicAWSCredentials("my-Access- Key","My- Secret-Key");
AmazonS3 s3client = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion(Regions.AP_EAST_1).build();
String bucketExists=String.valueOf(s3client.doesBucketExistV2("newBucketName"));
When I run this line of code, I am getting an exception saying that
com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID:RequestId...)
I don't want to maintain a credentials file in the .aws folder for
the following reason:
I am trying to variablilize the access credentials
based on the logged-in user from a secure LDAP system, so I can
confirm the feasibility only when I test it with hard-coded
credentials.
I have checked that the issue is not one of the below
I have created an IAM user with a valid Access ID and Secret Key in
the AWS console and have enabled the user for programmatic access.
I have also given applied the AmazonS3FullAccess policy for IAM user
The key is in Active state(have checked it through the console)
I have added the dependency for the AWS SDK to gradle (implementation group: 'com.amazonaws', name: 'aws-java-sdk-s3', version: '1.12.15')
Please let me know what the issue might be. My apologies if it is an amateur issue.
As indicated in the question comments, your code looks fine and it should work properly.
The most likely reason of the problem is that AWS is picking up other credentials from somewhere else. Please, try removing other credentials like the ones stored in the home directory in order to be sure that the SDK is using the right credentials when contacting S3.
In addition, please, verify that you are providing the right region according to your S3 bucket as well.
You are using the older V1 API. The Service Client name for V1 is AmazonS3. Likewise, the V2 service client name is S3Client. Amazon strongly recommends moving to V2:
The AWS SDK for Java 2.x is a major rewrite of the version 1.x code base. It’s built on top of Java 8+ and adds several frequently requested features. These include support for non-blocking I/O and the ability to plug in a different HTTP implementation at run time.
Try following this step by step set of instructions, which is based on V2.
Get started with the AWS SDK for Java 2.x
WHen working with V2, you can put your creds in a file located in a .aws folder named credentials, as explained in this document. Once you follow all steps in this document, you can programmatically access an Amazon S3 bucket.
We have a java app that runs within an EC2. This uses AWS SDK to interact with AWS services. The app uses the default credentials provider which in turn uses the temporary tokens from the EC2.
The problem is that this temporary token expires after a while and all subsequent AWS SDK calls fail. How can we refresh this temporary credentials so that the app can use the new valid credentials?
InstanceProfileCredentialsProvider retrieves credentials as needed; a client using it should never see a credentials timeout exception.
This implies that either (1) you are explicitly setting instance profile credentials on the client that is timing out, or (2) there is something else in the provider chain that is providing limited-time credentials.
Or, I suppose, you might be using an extremely old version of the SDK. I don't know at what point they added automatic refresh; according to git blame, the fetcher variable was last changed 13 months ago. But I've been using instance credentials with long-running processes for much longer than that.
This article may help you.
If you really need to get a new credentials, the following example shows pseudocode for how to use temporary security credentials if you're using an AWS SDK:
assumeRoleResult = AssumeRole(role-arn);
tempCredentials = new SessionAWSCredentials(
assumeRoleResult.AccessKeyId,
assumeRoleResult.SecretAccessKey,
assumeRoleResult.SessionToken);
s3Request = CreateAmazonS3Client(tempCredentials);
Read further info about temporary credentials in this docs.
I am currently curious about if there any risks when using S3 GUI client tool like S3Browser or Cyberduck to explore our S3 file with AWS secret keys.
https://s3browser.com/
https://cyberduck.io/
Can those aws keys will be stolen by the tool?
Both of these services store the keys locally on your host machine.
From experience I know that S3 Browser allows your credentials to be encrypted locally on the disk using a password of your choosing as the decryption key.
Obviously goes without saying but when using any service ensure the following to increase your security:
Regularly rotate your credentials.
Your IAM policies should locked down to the minimum settings you require.
If you're concerned about API calls enabled S3 API calls in CloudTrail
If its a 1 time key, immediately remove it afterwards from both the software and your AWS account.
S3 browser also supports the assuming of a role which allows it to generate temporary credentials.
Nothing is 100% risk free. As a developer who has built one of those s3 browser, this is what I'm doing to mitigate the risk:
the entire code is 100% open source (https://github.com/mickael-kerjean/filestash) so you can audit the code that actually runs
there's a special page from the application that shows the hash of the binary alongside the commit from which this version is built from to enable anyone to see the built they run and verify it as well
I support assume role so you can use short lived credentials
I'm beginning to work with Secrets Manager and created my first secret in AWS. During the process, it gave me some sample code to work with. I put that in a small application and ran it. The code:
String region = "us-east-1";
string secret = "";
MemoryStream memoryStream = new MemoryStream();
IAmazonSecretsManager client = new AmazonSecretsManagerClient(
RegionEndpoint.GetBySystemName(region));
GetSecretValueRequest request = new GetSecretValueRequest();
request.SecretId = "MySecretNameExample";
GetSecretValueResponse response = null;
response = client.GetSecretValue(request);
The problem is that:
I was able to successfully retrieve the secret that I created and
nowhere am I creating a Credentials object with any valid AWS credential data
Where is this code getting the credential information from?
If you refer to the documenation for the API for this line of code:
IAmazonSecretsManager client = new AmazonSecretsManagerClient(
RegionEndpoint.GetBySystemName(region));
AmazonSecretsManagerClient
You will find the following description:
Constructs AmazonSecretsManagerClient with the credentials loaded from
the application's default configuration, and if unsuccessful from the
Instance Profile service on an EC2 instance.
This means that you are either running on an EC2 or ECS service (or related service such as Beanstalk, ...) with a role assigned to the instance or your have configured your credentials in the standard method in a credentials file. The AWS SDK is helping you locate credentials.
This document link will explain in more detail how AWS credentials are managed and selected.
Working with AWS Credentials
I have seen a lot of developers get the little details wrong with how credentials work and how they are used within the SDKs. Given that AWS credentials hold the keys to the AWS kingdom, managing and protecting them is vitally important.
The AWS SDK uses the a resolution strategy that looks in a number of locations until it finds credentials it can use. Typically the DefaultProviderChain class is responsible for performing the resolution. More information is here, but the gist is the lookup is performed in the following order (for Java, other languages are similar):
environment variables
Java system properties
credentials file (e.g. in the home directory)
instance profile credentials (only available when running in AWS)
When you run within AWS infrastructure, you can assign a profile or role to the resource that's running your code. Doing that makes credentials automatically available to your code. The idea is that they've made it easy to avoid putting credentials directly into your code.
AWS .Net SDK specifies several ways to specify credentials. One of them is SDK store - http://docs.aws.amazon.com/AWSSdkDocsNET/V3/DeveloperGuide/net-dg-config-creds.html#net-dg-config-creds-sdk-store
In that same document two seemingly contradictory things are written:
"SDK Store profiles are specific to a particular user on a particular host. They cannot be copied to other hosts or other users. For this reason, SDK Store profiles cannot be used in production applications."
"The associated credentials are incorporated into the application during the build process."
IF the credentials are incorporated in the program at the build time, then why can't I use the SDK store method in production environment?
That leaves me with either storing credentials in .config files, or credentials files, or using roles.
To clarify, the documentation is stating that if you are only using the SDK store, you can't simply pick up your application and move it to any other machine while expecting it to work as-is. If you are using profile names, your application is referencing the profile name in the local SDK store (or, failing that, the credentials file used by other AWS resources like the AWS CLI).
For example, if you created a profile named dev-Amit on your personal computer and then moved to another machine in production, dev-Amit wouldn't exist there unless you had already configured the SDK store on that machine with the same profile name.
This behavior is contrary to the recommended practice of passing credentials in EC2, using temporary credentials via IAM Roles. Using this method, you can reliably pass credentials to your applications on EC2 instances by simply attaching an IAM role to the instance that has sufficient permissions to do whatever your application needs to do. This method is preferred because you don't have to login to the instance to configure anything; you simply attach the IAM role at creation time.
Further Resources:
AWS Documentation - Tutorial: Grant Access Using an IAM Role and the AWS SDK for .NET
AWS Documentation - Order in which credentials are searched for