I am using AWS Systems Manager Parameter Store to hold database connection strings which are used to dynamically build a DbContext in my .NET Core Application
I am using the .NET Core AWS configuration provider (from https://aws.amazon.com/blogs/developer/net-core-configuration-provider-for-aws-systems-manager/) which injects my parameters into the IConfiguration at runtime.
At the moment I am having to keep my AWS access key/secret in code so it can be accessed by the ConfigurationBuilder but would like to move this out of the code base and stored it in appsettings or similar.
Here is my method to create the webhost builder called at startup
public static IWebHostBuilder CreateWebHostBuilder(string[] args)
{
var webHost = WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>();
AWSCredentials credentials = new BasicAWSCredentials("xxxx", "xxxx");
AWSOptions options = new AWSOptions()
{
Credentials = credentials,
Region = Amazon.RegionEndpoint.USEast2
};
webHost.ConfigureAppConfiguration(config =>
{
config.AddJsonFile("appsettings.json");
config.AddSystemsManager("/ParameterPath", options, reloadAfter: new System.TimeSpan(0, 1, 0)); // Reload every minute
});
return webHost;
}
I need to be able to inject the BasicAWSCredentials parameter from somewhere.
You need to access an already built configuration to be able to retrieve the information you seek.
Consider building one to retrieve the needed credentials
public static IWebHostBuilder CreateWebHostBuilder(string[] args) {
var webHost = WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>();
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.Build();
var access_key = configuration.GetValue<string>("access_key:path_here");
var secret_key = configuration.GetValue<string>("secret_key:path_here");
AWSCredentials credentials = new BasicAWSCredentials(access_key, secret_key);
AWSOptions options = new AWSOptions() {
Credentials = credentials,
Region = Amazon.RegionEndpoint.USEast2
};
webHost.ConfigureAppConfiguration(config => {
config.AddJsonFile("appsettings.json");
config.AddSystemsManager("/ParameterPath", options, reloadAfter: new System.TimeSpan(0, 1, 0)); // Reload every minute
});
return webHost;
}
I would also suggest reviewing Configuring AWS Credentials from the docs to use the SDK to find a possible alternative way to storing and retrieving the credentials.
Related
I am trying to upload a file in S3 by AWS Assume Role. When I am trying to access it from CLI it works fine but from .Net SDK it gives me Access Denied error.
Here are the steps I followed in CLI -
Setup the access key/secret key for user using aws configure
Assume the Role - “aws sts assume-role --role-arn "arn:aws:iam::1010101010:role/Test-Account-Role" --role-session-name AWSCLI-Session”
Take the access key / secret key / session token from the assumed role and setup an AWS profile. The credentials are printed out/returned from the assumed role.
Switch to the assume role profile: “set AWS_PROFILE=”
Verify that the user has the role: “aws sts get-caller-identity”
Access the bucket using ls or cp or rm command - Works Successfully.
Now I am trying to access it from .Net core App -
Here is the code snippet- Note that I am using same Access and Secret key as CLI from my local.
try
{
var region = RegionEndpoint.GetBySystemName(awsRegion);
SessionAWSCredentials tempCredentials = await GetTemporaryCredentialsAsync(awsAccessKey, awsSecretKey, region, roleARN);
//Use the temp credentials received to create the new client
IAmazonS3 client = new AmazonS3Client(tempCredentials, region);
TransferUtility utility = new TransferUtility(client);
// making a TransferUtilityUploadRequest instance
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest
{
BucketName = bucketName,
Key = $"{subFolder}/{fileName}",
FilePath = localFilePath
utility.Upload(request); //transfer
fileUploadedSuccessfully = true;
}
catch (AmazonS3Exception ex)
{
// HandleException
}
catch (Exception ex)
{
// HandleException
}
The method to get temp credentials is as follow - GetTemporaryCredentialsAsync
private static async Task<SessionAWSCredentials> GetTemporaryCredentialsAsync(string awsAccessKey, string awsSecretKey, RegionEndpoint region, string roleARN)
{
using (var stsClient = new AmazonSecurityTokenServiceClient(awsAccessKey, awsSecretKey, region))
{
var getSessionTokenRequest = new GetSessionTokenRequest
{
DurationSeconds = 7200
};
await stsClient.AssumeRoleAsync(
new AssumeRoleRequest()
{
RoleArn = roleARN,
RoleSessionName = "mySession"
});
GetSessionTokenResponse sessionTokenResponse =
await stsClient.GetSessionTokenAsync(getSessionTokenRequest);
Credentials credentials = sessionTokenResponse.Credentials;
var sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
return sessionCredentials;
}
}
I am getting back the temp credentials but it gives me Access Denied while uploading the file. Not sure if I am missing anything here.
Also noted that the token generated via SDK is shorter than that from CLI. I tried pasting these temp credentials to local profile and then tried to access the bucket and getting the Access Denied error then too.
There is an AWS .NET V3 example that shows this exact use case. To assume a role, you use a AmazonSecurityTokenServiceClient. In this example, the user assumes the role that allows the role to be used to list all S3 buckets. See this .NET scenario here.
https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/dotnetv3/IAM/IAM_Basics_Scenario/IAM_Basics_Scenario/IAM_Basics.cs
Hello I am trying to download data from one of an Amazon S3 public bucket.
For example https://registry.opendata.aws/noaa-gfs-bdp-pds/
The bucket has web accessible folder and I want to download the files inside the bucket.
I know I can do this with AWS CLI tool
But I want to know if there anyway to do this with AWs SDK Api (s3 client) (c# visual studio)?
I think the issue is authentication when creating connection to s3 client it requires credentials like access key ,I don't have an AWS account,and the bucket I try to get to is public so
Does anyone know how to access to this public bucket without any credentials via API?
Thanks.
If you specify the AnonymousAWSCredentials as the credentials object, any requests that are made to S3 will be unsigned. After that, interacting with the bucket is done like any other call:
using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Model;
using System;
namespace S3TestApp
{
class Program
{
static void Main(string[] args)
{
var unsigned = new AnonymousAWSCredentials();
var client = new AmazonS3Client(unsigned, Amazon.RegionEndpoint.USEast1);
var listRequest = new ListObjectsRequest
{
BucketName = "noaa-gfs-bdp-pds",
Delimiter = "/",
};
ListObjectsResponse listResponse;
do
{
listResponse = client.ListObjects(listRequest);
foreach (var obj in listResponse.CommonPrefixes)
{
Console.WriteLine("PRE {0}", obj);
}
foreach (var obj in listResponse.S3Objects)
{
Console.WriteLine("{0} {1}", obj.Size, obj.Key);
}
listRequest.Marker = listResponse.NextMarker;
} while (listResponse.IsTruncated);
}
}
}
I am working on a requirement where i want to connect to s3 bucket using springboot application.
When i am connecting through my local environment i am using seeting loadCredentials(true) which uses Amazon STS which fetches the temperoriy credentials using a role and allow access to s3 bucket.
When i am deploying to the qa/prd envirment i am setting loadCredentials(false) which uses DefaultAWSCredentialsProviderChain() class to fetch the credential from aws instance profile(role is assigned to ec2 instance) and allow access to s3 bucket. My code is
#Configuration
public class AmazonS3Config
{
static String clientRegion = "ap-south-1";
static String roleARN = "arn:aws:iam::*************:role/awss3acess";
static String roleSessionName = "bucket_storage_audit";
String bucketName = "testbucket";
//Depending on environment is set to true(for local environment) or false(for qa and prd environment)
private static AWSCredentialsProvider loadCredentials(boolean isLocal) {
final AWSCredentialsProvider credentialsProvider;
if (isLocal) {
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceAsyncClientBuilder.standard()
.withCredentials(new ProfileCredentialsProvider())
.withRegion(clientRegion)
.build();
AssumeRoleRequest assumeRoleRequest = new AssumeRoleRequest().withDurationSeconds(3600)
.withRoleArn(roleARN)
.withRoleSessionName(roleSessionName);
AssumeRoleResult assumeRoleResult = stsClient.assumeRole(assumeRoleRequest);
Credentials creds = assumeRoleResult.getCredentials();
credentialsProvider = new AWSStaticCredentialsProvider(
new BasicSessionCredentials(creds.getAccessKeyId(),
creds.getSecretAccessKey(),
creds.getSessionToken())
);
} else {
System.out.println("inside default");
credentialsProvider = new DefaultAWSCredentialsProviderChain();
}
return credentialsProvider;
}
// Amazon s3client Bean return an instance of s3client
. #Bean
public AmazonS3 s3client() {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(Regions.fromName(clientRegion))
.withCredentials(loadCredentials(false))
.build();
return s3Client;
}
}
My question since the credentials of instance profile rotate after every 12 hours my application will fail after 12 hours.
What will i do to avoid this from happening in my code.
You can directly use ProfileCredentialsProvider instead of DefaultAWSCredentialsProviderChain as there is no need in your case to chain the credsproviders.
and about your question, AWSCredentialsProvider has refresh() method that will reread the config file.when an Authentication exception Occurs while using S3Client you can retry again and call refresh() first.
when i make post request to upload image file to aws s3 bucket form my local dot net core aws lambda serverless application it works but form my deployed application the image still uploded to s3 bucket but the image broken (shows black empty image)
here is the code
[HttpPut("PostImageFile")]
public async Task FileImageAsync(string Id)
{
var s3Client = new AmazonS3Client("*******", "*******", Amazon.RegionEndpoint.USEast1);
try
{
var httpRequest = HttpContext.Request;
//posted file
var file = httpRequest.Form.Files[0];
byte[] fileBytes = new byte[file.Length];
file.OpenReadStream().Read(fileBytes, 0, Int32.Parse(file.Length.ToString()));
var fileName = Guid.NewGuid() + file.FileName;
PutObjectResponse response = null;
using (var stream = new MemoryStream())
{
file.CopyTo(stream);
var request = new PutObjectRequest
{
BucketName = "imageakal",
Key = fileName,
InputStream = stream,
ContentType = file.ContentType,
CannedACL = S3CannedACL.PublicReadWrite
};
response = await s3Client.PutObjectAsync(request);
};
}
catch (Exception ex)
{
Console.Write("Upload Failed: " + ex.Message);
}
}
Without many more details, I would guess that your AWS settings could have a list of permitted/denied domains. I would check that your AWS instance is configured to allow requests from your domain.
Just put "multipart/form-data" at 'Binary Media Type' section in Api Gateway setting tab, and deploy it(don't forget).
I am working on my AWS cert and I'm trying to figure out how the following bit of js code works:
var AWS = require('aws-sdk');
var uuid = require('node-uuid');
// Create an S3 client
var s3 = new AWS.S3();
// Create a bucket and upload something into it
var bucketName = 'node-sdk-sample-' + uuid.v4();
var keyName = 'hello_world.txt';
s3.createBucket({Bucket: bucketName}, function() {
var params = {Bucket: bucketName, Key: keyName, Body: 'Hello'};
s3.putObject(params, function(err, data) {
if (err)
console.log(err)
else
console.log("Successfully uploaded data to " + bucketName + "/" + keyName);
});
});
This code successfully loads a txt file containing the words "Hello" in it. I do not understand how this ^ can identify MY AWS account. It does! But how! It somehow is able to determine that I want a new bucket inside MY account, but this code was taken directly from the AWS docs. I don't know how it could figure that out....
As per Class: AWS.CredentialProviderChain, the AWS SDK for JavaScript looks for credentials in the following locations:
AWS.CredentialProviderChain.defaultProviders = [
function () { return new AWS.EnvironmentCredentials('AWS'); },
function () { return new AWS.EnvironmentCredentials('AMAZON'); },
function () { return new AWS.SharedIniFileCredentials(); },
function () {
// if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is set
return new AWS.ECSCredentials();
// else
return new AWS.EC2MetadataCredentials();
}
]
Environment Variables (useful for testing, or when running code on a local computer)
Local credentials file (useful for running code on a local computer)
ECS credentials (useful when running code in Elastic Container Service)
Amazon EC2 Metadata (useful when running code on an Amazon EC2 instance)
It is highly recommended to never store credentials within an application. If the code is running on an Amazon EC2 instance and a role has been assigned to the instance, the SDK will automatically retrieve credentials from the instance metadata.
The next best method is to store credentials in the ~/.aws/credentials file.