I'm creating DynamoDB client using IAM credentials obtained via STS assume role.
#Provides
public DynamoDbEnhancedClient DdbClientProvider() {
final AWSSecurityTokenServiceClientBuilder stsClientBuilder = AWSSecurityTokenServiceClientBuilder.standard()
.withClientConfiguration(new ClientConfiguration());
final AssumeRoleRequest assumeRoleRequest = new AssumeRoleRequest().withRoleSessionName("some.session.name");
assumeRoleRequest.setRoleArn("arnRole");
final AssumeRoleResult assumeRoleResult = stsClientBuilder.build().assumeRole(assumeRoleRequest);
final Credentials creds = assumeRoleResult.getCredentials();
final AwsSessionCredentials sessionCredentials = AwsSessionCredentials.create(creds.getAccessKeyId()
, creds.getSecretAccessKey(), creds.getSessionToken());
final AwsCredentialsProviderChain credsProvider = AwsCredentialsProviderChain.builder()
.credentialsProviders(StaticCredentialsProvider.create(sessionCredentials))
.build();
final DynamoDbClient ddbClient = DynamoDbClient.builder().region(Region.US_EAST_1)
.credentialsProvider(credsProvider).build();
final DynamoDbEnhancedClient ddbEnhancedclient =
DynamoDbEnhancedClient.builder().dynamoDbClient(ddbClient).build();
return ddbEnhancedClient;
}
The main lambda handler looks like below:
public void LambdaMainHandler {
DynamoDbEnhancedClient ddbClient;
#Inject
public LambdaMainHandler(final DynamoDbEnhancedClient client) {
this.ddbClient = client;
}
public LambdaResponse processRequest(final LambdaRequest request) {
QueryResponse queryResponse = client.query("...")
return LambdaResponse.builder().setContent(queryResponse.getByteBuffer()).build();
}
}
I'm using the DDB client in LambdaMain constructor.
Since this is running in Lambda behind APIGateway, how do I make sure the credentials are refreshed when they expire while executing LambdaMain handler?
Related
I have a Spring Boot application that has a POST end-point that accepts 2 types of files. Based on the file category, I need to write them to S3 buckets which are in different regions. Example: Category 1 file should be written to Frankfurt (eu-central-1) and Category 2 file should be written to Ohio (us-east-2) S3 buckets. Spring boot accepts a static region (cloud.aws.region.static=eu-central-1) through property configuration and the connection is established when starting the Spring boot so the AmazoneS3 Client Bean is already created with a connection to Frankfurt itself.
I need to containerize this entire setup and deploy it in a K8 Pod.
What is the recommendation for establishing connections and writing objects to different regional buckets? How do I need to implement this? Looking for a dynamic region finding solution rather statically created Bean per region.
Below is a working piece of code that connects to Frankfurt bucket and PUT the object.
#Service
public class S3Service {
#Autowired
private AmazonS3 amazonS3Client;
public void putObject(MultipartFile multipartFile) {
ObjectMetadata objectMetaData = new ObjectMetadata();
objectMetaData.setContentType(multipartFile.getContentType());
objectMetaData.setContentLength(multipartFile.getSize());
try {
PutObjectRequest putObjectRequest = new PutObjectRequest("example-bucket", multipartFile.getOriginalFilename(), multipartFile.getInputStream(), objectMetaData);
this.amazonS3Client.putObject(putObjectRequest);
} catch (IOException e) {
/* Handle Exception */
}
}
}
Updated Code (20/08/2021)
#Component
public class AmazoneS3ConnectionFactory {
private static final Logger LOGGER = LoggerFactory.getLogger(AmazoneS3ConnectionFactory.class);
#Value("${example.aws.s3.regions}")
private String[] regions;
#Autowired
private DefaultListableBeanFactory beanFactory;
#Autowired
private AWSCredentialsProvider credentialProvider;
#PostConstruct
public void init() {
for(String region: this.regions) {
String amazonS3BeanName = region + "_" + "amazonS3";
if (!this.beanFactory.containsBean(amazonS3BeanName)) {
AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard().withPathStyleAccessEnabled(true)
.withCredentials(this.credentialProvider).withRegion(region).withChunkedEncodingDisabled(true);
AmazonS3 awsS3 = builder.build();
this.beanFactory.registerSingleton(amazonS3BeanName, awsS3);
LOGGER.info("Bean " + amazonS3BeanName + " - Not exist. Created a bean and registered the same");
}
}
}
/**
* Returns {#link AmazonS3} for a region. Uses the default {#link AWSCredentialsProvider}
*/
public AmazonS3 getConnection(String region) {
String amazonS3BeanName = region + "_" + "amazonS3";
return (AmazonS3Client)this.beanFactory.getBean(amazonS3BeanName, AmazonS3.class);
}
}
My Service layer will call the "getConnection()" and get the AmazonS3 Object to operate on it.
The only option that I am aware is to create different S3Client with S3ClientBuilder, one for each different region. You would need to register them as Spring Beans with different names so that you can later autowire them.
Update (19/08/2021)
The following should work (sorry for the Kotlin code but it is faster to write):
Class that may contain your configuration for each region.
class AmazonS3Properties(val accessKeyId: String,
val secretAccessKey: String,
val region: String,
val bucket: String)
Configuration for S3 that will create 2 S3Clients and stored the buckets for each region (later needed).
#Configuration
class AmazonS3Configuration(private val s3Properties: Map<String, AmazonS3Properties>) {
lateinit var buckets: Map<String, String>
#PostConstruct
fun init() {
buckets = s3Properties.mapValues { it.bucket }
}
#Bean(name = "regionA")
fun regionA(): S3Client {
val regionAProperties = s3Properties["region-a"]
val awsCredentials = AwsBasicCredentials.create(regionAProperties.accessKeyId, regionAProperties.secretAccessKey)
return S3Client.builder().region(Region.of(regionAProperties.region)).credentialsProvider { awsCredentials }.build()
}
#Bean(name = "regionB")
fun regionB(): S3Client {
val regionBProperties = s3Properties["region-b"]
val awsCredentials = AwsBasicCredentials.create(regionBProperties.accessKeyId, regionBProperties.secretAccessKey)
return S3Client.builder().region(Region.of(regionBProperties.region)).credentialsProvider { awsCredentials }.build()
}
}
Service that will target one of the regions (Region A)
#Service
class RegionAS3Service(private val amazonS3Configuration: AmazonS3Configuration,
#field:Qualifier("regionA") private val amazonS3Client: S3Client) {
fun save(region: String, byteArrayOutputStream: ByteArrayOutputStream) {
val inputStream = ByteArrayInputStream(byteArrayOutputStream.toByteArray())
val contentLength = byteArrayOutputStream.size().toLong()
amazonS3Client.putObject(PutObjectRequest.builder().bucket(amazonS3Configuration.buckets[region]).key("whatever-key").build(), RequestBody.fromInputStream(inputStream, contentLength))
}
}
I have a Cognito userpool on AWS account acc-1, and a Java code running on acc-2, which authenticates using "adminInitiateAuth", and for some reasons, I cannot use clientInitiateAuth.
I have created a cross-account role on acc-1, to be assumed by my Java code on acc-2
Question: How can I assume the role when I am sending an authentication request to Cognito? Is it possible to use withRoleArn()?
I came across this page, which explains how to "Configure cross-account Amazon Cognito authorizer for a REST API using the API Gateway console". But it is not what I am trying to do.
My Code:
protected AdminInitiateAuthRequest createInitialRequest(String username, String password) {
Map<String, String> authParams = new HashMap<>();
authParams.put("USERNAME", username);
authParams.put("PASSWORD", password);
return new AdminInitiateAuthRequest()
.withAuthFlow(AuthFlowType.ADMIN_NO_SRP_AUTH)
.withAuthParameters(authParams)
.withClientId(whoAmIService.getCognitoClientId())
.withUserPoolId(whoAmIService.getCognitoPoolId());
}
protected boolean isAuthenticatedByCognito(String username, String password) {
AWSCognitoIdentityProvider awsCognitoIDPClient = createCognitoIDPClient();
AdminInitiateAuthRequest authRequest = createInitialRequest(username, password);
try {
AdminInitiateAuthResult authResponse = awsCognitoIDPClient.adminInitiateAuth(authRequest);
AuthenticationResultType authenticationResultType = authResponse.getAuthenticationResult();
String cognitoAccessToken = authenticationResultType.getAccessToken();
whoAmIService.setCognitoAccessToken(cognitoAccessToken);
Map<String, String> challengeParams = authResponse.getChallengeParameters();
String cognitoUserIdForSrp = challengeParams.get("USER_ID_FOR_SRP");
String cognitoUserAttributes = challengeParams.get("userAttributes");
logger.debug("Cognito authenticated user ID: {} with user attributes: {}"
, cognitoUserIdForSrp, cognitoUserAttributes);
return true;
} catch (NotAuthorizedException nae) {
logger.error("Invalid Cognito username/password provided for {}", username);
return false;
} catch (AWSCognitoIdentityProviderException acipe) {
logger.error("Base exception for all service exceptions thrown by Amazon Cognito Identity Provider", acipe);
return false;
}
}
I found how to do it using STS. Change this line:
AWSCognitoIdentityProvider awsCognitoIDPClient = createCognitoIDPClient();
to:
String roleARN= "YOUR_CROSS_ACCOUNT_ROLE_ARN";
String roleSessionName = "GIVE_A_SESSION_NAME";
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder
.standard()
.withCredentials(new ProfileCredentialsProvider())
.build();
AssumeRoleRequest roleRequest = new AssumeRoleRequest()
.withRoleArn(roleARN)
.withRoleSessionName(roleSessionName);
AssumeRoleResult roleResponse = stsClient.assumeRole(roleRequest);
Credentials sessionCredentials = roleResponse.getCredentials();
BasicSessionCredentials awsCredentials = new BasicSessionCredentials(
sessionCredentials.getAccessKeyId(),
sessionCredentials.getSecretAccessKey(),
sessionCredentials.getSessionToken());
AWSCognitoIdentityProvider cognitoIPCB = AWSCognitoIdentityProviderClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.build();
I cannot access S3 through Java Code, but can through AWS CLI.
I am using Credentials from AWS SDK for MINIO
// import statements
public class S3Application {
private static final AWSCredentials credentials;
private static String bucketName;
static {
//put your accesskey and secretkey here
credentials = new BasicAWSCredentials(
"Q3AM3UQ867SPQQA43P2F",
"zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
);
}
public static void main(String[] args) throws IOException {
//set-up the client
AmazonS3 s3Client = AmazonS3ClientBuilder
.standard()
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("http://play.min.io:9000","us-east-1"))
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.build();
AWSS3Service awsService = new AWSS3Service(s3Client);
}
}
This is my log for the above mentioned code.
Exception in thread "main" com.amazonaws.SdkClientException: Unable to execute HTTP request: Connection reset
...
Caused by: java.net.SocketException: Connection reset
...
... 13 more
Process finished with exit code 1
You have might have to set PathStyle access to true.
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Builder.html#withPathStyleAccessEnabled-java.lang.Boolean-
Code like this might work.
// import statements
public class S3Application {
private static final AWSCredentials credentials;
private static String bucketName;
static {
//put your accesskey and secretkey here
credentials = new BasicAWSCredentials(
"Q3AM3UQ867SPQQA43P2F",
"zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
);
}
public static void main(String[] args) throws IOException {
//set-up the client
AmazonS3 s3Client = AmazonS3ClientBuilder
.standard()
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("http://play.min.io:9000","us-east-1"))
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withPathStyleAccessEnabled(true)
.build();
AWSS3Service awsService = new AWSS3Service(s3Client);
}
}
I'm trying to execute query in AWS Athena using Java API:
public class AthenaClientFactory
{
String accessKey = "access";
String secretKey = "secret";
BasicAWSCredentials awsCredentials = new
BasicAWSCredentials(accessKey, secretKey);
private final AmazonAthenaClientBuilder builder = AmazonAthenaClientBuilder.standard()
.withRegion(Regions.US_WEST_1)
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withClientConfiguration(new ClientConfiguration().withClientExecutionTimeout(10));
public AmazonAthena createClient()
{
return builder.build();
}
}
private static String submitAthenaQuery(AmazonAthena client) {
QueryExecutionContext queryExecutionContext = new QueryExecutionContext().withDatabase("my_db");
ResultConfiguration resultConfiguration = new ResultConfiguration().withOutputLocation("my_bucket");
StartQueryExecutionRequest startQueryExecutionRequest = new StartQueryExecutionRequest()
.withQueryString("select * from my_db limit 3;")
.withQueryExecutionContext(queryExecutionContext)
.withResultConfiguration(resultConfiguration);
StartQueryExecutionResult startQueryExecutionResult = client.startQueryExecution(startQueryExecutionRequest);
return startQueryExecutionResult.getQueryExecutionId();
}
public void run() throws InterruptedException {
AthenaClientFactory factory = new AthenaClientFactory();
AmazonAthena client = factory.createClient();
String queryExecutionId = submitAthenaQuery(client);
}
But I get an exception from startQueryExecutionResult.
The exception is:
Client execution did not complete before the specified timeout
configuration.
Has anyone encountered something similar?
The problem was in withClientExecutionTimeout(10).
Increasing this number to 5000 solved the issue
I'm trying to use AWS IAM to generate temporary tokens for a mobile app. I'm using the AWS C# SDK.
Here's my code...
The token generating service
public string GetIAMKey(string deviceId)
{
//fetch IAM key...
var credentials = new BasicAWSCredentials("MyKey", "MyAccessId");
var sts = new AmazonSecurityTokenServiceClient(credentials);
var tokenRequest = new GetFederationTokenRequest();
tokenRequest.Name = deviceId;
tokenRequest.Policy = File.ReadAllText(HostingEnvironment.MapPath("~/policy.txt"));
tokenRequest.DurationSeconds = 129600;
var tokenResult = sts.GetFederationToken(tokenRequest);
var details = new IAMDetails { SessionToken = tokenResult.GetFederationTokenResult.Credentials.SessionToken, AccessKeyId = tokenResult.GetFederationTokenResult.Credentials.AccessKeyId, SecretAccessKey = tokenResult.GetFederationTokenResult.Credentials.SecretAccessKey, };
return JsonConvert.SerializeObject(details);
}
The client
var iamkey = Storage.LoadPersistent<IAMDetails>("iamkey");
var simpleDBClient = new AmazonSimpleDBClient(iamkey.AccessKeyId, iamkey.SecretAccessKey, iamkey.SessionToken);
try
{
var details = await simpleDBClient.SelectAsync(new SelectRequest { SelectExpression = "select * from mydomain" });
return null;
}
catch (Exception ex)
{
Storage.ClearPersistent("iamkey");
}
The policy file contents
{ "Statement":[{ "Effect":"Allow", "Action":"sdb:* ", "Resource":"arn:aws:sdb:eu-west-1:* :domain/mydomain*" } ]}
I keep getting the following error...
User (arn:aws:sts::myaccountid:federated-user/654321) does not have permission to perform (sdb:Select) on resource (arn:aws:sdb:us-east-1:myaccountid:domain/mydomain)
Notice that my policy file clearly specifies two things
region should be eu-west-1
allowed action is a wild-card, ie, allow everything
But the exception thrown claims that my user doesn't have permission to us-east-1
Any ideas as to why I'm getting this error?
Ok figured it out.
You have to set the region endpoint on your call to the service from the client.
So
var simpleDBClient = new AmazonSimpleDBClient(iamkey.AccessKeyId, iamkey.SecretAccessKey, iamkey.SessionToken, Amazon.RegionEndpoint.EUWest1);