Access Denied when listing object using amazon s3 client - amazon-web-services

From the aws lambda I want to list objects inside s3 bucket. When testing the function locally I'm getting access denied error.
public async Task<string> FunctionHandler(string input, ILambdaContext context)
{
var secretKey = "***";
var uid = "***";
var bucketName = "my-bucket-name";
AmazonS3Client s3Client = new AmazonS3Client(uid, secretKey);
ListObjectsRequest listObjectsRequest = new ListObjectsRequest();
listObjectsRequest.BucketName = bucketName;
var listObjectsResponse = await s3Client.ListObjectsAsync(listObjectsRequest);
// exception is thrown
...
}
Amazon.S3.AmazonS3Exception: Access Denied at
Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext
requestContext, IWebResponseData httpErrorResponse,
HttpErrorResponseException exception, Stream responseStream) at
Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext
executionContext, HttpErrorResponseException exception) ....
The bucket I'm using in this example "my-bucket-name" is Publicly accessible and it has
Any idea?

First of all, IAM policies are a preferred way how to control access to S3 buckets.
For S3 permissions it is always very important to distinguish between bucket level actions and object level actions and also - who is calling that action. In your code I can see that you do use ListObjects, which is a bucket level action, so that is OK.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
What did catch my I is the following:
var secretKey = "***";
var uid = "***";
var bucketName = "my-bucket-name";
AmazonS3Client s3Client = new AmazonS3Client(uid, secretKey);
That means that you are using for your access an AWS role. But even in your screenshot you can see that "Authenticated users group (anyone with an AWS account)" does not have any permissions assigned.
If you already have a role I would suggest to give the read-bucket permissions to that particular role (user) via an IAM policy. But adding read ACL to your AWS users should help as well.

Related

AWS Assume Role via .Net SDK gives Access Denied but works with CLI

I am trying to upload a file in S3 by AWS Assume Role. When I am trying to access it from CLI it works fine but from .Net SDK it gives me Access Denied error.
Here are the steps I followed in CLI -
Setup the access key/secret key for user using aws configure
Assume the Role - “aws sts assume-role --role-arn "arn:aws:iam::1010101010:role/Test-Account-Role" --role-session-name AWSCLI-Session”
Take the access key / secret key / session token from the assumed role and setup an AWS profile. The credentials are printed out/returned from the assumed role.
Switch to the assume role profile: “set AWS_PROFILE=”
Verify that the user has the role: “aws sts get-caller-identity”
Access the bucket using ls or cp or rm command - Works Successfully.
Now I am trying to access it from .Net core App -
Here is the code snippet- Note that I am using same Access and Secret key as CLI from my local.
try
{
var region = RegionEndpoint.GetBySystemName(awsRegion);
SessionAWSCredentials tempCredentials = await GetTemporaryCredentialsAsync(awsAccessKey, awsSecretKey, region, roleARN);
//Use the temp credentials received to create the new client
IAmazonS3 client = new AmazonS3Client(tempCredentials, region);
TransferUtility utility = new TransferUtility(client);
// making a TransferUtilityUploadRequest instance
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest
{
BucketName = bucketName,
Key = $"{subFolder}/{fileName}",
FilePath = localFilePath
utility.Upload(request); //transfer
fileUploadedSuccessfully = true;
}
catch (AmazonS3Exception ex)
{
// HandleException
}
catch (Exception ex)
{
// HandleException
}
The method to get temp credentials is as follow - GetTemporaryCredentialsAsync
private static async Task<SessionAWSCredentials> GetTemporaryCredentialsAsync(string awsAccessKey, string awsSecretKey, RegionEndpoint region, string roleARN)
{
using (var stsClient = new AmazonSecurityTokenServiceClient(awsAccessKey, awsSecretKey, region))
{
var getSessionTokenRequest = new GetSessionTokenRequest
{
DurationSeconds = 7200
};
await stsClient.AssumeRoleAsync(
new AssumeRoleRequest()
{
RoleArn = roleARN,
RoleSessionName = "mySession"
});
GetSessionTokenResponse sessionTokenResponse =
await stsClient.GetSessionTokenAsync(getSessionTokenRequest);
Credentials credentials = sessionTokenResponse.Credentials;
var sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
return sessionCredentials;
}
}
I am getting back the temp credentials but it gives me Access Denied while uploading the file. Not sure if I am missing anything here.
Also noted that the token generated via SDK is shorter than that from CLI. I tried pasting these temp credentials to local profile and then tried to access the bucket and getting the Access Denied error then too.
There is an AWS .NET V3 example that shows this exact use case. To assume a role, you use a AmazonSecurityTokenServiceClient. In this example, the user assumes the role that allows the role to be used to list all S3 buckets. See this .NET scenario here.
https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/dotnetv3/IAM/IAM_Basics_Scenario/IAM_Basics_Scenario/IAM_Basics.cs

Rejected to connect from lambda to cognito due to not authorized

If CognitoIdentityProvider::Client doesn't provide access_key_id and secret_access_key, it will retrieve from the ENV by default.
The strange thing is it works at a totally new lambda but not in another.
client = Aws::CognitoIdentityProvider::Client.new(region: ENV['AWS_REGION'])
client.admin_get_user({
user_pool_id: Jets.application.config.cognito[:user_pool_id],
username: 'username'
})
I would get this error message but had no idea where can I set the policy for cognito.
"errorMessage": "User: arn:aws:sts::123123123123:assumed-role/firstage-api-stag-IamRole-1DYOOEVSCURMY/xxxxxx-api-stag-mes_controller-show is not authorized to perform: cognito-idp:AdminGetUser on resource: arn:aws:cognito-idp:ap-northeast-2:319924209672:userpool/ap-northeast-2_0cSCFMK4r",
"errorType": "Function<Aws::CognitoIdentityProvider::Errors::AccessDeniedException>",
I would like to use the default key_id and access_key in the lambda ENV rather than IAM user.
What should I do???
Best practice for Lambda functions is to ensure your IAM role associated with the Lambda function has the policy to let it invoke the specific AWS Service. Do not hard code creds in Lambda functions.
For some services, you need to write a custom policy. For example, for Pinpoint voice, you need to write a custom policy using JSON so the Lambda function can invoke that service.
For CognitoIdentityProvider, try using this policy for your IAM role:
When in doubt which services your IAM role can invoke - check the Access Advisor tab:
I just tested this use case and built a Lambda function that creates a CognitoIdentityProviderClient object and gets users in a specific user pool. I implemented this using the Lambda Java runtime API, but it does not matter what supported Lambda programming language you use. You still need to configure your IAM role in the same way.
This code works perfectly. Here is the Handler class:
public class Handler implements RequestHandler<Map<String,String>, String> {
#Override
public String handleRequest(Map<String,String> event, Context context) {
LambdaLogger logger = context.getLogger();
String pool = event.get("Pool");
logger.log("pool: " + pool);
CognitoInfo cog = new CognitoInfo();
String xml = cog.getUsers(pool);
logger.log("XML: " + xml);
return xml;
}
}
Here is a method named getUsers located in the CognitoInfo class that invokes the AWS Service:
public String getUsers( String userPoolId) {
CognitoIdentityProviderClient cognitoclient = CognitoIdentityProviderClient.builder()
.region(Region.US_EAST_1)
.build();
try {
ArrayList<String> userList = new ArrayList();
// List all users
ListUsersRequest usersRequest = ListUsersRequest.builder()
.userPoolId(userPoolId)
.build();
ListUsersResponse response = cognitoclient.listUsers(usersRequest);
for(UserType user : response.users()) {
userList.add(user.username());
}
return convertToString(toXml(userList));
} catch (CognitoIdentityProviderException e){
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
return "";
}
The Lambda function is configured to use an IAM role named lambda-support2 that has the AmazonCognitoPowerUser policy.
This Lambda function successfully retrieves the users in a specific user pool:
Make sure that all your Lambda functions have the proper IAM role configured.

How to use DefaultAWSCredentialsProviderChain() in java code to fetch credentials from instance profile and allow access to s3 bucket

I am working on a requirement where i want to connect to s3 bucket using springboot application.
When i am connecting through my local environment i am using seeting loadCredentials(true) which uses Amazon STS which fetches the temperoriy credentials using a role and allow access to s3 bucket.
When i am deploying to the qa/prd envirment i am setting loadCredentials(false) which uses DefaultAWSCredentialsProviderChain() class to fetch the credential from aws instance profile(role is assigned to ec2 instance) and allow access to s3 bucket. My code is
#Configuration
public class AmazonS3Config
{
static String clientRegion = "ap-south-1";
static String roleARN = "arn:aws:iam::*************:role/awss3acess";
static String roleSessionName = "bucket_storage_audit";
String bucketName = "testbucket";
//Depending on environment is set to true(for local environment) or false(for qa and prd environment)
private static AWSCredentialsProvider loadCredentials(boolean isLocal) {
final AWSCredentialsProvider credentialsProvider;
if (isLocal) {
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceAsyncClientBuilder.standard()
.withCredentials(new ProfileCredentialsProvider())
.withRegion(clientRegion)
.build();
AssumeRoleRequest assumeRoleRequest = new AssumeRoleRequest().withDurationSeconds(3600)
.withRoleArn(roleARN)
.withRoleSessionName(roleSessionName);
AssumeRoleResult assumeRoleResult = stsClient.assumeRole(assumeRoleRequest);
Credentials creds = assumeRoleResult.getCredentials();
credentialsProvider = new AWSStaticCredentialsProvider(
new BasicSessionCredentials(creds.getAccessKeyId(),
creds.getSecretAccessKey(),
creds.getSessionToken())
);
} else {
System.out.println("inside default");
credentialsProvider = new DefaultAWSCredentialsProviderChain();
}
return credentialsProvider;
}
// Amazon s3client Bean return an instance of s3client
. #Bean
public AmazonS3 s3client() {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(Regions.fromName(clientRegion))
.withCredentials(loadCredentials(false))
.build();
return s3Client;
}
}
My question since the credentials of instance profile rotate after every 12 hours my application will fail after 12 hours.
What will i do to avoid this from happening in my code.
You can directly use ProfileCredentialsProvider instead of DefaultAWSCredentialsProviderChain as there is no need in your case to chain the credsproviders.
and about your question, AWSCredentialsProvider has refresh() method that will reread the config file.when an Authentication exception Occurs while using S3Client you can retry again and call refresh() first.

How do I GetSessionToken from AmazonSecurityTokenServiceClient created with temporary saml credentials?

Here's what I'm trying to do-
var role = new AmazonSecurityTokenServiceClient(aws_access_key_id, aws_secret_access_key, aws_session_token, Amazon.RegionEndpoint.USWest2);
role.AssumeRole(new AssumeRoleRequest
{
DurationSeconds = 3600,
RoleArn = rolearn,
RoleSessionName = sessionname
});
GetSessionTokenResponse sessionTokenResponse = role.GetSessionToken(new GetSessionTokenRequest
{
DurationSeconds = 7200
});
Now my "role" is created with temporary credentials I obtained with saml. And it seems that since GetSessionToken provides you with temporary credentials, it needs "role" to be created with long term credentials. I can't seem to find a workaround for this.
Essentially, I'm trying to create an AmazonS3Client with assumed IAM role that has certain permissions. Here's what I plan to do if I manage to GetSessionToken-
var newcreds = sessionTokenResponse.Credentials;
var sessionCredentials = new SessionAWSCredentials(newcreds.AccessKeyId, newcreds.SecretAccessKey, newcreds.SessionToken);
AmazonS3Client newclient = new AmazonS3Client(sessionCredentials, Amazon.RegionEndpoint.USWest2);
It is not possible to call GetSessionToken using credentials returned by AssumeRoleWithSAML. From AWS doc:
The temporary security credentials created by AssumeRoleWithSAML can
be used to make API calls to any AWS service with the following
exception: you cannot call the STS service's GetFederationToken or
GetSessionToken API operations.

How do I set permissions so an IAM can upload to an Amazon S3 bucket in C#?

I've created a user in IAM, and attached 2 managed policies: AmazonS3FullAccess and AdministratorAccess. I would like to upload files to an S3 bucket called "pscfront".
I am using the following code to do the upload:
AWSCredentials credentials = new BasicAWSCredentials(Constants.AmazonWebServices.AccessKey, Constants.AmazonWebServices.SecretKey);
using (var client = new AmazonS3Client(credentials, RegionEndpoint.USEast1))
{
var loc = client.GetBucketLocation("s3.amazonaws.com/pscfront");
var tu = new TransferUtility(client);
tu.Upload(filename, Constants.AmazonWebServices.BucketName, keyName);
}
This fails with the exception "AccessDenied" (inner exception "The remote server returned an error: (403) Forbidden.") at the call to GetBucketLocation, or at the tu.Upload call if I comment that line out.
Any idea what gives?
smdh
Nothing wrong with the persmissions -- I was setting the bucket name incorrectly.You just pass the plan bucket name -- "pscfront" in this case.