I am trying to Terraform snowflake_stage and use the arn from the IAM role, that was also terraformed, as the credential.
The Snowflake SQL works when I use:
create stage dev
URL='s3://name_of_bucket/'
storage_integration = dev_integration
credentials=(AWS_ROLE='arn:aws:iam:999999999999:role/service-role-name')
encryption=(TYPE='AWS_SSE_KMS' KMS_KEY_ID='aws/key')
FILE_FORMAT=DATABASE.PUBLIC.SCHEMA.FORMAT_NAME
COPY_OPTION=(ON_ERROR='CONTINUE' PURGE='FALSE' RETURN_FAILED_ONLY='TRUE');
but when I try to write an equivalent Terraform resource "snowflake_stage" using:
resource "snowflake_stage" "stage" {
name = "dev"
url = "s3://name_of_bucket/"
storage_integration = "dev_integration"
schema = "public"
credentials = "AWS_ROLE='aws_iam_role.snowflake_stage.arn'"
encryption = "(TYPE='AWS_SSE_KMS' KMS_KEY_ID='aws/key')
file_format = "DATABASE.PUBLIC.SCHEMA.FORMAT_NAME"
copy_options = "(ON_ERROR='CONTINUE' PURGE='FALSE' RETURN_FAILED_ONLY='TRUE')"
}
I get :
SQL compilation error: invalid value [Not a property list: TOK_LIST] for parameter '{1}
The value on the encryption seems to need the "AWS_ROLE='..'" to be valid.
I've tried just using :
credentials = aws_iam_role.snowflake_stage.arn
but got a different set of errors.
How do I combine the :
credentials = "AWS_ROLE='
with the
aws_iam_role.snowflake_stage.arn
then append the :
`)"
for the credentials value ?
First, you are missing closing " in encryption. It should be:
encryption = "(TYPE='AWS_SSE_KMS' KMS_KEY_ID='aws/key')"
Second, for the role:
credentials = "AWS_ROLE='${aws_iam_role.snowflake_stage.arn}'"
A bit late on this, but encryption should be:
encryption = "TYPE='AWS_SSE_KMS' KMS_KEY_ID='aws/key'"
rather than:
encryption = "(TYPE='AWS_SSE_KMS' KMS_KEY_ID='aws/key')"
Moreover using storage integration on it's own will be fine as long as you configure it with the appropriate role and permission on the role ( S3, KMS, and STS policy document)
Then you can get rid of encryption and credentials field.
Related
I have the following s3 bucket defined:
module "bucket" {
source = "terraform-aws-modules/s3-bucket/aws"
version = "3.1.0"
bucket = local.test-bucket-name
acl = null
grant = [{
type = "CanonicalUser"
permission = "FULL_CONTROL"
id = data.aws_canonical_user_id.current.id
}, {
type = "CanonicalUser"
permission = "FULL_CONTROL"
id = data.aws_cloudfront_log_delivery_canonical_user_id.cloudfront.id
}
]
object_ownership = "BucketOwnerPreferred"
}
But when I try to terraform apply this, I get the error:
Error: error updating S3 bucket ACL (logs,private): MissingSecurityHeader: Your request was missing a required header status code: 400
This error message is not very specific. Am I missing some type of header?
I came across the same issue.
I was trying to update an ACL on a bucket which had previously had private set as the ACL and then modifying my terraform code to match manually created entries on the ACL that someone had done via the GUI.
To get it working for me, I removed one of the ACL entries from the S3 bucket manually of which I was trying to add to the bucket and then re-ran the terraform and it worked without an error
I see the same error in cloudtrail also.
Its like you cant set private acl to null without adding an ACL entry
im trying to deploy cognito for opensearch via terraform. I have a manually built cognito working and ow trying to port it to terraform.
does anyone know how to set the below part?:
Choose role from token
role resolution 'DENY'
Terraform for the identity pool:
resource "aws_cognito_identity_pool" "cognito-identity-pool" {
identity_pool_name = "opensearch-${var.domain_name}-identity-pool"
allow_unauthenticated_identities = false
cognito_identity_providers {
client_id = aws_cognito_user_pool_client.cognito-user-pool-client.id
provider_name = aws_cognito_user_pool.cognito-user-pool.endpoint
}
}
ive tried adding server_side_token_check = false but no joy..
You need to use a different resource, namely aws_cognito_identity_pool_roles_attachment [1]. In order to achieve the same thing you see in the AWS console, you need to add the following block:
resource "aws_cognito_identity_pool_roles_attachment" "name" {
identity_pool_id = aws_cognito_identity_pool.cognito-identity-pool.id
roles = {
"authenticated" = <your-role-arn>
}
role_mapping {
ambiguous_role_resolution = "Deny"
type = "Token"
identity_provider = "${aws_cognito_user_pool.cognito-user-pool.endpoint}:${aws_cognito_user_pool_client.cognito-user-pool-client.id}"
}
}
Note that the roles block is required and the key can be authenticated or unathenticated. Additionally, you will probably have to figure out what kind of permissions the role will need and create it. The example in the documentation can be used as a blueprint. There are also other settings like mapping_rule block which might be of use to you, but since the details are lacking I omitted it from the answer.
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_identity_pool_roles_attachment
I am trying to upload a file in S3 by AWS Assume Role. When I am trying to access it from CLI it works fine but from .Net SDK it gives me Access Denied error.
Here are the steps I followed in CLI -
Setup the access key/secret key for user using aws configure
Assume the Role - “aws sts assume-role --role-arn "arn:aws:iam::1010101010:role/Test-Account-Role" --role-session-name AWSCLI-Session”
Take the access key / secret key / session token from the assumed role and setup an AWS profile. The credentials are printed out/returned from the assumed role.
Switch to the assume role profile: “set AWS_PROFILE=”
Verify that the user has the role: “aws sts get-caller-identity”
Access the bucket using ls or cp or rm command - Works Successfully.
Now I am trying to access it from .Net core App -
Here is the code snippet- Note that I am using same Access and Secret key as CLI from my local.
try
{
var region = RegionEndpoint.GetBySystemName(awsRegion);
SessionAWSCredentials tempCredentials = await GetTemporaryCredentialsAsync(awsAccessKey, awsSecretKey, region, roleARN);
//Use the temp credentials received to create the new client
IAmazonS3 client = new AmazonS3Client(tempCredentials, region);
TransferUtility utility = new TransferUtility(client);
// making a TransferUtilityUploadRequest instance
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest
{
BucketName = bucketName,
Key = $"{subFolder}/{fileName}",
FilePath = localFilePath
utility.Upload(request); //transfer
fileUploadedSuccessfully = true;
}
catch (AmazonS3Exception ex)
{
// HandleException
}
catch (Exception ex)
{
// HandleException
}
The method to get temp credentials is as follow - GetTemporaryCredentialsAsync
private static async Task<SessionAWSCredentials> GetTemporaryCredentialsAsync(string awsAccessKey, string awsSecretKey, RegionEndpoint region, string roleARN)
{
using (var stsClient = new AmazonSecurityTokenServiceClient(awsAccessKey, awsSecretKey, region))
{
var getSessionTokenRequest = new GetSessionTokenRequest
{
DurationSeconds = 7200
};
await stsClient.AssumeRoleAsync(
new AssumeRoleRequest()
{
RoleArn = roleARN,
RoleSessionName = "mySession"
});
GetSessionTokenResponse sessionTokenResponse =
await stsClient.GetSessionTokenAsync(getSessionTokenRequest);
Credentials credentials = sessionTokenResponse.Credentials;
var sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
return sessionCredentials;
}
}
I am getting back the temp credentials but it gives me Access Denied while uploading the file. Not sure if I am missing anything here.
Also noted that the token generated via SDK is shorter than that from CLI. I tried pasting these temp credentials to local profile and then tried to access the bucket and getting the Access Denied error then too.
There is an AWS .NET V3 example that shows this exact use case. To assume a role, you use a AmazonSecurityTokenServiceClient. In this example, the user assumes the role that allows the role to be used to list all S3 buckets. See this .NET scenario here.
https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/dotnetv3/IAM/IAM_Basics_Scenario/IAM_Basics_Scenario/IAM_Basics.cs
I am trying to create cloudtrail for an organization in AWS. When I try to run the plan on a targeted apply for
resource "aws_cloudtrail" "nfcisbenchmark" {
name = "nf-cisbenchmark-${terraform.workspace}"
s3_bucket_name = aws_s3_bucket.nfcisbenchmark_cloudtrail.id
enable_logging = var.enable_logging
# 3.2 Ensure CloudTrail log file validation is enabled (Automated)
enable_log_file_validation = var.enable_log_file_validation
# 3.1 Ensure CloudTrail is enabled in all regions (Automated)
is_multi_region_trail = var.is_multi_region_trail
include_global_service_events = var.include_global_service_events
is_organization_trail = "${local.environments[terraform.workspace] == "origin"? true : var.is_organization_trail}"
# 3.7 Ensure CloudTrail logs are encrypted at rest using KMS CMKs (Automated)
kms_key_id = aws_kms_key.nfcisbenchmark.arn
depends_on = [aws_s3_bucket.nfcisbenchmark_cloudtrail]
cloud_watch_logs_role_arn = aws_iam_role.cloudwatch.arn
cloud_watch_logs_group_arn = "${aws_cloudwatch_log_group.nfcisbenchmark.arn}:*"
event_selector {
# 3.11 Ensure that Object-level logging for read events is enabled for S3 bucket (Automated)
read_write_type = "All"
include_management_events = true
}
}
I get Error: Error creating CloudTrail: InvalidCloudWatchLogsLogGroupArnException: Access denied. Check the permissions for your role. Any help with this issue would be greatly appreciated.
Version 3.0.0 of the AWS provider bundled a breaking change to the aws_cloudwatch_log_group resource's ARN output by stripping the :* suffix returned previously. Instead you now have to explicitly add this in places where the AWS API wants the :* suffix. All of the documentation was then updated to follow this pattern as well which is why you see this in the aws_cloudtrail resource documentation:
resource "aws_cloudwatch_log_group" "example" {
name = "Example"
}
resource "aws_cloudtrail" "example" {
# ... other configuration ...
cloud_watch_logs_group_arn = "${aws_cloudwatch_log_group.example.arn}:*" # CloudTrail requires the Log Stream wildcard
}
For you though, on v2.6.0, your ARN already includes this :* so you don't need to add it an extra time but you do need to remember to strip the :* suffix on resources where the AWS API doesn't want that suffix (by the looks of this issue then the aws_datasync_task resource is one of those).
Alternatively you could update your AWS provider to > v3.0.0 and keep the suffix there which will help you with a lot of other potential issues in the future.
I'm working with aws via terraform.
I'm trying to create an IAM user with Access type of "Programmatic access".
With the AWS management console this is quite simple:
When trying with Terraform (reference to docs) it seems that only the following arguments are supported:
name
path
permissions_boundary
force_destroy
tags
Maybe this should be configured via a policy?
Any help will be appreciated.
(*) Related question with different scenario.
You can use aws_iam_access_key (https://www.terraform.io/docs/providers/aws/r/iam_access_key.html) terraform resource to create Access keys for the user and that should imply that user has Programmatic Access.
Hope this helps.
The aws_iam_user resource needs to also have an aws_iam_access_key resource created for it.
The iam-user module has a comprehensive example of using it.
You could also use that module straight from the registry and let that do everything for you.
If you dont want to encrypt and just looking for Access key & Secret key into plain text you can use this
main.tf
resource "aws_iam_access_key" "sagemaker" {
user = aws_iam_user.user.name
}
resource "aws_iam_user" "user" {
name = "user-name"
path = "/"
}
data "aws_iam_policy" "sagemaker_policy" {
arn = "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
}
resource "aws_iam_policy_attachment" "attach-policy" {
name = "sagemaker-policy-attachment"
users = [aws_iam_user.user.name]
policy_arn = data.aws_iam_policy.sagemaker_policy.arn
}
output.tf
output "secret_key" {
value = aws_iam_access_key.user.secret
}
output "access_key" {
value = aws_iam_access_key.user.id
}
you will get the Access key and secret key into the plain text you can directly use it.