Securely upload secrets to Secret Manager/Parameter Store - amazon-web-services

I caught a misstake I have made in the way I have been uploading secrets to Secrets Manager. Through using cloudformation I have been sending in the secret as a plain text parameter into the template. The secret itself never gets exposed in the cloudformation yaml file. However, the secret is exposed as a parameter in cloudformation. Hence, being able to read/describe the stack is enough to get the secret.
Did some digging and found this. They suggest creating the parameter store/secret manager using cdk or cloudformation and after which you upload the secret using SDK/CLI.
To my question: does the SDK and CLI give traces themselves? Meaning, have I just moved the problem. Shifted from exposing the secret in cloudformation to exposing it to cloudtrail or any other monitoring in AWS.
How can I securely upload my own secrets in combination with IaC, without manually using the AWS console. Is there a way to turn of logging for certain SDK/CLI calls?

Depending on your use case there are different options:
If you set up new resources and need to create a new secret, you can have the SecretsManager generate the secret for you. See CloudFormation docs for the Secret resource.
If you want to store an existing secret, the option with the separate API-call is a good suggestion. The only place this could in principle be recorded is CloudTrail, which records any API-Call, but I have confirmed, that the secret value is not stored in the PutSecretValue event record.
A CreateSecret event from CloudTrail:
{
"eventVersion": "1.08",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDA2BFBC5RB4SDFSDQDI",
"arn": "arn:aws:iam::123456789123:user/myself",
"accountId": "123456789123",
"accessKeyId": "ASIA2BFSDFSD5RBR4L2JB7T",
"userName": "myself",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2021-07-05T11:38:38Z"
}
}
},
"eventTime": "2021-07-05T11:39:46Z",
"eventSource": "secretsmanager.amazonaws.com",
"eventName": "CreateSecret",
"awsRegion": "eu-central-1",
"sourceIPAddress": "95.48.10.191",
"userAgent": "aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.109-57.183.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.292-b10 java/1.8.0_292 vendor/Oracle_Corporation cfg/retry-mode/legacy",
"requestParameters": {
"name": "/demo",
"clientRequestToken": "5c59462b-d05c-4cfa-a224-a8d60f3edeff"
},
"responseElements": null,
"requestID": "6e61267a-ed8a-4383-8729-c33b8c217990",
"eventID": "23facc03-032c-4b24-bc36-d8f4e330445e",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"eventCategory": "Management",
"recipientAccountId": "123456789123",
"sessionCredentialFromConsole": "true"
}
A PutSecretValue event in CloudTrail:
{
"eventVersion": "1.08",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDA2BFSASB4SXNVRQDI",
"arn": "arn:aws:iam::123456789123:user/myself",
"accountId": "123456789123",
"accessKeyId": "ASIA2BFBSAWR4L2JB7T",
"userName": "myself",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2021-07-05T11:38:38Z"
}
}
},
"eventTime": "2021-07-05T11:40:09Z",
"eventSource": "secretsmanager.amazonaws.com",
"eventName": "PutSecretValue",
"awsRegion": "eu-central-1",
"sourceIPAddress": "11.11.190.191",
"userAgent": "aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.109-57.183.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.292-b10 java/1.8.0_292 vendor/Oracle_Corporation cfg/retry-mode/legacy",
"requestParameters": {
"clientRequestToken": "61297703-b519-4e9e-8984-aacd40db826b",
"secretId": "/demo"
},
"responseElements": null,
"requestID": "97693f1b-f586-4641-af4c-b46d66fd27c1",
"eventID": "192f8959-3c51-40f5-8ca6-88f9075dc2a3",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"eventCategory": "Management",
"recipientAccountId": "123456789123",
"sessionCredentialFromConsole": "true"
}

Related

Determine the exact problem point of calling AWS Backup to AWS KMS in order to decrypt S3 bucket

Here is a log in AWS CloudTrail presumably indicating that
the AWS Backup service ("userAgent": "backup.amazonaws.com",)
tries to call the AWS KMS service("eventSource": "kms.amazonaws.com",)
in order to decrypt("eventName": "Decrypt",)
some S3 bucket("principalId": "AROAY6JAXY37VKF726QCA:AWS_BACKUP_S3_71C3048C",) and fails.
But there are no clues as to which particular bucket or KMS key caused the problem, I'd appreciate any advice on how to determine the exact problem point, thanks.
BTW the account contains hundreds of keys and buckets that are being back-up, so to determine the problem point in some manual way like, like researching each bucket/key, is very undesirable:)
{
"eventVersion": "1.08",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAY6JAXY37VKF726QCA:AWS_BACKUP_S3_71C3048C",
"arn": "arn:aws:sts::****:assumed-role/AWSBackupDefaultServiceRole/AWS_BACKUP_S3_71C3048C",
"accountId": "****",
"accessKeyId": "ASIAY6JAXY37WHC7VFWN",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AROAY6JAXY37VKF726QCA",
"arn": "arn:aws:iam::****:role/service-role/AWSBackupDefaultServiceRole",
"accountId": "****",
"userName": "AWSBackupDefaultServiceRole"
},
"webIdFederationData": {},
"attributes": {
"creationDate": "2022-12-18T05:54:35Z",
"mfaAuthenticated": "false"
}
},
"invokedBy": "backup.amazonaws.com"
},
"eventTime": "2022-12-18T05:54:35Z",
"eventSource": "kms.amazonaws.com",
"eventName": "Decrypt",
"awsRegion": "eu-central-1",
"sourceIPAddress": "backup.amazonaws.com",
"userAgent": "backup.amazonaws.com",
"errorCode": "AccessDenied",
"errorMessage": "The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.",
"requestParameters": null,
"responseElements": null,
"requestID": "660b1b90-66d9-4a2b-9ab1-4af4e195b71f",
"eventID": "ef6fe50b-67f9-46c6-bbf5-057de88c3c1c",
"readOnly": true,
"eventType": "AwsApiCall",
"managementEvent": true,
"recipientAccountId": "****",
"eventCategory": "Management"
}

Instace Type Change log

How do I find out when the Instance size was changed using Cloudtrail? e.g. large -xlarge date, user and so on. One of the instance size have been changed and I'd like to find out which user has changed it
The Logging Amazon EC2, Amazon EBS, and Amazon VPC API Calls with AWS CloudTrail documentation states that:
All Amazon EC2, Amazon EBS, and Amazon VPC actions are logged by CloudTrail and are documented in the Amazon EC2 API Reference.
The Amazon EC2 API Reference shows that the action you're looking for is the one called ModifyInstanceAttribute; in your case you should target events that have "eventName": "ModifyInstanceAttribute" and have the instanceType key in the requestParameters object. The identity of the user (or role) that initiated the action is in userIdentity.
An example of such event in CloudTrail is:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "XXXXXXXXXXX",
"arn": "arn:aws:iam::XXXXXXXXXX:user/my_user",
"accountId": "XXXXXXXXXX",
"accessKeyId": "XXXXXXXXXX",
"userName": "my_user",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2020-11-26T15:49:37Z"
}
}
},
"eventTime": "2020-11-26T16:54:18Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "ModifyInstanceAttribute",
"awsRegion": "eu-west-1",
"sourceIPAddress": "111.22.33.444",
"userAgent": "console.ec2.amazonaws.com",
"requestParameters": {
"instanceId": "i-08999dedafc4xxyyz",
"instanceType": {
"value": "t3.nano"
}
},
"responseElements": {
"requestId": "11111111-2222-472f-ad77-bbeb506b242d",
"_return": true
},
"requestID": "11111111-2222-472f-ad77-bbeb506b242d",
"eventID": "aaaaaaa-c757-4501-8889-4f9d90720c0c",
"eventType": "AwsApiCall",
"recipientAccountId": "XXXXXXXXXX"
}

How to fix AWS Config generating AccessDenied error?

I am trying to allow AWS Config to write to a non-public S3 bucket.
Based on the official documentation, I should have two policies assigned to the AWS role. However, It is not possible to add any policy to the service-linked role, neither to create a custom new service-linked role for AWS config.
As such, how can I stop receiving the S3 AccessDenied error without making the bucket public?
edit: here is the error log:
{
"eventVersion": "1.07",
"userIdentity": {
"type": "AssumedRole",
"principalId": "xxxxxxxxxxxxxxxxxxxxx:AWSConfig-BucketConfigCheck",
"arn": "arn:aws:sts::xxxxxxxxxxxx:assumed-role/AWSServiceRoleForConfig/AWSConfig-BucketConfigCheck",
"accountId": "xxxxxxxxxxxx",
"accessKeyId": "xxxxxxxxxxxxxxxxxxxx",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "xxxxxxxxxxxxxxxxxxxxx",
"arn": "arn:aws:iam::xxxxxxxxxxxx:role/aws-service-role/config.amazonaws.com/AWSServiceRoleForConfig",
"accountId": "xxxxxxxxxxxx",
"userName": "AWSServiceRoleForConfig"
},
"attributes": {
"creationDate": "2020-04-30T00:43:57Z",
"mfaAuthenticated": "false"
}
},
"invokedBy": "AWS Internal"
},
"eventTime": "2020-04-30T00:43:57Z",
"eventSource": "s3.amazonaws.com",
"eventName": "PutObject",
"awsRegion": "eu-west-1",
"sourceIPAddress": "xxx.xxx.xxx.xxx",
"userAgent": "[AWSConfig]",
"errorCode": "AccessDenied",
"errorMessage": "Access Denied",
"requestParameters": {
"bucketName": "aws-config-bucket-xxxxxxxxxxxx",
"Host": "aws-config-bucket-xxxxxxxxxxxx.s3.eu-west-1.amazonaws.com",
"x-amz-acl": "bucket-owner-full-control",
"x-amz-server-side-encryption": "AES256",
"key": "AWSLogs/xxxxxxxxxxxx/Config/ConfigWritabilityCheckFile"
},
"responseElements": null,
"additionalEventData": {
"SignatureVersion": "SigV4",
"CipherSuite": "ECDHE-RSA-AES128-SHA",
"bytesTransferredIn": 0,
"AuthenticationMethod": "AuthHeader",
"x-amz-id-2": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=",
"bytesTransferredOut": 243
},
"requestID": "xxxxxxxxxxxxxxxx",
"eventID": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"readOnly": false,
"resources": [
{
"type": "AWS::S3::Object",
"ARN": "arn:aws:s3:::aws-config-bucket-xxxxxxxxxxxx/AWSLogs/xxxxxxxxxxxx/Config/ConfigWritabilityCheckFile"
},
{
"accountId": "xxxxxxxxxxxx",
"type": "AWS::S3::Bucket",
"ARN": "arn:aws:s3:::aws-config-bucket-xxxxxxxxxxxx"
}
],
"eventType": "AwsApiCall",
"managementEvent": false,
"recipientAccountId": "xxxxxxxxxxxx",
"vpcEndpointId": "vpce-xxxxxxxx",
"eventCategory": "Data"
}
I found the answer here: https://forums.aws.amazon.com/thread.jspa?threadID=314156
When AWS Config sends configuration information to an Amazon S3
bucket in another account, it first attempts to use the IAM role, but
this attempt fails if the access policy for the bucket does not grant
WRITE access to the IAM role. In this event, AWS Config sends the
information again, this time as the AWS Config service principal.
I checked my logs and there was an AWS Config service principal log, the same second as the AccessDenied, that was being accepted. Therefore, the error can be safely ignored. I have updated my Cloudwatch alarm to ignore it:
{($.errorCode="*UnauthorizedOperation") || (($.errorCode="AccessDenied*") && (($.userIdentity.type!="AssumedRole") || ($.userAgent!="[AWSConfig]")))}

Tracking IAM Access Key Usage Beyond "Last Used"

Is there a way to get a deeper history of when an access key was used, and for what service it was used?
If a key was used in multiple places, I can't be sure deactivating it is safe just because I know the last place it was used.
Yes pretty much possible with AWS CloudTrail.
Have a look at below link
Logging IAM Events with AWS CloudTrail
Example CloudTrail event
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::444455556666:user/Alice",
"accountId": "444455556666",
"accessKeyId": "AKIAI44QH8DHBEXAMPLE",
"userName": "Alice",
"sessionContext": {
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2014-07-15T21:39:40Z"
}
},
"invokedBy": "signin.amazonaws.com"
},
"eventTime": "2014-07-15T21:40:14Z",
"eventSource": "iam.amazonaws.com",
"eventName": "GetUserPolicy",
"awsRegion": "us-east-2",
"sourceIPAddress": "signin.amazonaws.com",
"userAgent": "signin.amazonaws.com",
"requestParameters": {
"userName": "Alice",
"policyName": "ReadOnlyAccess-Alice-201407151307"
},
"responseElements": null,
"requestID": "9EXAMPLE-0c68-11e4-a24e-d5e16EXAMPLE",
"eventID": "cEXAMPLE-127e-4632-980d-505a4EXAMPLE"
}
From above event information, you can determine that the request was
made to get a user policy named ReadOnlyAccess-Alice-201407151307 for
user Alice, as specified in the requestParameters element. You can
also see that the request was made by an IAM user named Alice on July
15, 2014 at 9:40 PM (UTC). In this case, the request originated in the
AWS Management Console, as you can tell from the userAgent element.

Create CloudWatch Alarm to notify about setting a S3 object to public

I want to create on CloudWatch a metric filter and an alarm based on it to notify me about S3 events, specially when a file or a bucket is set to public. This is the metric filter I used to create the metric:
{ ($.eventSource = s3.amazonaws.com) && (($.eventName = PutBucketAcl)
|| ($.eventName = PutObjectAcl)) &&
(($.requestParameters.AccessControlPolicy.AccessControlList.Grant.Grantee.type
= Group ))}
I tested this pattern by putting the following Custom log data :
{
"Records": [
{
"eventVersion": "1.03",
"userIdentity": {
"type": "IAMUser",
"principalId": "111122223333",
"arn": "arn:aws:iam::111122223333:user/myUserName",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"userName": "myUserName"
},
"eventTime": "2015-08-26T20:46:31Z",
"eventSource": "s3.amazonaws.com",
"eventName": "DeleteBucketPolicy",
"awsRegion": "us-west-2",
"sourceIPAddress": "127.0.0.1",
"userAgent": "[]",
"requestParameters": {
"bucketName": "myawsbucket"
},
"responseElements": null,
"requestID": "47B8E8D397DCE7A6",
"eventID": "cdc4b7ed-e171-4cef-975a-ad829d4123e8",
"eventType": "AwsApiCall",
"recipientAccountId": "111122223333"
},
{
"eventVersion": "1.03",
"userIdentity": {
"type": "IAMUser",
"principalId": "111122223333",
"arn": "arn:aws:iam::111122223333:user/myUserName",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"userName": "myUserName"
},
"eventTime": "2015-08-26T20:46:31Z",
"eventSource": "s3.amazonaws.com",
"eventName": "PutBucketAcl",
"awsRegion": "us-west-2",
"sourceIPAddress": "",
"userAgent": "[]",
"requestParameters": {
"bucketName": "",
"AccessControlPolicy": {
"AccessControlList": {
"Grant": {
"Grantee": {
"xsi:type": "Group",
"xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"ID": "d25639fbe9c19cd30a4c0f43fbf00e2d3f96400a9aa8dabfbbebe1906Example"
},
"Permission": "FULL_CONTROL"
}
},
"xmlns": "http://s3.amazonaws.com/doc/2006-03-01/",
"Owner": {
"ID": "d25639fbe9c19cd30a4c0f43fbf00e2d3f96400a9aa8dabfbbebe1906Example"
}
}
},
"responseElements": null,
"requestID": "BD8798EACDD16751",
"eventID": "607b9532-1423-41c7-b048-ec2641693c47",
"eventType": "AwsApiCall",
"recipientAccountId": "111122223333"
},
{
"eventVersion": "1.03",
"userIdentity": {
"type": "IAMUser",
"principalId": "111122223333",
"arn": "arn:aws:iam::111122223333:user/myUserName",
"accountId": "111122223333",
"accessKeyId": "AKIAIOSFODNN7EXAMPLE",
"userName": "myUserName"
},
"eventTime": "2015-08-26T20:46:31Z",
"eventSource": "s3.amazonaws.com",
"eventName": "GetBucketVersioning",
"awsRegion": "us-west-2",
"sourceIPAddress": "",
"userAgent": "[]",
"requestParameters": {
"bucketName": "myawsbucket"
},
"responseElements": null,
"requestID": "07D681279BD94AED",
"eventID": "f2b287f3-0df1-4961-a2f4-c4bdfed47657",
"eventType": "AwsApiCall",
"recipientAccountId": "111122223333"
}
]
}
I clicked Test Pattern and I get this message:
Results Found 0 matches out of 50 event(s) in the sample log.
Is the metric filter proper and correct ? I'm supposed to have one result but it is not coming up.
Calculating whether a policy is providing open access is quite complex, due to the many ways that rules can be specified in the Bucket Policy (for example, wildcards can provide access).
An easier approach would be to use the Amazon S3 Bucket Permissions check in Trusted Advisor:
Checks buckets in Amazon Simple Storage Service (Amazon S3) that have open access permissions or allow access to any authenticated AWS user.
You can then Monitor Trusted Advisor Check Results with Amazon CloudWatch Events.
However, that particular check is not included in the Free Tier for Trusted Advisor. You would need to be on a Support Plan for that check to operate.
The Amazon S3 console was also recently updated -- it now clearly shows any buckets with public permissions.