I have full access to AWS account, see all keys that was created and used and need to know the person who owns the private keys. But AWS UI does no provide any information about Key Pairs except ID, Name, Type, Fingerprint
Is it possible to identify who created a certain key pair in AWS EC2?
If CloudTrail is setup it would have recorded the event which includes the user. The Following is lifted from the AWS Cloudtrail docs:
{"Records": [{
"eventVersion": "1.0",
"userIdentity": {
"type": "IAMUser",
"principalId": "EX_PRINCIPAL_ID",
"arn": "arn:aws:iam::123456789012:user/Alice",
"accountId": "123456789012",
"accessKeyId": "EXAMPLE_KEY_ID",
"userName": "Alice",
"sessionContext": {"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2014-03-06T15:15:06Z"
}}
},
"eventTime": "2014-03-06T17:10:34Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "CreateKeyPair",
"awsRegion": "us-east-2",
"sourceIPAddress": "72.21.198.64",
"userAgent": "EC2ConsoleBackend, aws-sdk-java/Linux/x.xx.fleetxen Java_HotSpot(TM)_64-Bit_Server_VM/xx",
"requestParameters": {"keyName": "mykeypair"},
"responseElements": {
"keyName": "mykeypair",
"keyFingerprint": "30:1d:46:d0:5b:ad:7e:1b:b6:70:62:8b:ff:38:b5:e9:ab:5d:b8:21",
"keyMaterial": "\u003csensitiveDataRemoved\u003e"
}
}]}
If cloudTrail was not enabled when the keypair was created i think you might be out of luck. If you have paid support you could ask the question to AWS or otherwise use the AWS forums - they may know more than me.
Related
I caught a misstake I have made in the way I have been uploading secrets to Secrets Manager. Through using cloudformation I have been sending in the secret as a plain text parameter into the template. The secret itself never gets exposed in the cloudformation yaml file. However, the secret is exposed as a parameter in cloudformation. Hence, being able to read/describe the stack is enough to get the secret.
Did some digging and found this. They suggest creating the parameter store/secret manager using cdk or cloudformation and after which you upload the secret using SDK/CLI.
To my question: does the SDK and CLI give traces themselves? Meaning, have I just moved the problem. Shifted from exposing the secret in cloudformation to exposing it to cloudtrail or any other monitoring in AWS.
How can I securely upload my own secrets in combination with IaC, without manually using the AWS console. Is there a way to turn of logging for certain SDK/CLI calls?
Depending on your use case there are different options:
If you set up new resources and need to create a new secret, you can have the SecretsManager generate the secret for you. See CloudFormation docs for the Secret resource.
If you want to store an existing secret, the option with the separate API-call is a good suggestion. The only place this could in principle be recorded is CloudTrail, which records any API-Call, but I have confirmed, that the secret value is not stored in the PutSecretValue event record.
A CreateSecret event from CloudTrail:
{
"eventVersion": "1.08",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDA2BFBC5RB4SDFSDQDI",
"arn": "arn:aws:iam::123456789123:user/myself",
"accountId": "123456789123",
"accessKeyId": "ASIA2BFSDFSD5RBR4L2JB7T",
"userName": "myself",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2021-07-05T11:38:38Z"
}
}
},
"eventTime": "2021-07-05T11:39:46Z",
"eventSource": "secretsmanager.amazonaws.com",
"eventName": "CreateSecret",
"awsRegion": "eu-central-1",
"sourceIPAddress": "95.48.10.191",
"userAgent": "aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.109-57.183.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.292-b10 java/1.8.0_292 vendor/Oracle_Corporation cfg/retry-mode/legacy",
"requestParameters": {
"name": "/demo",
"clientRequestToken": "5c59462b-d05c-4cfa-a224-a8d60f3edeff"
},
"responseElements": null,
"requestID": "6e61267a-ed8a-4383-8729-c33b8c217990",
"eventID": "23facc03-032c-4b24-bc36-d8f4e330445e",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"eventCategory": "Management",
"recipientAccountId": "123456789123",
"sessionCredentialFromConsole": "true"
}
A PutSecretValue event in CloudTrail:
{
"eventVersion": "1.08",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDA2BFSASB4SXNVRQDI",
"arn": "arn:aws:iam::123456789123:user/myself",
"accountId": "123456789123",
"accessKeyId": "ASIA2BFBSAWR4L2JB7T",
"userName": "myself",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2021-07-05T11:38:38Z"
}
}
},
"eventTime": "2021-07-05T11:40:09Z",
"eventSource": "secretsmanager.amazonaws.com",
"eventName": "PutSecretValue",
"awsRegion": "eu-central-1",
"sourceIPAddress": "11.11.190.191",
"userAgent": "aws-internal/3 aws-sdk-java/1.11.1030 Linux/5.4.109-57.183.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.292-b10 java/1.8.0_292 vendor/Oracle_Corporation cfg/retry-mode/legacy",
"requestParameters": {
"clientRequestToken": "61297703-b519-4e9e-8984-aacd40db826b",
"secretId": "/demo"
},
"responseElements": null,
"requestID": "97693f1b-f586-4641-af4c-b46d66fd27c1",
"eventID": "192f8959-3c51-40f5-8ca6-88f9075dc2a3",
"readOnly": false,
"eventType": "AwsApiCall",
"managementEvent": true,
"eventCategory": "Management",
"recipientAccountId": "123456789123",
"sessionCredentialFromConsole": "true"
}
How do I find out when the Instance size was changed using Cloudtrail? e.g. large -xlarge date, user and so on. One of the instance size have been changed and I'd like to find out which user has changed it
The Logging Amazon EC2, Amazon EBS, and Amazon VPC API Calls with AWS CloudTrail documentation states that:
All Amazon EC2, Amazon EBS, and Amazon VPC actions are logged by CloudTrail and are documented in the Amazon EC2 API Reference.
The Amazon EC2 API Reference shows that the action you're looking for is the one called ModifyInstanceAttribute; in your case you should target events that have "eventName": "ModifyInstanceAttribute" and have the instanceType key in the requestParameters object. The identity of the user (or role) that initiated the action is in userIdentity.
An example of such event in CloudTrail is:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "XXXXXXXXXXX",
"arn": "arn:aws:iam::XXXXXXXXXX:user/my_user",
"accountId": "XXXXXXXXXX",
"accessKeyId": "XXXXXXXXXX",
"userName": "my_user",
"sessionContext": {
"sessionIssuer": {},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2020-11-26T15:49:37Z"
}
}
},
"eventTime": "2020-11-26T16:54:18Z",
"eventSource": "ec2.amazonaws.com",
"eventName": "ModifyInstanceAttribute",
"awsRegion": "eu-west-1",
"sourceIPAddress": "111.22.33.444",
"userAgent": "console.ec2.amazonaws.com",
"requestParameters": {
"instanceId": "i-08999dedafc4xxyyz",
"instanceType": {
"value": "t3.nano"
}
},
"responseElements": {
"requestId": "11111111-2222-472f-ad77-bbeb506b242d",
"_return": true
},
"requestID": "11111111-2222-472f-ad77-bbeb506b242d",
"eventID": "aaaaaaa-c757-4501-8889-4f9d90720c0c",
"eventType": "AwsApiCall",
"recipientAccountId": "XXXXXXXXXX"
}
Toying around with the AWS CodeTrail dashboard and investigating e.g. who disabled or deleted a user, I frequently see crucial fields in the Event record obfuscated and the contents replaced with asterisks. For example here:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::444455556666:user/Alice",
"accountId": "444455556666",
"accessKeyId": "AKIAI44QH8DHBEXAMPLE",
"userName": "Alice",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::444455556666:role/service-role/mylambda-role-agr9xeqt",
"accountId": "444455556666",
"userName": "mylambda-role-agr9xeqt"
},
"webIdFederationData": {},
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2020-06-16T14:00:45Z"
}
}
},
"eventSource": "cognito-idp.amazonaws.com",
"eventName": "AdminDisableUser",
"awsRegion": "us-east-2",
"sourceIPAddress": "82.95.39.57",
"userAgent": "aws-sdk-nodejs/2.631.0 linux/v12.16.3 exec-env/AWS_Lambda_nodejs12.x promise",
"requestParameters": {
"userPoolId": "myPool",
"username": "******"
},
"responseElements": null,
"requestID": "17daa5e6-8646-8762-a619-1e61e70cc6d3",
"eventID": "9fd9b856-3467-4e67-8165-78cbe298792d",
"eventType": "AwsApiCall"
}
As you can see, "username" within "requestParameters" has been replaced by "******" in the event record of an AdminDisableUser call, while "userPoolId" gives the actual data. There are other things that might be obfuscated like this automatically in the "requestParameters". For example "userAttributes" for event records of AdminUpdateUserAttributes.
I am assuming that this behaviour is due to requirements by the AWS Well-Architected-Framework, that certain sensitive data is being obfuscated by default because it could be a security risk. Or is maybe my role or account being limited in some way so that I don't have access to this information?
In any case, assuming that I have complete and unlimited priviledges for the account, how can I enable that these fields in the event records are not obfuscated like that in CloudTrail but that it should show me the actual real data for the API call instead?
There are cases where CloudTrail obfuscates the username if the username could not be proven to be valid, and this is because of the high incidence of people typing passwords into username fields. If that's the case here, then you can't get access to the actual username.
I often get Cloudwatch Authorization alerts because the role attached to my SageMaker instance doesn't seem to have enough SSM (Systems Manager) permissions to UpdateInstanceInformation. My understanding is that the agent amazon-ssm-agent wants to hit an AWS API but fails to do so.
My Role has full SSM permissions:
{
"Action": [
"ssm:*",
"ssmmessages:*"
],
"Resource": "*",
"Effect": "Allow"
}
but the error persists:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "AssumedRole",
"principalId": "XXXXXXXXXXXXX:SageMaker",
"arn": "arn:aws:sts::XXXXXXXXXXXXX:assumed-role/sagemaker_prod_Notebook_Instance_Role/SageMaker",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "XXXXXXXXXXXXX",
"arn": "arn:aws:iam::XXXXXXXXXXXXX:role/sagemaker_prod_Notebook_Instance_Role",
"accountId": "XXXXXXXXXXXXX",
"userName": "sagemaker_prod_Notebook_Instance_Role"
}
},
"invokedBy": "im.amazonaws.com"
},
"eventSource": "ssm.amazonaws.com",
"eventName": "UpdateInstanceInformation",
"sourceIPAddress": "im.amazonaws.com",
"userAgent": "im.amazonaws.com",
"errorCode": "AccessDenied",
"errorMessage": "An unknown error occurred",
"requestParameters": {
"instanceId": "i-045f627a2d2e469b1",
"agentVersion": "2.3.714.0",
"platformType": "Linux",
"agentName": "amazon-ssm-agent"
},
"eventType": "AwsApiCall"
}
Has anyone seen this before ?
This is a bit late but I had a similar issue so I reached out to AWS Support and it seems to be a somewhat of a bug.
I was told that the AWS Sagemaker team has ssm installed by default. The Sagemaker notebook runs in an aws service account, although when a customer assigns Sagemaker a role in their own account the role cannot perform UpdateInstance information via the customer assigned role.
Support suggested I create a lifecycle config and leverage the following code sample to fix it:
https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html
https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/disable-uninstall-ssm-agent/on-start.sh
Is there a way to get a deeper history of when an access key was used, and for what service it was used?
If a key was used in multiple places, I can't be sure deactivating it is safe just because I know the last place it was used.
Yes pretty much possible with AWS CloudTrail.
Have a look at below link
Logging IAM Events with AWS CloudTrail
Example CloudTrail event
{
"eventVersion": "1.05",
"userIdentity": {
"type": "IAMUser",
"principalId": "AIDACKCEVSQ6C2EXAMPLE",
"arn": "arn:aws:iam::444455556666:user/Alice",
"accountId": "444455556666",
"accessKeyId": "AKIAI44QH8DHBEXAMPLE",
"userName": "Alice",
"sessionContext": {
"attributes": {
"mfaAuthenticated": "false",
"creationDate": "2014-07-15T21:39:40Z"
}
},
"invokedBy": "signin.amazonaws.com"
},
"eventTime": "2014-07-15T21:40:14Z",
"eventSource": "iam.amazonaws.com",
"eventName": "GetUserPolicy",
"awsRegion": "us-east-2",
"sourceIPAddress": "signin.amazonaws.com",
"userAgent": "signin.amazonaws.com",
"requestParameters": {
"userName": "Alice",
"policyName": "ReadOnlyAccess-Alice-201407151307"
},
"responseElements": null,
"requestID": "9EXAMPLE-0c68-11e4-a24e-d5e16EXAMPLE",
"eventID": "cEXAMPLE-127e-4632-980d-505a4EXAMPLE"
}
From above event information, you can determine that the request was
made to get a user policy named ReadOnlyAccess-Alice-201407151307 for
user Alice, as specified in the requestParameters element. You can
also see that the request was made by an IAM user named Alice on July
15, 2014 at 9:40 PM (UTC). In this case, the request originated in the
AWS Management Console, as you can tell from the userAgent element.