I am trying to figure out usage of an AD user, using AWS via AssumeRoleWithSAML, following this ink, https://aws.amazon.com/blogs/security/how-to-easily-identify-your-federated-users-by-using-aws-cloudtrail/.
However, i dont see AssumeRoleWithSAML event at all in my Cloudtrails, though i can clearly see activity from this user. I went all the way to early July in cloudtrail to look up AssumeRoleWithSaml and dont see any event.
Am i missing something? Bcos of this event not coming, i am not able to correlate what this user is doing in AWS.
Thanks
Amit
You are right, there should be an event with name AssumeRoleWithSAML in the CloudTrail logs.
You already referenced the correct AWS security blog post which describes how to "identify a SAML federated user". [1]
Let's go into detail.
The IAM docs [2] contain an example how the AssumeRoleWithSAML event should look like:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "WebIdentityUser",
"principalId": "accounts.google.com:[id-of-application].apps.googleusercontent.com:[id-of-user]",
"userName": "[id of user]",
"identityProvider": "accounts.google.com"
},
"eventTime": "2016-03-23T01:39:51Z",
"eventSource": "sts.amazonaws.com",
"eventName": "AssumeRoleWithWebIdentity",
"awsRegion": "us-east-2",
"sourceIPAddress": "192.0.2.101",
"userAgent": "aws-cli/1.3.23 Python/2.7.6 Linux/2.6.18-164.el5",
"requestParameters": {
"durationSeconds": 3600,
"roleArn": "arn:aws:iam::444455556666:role/FederatedWebIdentityRole",
"roleSessionName": "MyAssignedRoleSessionName"
},
"responseElements": {
"provider": "accounts.google.com",
"subjectFromWebIdentityToken": "[id of user]",
"audience": "[id of application].apps.googleusercontent.com",
"credentials": {
"accessKeyId": "ASIACQRSTUVWRAOEXAMPLE",
"expiration": "Mar 23, 2016 2:39:51 AM",
"sessionToken": "[encoded session token blob]"
},
"assumedRoleUser": {
"assumedRoleId": "AROACQRSTUVWRAOEXAMPLE:MyAssignedRoleSessionName",
"arn": "arn:aws:sts::444455556666:assumed-role/FederatedWebIdentityRole/MyAssignedRoleSessionName"
}
},
"resources": [
{
"ARN": "arn:aws:iam::444455556666:role/FederatedWebIdentityRole",
"accountId": "444455556666",
"type": "AWS::IAM::Role"
}
],
"requestID": "6EXAMPLE-e595-11e5-b2c7-c974fEXAMPLE",
"eventID": "bEXAMPLE-0b30-4246-b28c-e3da3EXAMPLE",
"eventType": "AwsApiCall",
"recipientAccountId": "444455556666"
}
As we can see, the requestParameters contain an element durationSeconds which is the value you are looking for.
Why is the event missing?
First of all, it is necessary to know if you are using the AWS CloudTrail Console or if you are parsing the CloudTrail files which were delivered to the S3 bucket. If you use the CloudTrail console, you are able the view the last 90 days of recorded API activity and events in an AWS Region only!! [3]
So make sure that you use AWS Athena or another solution if you must go further back in time.
You must look into the trail of the correct region! You do this by inspecting the respective S3 prefix for a multi-region trail or by clicking onto the desired region in the top right corner if you use the AWS CloudTrail Console. This is important because regional services are logging to their respective trail!! AWS mentions this as follows:
If you activate AWS STS endpoints in Regions other than the default global endpoint, then you must also turn on CloudTrail logging in those Regions. This is necessary to record any AWS STS API calls that are made in those Regions. For more information, see Turning On CloudTrail in Additional Regions in the AWS CloudTrail User Guide. [4]
Make sure to look into the correct account! You must inspect the trail of the account whose role was assumed. I mention this explicitly because there are multi-account environments which might use centralized identity accounts etc.
References
[1] https://aws.amazon.com/de/blogs/security/how-to-easily-identify-your-federated-users-by-using-aws-cloudtrail/
[2] https://docs.aws.amazon.com/IAM/latest/UserGuide/cloudtrail-integration.html
[3] https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events-console.html
[4] https://docs.aws.amazon.com/IAM/latest/UserGuide/cloudtrail-integration.html
Related
Question
How to know the event format coming to Lambda from AWS services?
The AWS Lambda console includes a Test function, which can provide a sample event for most of the events that are generated by AWS.
You can modify these sample events to include your specific data.
For example, the Amazon S3 Put sample event simulates a new object being added to an Amazon S3 bucket. You can modify the event to include your own Bucket and Object names, then use it to test the function without actually using Amazon S3.
Generating Sample Event Payloads
$ sam local generate-event --help
Usage: sam local generate-event [OPTIONS] COMMAND [ARGS]...
You can use this command to generate sample payloads from different event
sources such as S3, API Gateway, and SNS. These payloads contain the
information that the event sources send to your Lambda functions.
Commands:
alexa-skills-kit
alexa-smart-home
apigateway
batch
cloudformation
cloudfront
cloudwatch
codecommit
codepipeline
cognito
config
connect
dynamodb
kinesis
lex
rekognition
s3
sagemaker
ses
sns
sqs
stepfunctions
S3 put
$ sam local generate-event s3 put
{
"Records": [
{
"eventVersion": "2.0",
"eventSource": "aws:s3",
"awsRegion": "us-east-1",
"eventTime": "1970-01-01T00:00:00.000Z",
"eventName": "ObjectCreated:Put",
"userIdentity": {
"principalId": "EXAMPLE"
},
"requestParameters": {
"sourceIPAddress": "127.0.0.1"
},
"responseElements": {
"x-amz-request-id": "EXAMPLE123456789",
"x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH"
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "testConfigRule",
"bucket": {
"name": "example-bucket",
"ownerIdentity": {
"principalId": "EXAMPLE"
},
"arn": "arn:aws:s3:::example-bucket"
},
"object": {
"key": "test/key",
"size": 1024,
"eTag": "0123456789abcdef0123456789abcdef",
"sequencer": "0A1B2C3D4E5F678901"
}
}
}
]
}
Firehose
$ sam local generate-event kinesis kinesis-firehose
{
"invocationId": "invocationIdExample",
"deliveryStreamArn": "arn:aws:kinesis:EXAMPLE",
"region": "us-east-1",
"records": [
{
"recordId": "49546986683135544286507457936321625675700192471156785154",
"approximateArrivalTimestamp": 1495072949453,
"data": "SGVsbG8sIHRoaXMgaXMgYSB0ZXN0IDEyMy4="
}
]
}
Update
As per the comment by #John Rotenstein, Lambda console can generate much more sample events.
For Go lang, aws-lambda-go/events/, provides a list of sample codes showing how to handle events from different sources and sample test data. Good resource to have a look.
Just print it out the first time you start your development. For python, the command is:
print(json.dumps(event))
The output should be available in CloudWatch log group for your lambda. This is must useful for debugging and testing with real-live events.
Sometimes, in documentation you can also find it. But I found that just printing it out is the fastest and most reliable way to get to know the event format.
I am trying to configure a Cloudwatch Event Rule (to trigger an SNS notification) for whenever
someone assumes a particular role:
{
"detail": {
"eventName": [
"AssumeRole"
],
"eventSource": [
"sts.amazonaws.com"
],
"requestParameters": {
"roleArn": [
"arn:aws:iam::0000:role/the_role_name"
]
}
},
"detail-type": [
"AWS API Call via CloudTrail"
]
}
Where 0000 is the account id and the_role_name is the role I want to alert on.
This is failing to trigger any notification, however when I search in Cloudtrail Insights for the
events:
filter eventName = 'AssumeRole'
| filter requestParameters.roleArn =~ 'the_role_name'
| sort #timestamp desc
| display #timestamp, requestParameters.roleSessionName, eventName, requestParameters.roleArn, userAgent, sourceIPAddress
I DO get results that SHOULD have triggered the rule:
requestParameters.roleSessionName eventName requestParameters.roleArn
my_username AssumeRole arn:aws:iam::0000:role/the_role_name
...
For the sake of trying to dumb things down and catch a broader set of events, I also tried the
following Rule (which would catch all AssumeRole events to any role):
{
"detail": {
"eventName": [
"AssumeRole"
]
},
"detail-type": [
"AWS API Call via CloudTrail"
]
}
This rule also is failing to trigger.
Does anyone have ideas on how to configure Cloudwatch Event Rules to trigger on AssumeRole events?
I read through this related question (which is trying to achieve something similar), but it did not have a solution: AWS CloudWatch Events trigger SNS on STS role assuming for cross account
First of all make sure whether the event is invoked or not by checking the monitoring metrics for the rule. It is possible that it is triggered, but it fails to invoke the target. In this case, you should check your IAM policies.
If it is not triggered, there could be issues with trail delivery to Cloudwatch Logs. Make sure that you created a trail in the same region, which delivers events to Cloudwatch Logs.
I've the following rule in us-east-1 region, which works fine:
{
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"sts.amazonaws.com"
],
"eventName": [
"AssumeRole"
]
},
"source": [
"aws.sts"
]
}
According an an AWS Support agent I was speaking with yesterday, and also indicated by the linked documents, Eventbridge Rules (formerly Cloudwatch Event Rules) unfortunately do not support STS events.
What's perplexing about this and might lead you down a wrong path, as it did me, is that the sts test-event-pattern api will in fact validate your event against a valid pattern and give no indication that it's an unsupported service.
Hopefully AWS adds STS event support in the future.
https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-service-event.html
When i create the ec2 instance the cloud watch event should trigger, But it is not working.. Below is the event Json. Could you please help.
The same event is triggered successful in other aws account, I not sure what is wrong with this account. IAM policy to lambda has complete access and when i test lambda code it is working fine.. But the problem is the event should be triggered.
{
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"ec2.amazonaws.com",
"rds.amazonaws.com",
"s3.amazonaws.com"
],
"eventName": [
"CreateVolume",
"RunInstances",
"RequestSpotInstances",
"CreateSnapshot",
"CreateVpnGateway",
"CreateSecurityGroup",
"CreateInternetGateway",
"CreateNatGateway",
"CreateVpc",
"CreateSubnet",
"CreateRouteTable",
"CreateCustomerGateway",
"CreateNetworkAcl",
"CreateVpnConnection",
"CreateImage",
"CreateDBInstance",
"CreateDBSnapshot",
"CreateDBCluster",
"CreateDBClusterParameterGroup",
"CreateDBClusterSnapshot",
"CreateDBInstanceReadReplica",
"CreateDBParameterGroup",
"CreateDBSecurityGroup",
"CreateDBSubnetGroup",
"CreateEventSubscription",
"CreateOptionGroup",
"CreateBucket"
]
}
}
Are you configuring Trails from AWS CloudTrail to track API Calls?
1) Check on your Trails configuration if the Logging status is enabled or not.
2) Check again on your policies as most of the cases, we mis-configured our IAM policies so one of the services cannot work as expected.
I am starting migrating our secrets from AWS parameter store to AWS secret manager, currently I am facing a problem that I don't how to solve, could anyone provide any insight?
We have a AWS account (let's call it identity account), which we manage all IAM users and groups. And we have another AWS account which hosts our infra (let's call this infra account). We'd like to manage all users in the identity account and let user assumes to poweruser role in the infra account so we can manage all users at one place.
In the infra account, we have RDS running and we want to create DB users for our developers so they can login to database for debugging purpose, but we also want to audit what they have done in case someone did something bad to our database, so we need to create one DB user per developer. All of those DB credentials are saved into AWS secret with a naming convention like
/dev/rds/mysql/users/foo
/dev/rds/mysql/users/bar
So here is the question: how can I manage user's IAM policy to restrict the permission of user so they can ONLY access their own secret? From this AWS doc that we CANNOT get aws:username when user is using assumed role to access AWS, thus the following policy would never work
actions = [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:UpdateSecretVersionStage"
]
resources = [
"arn:aws:secretsmanager:us-east-1:12345678912:secret:/dev/rds/mysql/users/${aws:username}"
]
The only IAM variable that I can use for assumed role is aws:userid but it would be something like this (assume user foo's username in identity account is foo#emaildomain.com)
"AROAJGHLP6KERYI375PJY:foo#emaildomain.com"
It also looks like that the role-id (AROAJGHLP6KERYI375PJY in this example) is random, with prefix AROA, which means I CANNOT use following policy either (and plus, having AROAJGHLP6KERYI375PJY:foo#emaildomain.com as a secret name in secret manager is pretty ugly)
actions = [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:UpdateSecretVersionStage"
]
resources = [
"arn:aws:secretsmanager:us-east-1:12345678912:secret:/dev/rds/mysql/users/${aws:userid}"
]
Currently my policy ended up with this
actions = [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:UpdateSecretVersionStage"
]
resources = [
"arn:aws:secretsmanager:us-east-1:12345678912:secret:/dev/rds/mysql/users/*"
]
which means as long as user assumed to infra account, they have access to other developers DB credentials as well.
I've looked into CloudWatch metric to see if I can setup a filter to filter out the API call that user foo is calling GetSecretValue API to get user bar's credential, but CloudWatch filter doesn't support user REGEX to extract certain value from JSON. Here's the example of the GetSecretValue event from CloudTrail log:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "AssumedRole",
"principalId": "AROAJGHLP6KERYI375PJY:foo#emaildomain.com",
"arn": "arn:aws:sts::12345678912:assumed-role/poweruser/foo#emaildomain.com",
"accountId": "12345678912",
"accessKeyId": "ASIATA5XIF7AFC2CQ7NO",
"sessionContext": {
"attributes": {
"mfaAuthenticated": "true",
"creationDate": "2018-07-11T21:31:20Z"
},
"sessionIssuer": {
"type": "Role",
"principalId": "AROAJGHLP6KERYI375PJY",
"arn": "arn:aws:iam::12345678912:role/poweruser",
"accountId": "12345678912",
"userName": "poweruser"
}
}
},
"eventTime": "2018-07-11T21:32:56Z",
"eventSource": "secretsmanager.amazonaws.com",
"eventName": "GetSecretValue",
"awsRegion": "us-east-2",
"sourceIPAddress": "1.2.3.4",
"userAgent": "aws-internal/3",
"requestParameters": {
"secretId": "/dev/rds/mysql/users/foo"
},
"responseElements": null,
"requestID": "f98ad2c2-8551-11e8-8a3f-751b0a8a6ca5",
"eventID": "73b8de89-bc8c-41a3-a172-58dd8d79a026",
"eventType": "AwsApiCall",
"recipientAccountId": "12345678912"
}
If I can extract foo#emaildomain.com from { $.userIdentity.principalId } and extract foo from to { $.requestParameters} then I can try some magic to compare foo#emaildomain.com == foo to trigger alert if user is trying to get other people's credential, but, I can't...
So, in this case, how could I manage my policy to lock users' permission?
From your first example, for each user, can you try to hardcode the secret in their user iam policy?
So for user foo, the user policy would look something like
...
actions = [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:UpdateSecretVersionStage"
]
resources = [
"arn:aws:secretsmanager:us-east-1:12345678912:secret:/dev/rds/mysql/users/foo"
]
...
You would have the same structure for bar, ...
If you have many users, this solution wouldn't scale
The problem seems to be that you are using a common assumed role for all the users. An alternative to this would be to create a resource policy on each secret that granted access permissions to the owning user in the identity account. This would let the users access the secret directly without calling assumerole. This would not prevent them from still assuming the infra account poweruser role and accessing the secret, so you would either have to drop Secrets Manager privileges from the role, or explicitly deny the infra power user in the resource policy you add to the secret.
Setting up cross account access like this also means you can not use the default KMS encryption key. You will need to setup a custom KMS key that grants the correct access permission to the identity account and re-encrypt the secrets with that new key.
Since resource policies and custom KMS keys can not be setup in the Secrets Manager Console, this all requires using the CLI or one of the SDKs.
I have a cross-account architecture and I'm setting up a CloudWatch event for the STS role assuming into another account. I have CloudTrail enabled on the account, the logs from CloudTrail are stored in a separate accounts s3 bucket. The SNS feeds into SES to send an email upon assumption.
For some reason, this event pattern won't trigger when the role is assumed! Any ideas?
{
"source": [
"aws.sts"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": ["sts.amazonaws.com"],
"eventName": ["AssumeRole"],
"requestParameters": {
"roleArn": ["arn:aws:iam::1111111111:role/RoleName"]
}
}
}
For anyone looking at this in the future..
IAM lives in the N. Virginia region! Make sure to audit the logs there for STS