IAM permission boundary for CDK apps with conditions - amazon-web-services

I am writing an IAM role for a CI/CD user which deploys our Cloud Development Kit (CDK) app. The CDK app consists of lambda functions, Fargate etc. The problem is, that CDK does not allow me to specify all the roles it needs. Instead it creates some of then on its own.
Couple of examples:
Each lambda function with log retention has another lambda created by CDK which sets log retention to the log group and log streams.
CloudTrail event executing a step function needs a role with states:StartExecution permission.
CDK creates these roles automatically and also puts inline policies to them. Which forces me to give my CI/CD role permissions to create roles and attach policies. So if anybody gets access to the CI/CD user (for example if our GitHub credentials leak), the attacker could create new roles and give them admin permissions.
I tried creating all the roles myself in a separate stack and then using these roles in CDK app. But as I mentioned above (see the examples above), it's not possible everywhere...
I also tried IAM permission boundary for the deployer role, but I can't figure out how to limit permissions for iam:PutRolePolicy. CDK essentially does the following:
iam:CreateRole
iam:PutRolePolicy
According to AWS documentation, conditions are quite basic string comparisons. I need to be able to select, which actions are allowed in the policy document passed to iam:PutRolePolicy.
This is a sample of my permission boundary allowing the principal to create roles and put role policies. See the condition comment.
permission_boundary = aws_iam.ManagedPolicy(
scope=self,
id='DeployerPermissionBoundary',
managed_policy_name='DeployerPermissionBoundary',
statements=[
aws_iam.PolicyStatement(
actions=['iam:CreateRole'],
effect=aws_iam.Effect.ALLOW,
resources=[f'arn:aws:iam::{core.Aws.ACCOUNT_ID}:role/my-project-lambda-role']
),
aws_iam.PolicyStatement(
actions=['iam:PutRolePolicy'],
effect=aws_iam.Effect.ALLOW,
resources=[f'arn:aws:iam::{core.Aws.ACCOUNT_ID}:role/my-project-lambda-role'],
conditions=Conditions([
StringLike('RoleName', 'Required-role-name'),
StringLike('PolicyName', 'Required-policy-name'),
StringEquals('PolicyDocument', '') # I want to allow only specified actions like logs:CreateLogStream and logs:PutLogEvents
])
)
]
)
deployer_role = aws_iam.Role(
scope=self,
id='DeployerRole',
assumed_by=aws_iam.AccountRootPrincipal(),
permissions_boundary=permission_boundary,
inline_policies={
'Deployer': aws_iam.PolicyDocument(
statements=[
aws_iam.PolicyStatement(
actions=['iam:PutRolePolicy'],
effect=aws_iam.Effect.ALLOW,
resources=[f'arn:aws:iam::{core.Aws.ACCOUNT_ID}:role/my-project-lambda-role']
),
...
...
]
)
}
)
What is the correct way of limiting the PutRolePolicy to selected actions only? I want to allow logs:CreateLogStream and logs:PutLogEvents and nothing else.
I've been fighting with this for quite some time and I don't want to fall back to giving out more permissions than necessary. Thanks everyone in advance!

Here's a solution in Python for CDK 1.4.0 inspired by #matthewtapper's code on GitHub. This allows you to set permission boundary to all the roles in your stack.
Needless to say it's very ugly, since python CDK does not provide construct objects in aspects. We have to dig deep into JSII to resolve the objects. Hope it helps someone.
from jsii._reference_map import _refs
from jsii._utils import Singleton
import jsii
#jsii.implements(core.IAspect)
class PermissionBoundaryAspect:
def __init__(self, permission_boundary: Union[aws_iam.ManagedPolicy, str]) -> None:
"""
:param permission_boundary: Either aws_iam.ManagedPolicy object or managed policy's ARN as string
"""
self.permission_boundary = permission_boundary
def visit(self, construct_ref: core.IConstruct) -> None:
"""
construct_ref only contains a string reference to an object. To get the actual object, we need to resolve it using JSII mapping.
:param construct_ref: ObjRef object with string reference to the actual object.
:return: None
"""
kernel = Singleton._instances[jsii._kernel.Kernel]
resolve = _refs.resolve(kernel, construct_ref)
def _walk(obj):
if isinstance(obj, aws_iam.Role):
cfn_role = obj.node.find_child('Resource')
policy_arn = self.permission_boundary if isinstance(self.permission_boundary, str) else self.permission_boundary.managed_policy_arn
cfn_role.add_property_override('PermissionsBoundary', policy_arn)
else:
if hasattr(obj, 'permissions_node'):
for c in obj.permissions_node.children:
_walk(c)
if obj.node.children:
for c in obj.node.children:
_walk(c)
_walk(resolve)
Usage:
stack.node.apply_aspect(PermissionBoundaryAspect(managed_policy_arn))

Here is the solution for CDK version 1.9.0 + with added and extra try_find_child() to prevent nested child errors on the node, also the stack.node.apply_aspect() method is depreceated by AWS, so there is a new usage implementation.
from aws_cdk import (
aws_iam as iam,
core,
)
import jsii
from jsii._reference_map import _refs
from jsii._utils import Singleton
from typing import Union
#jsii.implements(core.IAspect)
class PermissionBoundaryAspect:
"""
This aspect finds all aws_iam.Role objects in a node (ie. CDK stack) and sets
permission boundary to the given ARN.
"""
def __init__(self, permission_boundary: Union[iam.ManagedPolicy, str]) -> None:
"""
This initialization method sets the permission boundary attribute.
:param permission_boundary: The provided permission boundary
:type permission_boundary: iam.ManagedPolicy|str
"""
self.permission_boundary = permission_boundary
print(self.permission_boundary)
def visit(self, construct_ref: core.IConstruct) -> None:
"""
construct_ref only contains a string reference to an object.
To get the actual object, we need to resolve it using JSII mapping.
:param construct_ref: ObjRef object with string reference to the actual object.
:return: None
"""
if isinstance(construct_ref, jsii._kernel.ObjRef) and hasattr(
construct_ref, "ref"
):
kernel = Singleton._instances[
jsii._kernel.Kernel
] # The same object is available as: jsii.kernel
resolve = _refs.resolve(kernel, construct_ref)
else:
resolve = construct_ref
def _walk(obj):
if obj.node.try_find_child("Resource") is not None:
if isinstance(obj, iam.Role):
cfn_role = obj.node.find_child("Resource")
policy_arn = (
self.permission_boundary
if isinstance(self.permission_boundary, str)
else self.permission_boundary.managed_policy_arn
)
cfn_role.add_property_override("PermissionsBoundary", policy_arn)
else:
if hasattr(obj, "permissions_node"):
for c in obj.permissions_node.children:
_walk(c)
if hasattr(obj, "node") and obj.node.children:
for c in obj.node.children:
_walk(c)
_walk(resolve)
And the new implementation API for the stack is:
core.Aspects.of(stack).add(
PermissionBoundaryAspect(
f"arn:aws:iam::{target_environment.account}:policy/my-permissions-boundary"
)
)

Anyone still struggling in certain cases or wants a Java example:
#Slf4j
public class PermissionBoundaryRoleAspect implements IAspect {
private static final String BOUNDED_PATH = "/bounded/";
#Override
public void visit(final #NotNull IConstruct node) {
node.getNode().findAll().stream().filter(iConstruct -> CfnResource.isCfnResource(iConstruct) && iConstruct.toString().contains("AWS::IAM::Role")).forEach(iConstruct -> {
var resource = (CfnResource) iConstruct;
resource.addPropertyOverride("PermissionsBoundary", "arn:aws:iam::xxx:policy/BoundedPermissionsPolicy");
resource.addPropertyOverride("Path", BOUNDED_PATH);
});
if (node instanceof CfnInstanceProfile) {
var instanceProfile = (CfnInstanceProfile) node;
instanceProfile.setPath(BOUNDED_PATH);
}
}
}
Why I am doing it this way, is because I was faced with a case where not all Roles being created was of type CfnRole
In my case I had to create a CfnCloudFormationProvisionedProduct
This constructor had a weird way of creating Roles. Roles in this constructor is of type CfnResource and cannot be casted to "CfnRole"
Thus I am using iConstruct.toString().contains("AWS::IAM::Role") which works for every resource if its type AWS::IAM::Role and for any CfnRole

Related

Assign a role to the group using cloud resource manager api in gcp. I'm new to gcp, my understanding is of beginner level

As the title states I'm assigning a role to a group using cloud resource mananger api using python client library.
Here's the code :
from googleapiclient.discovery import build
from googleapiclient import errors
service = build('cloudresourcemanager', 'v1', credentials= delegated_credentials)
def assign_custom_role_to_group(project_id, role_name, group_email):
try:
# Define the request body
policy = service.projects().getIamPolicy(resource=project_id).execute()
#print("GET IAM", policy['version'])
bindings = policy.get('bindings', [])
# Add the custom role to the bindings
bindings.append({
"role": role_name,
"members": ["group:group_email"],
"etag" : "BwXy6qC04wk=",
"version" : 1
})
policy['bindings'] = bindings
# Execute the request
service.projects().setIamPolicy(resource=project_id, body=policy).execute()
print(f'Successfully assigned custom role {role_name} to group {group_email}')
except errors.HttpError as error:
print(f'An error occurred: {error}')
For this code I'm getting the following error.
enter image description here.
Thank you.

Last marker for boto3 pagination is not working

I'm working with roles and policies using AWS boto3 SDK. I want to get the policies attached to a given role and then do some processing. Here's the code.
def get_policies(role_name):
marker = None
while True:
print ('marker in get_policies is {} '.format(marker))
if marker:
response_iterator = iam.list_attached_role_policies(
RoleName=role_name,
# PathPrefix='string',
Marker=marker,
MaxItems=1
)
else:
response_iterator = iam.list_attached_role_policies(
RoleName=role_name,
# PathPrefix='string',
MaxItems=1
)
print("Next Page in get_policy : {} ".format(response_iterator['IsTruncated']))
print(response_iterator['AttachedPolicies'])
for policy in response_iterator['AttachedPolicies']:
detach_policy_from_entities(policy['PolicyArn'])
print(policy['PolicyName'] + " has to be deleted")
# delete_policy_response = iam.delete_policy(
# PolicyArn=policy['PolicyArn']
# )
print('deleted {}'.format(policy['PolicyName']))
if response_iterator['IsTruncated']:
marker = response_iterator['Marker']
else:
print("done with policy deletion")
return "finished"
The code works fine except it returns an empty list with the last marker. So, I have 3 policies attached to the given role.
The code works as follows:
initially marker is None, it just run the else part and returns 1 result with marker for next iteration
I use the marker to get another set of result. It works and returns 1 result with marker for the next iteration
Here I use the marker but it returns and empty list for the policy but I have one more policy
Any help will be greatly appreciated.
It looks like you are mutating the attached role policies and hence invalidating the pagination marker. Also, unless you specifically need it, I would remove MaxItems=1.
One solution is to change the code to simply append the policy ARNs to a list and then process that list for detachment after your for policy in ... loop.
As an FYI, you should consider using the resource-level IAM.Role as it simplifies access to the associated policies (they are available via simple policies and attached_policies collections on the role). For example:
import boto3
iam = boto3.resource("iam")
role = iam.Role("role-name-here")
for p in role.attached_policies.all():
print(p)
Your code works fine with your pagination logic, but I do not think you will need pagination as by default if you do not use MaxItems the list_attached_role_policies function is to return 100 values.
Also, as the Default quotas for IAM entities defines that you can have as default 10 Managed policies attached to an IAM role, if you don't request an increment (More information IAM object quotas can be found here).
So, for you logic works you need something like this:
import boto3
iam = boto3.client("iam")
role = "role_name"
policies = iam.list_attached_role_policies(RoleName=role)
policies_list = policies['AttachedPolicies']
for policy in policies_list:
print(policy['PolicyName'])
# you can add your detach logic here
Also the list_attached_role_policies method does not return inline policies, if the policies that are not being showed to you are inline policies you will need the list_role_policies method.
import boto3
iam = boto3.client("iam")
role = "role_name"
policies = iam.list_role_policies(RoleName=role)
policies_list = policies['PolicyNames']
for policy in policies_list:
print(policy)
# you can add your detach logic here

Code fails to update a table on BQ using DML, but succeeds for insertion and deletion with RPC

I wrote some code that uses service-account to write to BQ on google-cloud.
A very strange thing is that only "update" operation using DML fails. (Other insertion, deletion RPC calls succeeds).
def create_table(self, table_id, schema):
table_full_name = self.get_table_full_name(table_id)
table = self.get_table(table_full_name)
if table is not None:
return # self.client.delete_table(table_full_name, not_found_ok=True) # Make an API
# request. # print("Deleted table '{}'.".format(table_full_name))
table = bigquery.Table(table_full_name, schema=schema)
table = self.client.create_table(table) # Make an API request.
print("Created table {}.{}.{}".format(table.project, table.dataset_id, table.table_id))
#Works!
def upload_rows_to_bq(self, table_id, rows_to_insert):
table_full_name = self.get_table_full_name(table_id)
for ads_chunk in split(rows_to_insert, _BQ_CHUNK_SIZE):
errors = self.client.insert_rows_json(table_full_name, ads_chunk,
row_ids=[None] * len(rows_to_insert)) # Make an API request.
if not errors:
print("New rows have been added.")
else:
print("Encountered errors while inserting rows: {}".format(errors))
#Permissions Failure
def update_bq_ads_status_removed(self, table_id, update_ads):
affected_rows = 0
table_full_name = self.get_table_full_name(table_id)
for update_ads_chunk in split(update_ads, _BQ_CHUNK_SIZE):
ad_ids = [item["ad_id"] for item in update_ads_chunk]
affected_rows += self.update_bq_ads_status(f"""
UPDATE {table_full_name}
SET status = 'Removed'
WHERE ad_id IN {tuple(ad_ids)}
""")
return affected_rows
I get this error for update only:
User does not have bigquery.jobs.create permission in project ABC.
I will elaborate on my comment.
In GCP you have 3 types of IAM roles.
Basic Roles
include the Owner, Editor, and Viewer roles.
Predefined Roles
provide granular access for a specific service and are managed by Google Cloud. Predefined roles are meant to support common use cases and access control patterns.
Custom Roles
provide granular access according to a user-specified list of permissions.
What's the difference between predefinied and custom roles? If you change (add/remove) permission for a predefinied role it will become custom role.
Predefinied roles for BigQuery with permissions list can be found here
Mentioned error:
User does not have bigquery.jobs.create permission in project ABC.
Means that IAM Role doesn't have specific BigQuery Permission - bigquery.jobs.create.
bigquery.jobs.create permission can be found in two predefinied roles like:
BigQuery Job User - (roles/bigquery.jobUser)
BigQuery User - (roles/bigquery.user)
Or can be added to a different predefinied role, however it would change to custom role.
Just for addition, in Testing Permission guide, you can find information on how to test IAM permissions.
Please give the service account the bigquery.user role and try to run the code again.
BigQuery Job User
role: bigquery.user

AWS S3: Lambda triggered on OBJECT_CREATED does not list the newly created object

I have created an S3 bucket with a lambda function that gets triggerde upon object creation.
I have set this up using CDK:
class IndexedBucketStack(cdk.Stack):
def __init__(
self,
scope: cdk.Construct,
construct_id: str,
site_name: str,
hostname: str = None,
**kwargs
) -> None:
super().__init__(scope, construct_id, **kwargs)
# Bucket to store the files in.
# Important here that we destroy the S3 Bucket when the stack would be deleted.
bucket = s3.Bucket(
self,
"Bucket",
website_index_document=INDEX_FILE_NAME,
removal_policy=cdk.RemovalPolicy.DESTROY,
)
# Lambda that builds an index page every time a file gets uploaded or changed
create_index_lambda = _lambda.Function(
# snipped the code out here, but essentially packages a python function
)
# Allow lambda function to read the bucket
bucket.grant_read_write(create_index_lambda)
# Add event notification that triggers our lambda upon object creation...
bucket.add_event_notification(
s3.EventType.OBJECT_CREATED, s3n.LambdaDestination(create_index_lambda)
)
# ... and one on deletion of an object.
bucket.add_event_notification(
s3.EventType.OBJECT_REMOVED, s3n.LambdaDestination(create_index_lambda)
)
In the function, I list the objects using cloudpathlib's iterdir().
Hoever the results of that list (which uses ListObects v1 if I have it correctly) sometimes does not contain that just created file.
So I could just put a delay inside the lambda, but that seems a waste of time / money, right?
The goal is to have a lambda that creates a directory index.html everytime a file gets uploaded to a folder.
According to https://docs.aws.amazon.com/lambda/latest/dg/with-s3.html new object should be passed in the notification event - no need to call S3 API to find it on a list.
Apparently ListObjects (v1) is not strong consistent, while ListObjectsV2 is. The latter gives the intended results.
I ended up using S3Path instead of cloudpathlib, which does implement the V2 and offers the same convenience.

Setting .authorize_egress() with protocol set to all

I am trying to execute the following code
def createSecurityGroup(self, securitygroupname):
conn = boto3.resource('ec2')
response = conn.create_security_group(GroupName=securitygroupname, Description = 'test')
VPC_NAT_SecurityObject = createSecurityGroup("mysecurity_group")
response_egress_all = VPC_NAT_SecurityObject.authorize_egress(
IpPermissions=[{'IpProtocol': '-1'}])
and getting the below exception
EXCEPTION :
An error occurred (InvalidParameterValue) when calling the AuthorizeSecurityGroupEgress operation: Only Amazon VPC security
groups may be used with this operation.
I tried several different combinations but not able to set the protocol to all . I used '-1' as explained in the boto3 documentation. Can somebody pls suggest how to get this done.
(UPDATE)
1.boto3.resource("ec2") class actually a high level class wrap around the client class. You must create an extract class instantiation using boto3.resource("ec2").Vpc in order to attach to specific VPC ID e.g.
import boto3
ec2_resource = boto3.resource("ec2")
myvpc = ec2_resource.Vpc("vpc-xxxxxxxx")
response = myvpc.create_security_group(
GroupName = securitygroupname,
Description = 'test')
2.Sometime it is straightforward to use boto3.client("ec2") If you check boto3 EC2 client create_security_group, you will see this:
response = client.create_security_group(
DryRun=True|False,
GroupName='string',
Description='string',
VpcId='string'
)
If you use automation script/template to rebuild the VPC, e.g. salt-cloud, you need give the VPC a tag name in order to acquire it automatically from boto3 script. This will save all the hassle when AWS migrate all the AWS resources ID from 8 alphanumeric to 12 or 15 character.
Another option is using cloudformation that let you put everything and specify variable in a template to recreate the VPC stack.