i'm to implement a custom resource and i'm failing when I try to update its name.
The reason that's i'm failing is that the custom resource lambda is missing the permissions to delete the old resource. since i'm giving it the exact permission by arn, and when I try to change the name of the resource which affects the arn, then it only has permissions for the new resource.
is there any way to solve this wiht giving it permissions for arn:...:/* ?
Here is some code to make this clearer, this implements an SSM SecureString placeholder (since we don't want to deploy secrets that will get away in the stack outputs so just a placeholder)
See code below, note that the policy statement passed to add_to_policy, is for arn:aws:ssm:{region}:{account_id}:parameter/* - this is the only way it works.
if I try to do minimal privileges, and set it to the specific param name arn, then if I try to update the name, it will fail due to missing permissions to delete the old resource on update.
If I manually added the delete permissions for the old resource (hard coded) it worked.
param_name = some-value # This is the ssm param name that is part of the arn
policy = AwsCustomResourcePolicy.from_sdk_calls(resources=[self.arn_join(f'arn:aws:ssm:{region}:{account_id}:parameter', param_name)])
resource = AwsCustomResource(scope=self, id=f'{id_}AwsCustomResource', policy=policy, log_retention=log_retention,
on_create=on_create, on_update=on_update, on_delete=on_delete,
resource_type='Custom::AWSSecureString',
timeout=timeout)
resource.grant_principal.add_to_policy(iam.PolicyStatement(actions=['ssm:DeleteParameter'], resources=[f'arn:aws:ssm:{region}:{account_id}:parameter/*']))
Is it possible to do somehow without giving it '*' permissions for all params?
Thanks for your help!
Related
Terraform fails on terraform apply, because of failure on "already exists" error.
I think this happened, because I manually deleted the tfstate and ddb md5 entries. Which created the whacky state of Terraform.
Now when I do init, plan and apply, I am getting quite a few errors as follows example:
Error: error creating SSM parameter: ParameterAlreadyExists: The parameter already exists. To overwrite this value, set the overwrite option in the request to true.
......
Error: error creating SSM parameter: ParameterAlreadyExists: The parameter already exists. To overwrite this value, set the overwrite option in the request to true.
Error: Error creating DB Parameter Group: DBParameterGroupAlreadyExists: Parameter group abc already exists
I have taken a look into the import option, but it's too messy.
Is there an easy or cleaner approach on tacking this?
Thank you, any advice will be helpful.
The short answer is, it depends.
Each resource has it own functionalities, some allow you to overwrite existing resources and some don't.
For example, for ssm parameters, you can add a "overwrite" flag to the resource.
resource "aws_ssm_parameter" "foo" {
name = "foo"
type = "String"
value = "bar"
overwrite = true
}
Official reference: ssm_parameter
Now, a good way to avoid the issue of loosing tfstate is to store it in S3 in a bucket that has version control.
Is there a way I can use a terraform data call for a bucket (perhaps created and stored in a different state file) and then in the event nothing is in data, create the resource by setting a count?
I've been doing some experiments and continually get the following:
Error: Failed getting S3 bucket (example_random_bucket_name): NotFound: Not Found
status code: 404, request id: <ID here>, host id: <host ID here>
Sample code to test (this has been modified from the original code which generated this error):
variable "bucket_name" {
default = "example_random_bucket_name"
}
data "aws_s3_bucket" "new" {
bucket = var.bucket_name
}
resource "aws_s3_bucket" "s3_bucket" {
count = try(1, data.aws_s3_bucket.new.id == "" ? 1 : 0 )
bucket = var.bucket_name
}
I feel like rather than generating an error I should get an empty result, but that's not the case.
Terraform is a desired-state system, so you can only describe what result you want, not the steps/conditions to get there.
If Terraform did allow you to decide whether to declare a bucket based on whether there is already a bucket of that name, you would create a configuration that could never converge: on the first run, it would not exist and so your configuration would declare it. But on the second run, the bucket would then exist and therefore your configuration would not declare it anymore, and so Terraform would plan to destroy it. On the third run, it would propose to create it again, and so on.
Instead, you must decide as part of your system design which Terraform configuration (or other system) is responsible for managing each object:
If you decide that a particular Terraform configuration is responsible for managing this S3 bucket then you can declare it with an unconditional aws_s3_bucket resource.
If you decide that some other system ought to manage the bucket then you'll write your configuration to somehow learn about the bucket name from elsewhere, such as by an input variable or using the aws_s3_bucket data source.
Sadly you can't do this. data sources must exist, otherwise they error out. There is no build in way in TF to check if a resource exists or not. There is nothing in between, in a sense that a resource may, or may not exist.
If you require such functionality, you have to program it yourself using External Data Source. Or maybe simpler, provide an input variable bucket_exist, so that you explicitly set it during apply.
Data sources are designed to fail this way.
However, if you use a state file from external configuration, it's possible to declare an output in the external state, based on whether the s3 bucket is managed by that state and use it in s3_bucket resource as condition.
For example, the output in external state will be empty string (not managed) or value for whatever property is useful for you. Boolean is another choice. Delete data source from this configuration and add condition to the resource based on the output.
It's your call if any such workarounds complicate or simplify your configuration.
When I go to the Logs page the below error shows.
There was an error loading Log Streams. Please try again by refreshing this page.
Problem is there is another function that is identical except the code that is creating log files no problem.
Any suggestions?
I solved it.
I added CloudwatchLogsFullAccess and then it took some time under an hour and then it was working.
I'm not sure why I needed to do this for the second function but not the first but it's working now.
Below is the link that helped me.
https://blogs.perficient.com/2018/02/12/error-loading-log-streams/
Make sure your Lambda has already logged at least once!
Appears to me that this error occurs if that is not the case - I've tested fresh Lambdas both with and without any log statements to confirm: Without any log statements, a corresponding Log Group for the Lambda does not exist yet; after the first log statement is made, the statement then exists in a seemingly-newly-made corresponding Log Group.
Although this may seem obvious/intuitive after-the-fact, this is how I ran into this scenario: I think before any logging had occurred on my new Lambda, I tried to hook it up to CloudWatch events - I tried after that attempt to see if the Lambda was invoked (by the events) via viewing 'Monitoring' tab -> 'View logs in CloudWatch' button - and that is where I encountered this error. The Lambda had not been invoked [CloudWatch events hookup had failed], so no logging had occurred, and thus there was no corresponding Log Group made yet to examine (when trying to hyperlink into it from the Lambda Configuration).
(Fwiw, I imagine maybe a corresponding Log Group could be manually made before the first logging, but I have not tested that.)
Ensure your Lambda's Execution Role has a Policy that allows writing to CloudWatch Logs from your Lambda.
IAM console -> 'Roles' -> < your Lambda's role > -> 'Permissions' tab -> 'Permissions policies' accordion
Ensure there is a Policy listed that has parameters set like this:
'Service': "CloudWatch Logs"
'Access level': includes at least "Write"
'Resource': your Lambda is not excluded (i.e: its not set to another specific Lambda, or another directory of Lambdas, or another resource type)
'Request condition': does not preclude the context of your given Lambda execution
An example of an "AWS managed policy" that meets these requirements [out-of-the-box, being that it is AWS-managed] is "AWSLambdaBasicExecutionRole". It has these parameters:
'Service': "CloudWatch Logs"
'Access level': "Limited: Write"
'Resource': "All resources"
'Request condition': "None"
If your Role does not have such a policy already, either add a new one or edit and existing one to have the requirements listed here - then this error should be resolved.
For example, in my case before I fixed things, my Lambda's Role had a policy that was based off [AWS-managed] "AWSLambdaBasicExecutionRole", but somehow had a Resource that was limited to a different Lambda (which was my problem - insufficient permission to meet that policy from my different intended Lambda). I fixed this by adding the original [AWS-managed] "AWSLambdaBasicExecutionRole" Policy to my intended Lambda's role (I also deleted the prior-said Policy as it wasn't used by anything else, but that probably wasn't strictly necessary [although nice to tidy up]).
I resolved it by attaching CloudWatchFullAccess policy to the execution role of my lambda function
I am using the AWS command line tool and am having issue with copying the parameter group to a different region
Error:
An error occurred (DBParameterGroupNotFound) when calling the CopyDBParameterGroup operation: DB ParameterGroup not found, not allowed to do cross region copy.
The command is:
>aws rds copy-db-parameter-group --source-db-parameter-group-identifier arn:aws:rds:ap-southeast-1:MyActID:pg:source-para-group --target-db-parameter-group-identifier dest-para-group --target-db-parameter-group-description dest-para-group-description
I also tried with:
>aws rds copy-db-parameter-group --source-db-parameter-group-identifier arn:aws:rds:ap-southeast-1:MyActID:pg:source-para-group --target-db-parameter-group-identifier arn:aws:rds:ap-south-1:MyActID:pg:dest-para-group --target-db-parameter-group-description arn:aws:rds:ap-south-1:MyActID:pg:dest-para-group-description
Please let help me if anyone else come across the similar issue?
According to the copy-db-parameter-group, the tricky part :
If the source DB parameter group is in a different region than the copy, specify a valid DB parameter group ARN, for example arn:aws:rds:us-west-2:123456789012:pg:special-parameters .
Apparently it is a "from - to" copy option. Try using the TARGET region AWS credential when making the copy, then specify the source ARN.
I've noticed on AWS that if I get IAM role credentials (key, secret, token) and set them as appropriate environment variables in a python script, I am able to create and use SWF Layer1 objects just fine. However, it looks like the Layer2 objects do not work. For example, if I have boto and os imported, and do:
test = boto.swf.layer2.ActivityWorker()
test.domain = 'someDomain'
test.task_list = 'someTaskList'
test.poll()
I get an exception that the security token is not valid, and indeed, if I dig through the object, the security token is not set. This even happens with:
test = boto.swf.layer2.ActivityWorker(session_token=os.environ.get('AWS_SECURITY_TOKEN'))
I can fix this by doing:
test._swf.provider.security_token = os.environ.get('AWS_SECURITY_TOKEN')
test.poll()
but seems pretty hacky and annoying because I have to do this every time I make a new layer2 object. Anyone else noticed this? Is this behavior intended for some reason, or am I missing something here?
Manual management of temporary security credentials is not only "pretty hacky", but also less secure. A better alternative would be to assign an IAM role to the instances, so they will automatically have all permissions of that Role without requiring explicit credentials.
See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html