I am trying to create or update Lambdas on AWS using the Terraform CDKTF. During deployment, I am getting the error of
"An event source mapping with SQS arn (\" arn:aws:sqs:eu-west-2:*******:*****-*****-******** \") and function (\" ******-******-****** \") already exists. Please update or delete the existing mapping with UUID *******-****-****-****-***********"
**** are sensitive info I have swapped out.
Some of our Lambdas are called via SQS, which is what this mapping is referring to. I assumed the first fix would be to remove the mappings that might already exist (on a previous deployment that might have partly gone through), but I am unsure where to find them, nor if they are even available to delete. I originally assumed by calling cdktf deploy it would update these mappings and not throw the err at all.
Does anyone have any advice?
Your diagnosis seems right, there might be some stray resources left behind due to an aborted / unfinished Terraform run. You should be able to clean up after these runs by running terraform destroy in the stack directory ./cdktf.out/stacks/..../. That should delete all previously existing resources created through this Terraform stack.
Related
I accidentally created an instance of AWS::ApiGateway::GatewayResponse with a bad value - DEFAULT_400 instead of DEFAULT_4XX. The instance got created, or at least "partially created", although the Cloudformation stack in which it was contained returned an error. Now I can't delete it - every time I try I get
1 validation error detected: Value 'DEFAULT_400' at 'responseType' failed to satisfy constraint: Member must satisfy enum value set: [REQUEST_TOO_LARGE, RESOURCE_NOT_FOUND, AUTHORIZER_CONFIGURATION_ERROR, {...}]
How can I force deletion of this instance, ideally via the AWS CLI ?
PS it appears to have been "orphaned", ie the parent RestApi it belonged to has been deleted
There are few ways of deleting stacks which can be automatically removed. They are listed in AWS docs:
How do I delete an AWS CloudFormation stack that's stuck in DELETE_FAILED status?
I'm just starting my serverless/cloudformation/AWS SAM journey. I've created a stack that has a resource of type AWS::CloudFormation::Stack, and I've separated some of my resources into that child stack.
When I do sam build and then sam deploy, I get the following error:
Embedded stack arn:aws:cloudformation:us-
west-2:111111111111:stack/ParentStack-
ChildStack-1QK94LXRA71CS/f9885e30-631c-11eb-
bfd8-021cb123b7ed was not successfully created: The
following resource(s) failed to create: [DynamoDBTable].
-
The following resource(s) failed to create:
[ChildStack].
Of course, what I really want to know is which resource in the nested stack failed to create, and why. When I copy/paste the resources from the child stack into the parent .yaml file and rebuild/redeploy, I see:
One or more parameter values were invalid: Some index key
attributes are not defined in AttributeDefinitions. Keys:
[userID], AttributeDefinitions: [userId] (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code:
ValidationException; Request ID:
SMJDHUT0CQKM8IBQJVMAIJM4RRVV4KQNSO5AEMVJF66Q9ASUAAJG;
Proxy: null)
This is what I want to see in the output when I build the parent stack: the errors that caused the child stack to fail.
This has led me to use a rather tortuous workflow: build the resources in the main stack, then separate them to an independent stack when they build properly. There's got to be a better way, and I'm sure the community knows something here that I don't.
How do y'all debug child stacks when you're on the CloudFormation train?
This is normal behaviour you have to take help from the AWS Console or use AWSW CLI in this case.
Deploy error reporting is not showing the reason of failure when using nested stacks. #5974
Feature Requests for nested stacks
Why doesn't the error tell me what's wrong?:
CloudFormation passes the creation and updating of a resource to the service responsible for those resources. When a resource fails to create/update, the resource's backing service returns a reason to the stack, which gets logged as the Status Reason within the events. Child Stack is a CloudFormation::Stack resource, so it's being created by CloudFormation. As far as CloudFormation knows, it didn't run into an error trying to actually do anything. Everything it did relate to the CloudFormation side of things was fine; the blame is with the Service resource, which failed to create it. Therefore, Child Stack tells Parent-Stack that it only failed because Service failed to create, not because of a problem on the CloudFormation Service side of things.
you can read more about it here CloudFormation troubleshooting
I have a cloudformation stack which has a Lambda function that is mapped as a trigger to an SQS queue.
What happened was that I had to delete the mapping and create it again manually cos I wanted to change the batch size. Now when I want to update the mapping the cloudformation throws an error with The resource you requested does not exist. message.
The resource mapping code looks like this:
"EventSourceMapping":{
"Properties":{
"BatchSize":5,
"Enabled":"true",
"EventSourceArn":{
"Fn::GetAtt":[
"ProcessorQueue",
"Arn"
]
},
"FunctionName":{
"Fn::GetAtt":[
"ProcessorLambda",
"Arn"
]
}
},
"Type":"AWS::Lambda::EventSourceMapping"
}
I know that I've deleted the mapping cloudformation created initially and added it manually which is causing the issue. How do I fix this? Cos I cannot push any update now.
Please help
What you did, from my perspective, it is a mistake. When you use Cloud Formation you are not suppose to apply changes manually. You can, and maybe that's fine since one may don't care about the stack once is created. But since you are trying to update the stack, this tells me that you want to keep the stack and update it on a time basis.
To narrow down your problem, first let make clear that the manually-created mapping is out of sync with your cloud formation stack. So, from a cloud formation perspective, it doesn't matter if you keep that mapping or not. I'm wondering, what would happen if you keep the manually-created mapping and create a new from Cloud Formation? Maybe it will complain, since you would have repeated mappings for the same pair of (lambda,queue). Try this:
Create a change for your stack, where you completely remove the EventSourceMapping resource from your script. This step is to basically clean loosing references. Apply the change set.
Then, and this is where I think you may get some kind of issue, add back again EventSourceMapping to your stack.
If you get errors in the step 2, like "this mapping already exists", you will have to remove the manually-created mapping from the console. And then try again step 2.
You probably know now that you should not have removed the resource manually. If you change the CF, you can update it without changing resources which did not change in CF. You can try to replace the resource with the exact same physical name https://aws.amazon.com/premiumsupport/knowledge-center/failing-stack-updates-deleted/ The other option is to remove the resource from CF, update, and then add it back and update again - from the same doc.
While comments above are valid, I found it interesting, that no one mentioned much simpler option: using SAM commands (sam build/sam deploy). It's understandable that during the development process and designing the architecture, there might be flaws and situations where manual input in the console is necessary, therefore there's something I reference to every time I have similar issue.
Simply comment out the chunk of code that is creating troubles, run sam build/deploy on top of it, CloudFormation stack will recognize that the resource no longer in the template and will delete it.
Now, since the resource is no longer in the architecture anyway(removed manually prior), it will have no issues passing the step and successfully updating the stack.
Then simply uncomment, make any necessary changes (if any) and deploy.
Works every time.
After modifying my job to start using timestampLabel when reading from PubSub, resource setup seems to break every time I try to start the job with the following error:
(c8bce90672926e26): Workflow failed. Causes: (5743e5d17dd7bfb7): Step setup_resource_/subscriptions/project-name/subscription-name__streaming_dataflow_internal25: Set up of resource /subscriptions/project-name/subscription-name__streaming_dataflow_internal failed
where project-name and subscription-name represent the actual values of my project and PubSub subscription I'm trying to read from. Before trying to attach timestampLabel on message entry, the job was working correctly, consuming messages from the specified PubSub subscription, which should mean that my API/network settings are OK.
I'm also noticing two warnings with the payload
Internal Issue (119d3b54af281acf): 65177287:8503
but no more information can be found in the worker logs. For the few seconds that my job is setting up I can see the timestampLabel being set in the first step of the pipeline. Unfortunately I can't find any other cases or documentation about this error.
When using the timestampLabel feature, a second subscription is created for tracking purposes. Double check the permission settings on your topic to make sure it matches the permissions required.
Has anyone successfully set up their AWS IoT button?
When stepping through with default values I keep getting this message: Please correct validation errors with your trigger. But there are no validation errors on any of the setup pages, or the page with the error message.
I hate asking a broad question like this but it appears no one has ever had this error before.
This has been driving me nuts for a week!
I got it to work by using Custom IoT Rule instead of IoT Button on the IoT Type. The default rule name is iotbutton_xxxxxxxxxxxxxxxx and the default SQL statement is SELECT * FROM 'iotbutton/xxxxxxxxxxxxxxxx' (xxx... = serial number).
Make sure you copy the policy from the sample code into the execution role - I know that has tripped up a lot of people.
I was getting the same error. The cause turned out to be that I had multiple certificates associated with the button. This was caused by me starting over again on the wizard, generating cert & key, loading cert & key again. While on the device itself this doesn't seem to be a problem, the result was that on AWS I had multiple certs associated to the device.
Within the AWS IoT Resources view I eventually managed to delete all resources. Took some fiddling to get certs detached and able to be deleted. Once I deleted all resources I returned to the wizard, created yet another cert & key pair, pushed the Lambda code, and everything works.