Ansible Cloudwatch rule reports failed invocations - amazon-web-services

Question

I have created an AWS lambda that works well when I test it and when I create a cron job manually through a cloudwatch rule.
It reports metrics as invocations (not failed) and also logs with details about the execution.
Then I decided to remove that manually created cloudwatch rule in order to create one with ansible.
- name: Create lambda service.
lambda:
name: "{{ item.name }}"
state: present
zip_file: "{{ item.zip_file }}"
runtime: 'python2.7'
role: 'arn:aws:iam::12345678901:role/lambda_ecr_delete'
handler: 'main.handler'
region: 'eu-west-2'
environment_variables: "{{ item.env_vars }}"
with_items:
- name: lamda_ecr_cleaner
zip_file: assets/scripts/ecr-cleaner.zip
env_vars:
'DRYRUN': '0'
'IMAGES_TO_KEEP': '20'
'REGION': 'eu-west-2'
register: new_lambda
- name: Schedule a cloudwatch event.
cloudwatchevent_rule:
name: ecr_delete
schedule_expression: "rate(1 day)"
description: Delete old images in ecr repo.
targets:
- id: ecr_delete
arn: "{{ item.configuration.function_arn }}"
with_items: "{{ new_lambda.results }}"
That creates almost the exact same cloudwatch rule. The only difference I can see with the manually created one is in the targets, the lambda version / alias is set to Default when created manually while it is set to version, with a corresponding version number when created with ansible.
The cloudwatch rule created with ansible has only failed invocations.
Any idea why this is? I can't see any logs. Is there a way I can set the version to Default as well with the cloudwatchevent_rule module in ansible?

Answer 1

I've lost hours with this too, same error and same confusion (Why there isn't a log for failed invokations?), I'm going to share my ""solution"", it will solve the problem to someone, and will help others to debug and find the ultimate solution.
Note: Be carefull, this could allow any AWS account execute your lambda functions
Since you got invoke the function by creating the rule target manually, I assume you added the invoke permission to the lambda from CloudWatch, however it looks like the Source Account ID is different when the event is created by cli/api and when is created by de AWS dashboard/console
If you are adding the Source Account condition in the lambda invoke permission from principal "events.amazonaws.com" to prevent any AWS account execute your lambdas just remove it (under your responsability!).
So, if your lambda policy looks like this:
{
"Sid": "<sid>",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:InvokeFunction",,
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "<account-id>"
}
},
"Resource": "arn:aws:lambda:<region>:<account-id>:function:<lambda-function>"
}
Remove the "Condition" field
{
"Sid": "sid",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:InvokeFunction",,
"Resource": "arn:aws:lambda:<region>:<account-id>:function:<lambda-function>"
}
And "maybe" it will work for you.
I think something weird it is happening with the cloudwatch event owner/creator data when the event is created by cli/api... maybe a bug? Not sure. I will keep working on it

Answer 2

To extend the answered here https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CWE_Troubleshooting.html#LAMfunctionNotInvoked. Since you are creating it via API you should add permission to Lambda as mentioned before. Without compromising security you could do the following:
Add rule with PutRule api call, it will return you
{
"RuleArn": "string"
}
Use the RuleArn in Lambda AddPermission call
aws lambda add-permission \
--function-name MyFunction \
--statement-id MyId \
--action 'lambda:InvokeFunction' \
--principal events.amazonaws.com \
--source-arn arn-from-PutRule-request

Answer 3

If you are looking for the reason your invocations are failing, see the other answers UNLESS you're trying to implement AWS::Events::Rule and you're seeing failed invocations. The following answer may resolve the issue and negate to need to find these non-existent logs.
Cloudwatch failedinvocation error no logs available