Differences between AWS client and GUI - amazon-web-services

Question

I have a simple proof of concept which is working in the GUI. I have an S3 bucket, and a Lambda function, and when the S3 bucket's contents are altered, the Lambda logs the changes to CloudWatch.
I want to build on this and automate as much of the deployment as possible. I have written a shell script that zips up the Lambda's source and uploads it, creates a new version, and grabs the version's ARN to put into the S3 bucket's Event Notification so that the bucket uses the new version.
This works fine in the GUI. I paste the ARN in and the bucket calls whatever version I've pasted in.
However, when I try to script that last step, it always fails with:
An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Unable to validate the following destination configurations
There is never any content listed in the 'following destination configurations'
The command which is failing is:
aws s3api put-bucket-notification-configuration --bucket my_bucket_name --notification-configuration file://config.json
The config file is nearly identical to the output of:
aws s3api get-bucket-notification-configuration --bucket my_bucket_name
Except that the version number at the end of the ARN has changed. Using the output of that command as the input for the previous command is successful.
That looks like:
{
"LambdaFunctionConfigurations": [
{
"Id": "my_S3_event_notificaton",
"LambdaFunctionArn": "arn:aws:lambda:us-east-1:666666666666:function:my_lambda_name:4",
"Events": [
"s3:ObjectCreated:*",
"s3:ObjectRemoved:*"
],
"Filter": {
"Key": {
"FilterRules": [
{
"Name": "Prefix",
"Value": ""
}
]
}
}
}
]
}
Changing the '4' at the end to '7' fails.

Answer 1

TL;DR Workaround: omit the :7 qualifier on the notification configuration, use the unqualified (latest) lambda version.
Error replicated
I replicated the OP's error using the CLI and using boto3 directly. put-bucket-notification-configuration fails with qualified arns (e.g. :4) unless a given version has been previously added as a target manually in the S3 Console. Odd. The Chrome network traces of the Console's request-response traffic did not clarify.
# List the lambda versions
res_versions = lambda_client.list_versions_by_function(FunctionName=func)
versions = [v["Version"] for v in res_versions["Versions"]]
# -> ['$LATEST', '1', '2', '3']
# Put the configuration
config = { "LambdaFunctionConfigurations": [{ "LambdaFunctionArn": func + ":2", "Events": ["s3:ObjectCreated:*"] }] }
# fails unless the version has previously been manually set at the target in the Console
# unqualified arn always succeeds as expected
res_put = s3_client.put_bucket_notification_configuration(Bucket=bucket, NotificationConfiguration=config)
Workaround: use the unqualified lambda version
You are manually updating the S3 notification to the latest lambda version. This step is not strictly necessary. Life is easier if you omit the :4 version qualifier on your ARN. "When you invoke a function using an unqualified ARN, Lambda implicitly invokes $LATEST." The notification configuration is set once and does not change how many times you update your lambda.
AWS has invented the wheel
As #Maurice pointed out, you should consider AWS's mature infrastructure-as-code tools like CloudFormation and the Cloud Development Kit whose job it is to "automate as much of the deployment as possible".