I want to trigger a Lambda function whenever a file is uploaded to an Amazon S3 bucket with a certain prefix and suffix using SAM. Right now I'm using this code but it's giving error
"The ARN is not well formed(Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument"
Edit:
This is working but it's not giving an option to add suffix or prefix.
In the NotificationConfiguration you're simply using !Ref HelloWorld to reference your function, however, as the documentation for AWS::Serverless::Function states:
When the logical ID of this resource is provided to the Ref intrinsic function, it returns the resource name of the underlying Lambda function.
If we look at the documentation for the LambdaConfiguration it states:
The Amazon Resource Name (ARN) of the AWS Lambda function that Amazon S3 invokes when the specified event type occurs.
If you simply change the !Ref HelloWorld to !GetAtt HelloWorld.Arn it should pass in the correct value.
Beware however of the remark that is made on the NotificationConfiguration documentation that if you create the bucket at the same time as you're creating the notification configuration, you might end up with a circular dependency, since you also need to add a AWS::Lamdba::Permission (or use the Events of AWS::Serverless::Function) to allow S3 to invoke your lambda function.
Related
I have set up CI/CD for an AWS Lambda function such that the new version is automatically deployed using GitHub actions. By default, AWS creates a new Lambda ID (and thus URL) for this lambda function. This means that the front-end portion of my code will need to be updated to contain the updated URL. Is there a way to automatically perform such updating? By e.g. saving the URL as an environment variable and inserting it in the code with a GitHub action?
Or is there alternatively a way to re-use the old Lambda function URL for new deployments?
You can get the updated Lambda URL by using SAM template outputs as follows:
Resources:
MyFunction:
Type: 'AWS::Serverless::Function'
Outputs:
MyFunctionUrlEndpoint:
Description: "My Lambda Function URL Endpoint"
Value: !GetAtt MyFunctionUrl.FunctionUrl
Then you can access the output as described in this answer:
aws cloudformation describe-stacks --stack-name stack_name --query 'Stacks[0].Outputs[?OutputKey==`MyFunctionUrlEndpoint`].OutputValue' --output text
which can then be further processed in e.g. your front-end code.
There may be easier methods, but this should work!
Since the global uniqueness requirements of S3 bucket names, using the optional BucketName property in the AWS::S3::Bucket resource is problematic. Essentially, if I insist on using BucketName I need some way to attach a GUID there.
I can avoid this pain if I omit the BucketName property entirely, so CloudFormation reliably generates a unique name for me.
However, I face another problem: how do I work with this random bucket name in AWS Lambda/SAM/serverless.com/other code? I understand that CloudFormation templates can export the name, but how do I pass it to the Lambda code?
Is there a standard/recommended way of working with CloudFormation exports in AWS Lambda? The problem is not unique to S3 - e.g., AWS Amplify uses randomly generated DynamoDB table names, too.
If your Lambda is created through CloudFormation, you can pass the bucket name using Environment variables (Environment key in SAM and CloudFormation). You can refer to the bucket name using !Ref if the bucket is in the same spec and cross stack references if using different stacks. If you use cross stack references, you won't be able to modify or delete the output value in the original stack until you remove all references to it. If you are using Ref, the Lambda will also be updated if the bucket name changes.
If your Lambda isn't created through CloudFormation, you can use SSM parameter store as mentioned by Ervin in his comment. You can create a SSM Parameter and read it's value in your Lambda code.
I've one scenario like I want to invoke one lambda function by cloud custodian and want to pass newly created bucket name to that lambda function. is there any way to pass parameters to the lambda function from the custodian event? Thanks
-- below is my cloud custodian policy:-
policies:
name: lambda-s3-configure-standards-real-time
resource: aws.lambda
description: |
This policy is triggered when a new S3 bucket is created and it will invoke another lambda.
mode:
type: cloudtrail
events:
- CreateBucket
role: some-role
timeout: 200
actions:
- type: invoke-lambda
function: Lambda-function-name
Have you tried checking the lambda function's event dictionary that is passed as the payload to your invoked lambda function? I believe the bucket name should already be present as part of the payload readily.
As an example, when the bucket is created via console your payload should contain the Bucket Name at $.detail.requestParameters.bucketName
I have a problem that so far I'm unable to identify the root cause.
I have an AWS step machine that should be invoked once a file is uploaded to an S3 bucket.
So far when I upload the file to the S3 bucket, the lambda function that is defined in the StartAt key (StartAt: ImgUploadedEvent) starts as I can see in the lambda logs.
Here is the code:
stepFunctions:
stateMachines:
ValidateImageStateMachine:
loggingConfig:
level: ALL
includeExecutionData: true
destinations:
- Fn::GetAtt: [ StepFuncLogGroup, Arn ]
definition:
Comment: "This state function validates the images after users upload them to S3"
StartAt: ImgUploadedEvent
States:
ImgUploadedEvent:
Type: Task
Resource:
Fn::GetAtt: [ImgUploaded, Arn]
End: true
Below is the lambda function that is declared as the start of the StepMachine
This lambda function as I can see from the logs indeed get called once I modified an Object in S3
functions:
ImgUploaded:
handler: src/stepfunctions/imageWasUploadedEvent.handler
events:
- s3:
bucket: !Ref AttachmentsBucket
existing: true
iamRoleStatements:
- Effect: "Allow"
Action:
- "states:StartExecution"
Resource:
- "*"
To check that the Step Function was working I created a log group and added it to the Step Function.
resources:
Resources:
StepFuncLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: /aws/stepfunctions/${self:service}-${self:provider.stage}
I see this cloud watch log group correctly associated with the Step function in the AWS Console.
However when I upload an object to S3, I do see in the logs of the lambda functions that it was invoked, but I can not see any logs on the Step Function log group
My question is:
Is the Step Function indeed working and it is just an issue with the logs in the Step Function?
Or it is that the Step Function itself is not working and the lambda function is working just a lambda function totally independent of the Step Function?
What I have to do so the lambda function gets trigger as part of the Step Function?
BR
After studying how StepFunctions work I finally arrived to the conclusion that this is a wrong pattern.
In no place on the documentation of the plugins or Amazon it says that what I did is a pattern
StepFunctions can be started by events on CloudWatch and those events could be originated on a change on S3. That is not what I did here
There is no a direct link between an action on S3 and a StepFunction
The link in AWS documentation could confuse someone to think otherwise. Here is the title Starting a State Machine Execution in Response to Amazon S3 Events
But the state machine does not start because an S3 events but because the CloudWatch log that this event generates
A lambda proxy function is a good way of invoking an State Machine. This is a easy to use pattern and very common as we can use it with SQS etc
So the correct response to this question is
The state machine does not start because it is never invoked. We just called a lambda function that was used as the StartAt for an StateMachine. This does mean I invoked the State Machine.
That is the reason why there is no logs for the state machine meanwhile there are correct logs for the lambda function
Hope this response helps
I will add more details and reference to this response
BR
Or it is that the Step Function itself is not working and the lambda function is working just a lambda function totally independent of the Step Function?
To verify whether lambda was invoked as part of step function or not can't you just check execution history from the step function console. Also unless you have explicitly configured s3 to publish events to lambda, your lambda will not be automatically invoked upon uploading files to s3.
What I have to do so the lambda function gets trigger as part of the Step Function?
To be able to call trigger step function on file-upload to s3 you can follow this tutorial: https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-cloudwatch-events-s3.html
I have an aws lambda function on top of which I am trying to create a cloudwatch alarm using cloudformation template.
Lets say I have a lambda function named MyPackage-MyLambdaFunctionName but when I see my lambda in the aws console, it says MyPackage-MyLambdaFunctionName-M2DEESRWNF6I.
I am able to create the alarm by passing the below in the dimensions:
Dimensions:
- Name: FunctionName
Value: MyPackage-MyLambdaFunctionName-M2DEESRWNF6I
(Referring to How to set lambda alarm for specific lambda using CloudFormation)
But the issue is the lambda function is having a random component at the end (-M2DEESRWNF6I).
I have a cloudformation template which goes in a few accounts and creates these lambda functions. And since these random identifiers are different in each account, I cant put the above in cloudformation template, since it will work for one account but not for the others.
So, how can I achieve creating these cloudwatch alarms on my lamdba functions using the cloudformation template?
such as does "Value" takes regex? I tried
MyPackage-MyLambdaFunctionName.*
which didnt work. Or is there a way to restrict the lambda function name to be just MyPackage-MyLambdaFunctionName (without the random identifier)
Any guidance/help is appreciated.
Thanks
The random string on the end of your lambda's function name is there because you're not setting the lambda name explicitly when it's created. From docs:
FunctionName
The name of the Lambda function, up to 64 characters in
length. If you don't specify a name, AWS CloudFormation generates one.
In general, this is correct thing to do because if you do set it, you can't change any of the parameters that require replacement (I can't think of any parameters that require that on lambda though).
From the same doc:
If you specify a name, you cannot perform updates that require
replacement of this resource. You can perform updates that require no
or some interruption. If you must replace the resource, specify a new
name.
To create the alarm you can just reference the Lambda function in the alarm creation code.
Like this:
Dimensions:
- Name: "FunctionName"
Value:
Ref: LambdaFunctionReference