I have written and saved a lambda function. I see:
Congratulations! Your Lambda function "lambda_name" has been
successfully created. You can now change its code and configuration.
Choose Test to input a test event when you want to test your function.
Now how do I run it? I cannot see a 'run' or 'invoke' button as I would expect
Note
The lambda doesn't accept any arguments (it's extremely simple - for the purposes of this question, please presume it's simply 2 * 2 so when I run it it should not require any inputs and should return 4).
Also note
I can see a tonne of different ways to run the lambda here. I just want the simplest way (preferably a button in the browser)
Sending a test message via the Lambda console will run your Lambda function. The test message that you configure will define what is in the event parameter of your lambda handler function.
Since you are not doing anything with that message, you can send any arbitrary test message and it should work for you. You can just use the default hello world message and give it an arbitrary name.
It should then show you the results: any logs or returned objects right in the AWS Lambda console.
Further reading here
AWS Lambda functions are typically triggered by an event, such as an object being uploaded to Amazon S3 or a message being send to an Amazon SNS topic.
This is because Lambda functions are great at doing a small task very often. Often, Lambda functions only run for a few seconds, or even less than a second! Thus, they are normally triggered in response to something else happening. It's a bit like when somebody rings your phone, which triggers you to answer the phone. You don't normally answer your phone when it isn't ringing.
However, it is also possible to directly invoke an AWS Lambda function using the Invoke() command in the AWS SDK. For convenience, you can also use the AWS Command-Line Interface (CLI) aws lambda invoke command. When directly invoking an AWS Lambda function, you can receive a return value. This is in contrast to situations where a Lambda function is triggered by an event, in which case there is nowhere to 'return' a value since it was not directly invoked.
Related
I am new to Lambda functions, and am using the serverless framework to create them, specifically using python. I am trying to create a function that will do some heavy processing of a file. The type of work is not important, but each job might take several minutes to complete.
I'd prefer not to have an invocation request open for so long, and instead I'd like the Lambda to return immediately, but continue processing after it has returned. Once the job is done, it could email me the processed file using SMS, for example.
Is this possible with AWS Lambda?
AWS Lambda supports two invocation types:
RequestResponse
Event
RequestResponse will wait until the Lambda invocation is finished before returning the response. Event will invoke the Lambda function and then return a success status indicating that the function was successfully started. You can read more about this here.
In actual fact, Lambda functions are ‘almost’ always requested asynchronously.
For your specific use case, you can have the lambda be ‘triggered’ when a file is uploaded to s3 (instead of invoking it programmatically). This means every time a file is uploaded to s3, the lambda is triggered and will process the file.
That way, no connection is open for the duration of that processing. The function can then notify it once it has completed.
I currently have a bunch of AWS Lambda functions and would like to spin them each up in containers (preferably Docker) to integrate with the current CICD pipeline. It's totally OK to have these containers spun up in an Amazon service such as Fargate -- the important thing is that it must be spun up in a container. The closest thing I've found is this repo by Amazon that converts images into Lambdas, though this is the opposite of what I'm looking for.
Is this task possible, and if so what's the best way to go about it?
Behind the scenes, a Lambda "function" is a Python script that listens for invocation events and invokes the actual handler function. This means that you can pretty easily invoke the function using the standard Python "script" code:
if __name__ == "__main__":
args = # this is the tricky part
ctx = # some dict that looks like what Lambda expects
handler(args, ctx)
These two parameters are just dicts, so if you can populate them with something that looks right then you can wrap the entire thing in a container and deploy it to ECS or Batch, or your own Docker daemon.
The problem is how you populate them, and that depends on what your lambda functions do -- which you don't say -- and how they're invoked -- which you also don't say.
You could use command-line parameters. For example, if your lambda is intended to respond to an S3 PutObject operation, then all you really need is the bucket and key name. From that you can easily create a dummy S3 invocation event, using fake values for all of the other fields.
If you invoke your Lambda in response to an SQS message it's even easier: your "main" handler enters a loop that polls SQS, and reformats the messages that it pulls off the queue.
Other event sources are going to be much harder. It may make sense (depending on why you're doing this) to use SQS for all invocations, and simply pass a fully-formed event message into the queue.
I need my python lambda to invoke multiple lambda so they can run in parallel. I have it working - kind of. To test it, I invoke the code below for 5 different lambda calls (could be more in the future)... when i look at the Step Function console, I only see 2 lamdbas running concurrently and the other 3 are ignored after the first 2 finished. In researching concurrency issues, I didnt find a limit that is this low (5 should be nothing for AWS!)..
response = lambdaClient.invoke(
FunctionName=lambdaToInvoke <---this var is an ARN,
InvocationType='Event',
Payload=json.dumps(event) <--just passing state
)
All IAM is provisioned...Any thoughts are appreciated!
Usually when I need to do a "fan out" operation like you are describing I don't invoke the lambda functions directly.
Rather I would recommend you setup your 5 "slave" lambda functions to get activated from either SNS or SQS. From your "master" function you can use the SDK to create a SNS alert or SQS message.
What happens when you try to manually invoke the 3 functions that don't run?
Do you have logging setup in CW logs? Can you post?
I'm trying to implement an AWS Lambda function that should send an HTTP request. If that request fails (response is anything but status 200) I should wait another hour before retrying (longer that the Lambda stays hot). What the best way to implement this?
What comes to mind is to persist my HTTP request in some way and being able to trigger the Lambda function again in a specified amount of time in case of a persisted HTTP request. But I'm not completely sure which AWS service that would provide that functionality for me. Is SQS an option that can help here?
Or, can I dynamically schedule Lambda execution for this? Note that the request to be retried should be identical to the first one.
Any other suggestions? What's the best practice for this?
(Lambda function is my option. No EC2 or such things are possible)
You can't directly trigger Lambda functions from SQS (at the time of writing, anyhow).
You could potentially handle the non-200 errors by writing the request data (with appropriate timestamp) to a DynamoDB table that's configured for TTL. You can use DynamoDB Streams to detect when DynamoDB deletes a record and that can trigger a Lambda function from the stream.
This is obviously a roundabout way to achieve what you want but it should be simple to test.
As jarmod mentioned, you cannot trigger Lambda functions directly by SQS. But a workaround (one I've used personally) would be to do the following:
If the request fails, push an item to an SQS Delay Queue (docs)
This SQS message will only become visible on the queue after a certain delay (you mentioned an hour).
Then have a second scheduled lambda function which is triggered by a cron value of a smaller timeframe (I used a minute).
This second function would then scan the SQS queue and if an item is on the queue, call your first Lambda function (either by SNS or with the AWS SDK) to retry it.
PS: Note that you can put data in an SQS item, since you mentioned you needed the lambda functions to be identical you can store your first function's input in here to be reused after an hour.
I suggest that you take a closer look at the AWS Step Functions for this. Basically, Step Functions is a state machine that allows you to execute a Lambda function, i.e. a task in each step.
More information can be found if you log in to your AWS Console and choose the "Step Functions" from the "Services" menu. By pressing the Get Started button, several example implementations of different Step Functions are presented. First, I would take a closer look at the "Choice state" example (to determine wether or not the HTTP request was successful). If not, then proceed with the "Wait state" example.
My Amazon Lambda function (in Python) is called when an object 123456 is created in S3's input_bucket, do a transformation in the object and saves it in output_bucket.
I would like to notify my main application if the request was successful or unsuccessful. For example, a POST http://myapp.com/successful/123456 if the processing is successful and http://myapp.com/unsuccessful/123456 if its not.
One solution I thought is to create a second Amazon Lambda function that is triggered by a put event in output_bucket, and it to do the successful POST request. This solves half of the problem because but I can't trigger the unsuccessful POST request.
Maybe AWS has a more elegant solution using a parameter in Lambda or a service that deals with these types of notifications. Any advice or point in the right direction will be greatly appreciated.
Few possible solutions which I see as elegant
Using SNS Topic: From your transformation lambda, trigger a SNS topic, with success/unsuccess message, where SNS will call a HTTP/HTTPS endpoint with message payload. The advantage here is, your transformation lambda is loosely coupled with endpoint trigger and only connected through messaging.
Using Lambda Step Functions:
You could arrange to run a Lambda function every time a new object is uploaded to an S3 bucket. This function can then kick off a state machine execution by calling StartExecution. The advantage in using step functions is that you can coordinate the components of your application as series of steps in a visual workflow.
I don't think there is any elegant AWS solution, unless you re-architect, something like your lambda sends message to SQS or some intermediatery messaging service with STATUS and then interemdeiatery invokes POST to your application.
If you still want to go with your way of solving, you might need to configure "DeadLetter queue" to do error handling in failure cases (note that use cases described here are not comprehensive, so need to make sure it covers your case) like described here.