I a writing a Python Cloud Function and I would like to retrieve the "delivery_attempt" attribute.
Based on the documentation, the Cloud Function gets only 2 parameters: event (of type PubsubMessage) and context.
def hello_pubsub(event, context):
"""Background Cloud Function to be triggered by Pub/Sub.
Args:
event (dict): The dictionary with data specific to this type of
event. The `#type` field maps to
`type.googleapis.com/google.pubsub.v1.PubsubMessage`.
The `data` field maps to the PubsubMessage data
in a base64-encoded string. The `attributes` field maps
to the PubsubMessage attributes if any is present.
context (google.cloud.functions.Context): Metadata of triggering event
including `event_id` which maps to the PubsubMessage
messageId, `timestamp` which maps to the PubsubMessage
publishTime, `event_type` which maps to
`google.pubsub.topic.publish`, and `resource` which is
a dictionary that describes the service API endpoint
pubsub.googleapis.com, the triggering topic's name, and
the triggering event type
`type.googleapis.com/google.pubsub.v1.PubsubMessage`.
Returns:
None. The output is written to Cloud Logging.
"""
How can I retrieve the delivery_attempt on a message?
When trigging a Cloud Function via Pub/Sub, one does not have access to the delivery attempt field. Instead, the way to get access is to set up an HTTP-based Cloud Function and use the trigger URL as the push endpoint in a subscription you create separately.
Then, you can access the delivery attempt as follows:
def hello_world(request):
"""Responds to any HTTP request.
Args:
request (flask.Request): HTTP request object.
Returns:
The response text or any set of values that can be turned into a
Response object using
`make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
"""
request_json = request.get_json()
print(request_json["deliveryAttempt"])
Related
I have 3 webhooks that calls my API Gateway which calls my Lambda Function.
url/webhook/....
I want each webhook to call its own python method
startDelivery --> def start_delivery(event, context):
UpdateStatus--> def update_status(event, context):
EndDelivery--> def end_delivery(event, context):
I understand most likely one method will be executed via "url/webhook" which calls the appropriate python method.
def Process_task to call one of the three
What is the ideal way to set up this structure?
Creating different urls for Webhooks and API Gateway captures it and somehow calls the handler?
url/webhook/start
url/webhook/status
url/webhook/end
Sending a different query string for each webhook? and in the lamba parse the query string and call the right python method?
Keep in mind that a Lambda function has one handler (=> 1 invocation = 1 method called).
You can achieve the 1 route <-> 1 method by doing one of the following:
You have a single Lambda function triggered by your 3 APIGW routes.
You can then add a simple router to your function which parse the event['path'] and call the appropriate method.
def lambda_handler(event, context):
path = event['path']
if path == '/webhook/start':
return start_delivery(event, context)
elif path == '/webhook/status':
return update_status(event, context)
elif path == '/webhook/end':
return end_status(event, context)
else:
return { "statusCode": 404, "body": "NotFound" }
Create 1 Lambda function by route:
webhook/start triggers the StartDelivery Lambda function with start_delivery as handler
webhook/status triggers the UpdateDelivery Lambda function with update_delivery as handler
webhook/end triggers the EndDelivery Lambda function with end_delivery as handler
You can use Infrastructure as Code (Cloudformation) to easily manage these functions (SAM: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html)
acorbel's answer was helpful for me. The latest version is Format 2.0. Many fields have changed. For example event['path'] doesn't exist with Format 2.0.
Please check the below link for correct key names of event structure.
Working with AWS Lambda proxy integrations for HTTP APIs
Im trying to create a simple API gateway in which, with a POST method to a certain endpoint, a lambda function is executed.
Setting that up was easy enough, but I'm having some trouble with the request/response handling. Im sending the following request to the API Gateway (Im using python 3.7).
payload = {
"data": "something",
"data2": "sometsadas"
}
response = requests.post('https://endpoint.com/test', params = payload)
That endpoint activates a lambda function when accesed. That function just returns the same event it received.
import json
def lambda_handler(event, context):
# TODO implement
return event
How can I make it so the return value of my lambda function is actually the response from the request? (Or at least a way in which the return value can be found somewhere inside the response)
Seems it was a problem with how the information is sent, json format is required. Solved it by doing the following in the code.
payload{'data': 'someData'}
config_response = requests.post(endpointURL, data = json.dumps(config_payload))
I just found out that I can get a max of 100 records for DBClusterSnapshots, luckily AWS supports pagination where you can get a list by page. I was going over the documentation for aws-sdk-go to see how my Operation implements pagination. Unfortunately there isn't a pagination method for my Operation.
This is the operation I want to paginate. It says in the doc that it supports pagination.
https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-cluster-snapshots.html
However the pagination method for my operation doesn't appear to be supported
doc: https://docs.aws.amazon.com/sdk-for-go/api/service/rds/
It only supports DBSnapshotsPages but not DBClusterSnapshotsPages
The AWS SDK for Go has the DescribeDBClusterSnapshots function:
func (c *RDS) DescribeDBClusterSnapshots(input *DescribeDBClusterSnapshotsInput) (*DescribeDBClusterSnapshotsOutput, error)
It accepts a parameter of DescribeDBClusterSnapshotsInput, which includes:
Marker *string type:"string"
An optional pagination token provided by a previous DescribeDBClusterSnapshots request. If this parameter is specified, the response includes only records beyond the marker, up to the value specified by MaxRecords.
Therefore, your code can call DescribeDBClusterSnapshots, store the marker that is returned, then make another call to DescribeDBClusterSnapshots, passing in that value for marker. This will return the next 'page' of results.
On aws sdk you can handle pagination by yourself using the response's next_page method to verify there's no pages to retrieve. In order to retrieve the next page of results, attaching some ruby example:
# object initializtion:
rds_client = Aws::RDS::Client.new
# implementation:
def self.describe_all_db_snapshots(db_instance_identifier: db_instance_identifier)
response = rds_client.describe_db_snapshots({
db_instance_identifier: db_instance_identifier,
snapshot_type: "automated",
include_shared: false,
include_public: false,
max_records: 100 })
while response.next_page? do
# use the response data here...
puts #{response}
# next pagination iterator
response = response.next_page
end
end
For more details read aws sdk documentation.
I have been trying to find the answer to this but am unable to find it anywhere. On the Cloud Functions section in the Google Cloud Platform console there is a section title 'Testing' but I have no idea what one is supposed to put here to test the function, i.e. syntax.
I have attached an image for clarity:
Any help would be much appreciated.
HTTPS Callable functions must be called using the POST method, the Content-Type must be application/json or application/json; charset=utf-8, and the body must contain a field called data for the data to be passed to the method.
Example body:
{
"data": {
"aString": "some string",
"anInt": 57,
"aFloat": 1.23,
}
}
If you are calling a function by creating your own http request, you may find it more flexible to use a regular HTTPS function instead.
Click Here for more information
Example with the Cloud Function default Hello_World that is inserted automatically whenever you create a new Cloud Function:
def hello_world(request):
"""Responds to any HTTP request.
Args:
request (flask.Request): HTTP request object.
Returns:
The response text or any set of values that can be turned into a
Response object using
`make_response <http://flask.pocoo.org/docs/1.0/api/#flask.Flask.make_response>`.
"""
request_json = request.get_json()
if request.args and 'message' in request.args:
return request.args.get('message')
elif request_json and 'message' in request_json:
return request_json['message']
else:
return f'Hello World!'
Must be tested with a json as the input args:
{
"message": "Hello Sun!"
}
Out in the Testing Tab:
Hello Sun!
In the Testing tab editor: since we give the function the args in form of a json as we would elsewise write them like in python3 -m main.py MY_ARG, and since "message" is a key of that json, it is found by the elif and returns the value of the dictionary key as the message, instead of "Hello World". If we run the script without json args, else: is reached in the code, and output is "Hello World!":
This looks to be the same as gcloud functions call, with the JSON required being the same as the --data provided in the CLI.
You can check the docs for examples using the CLI, and the CLI documentation itself for further details.
There are multiple ways you could test you cloud function.
1) Use a google emulator locally if you want to test your code before deployment.
https://cloud.google.com/functions/docs/emulator.
This would give you a similar localhost HTTP endpoint that you can send request to for testing your function.
2) Using GUI on deployed function: The triggering event is the json object that the function expects in the request body. For example:
{
"key": "value"
}
Based on your function code dependency for the request it should trigger the function.
Simple Tests for Cloud Pub/Sub:
{"data":"This is data"}
Base64 'Hello World !' message :
{"data":"SGVsbG8gV29ybGQgIQ=="}
In an AWS Lambda code, how can I get the HTTP method (e.g. GET, POST...) of an HTTP request coming from the AWS Gateway API?
I understand from the documentation that context.httpMethod is the solution for that.
However, I cannot manage to make it work.
For instance, when I try to add the following 3 lines:
if (context.httpMethod) {
console.log('HTTP method:', context.httpMethod)
}
into the AWS sample code of the "microservice-http-endpoint" blueprint as follows:
exports.handler = function(event, context) {
if (context.httpMethod) {
console.log('HTTP method:', context.httpMethod)
}
console.log('Received event:', JSON.stringify(event, null, 2));
// For clarity, I have removed the remaining part of the sample
// provided by AWS, which works well, for instance when triggered
// with Postman through the API Gateway as an intermediary.
};
I never have anything in the log because httpMethod is always empty.
The context.httpMethod approach works only in templates. So, if you want to have access to the HTTP method in your Lambda function, you need to find the method in the API Gateway (e.g. GET), go to the Integration Request section, click on Mapping Templates, and add a new mapping template for application/json. Then, select the application/json and select Mapping Template and in the edit box enter something like:
{
"http_method": "$context.httpMethod"
}
Then, when your Lambda function is called, you should see a new attribute in the event passed in called http_method which contains the HTTP method used to invoke the function.
API Gateway now has a built-in mapping template that passes along stuff like http method, route, and a lot more. I can't embed because I don't have enough points, but you get the idea.
Here is a screenshot of how you add it in the API Gateway console:
To get there navigate to AWS Console > API Gateway > (select a resource, IE - GET /home) > Integration Request > Mapping Templates > Then click on application/json and select Method Request Passthrough from dropdown shown in the screenshot above
I had this problem when I created a template microservice-http-endpoint-python project from functions.
Since it creates an HTTP API Gateway, and only REST APIs have Mapping template I was not able to put this work. Only changing the code of Lambda.
Basically, the code does the same, but I am not using the event['httpMethod']
Please check this:
import boto3
import json
print('Loading function')
dynamo = boto3.client('dynamodb')
def respond(err, res=None):
return {
'statusCode': '400' if err else '200',
'body': err.message if err else json.dumps(res),
'headers': {
'Content-Type': 'application/json',
},
}
def lambda_handler(event, context):
'''Demonstrates a simple HTTP endpoint using API Gateway. You have full
access to the request and response payload, including headers and
status code.
To scan a DynamoDB table, make a GET request with the TableName as a
query string parameter. To put, update, or delete an item, make a POST,
PUT, or DELETE request respectively, passing in the payload to the
DynamoDB API as a JSON body.
'''
print("Received event: " + json.dumps(event, indent=2))
operations = {
'DELETE': lambda dynamo, x: dynamo.delete_item(**x),
'GET': lambda dynamo, x: dynamo.scan(**x),
'POST': lambda dynamo, x: dynamo.put_item(**x),
'PUT': lambda dynamo, x: dynamo.update_item(**x),
}
operation = event['requestContext']['http']['method']
if operation in operations:
payload = event['queryStringParameters'] if operation == 'GET' else json.loads(event['body'])
return respond(None, operations[operation](dynamo, payload))
else:
return respond(ValueError('Unsupported method "{}"'.format(operation)))
I changed the code from:
operation = event['httpMethod']
to
operation = event['requestContext']['http']['method']
How do I get this solution?
I simply returned the entire event, checked the JSON and put it to work with the correct format.
If event appears an empty object, make sure you enabled proxy integration for the method. Proxy integration for an HTTP method adds request information into event.
See Use Lambda Proxy integration on API Gateway page.
If you are using API gateway, http method will be automatically passed to the event parameter when the lambda is triggered.
export const handler: Handler<APIGatewayProxyEvent> = async (
event: APIGatewayEvent,
context: Context
): Promise<APIGatewayProxyResult> => {
const httpMethod = event.httpMethod;
...
}