I am using https://zeit.co (free) and was thinking setup a webhook for telegram chat bot.
I sent a message from the telegram app on the phone and it supposes to post a json to the webhook url. It does post the data but i cannot get the json. It seems zeit.co cannot handle the json?
It is like something stuck whenever i tried to call request.json
#app.route("/new_message", methods=["POST", "GET"])
def telegram_webhook_handler():
try:
print(request.json)
if request.method == 'POST':
r = request.get_json()
chat_id = r['message']['chat']['id']
text = "how are you?"
send_message(CHAT_ID, text)
else:
send_message(CHAT_ID, "This is a get")
except Exception as e:
print(e)
pass
return jsonify({"ping": "pong"})
The error message from zeit.co
12/27 01:42 PM (40s)
REPORT RequestId: 3462880b-09d4-11e9-b07e-77492ad19973 Duration: 300021.80 ms Billed Duration: 300000 ms Memory Size: 1024 MB Max Memory Used: 42 MB
12/27 01:42 PM (40s)
2018-12-27T12:42:42.838Z 3462880b-09d4-11e9-b07e-77492ad19973 Task timed out after 300.02 seconds
Any idea how i can get the webhook data?
Cheers
Your code has exceeded its duration limit.
Duration: 300021.80 ms Billed Duration: 300000 ms
If you want to increase the duration limit, you will have to upgrade your Zeit account.
Related
I'm doing a test where I want my function to fail on timeout.
So i set my timeout to 1 sec
My expectation is that it will fail when I invoke the function, however it still succeeds with a duration over the set timeout
Duration: 22744.46 ms Billed Duration: 22745 ms
Memory Size: 128 MB Max Memory Used: 74 MB Init Duration: 639.17 ms
When timeout, you will see line like this
2022-04-01T23:59:10.675Z xxxx Task timed out after 3.00 seconds
next the duration, memory stuffs you post this above
I used following python code, hopes help
import json
import time
def lambda_handler(event, context):
time.sleep(4)
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
I am using AWS Lambda to host a nodeJs service that fetch my open invoices on Stripe and execute a payment and update my database.
The problem is that most of the time, but not all the time (sometimes everything goes how it should), it hang on the call of invoice list and do nothing.
Here's the part of the code where log stops :
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY, {
maxNetworkRetries: 1,
timeout: 2000
});
[other imports]
const microservice = async (event, context, callback) => {
[some code including database connection]
console.log('retrieving all open invoices...')
let invoices;
try {
invoices = await stripe.invoices.list({
status: 'open',
limit: 100,
});
console.log(invoices.data.length + ' data retrieved.');
} catch (error) {
console.log('Unable fetch stripe invoices : ', error);
console.log('Exiting due to stripe connection error.');
reports.push(new Report('Unable fetch stripe invoices', 'ERROR'));
return {
statusCode: 500,
};
}
[code that process invoices]
return {};
};
module.exports.microservice = microservice;
And here the log output :
START RequestId: d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 Version: $LATEST
2021-10-26T00:04:05.741Z d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 INFO Connecting to database...
2021-10-26T00:04:05.929Z d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 INFO Executing (default): SELECT 1+1 AS result
2021-10-26T00:04:05.931Z d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 INFO Connection has been established successfully.
2021-10-26T00:04:05.931Z d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 INFO retrieving all open invoices...
END RequestId: d628aa1e-dee6-4cc6-9ce0-f7c11cf73249
REPORT RequestId: d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 Duration: 15015.49 ms Billed Duration: 15000 ms Memory Size: 400 MB Max Memory Used: 40 MB
2021-10-26T00:04:20.754Z d628aa1e-dee6-4cc6-9ce0-f7c11cf73249 Task timed out after 15.02 seconds
And when it gooes all right it's like that :
START RequestId: e5fb6b08-adf9-433f-b1da-fd9ec29dde31 Version: $LATEST
2021-10-25T14:35:03.369Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO Connecting to database...
2021-10-25T14:35:03.590Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO Executing (default): SELECT 1+1 AS result
2021-10-25T14:35:03.600Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO Connection has been established successfully.
2021-10-25T14:35:03.600Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO retrieving all open invoices...
2021-10-25T14:35:04.011Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO 0 data retrieved.
2021-10-25T14:35:04.011Z e5fb6b08-adf9-433f-b1da-fd9ec29dde31 INFO Everything went smoothly !
END RequestId: e5fb6b08-adf9-433f-b1da-fd9ec29dde31
REPORT RequestId: e5fb6b08-adf9-433f-b1da-fd9ec29dde31 Duration: 646.58 ms Billed Duration: 647 ms Memory Size: 400 MB
I don't get why it hangs with no error or log...
Network issues can happen due to various reasons. In your case, what you can try doing is to reduce the limit (e.g. limit : 30) and set your client library to retry the connection again by setting maxNetworkRetries : 3 or number that fits your application. When this is set, Stripe will retry the connection when the timeout error occurs.
This is a perfect match for Step functions use cases. It will allow you to orchestrate the steps of getting the invoices and processing them and easily design a retry mechanism in case of errors.
It is not really a solution, but what is causing the issue is that Stripe take a very long time to return less than 100 results.
We found workaround in order to not fetch this list.
I am working in AWS GovCloud I have the following configuration in AWS Lambda:
A Lambda function which decodes a payload
A Kinesis Stream set as a trigger for the aforementioned function
A Lambda Destination (we have tried Lambda functions as well as SQS, SNS)
No matter the configuration, I cannot seem to get Lambda to trigger the destination function (or queue in the event of SQS).
Here is the current Lambda Function. I have tried many permutations of the result/return payload without avail.
import base64
import json
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
for record in event['Records']:
payload = base64.b64decode(record['kinesis']['data']).decode('utf-8', 'ignore')
print("Success")
result = {
"statusCode": 202,
"headers": {
#'Content-Type': 'application/json',
},
"body": '{payload}'
}
return json.dumps(result)
I then send a message to Kinesis with the AWS CLI (I have noted that "Test" in the console does not observe desintations as per Jared Short ).
Every 0.1s: aws kinesis put-records --stream-name test-stream --records Data=SGVsbG8sIHRoaXMgaXMgYSB0ZXN0IGZyb20gdGhlIEFXUyBDTEkh,PartitionKey=partitionkey1 Thu Jul 8 19:03:54 2021
{
"FailedRecordCount": 0,
"Records": [
{
"SequenceNumber": "49619938447946944252072058244333476686328287240252293122",
"ShardId": "shardId-000000000000"
}
]
}
Using Cloudwatch metrics and logs I am able to observe the function being triggered by the messages sent to Kinesis every .1 second.
The metrics charts indicate a success (as I expect).
Here is an example log from Cloudwatch:
START RequestId: 0cf3fb87-06e6-4e35-9de8-b30147e7be9d Version: $LATEST
Loading function
Success
END RequestId: 0cf3fb87-06e6-4e35-9de8-b30147e7be9d
REPORT RequestId: 0cf3fb87-06e6-4e35-9de8-b30147e7be9d Duration: 1.27 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 51 MB Init Duration: 113.64 ms
START RequestId: e663fa4a-2d0b-42d6-9e38-599712b71101 Version: $LATEST
Success
END RequestId: e663fa4a-2d0b-42d6-9e38-599712b71101
REPORT RequestId: e663fa4a-2d0b-42d6-9e38-599712b71101 Duration: 1.04 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 51 MB
START RequestId: b1373bbe-d2c6-49fb-a71f-dcedaf9210eb Version: $LATEST
Success
END RequestId: b1373bbe-d2c6-49fb-a71f-dcedaf9210eb
REPORT RequestId: b1373bbe-d2c6-49fb-a71f-dcedaf9210eb Duration: 0.98 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 51 MB
START RequestId: e0382653-9c33-44d6-82a7-a82f0f416297 Version: $LATEST
Success
END RequestId: e0382653-9c33-44d6-82a7-a82f0f416297
REPORT RequestId: e0382653-9c33-44d6-82a7-a82f0f416297 Duration: 1.05 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 51 MB
START RequestId: f9600ef5-419f-4271-9680-7368ccc5512d Version: $LATEST
Success
However, viewing the cloudwatch logs/metrics for the destination lambda function or SQS queue clearly show that the destination is not being triggered.
Over the course of troubleshooting, I have over-provisioned IAM policies to the Lambda function execution role so I am fairly confident that it is not an IAM related issue. Additionally, both functions are sharing the same execution role.
One thing I am not clear on after reviewing AWS documentation and 3rd party information is the criteria by which AWS determines success or failure for a given function. I am currently researching the invokation docs in search of what might be wrong here - but my interpretation is that AWS knows our function is successful based on the above Cloudwatch metrics showing a 100% success rate.
Does anyone know what I am doing wrong or how to try to troubleshoot the destination trigger for lambda?
Edit: As pointed out, the code is not correct for multiple record events. This is a function of senseless troubleshooting/changes to the code to get the Destination to trigger. Even something as simple as this does not invoke the destination.
import base64
import json
def lambda_handler(event, context):
#print("Received event: " + json.dumps(event, indent=2))
# for record in event['Records']:
# payload = base64.b64decode(record['kinesis']['data']).decode('utf-8', 'ignore')
# print("Success")
# result = {
# "statusCode": 202,
# "headers": {
# 'Content-Type': 'application/json',
# },
# "body": '{"Success":True, payload}'
# }
return { "result": "OK" }
So, the question: Can someone demonstrate it is possible to have a Kinesis Stream Event Source Trigger a Lambda Function which successfully triggers a Lambda destination in AWS Govcloud?
I am attempting to write a simple Lambda function to query a table in Athena. But after a few seconds I see "Status: FAILED" in the Cloudwatch logs.
There is no descriptive error message on the cause of failure.
My test code is below:
import json
import time
import boto3
# athena constant
DATABASE = 'default'
TABLE = 'test'
# S3 constant
S3_OUTPUT = 's3://test-output/'
# number of retries
RETRY_COUNT = 1000
def lambda_handler(event, context):
# created query
query = "SELECT * FROM default.test limit 2"
# % (DATABASE, TABLE)
# athena client
client = boto3.client('athena')
# Execution
response = client.start_query_execution(
QueryString=query,
QueryExecutionContext={
'Database': DATABASE
},
ResultConfiguration={
'OutputLocation': S3_OUTPUT,
}
)
# get query execution id
query_execution_id = response['QueryExecutionId']
print(query_execution_id)
# get execution status
for i in range(1, 1 + RETRY_COUNT):
# get query execution
query_status = client.get_query_execution(QueryExecutionId=query_execution_id)
query_execution_status = query_status['QueryExecution']['Status']['State']
if query_execution_status == 'SUCCEEDED':
print("STATUS:" + query_execution_status)
break
if query_execution_status == 'FAILED':
#raise Exception("STATUS:" + query_execution_status)
print("STATUS:" + query_execution_status)
else:
print("STATUS:" + query_execution_status)
time.sleep(i)
else:
# Did not encounter a break event. Need to kill the query
client.stop_query_execution(QueryExecutionId=query_execution_id)
raise Exception('TIME OVER')
# get query results
result = client.get_query_results(QueryExecutionId=query_execution_id)
print(result)
return
The logs show the following:
2020-08-31T10:52:12.443-04:00
START RequestId: e5434651-d36e-48f0-8f27-0290 Version: $LATEST
2020-08-31T10:52:13.481-04:00
88162f38-bfcb-40ae-b4a3-0b5a21846e28
2020-08-31T10:52:13.500-04:00
STATUS:QUEUED
2020-08-31T10:52:14.519-04:00
STATUS:RUNNING
2020-08-31T10:52:16.540-04:00
STATUS:RUNNING
2020-08-31T10:52:19.556-04:00
STATUS:RUNNING
2020-08-31T10:52:23.574-04:00
STATUS:RUNNING
2020-08-31T10:52:28.594-04:00
STATUS:FAILED
2020-08-31T10:52:28.640-04:00
....more status: FAILED
....
END RequestId: e5434651-d36e-48f0-8f27-0290
REPORT RequestId: e5434651-d36e-48f0-8f27-0290 Duration: 30030.22 ms Billed Duration: 30000 ms Memory Size: 128 MB Max Memory Used: 72 MB Init Duration: 307.49 ms
2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds
I think I have the right permissions for S3 bucket access given to the role (if not, I would have seen the error message). There are no files created in the bucket either. I am not sure what is going wrong here. What am I missing?
Thanks
The last line in your log shows
2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds
To me this looks like the timeout of the Lambda Function is set to 30 seconds. Try increasing it to more than the time the Athena query needs (the maximum is 15 minutes).
I'm requesting a file with a size around 14MB from a slow server with urllib2.urlopen, and it spend more than 60 seconds to get the data, and I'm getting the error:
Deadline exceeded while waiting for HTTP response from URL:
http://bigfile.zip?type=CSV
Here my code:
class CronChargeBT(webapp2.RequestHandler):
def get(self):
taskqueue.add(queue_name = 'optimized-queue', url='/cronChargeBTB')
class CronChargeBTB(webapp2.RequestHandler):
def post(self):
url = "http://bigfile.zip?type=CSV"
url_request = urllib2.Request(url)
url_request.add_header('Accept-encoding', 'gzip')
urlfetch.set_default_fetch_deadline(300)
response = urllib2.urlopen(url_request, timeout=300)
buf = StringIO(response.read())
f = gzip.GzipFile(fileobj=buf)
...work with the data insiste the file...
I create a cron task who calls CronChargeBT. Here the cron.yaml:
- description: cargar BlueTomato
url: /cronChargeBT
target: charge
schedule: every wed,sun 01:00
and it create a new task and insert into a queue, here the queue configuration:
- name: optimized-queue
rate: 40/s
bucket_size: 60
max_concurrent_requests: 10
retry_parameters:
task_retry_limit: 1
min_backoff_seconds: 10
max_backoff_seconds: 200
Of coursethat the timeout=300 isn't working because the 60seconds limit in GAE but I think yhat I can avoid it using a task... anyone knows how I can get the data in the file avoiding this timeout.
Thanks a lot!!!
Cron jobs are limited to 10 minutes deadline, not 60 seconds. If your download fails, perhaps just retry? Does the download work if you download it from your computer? There's nothing you can do on GAE if the server you are downloading from is too slow or unstable.
Edit: According to https://cloud.google.com/appengine/docs/java/outbound-requests#request_timeouts, there is a maximum deadline of 60 seconds for cron job requests. Therefore, you can't get around it.