I was ready to use SES for production so I had my sending limits increased. This is the email from AWS:
"Congratulations! After reviewing your case, we have increased your sending quota to 50,000 messages per day and your maximum send rate to 14 messages per second in AWS Region US East (N. Virginia). Your account has also been moved out of the sandbox, so you no longer need to verify recipient addresses."
I configured sSMTP so I can send email using the mail command, using AWS endpoint and generated SMTP credentials. I send an email and I get this:
"Oct 17 14:08:10 ia sSMTP[20486]: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: root , root#ia.internal.vdopia.com"
The SMTP endpoint is: email-smtp.us-east-1.amazonaws.com:587
What am I doing wrong?
Updated
Output of syslog:
Output of syslog:
Oct 19 07:29:44 ia sSMTP[427]: Creating SSL connection to host
Oct 19 07:29:44 ia sSMTP[427]: 220 email-smtp.amazonaws.com ESMTP SimpleEmailService-1652178317 ANxvvoY79LhkdX5l8cYI
Oct 19 07:29:44 ia sSMTP[427]: EHLO ia.internal.vdopia.com
Oct 19 07:29:44 ia sSMTP[427]: 250 Ok
Oct 19 07:29:44 ia sSMTP[427]: STARTTLS
Oct 19 07:29:44 ia sSMTP[427]: 220 Ready to start TLS
Oct 19 07:29:44 ia sSMTP[427]: SSL connection using RSA_AES_128_CBC_SHA1
Oct 19 07:29:44 ia sSMTP[427]: EHLO ia.internal.vdopia.com
Oct 19 07:29:44 ia sSMTP[427]: 250 Ok
Oct 19 07:29:44 ia sSMTP[427]: AUTH LOGIN
--- removing some lines
Oct 19 07:29:44 ia sSMTP[427]: 235 Authentication successful.
Oct 19 07:29:44 ia sSMTP[427]: MAIL FROM: <root#ia.internal.vdopia.com>
Oct 19 07:29:44 ia sSMTP[427]: 250 Ok
Oct 19 07:29:44 ia sSMTP[427]: RCPT TO:<ayush.sharma#vdopia.com>
Oct 19 07:29:44 ia sSMTP[427]: 250 Ok
Oct 19 07:29:44 ia sSMTP[427]: DATA
Oct 19 07:29:44 ia sSMTP[427]: 354 End data with <CR><LF>.<CR><LF>
Oct 19 07:29:44 ia sSMTP[427]: Received: by ia.internal.vdopia.com (sSMTP sendmail emulation); Wed, 19 Oct 2016 07:29:44 +0000
Oct 19 07:29:44 ia sSMTP[427]: From: "root" <root#ia.internal.vdopia.com>
Oct 19 07:29:44 ia sSMTP[427]: Date: Wed, 19 Oct 2016 07:29:44 +0000
Oct 19 07:29:44 ia sSMTP[427]: Subject: testing
Oct 19 07:29:44 ia sSMTP[427]: To: <ayush.sharma#vdopia.com>
Oct 19 07:29:44 ia sSMTP[427]: X-Mailer: mail (GNU Mailutils 2.99.98)
Oct 19 07:29:44 ia sSMTP[427]:
Oct 19 07:29:45 ia sSMTP[427]: .
Oct 19 07:29:45 ia sSMTP[427]: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: root <root#ia.internal.vdopia.com>, root#ia.internal.vdopia.com
SMTP Response Codes Returned by Amazon SES
When your account is moved out of sand box, you do not have to verify the recipients' addresses. But you still have to verify the sender's address or domain. From your post, it appears you have not verified the sender's address. Remember to verify the address/domain that appears in:
From
Source
Sender / Return-Path
Can you post your actual mail command/script that you are using to send the mail?
Nevertheless this question is answered correctly, maybe this tip helps out other people. After the activation and switch from AWS SES Sandbox to Production mode (support request of limit increase) I realized, that using the old "SMTP IAM User" caused the same problem. Just create a new "SMTP IAM User" after production grant. I really cannot explain it but that worked several times now.
Related
I am running Django dev server in docker with celery and redis on my remote host machine.
Everything works fine like 30 mins, and then redis starts falling into infinite loop while MASTER <-> REPLICA sync started
Here's the console output:
redis_1 | 1:S 16 Feb 2023 17:42:37.119 * Non blocking connect for SYNC fired the event.
redis_1 | 1:S 16 Feb 2023 17:42:37.805 # Failed to read response from the server: No error information
redis_1 | 1:S 16 Feb 2023 17:42:37.805 # Master did not respond to command during SYNC handshake
redis_1 | 1:S 16 Feb 2023 17:42:38.057 * Connecting to MASTER 194.40.243.205:8886
redis_1 | 1:S 16 Feb 2023 17:42:38.058 * MASTER <-> REPLICA sync started
redis_1 | 1:S 16 Feb 2023 17:42:38.111 * Non blocking connect for SYNC fired the event.
redis_1 | 1:S 16 Feb 2023 17:42:39.194 * Master replied to PING, replication can continue...
redis_1 | 1:S 16 Feb 2023 17:42:39.367 * Partial resynchronization not possible (no cached master)
redis_1 | 1:S 16 Feb 2023 17:42:39.449 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
redis_1 | 1:S 16 Feb 2023 17:42:39.449 * MASTER <-> REPLICA sync: receiving 54992 bytes from master to disk
redis_1 | 1:S 16 Feb 2023 17:42:39.607 * MASTER <-> REPLICA sync: Flushing old data
redis_1 | 1:S 16 Feb 2023 17:42:39.608 * MASTER <-> REPLICA sync: Loading DB in memory
redis_1 | 1:S 16 Feb 2023 17:42:39.621 # Wrong signature trying to load DB from file
redis_1 | 1:S 16 Feb 2023 17:42:39.623 # Failed trying to load the MASTER synchronization DB from disk: Invalid argument
redis_1 | 1:S 16 Feb 2023 17:42:39.624 * Reconnecting to MASTER 194.40.243.205:8886 after failure
redis_1 | 1:S 16 Feb 2023 17:42:39.625 * MASTER <-> REPLICA sync started
redis_1 | 1:S 16 Feb 2023 17:42:39.709 * Non blocking connect for SYNC fired the event.
redis_1 | 1:S 16 Feb 2023 17:42:40.891 # Failed to read response from the server: No error information
redis_1 | 1:S 16 Feb 2023 17:42:40.891 # Master did not respond to command during SYNC handshake
redis_1 | 1:S 16 Feb 2023 17:42:41.069 * Connecting to MASTER 194.40.243.205:8886
redis_1 | 1:S 16 Feb 2023 17:42:41.069 * MASTER <-> REPLICA sync started
redis_1 | 1:S 16 Feb 2023 17:42:41.128 * Non blocking connect for SYNC fired the event.
redis_1 | 1:S 16 Feb 2023 17:42:42.167 # Failed to read response from the server: No error information
redis_1 | 1:S 16 Feb 2023 17:42:42.167 # Master did not respond to command during SYNC handshake
redis_1 | 1:S 16 Feb 2023 17:42:43.074 * Connecting to MASTER 194.40.243.205:8886
and this is only a few seconds.
My docker-compose file:
services:
redis:
image: redis:latest
restart: always
ports:
- "6379:6379"
django:
image: docker-app:django
container_name: django-app
build: .
volumes:
- .:/app/
ports:
- "8000:8000"
celery:
image: celery-app
restart: on-failure
build: .
command:
- celery
- -A
- myapp.celery_app
- worker
- -B
- --loglevel=INFO
volumes:
- .:/app/
container_name: celery
depends_on:
- django
I have no idea what's wrong here, 'cause on local machine everything works fine. I aslo tried apk-get update && apk-get upgrade, nothing changed
EDIT
Output on redis start
redis_1 | 1:C 16 Feb 2023 18:02:56.934 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 16 Feb 2023 18:02:56.935 # Redis version=7.0.4, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 16 Feb 2023 18:02:56.935 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 16 Feb 2023 18:02:56.936 * Increased maximum number of open files to 10032 (it was originally set to 1024).
redis_1 | 1:M 16 Feb 2023 18:02:56.936 * monotonic clock: POSIX clock_gettime
redis_1 | 1:M 16 Feb 2023 18:02:56.943 * Running mode=standalone, port=6379.
redis_1 | 1:M 16 Feb 2023 18:02:56.943 # Server initialized
redis_1 | 1:M 16 Feb 2023 18:02:56.945 * Loading RDB produced by version 7.0.8
redis_1 | 1:M 16 Feb 2023 18:02:56.945 * RDB age 1201 seconds
redis_1 | 1:M 16 Feb 2023 18:02:56.946 * RDB memory usage when created 1.44 Mb
redis_1 | 1:M 16 Feb 2023 18:02:56.946 * Done loading RDB, keys loaded: 0, keys expired: 0.
redis_1 | 1:M 16 Feb 2023 18:02:56.946 * DB loaded from disk: 0.001 seconds
redis_1 | 1:M 16 Feb 2023 18:02:56.946 * Ready to accept connections
I have an API Gateway which routes requests to SQS, API Gateway has a the needed permission to send messages to SQS (Created IAM role), when I send test data, I'll get the response below with UnknownOperationException.
SQS is also empty with no messages available.
Tue Nov 09 13:46:11 UTC 2021 : Sending request to https://sqs.ca-central-1.amazonaws.com/XXXXXXXX/XXXX
Tue Nov 09 13:46:11 UTC 2021 : Received response. Status: 404, Integration latency: 2 ms
Tue Nov 09 13:46:11 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=e63b5577-9866-569d-85d8-0f73e5851cf0, Date=Tue, 09 Nov 2021 13:46:11 GMT, X-Amzn-Trace-Id=Root=1-618a7ba3-dd93ee11df45d437f9c98705, Content-Type=null, Content-Length=29}
Tue Nov 09 13:46:11 UTC 2021 : Endpoint response body before transformations: <UnknownOperationException/>
Tue Nov 09 13:46:11 UTC 2021 : Method response body after transformations: <UnknownOperationException/>
Any ideas?
To fix that issue you have to add the Content-Type:'application/x-www-form-urlencoded' header to the integration request:
integration request headers
Execution failed due to configuration error: Illegal character in path at index 46: http://petstore-demo-endpoint.execute-api.com/{proxy}?type=fish
Logs :
Execution log for request 965a6cd1-1f68-4b33-8bab-3741e40b0028
Fri Mar 20 08:19:39 UTC 2020 : Starting execution for request: 965a6cd1-1f68-4b33-8bab-3741e40b0028
Fri Mar 20 08:19:39 UTC 2020 : HTTP Method: GET, Resource Path: /test/petstore/pets
Fri Mar 20 08:19:39 UTC 2020 : Method request path: {proxy=petstore/pets}
Fri Mar 20 08:19:39 UTC 2020 : Method request query string: {type=fish}
Fri Mar 20 08:19:39 UTC 2020 : Method request headers: {}
Fri Mar 20 08:19:39 UTC 2020 : Method request body before transformations:
Fri Mar 20 08:19:39 UTC 2020 : Endpoint request URI: http://petstore-demo-endpoint.execute-api.com/{proxy}?type=fish
Fri Mar 20 08:19:39 UTC 2020 : Endpoint request headers: {x-amzn-apigateway-api-id=ssg89ys6y6, User-Agent=AmazonAPIGateway_ssg89ys6y6}
Fri Mar 20 08:19:39 UTC 2020 : Endpoint request body after transformations:
Fri Mar 20 08:19:39 UTC 2020 : Sending request to http://petstore-demo-endpoint.execute-api.com/{proxy}?type=fish
Fri Mar 20 08:19:39 UTC 2020 : Execution failed due to configuration error: Illegal character in path at index 46: http://petstore-demo-endpoint.execute-api.com/{proxy}?type=fish
Fri Mar 20 08:19:39 UTC 2020 : Method completed with status: 500
Adding this here for others who face the issue
https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-proxy-path-character-error/
I'm using API Gateway's AWS Service integration type to add logs to the Cloudwatch Logs service using the PutLogEvents action, as described here: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html
I've successfully setup a similar type of method to add items into a DynamoDB table using the PutItem action and that worked just fine, so I'm not sure what I'm missing with this one.
I've double checked for typos and problems with my template. Have successfully used PutLogEvents using the CLI tools, so the setup seems all OK.
Here's some screenshots of my setup from the AWS dash:
Here's the mapping template I'm using:
{
"logGroupName": "FromAPI",
"logStreamName": "$input.path('$.streamName')",
"logEvents": [
{
"timestamp": $input.path('$.ts'),
"message": "$input.path('$.message')"
}
]
}
The response I get back has a 200 status code, but the following response body:
{
"Output": {
"__type": "com.amazon.coral.service#UnknownOperationException",
"message": null
},
"Version": "1.0"
}
Here's a redacted (xxx) execution log:
Execution log for request xxx
Fri Apr 19 02:28:58 UTC 2019 : Starting execution for request: xxx
Fri Apr 19 02:28:58 UTC 2019 : HTTP Method: POST, Resource Path: /log
Fri Apr 19 02:28:58 UTC 2019 : Method request path: {}
Fri Apr 19 02:28:58 UTC 2019 : Method request query string: {}
Fri Apr 19 02:28:58 UTC 2019 : Method request headers: {}
Fri Apr 19 02:28:58 UTC 2019 : Method request body before transformations: {
"streamName": "12345",
"ts": 1555641510000,
"message": "help!"
}
Fri Apr 19 02:28:58 UTC 2019 : Endpoint request URI: https://logs.xxx.amazonaws.com/?Action=PutLogEvents
Fri Apr 19 02:28:58 UTC 2019 : Endpoint request headers: {Authorization=xxx, X-Amz-Date=20190419T022858Z, x-amzn-apigateway-api-id=xxx, Accept=application/json, User-Agent=AmazonAPIGateway_xxx, X-Amz-Security-Token=xxx [TRUNCATED]
Fri Apr 19 02:28:58 UTC 2019 : Endpoint request body after transformations: {
"logGroupName": "FromAPI",
"logStreamName": "12345",
"logEvents": [
{
"timestamp": 1555641510000,
"message": "help!"
}
]
}
Fri Apr 19 02:28:58 UTC 2019 : Sending request to https://logs.xxx.amazonaws.com/?Action=PutLogEvents
Fri Apr 19 02:28:58 UTC 2019 : Received response. Status: 200, Integration latency: 38 ms
Fri Apr 19 02:28:58 UTC 2019 : Endpoint response headers: {x-amzn-RequestId=xxx, Content-Type=application/json, Content-Length=105, Date=Fri, 19 Apr 2019 02:28:58 GMT}
Fri Apr 19 02:28:58 UTC 2019 : Endpoint response body before transformations: {"Output":{"__type":"com.amazon.coral.service#UnknownOperationException","message":null},"Version":"1.0"}
Fri Apr 19 02:28:58 UTC 2019 : Method response body after transformations: {"Output":{"__type":"com.amazon.coral.service#UnknownOperationException","message":null},"Version":"1.0"}
Fri Apr 19 02:28:58 UTC 2019 : Method response headers: {X-Amzn-Trace-Id=Root=xxx, Content-Type=application/json}
Fri Apr 19 02:28:58 UTC 2019 : Successfully completed execution
Fri Apr 19 02:28:58 UTC 2019 : Method completed with status: 200
Nothing gets logged into my Cloudwatch Logs stream - the API Gateway integration request response body contains an UnknownOperationException error.
My best guess is that this request is not mapping to the PutLogEvents action for some reason. Strange that the status code is 200 in this situation.
I'm guessing it's just some typo - or an additional header that I need to send through? Any ideas?
It should work if you add the following lines at the top of your Mapping Template:
#set($context.requestOverride.header['X-Amz-Target'] = "Logs_20140328.PutLogEvents")
#set($context.requestOverride.header['Content-Type'] = "application/x-amz-json-1.1")
This is very tricky and not well documented. You can find those headers in the sample request for PutLogEvents.
As per here, I believe that setting a header of X-Amz-Invocation-Type: Event should set my Lambda invocation to be asynchronous.
However, by putting a import time;time.sleep(5000) at the beginning of my Lambda function, and sending requests to my API Gateway, I observe that:
$ aws apigateway get-integration --rest-api-id <api-id> \
--resource-id <resource-id> \
--http-method POST | jq -r '.requestParameters'
{
"integration.request.header.X-Amz-Invocation-Type": "'Event'"
}
$ aws apigateway get-integration --rest-api-id <api-id> \
--resource-id <resource-id> \
--http-method POST | jq -r '.uri'
arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:<account-id>:function:[...]Lambda-4HOA0ZSFAYCI/invocations
$ curl https://<api-id>.execute-api.us-east-1.amazonaws.com/LATEST/<path> -d '{}'
{"message": "Endpoint request timed out"}
[here I removed the sleep from my lambda function and made it return immediately]
$ curl https://<api-id>.execute-api.us-east-1.amazonaws.com/LATEST/<path> -d '{}'
{"body": "Request OK", "headers": {"Content-Type": "application/json"}, "statusCode": 200}
Assuming that the documentation is accurate, my best guess is that I've misconfigured the Integration somehow - perhaps the "'Event'" is incorrect and it should be "Event"? I'm pretty sure the single-quotes are required, however, to demarcate a static value (as opposed to value parsed from the request). In particular, I suspect that my responseParameters are not right:
$ aws apigateway get-integration --rest-api-id <api-id> \
--resource-id dzv1zj \
--http-method POST | jq -r '.integrationResponses'
{
"200": {
"responseTemplates": {
"application/json": null
},
"statusCode": "200"
}
}
Should null there be some VTL that staticly returns a 200 OK?
As for alternatives: I see that invoke-async is deprecated. I'd really rather not go to the overhead of going API Gateway -> SNS -> Lambda.
EDIT: Here are the logs from calling the API via the "Test" option on the console:
Execution log for request test-request
Wed Mar 07 17:24:57 UTC 2018 : Starting execution for request: test-invoke-request
Wed Mar 07 17:24:57 UTC 2018 : HTTP Method: POST, Resource Path: /<path>
Wed Mar 07 17:24:57 UTC 2018 : Method request path: {}
Wed Mar 07 17:24:57 UTC 2018 : Method request query string: {}
Wed Mar 07 17:24:57 UTC 2018 : Method request headers: {}
Wed Mar 07 17:24:57 UTC 2018 : Method request body before transformations: {"abc":"def"}
Wed Mar 07 17:24:57 UTC 2018 : Endpoint request URI: https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:<account-id>:function:[...]Lambda-4HOA0ZSFAYCI/invocations
Wed Mar 07 17:24:57 UTC 2018 : Endpoint request headers: {X-Amz-Date=20180307T172457Z, x-amzn-apigateway-api-id=<api-id>, Accept=application/json, User-Agent=AmazonAPIGateway_<api-id>, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=2c3fbda5f48b04e39d3a87f89e5bd00b48b6e5e3c4a093de65de0a87b8cc8b3b, X-Amzn-Trace-Id=Root=1-5aa02069-8670eb5d98dbc4ade9df03d8, x-amzn-lambda-integration-tag=test-request, Authorization=**********************************************************************************************************************************************************************************************************************************************************************************************************************************************bfe3de, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:<account-id>:<api-id>/null/POST/<path>, X-Amz-Invocation-Type=Event, X-Amz-Security-Token=[REDACTED] [TRUNCATED]
Wed Mar 07 17:24:57 UTC 2018 : Endpoint request body after transformations: {"abc":"def"}
Wed Mar 07 17:24:57 UTC 2018 : Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:<account-id>:function:[...]Lambda-4HOA0ZSFAYCI/invocations
Wed Mar 07 17:24:57 UTC 2018 : Received response. Integration latency: 43 ms
Wed Mar 07 17:24:57 UTC 2018 : Endpoint response body before transformations:
Wed Mar 07 17:24:57 UTC 2018 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, Connection=keep-alive, x-amzn-RequestId=75501cbf-222c-11e8-a1fc-2b19f19a9429, Content-Length=0, Date=Wed, 07 Mar 2018 17:24:57 GMT, X-Amzn-Trace-Id=root=1-5aa02069-8670eb5d98dbc4ade9df03d8;sampled=0}
Wed Mar 07 17:24:57 UTC 2018 : Method response body after transformations:
Wed Mar 07 17:24:57 UTC 2018 : Method response headers: {X-Amzn-Trace-Id=sampled=0;root=1-5aa02069-8670eb5d98dbc4ade9df03d8, Content-Type=application/json}
Wed Mar 07 17:24:57 UTC 2018 : Successfully completed execution
Wed Mar 07 17:24:57 UTC 2018 : Method completed with status: 200
Now I feel stupid - I needed to deploy my API. That seems like the API Gateway equivalent of "Have you tried turning it off and turning it on again?" :)
I found scubbo's answer but it didn't enlighten me until much later, so I post the same, but with a screenshot for any future lost soul