Amazon SES 535 Authentication Credentials Invalid trying to rotate access key - amazon-web-services

Question

We have an Amazon SES setup that works well and sends thousands of emails a day via SMTP. Trying to follow a best practice of "rotating" access keys we went to
https://console.aws.amazon.com/iam/home and creating a new access key for the exact same user which is used to send emails. The new key is supposedly active but when trying to email with the access keys, we keep getting
535 Authentication Credentials Invalid
Switching to the old access keys works well and emails are sent. Tried a couple of times to delete the new access keys and create others. Same machine, same software. We have proper copy+paste skills to ensure we're using the same ID/Password provided in the CSV coming from Amazon. Here the dialog from Amazon:
So what's going on? Is there a time limit till the new key becomes active? Is there some other hidden limitation somewhere?

Answer 1

You are confusing the SMTP credentials with access_key and secret. They are different.
access_key/secret --> Use in SDK and CLI
SMTP credentials --> Use to configure SES SMTP
You are creating a new access_key/secret and using it as SMTP credentials
Instead you create a new SMTP credentials and use it
Key rotation is different from SMTP credential rotation
No need to create a new user
It is likely you are using the SMTP credentials that does not change even if you generate another set of access_key/secret. In your case it looks like you are using the SMTP server and not the SDK. So generating a new set of access_key/secret has no effect on SMTP credentials.
If you want to create a new set of SMTP credentials, go to AWS SES dashboard and create SMTP credentials.
For more information: Obtaining Your Amazon SES SMTP Credentials

Answer 2

Yes, there's a hidden limitation in the way AWS approaches the SMTP password for SES. And they are using a very confusing way of handling these credentials.
The answer from helloV is on the right track, but it's not entirely correct. Both AWS and his answer tell us that Access_key/Secret_key and SES SMTP credentials are different things, but:
If you create fresh SES SMTP credentials, it creates a new IAM User with an Access Key/Secret Key pair
The Access Key Id is the same as the username for SMTP
If you delete or disable this key, you lose your SMTP access. So they are clearly very related.
The password for SMTP is derived from the Secret Key
It turns out that a new access_key/secret_key pair on an existing IAM user, can be used for SMTP, and therefore keys can be rotated without creating new users.
AWS converts the Secret Access Key to generate the SMTP password, as they explain in this documentation page:
The following pseudocode shows the algorithm that converts an AWS Secret Access Key to an Amazon SES SMTP password.
key = AWS Secret Access Key;
message = "SendRawEmail";
versionInBytes = 0x02;
signatureInBytes = HmacSha256(message, key);
signatureAndVer = Concatenate(versionInBytes, signatureInBytes);
smtpPassword = Base64(signatureAndVer);
So using the Secret Access key, the SMTP password can be generated
With bash and openssl installed, the following command will output the password for use in SMTP:
(echo -en "\x02"; echo -n 'SendRawEmail' \
| openssl dgst -sha256 -hmac $AWS_SECRET_ACCESS_KEY -binary) \
| openssl enc -base64
Just replace $AWS_SECRET_ACCESS_KEY with your key, or set the variable beforehand

Answer 3

Since, both Secret Key and SMTP Password are in different format, you need to convert Secret Key to SMTP Password using algorithm provided by AWS.
You can find it here:
https://aws.amazon.com/premiumsupport/knowledge-center/ses-rotate-smtp-access-keys/

Answer 4

Here is a working piece of code for transformning your secret key to smtp password using bash :
#!/usr/bin/env bash
# Convert AWS Secret Access Key to an Amazon SES SMTP password
# using the following pseudocode:
#
# date = "11111111";
# service = "ses";
# terminal = "aws4_request";
# message = "SendRawEmail";
# version = 0x04;
#
# kDate = HmacSha256(date, "AWS4" + key);
# kRegion = HmacSha256(region, kDate);
# kService = HmacSha256(service, kRegion);
# kTerminal = HmacSha256(terminal, kService);
# kMessage = HmacSha256(message, kTerminal);
# signatureAndVersion = Concatenate(version, kMessage);
# smtpPassword = Base64(signatureAndVersion);
#
# Usage:
# chmod u+x aws-ses-smtp-password.sh
# ./aws-ses-smtp-password.sh secret-key-here
# See: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html
#
if [ "$#" -ne 1 ]; then
echo "Usage: ./aws-ses-smtp-password.sh secret-key-here"
exit 1
fi
KEY="${1}"
DATE="11111111"
REGION="eu-west-1"
SERVICE="ses"
TERMINAL="aws4_request"
MESSAGE="SendRawEmail"
VERSION="4"
VERSION_IN_BYTES=$(printf \\$(printf '%03o' "${VERSION}"));
#SIGNATURE_IN_BYTES=$(echo -n "${MESSAGE}" | openssl dgst -sha256 -hmac "${KEY}" -binary);
SIGNATURE_IN_BYTES=$(echo -n "${DATE}" | openssl dgst -sha256 -mac HMAC -macopt "key:AWS4${KEY}" | sed 's/^.* //');
SIGNATURE_IN_BYTES=$(echo -n "${REGION}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${SIGNATURE_IN_BYTES}" | sed 's/^.* //');
SIGNATURE_IN_BYTES=$(echo -n "${SERVICE}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${SIGNATURE_IN_BYTES}" | sed 's/^.* //');
SIGNATURE_IN_BYTES=$(echo -n "${TERMINAL}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${SIGNATURE_IN_BYTES}" | sed 's/^.* //');
SIGNATURE_IN_BYTES=$(echo -n "${MESSAGE}" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:${SIGNATURE_IN_BYTES}" -binary | sed 's/^.* //');
SIGNATURE_AND_VERSION="${VERSION_IN_BYTES}${SIGNATURE_IN_BYTES}"
SMTP_PASSWORD=$(echo -n "${SIGNATURE_AND_VERSION}" | base64);
echo "${SMTP_PASSWORD}"