Authenticate an AWS SQS scaler in Keda - amazon-web-services

Question

I have a Keda deployment that I've been trying to get to work for about a month now. At the moment, my scaler looks like this:
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: {service-name}-scaler
spec:
scaleTargetRef:
deploymentName: {service-name}
containerName: {service-name}
pollingInterval: 30
cooldownPeriod: 600
minReplicaCount: 0
maxReplicaCount: 10
triggers:
- type: aws-sqs-queue
authenticationRef:
name: keda-trigger-authentication
metadata:
queueURL: https://sqs.ap-northeast-1.amazonaws.com/{AWS ID}/{Queue-name}
queueLength: "1"
awsRegion: "ap-northeast-1"
identityOwner: pod
The associated trigger authentication and secret are:
apiVersion: v1
kind: Secret
metadata:
name: keda-secrets
data:
AWS_ACCESS_KEY_ID: {base64-encoded-string}
AWS_SECRET_ACCESS_KEY: {base64-encoded-string}
KEDA_ROLE_ARN: {base64-encoded-string}
---
apiVersion: keda.k8s.io/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-authentication
spec:
env:
- parameter: awsRegion
name: AWS_REGION
- parameter: awsAccessKeyID
name: AWS_ACCESS_KEY_ID
- parameter: awsSecretAccessKey
name: AWS_SECRET_ACCESS_KEY
- parameter: awsRoleArn
name: KEDA_ROLE_ARN
secretTargetRef:
- parameter: awsRoleArn
name: keda-secrets
key: KEDA_ROLE_ARN
I understand that the KEDA_ROLE_ARN value is repeated here; I left both for debugging purposes. The order of deploying this is as follows:
Install common environment variables (this is where the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and KEDA_ROLE_ARN values are stored. The AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values are listed as AWS_ACCESS_KEY_ID_ASSUME and AWS_SECRET_ACCESS_KEY_ASSUME respectively in the file and will assume their appropriate values on the container. Again, these are duplicated for debugging purposes. I would prefer to use these values rather than a separate secret.
Install Keda pods with Helm
Deploy the keda-secrets secret and the keda-trigger-authentication trigger authentication
Deploy the container that should be scaled. This is where the AWS_ACCESS_KEY_ID_ASSUME value will assume the name of AWS_ACCESS_KEY_ID and the AWS_SECRET_ACCESS_KEY_ASSUME value will assume the name of AWS_SECRET_ACCESS_KEY and where the AWS_REGION value is defined.
The scaled object is deployed
For some reason, I keep getting an error from AWS when the scaler attempts to scale saying that there are no credential providers in the chain. It appears that the AWS credentials are not being sent. What am I doing wrong here?

Answer 1

I will show you two ways to successfully scale deployment based on AWS SQS
First way : Using AWS IAM role attached to node
If your IAM role (node role) has permission to SQS then accessing SQS becomes easier you just have to change identityOwner: pod field to identityOwner: operator so that KEDA can use node role to access AWS SQS
Sample ScaledObject file with SQS trigger
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: aws-sqs-queue-scaledobject
namespace: default
spec:
scaleTargetRef:
name: test-deployment
minReplicaCount: 0
maxReplicaCount: 2
triggers:
- type: aws-sqs-queue
metadata:
queueURL: https://sqs.us-east-1.amazonaws.com/3243234432432/Queue
queueLength: "5"
awsRegion: "us-east-1"
identityOwner: operator
Second way: Using IAM user
In this approach, we need to create below objects
Create IAM user in AWS.
Create secret in Kubernetes.
Create TriggerAuthentication in Kubernetes.
Create scaledObject in Kubernetes.
Create IAM user and give SQS permissions to this IAM user.
first encode IAM user Access Key and Secret key using base64 which will be required while creating Kubernetes secret.
Create secret
apiVersion: v1
kind: Secret
metadata:
name: test-secrets
namespace: default
data:
AWS_ACCESS_KEY_ID: <base64-encoded-key>
AWS_SECRET_ACCESS_KEY: <base64-encoded-secret-key>
Create TriggerAuthentication this will be used in scaledObject
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-auth-aws-credentials
namespace: default
spec:
secretTargetRef:
- parameter: awsAccessKeyID # Required.
name: test-secrets # Required.
key: AWS_ACCESS_KEY_ID # Required.
- parameter: awsSecretAccessKey # Required.
name: test-secrets # Required.
key: AWS_SECRET_ACCESS_KEY # Required.
Create scaledObject to map keda with deployment you want to scale based on SQS trigger
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: aws-sqs-queue-scaledobject
namespace: default
spec:
scaleTargetRef:
name: test-deployment
minReplicaCount: 0
maxReplicaCount: 2
triggers:
- type: aws-sqs-queue
authenticationRef:
name: keda-trigger-auth-aws-credentials
metadata:
queueURL: https://sqs.us-east-1.amazonaws.com/012345678912/Queue
queueLength: "5"
awsRegion: "us-east-1"