Post a Message to Elastic Beanstalk Worker Environment via SQS - amazon-web-services

Question

I have a docker application on elastic beanstalk with a web-server and worker environment.
The worker environment currently runs scheduled jobs via cron.
I'm trying to connect the server to the worker to achieve the following:
Client sends a request to the server (/trigger_job)
Server offloads the job to the worker by sending a JSON message to SQS queue (/perform_job)
Worker performs the job by reading the message from SQS
I haven't been able to find documentation on what the JSON message should look like. There are some HTTP headers mentioned in the official documentation. But there's no mention of header to specify the desired endpoint in the worker environment.
# server.py
from bottle import post, HTTPResponse
#post('/trigger_job')
def trigger_worker_job():
# should send a JSON message to sqs to trigger the '/perform_job'
# Need help with what the JSON message looks like
return HTTPResponse(status=200, body={'Msg': 'Sent message'})
# worker.py
from bottle import post, HTTPResponse
#post('/perform_job')
def perform_job():
# job is performed in the worker environment
return HTTPResponse(status=200, body={'Msg': 'Success'})

Answer 1

In Python, you can see how this from the python sample application where you can find on this aws doc step 4: Deploy a New Application Version.
You can configure SQS endpoint in beanstalk worker environment console. Configuration > Worker > Select a Worker Queue
# for example:
environ['HTTP_X_AWS_SQSD_TASKNAME']
environ['HTTP_X_AWS_SQSD_SCHEDULED_AT']
logger.info("environ X-Aws-Sqsd-Queue %s" % environ['HTTP_X_AWS_SQSD_QUEUE'])
# regarding your message attribute. For example, the attribute name is Email,
# you can extract it via environ['HTTP_X_AWS_SQSD_ATTR_EMAIL']).
# Make sure that the attribute name is all capital.
logger.info("environ X-Aws-Sqsd-Attr Email %s" % environ['HTTP_X_AWS_SQSD_ATTR_EMAIL'])
The message will contains the following info in the image. You can read more on aws AWS Elastic Beanstalk Worker Environments

Answer 2

I found from some of my failed cron jobs in the dead letter queue, there were the following three attributes:
beanstalk.sqsd.path
beanstalk.sqsd.task_name
beanstalk.sqsd.scheduled_time
which are similar to the attributes set on the cron job.
You can manually create a new message in the SQS queue for the worker environment and set those attributes to match the cron job you wish to execute, though that is tedious and I much prefer to do so by code.
I'm using the boto3 framework in python, though hopefully it is a similar process in other languages.
def send_elastic_beanstalk_message(name, path, message):
client = boto3.client('sqs')
return client.send_message(
QueueUrl=url,
MessageAttributes={
"beanstalk.sqsd.path": {
"DataType": "String",
"StringValue": path
},
"beanstalk.sqsd.task_name": {
"DataType": "String",
"StringValue": name
},
"beanstalk.sqsd.scheduled_time": {
"DataType": "String",
"StringValue": datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')
}
}
)
This will create a new message in the SQS queue that will be parsed by the worker daemon and trigger a cron job.
Unfortunately, a message body does not seem to be included when the message is parsed, so triggering the job is the only action I was able to solve.