We are trying to setup Amazon SQS between two AWS applications. Management wants to track cost associated with all Amazon resources. Is it possible to tag Amazon Simple Queue Service resources?
This feature is now supported on SQS: https://aws.amazon.com/blogs/aws/introducing-cost-allocation-tags-for-amazon-sqs/
Tagging for SQS is not yet supported. Perhaps you may manually calculate using the standard formula with few assumption of number of requests made like number of SQS Request etc.
In my opinion you can enable cost tagging for the AWS Resources which are support and for the remaining you can try having the accountability like misc. charges which can certainly include the SQS.
First 1 million Amazon SQS Requests per month are free
$0.50 per 1 million Amazon SQS Requests per month thereafter ($0.00000050 per SQS Request)
A single request can have from 1 to 10 messages, up to a maximum total payload of 256KB.
Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests.
Reference : http://aws.amazon.com/sqs/pricing/
Please use following AWS cli command to tag you SQS:
aws sqs tag-queue --queue-url --tags="Key-name=Value","Key-name=Value"
I have created a boto3 script to tag the SQS. Please pass your queue name in tag_sqs_queue()
import boto3
client = boto3.client('sqs')
def get_queue_url(queuename):
get_url = client.get_queue_url(
QueueName=queuename
)
def tag_sqs_queue(queuename):
response = client.tag_queue(
QueueUrl=get_queue_url(queuename),
Tags={
'Environment': 'Production',
'Owner': 'Joe Biden'
}
)
tag_sqs_queue('<queuname>')
script first gets the queue URL and then applies tags.
Related
SQS pricing page mentioned
Every Amazon SQS action counts as a request
SQS pricing strategy has some pricing buckets, like First 1 Million Requests/Month is Free, From 1 Million to 100 Billion Requests/Month for Standard Queue is $0.40, etc.
As per AWS doc, to trigger lambda from SQS, three SQS actions are required as given below.
sqs:ReceiveMessage
sqs:DeleteMessage and
sqs:GetQueueAttributes.
My questions are
For every lambda invocation (triggering) do all these three actions are execute?
If yes, do all these actions are treated as a request?
In that case, every trigger for each 64KB payload chunk consumes 3 requests. If this understanding is wrong then what is the request count calculation for every trigger execution?
Thanks in advance
I have a AWS Lambda function which is triggered by SQS. This function is triggered approximately 100 times daily, but request count to the SQS queue is approximately 20.000 times daily. I don't understand why the number of requests made to the SQS is too high. My expectation is that the number of requests made to the SQS should be same with the Lambda invocation.
I have only one Lambda function and one SQS queue in my account.
Can be related with polling of SQS queue? I tried to change the polling interval of SQS from the queue configuration but nothing changed. Another possibility is to change polling interval from Lambda function configuration. However, I cannot find any related parameter.
Briefy, I want to reduce number of SQS request, how can i do that while invoking Lmabda function with SQS?
When using SQS as an event source for AWS Lambda, AWS Lambda regularly polls the configured SQS queue to fetch new messages. While the official documentation isn't clear really about that, the blog post announcing that feature goes into the details:
When an SQS event source mapping is initially created and enabled, or when messages first appear after a period with no traffic, then the Lambda service will begin polling the SQS queue using five parallel long-polling connections.
According to the AWS documentation, the default duration for a long poll from AWS Lambda to SQS is 20 seconds.
That results in five requests to SQS every 20 seconds for AWS Lambda functions without significant load, which sums up to the ~21600 per day, which is close to the 20000 you're experiencing.
While increasing the long poll duration seems like an easy way to decrease the number of requests, that's not possible, as the 20 seconds AWS Lambda is using by default is already the maximum possible duration for an SQS queue. I'm afraid there is no easy way to decrease the requests to SQS, when using it as event source for AWS Lambda. Instead depending it could be worth evaluating if another event source, like SNS, would fit your use case as well.
Here is how we originally implemented when there is no SQS trigger.
Create a SNS trigger with the SQS Cloudwatch Metric
ApproximateNumberOfMessagesVisible > 0
Trigger a Lambda from SNS, Read Messages from SQS and deliver it to whichever the lambda needs the message.
Alternatively, you can use Kinesis to deliver it to Lambda.
SQS --> Cloudwatch (Trigger Lambda) --> Lambda(Reads Messages) -->
Kinesis (Set Batch Size) --> Lambda (Handle Actual Message)
You can also use Kinesis directly but there is no delayed delivery.
Hope it helps.
I have a PHP web application that is running on an ec2 server. The app is integrated with another service which involves subscribing to a number of webhooks.
The number of requests the server is receiving from these webhooks has become unmanageable, and I'm looking for a more efficient way to deal with data coming from these webhooks.
My initial thought was to use API gateway and put these requests into an SQS queue and read from this queue in batches.
However, I would like these batches to be read by the ec2 instance because the code used to process the webhooks is code reused throughout my application.
Is this possible or am I forced to use a lambda function with SQS? Is there a better way?
The approach you suggested (API Gateway + SQS) will work just fine. There is no need to use AWS Lambda. You'll want to use the AWS SDK for PHP when writing the application code that receives messages from your SQS queue.
I've used this pattern before and it's a great solution.
. . . am I forced to use a lamda function with SQS?
SQS plus Lambda is basically free. At this time, you get 1M (million) lambda calls and 1M (million) SQS requests per month. Remember that those SQS Requests may contain up to 10 messages and that's a potential 10M messages, all inside the free tier. Your EC2 instance is likely always on. Your lambda function is not. Even if you only use Lambda to push the SQS data to a data store like RDBMS for your EC2 to periodically poll, the operation would be bullet-proof and very inexpensive. With the introduction of SQS you could transition the common EC2 code to Lambda function(s). These now have a run time of 15 minutes.
To cite my sources:
SQS pricing for reference: https://aws.amazon.com/sqs/pricing/
Lambda pricing for reference: https://aws.amazon.com/lambda/pricing/
Does AWS expose a API/method if i can pull out number of message sent to SQS queue for a given time frame(for example a week).I am trying to build a tool to monitor cloud infra for an internal project in java.This is one of the requirement to display a graph with number of message pushed to SQS for a week. I also know we can check this in AWS console.
You can retrieve queue metrics from Amazon CloudWatch. For example:
NumberOfMessagesSent
NumberOfMessagesReceived
See: Amazon SQS Metrics and Dimensions - Amazon CloudWatch
I am willing to make a simple chat application in django.
After user sends message and the message in saved in Message model I would like to signal the publish event so that my mobile app gets message.
I went throught the SNS documentation and in its pricing I found Data transferred between Amazon SNS and Amazon EC2 within a single region is free of charge (i.e., $0.00 per GB)
My api is/will be hosted in ec2 in the same region lets say at Singapore the same region as sns. If I publish a event which I receive a message in my mobile application , will that cost me or it is considered as same region ? If it does than building a chat application using SNS will be very expensive.
If anyone can meke me clear about it that would be so helpful.
Thank you
Well I don't think it will be free even if they are in the same region but their prices for SNS are too low, if in free tier of Amazon you won't be charged for
1,000,000 Requests of Amazon Simple Queue Service every month**
1,000,000 Requests, 100,000 HTTP notifications and 1,000 email
notifications for Amazon Simple Notification Service every month**
as the SNS pricing suggests. Going above the limits will cost you :
It costs $1.00 to send one million mobile push notifications ($0.50 per million publishes, plus $0.50 per million mobile push notification deliveries). When you use SNS topics to broadcast identical messages to many recipients at once.