I facing some issue related to Amazon SES Statistic ordering, actually my recent sending mail static are in middle of data record it should be at top or lower of the record so what is the reason behind it?
My Question is How i do a simple report status with how much emails returned, invalid emails and emails sent if the "GetSendStatistics" returns a unordered list?
You will need to sort the data yourself, by using the Timestamp field of SendDataPoint. Unfortunately, the list returning from GetSendStatistics is not sorted.
This happens because due to the characteristics and nature of sending e-mail over the Internet, it might take a different amount of time for a server to determine whether a message (or a set of messages) is confirmed to have bounced, or to be delivered for sure. Please refer to this thread at AWS forums for a similar question and some insight into the issue you are facing.
Related
After I sent a message to my GCP subscription, it takes a minute or two (should be instant) to appear in my Nifi flow. At this point, I see a bunch of XML and my payload isn't there. Does anyone know what's possibly happening?
If your push messages are not acknowledged then it may slow down delivery of the rest significantly.
Your use case looks more like the endpoints don't acknowledge it's delivery instantly (or acknowledgement is late due to some other reasons). If the message is not acknowledged immediately then a system will retry to deliveer it (with some delay) and it will keep trying untill it's acknowledged.
Also look at the Message Flow Control documentation which albo may point you to a solution.
Similar topic was also discussed here in StackOverflow (which might help you).
I am noticing AWS SES stats are not being updated in real-time. After sending email, it takes time for sent count to increase on SES Dashboard. Sometimes it takes few minutes and sometimes it takes long.
Has anyone also experienced this? Any thoughts?
On the assumption that the console is simply making a call to a standard API action (rather than using some kind a console-only backend service that is not documented or user-accessible -- such things are not unheard-of, but are pretty rare in AWS, so it's a reasonably safe assumption), it looks like this is not really designed to be real-time. The stats are reported in 15 minute windows.
From the SES API reference:
GetSendStatistics
Returns the user's sending statistics. The result is a list of data points, representing the last two weeks of sending activity.
Each data point in the list contains statistics for a 15-minute interval.
— http://docs.aws.amazon.com/ses/latest/APIReference/API_GetSendStatistics.html
AWS/SES dashboard stats are for pure hint performace but not to rely on them. In such case, if you want to have real time notifications of sent emails you will need to create SNS notifications. Keep in mind that Spam-Complaint notifications can take up to a couple of days as this is based on information provided by the ISP to Amazon. And complaints within the Gmail evil-system will NEVER get to you.
Please help selecting a MQ app/system/approach for the following use-case:
Check for incoming messages for a specific user -> read the message if available -> delete from the queue, ideally, staying within AWS.
Context:
Social networking app, users receiving messages, i.e.
I need to identify incoming messages by recipient ID.
The app is doing long-polls for new messages every 30 seconds.
Message size is <1Kb.
As per current estimates, I'll need 100M+ message checks per months in total (however, much less messages, these are just checks).
While users acknowledge messages choosing OK or Ignore, however not sure if ACK support is required from MQ system for that.
I'm in AWS. Initially thought of SQS, but the more I read the less it looks like a good match - cannot set message recipient ID in a way to filter by recipient, etc, however maybe I'm wrong.
One of the options I also thought about is to just use DynamoDB's "messages" table, partition key being userId and sort key being a messageId, thus I'll be able to easily query by a user, however concerned with costs.
If possible, I would much more prefer to stay within AWS or at least use SAAS like SQS, as being a 1-person startup I really want to avoid headaches supporting self-hosted system.
Thank you!
D
You are right on both these counts:
SQS won't work, because of the limitation you pointed.
DynamoDB would work, but cost a lot.
I can suggest the following:
Create a Redis cluster, possibly on Amazon ElastiCache.
In it, make one List per user.
Whenever a new message comes, append it to concerned User's list.
To deliver the message, just read from the User's list. Also, flush the queue if needed.
What I am suggesting is very similar to how Twitter manages each User's news-feed and home-feed.
It should also be cheap.
So I need a second pair of eyes to correct or confirm my understand standing of Amazon SQS. From my understanding, you can add an unlimited amount of messages to one queue. A message can be 256 KB in size, and if it needs to be larger than that, you can use amazon s3 to store 2 GB. Reading around online, it appears there are many use cases for this queuing service. For example one use case of SQS can act as a database buffer.
But here's what I'm looking to do.. I'm looking to make a real time messaging system. My current functionality acts like more of a message board, so the implementation just inserts into the database then reads the data and packages it into JSON to be inserted on SQLITE mobile phone. That works great, but I'm getting a lot of requests from people to make it real-time.
So what I'm wondering is can I utilize amazon SQS to write and read messages for a chat application? So in my theoretical use case of SQS would have a message queue to write to, and pull from the that queue every second to check for messages on mobile. But here's where I'm confused. Since you cannot "Query" a particular message from the queue, would it make sense to have a queue per user then a generic queue for the app server to read from? Or am I just talking crazy and should spend cognitive resources thinking about implementing an open connection on an Ec2 instance?
Any help would be great,
Thanks!
Have you thought about using Amazon SNS to push the chat messages to your mobile devices? Each user publishes to a topic and the readers subscribe to that topic. You just have to be ok with missing messages if the app isn't running.
If you only have a few (or maybe, less than 100) users, you could have thought of having one SQS queue per user. If that is not so, the solution won't be operationally feasible.
If you were to have one generic queue, SQS won't help because it doesn't allow querying for a given field in all available messages.
I can think of following options for your use case:
Setup one Redis cluster, possibly on Amazon ElastiCache. Have one message List per user.
One Messages table in MySQL, possibly on AWS RDS. This will provide an easy way to query messages for a given user.
You can also use DynamoDB in #2.
Setting up Meteor to use "out of the box" AWS SES is simple, and one can use native Meteor "Email" methods without modification.
Steps to implement this can be found here. Thanks to Brian
Shamblen for putting together a detailed answer.
But one caveat with the "out of the box" SES is you need to both verify the sender and receiver email address.
To remedy this, you can put in a request with AWS SES for what they call, Production Access.
And further, according to Brian Shamblen,
The process to get production access is rather complicated. One will
need to handle bounce and complaint notifications from SES and prevent
messages from being sent to those addresses in the future.
Question
What is the Meteor code involved in handling bounce and complaint notifications from SES and prevent messages from being sent to those addresses in the future?
EDIT: Made modifications to question for clarity.
Requesting production access is fairly straightforward. You just need to contact them and they usually give it to you in a couple of hours.
Information about the process is here: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/request-production-access.html
Load up the URL : http://aws.amazon.com/ses/fullaccessrequest/ and let them know what you will be sending via Emails, for example if you will be sending transaction based email (verification of a transaction, etc)
With production access you can either send email from:
A specific verified email address, where you will be asked to click a link to an email sent to that address to verify you own it
Any email under an entire domain. Under this process you prove you own the domain by editing its DNS records to contain a 'key'.
Most use cases are covered under production access, they typically give you 2000 emails a day and rate limit emails to 5/sec (they queue them so the maximum send rate is 5/sec). If you need more than this you can contact them to raise this additionally.
The process of verification is to stop people quickly creating AWS accounts to mass-spam users. If they allowed this straight-off then AWS IPs would be looked at as spam by other email providers.
For bounce notifications, SES tracks these, and you have to make sure that you don't get an above average bounce rate. Typically these would come from sending unsolicited email, which I wouldn't advise sending via SES.
Production access is only approved by the AWS team. Wait a bit and they should easily give you 2.000 emails/day for free.
As per bounces-unsubscribes... You'll need to have the SES API notify you of each email address which has been 'marked' with such status.
You should store all those email addresses somewhere and tell your app not to send them ANYTHING else in the future.