I would like to implement a queuing mechanism for sending out email via PHPMailer on Amazon EC2. I have set up Beanstalkd correctly on the server and can access it via a console. The mail doesn't seem to go through (trying the various combinations of sample code). In addition do I need to set up a cron job also that would call one of the producer or consumer files?
Does anyone have working code for sending out email via phpmailer/pheanstalk please for Amazon EC2?
Thanks.
Beanstalkd is great, and I use it myself, however, don't use it for this; It's reinventing the wheel in a bad way. Instead, install a local mail server such as postfix and get that to do your queuing for you. This is also much, much simpler, faster, and easier to control. Email servers are built for managing queues, and they are extremely good at it.
Before you do so, get your mail sending script working – there's no point in even attempting to get something more complex working until you've done that. Also be aware that sending email from EC2 is difficult – Amazon wants you to use their SES service rather than sending directly – you may find sending is blocked altogether. Read the PHPMailer troubleshooting guide to see how to diagnose that.
Related
I want to enable event notifications for my customers. There are many possible ways to send notifications: emails, sms, XMPP/other IM, pre-recorded voice messages over SIP, phone-specific message push services, REST callbacks etc.
I don't want to develop all these transports myself, so I need a web service that can manage those notifications for customers. Also I don't want to store emails/phones/other personally identifiable information.
The notifications are transactional (i.e. it's not mass delivering same message to everyone). Paid solutions are welcome.
There is http://pagerduty.com but it is
designed to work within enterprise and not with outside customers
focused on full cycle of incident response as opposed to simple message delivery
So it puts more burden on respondents and I want something that requires zero effort for the users to setup.
Monitis is another example. It has multiple transports including Twitter, but again it's designed for insiders and not for service subscribers coming in bulk numbers.
Amazon SNS seems to be too low-level as it only manages delivery of push notifications, but for diplaying them I have to write a mobile app which I don't want.
XMPP servers as described in How best to deliver notifications to various IM / notification services? have traditionally supported the idea of different transports, but I'd like a third-party hosted service.
Twilio has only 2 transports: SMS and voice call and more oriented on full 2-side communications.
I cannot even find the right google keywords to search for the service/SaaS I want.
The question is, are there any such services? A sample of a few would give me an idea of what to look for.
This comes very late, perhaps too late but...
You should not need to implement any of the transport but you may be required to build some of the gateways and you will most likely need to assembly the application which talks to each of the gateways. You are not likely going to find a single service for this.
You've already outlined the strategy. You basically have these pieces:
transports
gateways
application
Each of the transports is accessed through some client via either an API or a CLI - so you'll need to figure out what your environment is. Java is probably a good choice but other cross-platform environments would likely work. Existing infrastructure like Apache ServiceMix has support for some of these transports:
https://cwiki.apache.org/confluence/display/SM/Components+list
and there may be other middle-ware with similar, distinct transports.
You will likely want a gateway for each provider for each transport type. You may be able to find a provider which adequately services multiple transports, e.g. Twilio's SMS and voice, but that will likely be the exception. You may also find that because of the differences in transports (and therefore functionality), it's more convenient to build a gateway for each transport type. So, you might have two configured providers in your SMS gateway, one for Twilio and one for Kannel, and you might have your Twilio account used in the SMS gateway and in the SIP gateway.
The final step is assembling your application into something meaningful. This might be something like:
sent.......: "Thanks for your purchase, ${username}!"
sent to the channels (i.e., provider-transport pair) configured, perhaps, by the user and being able to collect the response from the user:
response...: "It was a pleasure! --Bob"
You will need to store the basics of the each transport's endpoint, e.g., phone number for SMS, username for chat, etc., so if you have PII security issues to address you'll need to think though that. One option may be to turn all the PII over to each provider but you'll still need to keep each account for your users in each provider, and you will likely need to know something about the user, like "${username}" above, to personalize your notification appropriately within your application. So, removing all PII from your application seems unlikely.
I'm not sure how much this help but perhaps it gives you some ideas.
I have a Java/Spring application running in the Amazon AWS cloud.
My server instances are using load balancing and runs the same image of a Linus OS, with a Tomcat application server.
They are also connected to S3 as a shared file system (s3fs), and an RDS database.
My concern is to be sure the state of the different applications is synchronized. Today, the point of synchronization is the database, but when memory caching is needed, out of sync problems appear.
The solution I would like to use is to put in place a messaging system between the applications. For specific reasons, I cannot use Amazon SQS service, then JMS seems to fit my needs. After some reading, HornetQ seems also a very good implementation of it. Once an application state change, it communicates the change to all other applications. Each application is producer and consumer of the same queue.
As we are in a dynamic system where servers and IPs are automatically created and deleted, the automatic discovery of instances seems to be the best solution to use.
But in AWS, broadcast is not possible!
For HornetQ, I saw a kind of work around which is using JGroups additionally. But for me, this is a second framework to investigate and learn. Twice the work. And no more an out-of-the-box solution.
What is your opinion? Does anyone already build a solution for similar needs?
Maybe other out-of-the-box solutions exists?
Thanks in advance for your answer!
In my experience you could try to use TCPGOSSIP, that is a HornetQ configuration.
See https://docs.jboss.org/jbossclustering/cluster_guide/5.1/html/jgroups.chapt.html
I am running a web service that currently sends confirmation emails out to new users via the gmail smtp servers. As I'm only getting a few new users each day, this hasn't been a problem.
I've recently added new features to the webapp that will require a customized message to be sent out to each user every day. Think of this as similar to the regular messages LinkedIn sends out that give you a status report on the activity in your network. Every user's message will be different. With thousands of users, this means thousands of unique messages will be sent each day.
Edit: I've since found that these types of email are called "transactional or relationship messages". Spamtacular has a good article on differentiating between marketing and transactional email.
I don't think using gmail's smtp servers will cut it anymore, but I don't know that for sure. I don't know what gmail's maximum outgoing messages per account is (it might be 100/day), but they limit outgoing mail to 500 recipients per message. I'm not sending a single message to 500 recipients, but I'm going to be sending 1000's of customized messages with each recipient getting one per day.
I'm interested to learn any best practices for doing this (especially for Java-based webapps). Here are some of my thoughts and concerns on it:
Should I set up my own outgoing mail server? If I do this, it seems like I'll have all sorts of other issues to worry about, such as preventing mail server abuse, monitoring bounces, allowing ways to opt-out of emails, etc. Are there any tools or services to help with this? Maybe something like OpenEMM or a services like MailChimp? But those seem focused more toward email marketing campaigns.
I don't think I should have the webapp itself handle sending emails as it currently is for new user signups. I'm thinking I should setup a separate messaging server that can access the same backend/datastore as the webapp. Thoughts on this?
Should I consider setting up some sort of message queueing service to help with this, such as JMS, RabbitMQ, ActiveMQ, etc.?
Do I need to provide users a way to opt-out? Do I need to flag these as bulk messages? I don't really consider these email marketing messages, but I'm unsure what is considered appropriate or proper netiquette.
Any advice is appreciated. I'm also very interested in open source tools or web services that simplify things and could help me to ramp up as quickly as possible.
Thanks!
With regard to your first question, yes, you should set up your own mail server. Using gmail to do this might work for a while, but they are likely to shut you down in short order when they see this kind of activity. You could sign up for a business account and use app engine to send messages. Here's a link with information about mail quotas for that service.
Regarding your second and third questions, It would be a good idea to have messages queued by the web app and sent out by a centralized service rather than having the app send out the messages on its own.
Usually I would just use a database table as a queue - the web app inserts rows for each message it wants to send. A service/scheduled task app would grab new messages out of the table and send them off. This gives you lots of flexibility if you want to switch mail servers later, better reliability if the mail server is down, easier diagnostics if there are problems with recipients not getting messages, and the ability to resend messages. As for using JMS/MQ to do this - probably not necessary. IMO a database table used as a queue would give you more flexibility here than an actualy JMS-based queue system.
As for opt outs, YES - you should give people a way to opt out. I don't think you need to flag the messages as bulk though.
On the architecture side of things I would definitely consider decoupling the sending of the emails from the main service via some form of asynchronous message queuing (or facsimile thereof using database as an intermediary). Another benefit of this approach is that if the SMPT server\network is down you could build in retry semantics, additionally for future scalability you could implement multiple mail senders reading from the same queue or implement sending throttling or scheduling (i.e send n messages per hour), etc etc.
I'm working on the design of a remote control application. From my iPhone or a web browser, I'll send a few commands. Soon my home computer will perform the commands and send back results. I know there are remote desktop apps, but I want something programmable, something simpler, and something that I wrote.
My current direction is to use Amazon Simple Queue Service (SQS) as the message bus. The iPhone places some messages in a queue. My local Java/JRuby program notices the messages on the queue, performs the work and sends back status via a different queue.
This will be a very low-volume application. At $1.00 for a million requests (plus a handful of data transfer charges), Amazon SQS looks a lot more affordable than having my own server of any type. And super reliable, that's important for me too.
Are there better/standard toolkits or architectures for this kind of remote control? Cost is not a big issue, but I prefer the tons I learn by doing it myself.
I'm moderately concerned about security, but doubt it will be a problem. The list of commands recognized will be very short, and only recognized in specific contexts. No "erase hard drive" stuff.
update: I'll probably distribute these programs to some other people who want the same function, but who don't have Amazon SQS accounts. For now, they'll use anonymous access to my queues, with random 80-character queue names.
Well, I think it's a clever approach -- and as you said, the costs for your little traffic aren't even worth mentioning.
As I mentioned in the comment, it's a good way to leave your home machine behind your firewall and not have an open port on the internet.
I would suggest using OnlineMQ.com as a start; they have a free package.
This is a newbie question... I want to create a service which responds to emails sent to it.
Just to illustrate. If a user sends an email to handlethis#example.com with some instructions, I want a program at example.com (which is a domain I own) to read the email and act accordingly.
I strongly doubt I can do this with standard web hosts (which are shared), so I welcome suggestions on where I can host something like this (at the cheapest rate).
What will the "program" be written in? Can I use php, etc or is it some specific "email-handling" language!?
Thanks!
S
FWIW, I think this question should be posed on serverfault.com rather than here.
I confirm this is something most shared hostings can't do: You need to be able to create a scheduled task/cron job which checks your e-mail account every x minutes.
You might consider a virtual server hosting. A bit more pricey but much more flexible.
The script would be written in php on Linux or in VBS/JScript/.NET on Windows. I've written such a script in JScript on Windows, using a component that implements POP3.