I ran into an issue where my mail server only accepts 100 connections to the server every 5 minutes. My current code loops over my database, calling cfmail for each person on the list. I suppose the problem is im opening a new connection each time I use cfmail?
<CFLOOP QUERY="Customer" >
<!---send mail to Admin ----->
<cfmail to = "#cstEmail#"
from = "#FORM.fromAddressEmail#"
subject = "#FORM.subjectEmail#"
server = "#var.mailserver#"
port= "#var.mailport#"
username="#var.mailuser#"
password="#var.mailpass#"
failto="#var.failEmail#
type="html"
>
What I ran into was only 100 mails were being sent at a time, the rest were sent to cf's undelivered folder. I would send them to spool and again 100 would get through..
Now, I've read in older versions of cf there is a checkbox in cf administrator to "maintain connection" -Im running cf9 and dont see this option.
Would using cfmail's query attribute, force cfmail to only connect to the mail server once to send all the emails?
<cfmail query="Customer"
from = "#FORM.fromAddressEmail#"
to = "#cstEmail#"
subject = "#FORM.subjectEmail#">
Im not even sure how to test this without sending a couple hundred emails. Any thoughts if this is a viable solution to the problem?
Thanks for your help!
Biscotti
I ended up compromising by using a scheduled task to move the files every 5 minutes back over to the Spool dir from the Undelivr dir. Im not thrilled with this solution, but it works.
Thanks to Russ's Respooler extension. http://cfrespooler.riaforge.org/
By using the above code to call the QUERY within CFMAIL I only succeeded in speeding up the client side process. The mail server still rejected the mail after the 100th connection - leading me to determine there is no server side benefit to this method over simply looping CFMAIL like in my first example. I seems the only answer is to run the code within the enterprise edition of the cf environment, one that has the "maintain connection" feature enabled.
Related
I have a series of scheduled tasks that all run at various times of the day. Since the migration from Coldfusion version 7 to 10, these tasks have stopped running.
When I check the box, that outputs the results to a file, I get a text file that says nothing more than "Connection Failure". I have tried everything imaginable regarding the username and password for the task. It makes no difference. When I run the CFM page in my browser, the
page works correctly and generates an email just like it should. I just
can't make it run as a scheduled event.
Is the scheduled task folder has any check for the session or anything? I mean is the scheduled task folder is accessible without login? Please try with removing all the redirect rules for the application. That might work.
For me the requests were timing out. I increased the timeout in the administrator and that solved it. Doing a cfhttp in a test file and dumping the results helped me troubleshoot it.
Ever since upgrading to CF10, we've been having some odd issues with our automated ColdFusion emails. The processes always functioned properly in the past, but lately we've been getting some very out of the ordinary issues which I'll describe further below.
We discover the problem usually from contacts who usually receive these emails on a daily basis with or without attachments. We'll go to the CFMAIL directory for the corresponding server and find a slew of emails stuck in the 'Undelivr' emails. In some cases, we can just move these emails to the Spool folder and they process fine, but in most cases they result in one of the two errors below:
Error 1: In an email which normally does not contain a body and contains an attachment, the follow error is what we found in the logs:
"Error","scheduler-1","01/15/13","14:09:56",,"javax.mail.MessagingExce ption: missing body for message"
javax.mail.MessagingException: missing body for message
at coldfusion.mail.MailImpl.createMessage(MailImpl.java:696)
at coldfusion.mail.MailSpooler.deliver(MailSpooler.java:1295)
at coldfusion.mail.MailSpooler.sendMail(MailSpooler.java:1197)
at coldfusion.mail.MailSpooler.deliverFast(MailSpooler.java:1657)
at coldfusion.mail.MailSpooler.run(MailSpooler.java:1567)
at coldfusion.scheduling.ThreadPool.run(ThreadPool.java:211)
at coldfusion.scheduling.WorkerThread.run(WorkerThread.java:71)
Placing these emails that have always been sent out this way in the past without an attachment in the spool directory causes it to go right back in the 'Undelivr' folder and resulting in the same error. We ended up having to modify the email file and add random content in the body message, place it back in the spool directory, and it went through. - Mind boggling.
Error 2:
"Error","scheduler-2","02/04/13","09:08:17",,"javax.mail.MessagingExce ption: Exception reading response; nested exception is: java.net.SocketException: Connection reset"
Both errors occur randomly and we have not been able to find out what causes them randomly from time to time. All other emails go through fine, but certain emails will never go out and end up in the 'Undelivr' folder.
We are running them on Windows Server 2008 64bit.
I was facing second error connection reset couple of week ago but that was in CF9 and with SSL only. Here is blog post if that help
http://www.isummation.com/blog/getting-javaxmailmessagingexception-could-not-connect-to-smtp-host-xxxxxxx-port-465-response-1-error-in-coldfusion/
I have a Django project that needs to send out around 600 emails. I have Celery set up and it works, for the most part. I have the Django project set up to use my Google Apps (Business version -- ie: paid for) email account as the sending account. For testing purposes, I have every email sent to me -- not to the client.
The issue I am having is that Celery seems to randomly skip people in the list. When I start the process of sending all 600 emails, Celery works away, sending emails (I can see them show up in my inbox) but I only receive a total of about 420 emails. When Celery finishes, there are still 180 or so people that need the email. If I click "send emails" again with ONLY the remaining 180 people, it will finish the job and, at the end of two attempts, will have sent emails to all 600 people.
Why would Celery be skipping people?
Yes, you will get those gmail errors and it's not particularly predictable.
You could just use django-mailer instead -- I do, and deal with those gmail connection errors by letting django-mailer automatically retry the failed sending attempts until they succeed.
Check out this SO question for more folks suggesting you just use django-mailer vs celery for mail.
Advice on Python/Django and message queues
Here is my setup right now:
connection = mail.get_connection()
maillist = []
# my real setup is a little more complex for-loop, but basicly I add all recipients to a list.
for person in object_list:
mail_subject = "Mail subject here"
mail_body = "Mail body text...bla bla"
email_sender = "me#example.com"
maillist.append((mail_subject, mail_body, email_sender, [person.email]))
#send_mass_mail wants a tuple, so we convert the list
mailtuple = tuple(maillist)
mail.send_mass_mail(mailtuple, fail_silently=False, connection=connection)
However, the forloop iterates over 1000+ objects/persons and when I try this method I'm able to send 101 emails, and then it stops. No errors (as I can see) anywhere.
A fellow developer mentioned that maybe the POST size was too big? Any ideas from the SO-community?
Your SMTP server probably has some send limits. For example, I believe Gmail limits outgoing mail to 100 recipients.
As Micah suggested, there is a good chance you are hitting server limits.
Generally, when dealing with mass mail, it is always a good idea to throttle the sending. Doing 50 mails every 5 seconds for 300 seconds beats 3000 mails at once for many practical reasons including smtp server limitations.
Since you mentioned a POST limit - do you send out the emails in a view? I'm wondering how you handle canceled requests in your setup.
I'm using a management command to send out 1000+ newsletters. But instead of send_mass_mail i use the normal send method in a loop. It takes about 5 minutes (haven't a correct count atm) to send out the mails and i haven't run into any server limits yet.
My plan is to switch to celery to handle sending through a web interface. Perhaps you want to have a look at it in case you haven't already.
http://celeryproject.org/
Our CF server occasionally stops processing mail. This is problematic, as many of our clients depend on it.
We found suggestions online that mention zero-byte files in the undeliverable folder, so I created a task that removes them every three minutes. However, the stoppage has occurred again.
I am looking for suggestions for diagnosing and fixing this issue.
CF 8 standard
Win2k3
Added:
There are no errors in the mail log at the time the queue fails
We have not tried to run this without using the queue, due to the large amount of mail we send
Added 2:
It does not seem to be a problem with any of the files in the spool folder. When we restart the mail queue, they all seem to process correctly.
Added 3:
We are not using attachments.
What we ended up doing:
I wrote two scheduled tasks. The first checked to see if there were any messages in the queue folder older than n minues (currently set to 30). The second reset the queue every night during low usage.
Unfortunately, we never really discovered why the queue would come off the rails, but it only seems to happen when we use Exchange -- other mail servers we've tried do not have this issue.
Edit: I was asked to post my code, so here's the one to restart when old mail is found:
<cfdirectory action="list" directory="c:\coldfusion8\mail\spool\" name="spool" sort="datelastmodified">
<cfset restart = 0>
<cfif datediff('n', spool.datelastmodified, now()) gt 30>
<cfset restart = 1>
</cfif>
<cfif restart>
<cfset sFactory = CreateObject("java","coldfusion.server.ServiceFactory")>
<cfset MailSpoolService = sFactory.mailSpoolService>
<cfset MailSpoolService.stop()>
<cfset MailSpoolService.start()>
</cfif>
We have not tried to run this without using the queue, due to the large amount of mail we send
Regardless, have you tried turning off spooling? I've seen mail get sent at a rate of 500-600 messages in a half second, and that's on kind of a crappy server. With the standard page timeout at 60 seconds, that would be ~72,000 emails you could send before the page would time out. Are you sending more than 72,000 at a time?
An alternative I used before CFMail was this fast was to build a custom spooler. Instead of sending the emails on the fly, save them to a database table. Then setup a scheduled job to send a few hundred of the messages and reschedule itself for a few minutes later, until the table is empty.
We scheduled the job to run once a day; and it can re-schedule itself to run again in a couple of minutes if the table isn't empty. Never had a problem with it.
Have you tried just bypassing the queue altogether? (In CF Admin, under Mail Spool settings, uncheck "Spool mail messages for delivery.")
I have the same problem sometimes and it isn't due to a zero byte file though that problem did crop up in the past. It seems like one or two files (the oldest ones in the folder) will keep the queue from processing. What I do is move all of the messages to a holding folder, restart the mail queue and copy the messages back in a chunk at a time in reverse chronological order, wait for them to go out and move some more over. The messages which were holding up the queue are put in a separate folder to be examined latter.
You can probably programmatically do this by stopping the queue, moving the oldest file to another folder, then start the mail queue and see if sending begins successfully by checking folder file counts and dates. If removing the oldest file doesn't work, repeat the previous process until all of the offending mail files are moved and sending continues successfully.
I hope the helps.
We have actually an identical setup, 32bit CF8 on Win2K3.
We employed Ben's solution about a year ago, and that certain has helped auto re-queue emails that get stuck.
However recently for no particular reason one of our 7 web servers decided to get into this state with every email attempt.
An exception occurred when setting up mail server parameters.
This exception was caused by:
coldfusion.mail.MailSessionException:
An exception occurred when setting up mail server
parameters..
Each of our web servers are identical clones of each other, so why it was only happening to that one is bizarre.
Another item to note is that we had a script which reboot the machine in the middle of the night due to JRUN's memory management issues. The act of rebooting seemed to initiate the problem. A subsequent restarting of the CF service would then clear it, and the machine would be fine until it rebooted again.
We found that the problem is related to the McAfee virus scanner, after updating it to exclude the c:\ColdFusion8 directory, the problem went away.
Hope that helps.
There is a bug in Ben Doom's code. Thank you anyway ben, the code is great, and we use it now on one of our servers with CF8 installed, but:
if directory (\spool) is empty, the code fails (error: Date value passed to date function DateDiff is unspecified or invalid.) That's because if the query object spool is empty (spool.recordcount EQ 0), the datediff function produces an error.
we used this now:
<!--- check if request for this page is local to prevent "webusers" to request this page over and over, only localhost (server) can get it e.g. by cf scheduled tasks--->
<cfsetting requesttimeout="30000">
<cfset who = CGI.SERVER_NAME>
<cfif find("localhost",who) LT 1>
security restriction, access denied.
<cfabort>
</cfif>
<!--- get spool directory info --->
<cfdirectory action="list" directory="C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\Mail\Spool\" name="spool" sort="datelastmodified">
<cfset restart = 0>
<cfif spool.recordcount GT 0><!--- content there? --->
<cfif datediff('n', spool.datelastmodified, now()) gt 120>
<cfset restart = 1>
</cfif>
</cfif>
<cfif restart><!--- restart --->
<cfsavecontent variable="liste">
<cfdump var="#list#">
</cfsavecontent>
<!--- info --->
<cfmail to="x#y.com" subject="cfmailqueue restarted by daemon" server="xxx" port="25" from="xxxx" username="xxxx" password="xxx" replyto="xxxx">
1/2 action: ...try to restart. Send another mail if succeeded!
#now()#
Mails:
#liste#
</cfmail>
<cfset sFactory = CreateObject("java","coldfusion.server.ServiceFactory")>
<cfset MailSpoolService = sFactory.mailSpoolService>
<cfset MailSpoolService.stop()>
<cfset MailSpoolService.start()>
<!--- info --->
<cfmail to="x#y.com" subject="cfmailqueue restarted by daemon" server="xxx" port="25" from="xxxx" username="xxxx" password="xxx" replyto="xxxx">
2/2 action: ...succeeded!
#now()#
</cfmail>
</cfif>
There is/was an issue with the mail spooler and messages with attachments in CFMX 8 that was fixed with one of the Hotfixes. Version 8.0.1, at least, should have had that fixed.