I have a very basic app that plugs data into a stored procedure which in turn returns a recordset. I've been experiencing what I thought were 'timeouts'. However, I'm now no longer convinced that this is what is really happening. The reason why is that the DBA and I watched sql server spotlight to see when the stored procedure was finished processing. As soon as the procedure finished processing and returned a recordset, the ColdFusion page returned a 'timeout' error. I'm finding this to be consistent whenever the procedure takes longer than a minute. To prove this, I created a stored procedure with nothing more than this:
BEGIN
WAITFOR DELAY '00:00:45';
SELECT TOP 1000 *
FROM AnyTableName
END
If I run it for 59 seconds I get a result back in ColdFusion. If I change it to one minute:
WAITFOR DELAY '00:01';
I get a cfstoredproc timeout error. I've tried running this in different instances of ColdFusion on the same server, different databases/datasources. Now, what is strange, is that I have other procedures that run longer than a minute and return a result. I've even tried this locally on my desktop with ColdFusion 10 and get the same result. At this point, I'm out of places to look so I'm reaching out for other things to try. I've also increased the timeout in the datasource connections and that didn't help. I even tried ColdFusion 10 with the timeout attribute but no luck there either. What is consistent is that the timeout error is displayed when the query completes.
Also, I tried adding the WAITFOR in cfquery and the same result happened. It worked when set for 59 seconds, but timed out when changed to a minute. I can change the sql to select top 1 and there is no difference in the result.
Per the comments, it looks like your request timeout is set to sixty seconds.
Use cfsetting to extend your timeout to whatever you need.
<cfsetting requesttimeout = "{numberOfSeconds}">
The default timeout for all pages is 60s, you need to change this in the cfadmin if it is not enough, but most pages should not run this long.
Take some time to familiarise yourself with the cfadmin and all its settings to avoid such head scratching.
As stated use cfsetting tag to override for specific pages.
Related
I have an iOS app that inserts a record into an MySQL server via a Python-Django server, and then immediately queries that information.
I am using [NSURLConnection sendAsynchronousRequest: queue: completionHandler:] with a processing block to parse data from a server JSON object.
The insert statement and query statement are run from separate sendAsynchronousRequests, but the latter is called from the completionHandler block in the former, preventing any race conditions.
On the simulator, this works perfectly and returns the inserted data 100% of the time.
On a physical iPad, running iOS7, the response does not ever contain the immediately inserted data.
If I insert a long pause between the two sendAsynchronousRequests using [performSelector: withObject: afterDelay:] it does eventually work -- but I need to add a delay of 2 - 3 seconds.
On the simulator, the timing between the insert and the correct query response is less than 500 ms, so the very long delay should not be necessary and does not seem to be caused by a lower executing process on the simulator.
I have tried Cache-Control: max-age=0, and Expires= headers in my server code, and neither makes a difference, so I don't currently believe that it is a NSURLConnection caching issue.
What else am I overlooking here? Why does this work so perfectly on the Simulator but not on a physical iPad?
Thanks!
I finally figured this out.
My server code was filtering results based on a calculated time period, which ended "now". The time clock on my iPad was a few seconds into the future compared to the server, while my Mac's clock was in sync with the server. As a result, the records posted from the iPad were getting filtered out.
I have a loop the runs over a couple thousand records and for each record it hits is does some image resizing and manipulation on the server. The process runs well in testing over a small record set but when it moves to the live server I would like to suspend and resume the process after 50 records so the server is not taxed to the point of slow performance or quits altogether.
The code looks like this:
<cfloop query="imageRecords">
<!--create and save images to server - sometimes 3 - 7 images for each record -->
</cfloop>
Like I said, I would like to pause after 50 records, then resume where it left off. I looked at cfschedule but was unsure of how to work that into this.
I also looked at the sleep() function but the documentation talks about using this within cfthread tags. Others have posted about using it to simulate long processes.
So, I'm not sure sleep() can be safely used in the fashion I need it to.
Server is CF9 and db is MySQL.
I would create a column called worked in the database that is defaulted to 0 and once the image has been updated set the flag to 1. Then your query can be something like
SELECT TOP 50 imagename
FROM images
WHERE worked = 0
Then set up a CF scheduled task to run every x minutes
Here's a different approach which you could combine with probably any of the other approaches:
<cfscript>
th=CreateObject("java","java.lang.Thread");
th.setPriority(th.MIN_PRIORITY);
// Your work here
th.setPriority(th.NORM_PRIORITY);
</cfscript>
That ought to set the thread to have lower priority than the other threads which are serving your other requests. In theory, you'll get your work done in the shortest time, but with less affect on your other users. I've not had opportunity to test this yet, so your mileage may vary.
I bit of a puzzle to solve in CF here.
You have a timestamp passed as an argument to the server. For example 2012-5-10 14:55.
How would you see if 30 seconds has passed from the time given and return true or false?
Rules:
You cannot use the current server time to check if 30 seconds have passed because the timestamp comes from anywhere in the world and will always be wrong.
In this case you can always trust the argument timestamp passed to the server.
How would this be possible in ColdFusion 9 (if at all)?
Hmmm.... Your problem is that you don't know the latency. You can count 30 seconds from the time you receive the timestamp - but that could be seconds after the time stamp was created.
You can count 30 seconds easily enough with...
sleep(30000); //1000 miliseconds = 1 second. 30k = 30 seconds
So as soon as you get the var you could "wait" for thirty seconds and then do something. But from your question it seems like you need exact 30 seconds from the time the timestamp (created on the client) was created. You probably cannot be that exact because:
The 2 clocks are not in synch
You cannot figure out the latency of the request.
Even if you could, since you don't control the client you will have trouble guaranteeing the results because HTTP is stateless and not real time.
If you can dictate an HTML5 browser you could use websockets for this - it's practically begging for it :) But that's the only real solution I can think of.
You don't specify if the argument passed to the server is a API request, form submit or an ajax request, etc, so there will be differences in the implementation on the other end. However, the ColdFusion server end should be extremely similar and is basically bound with 3 issues; timezones, time disparity, and latency.
To solve the timezone issue you can require the argument to be passed as UTC time. When comparing the times with Coldfusion you would do something like:
<cfif DateDiff("s", Variables.PassedTimestamp, DateConvert( "Local2UTC", Now() )) EQ 30>
<!--- Exactly 30 seconds difference between timestamps --->
</cfif>
But you don't know if the time sent to you is synchronized with the time on your server. To help with this you can provide a method for the other end to query your time and either adjust their time accordingly or to echo back your time with their own time.
That time synching and the originally discussed send of the timestamp will both suffer from latency. Any communication over a network will suffer from latency, just varying levels of latency. Location and Internet connection type can have more of an impact than protocol type. You could do something like pinging the other end to get the response time and then halving that and adding it to the timestamp, but that's still an aproximation and not all servers will respond to a ping.
Additionally, the code on your server and the other end will also introduce latency. To reduce this you want the other end to calculate the timestamp as close to the end of the request as possible, and your code to calculate the timestamp as soon as possible. This can be done with ColdFusion by setting the request time as near the top of you Application.cfm|cfc as possible.
<cfset Request.Now = Now()>
And then change the checking code to:
DateDiff("s", Variables.PassedTimestamp, DateConvert( "Local2UTC", Request.Now ))
If, instead, you're wanting to loop until 30 seconds has passed and then respond, note that your response won't appear to happen after 30 seconds because the latency will delay the response going back to the other end.
Getting exactly 30 seconds is impossible, but you take steps that will get your closer. If you're just needing to see if approximately 30 seconds has passed, then that will be good enough.
I have a template doing a boat load of manipulations, I expect it to take 30-45 minutes to complete it's processing... I've had SOME success in setting my application and session vars to timeout # 2 hr. and I've set my request timeout to 9999 (which should be 2.77 hrs)...
However - there seems to be a magic threshold - somewhere around the 20 min mark, my browser goes to a white screen (no output) and it appears as though the CF engine has also stopped working on my task...
can anyone suggest a reliable way to keep this process going - until it's done or my astronomical timeout occurs? in addition , is there any way to push feedback to the browser so it doesn't time out....I've tried cfflush, but that doesn't seem to do it.
You could use cfthread to run the process in a separate thread and then on the page you are accessing in the browser, you could use javascript to periodically poll the system to check on its status. For example, inside the long running process in cfthread, as you work through, you could set a application variables indicating that the process is still running and how far along it is, and retrieve and report those in the browser. When its complete, you could clear the variables, or set a complete flag, etc, and your browser report page will be able to indicate that it is complete.
I strongly suggest refactoring the code to use a simple messaging / queue system. It wouldn't take but 30 minutes to implement (or write a simple one from scratch!) and would provide a lot of benefits over and above solving this issue.
For example, its not a pass/fail for the entire operation. If you hit a snag at say the 1.5 hour mark, you won't be re-doing the entire process again, only parts which fail.
Doing it this way there is literally no limit to how much processing you can do because you'll be adding and removing from the stack as needed.
If you give a little more background, I'd be happy to help you figure out logical divisions to make it possible.
If you have a process running for that long, then you'll want to run it as a scheduled task.
I imagine your browser is the one dying.
Did you check to see if the request is still running?
<cfsetting requesttimeout= "3600" /> will set the page to last for an hour. If you run it as a scheduled task, then session timeout shouldn't affect anything.
Please do not store queries in session like that. Depending on the size of the query and the number of concurrent users in the system, you could easily run out of memory, causing some current and all subsequent requests to fail.
The database should be more than able to handle the heavy lifting. I'd hazard a guess that much of the processing you're doing in the application could be re-factored to happen directly on the database and save you a considerable amount of time.
Regardless, you should look into something like CFTHREAD as Sean mentioned, a scheduled task or a queuing system to handle a long process like this. The user most likely doesn't want to wait for the process to end before seeing the next screen. If they're told up front that the process is lengthy, they'll cope with waiting as long as they can move on to other tasks.
I had the same problem. Generally the browser will timeout after 3 minutes of nothing being sent from the server. For most of these long operations I was able to periodically output a dot to keep the browser alive but when it came to some extremely long queries importing 20M records from a server side CSV file I had to think of another way.
cUrl was the answer.
So here's what I did.
<?
function get_page($page)
{
$ch = curl_init($page);
curl_setopt($ch, CURLOPT_TIMEOUT, 0);
curl_setopt($ch, CURLOPT_NOPROGRESS,false);
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION,'progress');
curl_setopt($ch, CURLOPT_BUFFERSIZE, 128);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_exec($ch);
}
function progress($clientp,$dltotal,$dlnow,$ultotal,$ulnow='')
{
echo '. ';
flush();
return(0);
}
get_page('http://www.example.com/my_extremely_long_operation_script.php');
?>
Even with no output from the server, curl updates the download progress periodically.
Solved!
I am profiling some AS code by measuring wall clock time. In order to minimize the error I need to run the code for a long period of time. However, flash seems to protect itself from unresponsive scripts by throwing an exception after some period of unresponsiveness, namely: Error #1502: A script has executed for longer than the default timeout period of 15 seconds.
Is there any way to disable this protection, or at least extend the timeout period?
If you are publishing with Adobe Flash CS/4/5 etc.
Goto the publish settings. select "flash" at the bottom of this screen there is a textbox which says "Script Timeout" I know you can increase this, I think the limit is 90seconds even though you can enter any value here.
Can you move execution of the script across separate frames, and add a timer to advance the frame before the timeout period has lapsed? I believe the error only occurs when you've dwelled on a frame for more than 15 seconds.