I am using ColdFusion 10 Update 23 with MySQL database.
When I make a change to a script where I am using cfqueryparm the script causes this error message: "The type for attribute value of tag queryparam could not be determined."
This script works perfectly:
<cfquery name="i" datasource="tasktrack">
UPDATE qa_commitments
SET
commit_division = <cfqueryparam cfsqltype='cf_sql_integer' value='#trim(commit_division)#'/>,
commit_source = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_source))#'/>,
commit_title = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_title))#'/>,
commit_responsibility = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_responsibility))#'/>,
<cfif Len(commit_cause)>commit_cause = <cfqueryparam cfsqltype='cf_sql_varchar' value='#commit_cause#'/>
<cfelse>commit_cause = NULL</cfif>,
commit_comments = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(PreserveSingleQuotes(commit_comments)))#'/>,
commit_issue_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_issue_date#'/>,
commit_response_due_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_response_due_date#'/>,
commit_response_compl_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_response_compl_date#'/>,
commit_action_due_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_action_due_date#'/>
<cfif IsDefined('commit_closeing_date')> ,commit_closeing_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_closeing_date#'/></cfif>
WHERE ID = #id#
If I open the script and add anything (i.e, a return, a tab, a comment), save the script I get the above error. If I restore the script from a older version it works fine again. I open the older one insert a line break w/enter key, save it and then it breaks again.
The really weird part is if I remove the cfqueryparam tags completely the script works again.
This happens on every script where I am using this tag.
The last update to CF was April 2017 and there are several scripts newer that work unless I edit them.
I have tried different editors with the same results.
I have googled my butt off with no results.
Can someone please point me in the right direction?
I can't tell you why it is broke, but I can tell you how I would fix it.
Putting <cfif>s inside of queries makes it so that the DB engine cannot cache the query. So I would move them out, and out the conditional logic within the query.
<cfquery name="i" datasource="tasktrack">
DECLARE #commit_clause varchar(40) = <cfqueryparam cfsqltype='cf_sql_varchar' value='#commit_cause#' null="#IIF(len(commit_clause)1, 0)#"/>
DECLARE #commit_closeing_date date = cfqueryparam cfsqltype='cf_sql_date' value='#commit_closeing_date#' null="#IIF(isDefined(Commit_closeing_date), 0, 1)#"/>.
UPDATE qa_commitments
SET
commit_division = <cfqueryparam cfsqltype='cf_sql_integer' value='#trim(commit_division)#'/>,
commit_source = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_source))#'/>,
commit_title = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_title))#'/>,
commit_responsibility = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(commit_responsibility))#'/>,
commit_cause = #commit_clause,
commit_comments = <cfqueryparam cfsqltype='cf_sql_varchar' value='#ucase(trim(PreserveSingleQuotes(commit_comments)))#'/>,
commit_issue_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_issue_date#'/>,
commit_response_due_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_response_due_date#'/>,
commit_response_compl_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_response_compl_date#'/>,
commit_action_due_date = <cfqueryparam cfsqltype='cf_sql_date' value='#commit_action_due_date#'/>,
commit_closeing_date = #commit_closeing_date
WHERE ID = #id# /* I would fix this too */
</cfquery>
Off topic
I would consider using Entities. I really don't like writing this stuff over and over.
I found the problem and the problem was McAfee scriptscan. I had IT guys turn it off and now the script works. IT is blaming it on the old version of Windows Server 2008 and it having a problem with McAfee Enterprise. Thanks for all the help. It really does help just to have someone to walk you through things.
Related
I have few variables that contain multiple values. Basically I want to store all the values into my database. I am using this code which I got here in Stackoverflow.
<cfquery datasource="databaseName">
INSERT INTO spreadsheet
([Local.Time.Stamp],
[Energy.Delivered..kVAh.],
[Energy.Received..kVAh.],
[Energy.Received..kVARh.],
[Energy.Delivered..kVARh.],
[Real.A..kW.],
[Real.B..kW.])
VALUES
(<cfloop query="excelquery">
'#excelquery.col_1#',
'#excelquery.col_2#',
'#excelquery.col_3#',
'#excelquery.col_4#',
'#excelquery.col_5#',
'#excelquery.col_6#',
'#excelquery.col_7#'
</cfloop>)
</cfquery>
However I always get a syntax error. I believe that my cfloop part is wrong, can someone please tell me the correct way for me to write that cfloop?
The problem is with the generated query not cfloop i.e., for entering multiple values the format should be like this:
INSERT INTO TableName (col,col,...) VALUES (val,val,...),(val,val,...),...
Also, use cfqueryparam to avoid sql injection.
You can try this:
<cfquery datasource="databaseName">
INSERT INTO spreadsheet
([Local.Time.Stamp],
[Energy.Delivered..kVAh.],
[Energy.Received..kVAh.],
[Energy.Received..kVARh.],
[Energy.Delivered..kVARh.],
[Real.A..kW.],
[Real.B..kW.])
VALUES
<cfloop query="excelquery">
<!--- cf_sql_varchar is just an example. --->
(<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_1#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_2#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_3#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_4#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_5#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_6#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#excelquery.col_7#">)
#excelQuery.currentRow NEQ excelQuery.recordCount ? ',': ''#
</cfloop>
</cfquery>
You might have to add a recordCount check before generating the query to avoid errors for no records.
I've just started using Mailgun for 2-way emails within a web app I'm developing for in-house use. I have a route set up to forward messages to a URL on my server - which is working fine.
However, the code at my end to process these messages is throwing errors. Malign sends the data as a http post, which I can then reference with a form variable. Initially I was getting 'not defined' errors on some fields (those with hyphens in the field names), however that now appears to be resolved. The code is now stumbling on the CFQUERY insert, with the error stating I have an error in the SQL syntax - but I can't see anything wrong with it!
This is the code I have for the page mailgun sends the post to;
<cfset thebody = form["body-plain"]>
<cfset thesender = form["sender"]>
<cfset therecipient = form["sender"]>
<cfset thesubject = form["subject"]>
<cfquery name="addmail">
INSERT INTO mailmessages(from,sender,recipient,subject,body,msgdate)
VALUES('#thesender#','#thesender#','#therecipient#','#thesubject#','#thebody#',#CreateODBCDateTime(Now())#)
</cfquery>
The error message states;
Error Executing Database Query. You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'from,sender,recipient,subject,body,msgdate) VALUES('lee#mydomain.com','l' at line 1
A cfdump of the form can be seen here (this is sent to be via cfmail in a cfcatch statement - the only way I'm able to see any errors);
The from column name is a reserved word, so this is what mySQL is complaining about.
If you escape the column name, it should fix this.
Along with cfqueryparam your code should be something like:
<cfset thebody = form["body-plain"]>
<cfset thesender = form["sender"]>
<cfset therecipient = form["sender"]>
<cfset thesubject = form["subject"]>
<cfquery name="addmail">
INSERT INTO mailmessages (
"from",
sender,
recipient,
subject,
body,
msgdate
)
VALUES (
<cfqueryparam value="#thesender#" cfsqltype="cf_sql_varchar">,
<cfqueryparam value="#thesender#" cfsqltype="cf_sql_varchar">,
<cfqueryparam value="#therecipient#" cfsqltype="cf_sql_varchar">,
<cfqueryparam value="#thesubject#" cfsqltype="cf_sql_varchar">,
<cfqueryparam value="#thebody#" cfsqltype="cf_sql_varchar">,
<cfqueryparam value="#CreateODBCDateTime(Now())#" cfsqltype="cf_sql_timestamp">
)
</cfquery>
As the error message suggests, there is a syntax error in the insert query.
Seems like you are missing single quotes (') around the date value you are passing. It should be written like this:
<cfquery name="addmail">
INSERT INTO mailmessages(from,sender,recipient,subject,body,msgdate)
VALUES('#thesender#','#thesender#','#therecipient#','#thesubject#','#thebody#','#CreateODBCDateTime(Now())#')
</cfquery>
To get rid of such syntax errors and prevent SQL injection always use cfqueryparams like this:
<cfquery name="addmail">
INSERT INTO mailmessages (from,sender,recipient,subject,body,msgdate)
VALUES (
<cfqueryparam value="#thesender#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#thesender#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#therecipient#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#thesubject#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#thebody#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#CreateODBCDateTime(Now())#" cfsqltype="CF_SQL_TIMESTAMP">
)
</cfquery>
Instead of CF_SQL_TIMESTAMP you can also use CF_SQL_DATE as per your DB or data type you have specified for the column.
I can't select Jedihomer Townend's comment above as the answer, but he was correct to point out that my fieldnames contained reserved words, and that was causing the problem. Changing those fixed it.
I am working on a software where I cannot create a new member (ie person) with special characters in the surname. For example, the surname cannot be "O'Connor", because it contains a special character. But that type of surname can be added in database directly.
I am using ColdFusion and PostgreSQL 9.3. It works by removing the "cleantrim" function before surname in coding, but it is a temporary solution. Can anyone help me how we can do it?
'#Call.tSurName#',
'#TrimCleanValue(Call.tFirstName)#',
'#TrimCleanValue(CAll.tMiddleName)#',
'#TrimCleanValue(Call.tPReferredName)#',
'#TrimCleanValue(Call.tScoutingName)#',
'#TrimCleanValue(Call.tPostNominal)#',
'#Call.tTitle#',
You should be using cfqueryparam for all database inserts. Especially something coming from the client. Not only does cfqueryparam help prevent SQL injection it also escapes quotes and can help make your queries faster.
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tSurName#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tFirstName#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tMiddleName">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tPReferredName">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tScoutingName">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tPostNominal">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#Call.tTitle">
I have the following cfquery:
<cfquery name="CQuery" datasource="XX.X.X.XXX">
INSERT INTO DatabaseName
(PhoneNumber_vch,
Company_vch,
date_dt)
VALUES(#PhoneNumber#,
#Company#,
#Date# )
</cfquery>
There are null values in case Company name doesnt' exist and I believe becasue of that Iam getting the following error:
Error Executing Database Query.
[Macromedia][SQLServer JDBC Driver][SQLServer]Incorrect syntax near ','.
The comma , pointed in the error message is after #Company# field. Please let me know is it's because of null values and best approach to fix it?
The values in the PhoneNumber, company and Date are extracted from a XML SOAP response with proper usage of trim function as discussed in my previous post.
Using cfif in coldfusion
Thanks
If you use CFQueryParam like you should on any database SQL that accepts dynamic parameters you can kill two birds with one stone. First and most important, prevent SQL Injection Attacks and second you can use the attribute of the null="" to insert a NULL value into your record.
<cfquery name="CQuery" datasource="XX.X.X.XXX">
INSERT INTO DatabaseName (PhoneNumber_vch, Company_vch, date_dt)
VALUES(
<cfqueryparam cfsqltype="cf_sql_varchar" value="#trim(PhoneNumber)#" null="#NOT len(trim(PhoneNumber))#" />
,<cfqueryparam cfsqltype="cf_sql_varchar" value="#trim(Company)#" null="#NOT len(trim(Company))#" />
,<cfqueryparam cfsqltype="cf_sql_timestamp" value="#Date#" null="#NOT len(trim(Date))#" />
)
</cfquery>
You will want to use <cfqueryparam> to take care of nulls (and injection attacks)
Try
<cfquery name="CQuery" datasource="XX.X.X.XXX">
INSERT INTO DatabaseName
(PhoneNumber_vch,
Company_vch,
date_dt)
VALUES(
<cfqueryparam value = "#PhoneNumber#" cfsqltype = "CF_SQL_VARCHAR">,
<cfqueryparam value = "#Company#" cfsqltype = "CF_SQL_VARCHAR"
null = "#IIF(Company EQ "", 1, 0)#">,
<cfqueryparam value = "#Date#" cfsqltype = "CF_SQL_TimeStamp"
null = "#IIF(Date EQ "", 1, 0)#" >
)
</cfquery>
Also see: cfqueryparam
You either need to qualify your varchar entries (surround all varchar entries with single quotes, or, better would be change them to cfqueryparams;
<cfquery name="CQuery" datasource="XX.X.X.XXX">
INSERT INTO DatabaseName
(PhoneNumber_vch,
Company_vch,
date_dt)
VALUES(<cfqueryparam value="#PhoneNumber#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#Company#" cfsqltype="CF_SQL_VARCHAR">,
<cfqueryparam value="#Date#" cfsqltype="CF_SQL_TIMESTAMP"> )
</cfquery>
Updates:
I have submitted the bug to Adobe and referenced this SO question
In my real-world code where the problem occurred I decided to just remove my use of cfqueryparam. I am now using a custom function to format the param based on type. There are security and speed concerns that I will have to deal with but it gets the particular process working acceptably under current load.
In the future I am planning on going to process that pulls the data files into temporary tables in the database. I'll then perform operations on the data and transfer data to live tables using SQL as much as possible, instead of relying on ColdFusion
I am having a problem with looping over queries using cfqueryparam tags while inserting data. (I have not tested with select or update queries). The looping progressively takes up more memory that is not released until the request is done. However, the problem only occurs when looping over a query while in a function.
It appears to be very sensitive to the number of cfqueryparam tags used. In this example there are 15 values being inserts however in my code that actually needs this to work I am inserting an unknown number of values that can make the problem more severe.
Below is code that shows the problem. Give it a datasource name (tested on MSSQL) and it will create a tmp table and insert records as example with and without being in a function. Memory usage is display before, after the non-function loop, then after the in-function loop. It also requests garbage collection and waits 10 seconds before outputting memory info to ensure it is displaying info as accurately as possible.
In my experience with this particular test the in-function loop resulted in over 200mb of memory being used. In my real world uses it crashes ColdFusion :-(
<cfsetting enablecfoutputonly="true">
<cfsetting requesttimeout="600">
<cfset insertCount = 100000>
<cfset dsn = "TmpDB">
<cfset dropTmpTable()>
<cfset createTmpTable()>
<cfset showMemory("Before")>
<cfflush interval="1">
<cfloop from="1" to="#insertCount#" index="i">
<cfquery name="testq" datasource="#dsn#">
INSERT INTO tmp ( [col1],[col2],[col3],[col4],[col5],[col6],[col7],[col8],[col9],[col10],[col11],[col12],[col13],[col14],[col15] )
VALUES ( <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR"> )
</cfquery>
</cfloop>
<cfset showMemory("After Non-Function INSERTS")>
<cfflush interval="1">
<cfset funcTest()>
<cfset showMemory("After Function based INSERTS")>
<cfset dropTmpTable()>
<cffunction name="funcTest" output="false">
<cfset var i = 0>
<cfset var testq = "">
<cfloop from="1" to="#insertCount#" index="i">
<cfquery name="testq" datasource="#dsn#">
INSERT INTO tmp ( [col1],[col2],[col3],[col4],[col5],[col6],[col7],[col8],[col9],[col10],[col11],[col12],[col13],[col14],[col15] )
VALUES ( <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR">, <cfqueryparam value="TestValue" cfsqltype="CF_SQL_CHAR"> )
</cfquery>
</cfloop>
</cffunction>
<cffunction name="showMemory" output="true">
<cfargument name="label" required="true">
<cfset var runtime = "">
<cfset var memoryUsed = "">
<cfset requestGC("10")>
<cfset runtime = CreateObject("java","java.lang.Runtime").getRuntime()>
<cfset memoryUsed = (runtime.totalMemory() - runtime.freeMemory()) / 1024 / 1024>
<cfoutput>
<h2>#arguments.label#</h2>
Memory Used: #Round(memoryUsed)#mb
</cfoutput>
</cffunction>
<cffunction name="requestGC">
<cfargument name="waitSeconds" required="false" default="0" type="numeric">
<cfscript>
createObject("java","java.lang.Runtime").getRuntime().gc();
createObject("java", "java.lang.Thread").sleep(arguments.waitSeconds*1000);
</cfscript>
</cffunction>
<cffunction name="dropTmpTable" output="false">
<cftry>
<cfquery datasource="#dsn#">
DROP TABLE tmp
</cfquery>
<cfcatch type="database"></cfcatch>
</cftry>
</cffunction>
<cffunction name="createTmpTable" output="false">
<cfquery datasource="#dsn#">
CREATE TABLE tmp(
col1 nchar(10) NULL, col2 nchar(10) NULL, col3 nchar(10) NULL, col4 nchar(10) NULL, col5 nchar(10) NULL, col6 nchar(10) NULL, col7 nchar(10) NULL, col8 nchar(10) NULL, col9 nchar(10) NULL, col10 nchar(10) NULL, col11 nchar(10) NULL, col12 nchar(10) NULL, col13 nchar(10) NULL, col14 nchar(10) NULL, col15 nchar(10) NULL
) ON [PRIMARY]
</cfquery>
</cffunction>
Just to show that memory can be released during an operation, here is example code that builds up a larger struct and shows memory used before and after the variable is overwritten and garbage collected. In my run of this memory used after population is 118mb and after overwriting and garbage collection it is 31mb.
<cfset showMemory("Before struct creation")>
<cfflush interval="1">
<cfset tmpStruct = {}>
<cfloop from="1" to="1000000" index="i">
<cfset tmpStruct["index:#i#"] = "testvalue testvalue testvalue testvalue testvalue testvalue testvalue testvalue testvalue testvalue">
</cfloop>
<cfset showMemory("After struct population")>
<cfflush interval="1">
<cfset tmpStruct = {}>
<cfset showMemory("After struct overwritten")>
Do you have debugging on in Administrator?
If so, even if you've got showdebugoutput="false", CF will be keeping debug information about all of those queries, and with that many queries, the debugging information could quickly build up.
Also, if you've really got 80,000 rows to insert, you probably want to be doing this a different way - e.g. generating an import script that runs directly against the DB, (without CF/JDBC getting in the way).
Maybe multiple insert can help? This technique itself typically works faster, saving some time can help you save some memory.
Yes I've seen your note "inserting an unknown number of values", but this should work if you have constant number of fields/values in a single insterting batch.
No idea if it will make a difference, but something to try - shrink the in-function loop, and loop round the function multiple times.
What this does with memory might help narrow down where it is being used up.
<cffunction name="funcTest" output="false">
<cfargument name="from" />
<cfargument name="to" />
<cfset var i = 0>
<cfset var testq = "">
<cfloop from="#arguments.from#" to="#arguments.to#" index="i">
<cfquery name="testq" datasource="#dsn#">
...
</cfquery>
</cfloop>
</cffunction>
<cfset BlockSize = 100 />
<cfloop index="CurBlock" from="1" to="#(InsertCount/BlockSize)#">
<cfset funcTest
( from : CurBlock*(BlockSize-1) + 1
, to : CurBlock*BlockSize
)/>
</cfloop>
I encountered a similar problem.
http://misterdai.wordpress.com/2009/06/24/when-not-to-use-cfqueryparam/
The approach depends on the few things. If you can trust the data, don't use cfqueryparam's, that'll reduce memory usage a lot. From there, minimize the SQL as much as possible. I was doing quite a bit of DB work per row, so I created a stored procedure instead. The big bonus in fighting memory usage was to buffer SQL calls to the database. Create an array, append your SQL to it, then every 50 rows (personal choice after testing) do an ArrayToList on the array, inside a CfQuery tag. This limits the database traffic to less, but larger, instead of many smaller ones.
After all of that, things worked for me. But I still think ColdFusion really isn't up to this type of task, more the domain of the database server itself if possible.
My first guess would be to type the values in your cfqueryparam - as in type="CF_SQL_CHAR". Why would this help? I'm not sure, but I can guess that there would be additional overhead with a non-typed variable.
Assuming you are using CF8... not sure if this happens in CF7...
Try turning off "Max Pooled Statements" (set it to zero) in your datasource "advanced settings"... I bet money your memory leak goes away...
That is where I have found the bug to be... this was causing all kinds of crashes on some CF servers until we found this... we are 100% more stable now because of this...
Patrick Steil
try to prepend "variables." before each query inside of your cffunctions. I've had a similiar issue and this fixed it.
So change:
<cfquery name="testq" datasource="CongressPlus">
to
<cfquery name="variables.testq" datasource="CongressPlus">
Cheers,
Thomas
It's been well documented all over the community that CF will not release memory until after the request is finished. Even calling the GC directly has no effect on freeing up memory during a running request. Don't know if this is by design or a bug.
I haven't a clue why you would even want to do something like this in CF anyways. There is no reason for you to be inserting 80K rows into a database using CF, no matter which database engine you're using.
Now, if there is a reason that you need to do this, such as you're getting the data from say an uploaded CSV or XML file; MSSQL has a TON of better ways to do this and workarounds.
One approach that I have done over the years is to create a stored procedure in MSSQL that calls BCP or BULK INSERT to read a file that contains the data to insert.
The best thing about this approach is that the only thing CF is doing is handling the file upload and MMSQL is doing all the work processing the file. MSSQL has no problems inserting millions of rows using BCP or BULK INSERT and will be INFINITELY faster then anything CF can process.
The way to prevent memory leaks from cfqueryparam in a large loop of queries was to not use cfqueryparam. However a broader answer is on avoiding CF's inefficiencies and memory leaks is to not use CF in these situations. I got the particular process to an acceptable level for the load at the time but in the long run will be rewriting it in another language, probably C# directly in the database engine.
I have no idea if that would fix your problem but what I usually do when I have multiple inserts like this is, a loop of the SQL statement itself instead of the entire cfquery.
So instead of having :
<cfloop from="1" to="#insertCount#" index="i">
<cfquery name="testq" datasource="#dsn#">
...
</cfquery>
</cfloop>
I do :
<cfquery name="testq" datasource="#dsn#">
<cfloop from="1" to="#insertCount#" index="i">
...
</cfloop>
</cfquery>
So instead of having multiple call to the database you only have one big one.
I have no idea how this would affect your memory leak problem, but I never experienced any memory leaks doing it that way.