I have a function which loops over a query and updates a database row for each item. After about 7000 iterations it's throwing an out of memory error - Java heap space.
Is there anything obviously wrong with this code ?
<cfloop query=loc.fixItems>
<cfset loc.count = loc.count + 1>
<cfset var categoryName = loc.fixItems.categoryName>
<cfinvoke component="Item" method="updateCode"
itemId="#loc.fixItems.itemId#" code="#loc.fixItems.newCode#"/>
<!--- Increment counter for category --->
<cfif structKeyExists(categoryMap, categoryName)>
<cfset var inc = structFind(categoryMap, categoryName) + 1>
<cfset structUpdate(categoryMap, categoryName, inc)>
<cfelse>
<cfset structInsert(categoryMap, categoryName, 1)>
</cfif>
</cfloop>
and in the update component:
<cffunction name="updateCode">
<cfargument name="itemId" type="numeric" required="yes">
<cfargument name="code" type="string" required="yes">
<cfset var loc = {}>
<cfquery name="loc.update">
update items
set code = <cfqueryparam value="#code#">
where id = <cfqueryparam value="#itemId#">
</cfquery>
</cffunction>
Don't use cfinvoke to create your Item component every iteration of your fixItems query. Create it once before that using createObject and simply call the updateCode method each time directly on the object.
The following can be done:
Change your <cfqueryparam> to use the appropriate cf_sql type. Are code and id really strings?
Don't give your <cfquery> a name. You are not keeping the result anyway. var loc doesn't help either
Bump up you memory to the JVM Addtional approach Use Java 7 and G1GC
Every 100 to 1000 iterations do a forced Garbage Collect
Update your data in bulk. XML based table variables can do this.
Make your function silent
Consider ORM on this
Related
I have a query where I am calling the columns like the way i want, I cannot use ColumnList of query because it sorts the column alphabatically, another thing i need to get the specific row of the query and its associated columns in a structure:
So here is my function which i am trying to bring the columns in the manner i want:
<cffunction name="rowToStruct" access="public" returntype="struct" output="false">
<cfargument name="queryObj" type="query" required="true" />
<cfargument name="row" type="numeric" required="true" />
<cfset var returnStruct = structNew()>
<cfset var colname = "">
<cfset arguments.queryObj = arrayToList(arguments.queryObj.getMeta().getColumnLabels())>
<cfloop list="#arguments.queryObj#" index="colname">
<cfset "returnStruct.#colname#" = arguments.queryObj[colname][arguments.row]>
</cfloop>
<cfreturn returnStruct/>
</cffunction>
before the above change the function was like this below:
<cffunction name="rowToStruct" access="public" returntype="struct" output="false">
<cfargument name="queryObj" type="query" required="true" />
<cfargument name="row" type="numeric" required="true" />
<cfset var returnStruct = structNew()>
<cfset var colname = "">
<cfloop list="#arguments.queryObj.columnList#" index="colname">
<cfset "returnStruct.#colname#" = arguments.queryObj[colname][arguments.row]>
</cfloop>
<cfreturn returnStruct/>
</cffunction>
Mine above one is giving me an error:
You have attempted to dereference a scalar variable of type class java.lang.String as a structure with members.
The getMetadata() function returns the columns in the order they were defined in the original statement. Docs: GetMetaData.
I can't see why your code would produce that error, although I'd do this:
<cfset returnStruct[colname] = arguments.queryObj[colname][arguments.row]>
Can you update your question with the exact error as it displays on the screen, including the code it focuses on, and the line numbers concerned (switch Robust Exception Handling on, if it's not on already)
The reason for the error is because you are overwriting the query object passed into the function. The new code resets the value of arguments.queryObj to a simple string here:
<cfset arguments.queryObj = arrayToList(arguments.queryObj.getMeta().getColumnLabels())>
Later in the code, you try and use the arguments.queryObj variable as if it were still a query object. Obviously, this causes an error because the variable now represents a string:
<cfset "returnStruct.#colname#" = arguments.queryObj[colname][arguments.row]>
The solution is to use a different variable to store the list of column names instead. Since it will also be function local variable, just be sure to scope it properly with local or var.
That said, as Adam mentioned in his answer, the feature you need is already supported using one of the built-in functions. So there is no really need to use the undocumented methods of the coldfusion.sql.QueryTable query class IMO. GetMetaData(queryObject) returns an array of structures containing two keys: Name (column name), IsCaseSensitive (boolean). To iterate through it, use an "array" loop instead of a "list" loop:
...
<cfset var returnStruct = {}>
<cfset var col = "">
<cfset var colArray = getMetaData(arguments.queryObj)>
<cfloop array="#colArray#" index="col">
<cfset returnStruct[col.Name] = arguments.queryObj[col.Name][arguments.row]>
</cfloop>
...
Update 1:
I cannot use ColumnList of query because it sorts the column
alphabatically
The bigger issue here is that CF structures are not ordered. Even if you add the values to the structure in the proper order, that order will not be maintained. You either need to address that in the calling code (ie use GetMetaData(query)) OR have the cffunction return both the ordered column names AND the row data. The latter seems a bit redundant, but it all depends on how you are using the function in your application.
Update 2:
IF you really do need a sorted structure .. there are also some java classes that do maintain the insertion order, such as a LinkedHashMap. Since it implements java.util.Map it can be used like a CF structure (in most ways).
<cfset var returnStruct = createObject("java", "java.util.LinkedHashMap").init()>
Runnable Example on trycf.com
NB: While having access to java objects is great, a common mistake is to forget that java is strongly typed, unlike CF. There are often subtle nuances that can easily bite you if you are not aware of them. So keep that in mind ... and be sure to read the API.
I have a query that gets called multiple times depending on the the variable vchrStatus
<cffunction name="getGalleriesByStatus" output="no" returntype="query">
<cfargument name="vchrStatus" type="string" required="yes">
<cfquery
name="getGalleries"
datasource="#Application.dsn#">
/// Long complicated query in here
</cfquery>
<cfset var result="#getGalleries#">
<!--- Return it --->
<cfreturn result>
</cffunction>
I tested the function with cfdump and it is outputing the desired results.
Now I want to loop over the results in a cfoutput
<cfoutput query="getGalleriesByStatus('Pending')">
But I'm getting the error: The value of the attribute query, which is currently getGalleriesByStatus('Pending'), is invalid.
<cfoutput> takes the name of the query variable (ie: a string). It does not take an expression which evaluates to a query. So you need this:
<cfset someVar = getGalleriesByStatus('Pending')>
<cfoutput query="someVar">
This is counter-intuitive, but is the case. It's also not in the docs, which sux.
I could not use student_id in cfoutput with query. My argument didn't pass in getStudentAnswers function. How can I do this?
I want to create JSON and need to add question and questions' answers near exam like this:
[
{
"id": "7",
"type": "2",
"question_id": "20",
"question": "Write a summary by answering the questions below: * What is convergence? * What will be the implications of convergence on classical media? on users? * Which devices/media forms do you think will converge nowadays/in the future? * What will be the opportunities and threats emerged by the convergence?",
"question_type": "0",
"answer": "",
"name": "Assignment3ver2",
"deadline_date": "2014-11-22 00:00:00.0"
}
Functions:
<cffunction name="getQuestion" >
<cfargument name="exam_id" type="string" required="true" >
<cfargument name="student_id" type="string" required="true" >
<cfset mert="#student_id#" >
<cfquery dataSource="mobileCourse" name="questselect">
SELECT id,question,question_type FROM dbo.Questions WHERE exam_id=#exam_id#
</cfquery>
<cfoutput query="questselect">
"question_id": "#id#",
"question": "#question#",
"question_type": "#question_type#",
#getStudentAnswers(mert, id, question)#
</cfoutput>
<cfreturn />
<cffunction name="getStudentAnswers" >
<cfargument name="student_id" type="string" required="true" >
<cfargument name="question_id" type="string" required="true" >
<cfargument name="question" type="string" required="true" >
<cfquery dataSource="mobileCourse" name="questselecta">
SELECT * FROM dbo.ANSWERS WHERE question_id=#question_id# AND student_id=#student_id#
</cfquery>
<cfif #questselecta.recordcount# gte 1>
<cfoutput query="questselecta">
"answer": "#answer#",
</cfoutput>
<cfelse>
"answer": "",
</cfif>
<cfreturn />
This is not a direct answer to your question. However, even ignoring the errors mentioned in the comments, the code above is not an efficient way to get the job done. It is not necessary to run a separate query for each question in the exam. Just run a single query using a JOIN instead. Since you appear to want all questions, even ones without an answer, make it an OUTER JOIN.
Note: Never use raw parameters in a cfquery because it puts your database at risk for sql injection. Instead use cfqueryparam with whichever cfsqltype matches actual data types of your columns (see documentation matrix).
<!--- adjust cfsqltypes as needed --->
<cfquery name="yourQueryName" ....>
SELECT q.id, q.question, q.question_type, a.answer, ... other columns
FROM Questions q LEFT JOIN Answers a
ON a.question_id = q.question_id
AND a.student_id = <cfqueryparam value="#ARGUMENTS.student_id#"
cfsqltype="cf_sql_integer">
WHERE q.exam_id = <cfqueryparam value="#ARGUMENTS.exam_id#"
cfsqltype="cf_sql_integer">
</cfquery>
Once you have the query results, use it to build your result object. The exact structure depends on your application, but assuming there can be multiple questions - an array of structures would be a good example.
Be sure to properly scope all variables. All function local variables should be var/local scoped to prevent potential threading issues: that includes query names.
<!--- localize all variables --->
<cfset var yourQueryName = "">
<cfset var resultArray = "" >
<cfset var data = "" >
.... run query ....
<!--- first initialize the array --->
<cfset resultArray = []>
<!--- loop through your query and build an array of structures --->
<cfloop query="yourQueryName">
<!--- create new structure to store current question --->
<cfset data = {}>
<cfset data["question_id"] = yourQueryName.question_id>
<cfset data["question"] = yourQueryName.question>
<cfset data["answer"] = yourQueryName.answer>
... other keys ....
<!--- append the question to your array --->
<cfset arrayAppend(resultArray, data)>
</cfloop>
Finally, use serializeJSON to return the array of structures in JSON format:
<cfreturn serializeJSON(resultArray)>
I have set of inserts that are wrapped in a <cftransaction> block, and I am getting a error and the insert is being rolled back.
Here is the code in question stubbed for space:
<cffunction name="InsertTCUV" access="public">
<cfargument name="vehicle required="true" type="xml" />
//Parsing the xml document here
<cftransaction>
<cfquery name="TCUVinsert datasource="mydb">
INSERT INTO tcuv
VALUES(...)
<cfquery>
<cfquery name="qLatestTCUVID" datasource="mydb">
SELECT TOP 1 tcuv_id FROM dbo.tcuv ORDER BY tcuv_id DESC
</cfquery>
<cfset curTCUVID = qLatestTCUVID.tcuv_ID>
<cfset optionsResult = insertOptions(curTCUVID,vehicle>
<cfset imagesResult = insertImages(curTCUVID,vehicle)>
<cfset standardFeaturesResult = insertStandardFeatures(curTCUVID,vehicle
</cftransaction>
</cffunction>
<cffunction name="insertOptions" access="private">
<cfargument name="TCUVID required="true type="numeric" />
<cfargument name="vehicleInfo" required="true" type="xml" />
<cfset var result = "good">
<cftry>
<cfset optionNode = xmlSearch(arguments.vehicleInfo[1], "p:RemarketingOption">
<cfloop index="i" from="1" to="#arrayLen(optionNode)#">
<cfset optionNodeNotes = XmlSearch(optionNode[#i#], "p:OptionNotes")>
<cfset optionNotes = "">
<cfloop index="j" from="1" to="#ArrayLen(optionNotesNodes)#">
<cfoutput>
<cfset optionNotes = optionNotes & " " & #optionNotesNodes[j].xmlText#>
</cfoutput>
</cfloop>
<cfquery name="insertOptions" datasource="mydb">
INSERT INTO dbo.tcuv_options
VALUES (
<cfqueryparam cfsqltype="cf_sql_integer" value="#arguments.TCUVID#">,
<cfqueryparam cfsqltype="cf_sql_varchar" value='#xmlSearch(optionNode[i], "p:OptionID")[1].xmlText#'>,
<cfqueryparam cfsqltype="cf_sql_varchar" value='#xmlSearch(optionNode[i], "p:OptionTypeCode")[1].xmlText#'>,
<cfqueryparam cfsqltype="cf_sql_varchar" value='#xmlSearch(optionNode[i], "p:OptionShortDescription")[1].xmlText#'>,
<cfqueryparam cfsqltype="cf_sql_varchar" value="#optionNotes#">
)
</cfquery>
</cfloop>
<cfcatch type="database">
//dumping cfcatch.* information
<cfset result = "error"
</cfcatch>
</cftry>
<cfreturn result>
</cffunction>
This whole thing is in a loop on the calling page, and everything works fine the first time through the loop. One the second pass, the TCUVInsert works, but I get to the insertOptions function, a coldfusion error is thrown saying that, variable insertOptions is undefined, and when I get the database, the second row isn't there, which tells me there was an error and the insert rolled back. So, there is an error with the insert of the options, and coldfusion isn't giving me the right error to diagnose it. So either I have to look in the database logs, which apparently are not setup, or try to extract the error from the cftransaction block, which I'm not sure how to do.
coldfusion 9, sql server 2008 r2
Any thoughts?
<cffunction name="insertOptions" access="private">
You are overwriting the function by using the same name for a query variable:
<cfquery name="insertOptions" datasource="imports">
Interestingly it is all because the query name was not var scoped. Functions are stored in the variables scope of the component. So by failing to localize the query name, you end up overwriting the function stored in variables.insertOptions when you run the query. Because insert statements do not return a resultset, that variable ends up being undefined. Hence the error. In this case the solution is to either scope the query name, or better yet remove it entirely (since it is not populated anyway).
Just one more reason to always var/local scope function variables - yes, query names too!
I want to get a specific row in a ColdFusion Query object without looping over it.
I'd like to do something like this:
<cfquery name="QueryName" datasource="ds">
SELECT *
FROM tablename
</cfquery>
<cfset x = QueryName[5]>
But it's giving me an error saying that the query isn't indexable by "5". I know for a fact that there are more than 5 records in this query.
You can't get a row in CF <= 10. You have to get a specific column.
<cfset x = QueryName.columnName[5]>
It's been 8 years since I posted this answer, however. Apparently CF11 finally implemented that feature. See this answer.
This can now be accomplished in coldfusion 11 via QueryGetRow
<cfquery name="myQuery" result="myresult" datasource="artGallery" fetchclientinfo="yes" >
select * from art where ARTID >
<cfqueryparam value="2" cfsqltype="CF_SQL_INTEGER">
</cfquery>
<cfdump var="#myQuery#" >
<cfset data = QueryGetRow(myQuery, 1) >
<cfdump var="#data#" >
I think there is a simpler solution...
I am guessing you know your column names and only want this column or that one. Then you don't need to put the whole row in a struct. You can reference the query by row number (remember its 1 based not 0).
yourQueryName["yourColumnName"][rowNumber]
<cfoutput>
#mycontacts["Name"][13]#
#mycontacts["HomePhone"][13]#
</cfoutput>
You have to convert the query to a struct first:
<cfscript>
function GetQueryRow(query, rowNumber) {
var i = 0;
var rowData = StructNew();
var cols = ListToArray(query.columnList);
for (i = 1; i lte ArrayLen(cols); i = i + 1) {
rowData[cols[i]] = query[cols[i]][rowNumber];
}
return rowData;
}
</cfscript>
<cfoutput query="yourQuery">
<cfset theCurrentRow = GetQueryRow(yourQuery, currentRow)>
<cfdump var="#theCurrentRow#">
</cfoutput>
Hope this points you in the right direction.
I know I come back to this thread any time I Google "cfquery bracket notation". Here's a function I wrote to handle this case using bracket notation. Hopefully this can help someone else too:
<cffunction name="QueryGetRow" access="public" returntype="array" hint="I return the specified row's data as an array in the correct order">
<cfargument name="query" required="true" type="query" hint="I am the query whose row data you want">
<cfargument name="rowNumber" required="true" hint="This is the row number of the row whose data you want">
<cfset returnArray = []>
<cfset valueArray = []>
<cfset cList = ListToArray(query.ColumnList)>
<cfloop from="1" to="#ArrayLen(cList)#" index="i">
<cfset row = query["#cList[i]#"][rowNumber]>
<cfset row = REReplace(row, "(,)", " ")>
<cfset returnArray[i] = row>
<cfset i++>
</cfloop>
<cfreturn returnArray>
</cffunction>
The REReplace is optional, I have it in there to cleanse commas so that it doesn't screw up the arrayToList function later on if you have to use it.
I wanted to extract a single row from a query, and keeping the column names (of course). This is how I solved it:
<cffunction name="getQueryRow" returntype="query" output="no">
<cfargument name="qry" type="query" required="yes">
<cfargument name="row" type="numeric" required="yes">
<cfset arguments.qryRow=QueryNew(arguments.qry.columnlist)>
<cfset QueryAddRow(arguments.qryRow)>
<cfloop list="#arguments.qry.columnlist#" index="arguments.column">
<cfset QuerySetCell(arguments.qryRow,arguments.column,Evaluate("arguments.qry.#arguments.column#[arguments.row]"))>
</cfloop>
<cfreturn arguments.qryRow>
</cffunction>
Methods previously described for obtaining query data by column name and row number (variables.myquery["columnName"][rowNumber]) are correct, but not convenient for getting a full row of query data.
I'm running Railo 4.1. And this is a cool solution. Too bad this can't be done the way we would want outright to get a full row of data, but the following method allows us to get what we want through a few hoops.
When you serializeJSON(variables.myquery) it changes the query to a JSON formatted cfml struct object with two items: "Columns" and "Data". Both of these are arrays of data. The "data" array is a two-dimensional array for rows and then columnar data.
The issue is that now we have an unusable string. Then if we re-serialize it it's NOT a query, but rather usable regular struct in the format described above.
Assume we already have a query variable named 'variables.myquery'. Then look at the following code:
<cfset variables.myqueryobj = deserializeJSON(serializeJSON(variables.myquery)) />
Now you get the two dimensional array by getting this:
<cfset variables.allrowsarray = variables.myqueryobj.data />
And you get one query row array by getting this:
<cfset variables.allrowsarray = variables.myqueryobj.data[1] />
OR the last row this way:
<cfset variables.allrowsarray = variables.myqueryobj.data[variables.myquery.recordCount] />
And you can get individual column values by column order number iteration:
<cfset variables.allrowsarray = variables.myqueryobj.data[1][1] />
Now this might be slow and possibly unwise with large query results, but this is a cool solution nonetheless.
Check out the documentation for queryGetRow. It accepts a query object and an index of the row with the first row being referenced with the index of 1 (NOT 0) The index used this way is required to be a positive integer.
<cfquery name="QueryName" datasource="ds">
SELECT *
FROM tablename
</cfquery>
<!---
This would retrieve the first record of the query
and store the record in a struct format in the variable 'x'.
--->
<cfset x = queryGetRow(QueryName, 1) />
<!---
This is an alternative using the member method form of queryGetRow
--->
<cfset x = QueryName.getRow(1) />