I am upgrading from Adobe CF10 to CF2016 and noticed that in CF2016 modifications to a cached query object appear to be persisted back to cache. This does not appear to be the case in CF10.
For example:
<cfquery datasource="myDs" name="rs" cachedwithin="#createtimespan(0,0,5,0)#">
SELECT 10 AS value
</cfquery>
<cfset rs.value = 2016>
<cfquery datasource="myDs" name="rs" cachedwithin="#createtimespan(0,0,5,0)#">
SELECT 10 AS value
</cfquery>
<!--- CF10 outputs 10 --->
<!--- CF2016 outputs 2016 --->
<cfoutput>#rs.value#</cfoutput>
Am I misunderstanding something here? I couldn't find this documented anywhere. RTFMs welcome.
Let's define a query object, name it variables.rs and place it into cache.
<cfquery datasource="myDs" name="rs" cachedwithin="#createtimespan(0,0,5,0)#">
SELECT 10 AS value
</cfquery>
This query object contains one record and one column (named value). If we output the value of this one record's column, referenced as variables.rs.value, we see the value of 10.
Now, let's define a struct named variables.rs, with a single key named value and a value of 2016. If we output this value, referenced as variables.rs.value, we see the value of 2016.
Finally, let's run that query again.
<cfquery datasource="myDs" name="rs" cachedwithin="#createtimespan(0,0,5,0)#">
SELECT 10 AS value
</cfquery>
But the query was run within the time of cachedwithin and the recordset never changed, so there's no need to update the query cache. Therefore, the variable variables.rs will not be updated.
This leaves variables.rs as a struct, with the key named value, whose value is 2016.
This seems to be the correct progression of events that should occur. It was probably a bug in previous versions that caused them to act as they did, which was fixed in 2016 (yay!?).
It's likely that no one has created a bug report outlining this exact problem with steps to recreate, therefore, the CF development team doesn't know this is a thing.
Of couse, it might just be easier for you to refactor your code than to hold your breath waiting for the issue to be triaged, accepted, fixed and released. But you could at least give it a shot. My company has filed a number of major changes in funcitonality between 2016 and previous versions and had most, if not all of them fixed.
Related
Can somebody please suggest what am I doing wrong in the query below:
Data type of imageUpdateDate column is date. I also tried with datetime with no luck.
Any help would be greatly appreciated.
imageUpdateDate ='#now()#'
The query is failing because of the single quotes around #now()#.
The reason is that the string representation of #now()# already contains single quotes:
{ts '2022-03-31 22:35:14'}
By wrapping that value in single quotes, you're creating an invalid sql statement:
UPDATE used_listings_v2
SET [image] = 'new_file_name.ext'
, imageUpdateDate ='{ts '2022-03-31 22:35:14'}'
WHERE id = '1'
Which when executed, fails with an error:
Error Executing Database Query. [Macromedia][SQLServer JDBC
Driver][SQLServer]Incorrect syntax near '2022'.
Although removing the single quotes, or using getDate() instead, would resolve the error, the fact that single quotes can so easily break the query means that anyone attempting to attack your database with malicious sql can do the exact same thing!. That's dangerous for any application. Especially one running on the EOL'd CF10, which no longer receives security updates. Consider upgrading or switching to Lucee.
Anyway, the correct way to resolve the error is by using cfqueryparam on all of the query variables. Not only does it eliminate the need for using single quotes around parameter values, it also helps protect your database against sql injection. (I don't know your column data types, so modify the cfsqltypes as needed)
<cfquery datasource="...">
UPDATE used_listings_v2
SET [image] = <cfqueryparam value="#dbFilename#" cfsqltype="cf_sql_varchar">
, imageUpdateDate = <cfqueryparam value="#now()#" cfsqltype="cf_sql_timestamp">
WHERE id = <cfqueryparam value="#edit#" cfsqltype="cf_sql_integer">
</cfquery>
*Note: It's a good practice to always scope your variables. For example, use FORM.dbFileName instead of just dbFileNameity updates.
I'm wondering if there is a better way to do what I'm doing. It works but I feel there should be a better way. If my query result in 20K records for example, I'm getting "The request has exceeded the allowable time limit Tag: CFQUERY"
<cfquery name="GetMyRecords" datasource="MyDSN">
SELECT idnumber,PrefAddr,...more colums
FROM um_valid
WHERE userid = <cfqueryparam cfsqltype="cf_sql_varchar"
value="#session.userid#">
AND session_id = <cfqueryparam cfsqltype="cf_sql_numeric"
value="#session.Session_Id#">
AND status NOT IN (<cfqueryparam cfsqltype="cf_sql_varchar"
value="X,C">)
</cfquery>
I also have an existing store procedure that expect some values from the query to do what it's supposed to do. So I'm looping it like this:
<cfloop query="GetMyRecords">
<cfstoredproc procedure="MyProc" datasource="MyDSN">
<cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR"
dbvarname="#id_number" value="#Trim(idnumber)#">
<cfprocparam type="In" cfsqltype="CF_SQL_VARCHAR"
dbvarname="#Aaddr_pref_ind" value="#Trim(PrefAddr)#">
----- still more params to be passed----
</cfstoredproc>
</cfloop>
Does ColdFusion has a better technique to avoid either time out error or 500 error?
Like another poster mentioned, reducing the number of database calls should be a priority to you. I suggest joining data (if possible) in your first query rather than looping your query and querying again.
To fix the time issue, you can put requestTimeout in your page to override the default timeout. The time is in seconds.
<cfsetting requestTimeOut = "600">
See this explanation.
Your current approach is to make 1 query, which contains n records. Then loop over that record set, calling a query for each record. This results in your calling n + 1 queries per request. As the volume of data returned by your first query increases, so too does the volume of overall queries made to the database. In your case, you're trying to make 20,001 calls to the database in a single request.
Ideally, you need a solution that involves one call to a stored procedure with a properly optimized query that can return all of your data in a single record set. If you don't need to dump all the data onto the page at the same time, then create a paginated query that will return x number of records per page. Then the user can go page by page through the query or provide a search form with additional filters to allow the user to reduce the overall size of the records returned.
Need to see full queries to give you an example, but in general, this is a really useful thing in CF to help you out!
You need to look into the attribute called group. This lets you specify a column to group your output by. This actually will eliminate the need for the stored proc you are calling entirely.
The way it works is letting you create sub-set outputs based on grouping. So for example, you could do this with your id's and output that group.
You can also have multiple of these and have header and footer sections for each one for display purposes or just logic manipulation.
This lets you query the entire dataset and then manipulate it in the loop without having subqueries which is ultra inefficient and cringe worthy.
This is something quite unique to ColdFusion, check it out!
http://help.adobe.com/en_US/ColdFusion/9.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7ff6.html
I've got query that is returning all translations for a site. It does this by getting all translations that are in the users desired language, then the remaining that are in the site default language, then any other strings that have not been translated. I'm using cachedwithin on that query since the data doesn't change often, and I'm resetting that queries cache if translations are modified. I'm then using ColdFusion's Query of Query to get the individual record that I'm after. This has increased performance considerably.
I was wondering if it's possible to further cache the Query of Query query to further increase performance. It appears to work as page load is 1/6 faster, however are there any gotchas with this technique?
The Query of Query is below.
<cfquery name="qryTranslation" dbtype="query">
SELECT
TranslationString
FROM
qryGetText
WHERE
TranslationHash = <cfqueryparam value="#StringHash#" cfsqltype="cf_sql_varchar">
AND DesiredLanguageID = <cfqueryparam value="#Arguments.LanguageID#" cfsqltype="cf_sql_bigint">
</cfquery>
Is it possible to cache a ColdFusion Query of Query
Yes, it is possible.
however are there any gotchas with this technique?
You queries will be cached based on its signature, so in your case the StringHash and Arguments.LanguageID. If you have a cached QofQ for every translation on a page, on many pages on your site, then you could potentially max out the "Maximum number of cached queries" value. If this happens other, potentially larger and more important, cached queries in the query cache could be evicted.
Calculating a suitable "Maximum number of cached queries" could be determined by load testing and using the build in server monitor to monitor the number of queries in the cache.
There is one big gotcha with caching a query of query.
The documentation for caching a query states that:
To use cached data, the current query must use the same SQL statement, data source, query name, user name, and password.
However a Query of Query does not have a data source, user name or password, so you lose a lot of "over cache" protection. The query as it stands in your question will conflict with any other queries on your server that have the same name and formatting. So if you have more than one website that uses this code then the first website that is loaded will dictate the translations used on the rest of the websites.
A quick way around this is to trick the query into being more constrained.
<cfquery name="qryTranslation" dbtype="query">
SELECT
TranslationString
FROM
qryGetText
WHERE
TranslationHash = <cfqueryparam value="#StringHash#" cfsqltype="cf_sql_varchar">
AND DesiredLanguageID = <cfqueryparam value="#Arguments.LanguageID#" cfsqltype="cf_sql_bigint">
AND '#Variables.DSN#' = '#Variables.DSN#'
</cfquery>
Change Variables.DSN to be the value of the datasource attribute in the main query. If you don't trust that variable, then also make it a cfqueryparam on both sides of the operator.
Given a query (pseudo-code):
<cfquery name="myquery">SELECT * FROM stuff</cfquery>
How do I get rid of the first record? In this case, altering the SQL is not an option.
I have tried: myquery.RemoveRows(0,1); but received an error:
No matching Method/Function for Query.REMOVEROWS(numeric, numeric) found
I'm on Railo 3 BTW
Lo and behold:
myquery.RemoveRow(1);
Does exactly what I wanted. Leave it to Railo to do things a little differently!
Can't think of a way offhand to remove a row from the original object. Two things I can think of are:
do a query of queries. That assumes you'd be able to identify the records you don't want and specify them in the WHERE.
construct a new query with queryNew(). Loop over the original query doing a querySetCell() into the new query for the records that you want. This functionality could be incorporated into a UDF pretty easily. In fact, stating that made me think to check cflib.org. See #3
Check cflib.org :) See http://www.cflib.org/udf/QueryDeleteRows
"RemoveRows" is actually a call to the underlying Java method, so you have to cast those numbers. like so:
myquery.RemoveRows(
JavaCast( "int", 0 ) // starting row, zero-based
,JavaCast( "int", 1 ) // number to delete, returns error if too many
)
So probably the method is "int,int", and if you don't cast it, it looks like "numeric,numeric". One might argue that this is undocumented, but it's so succinct that you could (as I did) wrap it in a function, so that if it changes you just need to replace the contents of the function with an alternative workaround.
Railo has added removeRows, see here. My ACF code that uses this method now runs on Railo too, no changes.
With this, the Railo implementation now matches ACF. (Note also that the original Railo implementation was 0 based, while the new one and the ACF version are both 1 based.)
The way I would normally do it is with a query of queries such as the following:
SELECT * FROM myquery
LIMIT {1,10000}
That should work in Railo. What it does is offset the query by one and pulls 10000 records.
You could also do this:
SELECT * FROM myquery
WHERE {primarykey} <> {value}
Where it selects all records except for the primary key value that you pass in.
The awesome thing about ColdFusion is that there are tons of ways to do exactly what you are looking for.
You could just skip the row when you process the results:
<cfoutput query="myquery">
<cfif myquery.currentrow GT 1>
<!---Do the work here. --->
</cfif>
</cfoutput>
I would like to add some code to my Application.cfc onRequestEnd function that, if a certain application variable flag is on, will log query sql and execution time to a database table. That part is relatively easy, since ColdFusion returns the sql and execution time as part of the query struct.
However, this site has probably close to 1000 pages, and modifying all of them just isn't realistic. So I'd like to do this completely programmatically in the onRequestEnd function. In order to do that I need to somehow get a list of all queries that have executed on the page and that's where I'm stumped.
How can I get a list of the names of all queries that have executed on the current page? These queries appear in the template's variables scope, but there are a myriad of other variables in there too and I'm not sure how to easily loop through that and determine which is a query.
Any help would be appreciated.
Since that information is available via the debugging templates, you might take a look at those files for some pointers.
Another thing to consider is encapsulating your queries in a CFC or custom tag and having that deal with the logging (but I suspect that your queries are spread all over the site so that might be a lot of pages to modify - although that speaks to why encapsulating data access is a good idea: it's easier to maintain and enhance for exactly this sort of situation).
The relevant code from the debug templates (modernized a bit), is:
<cfset tempFactory = createObject("java", "coldfusion.server.ServiceFactory") />
<cfset tempCfdebugger = tempFactory.getDebuggingService() />
<cfset qEvents = tempCfdebugger.getDebugger().getData() />
<cfquery dbType="query" name="qdeb">
SELECT *, (endTime - startTime) AS executionTime
FROM qEvents WHERE type = 'SqlQuery'
</cfquery>