When you add a debug IP to review the debug information from the collection of templates that are parsed to present the page, it includes a list of all queries from that page.
Is it possible to get access to that object for inspection?
I'm looking at trying to automate the documentation which data sources are in use for which page requests. We have a large number of small web apps that access different databases and have different dependencies. I'm hoping to find a way to automate the documentation of these dependencies rather than having to manually review all code for all of the webapps.
Not sure if the object doesn't get created until after the page creation is too far gone to actually do anything with the data, but who knows...
Here is a snippet of code that you can add to the end of your template to get a list of datasources used on the page:
<cfobject action="CREATE" type="JAVA" class="coldfusion.server.ServiceFactory" name="factory">
<cfset cfdebugger = factory.getDebuggingService()>
<cfset qEvents = cfdebugger.getDebugger().getData()>
<cftry>
<cfquery dbtype="query" name="cfdebug_qryDSN">
SELECT DISTINCT DATASOURCE FROM qEvents WHERE type = 'SqlQuery'
</cfquery>
<cfcatch type="Any">
<cfset cfdebug_qryDSN = queryNew('DATASOURCE')>
</cfcatch>
</cftry>
<cfdump var="#cfdebug_qryDSN#" label="cfdebug_qryDSN">
PS: most of the inspiration for this snippet came from {cfusion 10 home}\cfusion\wwwroot\WEB-INF\debug\classic.cfm. You can get some good ideas on how to gain access to debugger objects/data from this file.
For anyone stumbling across this....
If your [cfroot]/cfusion/lib/neo-datasource.xml file is WDDX encoded and you're not sandboxed, you can use the following (tested on CF2021)
<cflock type="readonly" scope="Server" timeout="5">
<CFSET LibPath=Server.System.Properties["coldfusion.libPath"]>
</cflock>
<CFFILE action="Read" file="#LibPath#/neo-datasource.xml" variable="DatasourcesWDDX">
<cfwddx action="wddx2cfml" input="#DatasourcesWDDX#" output="Datasources">
<cfoutput>#StructKeyList(Datasources[1])#</cfoutput>
<cfdump var=#Datasources#>
The first position of the Datasources array holds a structure containing information on each configured datasource with the main key being the name of the datasource.
Here's an idea that'll work for each application which uses an Application.cfc.
Enable Request Debugging Output in CF Administrator.
Configure Debugging IP Addresses so that every page receives debugging information.
Assuming that Select Debugging Output Format is set to classic.cfm, short circuit {cfusion 10 home}\cfusion\wwwroot\WEB-INF\debug\classic.cfm by making <cfreturn> the first executable statement in classic.cfm. This will prevent any pages from seeing the debug output.
In Application.cfc::OnRequestEnd() do what Scott Jibben suggested. You can wrap Scott's idea in an <cfif IsDebugMode()>.
Related
for a long period of time I have an issue with Session variables. I'm looking for some pointers or directions to address this problem. I try to explain what is happening and I understand that the information provided isn't sufficient to understand what is happening, but I'm trying to solve this and it's driving me nuts :-)
I have several Lucee webapplications (Lucee: 5.3.7.48) which are also available as Cordova app. Error messages are sent to my by e-mail. On a daily bases I receive about 100 messages that the session variable can not be found.
[APP] is a session variable, set in the file index.cfm.
<cfparam name="SESSION.auth.app" default="">
In line 592 there is something like
<cfif session.auth.app is 1>do something</cfif>
I don't want to focus on line 592, the real problem is that the session variable is/gets lost. What happens next is that the user is redirect to the login page (login.cfm), because the session is lost and then the problem repeats, it's a kind of loop. The application has a save username/password option.
This all happens when the application is running in the background, like the Cordova app running in the background. I know this because I reached out to a user when receiving 20 error messages, and he told me that I wasn't using the application at the time of the error messages.
In conclusion I measure a user when he is using the login.cfm page. As you van see in the image there is a peak in usage on March 16th, which has to do with the problem described.
I understand that I provided not much useful information to go on, but can someone give me some directions how to approach this problem?
UPDATE April 14th
I have changed the way session vars are set and followed the suggestions in the reactions.
In the application.cfc I have add:
<cffunction name="onSessionStart" access="public" returntype="void" output="false" hint="I fire when a new session begins.">
<cfset SessionRotate()>
<cfset SESSION.app = ''>
<cfset SESSION.device = ''>
<cfset SESSION.app_file_url = 'window.open'>
</cffunction>
Restarted Lucee just the be sure. But still the same error messages, APP doesn't exist. After some additional searching I found: https://www.bennadel.com/blog/1535-coldfusion-session-is-always-created-even-if-onsessionstart-fails.htm
Could this be a Lucee bug? Are do you have more suggestions to try?
In your onSessionStart function, Can you please change it to this?
<cfset session.auth = {}>
<cfset session.auth.app = ''>
I see you are checking <cfif session.auth.app is 1>do something</cfif> which should have the session.auth defined for it to work.
I hope the above helps.
Also, I'd be interested in knowing what are you dumping on the screen?
I'd like to initiate some logging of requests and, more importantly, queries within my application. Lucee makes this pretty easy, and I'm able to log all queries within any given page run with a few lines of code within the onRequestEnd function:
<cfset getQueries = getPageContext().variablesScope()>
<cfset queryArr = ArrayNew(2)>
<cfset x = 0>
<cfloop collection="#getQueries#" item="key">
<cfif IsQuery(getQueries[key])>
<cftry>
<cfset thisQ = getQueries[key]>
<cfset thisQT = thisQ.getExecutionTime() / 1000000>
<cfset thisSQL = thisQ.getSQL().getSQLString()>
<cfset x = x + 1>
<cfset queryArr[x][1] = thisQT>
<cfset queryArr[x][2] = thisSQL>
<cfcatch>
</cfcatch>
</cftry>
</cfif>
</cfloop>
This gives me an array with each query's SQL (with bind variables represented with ?) and the execution time. I can insert this into a logging database and have all kinds of fun with it.
The problem is that I have many pages that call CFCs, which run queries and then return data:
<cfset someVar = createObject("component","cfc.test").getSomeData(ID=7)>
After I wrote my logging code I realized that the queries within CFCs (the bulk of my data processing) were not being recorded as, of course, they run in their own scope. I'm looking for an easy way to record queries within CFCs as well.
I had never heard of onCFCRequest before a few minutes ago (when Google enlightened me), and I cannot find much information on its use. I was hoping it may work like onRequest, and I could do something like this:
<cffunction name="onCFCRequest">
<cfargument type="String" name="cfcName" required=true/>
<cfargument type="String" name="methodName" required=true/>
<cfargument type="struct" name="args" required=true/>
<cfset caller.getLog = getLog>
</cffunction>
However, that does not work, nor do any of the other variations I tried. In fact, this function doesn't appear to work at all, as this code did not insert anything into my test table:
<cffunction name="onCFCRequest">
<cfargument type="String" name="cfcName" required=true/>
<cfargument type="String" name="methodName" required=true/>
<cfargument type="struct" name="args" required=true/>
<cfquery>
insert into testTable (f1, f2)
values (<cfqueryparam cfsqltype="cf_sql_varchar" value="test1">,
<cfqueryparam cfsqltype="cf_sql_varchar" value="test2">)
</cfquery>
</cffunction>
What is the proper way to use onCFCRequest, and can it be used to write data from a cfc's local scope to the parent/caller request scope? If not, is there another function (like an onCFCRequestEnd) that can be used to do so? I do not want to have to make largescale changes to every cfQuery or cfc file in my application to accomplish this goal.
I am aware there are some commercial applications designed to do this far better that I can with just a few hours of coding. However, this is for a personal site that I can't afford to spend money on, and the experience of doing it myself is enlightening.
As your actual question is how to log all query executions no matter where they happened, I think there are two approaches to do that.
Enabling Logging
Whenever a query is executed, you can write the SQL string and the parameters into a log.
You can set up the logging to a database table of your wish within the Lucee Admin.
Once that's set up, you need to call <cflog> and use the name of the specified log within the file attribute.
Downside to this approach is that you have to manually place a <cflog> whenever you execute a query. This might be mitigated by centralizing all database communication in a function which also does the logging. So you whenever you want to query something, you'd call that function instead of calling <cfquery> manually.
Enabling debugging
Lucee's debugging functionality allows you to output all queries executed within a page run besides other useful information.
You first need to add a debugging template:
Important note: The predefined templates only output the data to the generated HTML of your website. Though you have the possibility to create your own debugging template. In order to do that you need to create your own template CFC within lucee-server/context/context/admin/debug/ and provide it with a unique name. There's also a slide show of how to create your own debugging template of the head behind Lucee, Gert Franz, which explains in more detail how to do that.
When done correctly, you should see the name of your template listed in the template types drop-down list.
Then, all you need to do is to enable debugging of database activity:
I've a simple CFC file that contains the different functions for different queries & a separate function that displays the reports dynamically based on the queries.
All the queries work except one which returns approx. 50k rows. Its just a blank screen & I get no error. When I dump the query results, they do get dumped on the screen but while displaying it in a tabular report it gives nothing.
I've a another CFM file that returns 100k rows & works fine.
Below is CFC code that is not working.
<cfcomponent>
<cfparam name="qry1" default="">
<cffunction name="showqry1" access="remote">
<cfquery name="qry1" dataSource="myds" cachedwithin="#CreateTimeSpan(0, 2, 0, 0)#">
<!--- myquery --->
</cfquery>
<cfset Display()>
</cffunction>
<cffunction name="showqry2" access="remote">
<cfquery name="qry1" dataSource="myds" cachedwithin="#CreateTimeSpan(0, 2, 0, 0)#">
<!--- myquery --->
</cfquery>
<cfset Display()>
</cffunction>
<cffunction name="Display" access="private">
<cfdump var="#rptQry#" top="20">
<cfsavecontent variable="myrpt">
<table>
<!--- make a tabular report here using cfloop over the query--->
</table>
</cfsavecontent>
<cfform action="test.cfm" method="post" name="ExcelData">
<cfoutput>#myrpt#</cfoutput>
<cfinput type="hidden" name="excel_data" value="#myrpt#"/><!---This is giving the error. --->
<cfinput type="submit" name="test" value="Export" />
</cfform>
</cffunction>
</cfcomponent>
Any idea why CFM works fine but CFC doesn't? I need my CFC to work & dont want it to convert it to CFM...
UPDATE:
I've added a comment("This is giving the error") in the above code that is cause of the error. Irrespective of CFC/CFM this doesn't work.
I use the hidden field to pass data to another file which exports data to excel. Any alternate suggestions??
Any help is highly appreciated.
Thanks
You still need to read that doc I put in the comment about how to ask questions clearly.
However you are putting your recordset into a variable qry1, but trying to dump a variable rptQry. But that would just error, unless there's some code you're not showing us that populates rptQry.
Also, from a coding practice POV, you shouldn't really be outputting stuff in your functions: that's best done in a CFM page. Get your data with a CFC method; display it with a CFM.
I also recommend you read up on how to do OO with CFML (or in general). Perhaps get Matt Gifford's book "Object-Oriented Programming in ColdFusion"
Your Display function has cfsavecontent with tabular data and you are putting it into a cfform inside a cfc. I don't know why you are doing that. Insted, simply do an ajax call which return that cfsavecontent and then show it in the cfm.
Else, I guess you may have to output the cfform in the Display function. I may be wrong, but I don't think you can simply place a cfform inside a cfc and expect it to show up on the browser. CFC is not for browser rendering, it should be in a cfm.
Regarding the comment, "This is because I need to export to excel on click of a button for which I'd need cfform. Can you suggest some alternate to this functionality?", I will give you some things to think about.
First, you talk about recordsets containing several thousand rows and you have code where you attempt to display that in a browser. Quite simply, that will take an enormous amount of time to render. So, it's a bad idea.
Next, your code has functions for various queries but just one display function. Unless that's a cleverly written function that figures out the column names, it will only work if all the queries have the same columns. If that's the case, maybe you only need one query and some variables.
My suggestion is to start with a form where the user sends the appropriate information which determines what sql gets written. This form should also include a way for them to choose whether they want the results rendered in excel or html. If they choose html, do something to ensure that the data being returned does not overwhelm their browser.
By the way, re-useable code for displaying query results is a good idea. However, a custom tag might be a more conventional way to do it.
I have a ColdFusion app in which I wish to restrict access to certain pages, based on some criteria. I am currently doing it like this, in Application.cfc:
<cffunction name="OnRequestStart" access="public" returntype="boolean" output="true">
<cfargument name="TargetPage" type="string" required="true" />
<cfif not SESSION.isAdmin and REFindNoCase("/admin",ARGUMENTS.TargetPage) >
<!--- Deny non-admin access to admin pages. --->
<cfinclude template="/notauth.cfm">
<cfreturn false />
</cfif>
<cfreturn true />
</cffunction>
My main concern is: How vulnerable is the general approach of checking TargetPage against a regex, and are there ways to improve the security of this design? Specifically, I'm concerned about avoiding "canonical representation vulnerabilities." See here.
For example, using just a REFind instead of REFindNoCase would let people slide right on through if they went to "/ADMIN/". Are there are other things to watch out for here?
I know there are other designs, like using another Application.cfc in a subfolder, or doing checks right in the page code. But I like the idea of having all my security code in one place. So please only suggest those in your answer if there's no way to do the above securely, or if it's just really a bad idea for some reason. Thanks.
I'm sure there are reams of this stuff on the internets but here is my take on it :)
They way I would solve your specific example is to maintain a database list of scripts that are restricted (a blacklist) unless you are a member of a certain group (i.e. you are an admin).
You can make this as complicated as you wish but for a simple start you could compare the full script name (CGI.SCRIPT_NAME) to a query of queries representing blacklisted pages you store in the APPLICATION scope that you loaded in onApplicationStart() called qRestrictedList.
So in onRequestStart you could do the following:
<cfquery name="qThisPageRestricted" dbtype="query">
SELECT * FROM qRestrictedList
WHERE ScriptName = '#CGI.SCRIPT_NAME#'
</cfquery>
<cfif qThisPageRestricted.recordCount and not SESSION.isAdmin>
<cfinclude template="/notauth.cfm">
<cfreturn false />
</cfif>
Even better, you can expand on this at a later date by wrapping all this in a 'authentication' CFC and creating user groups and levels, i.e. move your logic out of onRequestStart() and encapsulate it.
But as a start, storing the data in the database might be a more maintainable way for you to get this done and provide a better foundation for future changes to how your authentication works.
I hope this helps.
It may worth to make regex a bit stricter:
REFindNoCase("\/admin\/([A-Za-z_]+)\.cfm", ARGUMENTS.thePage)
A better approach would be to put an application.cfc in the /admin directory that controls access (maybe based on a SESSION variable set through logging in as an admin), and have that "child" application.cfc reference the parent one if necessary.
See this question for an example on how to do this: Extending application.cfc in a subdirectory
I am trying to store coldfusion code in a database to be used for the subject of a cfmail. The code stored is as follows:
"RE: <cfif myData.general.legalName NEQ """"> {{dotlegalname}}<cfelse>{{docketLegalName}}</cfif>,
DOT## {{dot}}, Docket ##(s) {{docketString}}"
When I retrieve string from the database, I use cfsavecontent to attempt to evaluate it.
<cfsavecontent variable="subject">
<cfoutput>#myData.email.subject#</cfoutput>
</cfsavecontent>
I also tried
<cfsavecontent variable="subject">
<cfoutput>#evaluate(myData.email.subject)#</cfoutput>
</cfsavecontent>
And then I replace all the {{ }} with the appropriate values.
However, the subject of the email is stubbornly refusing to contain an evaluated cfif, and is instead showing the cfif as if it were a string.
Any ideas?
The only way to dynamically evaluate code that you are creating at runtime is via writing it out to a file, and then executing it.
The easiest way would be to write it a .cfm page in the Virtual File System (probably name the file after a UUID, so it's unique), and then it where you need to run the contents.
I wouldn't normally advocate generating code at runtime like this, but it can be the most elegant solution in some cases.
As an alternative, instead of storing the CFML code in the database, you have a set of CFML email template files that get stored in a directory on your server, and in your database you simply record which template needs to be included either via cfinclude or cfmodule.
You can't dynamically evaluate CFML stored in a database without first writing it to file and then using <cfinclude> to include it.
Further to Mark's answer here is some psuedo code:
<cfset fileName = createUUID() & ".cfm">
<cfset fileWrite( fileName, [CODE_FROM_DB]>
<cfinclude template="#fileName#">
<cfset fileDelete( fileName )>
I have used code like this before with no problems. Anything in the Virtual File System flies as it is all run in RAM. For best practice do remember to delete the files created ;)
If you absolutely have to do this, look at the evaluate() function. This, essentially, fires up a new CF thread, compiles the string passed to it, runs it, and returns the result.
If at all possible, I would try to find a way to move your logic to the actual file being run, not the string from the database. I assume you are pulling the data based on some string you've already built, so you might consider appending something to it, so you are looking up subjectDotLegal and subjectDocketLegal or something similar.
Remember, evaluate() is slow, ugly, and can be dangerous (it will run anything passed to it!). If there's a way around it, I suggest you use it.
why not just use something like mustache?
http://mustache.github.com/
https://github.com/pmcelhaney/Mustache.cfc
it has the ability to not only do some of the logic that you want in your script dynamically. i really would suggest you check out the project and maybe even improve and contribute on it.
OH and just for the chance to be on a soapbox: I've been emailing Adobe for years saying that we need the ability to dynamically parse and render CFML. Sadly my cries have only gotten ignored. maybe if more people complained that this feature needs to be added, it would get the attention it deserves.
To give an example: Assume code.txt is a text file that contains the following (just to facilitate simulating CFML stored in a db): <cfoutput>#now()#</cfoutput>
The following code would work:
<cfset q = queryNew("code") />
<cfset queryAddRow(q,1) />
<cfset querySetCell(q, "code", fileRead(expandPath('code.txt')), 1) />
<cfdump var="#q#">
<cfset newCodeFile = expandPath('dynamic.cfm') />
<cfset fileWrite(newCodeFile, q.code[1]) />
<cfinclude template="dynamic.cfm" />
In OpenBlueDragon there is the render function, which can do this.
You can mimic this function in Railo by creating a custom built-in function that saves the file into RAM then cfincludes it, using the following code:
<cffunction name="render" output="Yes" returntype="string"><!---
---><cfargument name="Code" required="Yes" type="string"><!---
---><cfset local.mapping = {'/render_ram_resource':'ram://'}><!---
---><cfapplication action="update" mappings="#local.mapping#"><!---
---><cfset local.fileName = "/render_ram_resource/_render_" &
createUUID() & ".cfm"><!---
---><cffile action="WRITE" file="#fileName#"
output="#arguments.Code#"><!---
---><cfinclude template="#fileName#"><!---
---><cffile action="DELETE" file="#fileName#"><!---
---></cffunction>
(This looks unusual because it needs to allow output, but prevent extra whitespace, hence why all the comments. Unfortunately SO's syntax highlighting seems to be confused by them.)
If you need an ACF-compatible solution, you'll need to use the regular filesystem and a pre-created mapping. (Well, in ACF9 and above you can use the RAM virtual filesystem, but afaik you can't create mappings on the fly like this.)
There's a better way, namely using in memory files. This way you don't have any I/O on the disk and therefore much faster:
For tags that take logical path, define mapping in Administrator. Execute in-memory CFM pages using the cfinclude tag:
Create a mapping for ram:/// so that it can be used in the tags. In this example, /inmemory is the mapping that points to ram:///.
For tags that take absolute path, specify the syntax as provided in the following example:
You can also delete the file from the ram usinf cffile and action delete.
Here's how I stored my header and footers for all pages in a record. This code can go at the top of each page. But I have it in the APPLICATION.cfm and it seems to be working great.
The key here is not use #pound# signs on your expressions. User [square braces]. The code will pick them and evaluate them and return the result back to the template.
It will substitute the number 0 if it can not evaluate an expression as a means of error handling.
<CFSET FooterID=1234> <!-- ID of the record you want to use -->
<CFQUERY NAME="StoredHeader" Datasource="DS1">
Select Body from templates where id=#FooterID#
</CFQUERY>
<CFSET Parse=StoredHeader.Body>
<CFLOOP CONDITION="FindNoCase('[',Parse,1) GT 0">
<CFSET STB=FindNoCase('[',Parse,1)>
<CFSET ENB=FindNoCase(']',Parse,1)>
<CFIF ENB-STB GT 0>
<CFSET BracketExp=Mid(Parse,STB+1,ENB-1-STB)>
<CFTRY>
<CFSET BracketValue=Evaluate(BracketExp)>
<CFSET Parse=ReplaceNoCase(Parse,'['&BracketExp&']',Evaluate(#BracketExp#))>
<cfcatch type="any">
<div>'Using ZERO 0 for missing <cfoutput>#BracketExp#' </cfoutput> </div>
<CFSET Parse=ReplaceNoCase(Parse,'['&BracketExp&']','0')>
</cfcatch>
</CFTRY>
</CFIF>
</CFLOOP>
<CFSET Footer=Parse>
<cfoutput>FOOTER</cfoutput>
I would try the built-in QuoteName function.