I have a very large query which is getting called from 3 different pages.
Instead of writing the same query in all the 3 cfm files, I am trying to find an alternative way to save the query (alongwith #variable(s)#) in a Query.cfm file.
Query.cfm example :
SELECT *
FROM A
WHERE TRADE_DATE BETWEEN to_date('#f_startDate#','dd/mm/yyyy') AND to_date('#f_endDate#','dd/mm/yyyy')
variables : #f_startDate# and #f_endDate#
Then I read the filecontents, store it in a variable and replace the #variable(s)# with the values to run the function from each of the pages.
Calling page (code so far which is not working):
<cffile action = "read" file = "#ExpandPath( './Query.cfm')#" variable = "Query">
<cfset Query = #ReplaceList(Query,"#f_startDate#,#f_endDate#", "01/01/2000,01/01/2002")#>
<cfquery name="Q_DailyPrice" datasource="#f_datasource#">
#PreserveSingleQuotes(Query)#
</cfquery>
How to set the variable values into the string?
Further details about each page:
Returns the JSON of the query to load charts
Used to generate query data in xls
Used further to generate subset of the query data (QoQ) to create a table.
Database : Oracle
Your options include:
Put the query in a .cfm template and access it with cfinclude
Put the query into a user defined function in a .cfm file. Then cfinclude the file and call the function.
Put the query into a user defined function in a .cfc file. Then you can either run the function with cfinvoke, or create an object and then call the function.
There are probably other options as well. I suggest looking at the three I suggested and determine which one best meets your needs.
Whatever method you use, ColdFusion has a parsedatetime function that will convert your strings to date objects. Using these might be faster that oracle's to_date function. You'll have to test and see. In any event, use cfqueryparameter for a variety of reasons.
Also, be careful about using between with oracle. Its date fields include a time component. If any of your records have one, you are safer with
where trade_date >= YourStartDate
and trade_date < TheDayAfterYourEndDate
Related
I have several fields that contain exactly the same sql query! Is it possible to place the sql question centrally in APEX in the same way as list of values or as a function in oracle? I am using APEX 18.2
Here are two extended solutions
Pipelined SQL
https://smart4solutions.nl/blog/on-apex-lovs-and-how-to-define-their-queries/
Dynamic SQL
http://stevenfeuersteinonplsql.blogspot.com/2017/01/learn-to-hate-repetition-lesson-from.html
Call me dense, but I don't think I understand why you'd have multiple fields (presumably on the same form) whose source is the same SQL query.
Are you passing a parameter to the SQL to get a different value for each field?
If you are passing a parameter to the SQL query, why not create a database view to hold the query, then pass the parameter to the view. That way, if you need to change it, it's in one place.
If they really are all the same value from the same query, how about using the SQL for one field/page_item, then make the source for the others be the first page item?
I would create a hidden item with the query in the source as text and use &HIDDEN_ITEM_NAME. to reference its value in the source of any item I was trying to display the query.
Finally isolved it with a function and use the type PL/SQL Function Body returning SQL Query in APEX, then i have all in one place. I created a function in SQL developer that returns a SQL query.
I've been given a project that uses AngularJS and ColdFusion as a Service. I understand Angular but I've never worked with ColdFusion before. Within the CFFunction Tag in a ColdFusionComponent I have some complex SQL that is being generated. In addition to the actual data being returned from the Service I would like to have the service return the actual text of the SQL executed. Can someone tell me how this can be done?
From the comments
In order to get the SQL statement that was executed you can use the result attribute of the <cfquery> tag. When you include that attribute then ColdFusion will return more information about the query including the SQL statement that was executed. See the docs here under the "usage" section (about midway down the page) for more information.
From the referenced documentation:
The cfquery tag also returns the following result variables in a structure. You can access these variables with a prefix of the name you specified in the result attribute. For example, if you assign the name myResult to the result attribute, you would retrieve the name of the SQL statement that was executed by accessing #myResult.sql#. The result attribute provides a way for functions or CFCs that are called from multiple pages, possibly at the same time, to avoid overwriting results of one call with another. The result variable of INSERT queries contains a key-value pair that is the automatically generated ID of the inserted row; this is available only for databases that support this feature. If more than one record was inserted, the value can be a list of IDs. The key name is database-specific.
Variable name Description
result_name.sql The SQL statement that was executed.
result_name.recordcount Number of records (rows) returned from the query.
result_name.cached True if the query was cached; False otherwise.
result_name.sqlparameters An ordered Array of cfqueryparam values.
result_name.columnList Comma-separated list of the query columns.
result_name.ExecutionTime Cumulative time required to process the query.
result_name.IDENTITYCOL SQL Server only. The ID of an inserted row.
result_name.ROWID Oracle only. The ID of an inserted row. This is not the
primary key of the row, although you can retrieve rows
based on this ID.
result_name.SYB_IDENTITY Sybase only. The ID of an inserted row.
result_name.SERIAL_COL Informix only. The ID of an inserted row.
result_name.GENERATED_KEY MySQL only. The ID of an inserted row. MySQL 3 does not
support this feature.
result_name.GENERATEDKEY Supports all databases. The ID of an inserted row.
I have two tables in different databases. In a table A is the data, in the other table B are information for incremental load of the data from the first table. I want to load from table B and store the date of the last successful load from table A in a mapping variable $$LOAD_DATE. To achieve this, I read a date from table B and use the SETVARIABLE() function in a expression to set the $$LOAD_DATE variable. The port in which I do this is marked as output and writes into a dummy flat file. I only read on row of this source!
Then I use this $$LOAD_DATE variable in the Source Filter of the Source Qualifier of table A to only load new records which are younger than the date stored in the $$LOAD_DATE variable.
My problem is that I am not able to set the $$LOAD_DATE variable correctly. It is always the date 1753-1-1-00.00.00, which is the default value for mapping variables of the type date/time.
How do I solve this? How can I store a date in that variable and use it later in a Source Qualifiers source filter? Is it even possible?
EDIT: Table A has too much records to read them all and filter them later. This would be to expensive, so they have to be filtered at source filter level.
Yes, it's possible.
In the first map you have to initialize the variable, like this:
In first session configuration you have to define the Post-session on success variable assignment:
The second map (with your table A) will get the variable after this configuration of the session in Pre-session variable assignment:
It will work.
It is not possible to set a mapping variable and use it's value somewhere else in the same run, because, the variable is actually set when the session completes.
If you really want to implement it using mapping variables you have to create two mappings, one for setting the mapping variable and another for actual incremental load. You can pass a mapping variable value from one session to another in a workflow using a workflow variable. https://stackoverflow.com/a/26849639/2626813
Other solutions could be to use a lookup on B and a filter after that.
You can also write some scripts to query table B and modify the parameter file with the latest $LOAD_DATE value prior to executing the mapping.
Since we're having two different DBs, use two sessions. Get values in the first one and pass the parameters to the second one.
I am having an array of structure. I need to insert all the rows from that array to a table.
So I have simply used cfquery inside cfloop to insert into the database.
Some people suggested me not to use cfquery inside cfloop as each time it will make a new connection to the database.
But in my case Is there any way I can do this without using cfloop inside cfquery?
Its not so much about maintaining connections as hitting the server with 'n' requests to insert or update data for every iteration in the cfloop. This will seem ok with a test of a few records, but then when you throw it into production and your client pushes your application to look around a couple of hundred rows then you're going to hit the database server a couple of hundred times as well.
As Scott suggests you should see about looping around to build a single query rather than the multiple hits to the database. Looping around inside the cfquery has the benefit that you can use cfqueryparam, but if you can trust the data ie. it has already been sanatised, you might find it easier to use something like cfsavecontent to build up your query and output the string inside the cfquery at the end.
I have used both the query inside loop and loop inside query method. While having the loop inside the query is theoretically faster, it is not always the case. You have to try each method and see what works best in your situation.
Here is the syntax for loop inside query, using oracle for the sake of picking a database.
insert into table
(field1, field2, etc)
select null, null, etc
from dual
where 1 = 2
<cfloop>
union
select <cfqueryparam value="#value1#">
, <cfqueryparam value="#value2#">
etc
from dual
</cfloop>
Depending on the database, convert your array of structures to XML, then pass that as a single parameter to a stored procedure.
In the stored procedure, do an INSERT INTO SELECT, where the SELECT statement selects data from the XML packet. You could insert hundreds or thousands of records with a single INSERT statement this way.
Here's an example.
There is a limit to how many <CFQUERY><cfloop>... iterations you can do when using <cfqueryparam>. This is also vendor specific. If you do not know how many records you will be generating, it is best to remove <cfqueryparam>, if it is safe to do so. Make sure your data is coming from trusted sources & is sanitised. This approach can save huge amounts of processing time, because it is only make one call to the database server, unlike an outer loop.
I would like to add some code to my Application.cfc onRequestEnd function that, if a certain application variable flag is on, will log query sql and execution time to a database table. That part is relatively easy, since ColdFusion returns the sql and execution time as part of the query struct.
However, this site has probably close to 1000 pages, and modifying all of them just isn't realistic. So I'd like to do this completely programmatically in the onRequestEnd function. In order to do that I need to somehow get a list of all queries that have executed on the page and that's where I'm stumped.
How can I get a list of the names of all queries that have executed on the current page? These queries appear in the template's variables scope, but there are a myriad of other variables in there too and I'm not sure how to easily loop through that and determine which is a query.
Any help would be appreciated.
Since that information is available via the debugging templates, you might take a look at those files for some pointers.
Another thing to consider is encapsulating your queries in a CFC or custom tag and having that deal with the logging (but I suspect that your queries are spread all over the site so that might be a lot of pages to modify - although that speaks to why encapsulating data access is a good idea: it's easier to maintain and enhance for exactly this sort of situation).
The relevant code from the debug templates (modernized a bit), is:
<cfset tempFactory = createObject("java", "coldfusion.server.ServiceFactory") />
<cfset tempCfdebugger = tempFactory.getDebuggingService() />
<cfset qEvents = tempCfdebugger.getDebugger().getData() />
<cfquery dbType="query" name="qdeb">
SELECT *, (endTime - startTime) AS executionTime
FROM qEvents WHERE type = 'SqlQuery'
</cfquery>