Excessive recrawl of ColdFusion dynamic pages - coldfusion

The folks who use ColdFusion and serversideincludes are having issues with excessive recrawls on dynamic pages because there is no datelastmodfied set, which causes excessive server traffic. You can laugh if you want, but when I tell them the solution is setting a last modified date on the pages I get a universal huh? how do you do that? I opened a case with google originally and was told that yep, it's a page date problem. I have done a lot of research to try and find how to code this in the header and most of what I found talked about pulling a date from a page.
I did determine that it probably could be done using the CFHEADER tag. I'm just not sure about implementing.
Can I tell them that adding something like
<cfheader NAME="datelastmodified="Mon, 01 Feb 2013 08:00:00 GMT">
will suffice? Not sure about the date format, if the day name is required.
Have I tried just asking one of the webmasters to try this? No I haven't. I would like to know that I am at least on the right track before taking up too much of their time. And so far none of them have come up with a solution on their own other than useing robots.txt to block the crawl or things along those lines.
Any suggestions or thoughts would be appreciated.

Fortunately, none of these things need to be mysterious, as they're all well documented.
last-modified HTTP header
HTTP date/time formats
<cfheader>
and even a function to format the date correctly: getHttpTimeString()
This all comes together to suggest this sort of thing:
<cfheader name="Last-Modified" value="#getHttpTimeString(now())#"> <!--- although use some timestamp indicating when the content of the page was last updated,which would be a system-specific sort of thing --->
NB: I didn't know any of the specifics to this until I googled it about 5min ago.

Google's crawlers do tend to respect the meta tag details and HTTP response values for pages they encounter and the way to set such in CF is indeed with the CFHEADER tag. You'll want to craft it to look something like this:
<CFHEADER NAME="Last-Modified" VALUE="#DateFormat(now (), 'ddd, dd mmm yyyy')# #TimeFormat(now(), 'HH:mm:ss')# GMT#gmt#">
<CFHEADER NAME="Expires" VALUE="Mon, 10 Mar 2013 05:00:00 GMT">
You will likely want a CF dev to do that work as I'm showing you two examples for the datetime value there. The first one dynamically sets it to right now (using the DateFormat() and Now() functions) and the second example sets the Expires header value with a hard coded date.
You'll probably want to include both the last-modified and expires tags and decide whether you want the dates applied to each to be either dynamic or hard coded.

Related

opencart: I can edit order but cannot delete it. (with Error log)

I use opencart version 2.1.0.1
Everytime I click admin > sales > order, it will pop up "error undefined." By closing that popup window, I can still edit order but cannot delete order (no response).
In my log, there is:
PHP Notice: Undefined variable: order_id in
/var/www/html/opencart2101/system/storage/modification/admin/view/template/sale/order_list.tpl on line 821
The line 821 is:
url: 'index.php?route=extension/openbay/addorderinfo&token=<?php echo $token; ?>&order_id=<?php echo $order_id; ?>&status_id=' + status_id,
However, I haven't installed any openbay related module. Also, line 821 is inside <!-- --> mark. It should have no effect.
Help!
Although this is now an older version of opencart, I still see this being reported a lot around and about.
The problem occurs due to the store front adding the http url rather than the https url to the order. So firstly you need to fix that. If you dont want to read all of my explanation, you can just hit up the bold points :)
Either way BACKUP EVERYTHING actually not really, back up the file you are going to edit and backup your whole database.
open:
catalog/controller/checkout/confirm.php at around line 100
Find:
$order_data['store_url'] = HTTP_SERVER;
Change to:
$order_data['store_url'] = HTTPS_SERVER;
Now you will want to fix your database because for reasons I cannot fathom, the domain name is placed in the order along with the stores id. and when editing orders it is the usage of that directly within your admin order page that throws up the undefined notice. Basically the browser blocks the request because its trying to make an insecure request from a secure page.
Crack open phpmyadmin or whatever database tool you have on hand.
locate the table, default is oc_orders
Browsing the table, look for the column that contains your store url (i cant remember the name off hand, i think its just store_url but it will be obvious anyway. if you are multi store you will need to run the query for each
I am sure somebody can come up with a clever way to automatically convert just the http into https with a single use sql query on the one column, but this works for me.
Run SQL: adjust as appropriate
UPDATE `oc_orders` SET `store_url` = 'https://example.com' WHERE store_id = 0;

Can't delete Solr keys

Having trouble deleting keys from a Solr collection for files.
Updating the Solr collection with this:
<cfoutput query="fileQuery">
<cfset theFile = defaultpath & "#fileID#.pdf" />
<cfif fileExists(theFile)>
<cfindex
action="update"
collection="file_vault_solr"
type="file"
key="#theFile#"
title="#documentName#"
body="fileNumber,documentName"
custom1="/filevault/#filealias#"
custom2="#fileNumber#"
custom3="#documentName#"
>
</cfif>
</cfoutput>
However, when attempting to delete the key from the catalog it simply doesn't work. Here's the code being used to (try to) delete the keys:
<cfoutput query="deletedFile">
<cfset theFile = defaultpath & "#fileID#.pdf" />
<!--- Remove the deleted file from the collection. --->
<cfindex
collection="file_vault_solr"
type="file"
action="Delete"
key="#theFile#"
>
</cfoutput>
The key is not deleted, however. The only thing that has worked has been to purge the whole catalog and re-index all of the documents.
Any insights?
After a lot of debugging I found out.
The reason for this behavior is a very… uh… unfortunate uhm… "design decision" Adobe took when implementing the interface between ColdFusion and Solr.
So you have a Solr collection of indexed files and want to selectively purge the ones that do no longer exist on disk. I'm pretty sure that's the exact situation you've been in.
Let's assume:
there is a file called /path/to/file on your system and
it is indexed in the Solr collection foo.
When you issue a <cfindex collection="foo" action="delete" key="/path/to/file">, ColdFusion sends the following HTTP request to Solr:
POST /solr/foo/update?wt=xml&version=2.2 (application/xml; charset=UTF-8)
<delete><id>1247603285</id></delete>
This is a perfectly reasonable request that Solr will happily fulfill. The only strange thing is the number in the <id>. In any case, the file will be gone from the index after this operation.
Re-index the file and delete it from disk. Now:
there no longer is a file called /path/to/file on your system, but
it is still indexed in the Solr collection foo.
Let's do the same <cfindex action="delete"> operation again.
POST /solr/foo/update?wt=xml&version=2.2 (application/xml; charset=UTF-8)
<delete><id>/path/to/file</id></delete>
Huh? Shouldn't there be a number in the ID?
As it turns out, someone at Adobe thought it would be a jolly smart idea to use numbers for unique IDs of indexed files, to, uhhh, save space, I assume.
However for some inexplicable reason this only happens when the file in question still exists. If it does not exist anymore, ColdFusion will notice and pass the path instead.
Inspecting the number reveals that it would fit into a 32 bit signed integer value. (I've checked, there are plenty of negative values in the uid field of the collection.)
So this looks as if they use some kind of hashing algorithm that returns 32 bits and chuck that in a int. CRC32 springs to mind, but that's not it. Also, java.util.zip.CRC32 returns a long, so there wouldn't be any negative values in the first place.
The other readily available 32 bit hash in Java is ... java.lang.Object.hashCode().
Bingo.
"/path/to/file".hashCode() // -> 1247603285
So the solution is to never delete a file by its path, but always like this:
<cfindex collection="foo" action="delete" key="#path.hashCode()#">
For files that no longer exist this does the right thing.
More importantly: For files that still exist this does the right thing as well - ColdFusion would have sent the hash code anyway.
Until Adobe fixes this problem this is a safe and easy work-around.
Note that the file path is case sensitive and must match exactly with the one stored in the index.
A quick
<cfsearch collection="foo" name="foo">
without any criteria will return all index entries, so retrieving the exact path of orphaned entries it not a big problem.
Eric Lippert explains object hash codes and why it is a bad idea to use them for anything "practical" in an application It's a .NET article but applies to Java just as well.
It boils down to: Adobe should store the actual path in the Solr collection and leave the performance optimization they seem to have attempted to Solr.
I've filed Bug 3589991 against Adobe's ColdFusion bug database.
The key has to match exactly what is in Solr's index. So ensure that "defaultpath" is the same in both and check that the case matches as I believe Solr is case sensitive.
To debug this I would suggest that you add the status="myStatusVar" to the cfindex call . Then on both the add and delete to see what is going on. If the delete is not returning a Deleted Count. Then there is a Key mismatch.
<cfindex
collection="file_vault_solr"
type="file"
action="Delete"
key="#theFile#"
status="myStatusVar"
>

List of tags not available ColdFusion 9 script syntax?

I'm looking for a complete list of tags that are not available in ColdFusion 9 script syntax.
Example:
CFSetting: is one example that is available in Railo but not in CF9 for use in cfscript
CFDocument: I can't find this one so far.
Not an official list by any measure, but this is a list I presented to a private forum a while back, and it didn't receive too much correction (and those corrections have been integrated). It was in the context of what CF does and doesn't need to be implemented, to claim 100% coverage in CFScript.
Summary of omissions:
These ones are significant omissions:
<cfcollection>
<cfexchangecalendar>
<cfexchangeconnection>
<cfexchangecontact>
<cfexchangefilter>
<cfexchangemail>
<cfexchangetask>
<cfexecute>
<cfindex>
<cfinvoke> (support for dynamic method names)
<cflogin>
<cfloginuser>
<cflogout>
<cfmodule>
<cfoutput> (implementation of query looping with grouping)
<cfparam> (fix the bug in that enforced requiredness doesn’t work (ie: param name="foo";))
<cfsearch>
<cfsetting>
<cfwddx>
<cfzip>
<cfzipparam>
There’s a reasonable case for these ones to be implemented:
<cfassociate>
<cfcache>
<cfcontent>
<cfflush>
<cfhtmlhead>
<cfheader>
<cfntauthenticate>
<cfprint>
<cfschedule>
<cfsharepoint>
These ones... I’m ambivalent:
<cfgridupdate>
<cfinsert>
<cfobjectcache>
<cfregistry>
<cfreport>
<cfreportparam>
<cftimer>
<cfupdate>
We don’t need these ones at all, I think:
<cfajaximport>
<cfajaxproxy>
<cfapplet>
<cfcalendar>
<cfchart>
<cfchartdata>
<cfchartseries>
<cfcol>
<cfdiv>
<cfdocument>
<cfdocumentitem>
<cfdocumentsection>
<cffileupload>
<cfform>
<cfformgroup>
<cfformitem>
<cfgraph>
<cfgraphdata>
<cfgrid>
<cfgridcolumn>
<cfgridrow>
<cfinput>
<cflayout>
<cflayoutarea>
<cfmap>
<cfmapitem>
<cfmediaplayer>
<cfmenu>
<cfmenuitem>
<cfpod>
<cfpresentation>
<cfpresentationslide>
<cfpresenter>
<cfselect>
<cfsilent>
<cfslider>
<cfsprydataset>
<cftable>
<cftextarea>
<cftextinput>
<cftooltip>
<cftree>
<cftreeitem>
<cfwindow>
If there's anything here that you think ought to be included in CFScript, please raise an issue here - http://cfbugs.adobe.com/cfbugreport/flexbugui/cfbugtracker/main.html - and cross reference the issue number here.
HTH.
I would argue that there are no commands that are not available as script as you can extend and write the missing bits using cfc's.
Thus wrap your favourite missing <cftag in a cfc and call it using new
However, here is a list of what is supported
http://help.adobe.com/en_US/ColdFusion/9.0/Developing/WSe9cbe5cf462523a02805926a1237efcbfd5-7ffe.html

Cleansing string / input in Coldfusion 9

I have been working with Coldfusion 9 lately (background in PHP primarily) and I am scratching my head trying to figure out how to 'clean/sanitize' input / string that is user submitted.
I want to make it HTMLSAFE, eliminate any javascript, or SQL query injection, the usual.
I am hoping I've overlooked some kind of function that already comes with CF9.
Can someone point me in the proper direction?
Well, for SQL injection, you want to use CFQUERYPARAM.
As for sanitizing the input for XSS and the like, you can use the ScriptProtect attribute in CFAPPLICATION, though I've heard that doesn't work flawlessly. You could look at Portcullis or similar 3rd-party CFCs for better script protection if you prefer.
This an addition to Kyle's suggestions not an alternative answer, but the comments panel is a bit rubbish for links.
Take a look a the ColdFusion string functions. You've got HTMLCodeFormat, HTMLEditFormat, JSStringFormat and URLEncodedFormat. All of which can help you with working with content posted from a form.
You can also try to use the regex functions to remove HTML tags, but its never a precise science. This ColdFusion based regex/html question should help there a bit.
You can also try to protect yourself from bots and known spammers using something like cfformprotect, which integrates Project Honeypot and Akismet protection amongst other tools into your forms.
You've got several options:
"Global Script Protection" Administrator setting, which applies a regular expression against post and get (i.e. FORM and URL) variables to strip out <script/>, <img/> and several other tags
Use isValid() to validate variables' data types (see my in depth answer on this one).
<cfqueryparam/>, which serves to create SQL bind parameters and validate the datatype passed to it.
That noted, if you are really trying to sanitize HTML, use Java, which ColdFusion can access natively. In particular use the OWASP AntiSamy Project, which takes an HTML fragment and whitelists what values can be part of it. This is the same approach that sites like SO and slashdot.org use to protect submissions and is a more secure approach to accepting markup content.
Sanitation of strings in coldfusion and in quite any language is very important and depends on what you want to do with the string. most mitigations are for
saving content to database (e.g. <cfqueryparam ...>)
using content to show on next page (e.g. put url-parameter in link or show url-parameter in text)
saving files and using upload filenames and content
There is always a risk if you follow the idea to prevent and reduce a string by allow basically everything in the first step and then sanitize malicious code "away" by deleting or replacing characters (blacklist approach).
The better solution is to replace strings with rereplace(...) agains regular expressions that explicitly allow only the characters needed for the scenario you use it as an easy solution, whenever this is possible. use cases are inputs for numbers, lists, email-addresses, urls, names, zip, cities, etc.
For example if you want to ask for a email-address, you could use
<cfif reFindNoCase("^[A-Z0-9._%+-]+#[A-Z0-9.-]+\.(?:[A-Z]{5})$", stringtosanitize)>...ok, clean...<cfelse>...not ok...</cfif>
(or an own regex).
For HTML-Imput or CSS-Imput I would also recommend OWASP Java HTML Sanitizer Project.

Can I use any other CF tags to replace CFDUMP?

In the application I designed, I named one of my web pages "error.cfm". I want it to display whenever there is an error from the application. So I put the following code inside "error.cfm":
An uncaught exception just 'happened' :-(
<br><br>
<b><cfoutput>#exception.message#</cfoutput></b><br />
<cfoutput>#exception.detail#</cfoutput><br /><br />
<cfif isdefined('exception.cause')>
<b><cfoutput>#exception.cause.message#</cfoutput></b><br />
<cfoutput>#exception.cause.detail#</cfoutput>
</cfif>
<cfdump var="#exception#">
So after hosting the website, I discovered that this particular page refused to load and instead a '500 Internal Error' was displayed. I then complained to my hosting company and I was sent these details:
Dear Customer,
The actual error message is the following:
Security: The requested template has been denied access to createobject(java).
The following is the internal exception message: access denied (coldfusion.runtime.FunctionPermission createobject(java))
The error occurred in C:\inetpub\vhosts\plat4ad.com\httpdocs\cms\error.cfm: line 10
8 :
9 :
10 :
Unfortunately some tags and functions are disabled on our servers due to security purposes. You can check full list here:
https://support.dailyrazor.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=293&nav=0,29,76
Please let us know if you have any other questions.
Best wishes,
Support-GG
DailyRazor Support Team.
Now checking the lists of the tags they disabled on their servers, CFDUMP was among them:
On the shared ColdFusion servers you will have access to all tags and
functions except for the following:
CF Tags:
CFCOLLECTION
CFCONTENT
CFDUMP
CFEXECUTE
CFLOG
CFOBJECT
CFOBJECTCACHE
CFREGISTRY
CF Functions:
SetProfileString
CreateObject(COM)
CreateObject(CORBA)
CreateObject(JAVA)
Please is there any alternative for CFDUMP? Or does anyone know any ColdFusion hosting company that doesn't have these restrictions? I appreciate any ideas from you.
How about this: http://www.coldfusionjedi.com/index.cfm/2010/10/6/Cant-dump-Try-a-fake-dump
If you get yourself VPS (google:coldfusion VPS) you'll be able to do on that machine whatever you want.
The whole point of not having cdump is security measure. Usually all debugging and dumping raw data is done on development server and on hosted server you run applications, right? Error.cfm is used to hide caught exception details, like path to your files, data about used libraries etc.
There're ways to display data without cfdump. In your case I don't see why you couldn't use plain cfoutput and write exception details as 2 lines of output.
In your place, I'd take source codes of fw/1 or Mura CMS to see how they handle error messages, better to see how more experienced people are doing it then to "waste" time reinventing the wheel.
CFDUMP began life as a custom tag before it was ever included in ColdFusion.
It still appears to be downloadable: http://www.adobe.com/cfusion/exchange/index.cfm?event=extensionDetail&extid=1002037
You may need to make changes to it to work (better) in more modern versions of CF, but it should probably do more than 90 per cent of what you need it for.
I will add that if you choose to "dump" details, you can also wrap the dump code in an "if" block that looks at your cgi.REMOTE_ADDR and if it matches your IP address, does the dumpout, otherwise doesn't.
<cfif cgi.REMOTE_ADDR EQ "167.96.177.66">
<!--- execute dump code here --->
</cfif>
Caveats: YMMV, have to have a static IP, have to BE at that IP, etc.