I'm trying to use the Twitter API to pull down statuses from the Lists API using ColdFusion and am parsing everything I need just fine using the JSON format and a JSON component.
The problem I've come across is trying to convert the Unicode characters so they display correctly on screen.
here is the sample data that comes from the JSON feed
F\u00e0bregas
Is there some regex I could use to convert this?
Currently I have it writing out the raw data from the JSON feed
#node.user.name#
Which is fine, but it contains the \u00e0 which I need to convert so it displays as Fàbregas with the correct accent over the 'a'.
First up I think this is more of an character encoding issue than a regex issue.
How are you getting the Twitter data? If it's using <cfhttp> you could try setting the charset attribute to UTF-8. This will ensure that the data from Twitter arrives in UTF-8.
Then you should explicitly set the character encoding on the page you are trying to output the data on (the FORM and URL encoding while you are at it). For example:
<!--- URL and FORM encoding to UTF-8 --->
<cfset setEncoding("URL", "UTF-8") />
<cfset setEncoding("FORM", "UTF-8") />
<cfcontent type="text/html; charset=UTF-8" />
<cfoutput>#node.user.name#</cfoutput>
You'll find some more info here. Hope that helps!
There has to be a better way but until then I think this works
<cfset y = 'F\u00e0bregas'/>
<cfset x = evaluate(de(rereplace(y,'\\u([a-fA-f0-9]{4})','##chr(inputbasen(''\1'',16))##','all')))/>
Related
I'm able to get all the values that I want from cfldap.
But when I try to update the user image I don't know how to send the correct value for the binary image attribute.
I tried getting the image variable from the cffile upload
<cffile action="UPLOAD" filefield="file" destination="c:\inetpub\wwwroot\test" nameconflict="OVERWRITE" result="image" />
Also tried using cfimage with a static image -
<cfimage action="read" source="c:\inetpub\wwwroot\test\image.png" name="anotherImage">
Or even with
<cffile action="READBINARY" file="c:\inetpub\wwwroot\test\image.png" variable="BinaryImageContent">
But in any case, when I call
<cfldap action="modify"
DN="#results.dn#"
attributes="thumbnailPhoto=#obj.image#"
modifytype="replace"
server="myserver"
username="mydomain\myuser"
password="mypass">
The #results.dn# is the DN from the user that I get before (Everything ok on that)
I created the #obj.image# to be able to try all types of variables
Also tried these params:
<cfset obj.test1 = BinaryImageContent />
<cfdump var="#imageGetBlob(anotherImage)#" />
<cfdump var="#toString(obj.test1)#" />
By the way, the error that I get its
One or more of the required attributes may be missing or incorrect or
you do not have permissions to execute this operation on the server.
The problem is that I'm using the domain administrator account to update that
(THIS ERROR IS SOLVED - The network guys hadn't given me this permission... now I have it).
Now what I'm using is the following:
<cffile action="UPLOAD" filefield="file" destination="c:\inetpub\wwwroot\test" nameconflict="OVERWRITE" result="imagem" />
<cfset filename = "C:\inetpub\wwwroot\test\#imagem.serverFile#">
<cffile action="readbinary" file="#filename#" variable="img">
<cfset imgStr = BinaryEncode(img, "hex")>
<cfset imgStr2 = REReplace(imgStr, "..", "\\\0", "ALL")>
<cfldap
action="modify"
DN="#results.dn#"
attributes="thumbnailPhoto=#imgStr2#"
modifytype="replace"
server="myserver"
username="mydomain\myuser"
password="mypass"
>
but I get this binary code
Whats strange, is that before I had a binary code like -1-41 and now, nothing similar...
and when I try to show the pic
And this is one correct image....
EDIT: The original code sample below shows how it could work if ColdFusion wouldn't have a bug (or "very unfortunate design decision") in CFLDAP.
CFLDAP encodes the parameter values you pass to it before sending them to the server. This is nice because you don't have to worry about value encoding. But... it is also not helpful because it means you can't send encoded values yourself anymore, since CF invariably encodes them again.
Bottom line: As far as LDAP is concerned, encoding a file into a hex-string is correct, but CFLDAP mangles that string before sending it to the server. Combined with the fact that CFLDAP does not accept raw binary data this means that you can't use it to update binary attributes.
The comments contain a suggestion for a 3rd-party command line tool that can easily substitute CFLDAP for this task.
You need to send an encoded string to the server as the attribute value. The encoding scheme for binary data in LDAP queries has the form of attribute=\01\02\03\ab\af\cd.
Read your image into a byte array, encode that array into a hex string and prefix every encoded byte with a backslash.
<cffile action="readbinary" file="#filename#" variable="img">
<cfset imgStr = BinaryEncode(img, "hex")>
<cfset imgStr = REReplace(imgStr, "..", "\\\0", "ALL")>
<cfldap
action="modify"
DN="#results.dn#"
attributes="thumbnailPhoto=#imgStr#"
modifytype="replace"
server="myserver"
username="mydomain\myuser"
password="mypass"
>
Also don't forget what the documentation has to say about modifyType.
When a user registers on a site, should we use EncodeForHTML() or EncodeForURL() before storing the value in a DB?
The reason I ask this is that when I send an e-mail to someone that includes a URL that contains an email address as a URL variable, I have to use EncodeForURL(). But if this email address is already encoded using EncodeForHTML(), it will mean I have to Canonicalize() it before using EncodeForURL() on it again.
I would therefore think that EncodeForURL() is probably good, but is it 'safe' and 'correct' when storing the value in a database?
Update: Upon reading the docs it says that EncodeForURL is only for using a value in a URL. Thereofore it seems to make sense that I should store it as EncodedForHTML, but then Canonicalize and re-encode for URL when using it in a URL context. I don't know how much of a performance hit all this encoding is going to take on my server...??
Copying this from my company's internal documentation. Not sure if the images uploaded correctly since imagr is blocked # work. If so, I'll re-upload them later. I'll be publishing this and more related content to a Githib repo in the future.
You should store it as simple text, but make sure you scrub your data on the way in using an AntiSamy library. Once the data is safe, make sure to encode the data on the way out using the proper encoder. And FYI, there's a big difference between the output of encodeForHTML() and encodeForHTMLAttribute().
In the below examples, substitute the variables that define email addresses with data from the DB.
PROTIP: Don't use these encoders in CFFORM tags. Those tags take care of the encoding for you. CF 9 and below use HTMLEditFormat(), CF 10 and above most likely use encodeForHTMLAttribute().
Simple Implementation
A basic implementation is to include a single e-mail address in order to populate the "To" field of a new e-mail window.
CFML
<cfset email = "someone#example.com" />
E-mail
HTML Output
E-mail
CFML with Proper Encoding
<cfset email = "someone#example.com" />
E-mail
Encoded HTML Output
Notice that the "#" symbol is properly percent encoded as "%40".
E-mail
Results when clicked
And if you plan on showing the e-mail address on the page as part of the link:
<cfset email = "someone#example.com" />
#encodeForHTML(email)#
Attack Vector
An advanced implementation includes e-mail addresses for "To" & "CC". It can also pre-populate the body and subject of the new e-mail.
CFML without encoding
<cfset email = "someone#example.com" />
<cfset email_cc = "someone_else#example.com" />
<cfset subject = "This is the subject" />
<cfset body = "This is the body" />
E-mail
HTML Output
E-mail
Results when clicked
Notice that the subject and body parameters contain spaces. While this string will technically work, it is still prone to attack vectors.
Imagine the value of body is set by the result of a database query. This record has been "infected" by a malicious user and the default body message has an appended "BCC" address, so some evil user can get copies of e-mails sent via this link.
Infected Data
<cfset body = "This is the body&bcc=someone#evil.com" />
HTML Output
E-mail
Results when clicked
In order to stop this MAILTO link from being infected, this string needs to be properly encoded.
CFML with HTML Attribute Encoding
Since "href" is an attribute of the <a> tag, you might think to use the HTML Attribute encoder. This would be incorrect.
<cfset email = "someone#example.com" />
<cfset email_cc = "someone_else#example.com" />
<cfset subject = "This is the subject" />
<cfset body = "This is the body&bcc=someone#evil.com" />
E-mail
HTML Output
E-mail
Results when clicked
CFML with URL Encoding
The correct encoding of a MAILTO link is done with the URL encoder.
<cfset email = "someone#example.com" />
<cfset email_cc = "someone_else#example.com" />
<cfset subject = "This is the subject" />
<cfset body = "This is the body&bcc=someone#evil.com" />
E-mail
HTML Output with Correct Encoding
Notice these things about the URL encoder:
Each space (" ") is converted to a plus sign ("+") instead of its expected percent value ("%20").
Encoding is otherwise done using percent ("%") values.
Since the individual query paramters are encoded, the ampersands ("&") connecting each paramter were not encoded.
When the "body" paramter is encoded, it includes the "&body=" string that was maliciously injected. This entire string is now part of the message body, which prevents the unintended "bcc" of the e-mail.
E-mail
Results when clicked
What's with the plus signs? It is up to the individual mail client (e.g. Outlook, GMail, etc.) to correctly decode these URL encoded values.
Store the email addresses in plain text, then encode them when you use them, depending on the context. If it's going to be a part of URL, use EncodeForURL(). If it's going to be displayed in HTML as text, use EncodeForHtml().
I'm working on an app that creates customized images based on user inputs using canvas and I was wondering if it was possible to allow users to email themselves a copy of the final product in base64 or if I would have to convert it to .jpg or .png and then embed it as that.
Thanks.
I'd suggest converting the Base64 to an image, writing to disk, and using cfmailparam to attach/inline it as well as automatically remove it from disk.
I have had nothing but issues trying to directly inline/attach base64 images to emails using cfmail. I have had partial success converting the base64 to an image object (using ImageReadBase64()) and then using the image object as the value of the cfmailparam content attribute and ommitting the file attribute, however the image comes through with inverted colors oddly enough.
On to some example code...
<cfsavecontent variable="mailContent">
...
<img src="cid:signature">
...
</cfsavecontent>
<cfset sigImage = ImageReadBase64(SIGNATURE_IMAGE_BASE64)>
<cfimage source="#sigImage#" destination="tmpSigImage.png" action="write" overwrite="true">
<cfmail ...>
#mailContent#
<cfmailparam
file="#ExpandPath('./tmpSigImage.png')#"
contentid="signature"
disposition="inline"
remove="yes"
/>
</cfmail>
I am using cfhttp to get a website . I want to replace all the links inside the body tags. Importantly I don't want to mess up the stylesheets etc in the head.
I want to do the following:
In the external web page body we may find a link:
External Link
I want to replace it with the following:
External Link
Its easy enough using Replace() but then I also replace all the linked stylesheets etc. I just want to edit the href's of clickable links.
I've modified an HTML document's DOM to add tracking parameters to links in outbound email messages using the jsoup library. (jsoup is an open source Java HTML Parser and can be download at http://jsoup.org/.) You'll note that it uses jQuery-like select methods, but all manipulations are performed on the server-side (I've also used it for removing ads from CFHTTTP-fetched HTML.)
Here's a quick sample of working ColdFusion code that will do exactly what you want on the server-side:
<CFSET TheHTML = CFHTTP.FileContent>
<CFSET jsoup = CreateObject("java", "org.jsoup.Jsoup")>
<CFSET TempHTML = jsoup.parse(TheHTML)>
<CFLOOP ARRAY="#TempHTML.select('a')#" INDEX="ThisLink">
<CFSET TheLink = thisLink.attr("href").toString()>
<CFSET TheHTML = replace(TheHTML, TheLink, "http://mywebsite.com/?u=" & URLEncodedFormat(TheLink))>
</CFLOOP>
What would be the correct way to stop the white space that ColdFusion outputs?
I know there is cfcontent and cfsetting enableCFoutputOnly. What is the correct way to do that?
In addition to <cfsilent>, <cfsetting enablecfoutputonly="yes"> and <cfprocessingdirective suppressWhiteSpace = "true"> is <cfcontent reset="true" />. You can delete whitespaces at the beginning of your document with it.
HTML5 document would then start like this:
<cfcontent type="text/html; charset=utf-8" reset="true" /><!doctype html>
XML document:
<cfcontent reset="yes" type="text/xml; charset=utf-8" /><CFOUTPUT>#VariableHoldingXmlDocAsString#</CFOUTPUT>
This way you won't get the "Content is not allowed in prolog"-error for XML docs.
If you are getting unwanted whitespaces from a function use the output-attribute to suppress any output and return your result as string - for example:
<cffunction name="getMyName" access="public" returntype="string" output="no">
<cfreturn "Seybsen" />
</cffunction>
You can modify the ColdFusion output by getting access to the ColdFusion Outpout Buffer. James Brown recently demo'd this at our user group meeting (Central Florida Web Developers User Group).
<cfscript>
out = getPageContext().getOut().getString();
newOutput = REreplace(out, 'regex', '', 'all');
</cfscript>
A great place to do this would be in Application.cfc onRequestEnd(). Your result could be a single line of HTML which is then sent to the browser. Work with your web server to GZip and you'll cut bandwidth a great deal.
In terms of tags, there is cfsilent
In the administrator there is a setting to 'Enable whitespace management'
Futher reading on cfsilent and cfcontent reset.
If neither <cfsilent> nor <cfsetting enablecfoutputonly="yes"> can satisfy you, then you are probably over-engineering this issue.
When you are asking solely out of aesthetic reasons, my recommendation is: Ignore the whitespace, it does not do any harm.
Alternatively, You can ensure your entire page is stored within a variable and all this processing is done within cfsilent tags. e.g.
<cfsilent>
<!-- some coldfusion -->
<cfsavecontent variable="pageContent">
<html>
<!-- some content -->
</html>
</cfsavecontent>
<!-- reformat pageContent if required -->
</cfsilent><cfoutput>#pageContent#</cfoutput>
Finally, you can perform any additional processing after you've generated the pagecontent but before you output it e.g. a regular expression to remove additional whitespace or some code tidying.
Here's a tip if you use CFC.
If you're not expecting your method to generate any output, use output="false" in <cffunction> and <cfcomponent> (not needed only if you're using CF9 script style). This will eliminate a lot of unwanted whitespaces.
If you have access to the server and want to implement it on every page request search for and install trimflt.jar. It's a Java servlet filter that will remove all whitespace and line breaks before sending it off. Drop the jar in the /WEB-INF/lib dir of CF and edit the web.xml file to add the filter. Its configurable as well to remove comments, exclude files or extensions, and preserve specific strings. Been running it for a few years without a problem. A set it and forget it solution.
I've found that even using every possible way to eliminate whitespace, your code may still have some unwanted spaces or line breaks. If you're still experiencing this you may need to sacrifice well formatted code for desired output.
for example, instead of:
<cfprocessingdirective suppressWhiteSpace = "true">
<cfquery ...>
...
...
...
</cfquery>
<cfoutput>
Welcome to the site #query.userName#
</cfoutput>
</cfprocessingdirective>
You may need to code:
<cfprocessingdirective suppressWhiteSpace = "true"><cfquery ...>
...
...
...
</cfquery><cfoutput>Welcome to the site #query.UserName#</cfoutput></cfprocessingdirective>
This isn't CF adding whitespace, but you adding whitespace when formatting your CF.
HTH