ColdFusion's binaryDecode(input, 'base64') is picky because padding is mandatory.
What is the correct way to add padding with = to a base64 value?
1.) Ben Nadel uses:
value &= repeatString( "=", ( 4 - ( len( value ) % 4 ) ) );
2.) Arlo Carreon uses
<cfset res = Len(raw_str) % 4>
<cfif res eq 2>
<cfset raw_str &= "==">
<cfelseif res eq 3>
<cfset raw_str &= "=">
</cfif>
While they both seem to work, the 1st solution may return 1 to 4 ='s, while the 2nd solution may return 0, 1 or 2 ='s. Wikipedia on Base64 Padding seems to indicate that a valid base64 value should really have only 1 or 2 ='s.
1st solution seems to work with all base64 value lengths, but it may sometimes return 3 or 4 ='s which is kind of strange. Where as the 2nd solution may fail for base64 value that has remainder of 1. CF throws The input and output encodings are not same.
Padding is mandatory by specification (RFC 2045, 3548/4648).
Implementations MUST include appropriate pad characters at the end of encoded data unless the specification referring to this document explicitly states otherwise.
The correct way to fix missing padding is to append = until ( len(value) % 4 ) eq 0. This means a correctly padded Base64 string can only end:
without =
with =
with ==
The specification allows ("may") ignoring excessive padding.
If more than the allowed number of pad characters is found at the end of the string (e.g., a base 64 string terminated with "==="), the excess pad characters MAY also be ignored.
Can you elaborate what you mean by The input and output encodings are not same.? This sounds like an invalid Base64 encoded string. You may want to check what toBinary() returns for the input. It will probably tell you The parameter 1 of function ToBinary, which is now ... must be a base-64 encoded string, which is exactly the problem.
Related
I have fields to test and make sure they only accept integers. There is few functions but I wasn't sure which one is the best. First I tried isValid("integer",value) but I have discovered that "1,5" will be accepted as an integer. So then I tried isNumeric(value) but this will accept values like 1.5. I'm wondering what should be the best way to check for integers? Maybe two combine these two functions like:
<cfif isValid("integer",value) AND isNumeric(value)>
Or there is better way to do this?
cfscript
// Returns if the provided value is a signed integer up to 32 Bit.
function isINT(any value) {
return (
isSimpleValue(ARGUMENTS.value) &&
(reFind("^\-?[0-9]{1,10}$", ARGUMENTS.value) > 0) &&
(ARGUMENTS.value <= 2147483647) &&
(ARGUMENTS.value >= -2147483648)
);
}
cftag
<cffunction name="isINT" access="public" output="false" returnType="boolean"
hint="Returns if the provided value is a signed integer up to 32 Bit.">
<cfargument name="value" type="any" required="true">
<cfreturn (
isSimpleValue(ARGUMENTS.value) and
(reFind("^\-?[0-9]{1,10}$", ARGUMENTS.value) gt 0) and
(ARGUMENTS.value lte 2147483647) and
(ARGUMENTS.value gte -2147483648)
)>
</cffunction>
isSimpleValue making sure the input is a primitive type (by CF means), because all numbers are considered simple values in CF (string conversion)
reFind regular expression checking digits-only (with or without sign), minimum of one digit, maximum of ten digits (implicit call of toString here)
check the range, all numeric types fit into 4 Bytes, thus no need to "upgrade" the type (as you would need to with BigInteger, BigDecimal etc.)
If you don't need the range check for 4 Byte integers, #DanBracuk posted an answer with a function that performs around 5-6 times faster than this one.
Here's the isInteger UDF that I prefer using:
function isInteger(num){
return YesNoFormat(refind("^-?\d+$", num) AND VAL(num) LTE 2147483647 AND VAL(num) GTE -2147483648);
}
Here are some tests to determine how it functions and compares against the various built-in functions.
https://gist.github.com/JamoCA/fab1104a3a9074434ff336630dd5ffd1
View the results using TryCF.com
https://trycf.com/gist/fab1104a3a9074434ff336630dd5ffd1
You could try this:
value = replace(value, ',', '', 'all');
numberIsInteger = isNumeric(value) && round(value) == value ? true : false;
Note
People often include commas in large numbers such as 1,000,000. isNumeric will return false for that string, as will the refind function in the other answers.
I am attempting to use ColdFusion's hmac() function to calculate an HMAC value using binaryEncode(binaryObj,'Base64') instead of toBase64() since that function is deprecated. It works perfectly with toBase64() but not with binaryEncode(). The docs are not very informative. Can someone help me understand why I cannot get the same value using binaryEncode?
From what I understand, the hmac() function returns the results in hexadecimal format. binaryEncode() expects a binary value, so thehmac() results must be first converted from hex to binary, before it can be converted to base64.
<cfset string = "1234567890" />
<cfset secretKey = "abcdefghijklmnopqrstuvwxyz" />
<!--- Get Hex results from HMAC() --->
<cfset hmacHex = hmac(string,secretKey,'HMACSHA256') />
<!--- Decode the binary value from hex --->
<cfset hmacAsBinary = binaryDecode(hmacHex,'hex') />
<!--- Convert binary object to Base64 --->
<cfset hmacBase64 = binaryEncode(hmacAsBinary, 'base64') />
<cfoutput>
<!--- incorrect hmac signature --->
hmacBase64: #hmacBase64#<br>
<!--- correct hmac signature --->
toBase64: #toBase64(hmac(string,secretKey,'HMACSHA256'))#<br>
</cfoutput>
The results are:
hmacBase64: VEVGNnqg9b0eURaDCsA4yIOz5c+QtoJqIPInEZOuRm4=
toBase64: NTQ0NTQ2MzY3QUEwRjVCRDFFNTExNjgzMEFDMDM4Qzg4M0IzRTVDRjkwQjY4MjZBMjBGMjI3MTE5M0FFNDY2RQ==
One thing I noticed is the results are much longer when using toBase64(). I can't seem to figure out why I can't use binaryEncode(). However, I would like to, since toBase64() is being deprecated. Any insight is much appreciated. Thanks!
Update based on comments:
Well using ToBase64(Hmac(...)) is not the correct way to convert a hex string to base64 ;-) However, it sounds like the API requires something other than a straight conversion. If so, just do what the ToBase64(hmac(...)) code is doing. ie Decode the hex string as UTF8 and re-encode it as base64:
matchingResult = binaryEncode(charsetDecode(hmacHex, "utf-8"), "base64")
Short answer:
The two methods are encoding totally different values. That is why the results do not match. The correct way to convert the hex string to base64 is using BinaryEncode/Decode().
Longer answer:
<!--- correct hmac signature --->
toBase64: #toBase64(hmac(string,secretKey,'HMACSHA256'))#<br>
Actually that is not the correct way to convert hex to base64.
Hexadecimal and Base64 are just different ways of representing a binary value. In order to get the same results, the two methods need to start with the same binary. In this case, are actually encoding totally different values. Hence the difference in the results.
With a hexadecimal string, each byte is represented by two characters. So the binary will be half the size of the original string. In the case of HMAC(HMACSHA256), the resulting hex string is 64 characters long. So the binary value should be 32 bytes. To obtain the correct binary value, the string must be decoded as hex:
original string length = #len(hmacHex)#
binary size = #arrayLen(binaryDecode(hmacHex, "hex"))#
The problem with ToBase64 is that it decodes the string incorrectly. It treats the input as UTF8 and decodes the characters in the string individually. So the binary value is double the size it should be. Notice it is 64 bytes, instead of 32? That is why the final string is longer as well.
UTF8 binary size = #arrayLen(charsetDecode(hmacHex, "utf-8"))#
ToBase64 binary size = #arrayLen(binaryDecode(toBase64(hmacHex), "base64"))#
So again, the two methods produce different results because they are encoding totally different values. However, strictly speaking, only the first method is correct. To re-encode a hex string as base64 use binaryEncode/binaryDecode:
correctResult = binaryEncode(binaryDecode(hmacHex, "hex"), "base64")
When I pack the data to fixed length and then while unpacking I am unable to retrieve the data with out mentioning the actual length of the data.
How do I retrieve only data without the \x00 characters without calculating the length in prior.
>>> import struct
>>> with open("forums_file.dat", "w") as file:
file.truncate(1024)
>>> country = 'india'
>>> data = struct.pack('20s', country)
>>> print data
india
>>> data
'india\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>>> print len(data)
20
>>> unpack_data = struct.unpack('5s', country)
>>> unpack_data
('india',)
In the above code snippet I had mentioned the length of the data(5s) while unpacking.
Short answer: You can't do it directly.
Longer answer:
The more indirect solution is actually not that bad. When unpacking the string, you use the same length as you used for packing. That returns the string including the NUL chars (0 bytes).
Then you split on the NUL char and take the first item, like so:
result_with_NUL, = struct.unpack('20s', data)
print(repr(result_with_NUL))
result_string = result_with_NUL.split('\x00', 1)[0]
print(repr(result_string))
The , 1 parameter in split() is not strictly necessary, but makes it more efficient, as it splits only on the first occurrence of NUL instead of every single one.
Also note that when packing and unpacking with the goal to read/write files or exchange data with different systems, it's important to explicitly precede your format strings with "<" or ">" (or "=" in certain very special cases), both for packing and unpacking, since otherwise it will align and pad the structures, which is heavily system dependent and might cause hard to find bugs later.
1.val Matcher = """.+/(.*)""".r
2.val Matcher(title) = """http://en.wikipedia.org/wiki/Château_La_Louvière"""
3.val lowerCase = title.toLower
4.if(lowercase.equals("château_la_louvière")) //do something
The above comparison returns false because I guess line 2 results in Ch?teau_La_Louvi?re. Any ideas how I can accomplish this?
As 4e6 says the problem lies within the standard configuration of Java. Which assumes all files encoded in Latin1.
1.val Matcher = """.+/(.*)""".r
2.val Matcher(title) = """http://en.wikipedia.org/wiki/Château_La_Louvière"""
This could be fixed by setting the following java-OPTS
export JAVA_OPTS='-Dfile.encoding=UTF-8'
Still 1. and 2. will work, even if you don't change the encoding. The Problem lies in 3. and 4. .
3.val lowerCase = title.toLower
4.if(lowercase.equals("château_la_louvière")) //do something
''toLower'' will cause the test in 4. to fail , because "â" and "è" will be interpreted wrongly. These characters would be encoded as two up to four bytes, which each will be lowercased independently thus yielding a completely different result as ''château_la_louvière'' .
I normally don't work in ColdFusion but there's a FTP process at work I have to create a report for with the only option right now being a ColdFusion 8 server. This FTP feed has a few issues (trash too).
So, I make the query and then I need to convert some of the string values during the output to do some math. Before that:
How do I tell if a field in the output loop: is not blank or null, is string that can be converted into a valid number, and is not 0?
Is there a simple way of doing this w/o a lot of if statements?
Thanks!
So you want to make sure that the variable is numeric but not zero?
Then you want this:
<cfif IsNumeric(MyVar) AND MyVar NEQ 0 >
Determining if a string is not null/blank and is a number and not 0?
Here's the code I would use in this case.
<cfif isDefined(stringVar) and len((trim(stringVar))) and isNumeric(stringVar)>
do stuff here
</cfif>
isDefined returns a true if the variable exists. If you know the scope of the variable, i.e., its in the form or url scope for instance, you can use structkeyExists(form,"stringVar"). I would recommend using this approach if you know the scope of the variable.
Len(trim(stringVar)) is the second check. First off it trims any leading or trailing empty spaces from the string - this makes sure that any empty variables are not passed along. Then if something is there it will return the length of the string. If its empty len will return a 0.
isNumeric(stringVar) returns a true if the variable is a number and false otherwise.
<cfif Len(field) and Val(field)>
Len() will verify the field has length (not blank--there are no NULLs in CF) and Val() will automatically convert the first character in the string into into a number--or return 0 if it cannot.
Take note of Peter's comment below; although this is the least verbose answer, Val() may fail in certain edge conditions below, ie. The field is a string but starts with a number, incorrectly converting it to a number, and evaluating to TRUE.
<cfif isNumeric(myfield) and myfield gt 0>