I am working on implementing the Amazon REST API in our application. The application is build with WinDev. In order too test my signature calculation i desided to try the test suite provided by amazon:
https://docs.aws.amazon.com/general/latest/gr/signature-v4-test-suite.html
This is how i derive the hex value of my canonical request:
sCanonicalRequestHash = :HashCanonicalRequest([
GET
/
Param1=value1&Param2=value2
host:example.amazonaws.com
x-amz-date:20150830T123600Z
host;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
])
The method HashCanoncialRequest removes all the char 10 ( this is done in order to hash the string correctly) hashes the string in to binary using windev's hashstring function. This binary function is converted to a hex value, all the whitespace is removed and changed to lower case.
//Remove char 13 ,otherwise the hash fails( Windows enter )
sResult = Replace(sResult, Charact(13), "")
//Create hash
sResult = HashString(HA_SHA_256, sResult)
//Convert hash to lower case hex
sResult = Lower(BufferToHexa(sResult, 1, 32))
//Remove spaces
sResult = Replace(sResult," ", "")
This results the following value:
816cd5b414d056048ba4f7c5386d6e0533120fb1fcfa93762cf0fc39e2cf19e0
This is the value expected by the test suite. So far so good.
Next up is the string to sign, this looks as followed:
AWS4-HMAC-SHA256
20150830T123600Z
20150830/us-east-1/service/aws4_request
816cd5b414d056048ba4f7c5386d6e0533120fb1fcfa93762cf0fc39e2cf19e0
Now it's time to calculate the signingkey.
First up some values given by the test suite:
sSecret is string = "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY"
sDate is string = "20150830"
sRegion is string = "us-east-1"
And now the calculation:
bufDateKey is Buffer = WL.HashString(HA_HMAC_SHA_256, sDate, "AWS4" + sSecret)
bufRegionKey is Buffer = WL.HashString(HA_HMAC_SHA_256, sRegion, bufDateKey)
bufServiceKey is Buffer = WL.HashString(HA_HMAC_SHA_256, "service", bufRegionKey)
bufSigningKey is Buffer = WL.HashString(HA_HMAC_SHA_256, "aws4_request", bufServiceKey)
Amazon provides a different test in order too check your calculations here and this calculation is tested and returns the value expected.
Now for the part that doesn't do what the test suite expects. The signature calculation.
//Hashing the ss with psSigningKey as the key
bufSignature = WL.HashString(HA_HMAC_SHA_256, ss, bufSigningKey)
//Converting the hash to hex
bufSignature = BufferToHexa(bufSignature, 1, 32)
//Converting the hex value to lower case and remove any whitespace
bufSignature = Replace(Lower(bufSignature), " ", "")
ss is the string value of the string to sign as shown in the third code snipped
bufSigningKey is the binary value of the result for the for last code snipped. This is converted to hex and all the white space is removed and the string is converted to lower case. This do's not return the signature as shown by the test suite.
If hope someone can help.
Related
I am trying to make a (assembly) parser which uses a string as a guide for how to cut the text to get the tokens I want.
string s = "$t4,";
string guide = "$!,$!,$!";
int i = 1;
string test =s.substr(0, s.find_first_of(" ,.\t"+to_string(guide[i+1]) ));
cout << test << "\n";
if s = "$t4" then test = "$t"
what I am expecting it to do is test to be "$t4", this works for every other $tX except for specifically the number 4 even though it's not in the (" ,.\t"+to_string(guide[i+1])) string
s.find_first_of(" ,.\t" + std::to_string(guide[i + 1]))
Assuming ASCII, that string will be:
,.\t44
44 is the ASCII value of the , in guide[i + 1].
The first character in "$t4," that it'll find is 4 at position 2, and you then create a substring from 0 and length 2, that is $t.
server -> AUTH GateKeeper S :GKSSP\0\0\0\0\0\0\0\0\0r!W\tvM
Client -> AUTH GateKeeper S :GKSSP\0\0\0\0\0\0\0\0\0"g#çójDî3(ƒP¡Á"
VB.NET tcp code:
Do While Stream.CanRead
responseData = Trim(System.Text.Encoding.UTF8.GetString(data2, 0, bytes))
Feed = responseData.Split(Chr(13), Chr(10))
Dim datax As [Byte]()
For Each line As String In Feed
If line <> "" Then
Select Case UCase(Split(line.Trim, " ")(0))
Case "AUTH"
If (Split(line, " ")(2) = "S") And (Split(line, " ")(3) <> ":OK") Then
ChOK = ":GKSSP\0\0\0" & Chr(2) & "\0\0\0" & Chr(3) & "\0\0\0" & Challenge_1(Mid(line, InStr(line, "\0\0\0\0\0\0") + Len("\0\0\0\0\0\0"))) & "Sm(" &
HexToAsc("e4") & "HS" & HexToAsc("c1") & "M" & HexToAsc("847f8293f98d") & "UC"
message = "AUTH GateKeeper S " & ChOK & vbCrLf
VB.NET challenge functions
Public Function Challenge_1(Challenge As String) As String
Dim c1 As String, a1 As String, c2 As String, a2 As String
c1 = "edp{}e|wxrdse}}u666666666666666666666666666666666666666666666666" + Challenge
a1 = HexToAsc(GetIRC7PWD(c1))
Dim Aaah As New String("\"c, 48)
c2 = HexToAsc("0f0e1a11170f161d12180e190f17171f") & Aaah & a1
a2 = HexToAsc(GetIRC7PWD(c2))
Challenge_1 = a2
End Function
Public Function GetIRC7PWD(source As String)
Dim md5Hash As MD5 = MD5.Create()
' Dim source As String = "edp{}e|wxrdse}}u666666666666666666666666666666666666666666666666"
Dim data As Byte() = md5Hash.ComputeHash(Encoding.Default.GetBytes(source))
Dim sBuilder As New StringBuilder()
Dim i As Integer
For i = 0 To data.Length - 1
sBuilder.Append(data(i).ToString("x2"))
Next i
Return sBuilder.ToString()
End Function
The above code authenticates successfully.
My C++ code works fine, until it reads data sent from the AUTH and MD5's it. However, it creates the wrong MD5 hash:
while (true)
{
ZeroMemory(buff, 1024);
int bytesReceived = NetDll_recv(XNCALLER_SYSAPP,Sock, buff, 1024, 0);
if (bytesReceived == SOCKET_ERROR)
{
}
if (bytesReceived > 0) {
std::string Recieved(buff, bytesReceived);
std::stringstream iss(Recieved);
while(iss.good())
{
std::string SingleLine;
getline(iss,SingleLine);
if (!SingleLine.empty()) {
stringstream ss(SingleLine);
string s;
int xUp = 1;
vector <string> tokens;
while (getline(ss, s, ' ')) {
tokens.push_back(s);
xUp++;
}
if (tokens[0] == "AUTH" && tokens[2] == "S") {
MD5 md5;
string ChOK3 = SingleLine.substr(46);
string ChallengeCode = "edp{}e|wxrdse}}u666666666666666666666666666666666666666666666666" + ChOK3;
char* cz = const_cast<char*>(ChallengeCode.c_str());
string g1 = md5.digestString( cz );
string a1 = hexToASCII(g1);
int n = 48;
char cx = '\\';
string c2 = hexToASCII("0f0e1a11170f161d12180e190f17171f") + std::string(n, cx) + a1;
char* c = const_cast<char*>(c2.c_str());
string a2 = md5.digestString( c );
string a3 = hexToASCII(a2);
//string ChOK4 = ChOK + a2 + "Sm(" + hexToASCII("e4") + "HS" + hexToASCII("c1") + "M" + hexToASCII("847f8293f98d") + "UC\r\n";
string ChOK4 = "AUTH GateKeeper S :GKSSP\\0\\0\\0\x02\\0\\0\\0\x03\\0\\0\\0" + a3 + "Sm(" + hexToASCII("e4") + "HS" + hexToASCII("c1") + "M" + hexToASCII("847f8293f98d") + "UC\r\n";
const char *cstr = ChOK4.c_str();
NetDll_send(XNCALLER_SYSAPP, Sock,ChOK4.c_str(),ChOK4.length(), 0);
}
It looks like my code is the same.
Is it because the characters are UTF-16? How can I read the buffer in UTF-16 so that I can properly MD5 the string?
The two codes are NOT doing the same thing, which is why they are producing different results. Differences are highlighted below:
While reading from the socket:
the VB code is:
reading arbitrary bytes (without regard to message boundaries)
converting the bytes from UTF-8 to a UTF-16 string (without regard to codeunit boundaries)
trimming the string
splitting the string on line breaks
parsing each line
the C++ code is:
reading arbitrary bytes (without regard to message boundaries)
converting the bytes as-is to a std::string (without regard to encoding)
splitting the string on line breaks (without trimming it first)
parsing each line
Up to this point, given the data shown, the results are logically the same for both codes (though technically different layouts in memory), even though they are being processed differently. But even up to this point, both codes are NOT handling the socket data adequately, and there IS potential for data corruption and data loss here.
Now, assuming no corruption/loss has actually occurred, then for the actual AUTH command:
the VB code is:
extracting the challenge from position 37, after the first "\0\0\0\0\0\0" substring (thus, the extracted challenge is "\0\0\0r!W\tvM")
AUTH GateKeeper S :GKSSP\0\0\0\0\0\0\0\0\0r!W\tvM
^ ^ ^
1 25 37
converting the challenge from UTF-16 to ANSI bytes
calculating an MD5 for those bytes
calculating a 2nd MD5 based on the 1st MD5
creating a response for the 2nd MD5
the C++ code is:
extracting the challenge from fixed index 46 (position 47) (thus, the extracted challenge is "tvM")
AUTH GateKeeper S :GKSSP\0\0\0\0\0\0\0\0\0r!W\tvM
^ ^ ^ ^
0 24 36 46
getting the challenge's raw UTF-8 bytes
calculating an MD5 for those bytes
calculating a 2nd MD5 based on the 1st MD5
creating a response for the 2nd MD5
So, the biggest issues I see that you need to fix are the following:
in both languages, fix the socket reading logic to avoid data loss. Right now, you are extracting all bytes from the socket and throwing away any incomplete lines that are waiting for more bytes to arrive. You need to save all raw bytes from the socket into an intermediate buffer first, and then extract only complete lines from that buffer, leaving behind incomplete lines in the buffer so they can be finished on subsequent reads.
on the VB side, for ASCII-only data (as you have shown), the conversion from UTF-8 to UTF-16 to ANSI is loss-less, but for non-ASCII data this will be lossy. Consider converting the UTF-16 data back to UTF-8 instead of ANSI, to match the C++ code.
on the C++ side, fix the code to extract the server's challenge from the correct string position to match the VB code.
I have command output that has been stored in an array of strings, and this numerical value could or could not exist in a string, and if it does exit it can be any integer value. I've tried various methods with regex and parseInt, and I am not getting the desired effect, I believe because of the way the string is encoded. How have others gone about this in the past?
EDIT:
The issue is the string is coming back, encoded as UT8. So the initial string I get is this: disk: 40 Which I can reduce down to: 40 After that however, using a isNaN(decode_utf8(str)) Still produces a true for isNaN. I am not sure exactly how to overcome this
Try testing if a string is a number like this:
var toTest = "232";
if (!isNaN(+toTest)) {
var aNumber = +toTest
console.log('That is a number: ', aNumber)
if it must be a integer you could check like this:
// check if integer ...
var isInt = typeof aNumber === "number" && aNumber % 1 === 0;
}
If you are using latest JavaScript ES6 then isNumber is built in:
Number.isInteger(123) // true
I have the following string:
{'output',{'variable','VGRG_Pos_Var1/Parameters/D_foo'},'date',734704.60904050921}
I would like to verify the format of the string that the word 'variable' is the second word and i would like to retrive the string after the last '/' in the 3rd string (In this example 'D_foo').
how could i verify this and retrive the sting i search?
I tried the following:
regexp(str,'{''\w+'',{''variable'',''([(a-z)|(A-Z)|/|_])+')
without success
REMARK
The string to analysis is not splited after the komma, it is only due to length of the string.
EDIT
my string is:
'{''output'',{''variable'',''VGRG_Pos_Var1/Parameters/D_foo''},''date'',734704.60904050921}';
and not a cell, which could be understood. I added the sybol ' at the start and end of the string to symbolizied that it is a string.
I realise that you mention using regexp in the question, but I'm not sure if this is a requirement? If other solutions are acceptable you could try this:
str='{''output'',{''variable'',''VGRG_Pos_Var1/Parameters/D_foo''},''date'',734704.60904050921}';
parts1=textscan( str, '%s','delimiter',{',','{','}'},'MultipleDelimsAsOne',1);
parts2=textscan( parts1{1}{3}, '%s','delimiter',{'/',''''},'MultipleDelimsAsOne',1);
string=parts2{1}{end}
match=strcmp(parts1{1}{2},'variable')
To answer the first part of your question, you can write this:
str = {'output',{'variable','VGRG_Pos_Var1/Parameters/D_foo'},'date',734704.60904050921};
temp = str(2); %this holds the cell containing the two strings
if cmpstr(temp{1}(1), 'variable')
%do stuff
end
For the second part you can do this:
str = {'output',{'variable','VGRG_Pos_Var1/Parameters/D_foo'},'date',734704.60904050921};
temp = str(2); %like before, this contains the cell
temp = temp{1}(2); %this picks out the second string in the cell
temp = char(temp); %turns the item from a cell to a string
res = strsplit(temp, '/'); %splits the string where '/' are found, res is an array of strings
string = res(3); %assuming there will always be just 2 '/'s.
I have a file that contains rows and columns of information like:
104857 Big Screen TV 567.95
573823 Blender 45.25
I need to parse this information into three separate items, a string containing the identification number on the left, a string containing the item name, and a double variable containing the price. The information is always found in the same columns, i.e. in the same order.
I am having trouble accomplishing this. Even when not reading from the file and just using a sample string, my attempt just outputs a jumbled mess:
string input = "104857 Big Screen TV 567.95";
string tempone = "";
string temptwo = input.substr(0,1);
tempone += temptwo;
for(int i=1 ; temptwo != " " && i < input.length() ; i++)
{
temptwo = input.substr(j,j);
tempone += temp2;
}
cout << tempone;
I've tried tweaking the above code for quite some time, but no luck, and I can't think of any other way to do it at the moment.
You can find the first space and the last space using std::find_first_of and std::find_last_of . You can use this to better split the string into 3 - first space comes after the first variable and the last space comes before the third variable, everything in between is the second variable.
How about following pseudocode:
string input = "104857 Big Screen TV 567.95";
string[] parsed_output = input.split(" "); // split input string with 'space' as delimiter
// parsed_output[0] = 104857
// parsed_output[1] = Big
// parsed_output[2] = Screen
// parsed_output[3] = TV
// parsed_output[4] = 567.95
int id = stringToInt(parsed_output[0]);
string product = concat(parsed_output[1], parsed_output[2], ... ,parsed_output[length-2]);
double price = stringToDouble(parsed_output[length-1]);
I hope, that's clear.
Well try breaking down the files components:
you know a number always comes first, and we also know a number has no white spaces.
The string following the number CAN have whitespaces, but won't contain any numbers(i would assume)
After this title, you're going to have more numbers(with no whitespaces)
from these components, you can deduce:
grabbing the first number is as simple as reading in using the filestream <<.
getting the string requires you to check until you reach a number, grabbing one character at a time and inserting that into a string. the last number is just like the first, using the filestream <<
This seems like homework so i'll let you put the rest together.
I would try a regular expression, something along these lines:
^([0-9]+)\s+(.+)\s+([0-9]+\.[0-9]+)$
I am not very good at regex syntax, but ([0-9]+) corresponds to a sequence of digits (this is the id), ([0-9]+\.[0-9]+) is the floating point number (price) and (.+) is the string that is separated from the two number by sequences of "space" characters: \s+.
The next step would be to check if you need this to work with prices like ".50" or "10".