I'm using gSOAP to use a service and I'm having the basic authentication string encoded in two different string, one from linux 32 bits and another one from AS400.
The code is the same.
I guess this could be the AS400 using EBCDIC, but I'm converting it to ASCII and it gives the same result.
Has someone had the same problem or similar?
This is the Linux encoded string:
c2FudGFuZGVyY29uc3VtZXI6Z2Vyc29hMg==
This is the AS400 encoded string:
ooGVo4GVhIWZg5aVoqSUhZl6h4WZopaB8g==
This is the encoding code:
if (!t)
t = (char*)soap_malloc(soap, output_length /*(n + 2) / 3 *
* 4 + 1 */);
if (!t)
return NULL;
p = t;
for (int i = 0, j = 0; i < input_length;) {
uint32_t octet_a = i < input_length ? (unsigned char)s[i++] : 0;
uint32_t octet_b = i < input_length ? (unsigned char)s[i++] : 0;
uint32_t octet_c = i < input_length ? (unsigned char)s[i++] : 0;
uint32_t triple = (octet_a << 0x10) + (octet_b << 0x08) + octet_c;
t[j++] = encoding_table[(triple >> 3 * 6) & 0x3F];
t[j++] = encoding_table[(triple >> 2 * 6) & 0x3F];
t[j++] = encoding_table[(triple >> 1 * 6) & 0x3F];
t[j++] = encoding_table[(triple >> 0 * 6) & 0x3F];
}
for (int i = 0; i < mod_table[input_length % 3]; i++)
t[output_length - 1 - i] = '=';
t[output_length - mod_table[input_length % 3] + 2] = '\0';
The code from the Linux system is working fine. From AS400 isn't working.
I guess it is some AS400 encoding problem, but I'm not sure and I have limited access to the AS400 system, so I can't trace a lot.
It is because you are converting a byte sequence which represents text encoded in EBCDIC.
The first string is the following bytes:
115,97,110,116,97,110,100,101,114,99,111,110,115,117,109,101,114,58,103,101,114,115,111,97,50
Which decodes in ASCII as santanderconsumer:gersoa2. You are going to have to change that password now, by the way.
The second Base64 string is the following bytes:
162,129,149,163,129,149,132,133,153,131,150,149,162,164,148,133,153,122,135,133,153,162,150,129,242
Checking the EBCDIC table at https://en.wikipedia.org/wiki/EBCDIC we can see that this is the same string.
Related
What is the best way to convert a unsigned 16 bit integer into ASCII HEX? I'm trying to integrate my Arduino with a serial communication protocol that expects the payload as an array of 2 byte ASCII HEX values. I'd like to be able to store each character of the HEX representation in a char array since the full message (with start and stop characters) needs to be CRC checksummed before being transmitted.
For example, to transmit a decimal value of 129, it would expect a string 0081 (0x00 and 0x81). To transmit a decimal value of 35822, it would expect a string of 8BEE.
I mostly work with Python so I'm not very familiar with casting to different data types.
Thanks!
EDIT: I'm actually working with a Teensy 4.0, just wrote Arduino out of habit
static const char *digits = "0123456789ABCDEF";
char *toHex(char *buff, uint16_t val, int withNULL)
{
buff[0] = digits[(val >> 12)];
buff[1] = digits[((val >> 8) & 0xf)];
buff[2] = digits[((val >> 4) & 0xf)];
buff[3] = digits[(val & 0xf)];
if(withNULL) buff[4] = 0;
return buff;
}
char *toHex1(char *buff, uint16_t val, int withNULL)
{
unsigned char d;
buff[0] = (char)((d = (val >> 12)) > 9 ? ('A' + d - 10) : ('0' + d));
buff[1] = (char)((d = ((val >> 8) & 0xf)) > 9 ? ('A' + d - 10) : ('0' + d));
buff[2] = (char)((d = ((val >> 4) & 0xf)) > 9 ? ('A' + d - 10) : ('0' + d));
buff[3] = (char)((d = (val & 0xf)) > 9 ? ('A' + d - 10) : ('0' + d));
if(withNULL) buff[4] = 0;
return buff;
}
I'm writing a tool for operations on long strings of 6 different letters (e.g. >1000000 letters), so I'd like to encode each letter in less than eight bits (for 6 letters 3 bits is sufficient)
Here is my code:
Rcpp::RawVector pack(Rcpp::RawVector UNPACKED,
const unsigned short ALPH_SIZE) {
const unsigned int IN_LEN = UNPACKED.size();
Rcpp::RawVector ret((ALPH_SIZE * IN_LEN + BYTE_SIZE - 1) / BYTE_SIZE);
unsigned int out_byte = ZERO;
unsigned short bits_left = BYTE_SIZE;
for (int i = ZERO; i < IN_LEN; i++) {
if (bits_left >= ALPH_SIZE) {
ret[out_byte] |= (UNPACKED[i] << (bits_left - ALPH_SIZE));
bits_left -= ALPH_SIZE;
} else {
ret[out_byte] |= (UNPACKED[i] >> (ALPH_SIZE - bits_left));
bits_left = ALPH_SIZE - bits_left;
out_byte++;
ret[out_byte] |= (UNPACKED[i] << (BYTE_SIZE - bits_left));
bits_left = BYTE_SIZE - bits_left;
}
}
return ret;
}
I'm using Rcpp, which is an R interface for C++. RawVector is in fact vector of char's.
This code works just perfectly - except it is too slow. I'm performing operations bit by bit while I could vectorize it somehow. And here is a question - is there any library or tool to do it? I'm not acknowledged with C++ tools.
Thanks in advance!
This code works just perfectly - except it is too slow.
Then you probably want to try out 4-bits/letter. Trading space for time. If 4-bits meets your compression needs (just 33.3% larger) then your code works on nibbles which will be much faster and simpler than tri-bits.
You need to unroll your loop, so optimizer could make something useful out of it. It will also get rid of your if, which kills any chance for quick performance. Something like this:
int i = 0;
for(i = 0; i + 8 <= IN_LEN; i += 8) {
ret[out_byte ] = (UNPACKED[i] ) | (UNPACKED[i + 1] << 3) | (UNPACKED[i + 2] << 6);
ret[out_byte + 1] = (UNPACKED[i + 2] >> 2) | (UNPACKED[i + 3] << 1) | (UNPACKED[i + 4] << 4) | (UNPACKED[i + 5] << 7);
ret[out_byte + 2] = (UNPACKED[i + 5] >> 1) | (UNPACKED[i + 6] << 2) | (UNPACKED[i + 7] << 5);
out_byte += 3;
}
for (; i < IN_LEN; i++) {
if (bits_left >= ALPH_SIZE) {
ret[out_byte] |= (UNPACKED[i] << (bits_left - ALPH_SIZE));
bits_left -= ALPH_SIZE;
} else {
ret[out_byte] |= (UNPACKED[i] >> (ALPH_SIZE - bits_left));
bits_left = ALPH_SIZE - bits_left;
out_byte++;
ret[out_byte] |= (UNPACKED[i] << (BYTE_SIZE - bits_left));
bits_left = BYTE_SIZE - bits_left;
}
}
This will allow optimizer to vectorize whole thing (assuming it's smart enough). With your current implementation i doubt any current compiler can find out, that your code loops after 3 written bytes and abuse it.
EDIT:
with sufficient constexpr / template magic you might be able to write some universal handler for body of the loop. Or just cover all small values (like write specialized template function for every bitcount from 1 to let's say 16). Packing values bitwise after 16 bits is overkill.
I want to base64 a big file (500MB)
I use this code but it doesn't work for a large file
I test CryptStringToBinary but it doesn't work too
what should I do????
The issue is clearly that there is not enough memory to store a 500 megabyte string in a 32-bit application.
The one solution is alluded to by the this link, which writes the data to a string. Assuming that the code works correctly, it is not that hard to adjust it to write to a file stream.
#include <windows.h>
#include <fstream>
static const wchar_t *Base64Digits = L"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
int Base64Encode(const BYTE* pSrc, int nLenSrc, std::wostream& pDstStrm, int nLenDst)
{
wchar_t pDst[4];
int nLenOut = 0;
while (nLenSrc > 0) {
if (nLenDst < 4) return(0);
int len = 0;
BYTE s1 = pSrc[len++];
BYTE s2 = (nLenSrc > 1) ? pSrc[len++] : 0;
BYTE s3 = (nLenSrc > 2) ? pSrc[len++] : 0;
pSrc += len;
nLenSrc -= len;
//------------------ lookup the right digits for output
pDst[0] = Base64Digits[(s1 >> 2) & 0x3F];
pDst[1] = Base64Digits[(((s1 & 0x3) << 4) | ((s2 >> 4) & 0xF)) & 0x3F];
pDst[2] = Base64Digits[(((s2 & 0xF) << 2) | ((s3 >> 6) & 0x3)) & 0x3F];
pDst[3] = Base64Digits[s3 & 0x3F];
//--------- end of input handling
if (len < 3) { // less than 24 src bits encoded, pad with '='
pDst[3] = L'=';
if (len == 1)
pDst[2] = L'=';
}
nLenOut += 4;
// write the data to a file
pDstStrm.write(pDst,4);
nLenDst -= 4;
}
if (nLenDst > 0) *pDst = 0;
return (nLenOut);
}
The only changes done were to write the 4 bytes to a wide stream instead of appending the data to a string
Here is an example call:
int main()
{
std::wofstream ofs(L"testfile.out");
Base64Encode((BYTE*)"This is a test", strlen("This is a test"), ofs, 1000);
}
The above produces a file with the base64 string VGhpcyBpcyBhIHRlc3Q=, which when decoded, produces This is a test.
Note that the parameter is std::wostream, which means any wide output stream class (such as std::wostringstream) will work also.
I am trying to base64 encode a unicode string. I am running into problems, after the encoding, the output is my string base64'ed however, there is null bytes at random places in throughout the code, I don't know why, or how to get them out.
Here is my Base64Encode function:
static char Base64Digits[] =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
int Base64Encode(const BYTE* pSrc, int nLenSrc, wchar_t* pDst, int nLenDst)
{
int nLenOut= 0;
while ( nLenSrc > 0 ) {
if (nLenOut+4 > nLenDst) return(0); // error
// read three source bytes (24 bits)
BYTE s1= pSrc[0]; // (but avoid reading past the end)
BYTE s2= 0; if (nLenSrc>1) s2=pSrc[1]; //------ corrected, thanks to jprichey
BYTE s3= 0; if (nLenSrc>2) s3=pSrc[2];
DWORD n;
n = s1; // xxx1
n <<= 8; // xx1x
n |= s2; // xx12
n <<= 8; // x12x
n |= s3; // x123
//-------------- get four 6-bit values for lookups
BYTE m4= n & 0x3f; n >>= 6;
BYTE m3= n & 0x3f; n >>= 6;
BYTE m2= n & 0x3f; n >>= 6;
BYTE m1= n & 0x3f;
//------------------ lookup the right digits for output
BYTE b1 = Base64Digits[m1];
BYTE b2 = Base64Digits[m2];
BYTE b3 = Base64Digits[m3];
BYTE b4 = Base64Digits[m4];
//--------- end of input handling
*pDst++ = b1;
*pDst++ = b2;
if ( nLenSrc >= 3 ) { // 24 src bits left to encode, output xxxx
*pDst++ = b3;
*pDst++ = b4;
}
if ( nLenSrc == 2 ) { // 16 src bits left to encode, output xxx=
*pDst++ = b3;
*pDst++ = '=';
}
if ( nLenSrc == 1 ) { // 8 src bits left to encode, output xx==
*pDst++ = '=';
*pDst++ = '=';
}
pSrc += 3;
nLenSrc -= 3;
nLenOut += 4;
}
// Could optionally append a NULL byte like so:
// *pDst++= 0; nLenOut++;
return( nLenOut );
}
Not to fool anyone, but I copied the function from here
Here is how I call the function:
wchar_t base64[256];
Base64Encode((const unsigned char *)UserLoginHash, lstrlenW(UserLoginHash) * 2, base64, 256);
So, why is there random null-bytes or "whitespaces" in the generated hash? What should be changed so that I can get rid of them?
Try something more like this. Portions copied from my own base64 encoder:
static const wchar_t *Base64Digits = L"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
int Base64Encode(const BYTE* pSrc, int nLenSrc, wchar_t* pDst, int nLenDst)
{
int nLenOut = 0;
while (nLenSrc > 0) {
if (nLenDst < 4) return(0); // error
// read up to three source bytes (24 bits)
int len = 0;
BYTE s1 = pSrc[len++];
BYTE s2 = (nLenSrc > 1) ? pSrc[len++] : 0
BYTE s3 = (nLenSrc > 2) ? pSrc[len++] : 0;
pSrc += len;
nLenSrc -= len;
//------------------ lookup the right digits for output
pDst[0] = Base64Digits[(s1 >> 2) & 0x3F];
pDst[1] = Base64Digits[(((s1 & 0x3) << 4) | ((s2 >> 4) & 0xF)) & 0x3F];
pDst[2] = Base64Digits[(((s2 & 0xF) << 2) | ((s3 >> 6) & 0x3)) & 0x3F];
pDst[3] = Base64Digits[s3 & 0x3F];
//--------- end of input handling
if (len < 3) { // less than 24 src bits encoded, pad with '='
pDst[3] = L'=';
if (len == 1)
pDst[2] = L'=';
}
nLenOut += 4;
pDst += 4;
nLenDst -= 4;
}
if (nLenDst > 0) *pDst = 0;
return (nLenOut);
}
The problem, from what I can see, is that as the encoder works, occasionally it is adding a value to a certain character value, for example, let's say U+0070 + U+0066 (this is just an example). At some point, these values equal the null terminator (\0) or something equivalent to it, making it so the program doesn't read past that point when outputting the string and making it appear shorter than it should be.
I've encountered this problem with my own encoding algorithm before, and the best solution appears to be to add more variability to your algorithm; so, instead of only adding characters to the string, subtract some, multiply or XOR some at some point in the algorithm. This should remove (or at least reduce the chances of) null terminators appearing where you don't want them. This may, however, take some trial-and-error on your part to see what works and what doesn't.
I have declared an array of bytes:
uint8_t memory[123];
which i have filled with:
memory[0]=0xFF;
memory[1]=0x00;
memory[2]=0xFF;
memory[3]=0x00;
memory[4]=0xFF;
And now i get requests from the user for specific bits. For example, i receive a request to send the bits in position 10:35, and i must return those bits combined in bytes. In that case i would need 4 bytes which contain.
response[0]=0b11000000;
responde[1]=0b00111111;
response[2]=0b11000000;
response[3]=0b00000011; //padded with zeros for excess bits
This will be used for Modbus which is a big-endian protocol. I have come up with the following code:
for(int j=findByteINIT;j<(findByteFINAL);j++){
aux[0]=(unsigned char) (memory[j]>>(startingbit-(8*findByteINIT)));
aux[1]=(unsigned char) (memory[j+1]<<(startingbit-(8*findByteINIT)));
response[h]=(unsigned char) (aux[0] | aux[1] );
h++;
aux[0]=0x00;//clean aux
aux[1]=0x00;
}
which does not work but should be close to the ideal solution. Any suggestions?
I think this should do it.
int start_bit = 10, end_bit = 35; // input
int start_byte = start_bit / CHAR_BIT;
int shift = start_bit % CHAR_BIT;
int response_size = (end_bit - start_bit + (CHAR_BIT - 1)) / CHAR_BIT;
int zero_padding = response_size * CHAR_BIT - (end_bit - start_bit + 1);
for (int i = 0; i < response_size; ++i) {
response[i] =
static_cast<uint8_t>((memory[start_byte + i] >> shift) |
(memory[start_byte + i + 1] << (CHAR_BIT - shift)));
}
response[response_size - 1] &= static_cast<uint8_t>(~0) >> zero_padding;
If the input is a starting bit and a number of bits instead of a starting bit and an (inclusive) end bit, then you can use exactly the same code, but compute the above end_bit using:
int start_bit = 10, count = 9; // input
int end_bit = start_bit + count - 1;