Access violation at ABABABAB after realloc - c++

I ran into this error and I don't quite understand what happens here.
I hope the codesnippet is sufficient to make my point.
I am modifying a callback to inject my own data into a response to a server.
Basically the callstack looks as follows:
mainRoutine(browseFunction(browseInternal), myBrowseFunction))
^ takes response ^ sends response ^ catches and modifies response
and sends callback
So what happens:
I have a server and a few hundred static nodes to be read. Now I want to support a highly dynamic messagingsystem, creating a node takes 300kb so is not an option as they are ten thousands, created and deleted within seconds. Therefore I inject the message into the response and pretend a node was read.
So much for the theory. This system worked in other context already, so there is no doubt the server can handle the fake response...
Some code - written in c++ but with the serverstack in C there are no new() or delete() methods available. All variables are initialized and filled with sensible values, as far as possible.
volatile int pNoOfNodesToAppend = 5;
Boolean xAdapter::Browse(BaseNode *pNode, BrowseContext* pBrowseCtx, int i)
{
[... some initializations....]
BrowseResult* pBrowseResult = &pResponse->Results[i];
int NoOfReferences = pBrowseResult->NoOfReferences + pNoOfNodesToAppend;
pResponse->NoOfResults = NoOfReferences;
// Version one
ReferenceDescription* refDesc = reinterpret_cast <ReferenceDescription *>(realloc(
pBrowseResult->References,
sizeof(OpcUa_ReferenceDescription) * NoOfReferences));
//Version two I tried just out of curiousity to see whether copying "by hand" would cause the programm to crash, as it didn't allocate enough memory - but no problem there.
/*
ReferenceDescription* refDesc = reinterpret_cast <ReferenceDescription *>(malloc( NoOfReferences * sizeof(ReferenceDescription)));
for (int k = 0; k < NoOfReferences; k++)
{
memcpy(&pBrowseResult->References[0], &pBrowseResult->References[k], sizeof(ReferenceDescription));
}
*/
int size = _msize(refDesc);
pBrowseResult->NoOfReferences = NoOfReferences;
if (refDesc != NULL)
{
pBrowseResult->References = refDesc;
}
else
{
return False;
/* Errorhandling ... */
}
[Fill with data... check for errors, handle errors]
return True;
}
I know this code looks cumbersome, but most of it cannot be done easier due to the underlying stack, as it gives me a hard time casting types to and forth, containing lots and lots of structures.
This code compiles and runs fine, once the callback is sent it crashes with an access violation at ABABABAB, which as I found out is a magic number used by Microsoft debug to mark guard bits around heapAlloc() memory (4 bits before and after).
See here: Magic_debug_values 1
Edit: This section is solved. I was just too blind to realize that we are talking about HEX here and thus too dumb to calculate my numbers correctly. So consider it unworthy of reading except for understanding of comments.
What really gives me headache is the memsize of the allocated new array.
NoOfReferences: 6
sizeof(ReferenceDescription) 0x00000080 unsigned int
(NoOfReferences * sizeof(ReferenceDescription)) 0x00000300 unsigned long
sizeof(*refDesc) 0x00000080 unsigned int //pointer to first element of array
_msize says:
size of (*refDesc) 0x00000300 int
Now WHY is the size of the newly allocated space 300? If my mind is not playing tricks on me then 6*80 is 480, even if there where 8 guard bits around every single element it would still be 72*6 > 300 bit. Anyway the system proceeds normally.
Now in the next chunk of code the structures in the array are filled with useful data and handed back to the Response structure.
The Callback is sent, the server ist going back to the ServerMain() and then crashes with first chance and unhandled exception
Unhandled exception at 0x5f95ed6a in demoserver.exe: 0xC0000005:
Access violation reading location 0xabababab.
Memory
0x5F95ED6A f3 a5 ff 24 95 84 ee 95 5f 90 8b c7 ba 03 00 00 00 83 e9 04 72 ó¥ÿ$..î._..Ǻ....ƒé.r
0x5F95ED7F 0c 83 e0 03 03 c8 ff 24 85 98 ed 95 5f ff 24 8d 94 ee 95 5f 90 .ƒà..Èÿ$.˜í._ÿ$.”î._.
0x5F95ED94 ff 24 8d 18 ee 95 5f 90 a8 ed 95 5f d4 ed 95 5f f8 ed 95 5f 23 ÿ$..î._.¨í._Ôí._øí._#
0x5F95EDA9 d1 8a 06 88 07 8a 46 01 88 47 01 8a 46 02 c1 e9 02 88 47 02 83 ÑŠ.ˆ.ŠF.ˆG.ŠF.Áé.ˆG.ƒ
0x5F95EDBE c6 03 83 c7 03 83 f9 08 72 cc f3 a5 ff 24 95 84 ee 95 5f 8d 49 Æ.ƒÇ.ƒù.rÌó¥ÿ$..î._.I
0x5F95EDD3 00 23 d1 8a 06 88 07 8a 46 01 c1 e9 02 88 47 01 83 c6 02 83 c7 .#ÑŠ.ˆ.ŠF.Áé.ˆG.ƒÆ.ƒÇ
0x5F95EDE8 02 83 f9 08 72 a6 f3 a5 ff 24 95 84 ee 95 5f 90 23 d1 8a 06 88 .ƒù.r¦ó¥ÿ$..î._.#ÑŠ.ˆ
0x5F95EDFD 07 83 c6 01 c1 e9 02 83 c7 01 83 f9 08 72 88 f3 a5 ff 24 95 84 .ƒÆ.Áé.ƒÇ.ƒù.rˆó¥ÿ$..
0x5F95EE12 ee 95 5f 8d 49 00 7b ee 95 5f 68 ee 95 5f 60 ee 95 5f 58 ee 95 î._.I.{î._hî._`î._Xî.
0x5F95EE27 5f 50 ee 95 5f 48 ee 95 5f 40 ee 95 5f 38 ee 95 5f 8b 44 8e e4 _Pî._Hî._#î._8î._.DŽä
0x5F95EE3C 89 44 8f e4 8b 44 8e e8 89 44 8f e8 8b 44 8e ec 89 44 8f ec 8b .D.ä.DŽè.D.è.DŽì.D.ì.
0x5F95EE51 44 8e f0 89 44 8f f0 8b 44 8e f4 89 44 8f f4 8b 44 8e f8 89 44 DŽð.D.ð.DŽô.D.ô.DŽø.D
0x5F95EE66 8f f8 8b 44 8e fc 89 44 8f fc 8d 04 8d 00 00 00 00 03 f0 03 f8 .ø.DŽü.D.ü........ð.ø
0x5F95EE7B ff 24 95 84 ee 95 5f 8b ff 94 ee 95 5f 9c ee 95 5f a8 ee 95 5f ÿ$..î._.ÿ”î._œî._¨î._
0x5F95EE90 bc ee 95 5f 8b 45 08 5e 5f c9 c3 90 8a 06 88 07 8b 45 08 5e 5f .î._.E.^_ÉÃ.Š.ˆ..E.^_
0x5F95EEA5 c9 c3 90 8a 06 88 07 8a 46 01 88 47 01 8b 45 08 5e 5f c9 c3 8d ÉÃ.Š.ˆ.ŠF.ˆG..E.^_ÉÃ.

So the mistake was found. The problem was neither the allocation nor the reassignment of the array but rather the fact, that the API didn't behave as expected and marshalled several callbacks. Trying to add mine by appeding it crashed and caused the exception as it was not ment to be done that way. (Solution and structure would be too complex to post it here.)
Thank you for your time and hints anyway, I learned a lot while chasing the errors!

Related

How to de-compress old data compressed with addCRYPT?

since the 2000th years we were using a compression-lib called addCRYPT.dll from a company called littleBIGware for our Windows-apps. Lib and company doesn't exists anymore, but users of our apps created lots of files up to now which are compressed with this lib.
Now for a new version I don't want to use this old lib, but must still be able to decompress the existing data.
I tried Zip, GZip and some others, but with no success.
Does anybody now, how to decompress these data? Here is a hex-dump of the first 64 bytes of some compressed blocks. There is a kind of a pattern visible, but what?
015104: 40 9c 00 00 42 5a 68 39 31 41 59 26 53 59 63 98 00 a8 00 30 ba 70 00 7f ff fa 10 8f fb e0 51 7c 77 db ed 7d bb 92 35 77 63 91 75 89 5d 60 65 91 26 b0 bb 32 ed aa 14 b5 a5 35 8d 34 ae dd c6 fa ...
017640: 40 9c 00 00 42 5a 68 39 31 41 59 26 53 59 2d fc 84 c1 00 49 2e 70 00 7f f8 7a 10 ef fb e0 5a ee 3e 9f 1e da ed a5 68 16 d5 ad ad 6b 3d cf 5e a4 f7 6d d6 d3 6b 47 bb ab 76 39 34 d8 ad bd ec ea ...
015760: 40 9c 00 00 42 5a 68 39 31 41 59 26 53 59 a5 a4 8a 00 00 4a 8b f0 00 7f ff fa 14 cf fb e0 51 dd b6 d5 ad 5e d6 fa 9b b9 bb 1a 77 77 6e dd 54 80 52 aa e5 98 cd 6d ad 92 18 bb 7a cb af 53 b1 da ...
015336: 40 9c 00 00 42 5a 68 39 31 41 59 26 53 59 4f 3c 3e 6f 00 2d cd f0 00 7f ff fe 10 8f fb e0 53 9f 6f 66 cb ec f6 6f 64 a2 da da 61 8d 2a 85 6b 4d 04 80 03 2b 1b 20 16 d9 de cf 6a b3 d5 a0 32 52 ...
016240: 40 9c 00 00 42 5a 68 39 31 41 59 26 53 59 af b2 df 80 00 27 bd f0 00 7f fd fa 10 8f fb e0 57 bd 5e 9f 19 f7 67 02 35 b0 dd ba 5d 67 b3 72 ce cd db 7a eb ba ce bd de f6 af 0c d6 ba d7 57 dc 1c ...

Decoding an RSA PublicKey in C++/Qt

I had to port a Java app to C++/Qt, but I have a problem about the RSA encryption.
Basically my app ask the server for a key and the the server return it like this :
30 81 9f 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03
81 8d 00 30 81 89 02 81 81 00 b4 74 05 71 53 79 39 16 29
fc 17 d3 dd 27 c0 84 72 1e 4a 66 b8 43 e8 57 58 f6 13 65
3c a6 85 ff e1 c3 b3 76 11 3e 10 33 00 7d a4 ab 94 d3 86
a2 99 71 d6 c9 99 a6 54 ec 3c 32 de 25 60 61 c3 84 6d 00
32 6a 4e 7c f7 25 38 3c 12 f7 f7 c0 08 e0 8a ca 6f dc 1d
3c 37 bc e9 77 70 11 e7 9e 67 f0 96 d6 df a1 37 d2 8e 83
1c 75 cf 86 ff 39 bb 53 a6 63 f7 ce 99 eb b3 f2 20 39 37
52 41 31 6e 9f 02 03 01 00 01
After that, all i have to do is to encrypt the credentials and sending them to the server. But I can't find any way to use that key.
The Java version look like this
byte[] key;
// Get the key from the server
// ...
//
KeyFactory kf;
EncodedKeySpec eks = new X509EncodedKeySpec(key);
PublicKey localPublicKey = kf.generatePublic(eks);
I've looked both OpenSSL and QSsl libs but I didn't found anything like this.
If anyone had any ideas ...

Example of LUC algorithm with Crypto++

I am looking for an example of LUC algorithm, but I can't find anything. I know that it is in Crypto++, but I don't know C++ too well to use it.
I look for an example of algorithm of LUC...
It kind of depends on what you want to do. You might want to browse luc.h to see some of the things Crypto++ offers for LUC. There's a LUCES, a LUCSS and a LUC_IES. The *ES is encryption scheme, the *SS is a signature scheme, and the *IES is an integrated encryption scheme (which includes a key agreement algorithm and mask function).
Generally speaking, LUC is a public key encryption system. Using it is like using any other public key encryption system offered by Crypto++. That's because all the public key encryption systems inherit from the same classes (more correctly, base interfaces). You can see the design in the comments for file pubkey.h.
$ grep -R LUCES *
...
typedef LUCES<OAEP<SHA> >::Decryptor LUCES_OAEP_SHA_Decryptor;
typedef LUCES<OAEP<SHA> >::Encryptor LUCES_OAEP_SHA_Encryptor;
And that's pretty much all you need, though you may not know it.
Here's the easier problem to solve. How do you perform RSA encryption in Crypto++?
$ grep -R RSAES *
...
typedef RSAES<PKCS1v15>::Decryptor RSAES_PKCS1v15_Decryptor;
typedef RSAES<PKCS1v15>::Encryptor RSAES_PKCS1v15_Encryptor;
typedef RSAES<OAEP<SHA> >::Decryptor RSAES_OAEP_SHA_Decryptor;
typedef RSAES<OAEP<SHA> >::Encryptor RSAES_OAEP_SHA_Encryptor;
If you find an RSAES_PKCS1v15_Decryptor or RSAES_OAEP_SHA_Decryptor example, you just copy/replace with LUCES_OAEP_SHA_Decryptor and it will work just fine. And if you find an RSAES_PKCS1v15_Encryptor or RSAES_OAEP_SHA_Encryptor example, you just copy/replace with LUCES_OAEP_SHA_Encryptor and it will work just fine.
You can find the examples of using RSAES_OAEP_SHA_Encryptor and RSAES_OAEP_SHA_Decryptor on the Crypto++ wiki page for RSA Encryption Schemes. Or you can use the ECIES examples at Elliptic Curve Integrated Encryption Scheme (remember, all the public key systems inherit from the same base interfaces, so they all have the same methods and you use them the same way).
This should get you started. It creates a private key, saves it, then creates a public key, and saves it.
try
{
AutoSeededRandomPool prng;
FileSink fs1("lucs-private.der", true);
FileSink fs2("lucs-public.der", true);
InvertibleLUCFunction params;
params.GenerateRandomWithKeySize(prng, 2048);
LUC::PrivateKey privateKey(params);
privateKey.DEREncode(fs1);
LUCES_OAEP_SHA_Decryptor decryptor(privateKey);
// ...
LUC::PublicKey publicKey(params);
publicKey.DEREncode(fs2);
LUCES_OAEP_SHA_Encryptor encryptor(publicKey);
// ...
}
catch(CryptoPP::Exception& ex)
{
cerr << ex.what() << endl;
}
If you don't want to use InvertibleLUCFunction, the do something like this to generate the key. Note: RSA has an InvertibleRSAFunction.
LUC::PrivateKey privateKey;
privateKey.Initialize(prng, 2048);
...
LUC::PublicKey publicKey(privateKey);
...
An here's yet another way to do it:
FileSink fs1("lucs-private.der", true);
FileSink fs2("lucs-public.der", true);
LUCES_OAEP_SHA_Decryptor decryptor;
decryptor.AccessKey().Initialize(prng, 2048);
decryptor.AccessKey().DEREncode(fs1);
...
LUCES_OAEP_SHA_Encryptor encryptor(decryptor);
encryptor.AccessKey().DEREncode(fs2);
...
And here's a dump of the private key created by the test program:
$ dumpasn1 lucs-private.der
0 662: SEQUENCE {
4 1: INTEGER 0
7 257: INTEGER
: 00 B8 7A CA 6A 61 D9 CF 2F D8 89 5C A4 7D 74 7B
: AC F5 10 4C 3D 95 BF DD 2E F5 4E E5 F4 20 CF CD
: 44 7F C7 27 41 48 6B 83 E0 7C D9 66 16 8D 54 36
: 97 B9 CE 2D 80 A6 F6 E5 25 87 83 6E B9 41 45 DC
: 2A EB EC 4E EC D9 C0 17 B4 E0 04 F0 58 61 60 F8
: 87 18 27 16 58 BA 56 4E DD 9B C8 CD 18 46 28 38
: A2 6A A6 14 36 D0 A6 FF 9C B8 A8 B5 0F 3A 11 B5
: 00 08 44 B3 31 58 AF 01 F8 57 17 E8 FC 68 B2 5F
: [ Another 129 bytes skipped ]
268 1: INTEGER 17
271 129: INTEGER
: 00 C8 DF 47 D0 B2 6F C2 1A E4 B7 E8 3D 12 BB FF
: 04 F7 34 40 A0 0E ED DC F7 24 7B D9 46 EE 10 C4
: D5 E2 9C 93 05 CF 13 53 40 F4 50 EC 1F 6D D7 33
: FF FF 46 42 88 8D FC F4 EE 7F 0C 8B 71 71 51 D2
: 3C 32 E3 9A 11 B7 D8 CF EA 10 B2 07 49 3F 93 CD
: A0 3F 71 A9 23 27 35 1F 6A C9 1D FE CE 24 75 33
: 8F 53 71 B9 0B DE BC 05 93 98 A3 EA 94 8E 04 B1
: 29 A1 4F 4C 82 34 7A 08 3A 0E 07 98 8B 00 30 D7
: 5B
403 129: INTEGER
: 00 EB 1B D0 EF 5C 0F FC FC B7 56 A7 70 8C AA B7
: A6 90 C8 1F AA AD A0 0B 66 E5 33 75 F2 BE 68 35
: 29 2E 57 AC E0 E0 C8 04 A7 C4 13 1D 10 30 8B 50
: 20 17 0C 83 A7 14 4A 7D 25 31 77 50 66 08 36 13
: BE 9D C0 4E F4 44 74 7A BB D2 92 D0 F7 AE 7C EB
: 8E 84 5C 27 61 2C C9 7A D1 D0 C5 A0 13 98 96 E3
: 76 CD B0 E7 E8 7E 4E 0A 2D 00 86 07 57 DB 8A 51
: 1E 59 76 EA 88 44 4D DA F3 D6 AB 75 CB A6 45 F3
: F3
535 128: INTEGER
: 2E 6A AA BA B4 E8 DD 11 2D 31 A4 D5 F7 08 AB E3
: 1A 9A 15 58 AE C8 59 BE C4 75 85 90 6D 5D A4 18
: 39 27 8F FF 1C 9A FD 0F 0C 29 05 98 9C 16 FE 84
: A4 5C 85 15 F7 98 E6 D5 5B 23 CA 2F A2 27 8A 00
: 6E B1 BB 02 6E 93 53 85 30 30 61 F5 1C 49 5D 19
: EF DF CD 6F 11 7C 6D DC AE F6 A2 06 53 BB 7E 03
: C3 E5 4E E9 59 E0 D8 5F C3 28 0E E0 17 5C 63 6E
: 8E A6 18 FC AD A5 9B 08 D1 8B 7B 28 9D E2 CF E2
: }
0 warnings, 0 errors.

RSAES_OAEP_SHA_Decryptor MaxPlaintextLength returning zero

I have to write a small authentication server that, when a client connects, sends a RSA public exponent and modulus to it, and the client returns the username and password encrypted with them. The part where I get the exponent and modulus works fine, the client receives them, and sends the encoded block back; that's not a problem. The problem is in decoding it: the RSA Decryptor always returns 0 as MaxPlaintextLength, and no matter how many different ways I try to move the data around, I can't get it to work.
I'm providing hex dumps of the different parts after the code.
// These are static in the namespace to keep the key loaded.
static CryptoPP::AutoSeededRandomPool rsaRng;
static CryptoPP::InvertibleRSAFunction rsaParameters;
static CryptoPP::RSA::PrivateKey rsaPrivateKey(rsaParameters);
// This is done when the client connects.
{
rsaPrivateKey.GenerateRandomWithKeySize(rsaRng, 1024);
// This is where GetPublicExponent() and GetModulus() are
// encoded and sent to the client. They are sent correctly
// and I receive the encrypted data in a char buffer.
}
// This runs when I receive the encrypted data, which starts
// at &dataBuffer[7] and has a length of rsaPayloadLen.
{
int rsaPlainSize;
byte *rsaBuffer;
rsaBuffer = new byte[rsaPayloadLen];
memcpy(&rsaBuffer[0], &dataBuffer[7], rsaPayloadLen);
CryptoPP::SecByteBlock rsaCypher(rsaBuffer, rsaPayloadLen);
CryptoPP::RSAES_OAEP_SHA_Decryptor rsaDecryptor(rsaPrivateKey);
// At this point I inspected rsaCypher and it does contain the
// expected binary data, and rsaCypher.size() returns 256.
rsaPlainSize = rsaDecryptor.MaxPlaintextLength(rsaCypher.size());
// rsaPlainSize is 0. I have no idea why!
}
Hex dumps of everything at the time of calling MaxPlaintextLength:
rsaPrivateKey.GetPublicExponent():
000000: 11 .
rsaPrivateKey.GetPrivateExponent():
000000: 10 7a fd fd 9e a9 72 8c c3 5d 5b 80 e8 f4 6f bc .z....r..][...o.
000010: bc 6a 7a 51 4f 9f af d3 e4 76 a5 4a 9d fe 17 37 .jzQO....v.J...7
000020: 03 cf 82 24 33 e2 a0 d8 97 26 0a 6b ac 9d b1 de ...$3....&.k....
000030: 39 d5 3a 93 aa 65 66 be 17 43 3c 00 20 77 68 0a 9.:..ef..C<. wh.
000040: ac 2f 77 1e b8 c4 7f 64 52 54 7c 17 54 b6 e6 a4 ./w....dRT|.T...
000050: 95 49 60 7b 7b 16 6a 41 72 54 03 a2 2d 3a 80 8b .I`{{.jArT..-:..
000060: aa 74 fa 77 22 5d 0a d9 81 b2 b2 48 01 db 43 e8 .t.w"].....H..C.
000070: 16 1c c4 c3 a6 bf 45 7e 90 d3 6a 37 10 40 9f 71 ......E~..j7.#.q
rsaPrivateKey.GetModulus():
000000: d2 20 26 61 a6 f0 74 82 ba e6 4e ab 9a 2c 90 a6 . &a..t...N..,..
000010: 62 4d 97 8c b7 34 01 cd a0 e8 bb 77 5e 67 a7 fd bM...4.....w^g..
000020: 70 95 bb 4d 95 89 82 c9 87 25 04 dc d8 da 9b d1 p..M.....%......
000030: 61 5e aa da bc 8c dd f7 a8 99 3d 01 9d f2 6e 89 a^........=...n.
000040: e4 75 ec 91 31 e9 86 f4 da 43 4a ca a4 66 6b 04 .u..1....CJ..fk.
000050: c2 c9 a1 18 1d fa 81 b0 6e ef a5 13 04 44 88 89 ........n....D..
000060: 42 41 be 9c 7c 77 75 96 50 07 70 ad eb 60 e5 05 BA..|wu.P.p..`..
000070: aa a8 d8 27 03 28 cf bb c7 f5 cb 0d b3 b3 96 7f ...'.(..........
rsaPrivateKey.GetPrime1():
000000: d7 9e af ac e4 04 42 e4 58 9c 39 19 0e 56 7c ef ......B.X.9..V|.
000010: b3 bf b6 26 73 25 d8 ab d7 5e d1 e0 56 49 ae 66 ...&s%...^..VI.f
000020: c4 d8 81 bc d0 be c2 ef f4 6a 09 72 ef 72 35 7e .........j.r.r5~
000030: 15 f4 f9 3b f8 be f9 3a a1 0d 3e d0 eb c8 34 11 ...;...:..>...4.
rsaPrivateKey.GetPrime2():
000000: f9 7a 0e 1c 9a 1b eb d1 67 f1 e3 88 1d f3 f1 62 .z......g......b
000010: 9f a2 5c cb 49 76 de 42 25 e1 a4 de ed 50 f3 2d ..\.Iv.B%....P.-
000020: c0 15 c3 70 b5 96 68 51 25 f7 06 24 e4 43 0d b8 ...p..hQ%..$.C..
000030: 7a c5 12 2c 7c 63 20 73 70 61 01 fe b8 b3 71 8f z..,|c spa....q.
Plain text buffer that was encrypted:
000000: 73 74 72 69 6e 67 62 75 66 66 65 72 00 00 00 00 stringbuffer....
000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ..............
rsaCypher:
000000: 0e 9e bd 34 10 16 98 a5 b8 e4 0c 9b 4f 23 71 6d ...4........O#qm
000010: af d6 e8 c1 4d 97 b9 32 cb 25 eb 01 fe 4f 5c 79 ....M..2.%...O\y
000020: 2d d8 32 c4 4f fa e9 2e 58 dd fd 37 7f 08 97 d8 -.2.O...X..7....
000030: 95 bb 6f 04 46 fa 83 77 05 01 43 75 ca be b4 4a ..o.F..w..Cu...J
000040: 60 f9 e7 4a 91 3d bc ac fb e9 41 f3 9d b7 df d3 `..J.=....A.....
000050: a7 03 80 3a 7f 35 98 46 ca 06 b1 f3 d1 7b 56 83 ...:.5.F.....{V.
000060: 1b 00 7d 97 59 39 be 46 d5 cf 6d 2c b3 a7 8e 30 ..}.Y9.F..m,...0
000070: 39 ca ca d5 59 a2 71 43 e7 7e 75 b3 3c d6 a3 a5 9...Y.qC.~u.<...
000080: aa 89 e3 e9 32 e1 a9 c1 a5 a8 f5 66 be 7f c9 ba ....2......f....
000090: 65 35 0f 61 a0 d4 fa c7 ac 8e 28 7c 39 26 3f 01 e5.a......(|9&?.
0000a0: 34 ad 82 69 5e c4 ab 92 48 47 42 04 02 48 79 c4 4..i^...HGB..Hy.
0000b0: 39 6e f2 2c 7c 19 71 34 36 38 1c eb c1 f2 33 f0 9n.,|.q468....3.
0000c0: 49 b9 7e bb c3 16 ed d7 f7 3e 10 a7 cc 2b 8c 31 I.~......>...+.1
0000d0: f1 17 c7 a5 49 ce dd a3 c6 e2 9c 3c 2f 37 e4 97 ....I......</7..
0000e0: ac b7 24 17 b3 f8 75 6f 2a 85 cb 23 7a e1 77 72 ..$...uo*..#z.wr
0000f0: 02 0b 90 28 9b 9b ff 5d 6f 9b 11 11 d3 8b dd 4b ...(...]o......K
rsaCypher.size(): 256
rsaDecryptor.MaxPlaintextLength(rsaCypher.size()): 0
I'm really completely at a loss and the only reason I've spent the whole weekend fighting this is because I'm going to need CryptoPP later for Blowfish and Adler32, otherwise I would have just used another RSA library.
RSAES_OAEP_SHA_Decryptor MaxPlaintextLength returning zero...
Your calculation of rsaPayloadLen is probably not correct. I would suggest you check the size of the ciphertext before you transmit it and after you recover it. I think its size is rsaPayloadLen in you code above.
From the comments in cryptlib.h:
//! maximum length of plaintext for a given ciphertext length
/*! \note This function returns 0 if ciphertextLength is not valid (too long or too short). */
virtual size_t MaxPlaintextLength(size_t ciphertextLength) const =0;
I cannot duplicate when I take plain text, encrypt it, and then pass ciphertext.size() into MaxPlaintextLength. I can duplicate if I pass ciphertext.size() - 1 into MaxPlaintextLength.
As a work around to get you a little further along, you might try using FixedMaxPlaintextLength(). It takes no arguments and only depends upon the public key length. You can use it to find the largest buffer based on the key. But you could fail on decryption.
You should probably show us your encryption code, too. You might not be sizing a buffer correctly.
The whole problem was the key size. The doc said I should use a 1024-bit key, but the other program was expecting a 1152-byte key; apparently that change was never documented. Everything works fine after changing a single line:
rsaPrivateKey.GenerateRandomWithKeySize(rsaRng, 1152);
Leaving this answer here in case someone else ever has this problem. Check the key sizes, and don't trust the documentation!

Writing BMP data getting garbage

I'm working on understanding and drawing my own DLL for PDF417 (2d barcodes). Anyhow, the actual drawing of the file is perfect, and in correct boundaries of 32 bits (as monochrome result). At the time of writing the data, the following is a memory dump as copied from C++ Visual Studio memory dump of the pointer to the bmp buffer. Each row is properly allocated to 36 wide before the next row.
Sorry about the wordwrap in the post, but my output was intended to be the same 36 bytes wide as the memory dump so you could better see the distortion.
The current drawing is 273 pixels wide by 12 pixels high, monochrome...
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02 eb 00 00
00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02 eb 00 00
00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02 eb 00 00
00 ab 85 7e d0 29 e8 14 f4 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79 bc 5c 0a 8e 02 eb 00 00
00 ab 85 7e d0 29 e8 14 f4 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79 bc 5c 0a 8e 02 eb 00 00
00 ab 85 7e d0 29 e8 14 f4 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79 bc 5c 0a 8e 02 eb 00 00
00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c db ce 6d 5f be 02 eb 00 00
00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c db ce 6d 5f be 02 eb 00 00
00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c db ce 6d 5f be 02 eb 00 00
Here is the code to WRITE the file out -- verbatim immediately at the time of the memory dump from above
FILE *stream;
if( fopen_s( &stream, cSaveToFile, "w+" ) == 0 )
{
fwrite( &bmfh, 1, (UINT)sizeof(BITMAPFILEHEADER), stream );
fwrite( &bmi, 1, (UINT)sizeof(BITMAPINFO), stream );
fwrite( &RGBWhite, 1, (UINT)sizeof(RGBQUAD), stream );
fwrite( ppvBits, 1, (UINT)bmi.bmiHeader.biSizeImage, stream );
fclose( stream );
}
Here's what ACTUALLY Gets written to the file.
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab a8 61 d7 18 ed 18 f7 a3 89 1c dd 70 86 f5 f7 1a 20 91 3b c9 27 e7 67 12 1c 68 ae 3c b7 3e 02 eb 00 00
00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0d 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02 eb 00
00 00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0d 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02 eb
00 00 00 ab 81 4b ca 07 6b 9c 11 40 9a e6 0c 76 0d 0a fc a3 33 70 bb 30 55 87 e9 c4 10 58 d9 ea 0d 48 3e 02
eb 00 00 00 ab 85 7e d0 29 e8 14 f4 0d 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79 bc 5c 0d 0a
8e 02 eb 00 00 00 ab 85 7e d0 29 e8 14 f4 0d 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79 bc 5c
0d 0a 8e 02 eb 00 00 00 ab 85 7e d0 29 e8 14 f4 0d 0a 7a 05 3c 37 ba 86 87 04 db b6 09 dc a0 62 fc d1 31 79
bc 5c 0d 0a 8e 02 eb 00 00 00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c
db ce 6d 5f be 02 eb 00 00 00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c
db ce 6d 5f be 02 eb 00 00 00 ab 85 43 c5 30 e2 26 70 4a 1a f3 e4 4d ce 2a 3f 79 cd bc e6 de 73 6f 39 b7 9c
db ce 6d 5f be 02 eb 00 00
Notice the start of the distortion with the "0d" in the result from reading the file back in the 4th line, about the 15th byte over... Then, there are a few more staggered around which in total, skew the image off by 9 bytes worth...
Obviously, the drawing portion is working ok as everything remains properly aligned in memory for the 12 lines.
Shouldn't you open the file in a compound mode i.e. writable & binary as in wb+?
Notice the start of the distortion with the "0d"
That's ASCII code for Carriage Return (CR) -- added on some OSes with newline (where a newline is actually a sequence of CR/LF). This should go away once you start writing the output in binary mode.
Your code looks neat otherwise. Cheers!
Your 0x0A (\n) gets converted to DOS format 0x0D0A (\r\n), becouse you're write the file in text mode. Switch to binary mode.
I actually just did a similar thing in java (printing bmp data to a thermal receipt printer). There are a couple of things i want to share with you:
bmp image data != an image format from microsoft. the MS bitmap has about 54 bytes of header information before any image data. (i spent a day or two working on this before I realized the difference)
bmp image data reads left to right, top to bottom, with the most significant bit on the left.
make sure the barcode image has a bitdepth of 1. this means 1 bit = 1 pixel. hexidecimal "ab" is 10101011 in binary, those 8 pixels will be filled in accordingly.
if you have a barcode 36 bytes wide, the barcode resolution is 288 x 12, not 273 x 12. (36 * 8 = 288).
the image data should be 432 bytes in size (12 rows of 36 bytes).
i dont know what this means:
Anyhow, the actual drawing of the file is perfect, and in correct boundaries of 32 bits (as monochrome result).
monochrome means its either 1 color or another. the pixel (think bit) is either filled in or it isnt.
Hope this helps