crypto++ saving key to a byte queue and back to a key - c++

Question

I want to license my software using RSA encryption. My software has several executables and I plan to have each check the signature of a common license file before they proceed to do what it is they do. My goal is not to make it impossible to circumvent the licensing protection, just make it very difficult. I know that no one can make it impossible.
The executables currently run in a Windows environment, but they will only be released to work in a Linux environment.
My current plan is to put the public key within each executable for signature verification. The programs already have a 'safe' encrypted area in which to put the key.
My question for this post is, does my implementation method make sense? Is there another alternative? The alternative to have the public key in a separate file would allow a hacker to replace that file and use their own license file and signature. That doesn't seem as safe.
Additionally I've been reading through the crypto++ documentation and running the example code to try and accomplish this task. I cannot get any code to work that puts the key into a non-file sink and back again. All the examples write and read to files. I need to be able to save and load from a string or a byte queue. A simple attempt to do this is below, but when it runs I get this error, when r2.BERDecodePrivateKey() executes:
Unhandled exception at 0x7630c41f in MyRSA.exe: Microsoft C++ exception: CryptoPP::BERDecodeErr at memory location 0x002ef32c..
#include <osrng.h>
#include "rsa.h"
using CryptoPP::RSA;
#include <queue.h>
using CryptoPP::ByteQueue;
int myCode_key2ByteQueueToKey(void)
{
////////////////////////////////////////////////////////////////////////
// Generate the keys
AutoSeededRandomPool rnd;
CryptoPP::RSA::PrivateKey r1;
r1.GenerateRandomWithKeySize(rnd, 2048);
CryptoPP::RSA::PublicKey rsaPublic(r1);
////////////////////////////////////////////////////////////////////////
// Put the 'inner' part of the key into a ByteQueue - whatever that is.
ByteQueue queue;
r1.DEREncodePublicKey(queue);
////////////////////////////////////////////////////////////////////////
// Copy the byte queued inner key into another key
CryptoPP::RSA::PrivateKey r2;
r2.BERDecodePrivateKey(queue, false /*optParams*/, queue.MaxRetrievable());
////////////////////////////////////////////////////////////////////////
// Validate the key made the trip in and out of a byte queue ok.
if(!r1.Validate(rnd, 3)) printf("Validation of oringal key failed.\n");
if(!r2.Validate(rnd, 3)) printf("Validation of reloaded key failed.\n");
if(r1.GetModulus() != r2.GetModulus() ||
r1.GetPublicExponent() != r2.GetPublicExponent() ||
r1.GetPrivateExponent() != r2.GetPrivateExponent())
{
printf("Key didn't survive round trip in and out of a byte queue.");
}
return 0;
}
I do not have a fix for the above code. There's something I don't understand about library and as a result something's missing, but I've got to get a move on.
I thought I'd post an alternative I've found. Its an example on the Crypto++ wiki that puts the keys into strings (and not files) and back again. The round trip is shown to work.
http://www.cryptopp.com/wiki/BERDecode
Instead of
CryptoPP::RSA::PrivateKey
it uses
CryptoPP::RSAES_OAEP_SHA_Decryptor
and similarly for the public key. This allows the use of a member function AccessKey() which isn't available for the PrivateKey class.
If anyone has a fix for the original code, I urge you to please post it as it would help me better understand this library.

Answer 1

So basically you do this:
Generate 2048 bits private key
Encode a public key in DER format from the exponents of your private key to your bytequeue
Try to decode it as a private key from the bytequeue <-- here is the error
Validate the key...
You cannot decode a public key to a private key, some encoding flags differs.
As for CryptoPP::RSAES_OAEP_SHA_Decryptor
it's a good practice to use it for key generation as it is aware of safe primes.
It is also simpler to use for general decryption task as it includes everything you need in one object.
See http://www.cryptopp.com/wiki/Keys_and_Formats#High_Level_Objects
Hope it helped even it's a late answer ;)