Background
I am designing a system which enables the development of dynamic authentication schemes for a user of static web content. The motivation is to pre-generate large amounts of complex-to-generate, yet sensitive web-content, and then serve it statically with cookie-based (embedding reversably encrypted information) authentication in place, enforced by the web-server inately. Using an AEAD-mode encryption primitive.
The Problem
I need to generate IVEC's and keys that are valid for a duration of time, say one week (the current-valid pair). and that past IVECs/Keys are also valid for say 2 weeks(historically-valid) and any data encrypted with the historically valid secrets will just be re-encrypted with the current-valid IVEC/KEY.
What I need is a deterministic CSPRNG that seeds of a random number and a passphrase and that can produce in an indexed fashion 64-bit or 128-bit blocks of numbers. If I use a weeks-since-"jan 1 1970" as one of the index element of my hypothetical CSPRNG I should be able to build a system that innately changes keys automatically as time goes by.
Approach I am Considering
Now I don't see such functionality in cryptopp, or I do now know the terminology well enough, and as cryptopp is the most advanced of the encryption libraries out there, I don't have confidence I will find another one. So, If I can't find an implementation out there, I should roll my own. Will generating a static string structure out of the concatinated data and then hashing it (shown below) do the trick ?
RIPEMD160(RandomPreGeneratedFixedNonce:PassPhrase:UInt64SinceEpoch:128BitBlockIndexNumber);
Note: The blocknumbers will be assigned and have a regular structure, so for example for a 128-bit digest, the first 64-bits of block 0 will be for the ivec, and all of element 1 for the 128-bit key.
Is this a sound approach (--.i.e, cryptographically secure) ?
-- edit: post accept comment --
After some reflection, I have decided to merge what I originally considered the passphrase and the nonce/salt into a 16-byte (cryptographicall strong) key, and use the techniques outlined in the PKCS #5 to derive multiple time-based keys. There isn't a need for a salt, as passphrases aren't used.
Interesting question.
First, your Initial Vectors don't have to be cryptographically strong random quantities, but they should be unique per-message. The IV is really just a kind of salt value that ensures that similar messages encrypted using the same key don't look similar once they're encrypted. You can use any quick pseudo-random generator to generate the IV, and then send it (preferably encrypted) along with the encrypted data.
The keys, of course, should be as strong as you can practically make them.
Your proposal to hash a text string containing a nonce, passphrase, and validity data seems to me to be very reasonable -- it's broadly in line with what is done by other system that use a passphrase to generate a key. You should hash more many times -- not just once -- to make the key generation computationally expensive (which will be a bigger problem for anyone trying to brute-force the key than it will for you).
You might also want to have a look at the key-generation scheme set out in PKCS#5 (e.g. at http://www.faqs.org/rfcs/rfc2898.html) which is implemented in cryptopp as PasswordBasedKeyDerivationFunction. This mechanism is already widely used and known to be reasonable secure (note that PKCS#5 recommends hashing the passphrase data at least 1000 times). You could just append your validity period and index data to the passphrase and use PasswordBasedKeyDerivationFunction as it stands.
You don't say what encryption algorithm you propose to use to encrypt the data, but I would suggest that you should pick something widely-used and known to be secure ... and in particular I'd suggest that you use AES. I'd also suggest using one of the SHA digest functions (maybe as an input to PasswordBasedKeyDerivationFunction). SHA-2 is current, but SHA-1 is sufficient for key generation purposes.
You also don't say what key length you're looking to generate, but you should be aware that the amount of entropy in your keys depends on the length of the passphrase that you use, and unless the passphrase is very long that will be much less than the keylength ideally requires.
The weakest link in this scheme is the passphrase itself, and that's always going to limit the level of security you can achive. As long as you salt your data (as you are doing) and make the key-generation expensive to slow down brute-force attacks you should be fine.
What I need is a deterministic CSPRNG that seeds of a random number and a passphrase and that can produce in an indexed fashion 64-bit or 128-bit blocks of numbers. If I use a weeks-since-"jan 1 1970" as one of the index element of my hypothetical CSPRNG I should be able to build a system that innately changes keys automatically as time goes by.
Well, I think part of the solution is to use a non-time based generator. That way, if both sides start with the same seed, then they both produce the same random stream. You can layer your "weeks since Week 1, 1970" logic on top of that.
To do that, you would use OFB_mode<T>::Encryption. It can be used as a generator because OFB mode uses AdditiveCipherTemplate<T>, which derives from RandomNumberGenerator.
In fact, Crpyto++ uses the generator in test.cpp so that results can be reproduced if something fails. Here's how you would use OFB_mode<T>::Encryption. It also applies to CTR_Mode<T>::Encryption:
SecByteBlock seed(32 + 16);
OS_GenerateRandomBlock(false, seed, seed.size());
for(unsigned int i = 0; i < 10; i++)
{
OFB_Mode<AES>::Encryption prng;
prng.SetKeyWithIV(seed, 32, seed + 32, 16);
SecByteBlock t(16);
prng.GenerateBlock(t, t.size());
string s;
HexEncoder hex(new StringSink(s));
hex.Put(t, t.size());
hex.MessageEnd();
cout << "Random: " << s << endl;
}
The call to OS_GenerateRandomBlock fetches bytes from /dev/{u|s}random and then uses that as a simulated shared seed. Each run of the program will be different. Within each run of the program, it prints similar to the following:
$ ./cryptopp-test.exe
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
Random: DF3D3F8E8A21C39C0871B375013AA2CD
There's another generator available that does the same, but its not part of the Crypto++ library. Its called AES_RNG, and its based on AES-256. Its a header only implementation, and you can find it at the Crypto++ wiki under RandomNumberGenerator.
Also see the topic Reproducibility for RandomNumberGenerator class on the Crypto++ wiki.
Related
I am struggling with finding examples for OpenWall's bcrypt implementation that use crypt_gensalt_rn and crypt_r that also explain in depth exactly what is happening in terms of input, settings etc and more generally the cryptographic component. crypt and crypt_gensalt are not really viable due to them not being MT-Safe so I am trying to better understand the rn, ra, and r implementations.
Summary: I want to better understand what the
the parameters to the function are and what their purposes are.
What are the best practices cryptographically for password hashing using these re-entrant ones and how to use them safely in a MT environment so I am not one of those developers who just uses crypto functions without understanding the implications of them and pitfalls.
when generating random bytes for the salt generator, what is a cryptographically strong source for random bytes?
I am also open to recommendations to other libraries etc. but they need to be production ready.
Tried to solve this on my own. Here is what I found out:
1a. crypt_gensalt_rn:
prefix = Prefix code in the form of $<Algorithim>$ ex. $2a$
count : The number of rounds you want the hashing algorithim to run. Set this value by response time. (ie. if you want to finish a hash in 1s, then run a benchmark and figure out the # of rounds needed to respond in 1s)
rbytes, nrbytes : rbytes is a set of random bytes and nrbytes is the length of said char array of randombytes. You can pass NULL to rbytes and 0 to nrbytes to have the OS get them for you (best practice typically unless OS doesn't have random bytes hardware needed which can result in a security risk)
output, output_size : This is defined by each algorithm individually. In the case of bcrypt it is 32 or the length of the setting string for crypt_ra. This stores in the format of $<Algorithim>$<rounds>$<salt> for bcrypt and many others.
1b. crypt_ra(const char *phrase, const char *setting, void **data, int *size);
phrase : the text you want to hash
setting: the setting string (ie. char* output) made in crypt_gensalt_rn.
data : a pointer to a generic crypt_struct used by most linux libraries. This is where crypt_ra will allocate a struct you need to free.
size : A pointer to an integer that crypt_ra will set to the length in bytes of the crypt struct.
Ra and RN are safe in Multithreaded environments. Make sure if your server doesn't support Random Byte Generation via hardware there can be a security risk (this was reported). Set rounds to a time you want it to take to verify a password not a # of rounds.
You can use the OS if it has the appropriate hardware. Otherwise you can use RNG like mersenne twister.
I'm trying to encrypt a message with a public key and decrypt the cipher with the private key using crypto++ like this in the shell:
openssl rsautl -encrypt -inkey id_rsa.pub.pem -pubin -in message -out message.enc
and
openssl rsautl -decrypt -inkey id_rsa.pem -in message.enc -out message.dec
Encryption/Decryption is done in separate applications. I started with the example from https://www.cryptopp.com/wiki/RSA_Cryptography.
My code:
std::string publicEncrypt(std::string const& plain) {
auto cipher = std::string{};
CryptoPP::RSAES_OAEP_SHA_Encryptor e(getPublicKey());
CryptoPP::StringSource(plain, true,
new CryptoPP::PK_EncryptorFilter(CryptoPP::NullRNG(), e,
new CryptoPP::StringSink(cipher)));
return cipher;
}
std::string privateDecrypt(std::string const& cipher) {
auto decrypted = std::string{};
CryptoPP::RSAES_OAEP_SHA_Decryptor d(getPrivateKey());
CryptoPP::StringSource(cipher, true,
new CryptoPP::PK_DecryptorFilter(CryptoPP::NullRNG(), d,
new CryptoPP::StringSink(decrypted)));
return decrypted;
}
My questions:
Why is a random number generator (RNG) needed for EncryptorFilter/DecryptorFilter?
The RNG has to be the same for encryption/decription, right? So, how to share between processes?
Using the NullRNG() as recommended by https://stackoverflow.com/users/608639/jww in Unable to do RSA Encrption/Decryption using Crypto++ (isValidCoding is false) leads to
std::exception NullRNG: NullRNG should only be passed to functions that don't need to generate random bytes.
I guess I fundamentally miss something. Thanks for hints and advices.
If I use this code in a unit test with a global RNG, everything works fine.
Why is a random number generator (RNG) needed for EncryptorFilter/DecryptorFilter?
The signing and verification classes are abstract interfaces setup in cryptlib.h. Some cryptosystems use them, others do not. A class will specialize and can forgo using a generator. Sometimes a class does not need a generator for one of the operations. NullRNG can be used if not needed.
The reason RSA needs a RNG during public key operations is message padding. Padding is often part of the message formatting function. As #PuzzlePalace pointed out, OAEP padding is randomized and not deterministic.
The reason RSA needs a RNG during private key operations is blinding. For RSA and other RSA-like schemes (like Rabin-Williams), blinding is just a multiplication by a random value to mask the inversion by the priavte key to recover the original value. Later, after signing or decryption, the blinding value is removed and the result of the operation remains.
Related, a reason DSA or ECDSA would not need a RNG during private key operations is RFC 6979, Deterministic Usage of the Digital Signature Algorithm (DSA) and Elliptic Curve Digital Signature Algorithm (ECDSA). Deterministic signatures don't use randomized formatting or randomized k's.
Another reason a RNG is needed for public key and private key operations is validation checks on the key. For example, a key might be checked to ensure a particular constraint holds, like its prime or it has a particular Jacobi symbol.
The RNG has to be the same for encryption/decryption, right? So, how to share between processes?
No, the generators can be different. The only requirements is they produce a random stream for some reasonable definition of what it means to be "random". Without splitting too many hairs, it means the generator produces a uniform distribution.
You can find more reading on Crypto++ generators at RandomNumberGenerator on the wiki.
If I use this code in a unit test with a global RNG, everything works fine.
One quick word of caution... GlobalRNG is part of the Test namespace. It is defined in test.cpp : 115:
NAMESPACE_BEGIN(CryptoPP)
NAMESPACE_BEGIN(Test)
ANONYMOUS_NAMESPACE_BEGIN
OFB_Mode<AES>::Encryption s_globalRNG;
NAMESPACE_END
RandomNumberGenerator & GlobalRNG()
{
return dynamic_cast<RandomNumberGenerator&>(s_globalRNG);
}
NAMESPACE_END // Test
NAMESPACE_END // CryptoPP
GlobalRNG is a deterministic generator and its not part of the library proper. Your code will fail to compile in the field if you depend on it.
Use one of the other generators discussed at RandomNumberGenerator on the wiki. AutoSeededRandomPool is a good choice.
Using the NullRNG() as recommended by https://stackoverflow.com/users/608639/jww in Unable to do RSA
Encrption/Decryption using Crypto++ (isValidCoding is false) leads to
std::exception NullRNG: NullRNG should only be passed to functions that don't need to generate random bytes.
That information is incorrect. I need to fix it. Thanks.
Interestingly (in a morbid sort of way), Crypto++ took CVE-2015-2141 due to blinding in Rabin-Williams. The blinding value needed to be a quadratic residue; otherwise an attacker could prepare special messages to reveal the private key.
The full paper by Evgeny Sidorov is available at Breaking the Rabin-Williams digital signature system implementation in the Crypto++ library. Here's what the new and improved inverse function looks like after fixing Sidorov's attack (from rw.cpp):
ModularArithmetic modn(m_n), modp(m_p), modq(m_q);
Integer r, rInv;
do
{
// Do this in a loop for people using small numbers for testing
r.Randomize(rng, Integer::One(), m_n - Integer::One());
// Fix for CVE-2015-2141. Thanks to Evgeny Sidorov for reporting.
// Squaring to satisfy Jacobi requirements suggested by Jean-Pierre Munch.
r = modn.Square(r);
rInv = modn.MultiplicativeInverse(r);
} while (rInv.IsZero());
If you read Section 6 of Sidorov's paper, he suggests generating a random r, and then checking the Jacobi symbol of r to ensure its a quadratic residue. If it was not a QR, then try a new random r. The triage used the method, but it showed the scheme slowed down considerably because a random r satisfies the condition with probability 1/16.
However, we knew squaring r ensured we satisfied Jacobi on the first try because r2 mod n was always a quadratic residue. The squaring/multiplication only takes log (exp) (not n log (n)), so it turned out to be a significant speedup over trial and error. Before we released the next version of the library, we switched to the squaring method.
I am wondering if there is a way to generate a Cryptographic Nonce using OpenSSL or Crypto++ libraries. Is there anything more to it than just generating a set of random bytes using autoseeded pools?
I am wondering if there is a way to generate a cryptographic nonce using OpenSSL or Crypto++ libraries.
Crypto++:
SecByteBlock nonce(16);
AutoSeededRandomPool prng;
prng.GenerateBlock(nonce, nonce.size());
OpenSSL:
unsigned char nonce[16];
int rc = RAND_bytes(nonce, sizeof(nonce));
unsigned long err = ERR_get_error();
if(rc != 1) {
/* RAND_bytes failed */
/* `err` is valid */
}
/* OK to proceed */
Is there anything more to it than just generating a set of random bytes using autoseeded pools?
A nonce is basically an IV. Its usually considered a public parameter, like an IV or a Salt.
A nonce must be unique within a security context. You may need a nonce to be unpredictable, too.
Uniqueness and unpredictability are two different properties. For example, a counter starting at 0000000000000000 is unique, but its also predictable.
When you need both uniqueness and unpredictability, you can partition the nonce into a random value and a counter. The random value will take up 8 bytes of a 16 byte nonce; while the counter will take up the remaining 8 bytes of a 16 byte nonce. Then you use an increment function to basically perform i++ each time you need a value.
You don't need an 8-8 split. 12-4 works, as does 4-12. It depends on the application and the number of nonces required before rekeying. Rekeying is usually driven by plain text byte counts.
16-0 also works. In this case, you're using random values, avoiding the counter, and avoiding the increment function. (The increment function is basically a cascading add).
NIST SP800-38C and SP800-38D offer a couple of methods for creating nonces because CCM and GCM uses them.
Also see What are the requirements of a nonce? on the Crypto Stack Exchange.
You need a unique number for each nonce. You can use either a serial number or a random number. To help ensure uniqueness, it is common, though not required, to add a timestamp to the nonce. Either passing the timestamp as a separate field or concatenating it with the nonce. Sometimes information such as IP addresses and process IDs are also added.
When you use a serial number, you don't need to worry about skipping numbers. That's fine. Just make sure you never repeat. It must be unique across restarts of your software. This is one place where adding a timestamp can help. Because time-in-millis+serial-number is almost certainly unique across restarts of the server.
For the pseudo random number generator, anyone should be fine. Just make sure that you use a sufficiently large space to make the chances of getting a duplicate effectively impossible. Again, adding time will reduce the likelihood of you getting duplicates as you'll need to get the same random number twice in the same millisecond.
You may wish to hash the nonce to obscure the data in it (eg: process ID) though the hash will only be secure if you include a secure random number in the nonce. Otherwise it may be possible for a viewer of the nonce to guess the components and validate by redoing the hash (ie: they guess the time and try all possible proc IDs).
No. If the nonce is large enough then an autoseeded DRBG (deterministic random bit generator - NIST nomenclature) is just fine. I would suggest a nonce of about 12 bytes. If the nonce needs to be 16 bytes then you can leave the least significant bits - most often the rightmost bytes - set to zero for maximum compatibility.
Just using the cryptographically secure random number generators provided by the API should be fine - they should be seeded using information obtained from the operating system (possibly among other data). It never hurts to add the system time to the seed data just to be sure.
Alternatively you could use a serial number, but that would require you to keep some kind of state which may be hard across invocations. Beware that there are many pitfalls that may allow a clock to repeat itself (daylight saving, OS changes, dead battery etc. etc.).
It never hurts to double check that the random number generator doesn't repeat for a large enough output. There have been issues just with programming or system configuration mistakes, e.g. when a fix after a static code analysis for Debian caused the OpenSSL RNG not to be seeded at all.
This question already has answers here:
Generate a random number using the C Preprocessor
(2 answers)
Closed 6 years ago.
I'm developing a method for encrypting game data files which will be used in a game engine that I'm creating. The game data file will be encrypted with a private key and decrypted once loaded into the game engine.
The private key will be set manually in the engine (as a const) so that it is compiled into the engine. The trick, however, is obfuscating it in there so that your average Joe Bloggs can't (easily) find the key simply by disassembling and reverse-engineering the code.
struct Example {
u_int8_t random1;
u_int64_t segment31;
u_int8_t random2;
u_int8_t random3;
u_int64_t segment3;
u_int64_t segment14;
// Etc
};
My idea is to store the key in segments which are jumbled together with randomly generated bytes. My question is this:
Is there any way to randomly generate a number in a C++ macro? The number must be randomly generated at compile time (or the random values each set to a random user-selected value), so any runtime functions are no good.
I'm open to any suggestions if there is a better way of doing this, but I definitely want the game data files encrypted with a private/public key pair, and the public key kept as private as possible.
Since you are generating a constant random number at compile time which will become baked into your program, I don't see a difference between that and simply hard-coding a random number (of your own choosing) into it yourself. I suppose with a macro you get a different random number with each compile. Is that really so critical?
I recommend doing something simpler: If the key were simply ASCII text (bits are bits, right?) you could use an inocuous-looking diagnostic string as your key and no one would notice it.
Still, security by obscurity is generally frowned on by experts in the field (they do a lot of frowning). If you are really doing public key encryption, it is perfectly OK to have the public key exposed. That is the point after all. It is the private key which must be kept secret.
If you really want to do this, then I would look at a random number generation algorithm, and unroll an iteration of it into a (large) macro that essentially takes a seed and evaluates to a large expression that yields a pseudorandom integer as its result. That should do it. If the seed were somehow derived from an unpredictable thing like the time of the compile (__TIME__), it just might work.
There's not currently any rand macro or template metafunction, but it could be created...with a lot of effort.
On the other hand, you could simply touch /dev/random as part of your build process, or some other random source, and create a header with the result. Include this header where you get a random number constant.
I was wondering, is it fair to conclude, XOR string encryption is less secure than other encryption method, say Blowfish
This is because for both methods, their input are
Unencrypted string
A secret key
string XOR(string value,string key)
{
string retval(value);
short unsigned int klen=key.length();
short unsigned int vlen=value.length();
short unsigned int k=0;
short unsigned int v=0;
for(v;v<vlen;v++)
{
retval[v]=value[v]^key[k];
k=(++k<klen?k:0);
}
return retval;
}
Is there any proof that XOR encryption method is more easy to be "broken" than Blowfish if the same key is being chosen?
If your key is (a) truly random, (b) at least as long as the plaintext, and (c) never re-used then XOR encryption is proveably unbreakable.
If you can't meet those stringent criteria then XOR encryption is proveably weaker than proper encryption algorithms like Blowfish, though I'm not in a position to prove it myself.
XOR encryption is wildly unsafe if you try encoding the same string more than once with it because it allows malicious users to recover the XOR of two messages by XORing their encryption. If they can somehow trick you into XORing a known string, then they can recover your key and break the encryption entirely.
More advanced encryption algorithms like Blowfish and AES have much stronger security guarantees and are assumed to be so strong that even if the attacker lets you encrypt known data, they can't recover your key or recover any individual bits. They should always be used instead of your own custom encryption in any scenario where security is important.
As an interesting aside, XOR encryption where on each invocation you use a new (randomly-chosen) key is cryptographically unbreakable from an information-theoretic perspective. This is called a "one-time pad" and could in theory be useful in some settings.
Actually, "XOR encryption" is proven to be perfectly secure against eavesdropping (you are still in trouble when someone can change your stream), given an encryption key that is
as long as the string to be encrypted
perfectly random (bits are uncorrelated and have equal probabilities of being 1 and 0)
used only once.
Otherwise, you are in deep trouble. If your bits aren't random, you can extract some information about the plaintext bits. If you reuse your key, you can XOR two ciphertexts to get the XOR of the plaintext, or, if by chance someone gets you to post a message with a plaintext known to him, he can extract the key (they actually did that for ENIGMA in WW2, so that's not unrealistic!). If the key is shorter than the message, you can employ statistics - say they key is 10 bytes long, you can divide the ciphertext into 10 parts which are all encoded with the same byte, and employ frequency analysis ... just to name a few attacks.
I don't know why it so rarely gets mentioned but the one time pad (OTP) has one major short-coming which is avoided by using the likes of RSA. Imagine that it is known that you will send one of two messages regarding stock ABC to your broker based on some news you will get at 2:00pm. You will either send BUY or SELL. With OTP, it doesn't take much crypto-analysis to figure out what you sent, because the length alone gives it away.
IMHO, it's better to use something that changes the length when converting to ciphertext as most common encryption systems do.