I hope someone can help me to understand how DEK and KEK works in layman term.
This is my understanding so far
Part 1
Uploaded data is stored in chunks. Each chunk is encrypted with DEK. DEK is stored near the chunked data.
Part 2 DEK of the data chunk is wrapped with KEK
Part 3 KEK is stored in KMS.
This sentence from a Google's course material really tripped me.
The wrapped data encryption keys are then stored with this data
My brain goes crazy with these questions:
What is this wrapped data encryption keys? Isn't that the KEK in part 2?
If that's the KEK in part 2, shouldn't that be in KMS?
If yes, does KMS store data?
If no, how many DEKs a chunk of data has? 2 DEKs?
What keys do we(users/customers)keep?
We store the data with which keys?
What is this wrapped data encryption keys? Isn't that the KEK in part 2?
The wrapped DEK is the result of encrypting the DEK with the KEK. The key encryption key is called that because it encrypts (data encryption) keys.
If that's the KEK in part 2, shouldn't that be in KMS?
Does KMS store data?
No. As the name suggests, a key management system only stores keys, and specifically KEKs.
How many DEKs does a chunk of data have? 2 DEKs?
One. Data is encrypted only once, so there is only one key.
What keys do we (users/customers) keep?
Only the KEK. When using a KMS, not even that key is known to you, the user of the KMS.
We store the data with which keys?
With the wrapped DEK.
Here is a concrete example:
Data (i.e. plaintext): yellow submarine
KEK (aka. master key): my-secret-master
DEK (randomly generated for each piece of data): ttlly-random-dek
Ciphertext: Base64(AES(Data, DEK)) = TJ4SLFTy0sMdvGe55QCuYQ==
Wrapped DEK: Base64(AES(DEK, KEK)) = lOEmql1JuSONZ8uLorc/vQ==
The second operation is done by the KMS, if there is one (it has to be, because it's the only one in possession of the KEK, and it will never disclose it).
What you store together: TJ4SLFTy0sMdvGe55QCuYQ== and lOEmql1JuSONZ8uLorc/vQ==.
And to reconstruct the plaintext given TJ4SLFTy0sMdvGe55QCuYQ== + lOEmql1JuSONZ8uLorc/vQ==:
DEK: AES'(Base64'(lOEmql1JuSONZ8uLorc/vQ==), KEK) // = ttlly-random-dek Plaintext: AES'(Base64'(TJ4SLFTy0sMdvGe55QCuYQ==), DEK)
The first operation is done by the KMS if there is one (again, it has to be, because the KEK is involved).
In the example above I conveniently chose 16 bytes of data and 16-byte keys, so I could gloss over block cipher modes and IVs/nonces. In practical applications the IVs for each encryption have to be retained as well, of course.
Note that the KMS never sees your data, and it only has to encrypt and decrypt tiny amounts. That is why we do all of this; only you, the owner of the data ever sees it in plaintext (assuming the KMS is an honest party, obviously).
Related
I Have gone through several approaches on stack overflow but none works now.
Encrypting a text field value which is the password and saving it in the database is the requirement.
I need the approach of converting the plain text into encrypted
I have tried
https://wiki.qt.io/Simple_encryption_with_SimpleCrypt
it gives different encryptions for same text so I can not compare and validate
Simplecrypt you linked to has this piece of code in it:
//prepend a random char to the string
char randomChar = char(qrand() & 0xFF);
ba = randomChar + integrityProtection + ba;
What this means is, any piece of data can result in 256 different possible encrypted datas. This is useful in encryption, where you (among many other things) don't want an attacker to be able to see if two separate encrypted pieces of data are actually same data or not.
If you want to use SimpleCrypt, you have to compare the passwords after decrypting. You could also modify the algorithm to have a known (given by you) randomChar. But I advise against it, as that is going to extra effort to do something poorly.
You should really use something else, for example QCryptographicHash. Just remember to use salt when hashing the password (this prevents an attacker from seeing if some passwords in the database are the same).
There is this question on SO:
What is the difference between encrypting and signing in asymmetric encryption?
My question is not a duplicate. I am asking if there is difference between two programmatically.
I have an openssl C++ API to sign and verify a message. Let say two functions:
sign
verify
Let say two sides who interchange messages are A and B.
Currenlty sign function uses A_privateKey, and verify function uses B_publicKey.
Now, I want to encrypt some part of the message.
So, could I do it just using sign function with B_publicKey?
Or the algorithm to encrypt is totally different and I need to implement a new function?
Or with some modification in sign function (removing generating hash, for ex) does it just work?
Now, I want to encrypt some part of the message.
So, could I do it just using sign function with B_publicKey?
The output of the sign function is an encrypted hash of the original message. If "A" were to tranmit the output of sign as you describe, the only thing B could do is decrypt it back to the hash of the original message. He wouldn't have the actual message.
Because assymetric encryption can be a very computationally expensive operation that doesn't scale well for long messages, the standard pattern is this:
Alice generates a message she wants to send to Bob.
Alice generates a symmetric key first
Alice encrypts the message with this symmetric key.
Alice encrypts the symmetric key with Bob's public key
Alice transmits the encrypted message and encrypted key to Bob.
Then Bob does this:
Bob receives both the message that was symmetrically encrypted and
the assymetrically encrypted key.
Bob decrypts the symmetric key with his public key
Bob uses the symmetric key to decrypt the encrypted message
There's probably a ton of other security principals missing as well, but the above is the general idea.
As reported in the AWS documentation
The maximum size of the data payload of a record before base64-encoding is up to 1 MiB.
Since that I need to process records that may have the size larger than 1 MB this limit may be an issue.
Is there any workaround to overcome this limit? And in case any proven solution already implemented and used by anyone? (I'd like avoiding "reinventing the wheel")
You have two choices: break the payload into multiple records or save it outside the stream, for example in S3.
For the first option, you can utilize PartitionKey and SequenceNumberForOrdering (doc). Assign a unique partition key (such as a UUID) to each source record. If you need to break the source into sub-1MB chunks, you set the sequence number for chunks 2..N to the returned sequence number of the previous chunk.
This will then require the clients to examine the partition key for retrieved records, and reconstruct the original record if necessary. Note that they may need to buffer several chunks (for different source records).
Externalizing the data will simplify both the producer and consumer code. Again, create a unique identifier for each source record, but rather than writing the record to the stream write it to S3 with that identifier as its key. Then write the key to the stream. The consumer will then retrieve the actual data from S3 when it reads the ID from the stream.
This second approach does require more management: you will need to add a lifecycle rule to S3 to delete the records, and you'll need to ensure that this life-cycle rule lets the objects live at least as long as the stream's retention period (I would probably set an 8 day TTL regardless of stream retention period, because S3 is cheap).
If you only have infrequent large records, and especially if you have lots of small records, then writing everything to S3 will be inefficient. In that case you can adopt a hybrid model, in which you write a data structure to the stream that either contains the actual data or a reference to external storage.
I'm having trouble encrypting data with node and decrypting with RNCryptor. My understanding is that RNCryptor uses a special data format?
The cryptotext is being generated as follows:
var crypto = require('crypto');
var cipher = crypto.createCipher('aes-256-cbc','InmbuvP6Z8');
var text = "123|123123123123123";
var crypted = cipher.update(text,'utf8','base64');
crypted += cipher.final('base64');
And I'm decrypting like this:
[RNDecryptor decryptData:plainText withPassword:password error:&error];
How am I supposed to do this? When I try to decrypt currently I get an empty NSData and no error.
Yes, RNCryptor outputs encrypted data in its own format. If you build this format in your own encryption code (and use the same encryption params), you can pass it to JNCryptor to decrypt.
For RNCryptor format version 3, the first 34 bytes are as follows:
byte[0] is the version (3).
byte[1] defines whether a password or a key is used (1 for password, 0 for key).
bytes[2-9] carry the encryption salt.
bytes[10-17] carry the HMAC salt.
bytes[18-33] carry the IV.
Then comes the encrypted ciphertext.
Then the last 32 bytes hold the (SHA256) HMAC for the ciphertext.
The spec is here: https://github.com/RNCryptor/RNCryptor-Spec/blob/master/RNCryptor-Spec-v3.md
RNCryptor is more than just encryption, it is an entire secure "stack" including authentication, password key derivation, versioning and random iv. Since your encryption is just the encryption portion it will be incompatible.
Either use RNCryptor for the encryption or change to a simple decryption method. Since the question has a commoncrypto tag and the encryption code look like it might be Swift (there is no language tag) just consider using Common Crypto in Objective-C for the decryption. See the SO answer for example code.
I just want to know the exact formula (or algorithm) used for generating the pseudo random values used in encrypting the zip file. I am trying to create a password hacker(for zip files) and I also require to know how to verify if the random password generated by my program is correct. I have tried searching for an answer to this in Google but I could't find a direct solution.
I am trying to program this zip hacker in c++.
note: by formula (or algorithm) I meant: key derivation function.
I just want the necessary information as quick as possible, that's why I posted it here!
Different versions of zip-files do it differently, but basically you have an encryption-header specifying what encryption is used according to the zip-file specification.
For example, the strong encryption header looks like below and specifies the encryption algorithm in the AlgID-field.
4.5.12 -Strong Encryption Header (0x0017):
Value Size Description
----- ---- -----------
0x0017 2 bytes Tag for this "extra" block type
TSize 2 bytes Size of data that follows
Format 2 bytes Format definition for this record
AlgID 2 bytes Encryption algorithm identifier
Bitlen 2 bytes Bit length of encryption key
Flags 2 bytes Processing flags
CertData TSize-8 Certificate decryption extra field data
(refer to the explanation for CertData
in the section describing the
Certificate Processing Method under
the Strong Encryption Specification)
7-zip uses AES-256 encrpytion for 7z/zip archives.(see here)
7-Zip also supports encryption with AES-256 algorithm. This algorithm uses cipher key with length of 256 bits. To create that key 7-Zip uses derivation function based on SHA-256 hash algorithm. A key derivation function produces a derived key from text password defined by user. For increasing the cost of exhaustive search for passwords 7-Zip uses big number of iterations to produce cipher key from text password.
Also, keep in mind that Brute force attacks are a waste of time. I won't go into details why, I will instead direct you to Jeff Atwood's blog, he has an excellent post.