I am using CryptoApi to encrypt a file (asymmetric encryption). Everywthing is ok but when the file is large, it can not encrypt it. I searched and found that I must encrypt block by block. Except for the last block the Final flag in CryptEncrypt function must be false.
I know all of above conception but I dont know how I can to implement them. I mean I dont know how read, encrypt and write block by block.
Can you give me a real code example.
Update:
I used the code of this website: http://blogs.msdn.com/b/alejacma/archive/2008/01/28/how-to-generate-key-pairs-encrypt-and-decrypt-data-with-cryptoapi.aspx
I am writing this solution for programmers who will have this problem in the future:
In this link has been shown how to encrypt large file (block by block):
https://msdn.microsoft.com/en-us/library/windows/desktop/aa382358%28v=vs.85%29.aspx
Note: Somethings must be change when you want to use the above code
1) In encryption, block size must be set to 128 - 11 ( DWORD dwBlockLen = 128 - 11 )
2) In decryption, block size must be set to 128 ( DWORD dwBlockLen = 128 )
Both tested in win 7.
Try something like:
final_flag <- false
repeat
this_block <- read_next_block(file)
if (is_EoF(file)) {final_flag <- true }
encrypt(this_block, final_flag)
until (final_flag == true)
I don't know enough about the C++ file system handling to write a working check for the end of a file, but there should be one in there somethere.
Related
before = new unsigned char[mSizeNeeded*4];
uLong value = compressBound(mSizeNeeded*4);
after = new unsigned char[value];
compress(after, &value, before, mSizeNeeded*4);
fwrite(&after, 1, value, file);
'before' has a bunch of audio data stored into it and I am trying to compress it and store it into 'after'. I then write it into a file. The file is the same size as the original file, it also contains the same data that was in before (as far as I can tell).
Compress also returns OK so I know that the compression is not failing.
Okay, so it looks like my only problem is somewhere in the compression (I think). I am able to run compress and then I can uncompress and get the correct data out. Also, it is writing into the file and fwrite returns 561152 but the count (value) is 684964. So it looks like something is wrong with fwrite. I looked more carefully and the after data is different than the before data.
561152 is the same size as the original audio data in a .wav file that I have (stripped of the .wav headers of course).
Based on your original text:
fwrite (&before, ...
I am trying to compress it and store it into 'after'. I then write it into a file.
I think not. You are writing the original data to the file, you should probably be writing after instead.
The other thing you should get in the habit of doing is checking return values from functions that you care about. In other words, compress() will tell you if a problem occurs yet you seem to be totally ignoring the possibility.
Similarly, fwrite() also uses its return value to indicate whether it was successful or not. Since you haven't included the code showing how that's set up, this is also a distinct possibility. In particular fwrite is under no obligation to write your entire block to the file in one hit (device may be full, etc), that's why it has a return value, so you can detect and adjust for that situation. Often, a better option than:
fwrite (&after, 1, value, file);
is:
fwrite (&after, value, 1, file);
since the latter will always give you one for a fully successful write, something else for a failure of some description.
That would be my first step in establishing where the problem lies.
On top of that, there are numerous other (generally-applicable) methods you can use to track down the issue, such as:
outputting all variables after they change or are set (like the return values of functions, after, before, value and so on).
delete the output file before running your program, to ensure it's created afresh.
run the code through a debugger so you can see what's happening under the covers.
clearing after to all zero bytes (or a known pattern) to ensure you don't get stale data in there.
And, as a final approach (given that the zlib source code is freely available), you can also modify (or debug into) it so that you can clearly see what's going on under the covers.
I'm having a lot of difficulty understanding and implementing the Windows Crypto API to Import and Export Keys in c++.
Despite reading through the MSDN documentation many many times I can't seem to get it to work in the way I want.
Below is a snippet of code from what i'm working on.
if(CryptAcquireContext(&CryptoHandle,NULL,provPointer, PROV_RSA_AES, 0xF0000000))
{
HCRYPTKEY aesKey;
//We now have context on Enhanced AES
if(CryptGenKey(CryptoHandle,CALG_AES_128,CRYPT_EXPORTABLE,&aesKey))
{
DWORD dwBlobLen;
BYTE* pbKeyBlob;
CryptExportKey(aesKey,0,PLAINTEXTKEYBLOB,0,NULL,&dwBlobLen);
if(pbKeyBlob=new BYTE[dwBlobLen])
{
if(CryptExportKey(aesKey, NULL,PLAINTEXTKEYBLOB, 0,pbKeyBlob, &dwBlobLen))
{
//Blah Blah
}
}
}
}
*(Where provPointer is a pointer to the Enhanced crypto api string.
As you might be able to tell from the snippet i'm trying to export a AES 128 key to plaintext.
In the debugger it all executes fine (No visible errors) but I don't understand the outcome at all.
The first call to CryptExportKey fills the dwBlobLen with '28' (What does this mean? Why?)
After the second CryptExport key i've tried writing pbKeyBlob(Which I assume points to the key) to file But I just end up with a constant set of bytes (Same for every try) followed by a set of bytes that I different every time (I assume this is some of the key) (Which add to 28 bytes total)
I'd really appreciate if someone could identify where I've gone wrong. I'm pretty clueless with the whole crypto lingo (Sessions,machine keys, blobs etc.)
In the future I'd like to be able to generate an AES key, use it and export it into a file in a form where I can import it again later.
Thanks in advance.
I'm not an expert on the Windows Cryptography API (or on cryptography in general) but I believe I can shed some light on what's going on here.
The first call to CryptExportKey puts 28 in dwBlobLen because that is the size of the blob that it will created when the key is exported. This is in the MSDN docs: http://msdn.microsoft.com/en-us/library/windows/desktop/aa379931%28v=vs.85%29.aspx
AS far as what you're doing wrong. You aren't doing anything wrong. You are asking CryptExportKey to export a plaintext blob which has the following layout:typedef struct _PLAINTEXTKEYBLOB {
BLOBHEADER hdr;
DWORD dwKeySize;
BYTE rgbKeyData[];
} PLAINTEXTKEYBLOB, *PPLAINTEXTKEYBLOB;
As you can see, the blob starts with a header and a key size (which is the constant set of bytes which you have reported, and should be 12 bytes long), followed by the key data (which is the data that changes every time, and should be 16 bytes long). Remember you are generating a 128 bit key (which is 16 bytes).
The BLOBHEADER has the following layout:typedef struct _BLOBHEADER {
BYTE bType;
BYTE bVersion;
WORD Reserved;
ALG_ID aiKeyAlg;
} BLOBHEADER;
By the way, from the doc on the CryptImportKey function, you can't import the PLAINTEXTBLOB directly, because the BYTE array that you pass to CryptImportKey does not include the keysize. You need to pass a buffer with the BLOBHEADER followed by the key data.
I'm making simple dll packet sniffer using C++, that will hook to the apps, and write the received packet into INI file. Unfortunately after 20-30 minutes it crashed the main apps.
When the packet is received, receivedPacket() will be called. After 20+ minutes, WriteCount value is around 150,000-200,000.. and starting to get C++ runtime error/crash, GetLastError() code that I get is 0x8, which is ERROR_NOT_ENOUGH_MEMORY, and the WritePrivateProfileStringA() returns 0
void writeToINI(LPCSTR iSec,LPCSTR iKey,int iVal){
sprintf(inival, _T("%d"), iVal);
WritePrivateProfileStringA(iSec,iKey,inival,iniloc);
//sprintf(strc, _T("%d \n"), WriteCount);
//WriteConsole(GetStdHandle(STD_OUTPUT_HANDLE), strc, strlen(strc), 0, 0);
WriteCount++;
}
void receivedPacket(char *packet,WORD size){
switch ( packet[2] )
{
case 0x30:
// Size : 0x5F
ID = *(signed char*)&packet[0x10];
X = *(signed short*)&packet[0x20];
Y = *(signed short*)&packet[0x22];
Z = *(signed short*)&packet[0x24];
sprintf(inisec, _T("PACKET_%d"), (ID+1));
writeToINI(inisec,"id",ID);
writeToINI(inisec,"x",X);
writeToINI(inisec,"y",Y);
writeToINI(inisec,"z",Z);
}
[.....OTHER CASES.....]
}
Thanks :)
WritePrivateProfileString() and GetPrivateProfileString() are very slow (due to parsing INI file each call), instead you can:
use one of existing parsing libraries, but i am not sure about memory efficiency nor supporting sequential write.
write your own sequential INI writter:
read file (or only part, by part, if it is too big)
find section and key (if not found, create new section at end of file, or find insertion position, if you want sorted sections), save file position of key and next key
change value
save (beginning of original file to position of key + actual changed key + position of next key in original file to end of file) (if new section is created at end, you can simply append new section to original file) (if packets rewrite same ID often, you can add padding whitespace after each key, large to hold any value of desired type (example: change X=1---\n to X=100-\n (change - to whitespace), so you have constant size of key, you can update only part of file) )
database, for example MySQL
write binary file (fastest solution) and make program to read values, or to convert to text
Little note: I use GetPrivateProfileString() few years ago to read settings file (about 1KB of size), reading form HDD: 50ms, reading from USB flash disk: 1000ms!, after changing (1. read file to memory 2. run my own parser) it run in 1ms both on HDD and USB.
Thanks for the reply guys, but looks like the problem wasn't come from WritePrivateProfileStringA().
I just need to add extra size in malloc() for the Hook.
:)
I'd like to use the STL (C++0x in vs2010) to open a file and binary search through it and then replace when a match is found.
I can use fstream to open a file but i'm a bit confused to how to search the file.
Reading one byte at a time isn't good so it should read a block into a buffer and we then search in the buffer but is this functionality already in the STL ?
I want the buffering part to be automatic, when the search reach the end of buffer it should automatically read in the next block from the file as if it was reading it directly (using the same file offsets, etc..)
The other problem is when it finds a match exactly how should it update the file.
I know that this file functionality exists in windows using CreateFileMapping and MapViewOfFile but i want to use the STL to make my code portable and also by using the STL also more flexible. Using these windows function you can read the file without worrying about buffering, etc, they will even update the file when you change data. This is the functionality i'm after using the STL.
Update: I've changed 'binary search' to 'raw byte search' to clear up the confusion, sorry for that.
An example of the raw byte search function ( if you have a better please do tell )
// NOTE: This function doesn't support big files > 4 gb
DWORD_PTR RawByteSearch( PBYTE pBaseAddress, DWORD dwLowSize, PBYTE pSignature, DWORD sigSize )
{
PBYTE pBasePtr = pBaseAddress;
PBYTE pEndPtr = pBaseAddress + dwLowSize;
DWORD_PTR i;
while ( pBasePtr < pEndPtr )
{
for (i = 0; i < sigSize; ++i)
{
if ( pSignature[i] != pBasePtr[i] )
break;
}
if ( i == sigSize ) // Found a match
{
return pBasePtr - pBaseAddress;
}
++pBasePtr; // Next byte
}
return 0;
}
std::search already does what you need.
std::search(pBasePtr, bEndPtr, pSignature, pSignature+sigSize)
Out of completeness, you don't need to use memory mapping (although that will likely be the most efficient).
A 100% portable solution is to instead use the standard library stream iterators:
std::ifstream file;
std::search(std::istreambuf_iterator<char>(file.rdbuf()) // search from here
, std::istreambuf_iterator<char>() // ... to here
, pSignature // looking for a sequence starting with this
, pSignature+sigSize); // and ending with this
You should avoid the term "binary search". This is a term used for searching in sorted data in logarithmic time, what you are doing is a linear search (in binary data). The C++ standard library has a function for that, it's std::search. It would work something like:
PWORD pos = std::search(pBaseAddress, pBaseAddress+dwLowSize, pSignature, pSignature+pSigSize);
P.S. Hungarian Notation sucks ;)
std::fstream has a function called read that can read a block into a buffer. Am I misunderstanding something?
Edit: I just did a quick search on memory-mapped file implementations. Maybe this can be helpful for you:
http://www.boost.org/doc/libs/1_40_0/libs/iostreams/doc/classes/mapped_file.html
BinarySearch actually means starting at the midway point and comparing to slice your search space in two, then going to the midway point of the new search space to divide it further. As your search space halves with every iteration, the operation is O(log N) which is therefore in general a lot faster.
You may wish to do that here by building up an index.
A lot of this depends how your file is formatted. Does it consist of fixed-length "records" with a key at the start of the record and with that key sorted?
If so, you can actually scan every Nth record to create an index in memory, then use binary search to locate where approximately in the file the record would be, then load just that section of the file into memory.
For example, let's say each record is 128 bytes, the key is 4 bytes and there are 32K records, so the file is about 4MB big. If you load every 64th record, you are reading 512 of them and storing 2K of data in memory. When you search your local index you then load an 8K portion into memory and search in that for the actual record you wish to change, and modify it appropriately.
If your records are not all the same length, you will have a problem if your modification tramples over the next record.
I'm using two different libraries to generate a SHA-1 hash for use in file validation - an older version of the Crypto++ library and the Digest::SHA1 class implemented by Ruby. While I've seen other instances of mismatched hashes caused by encoding differences, the two libraries are outputting hashes that are almost identical.
For instance, passing a file through each process produces the following results:
Crypto++
01c15e4f46d8181b984fa2a2c740f8f67130acac
Ruby:
eac15e4f46d8181b984fa2a2c740f8f67130acac
As you can see, only the first two characters of the hash string are different, and this behavior repeats itself over many files. I've taken a look at the source code for each implementation, and the only difference I found at first glance was in the data hex that is being used for the 160-bit hashing. I have no idea how that hex is used in the algorithm, and I figured it'd probably be quicker for me to ask the question in case anyone had encountered this issue before.
I've included the data from the respective libraries below. I also included the values from OpenSSL, since each of the three libraries had slightly different values.
Crypto++:
digest[0] = 0x67452301L;
digest[1] = 0xEFCDAB89L;
digest[2] = 0x98BADCFEL;
digest[3] = 0x10325476L;
digest[4] = 0xC3D2E1F0L;
Ruby:
context->state[0] = 0x67452301;
context->state[1] = 0xEFCDAB89;
context->state[2] = 0x98BADCFE;
context->state[3] = 0x10325476;
context->state[4] = 0xC3D2E1F0;
OpenSSL:
#define INIT_DATA_h0 0x67452301UL
#define INIT_DATA_h1 0xefcdab89UL
#define INIT_DATA_h2 0x98badcfeUL
#define INIT_DATA_h3 0x10325476UL
#define INIT_DATA_h4 0xc3d2e1f0UL
By the way, here is the code being used to generate the hash in Ruby. I do not have access to the source code for the Crypto++ implementation.
File.class_eval do
def self.hash_digest filename, options = {}
opts = {:buffer_length => 1024, :method => :sha1}.update(options)
hash_func = (opts[:method].to_s == 'sha1') ? Digest::SHA1.new : Digest::MD5.new
open(filename, "r") do |f|
while !f.eof
b = f.read
hash_func.update(b)
end
end
hash_func.hexdigest
end
end
I would guess that you are off by a byte in printing out the SHA-1 hashes. Can we see the code that is printing them? If not, here are a couple of potentially useful diagnostics:
Make a very short file (say, one word), and put its contents in as a hex string at http://www.fileformat.info/tool/hash.htm. You would need to know exactly the hex contents of the file, though. You can use xxd to that on Unix, but you'll have to watch out for endianness issues. I'm not sure how to do it on other OSs.
Does running the same file through the same SHA-1 implementation several times always print out the same value in that first byte? If so, does that value change when you change files?
This isn't making much sense. If there were something wrong with the SHA1 implementation, such as with those numbers, it would likely produce hashes that are completely different than the real SHA1 hashes, rather than just one byte off. Even if there were something wrong with your file reading loop, that it would drop a newline or something, you would still get a completely different hash by changing one byte in the stream, it wouldn't be one byte off of the real SHA1 hash.
If I do use your method in the following program, I get the correct results.
#!/usr/bin/env ruby
require 'digest/sha1'
require 'digest/md5'
File.class_eval do
def self.hash_digest filename, options = {}
opts = {:buffer_length => 1024, :method => :sha1}.update(options)
hash_func = (opts[:method].to_s == 'sha1') ? Digest::SHA1.new : Digest::MD5.new
open(filename, "r") do |f|
while !f.eof
b = f.read
hash_func.update(b)
end
end
hash_func.hexdigest
end
end
puts File.hash_digest(ARGV[0])
And its output compared with that of OpenSSL.
tmp$ dd if=/dev/urandom of=random.bin bs=1MB count=1
1+0 records in
1+0 records out
1000000 bytes (1.0 MB) copied, 0.287903 s, 3.5 MB/s
tmp$ ./digest.rb random.bin
a511d8153426ebea4e4694cde78db4e3a9e413d1
tmp$ openssl sha1 random.bin
SHA1(random.bin)= a511d8153426ebea4e4694cde78db4e3a9e413d1
So there's nothing wrong with your hashing method. Something is going wrong between its return value and it being printed.