Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
tl;dr: What are necessary control characters that custom file must have in order not to trigger [or trigger] badbit, failbit, badbit, eofbit dependant on OS?
I am working with Cygwin and Notepad++, using Windows 7 on an x64 laptop.
I have an file called genesis.o that I typed in hex editor that is in same directory as sandbox.cpp from which I am doing reading/writing. The file "genesis.o" contains:
genesis.o
-----------------
58 - 4f - 58 - 4f
58 - 58 - 58 - 58
3f - 3f - 3f - 3f
00 - 00 - 00 - 3f
00 - 00 - 3f - 00
00 - 3f - 00 - 00
3f - 00 - 00 - 00
00 - 00 - 00 - 58
00 - 00 - 58 - 00
00 - 58 - 00 - 00
58 - 00 - 00 - 00
48 - 45 - 4c - 4c
So far, I have read and manipulated custom structures without any regard of validity of the file. I used stat to check if file is accessible and that was enough.
This file (with and without extension ".o") passes all check ups I made except file.rdstate in which it always returns 4 (std::ios::failbit).
Error doesn't show itself on any other normal file, so I am guessing that some sort of control character sequence before/after or in file actually tells the std::fstream that file is valid.
Since no other file (except those typed in a hex editor) triggers this behaviour, is there a way to structure an custom binary file to be recognised by fstream? Some sort of control characters, preset flags etc.?
I am using std::ios:in | std::ios:binary. I am reading it by getting stat buffer.st_size -> divide it by 4 (since I read 4 byte integers) and:
uitn32_t temp = 0;
file.read( (char *)(&temp), sizeof(uint32_t) );
It is notable to mention, that I can read that binary file even if file.rdstate returns an failbit.
Minimal testable example. Just make an "genesis.o" file with character specification above.
#include <fstream>
#include <iostream>
#include <string>
#include <sys\types.h>
#include <sys\stat.h>
#include <vector>
struct handl{
std::string name = "genesis";
std::string ext = "o";
std::vector<uint32_t> mem;
bool acces = false;
struct stat buffer;
handl():mem(0),name("genesis"),ext("o"){}
const char *f_name(){
std::string f_n = this->name;
f_n.append(".");
f_n.append(this->ext);
return f_n.c_str();
}
void recheck(){
this->acces = ( stat(this->f_name(), &this->buffer ) == 0 );
}
virtual bool header( std::fstream &file )
{return true;}
virtual bool footer( std::fstream &file )
{return true;}
void operator()(){ this->recheck(); }
void operator()( const char *name, const char *ext ){
this->name = std::string(name);
this->ext = std::string(ext);
this->recheck();
}
void prefix( const char* pre ){
std::string pn(pre);
pn.append( this->name );
this->name = pn;
}
void suffix( const char *su ){
this->name.append(su);
this->recheck();
}
int read(){
this->recheck();
if( !this->acces ){return 0;}
std::fstream file;
file = std::fstream( this->f_name(), std::ios::in | std::ios::binary );
if( this->header(file) && this->footer(file) ){
int byte_size = this->buffer.st_size;
std::cout << file.rdstate() << std::endl;
std::cout << "gb\t" << std::ios::goodbit << std::endl;
std::cout << "bb\t" << std::ios::badbit << std::endl;
std::cout << "eb\t" << std::ios::eofbit << std::endl;
std::cout << "fb\t" << std::ios::failbit << std::endl;
file.close();
return 1;
}else{
file.close();
return 0;
}
}
};
int main(){
handl f;
std::cout << f.read() << std::endl;
return 0;
}
return f_n.c_str();
Your local f_n object is destroyed and the pointer to its internal memory is dangling. Using it is undefined behaviour. File content is irrelevant.
I tried your code, and this line: if( !this->acces ){return 0;} is failing for me. By "failing", I mean this is causing the read function to return 0.
I am trying to perform AES decryption using the crypto++ library. I have an encrypted file whose first 8 bytes are the filelength, subsequent 16 bytes are the initialization vector, and the remaining data is the data of interest. I also have a string representation of my key (which I hash using SHA256)
I get the following error when trying to perform AES decryption:
StreamTransformationFilter: invalid PKCS #7 block padding found
I am using the following c++ code:
std::string keyStr = "my_key";
std::string infilePath = "my/file/path";
CryptoPP::SHA256 hash;
unsigned char digest[CryptoPP::SHA256::DIGESTSIZE];
hash.CalculateDigest( digest, reinterpret_cast<const unsigned char*>(&keyStr[0]), keyStr.length() );
auto key = CryptoPP::SecByteBlock(digest, CryptoPP::SHA256::DIGESTSIZE);
std::ifstream fin(infilePath, std::ifstream::binary);
// First 8 bytes is the file size
std::vector<char> fileSizeVec(8);
fin.read(fileSizeVec.data(), fileSizeVec.size());
// Read the next 16 bytes to get the initialization vector
std::vector<char> ivBuffer(16);
fin.read(ivBuffer.data(), ivBuffer.size());
CryptoPP::SecByteBlock iv(reinterpret_cast<const unsigned char*>(ivBuffer.data()), ivBuffer.size());
// Create a CBC decryptor
CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryption;
decryption.SetKeyWithIV(key, sizeof(key), iv);
CryptoPP::StreamTransformationFilter decryptor(decryption);
std::vector<char> buffer(CHUNK_SIZE, 0);
while(fin.read(buffer.data(), buffer.size())) {
CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), buffer.size());
decryptor.Put(tmp, tmp.size());
}
decryptor.MessageEnd();
size_t retSize = decryptor.MaxRetrievable();
std::vector<char> decryptedBuff;
decryptedBuff.resize(retSize);
decryptor.Get(reinterpret_cast<CryptoPP::byte*>(decryptedBuff.data()), decryptedBuff.size());
I am not sure what is giving me the error. I am working off the following python code. When I run the python code with the same input file, it successfully decrypts the file.
def decrypt_file(in_filename, out_filename=None):
key = hashlib.sha256(PASSWORD).digest()
"""loads and returns the embedded model"""
chunksize = 24 * 1024
if not out_filename:
out_filename = os.path.splitext(in_filename)[0]
with open(in_filename, 'rb') as infile:
# get the initial 8 bytes with file size
tmp = infile.read(8)
iv = infile.read(16)
decryptor = AES.new(key, AES.MODE_CBC, iv)
string = b''
# with open(out_filename, 'wb') as outfile:
while True:
chunk = infile.read(chunksize)
if len(chunk) == 0:
break
string += decryptor.decrypt(chunk)
return string
In addition to solving the error, I would also love some general c++ coding feedback on how I can improve.
Thanks in advance!
Edit:
It looks like I wasn't reading the input file all the way to the end (as the length of the last chunk is smaller than CHUNK_SIZE). The following code now reads the entire file, however I still get the same issue. I have also confirmed that the IV and key match exactly that produced from the python code.
// Get the length of the file in bytes
fin.seekg (0, fin.end);
size_t fileLen = fin.tellg();
fin.seekg (0, fin.beg);
std::vector<char> buffer(CHUNK_SIZE, 0);
size_t readSize = CHUNK_SIZE;
while(fin.read(buffer.data(), readSize)) {
CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), CHUNK_SIZE);
decryptor.Put(tmp, tmp.size());
std::fill(buffer.begin(), buffer.end(), 0);
size_t bytesReamining = fileLen - fin.tellg();
readSize = CHUNK_SIZE < bytesReamining ? CHUNK_SIZE : bytesReamining;
if (!readSize)
break;
}
}
Note that I have tried this line as both CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), CHUNK_SIZE);
and CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), readSize); (Using CHUNK_SIZE pads with 0)
I have an encrypted file whose first 8 bytes are the filelength, subsequent 16 bytes are the initialization vector, and the remaining data is the data of interest...
I think I'll just cut to the chase and show you an easier way to do things with the Crypto++ library. The key and iv are hard-coded to simplify the code. The derivation is not needed for the example. By the way, if Python has it, you should consider using HKDF for derivation of the AES key and iv. HKDF has provable security properties.
Crypto++ handles the chunking for you. You don't need to explicitly perform it; see Pumping Data on the Crypto++ wiki.
I believe the Python code has a potential padding oracle present due to the use of CBC mode without a MAC. You might consider adding a MAC or using an Authenticated Encryption mode of operation.
#include "cryptlib.h"
#include "filters.h"
#include "osrng.h"
#include "modes.h"
#include "files.h"
#include "aes.h"
#include "hex.h"
#include <string>
#include <iostream>
const std::string infilePath = "test.dat";
int main(int argc, char* argv[])
{
using namespace CryptoPP;
const byte key[16] = {
1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4
};
const byte iv[16] = {
8,7,6,5, 8,7,6,5, 8,7,6,5, 8,7,6,5
};
const byte data[] = // 70 characters
"Now is the time for all good men to come to the aide of their country.";
HexEncoder encoder(new FileSink(std::cout));
std::string message;
// Show parameters
{
std::cout << "Key: ";
StringSource(key, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "IV: ";
StringSource(iv, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "Data: ";
StringSource(data, 70, true, new Redirector(encoder));
std::cout << std::endl;
}
// Write sample data
{
FileSink outFile(infilePath.c_str());
word64 length = 8+16+70;
outFile.PutWord64(length, BIG_ENDIAN_ORDER);
outFile.Put(iv, 16);
CBC_Mode<AES>::Encryption enc;
enc.SetKeyWithIV(key, 16, iv, 16);
StringSource(data, 70, true, new StreamTransformationFilter(enc, new Redirector(outFile)));
}
// Read sample data
{
FileSource inFile(infilePath.c_str(), true /*pumpAll*/);
word64 read, l;
read = inFile.GetWord64(l, BIG_ENDIAN_ORDER);
if (read != 8)
throw std::runtime_error("Failed to read length");
SecByteBlock v(16);
read = inFile.Get(v, 16);
if (read != 16)
throw std::runtime_error("Failed to read iv");
CBC_Mode<AES>::Decryption dec;
dec.SetKeyWithIV(key, 16, v, 16);
SecByteBlock d(l-8-16);
StreamTransformationFilter f(dec, new ArraySink(d, d.size()));
inFile.CopyTo(f);
f.MessageEnd();
std::cout << "Key: ";
StringSource(key, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "IV: ";
StringSource(v, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "Data: ";
StringSource(d, d.size(), true, new Redirector(encoder));
std::cout << std::endl;
message.assign(reinterpret_cast<const char*>(d.data()), d.size());
}
std::cout << "Message: ";
std::cout << message << std::endl;
return 0;
}
Running the program results in:
$ g++ test.cxx ./libcryptopp.a -o test.exe
$ ./test.exe
Key: 01020304010203040102030401020304
IV: 08070605080706050807060508070605
Data: 4E6F77206973207468652074696D6520666F7220616C6C20676F6F64206D656E20746F2063
6F6D6520746F207468652061696465206F6620746865697220636F756E7472792E
Key: 01020304010203040102030401020304
IV: 08070605080706050807060508070605
Data: 4E6F77206973207468652074696D6520666F7220616C6C20676F6F64206D656E20746F2063
6F6D6520746F207468652061696465206F6620746865697220636F756E7472792E
Message: Now is the time for all good men to come to the aide of their country.
Prior to this Stack Overflow question, the Crypto++ library did not provide PutWord64 and GetWord64. Interop with libraries like Python is important to the project, so they were added at Commit 6d69043403a9 and Commit 8260dd1e81c3. They will be part of the Crypto++ 8.3 release.
If you are working with Crypto++ 8.2 or below, you can perform the 64-bit read with the following code.
word64 length;
word32 h, l;
inFile.GetWord32(h, BIG_ENDIAN_ORDER);
inFile.GetWord32(l, BIG_ENDIAN_ORDER);
length = ((word64)h << 32) | l;
Here is the data file used for this example.
$ hexdump -C test.dat
00000000 00 00 00 00 00 00 00 5e 08 07 06 05 08 07 06 05 |.......^........|
00000010 08 07 06 05 08 07 06 05 b0 82 79 ee a6 d8 8a 0e |..........y.....|
00000020 a6 b3 a4 7e 63 bd 9a bc 0e e4 b6 be 3e eb 36 64 |...~c.......>.6d|
00000030 72 cd ba 91 8d e0 d3 c5 cd 64 ae c0 51 de a7 c9 |r........d..Q...|
00000040 1e a8 81 6d c0 d5 42 2a 17 5a 19 62 1e 9c ab fd |...m..B*.Z.b....|
00000050 21 3d b0 8f e2 b3 7a d4 08 8d ec 00 e0 1e 5e 78 |!=....z.......^x|
00000060 56 6d f5 3e 8c 5f fe 54 |Vm.>._.T|
Looks like the issue had to do with padding. I instead switched to using a StringSource, which only worked once I specified CryptoPP::BlockPaddingSchemeDef::BlockPaddingScheme::ZEROS_PADDING as an argument for StreamTransformationFilter
Here is the working code for anyone that is interested:
void Crypto::decryptFileAES(CryptoPP::SecByteBlock key, std::string infilePath) {
std::ifstream fin(infilePath, std::ifstream::binary);
// Get the length of the file in bytes
fin.seekg (0, fin.end);
size_t fileLen = fin.tellg();
fin.seekg (0, fin.beg);
// First 8 bytes is the file size
std::vector<char> fileSizeVec(8);
fin.read(fileSizeVec.data(), fileSizeVec.size());
// Read the first 16 bytes to get the initialization vector
std::vector<char> ivBuffer(16);
fin.read(ivBuffer.data(), ivBuffer.size());
CryptoPP::SecByteBlock iv(reinterpret_cast<const unsigned char*>(ivBuffer.data()), ivBuffer.size());
// Create a CBC decryptor
CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryption;
decryption.SetKeyWithIV(key, sizeof(key), iv);
size_t bytesReamining = fileLen - fin.tellg();
std::vector<char> buffer(bytesReamining);
if(!fin.read(buffer.data(), bytesReamining)) {
throw std::runtime_error("Unable to read file");
}
std::string decryptedText;
CryptoPP::StringSource ss(reinterpret_cast<const unsigned char*>(buffer.data()), buffer.size(), true,
new CryptoPP::StreamTransformationFilter(decryption,
new CryptoPP::StringSink(decryptedText), CryptoPP::BlockPaddingSchemeDef::BlockPaddingScheme::ZEROS_PADDING));
std::cout << decryptedText << std::endl;
}
I am getting a memory leak when I run try to read from Clipboard.
Sample code:
void SomeFunction()
{
OpenClipboard(nullptr);
HGLOBAL hglb = GetClipboardData(CF_TEXT);
char* ch = static_cast<char*>(GlobalLock(hglb));
CString rawClipboardData(ch);
GlobalUnlock(hglb);
CloseClipboard();
}
It is the middle row above which causes the memory leak according to Visual Studio. This row:
CString rawClipboardData(ch);
If I do not run it, there is no leak reported.
But if I do run it I get the following debug output in visual studio output window:
Detected memory leaks!
Dumping objects ->
f:\dd\vctools\vc7libs\ship\atlmfc\src\strcore.cpp(158) : {75645} normal block at 0x00000000072C89A0, 52 bytes long.
Data: <`x > 60 78 F7 D3 FE 07 00 00 0D 00 00 00 0D 00 00 00
Object dump complete.
Any ideas?
UPDATE: Added OpenClipboard(nullptr) in code above. Also in real code there are nullptr-checks. Just keeping it clean here to reduce amount of guard-clause code.
GlobalLock(hglb) should be a LPTSTR so I would assume that the leak is caused by the cast to char*. For Unicode platforms, TCHAR is defined as synonymous with the WCHAR type.
you should be able to do something like
CString rawClipboardData = GlobalLock(hglb);
If not then
CString rawClipboardData;
LPTSTR lptstr = GlobalLock(hglb);
rawClipboardData = lptstr;
will definitely work
Hi everyone i have an issue while reading binary data from a binary file as following:
File Content:
D3 EE EE 00 00 01 D7 C4 D9 40
char * afpContentBlock = new char[10];
ifstream inputStream(sInputFile, ios::in|ios::binary);
if (inputStream.is_open()))
{
inputStream.read(afpContentBlock, 10);
int n = sizeof(afpContentBlock)/sizeof(afpContentBlock[0]); // Print 4
// Here i would like to check every byte, but no matter how i convert the
// char[] afpContentBlock, it always cut at first byte 0x00.
}
I know this happens cause of the byte 0x00. Is there a way to manage it somehow ?
I have tried to write it with an ofstream object, and it works fine since it writes out the whole 10 bytes. Anyway i would like to loop through the whole byte array to check bytes value.
Thank you very much.
It's much easier to just get how many bytes you read from the ifstream like so:
if (inputStream.is_open()))
{
inputStream.read(afpContentBlock, 10);
int bytesRead = (int)inputStream.gcount();
for( int i = 0; i < bytesRead; i++ )
{
// check each byte however you want
// access with afpContentBlock[i]
}
}
I'm new to C++ and I've to do an assignment for school.
I need to copy a binary* file without using api calls or system integrated commands. At school we use a windows machine.
I've searched around a bit, and I found out that the best way to copy data without using any api's is to use iostream (ifstream/fstream)
Here's the code I'm using:
int Open(string Name){
int length;
char * buffer;
ifstream is;
fstream out;
FILE* pFile;
is.open (Name.c_str(), ios::binary );
// get length of file:
is.seekg (0, ios::end);
length = is.tellg();
is.seekg (0, ios::beg);
// allocate memory:
buffer = new char [length];
// read data as a block:
is.read (buffer,length);
is.close();
pFile = fopen ( "out.exe" , "w" );
fclose(pFile);
out.open("out.exe", ios::binary);
out.write( buffer, length);
out.close();
delete[] buffer;
return 0;
}
out.exe isnt working properly, and after looking at it in winhex.exe
I see that the data has been modefied, while I'm not doing anything with it
Can anyone help me?
*the file is a simple hello world program, it messageboxes "hello world"
EDIT:
Sorry for my unresponsiveness, It was sleeping.
Anyways, I've opened both (the result and the original) programs inside an hex editor.
It seems that with everything I try this line:
Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F
00000200 4C 00 00 00 00 30 00 00 00 02 00 00 00 0D 0A 00 L 0
Changes into this:
Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F
00000200 4C 00 00 00 00 30 00 00 00 02 00 00 00 0A 00 00 L 0
As you can or cannot see somehow during the reading or writing process a byte is being removed (or added, that sometimes happens as well)
Passing only ios_base::binary to fstream's ctor is not specified (in and/or out must be supplied too).
To avoid that, you could use ofstream (note the exra 'o') for out instead of fstream. As a bonus, this would avoid the need to first fopen with the "w" flag since ofstream's ctor creates the file by default.
is.read(buffer, length) is not guaranteed to read length bytes.
I forget if the same is true for out.write or not.
Lets make that a bit neater:
// Pass strings by const reference (just good habit)
// But may also save a copy. And it indicates that the function should
// not be messing with the name!
int Open(std::string const& Name, std::string const& out)
{
// Declare variables as close to use as possable.
// It is very C-Like to declare all the variables at the
// head of a function.
// Use the constructor to open the file.
std::ifstream is(Name.c_str(), ios::binary);
if (!is) // Failed to open
{ return -1;
}
// get length of file:
is.seekg (0, ios::end);
std::size_t length = is.tellg(); // Use the correct type. int is not correct
is.seekg (0, ios::beg);
// allocate memory:
// Using new/delete is risky. It makes the code not exception safe.
// Also because you have to manually tidy up the buffer you can not
// escape early. By using RAII the cleanup becomes automative and there
// is no need to track resources that need to be tidied.
//
// Look up the concept of RAII it makes C++ lfe so much easier.
// std::vector implements the new/delete internally using RAII
std::vector<char> buffer(length);
std::size_t read = 0;
do
{
// read does not gurantee that it will read everything asked for.
// so you need to do int a loop if you want to read the whole thing
// into a buffer.
is.read(&buffer[read], length - read);
std::size_t amount = is.gcount();
if (amount == 0)
{ return -2; // Something went wrong and it failed to read.
}
read += amount;
} while(length != read);
fstream out(out.c_str(), ios::binary );
if (!out)
{ return -3; // you may want to test this before spending all the time reading
}
// Probably need to loop like we did for read.
out.write( &buffer[0], length);
return 0;
}
Generally, files end in a newline. That 0d0a ("\r\n") might not be a readable part of the source file. Windows usually uses "\r\n" for newline, while UNIX uses just "\n". For some reason, when it writes a new file, it's using just 0a for the final newline. It might be interesting to see what happens if you read in and copy the file you wrote the first time.
The short answer is, this is just the kind of problem that crops up when you use a Windows system. :D
To hack it, you could always unconditionally write an extra "\r" as the last thing you output.
I think that
ifstream src(source.c_str(), ios::binary);
ofstream dest(destination.c_str(), ios::binary | ios::trunc);
dest << src.rdbuf();
src.close();
dest.close();
would do the trick.