Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
tl;dr: What are necessary control characters that custom file must have in order not to trigger [or trigger] badbit, failbit, badbit, eofbit dependant on OS?
I am working with Cygwin and Notepad++, using Windows 7 on an x64 laptop.
I have an file called genesis.o that I typed in hex editor that is in same directory as sandbox.cpp from which I am doing reading/writing. The file "genesis.o" contains:
genesis.o
-----------------
58 - 4f - 58 - 4f
58 - 58 - 58 - 58
3f - 3f - 3f - 3f
00 - 00 - 00 - 3f
00 - 00 - 3f - 00
00 - 3f - 00 - 00
3f - 00 - 00 - 00
00 - 00 - 00 - 58
00 - 00 - 58 - 00
00 - 58 - 00 - 00
58 - 00 - 00 - 00
48 - 45 - 4c - 4c
So far, I have read and manipulated custom structures without any regard of validity of the file. I used stat to check if file is accessible and that was enough.
This file (with and without extension ".o") passes all check ups I made except file.rdstate in which it always returns 4 (std::ios::failbit).
Error doesn't show itself on any other normal file, so I am guessing that some sort of control character sequence before/after or in file actually tells the std::fstream that file is valid.
Since no other file (except those typed in a hex editor) triggers this behaviour, is there a way to structure an custom binary file to be recognised by fstream? Some sort of control characters, preset flags etc.?
I am using std::ios:in | std::ios:binary. I am reading it by getting stat buffer.st_size -> divide it by 4 (since I read 4 byte integers) and:
uitn32_t temp = 0;
file.read( (char *)(&temp), sizeof(uint32_t) );
It is notable to mention, that I can read that binary file even if file.rdstate returns an failbit.
Minimal testable example. Just make an "genesis.o" file with character specification above.
#include <fstream>
#include <iostream>
#include <string>
#include <sys\types.h>
#include <sys\stat.h>
#include <vector>
struct handl{
std::string name = "genesis";
std::string ext = "o";
std::vector<uint32_t> mem;
bool acces = false;
struct stat buffer;
handl():mem(0),name("genesis"),ext("o"){}
const char *f_name(){
std::string f_n = this->name;
f_n.append(".");
f_n.append(this->ext);
return f_n.c_str();
}
void recheck(){
this->acces = ( stat(this->f_name(), &this->buffer ) == 0 );
}
virtual bool header( std::fstream &file )
{return true;}
virtual bool footer( std::fstream &file )
{return true;}
void operator()(){ this->recheck(); }
void operator()( const char *name, const char *ext ){
this->name = std::string(name);
this->ext = std::string(ext);
this->recheck();
}
void prefix( const char* pre ){
std::string pn(pre);
pn.append( this->name );
this->name = pn;
}
void suffix( const char *su ){
this->name.append(su);
this->recheck();
}
int read(){
this->recheck();
if( !this->acces ){return 0;}
std::fstream file;
file = std::fstream( this->f_name(), std::ios::in | std::ios::binary );
if( this->header(file) && this->footer(file) ){
int byte_size = this->buffer.st_size;
std::cout << file.rdstate() << std::endl;
std::cout << "gb\t" << std::ios::goodbit << std::endl;
std::cout << "bb\t" << std::ios::badbit << std::endl;
std::cout << "eb\t" << std::ios::eofbit << std::endl;
std::cout << "fb\t" << std::ios::failbit << std::endl;
file.close();
return 1;
}else{
file.close();
return 0;
}
}
};
int main(){
handl f;
std::cout << f.read() << std::endl;
return 0;
}
return f_n.c_str();
Your local f_n object is destroyed and the pointer to its internal memory is dangling. Using it is undefined behaviour. File content is irrelevant.
I tried your code, and this line: if( !this->acces ){return 0;} is failing for me. By "failing", I mean this is causing the read function to return 0.
Related
I have a utility that should copy files from one location to another.
The problem I have is when reading X bytes using the QDataStream and writing it, the number of bytes being read/written exceeds the number of bytes the file has. I see this problem happen with a number of files.
I am using a QDataStream::readRawData() and QDataStream::writeRawData() to facilitate reading/writing to and from files as shown below
QDataStream in(&sourceFile);
QDataStream out(&newFile);
// Read/Write byte containers
qint64 fileBytesRead = 0;
quint64 fileBytesWritten = 0;
qint64 bytesWrittenNow = 0;
quint8* buffer = new quint8[bufSize];
while ((fileBytesRead = in.readRawData((char*)buffer, bufSize)) != 0) {
// Check if we have a read/write mismatch
if (fileBytesRead == -1) {
printCritical(TAG, QString("Mismatch read/write: [R:%1/W:%2], total file write/max [W:%3/M:%4]. File may be corrupted, skipping...").arg(QString::number(fileBytesRead), QString::number(bytesWrittenNow), QString::number(fileBytesWritten), QString::number(storageFile.size)));
// close source file handle
sourceFile.close();
// Close file handle
newFile.close();
return BackupResult::IOError;
}
// Write buffer to file stream
bytesWrittenNow = out.writeRawData((const char*)buffer, fileBytesRead);
// Check if we have a read/write mismatch
if (bytesWrittenNow == -1) {
printCritical(TAG, QString("Mismatch read/write: [R:%1/W:%2], total file write/max [W:%3/M:%4]. File may be corrupted, skipping...").arg(QString::number(fileBytesRead), QString::number(bytesWrittenNow), QString::number(fileBytesWritten), QString::number(storageFile.size)));
// close source file handle
sourceFile.close();
// Close file handle
newFile.close();
return BackupResult::IOError;
}
// Add current buffer size to written bytes
fileBytesWritten += bytesWrittenNow;
if(fileBytesWritten > storageFile.size) {
qWarning() << "Extra bytes read/written exceeding file length"; <================= this line is hit every now and then
}
//...
This problem isn't consistent, but it happens every now and then, I have no idea why. Anyone have thoughts on a possible cause?
The name of the function QDataStream::writeRawData() sounds like ideal for writing binary data. Unfortunately, that's only half of the story.
The open-mode of the file is relevant as well under certain conditions – e.g. if the QFile is opened on Windows with QIODevice::Text:
QIODevice::Text
When reading, the end-of-line terminators are translated to '\n'. When writing, the end-of-line terminators are translated to the local encoding, for example '\r\n' for Win32.
I prepared an MCVE to demonstrate that:
// Qt header:
#include <QtCore>
void write(const QString &fileName, const char *data, size_t size, QIODevice::OpenMode mode)
{
qDebug() << "Open file" << fileName;
QFile qFile(fileName);
qFile.open(mode | QIODevice::WriteOnly);
QDataStream out(&qFile);
const int ret = out.writeRawData(data, size);
qDebug() << ret << "bytes written.";
}
// main application
int main(int argc, char **argv)
{
const char data[] = {
'\x00', '\x01', '\x02', '\x03', '\x04', '\x05', '\x06', '\x07',
'\x08', '\x09', '\x0a', '\x0b', '\x0c', '\x0d', '\x0e', '\x0f'
};
const size_t size = sizeof data / sizeof *data;
write("data.txt", data, size, 0);
write("test.txt", data, size, QIODevice::Text);
}
Built and tested in VS2017 on Windows 10:
Open file "data.txt"
16 bytes written.
Open file "test.txt"
16 bytes written.
Result inspected with the help of cygwin:
$ ls -l *.txt
-rwxrwx---+ 1 scheff Domänen-Benutzer 427 Jun 23 08:24 CMakeLists.txt
-rwxrwx---+ 1 scheff Domänen-Benutzer 16 Jun 23 08:37 data.txt
-rwxrwx---+ 1 scheff Domänen-Benutzer 17 Jun 23 08:37 test.txt
$
data.txt has 16 bytes as expected but test.txt has 17 bytes. Oops!
$ hexdump -C data.txt
00000000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f |................|
00000010
$ hexdump -C test.txt
00000000 00 01 02 03 04 05 06 07 08 09 0d 0a 0b 0c 0d 0e |................|
00000010 0f |.|
00000011
$
Obviously, the underlying Windows file function “corrected” the \n to \r\n – 09 0a 0b became 09 0d 0a 0b. Hence, there occurs one additional byte which was not part of the originally written data.
Similar effects may happen when the QFile is opened for reading with QIODevice::Text involved.
I am trying to perform AES decryption using the crypto++ library. I have an encrypted file whose first 8 bytes are the filelength, subsequent 16 bytes are the initialization vector, and the remaining data is the data of interest. I also have a string representation of my key (which I hash using SHA256)
I get the following error when trying to perform AES decryption:
StreamTransformationFilter: invalid PKCS #7 block padding found
I am using the following c++ code:
std::string keyStr = "my_key";
std::string infilePath = "my/file/path";
CryptoPP::SHA256 hash;
unsigned char digest[CryptoPP::SHA256::DIGESTSIZE];
hash.CalculateDigest( digest, reinterpret_cast<const unsigned char*>(&keyStr[0]), keyStr.length() );
auto key = CryptoPP::SecByteBlock(digest, CryptoPP::SHA256::DIGESTSIZE);
std::ifstream fin(infilePath, std::ifstream::binary);
// First 8 bytes is the file size
std::vector<char> fileSizeVec(8);
fin.read(fileSizeVec.data(), fileSizeVec.size());
// Read the next 16 bytes to get the initialization vector
std::vector<char> ivBuffer(16);
fin.read(ivBuffer.data(), ivBuffer.size());
CryptoPP::SecByteBlock iv(reinterpret_cast<const unsigned char*>(ivBuffer.data()), ivBuffer.size());
// Create a CBC decryptor
CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryption;
decryption.SetKeyWithIV(key, sizeof(key), iv);
CryptoPP::StreamTransformationFilter decryptor(decryption);
std::vector<char> buffer(CHUNK_SIZE, 0);
while(fin.read(buffer.data(), buffer.size())) {
CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), buffer.size());
decryptor.Put(tmp, tmp.size());
}
decryptor.MessageEnd();
size_t retSize = decryptor.MaxRetrievable();
std::vector<char> decryptedBuff;
decryptedBuff.resize(retSize);
decryptor.Get(reinterpret_cast<CryptoPP::byte*>(decryptedBuff.data()), decryptedBuff.size());
I am not sure what is giving me the error. I am working off the following python code. When I run the python code with the same input file, it successfully decrypts the file.
def decrypt_file(in_filename, out_filename=None):
key = hashlib.sha256(PASSWORD).digest()
"""loads and returns the embedded model"""
chunksize = 24 * 1024
if not out_filename:
out_filename = os.path.splitext(in_filename)[0]
with open(in_filename, 'rb') as infile:
# get the initial 8 bytes with file size
tmp = infile.read(8)
iv = infile.read(16)
decryptor = AES.new(key, AES.MODE_CBC, iv)
string = b''
# with open(out_filename, 'wb') as outfile:
while True:
chunk = infile.read(chunksize)
if len(chunk) == 0:
break
string += decryptor.decrypt(chunk)
return string
In addition to solving the error, I would also love some general c++ coding feedback on how I can improve.
Thanks in advance!
Edit:
It looks like I wasn't reading the input file all the way to the end (as the length of the last chunk is smaller than CHUNK_SIZE). The following code now reads the entire file, however I still get the same issue. I have also confirmed that the IV and key match exactly that produced from the python code.
// Get the length of the file in bytes
fin.seekg (0, fin.end);
size_t fileLen = fin.tellg();
fin.seekg (0, fin.beg);
std::vector<char> buffer(CHUNK_SIZE, 0);
size_t readSize = CHUNK_SIZE;
while(fin.read(buffer.data(), readSize)) {
CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), CHUNK_SIZE);
decryptor.Put(tmp, tmp.size());
std::fill(buffer.begin(), buffer.end(), 0);
size_t bytesReamining = fileLen - fin.tellg();
readSize = CHUNK_SIZE < bytesReamining ? CHUNK_SIZE : bytesReamining;
if (!readSize)
break;
}
}
Note that I have tried this line as both CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), CHUNK_SIZE);
and CryptoPP::SecByteBlock tmp(reinterpret_cast<const unsigned char*>(buffer.data()), readSize); (Using CHUNK_SIZE pads with 0)
I have an encrypted file whose first 8 bytes are the filelength, subsequent 16 bytes are the initialization vector, and the remaining data is the data of interest...
I think I'll just cut to the chase and show you an easier way to do things with the Crypto++ library. The key and iv are hard-coded to simplify the code. The derivation is not needed for the example. By the way, if Python has it, you should consider using HKDF for derivation of the AES key and iv. HKDF has provable security properties.
Crypto++ handles the chunking for you. You don't need to explicitly perform it; see Pumping Data on the Crypto++ wiki.
I believe the Python code has a potential padding oracle present due to the use of CBC mode without a MAC. You might consider adding a MAC or using an Authenticated Encryption mode of operation.
#include "cryptlib.h"
#include "filters.h"
#include "osrng.h"
#include "modes.h"
#include "files.h"
#include "aes.h"
#include "hex.h"
#include <string>
#include <iostream>
const std::string infilePath = "test.dat";
int main(int argc, char* argv[])
{
using namespace CryptoPP;
const byte key[16] = {
1,2,3,4, 1,2,3,4, 1,2,3,4, 1,2,3,4
};
const byte iv[16] = {
8,7,6,5, 8,7,6,5, 8,7,6,5, 8,7,6,5
};
const byte data[] = // 70 characters
"Now is the time for all good men to come to the aide of their country.";
HexEncoder encoder(new FileSink(std::cout));
std::string message;
// Show parameters
{
std::cout << "Key: ";
StringSource(key, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "IV: ";
StringSource(iv, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "Data: ";
StringSource(data, 70, true, new Redirector(encoder));
std::cout << std::endl;
}
// Write sample data
{
FileSink outFile(infilePath.c_str());
word64 length = 8+16+70;
outFile.PutWord64(length, BIG_ENDIAN_ORDER);
outFile.Put(iv, 16);
CBC_Mode<AES>::Encryption enc;
enc.SetKeyWithIV(key, 16, iv, 16);
StringSource(data, 70, true, new StreamTransformationFilter(enc, new Redirector(outFile)));
}
// Read sample data
{
FileSource inFile(infilePath.c_str(), true /*pumpAll*/);
word64 read, l;
read = inFile.GetWord64(l, BIG_ENDIAN_ORDER);
if (read != 8)
throw std::runtime_error("Failed to read length");
SecByteBlock v(16);
read = inFile.Get(v, 16);
if (read != 16)
throw std::runtime_error("Failed to read iv");
CBC_Mode<AES>::Decryption dec;
dec.SetKeyWithIV(key, 16, v, 16);
SecByteBlock d(l-8-16);
StreamTransformationFilter f(dec, new ArraySink(d, d.size()));
inFile.CopyTo(f);
f.MessageEnd();
std::cout << "Key: ";
StringSource(key, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "IV: ";
StringSource(v, 16, true, new Redirector(encoder));
std::cout << std::endl;
std::cout << "Data: ";
StringSource(d, d.size(), true, new Redirector(encoder));
std::cout << std::endl;
message.assign(reinterpret_cast<const char*>(d.data()), d.size());
}
std::cout << "Message: ";
std::cout << message << std::endl;
return 0;
}
Running the program results in:
$ g++ test.cxx ./libcryptopp.a -o test.exe
$ ./test.exe
Key: 01020304010203040102030401020304
IV: 08070605080706050807060508070605
Data: 4E6F77206973207468652074696D6520666F7220616C6C20676F6F64206D656E20746F2063
6F6D6520746F207468652061696465206F6620746865697220636F756E7472792E
Key: 01020304010203040102030401020304
IV: 08070605080706050807060508070605
Data: 4E6F77206973207468652074696D6520666F7220616C6C20676F6F64206D656E20746F2063
6F6D6520746F207468652061696465206F6620746865697220636F756E7472792E
Message: Now is the time for all good men to come to the aide of their country.
Prior to this Stack Overflow question, the Crypto++ library did not provide PutWord64 and GetWord64. Interop with libraries like Python is important to the project, so they were added at Commit 6d69043403a9 and Commit 8260dd1e81c3. They will be part of the Crypto++ 8.3 release.
If you are working with Crypto++ 8.2 or below, you can perform the 64-bit read with the following code.
word64 length;
word32 h, l;
inFile.GetWord32(h, BIG_ENDIAN_ORDER);
inFile.GetWord32(l, BIG_ENDIAN_ORDER);
length = ((word64)h << 32) | l;
Here is the data file used for this example.
$ hexdump -C test.dat
00000000 00 00 00 00 00 00 00 5e 08 07 06 05 08 07 06 05 |.......^........|
00000010 08 07 06 05 08 07 06 05 b0 82 79 ee a6 d8 8a 0e |..........y.....|
00000020 a6 b3 a4 7e 63 bd 9a bc 0e e4 b6 be 3e eb 36 64 |...~c.......>.6d|
00000030 72 cd ba 91 8d e0 d3 c5 cd 64 ae c0 51 de a7 c9 |r........d..Q...|
00000040 1e a8 81 6d c0 d5 42 2a 17 5a 19 62 1e 9c ab fd |...m..B*.Z.b....|
00000050 21 3d b0 8f e2 b3 7a d4 08 8d ec 00 e0 1e 5e 78 |!=....z.......^x|
00000060 56 6d f5 3e 8c 5f fe 54 |Vm.>._.T|
Looks like the issue had to do with padding. I instead switched to using a StringSource, which only worked once I specified CryptoPP::BlockPaddingSchemeDef::BlockPaddingScheme::ZEROS_PADDING as an argument for StreamTransformationFilter
Here is the working code for anyone that is interested:
void Crypto::decryptFileAES(CryptoPP::SecByteBlock key, std::string infilePath) {
std::ifstream fin(infilePath, std::ifstream::binary);
// Get the length of the file in bytes
fin.seekg (0, fin.end);
size_t fileLen = fin.tellg();
fin.seekg (0, fin.beg);
// First 8 bytes is the file size
std::vector<char> fileSizeVec(8);
fin.read(fileSizeVec.data(), fileSizeVec.size());
// Read the first 16 bytes to get the initialization vector
std::vector<char> ivBuffer(16);
fin.read(ivBuffer.data(), ivBuffer.size());
CryptoPP::SecByteBlock iv(reinterpret_cast<const unsigned char*>(ivBuffer.data()), ivBuffer.size());
// Create a CBC decryptor
CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryption;
decryption.SetKeyWithIV(key, sizeof(key), iv);
size_t bytesReamining = fileLen - fin.tellg();
std::vector<char> buffer(bytesReamining);
if(!fin.read(buffer.data(), bytesReamining)) {
throw std::runtime_error("Unable to read file");
}
std::string decryptedText;
CryptoPP::StringSource ss(reinterpret_cast<const unsigned char*>(buffer.data()), buffer.size(), true,
new CryptoPP::StreamTransformationFilter(decryption,
new CryptoPP::StringSink(decryptedText), CryptoPP::BlockPaddingSchemeDef::BlockPaddingScheme::ZEROS_PADDING));
std::cout << decryptedText << std::endl;
}
Problem solved! Thanks to #ewindes! I may go further with mu project! Bug in Notepad++ Hex Editor Plugin v0.9.5. I open with WinHex and Xplore and see 79 00 00 10. I trust too match...
I expect see in hex editor 79 00 00 10 but see 79 e1 80 80.
Using codeblocks with mingw 5.3.0. Tried visual studio 10 and got same result.
Why can't save variable right?
Code is simple and looks correct:
#include <iostream>
#include <fstream>
#include <stdint.h>
using namespace std;
const uint32_t uid1 = 0x10000079;
int main()
{
std::fstream fs;
fs.open("e32.exe", std::fstream::trunc | std::fstream::binary | std::fstream::out);
fs.write((const char*)&uid1, sizeof(uid1));
fs.close();
cout << "Hello world!" << endl;
cout << "sizeof(uid1): " << sizeof(uid1) << endl;
return 0;
}
I play more with code and got very strange results. File "test" contain crap as above:
FILE *fp;
if((fp=fopen("test", "wb"))==NULL) {
printf("Cannot open file.");
return 1;
}
char *data = (char *)&uid1;
fwrite(&data[0], 1, 1, fp);
printf("data[0] %x\n", data[0]);
fwrite(&data[1], 1, 1, fp);
printf("data[1] %x\n", data[1]);
fwrite(&data[2], 1, 1, fp);
printf("data[2] %x\n", data[2]);
fwrite(&data[3], 1, 1, fp);
printf("data[3] %x\n", data[3]);
fclose (fp);
But uid1 contain valid data:
data[0] 79
data[1] 0
data[2] 0
data[3] 10
Bug in Notepad++ Hex Editor Plugin v0.9.5. I open with WinHex and Xplore and see 79 00 00 10. I trust that too match...
I recently experienced a memory leak issue. I troubleshooted the issue for quite a long time and subsequently found out that throwing an exception (I use my own exception classes) causes this memory leak. The code of throwing the exception is as following:
HINSTANCE lib = LoadLibrary(path.c_str());
if(!lib)
{
DWORD werror = GetLastError();
ostringstream stream;
stream << werror;
string errstring = "Error " + stream.str();
errstring.append(" - " + libraryName);
throw LibraryLoadException(MOD_ERROR_LIB_LOAD, errstring.c_str());
}
The resulting output looks like:
Detected memory leaks!
Dumping objects ->
{351} normal block at 0x0044D208, 32 bytes long.
Data: <Error 126 - note> 45 72 72 6F 72 20 31 32 36 20 2D 20 6E 6F 74 65
{347} normal block at 0x0043BD98, 8 bytes long.
Data: <4 > > 34 F2 3E 00 00 00 00 00
{344} normal block at 0x0043FDE8, 32 bytes long.
Data: <126 > 31 32 36 CD CD CD CD CD CD CD CD CD CD CD CD CD
{302} normal block at 0x004409D8, 8 bytes long.
Data: <4 > > 34 F3 3E 00 00 00 00 00
{301} normal block at 0x0043FAF0, 8 bytes long.
Data: <P > > 50 F3 3E 00 00 00 00 00
Object dump complete.
As seen in the output of the visual studio leak CrtDbg, there are actual values of the objects used in the if block. All these objects including the exception itself (and all its attributes) are allocated on stack, though, so there cannot be a fault of me forgetting to deallocate something on heap.
I empirically tested this and the leak is definitely caused by the objects in the if block (after removing several objects like the string, DWORD and the stream the leaks grow fewer).
Can anyone tell me what am I doing (or what is) wrong over here?
Thank You in advance
As for the comments asking for more detailed code, here is the method causing memory leak:
void ModuleLoader::load(std::string libraryName, std::string type, void(CALLBACK * receiveData)(Message))
{
path = s2ws(libraryName); // conversion to wide string
HINSTANCE lib = LoadLibrary(path.c_str());
if(!lib)
{
DWORD werror = GetLastError();
ostringstream stream;
stream << werror;
string errstring = "Error " + stream.str();
errstring.append(" - " + libraryName);
throw LibraryLoadException(MOD_ERROR_LIB_LOAD, errstring.c_str());
}
DllModule *module = new DllModule(libraryName, lib);
module->setModType(type);
try
{
startModule(module, receiveData);
moduleMap.insert(std::pair<std::string, DllModule *>(type, module));
}
catch (ProbeCoreException e)
{
delete module;
throw e;
}
}
It is a method of a singleton class that loads dynamic modules, defined as following:
class ModuleLoader
{
// Function pointer definitions
typedef void (*StopFuncPointer)();
typedef int (*StartFuncPointer)(void(CALLBACK * receiveData)(Message));
typedef void (*SetDataFunctionPointer)(Message);
private:
std::map<std::string, DllModule *> moduleMap; // map of loaded modules
std::wstring path;
std::wstring s2ws(const std::string &s);
void startModule(DllModule * module, void(CALLBACK * receiveData)(Message));
void stopModule(DllModule * module);
// singleton functions
ModuleLoader() {}; // private constructor
ModuleLoader(ModuleLoader const&);
void operator = (ModuleLoader const&);
public:
void load(std::string libraryName, std::string type, void(CALLBACK * receiveData)(Message));
void unload(std::string libraryType);
void unloadAll();
vector<DllModule> getLoadedModules();
int containsModuleType(string modType);
HINSTANCE getModuleLibraryByType(std::string type);
// singleton getInstance function
static ModuleLoader & getInstance()
{
static ModuleLoader instance;
return instance;
}
};
The s2ws method converts a common string to wide string (i post it just in case):
std::wstring ModuleLoader::s2ws(const std::string& s)
{
int len;
int slength = (int)s.length() + 1;
len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0);
wchar_t* buf = new wchar_t[len];
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, buf, len);
std::wstring r(buf);
delete[] buf;
return r;
}
I checked many times and deallocating heap objects when the exception is thrown should be carried out at all levels of the application.
Furthermore, if I do remove the DWORD, ostringstream and string objects (allocated on stack), the memory leaks grow fewer...so it HAS to be in connection with these as well. I cannot imagine how removing this part of the code should help heap memory deallocation elsewhere:
DWORD werror = GetLastError();
ostringstream stream;
stream << werror;
string errstring = "Error " + stream.str();
errstring.append(" - " + libraryName);
OK, I managed to reduce the leaks to just two of original 5:
Dumping objects ->
{312} normal block at 0x0045FDC8, 8 bytes long.
Data: <( ( > 28 ED 28 00 00 00 00 00
{311} normal block at 0x0045F810, 8 bytes long.
Data: <D ( > 44 ED 28 00 00 00 00 00
Object dump complete.
I used _CrtSetBreakAlloc(x) function with the x being a number of the leak (e.g. 311 or 312 in the case like above) and found out, where is the unallocated memory allocated. It is really difficult to believe it, but the allocations really occured on these lines:
string errstring = "Error " + stream.str();
and
errstring.append(" - " + libraryName);
I removed the leaks by making the string and stream dynamically allocated on heap, then creating the exception and storing it in a temporary variable, subsequently deallocating the string and stream variables and finally throwing the exception itself:
DWORD werror = GetLastError();
ostringstream *stream = new ostringstream();
*stream << werror;
string *errstring = new string("Error ");
errstring->append(stream->str());
errstring->append(" - ");
errstring->append(libraryName);
ProbeCoreException e = LibraryLoadException(MOD_ERROR_LIB_LOAD,
errstring->c_str());
delete errstring;
delete stream;
throw e;
The last two allocations occur (once again unbelievably) during passing string parameters to the "load" function itself:
loader.load("notexisting.dll", "TEST", &callbackFunction);
I was contemplating about the leaks occuring due to the class being a singleton, the class was created according to leak-proof singleton rules, though, mentioned here:
C++ Singleton design pattern
It seems that the only chance how to get rid of the resting leaks is to pass string pointers even as parameters and then dealloc them explicitly...
I'm new to C++ and I've to do an assignment for school.
I need to copy a binary* file without using api calls or system integrated commands. At school we use a windows machine.
I've searched around a bit, and I found out that the best way to copy data without using any api's is to use iostream (ifstream/fstream)
Here's the code I'm using:
int Open(string Name){
int length;
char * buffer;
ifstream is;
fstream out;
FILE* pFile;
is.open (Name.c_str(), ios::binary );
// get length of file:
is.seekg (0, ios::end);
length = is.tellg();
is.seekg (0, ios::beg);
// allocate memory:
buffer = new char [length];
// read data as a block:
is.read (buffer,length);
is.close();
pFile = fopen ( "out.exe" , "w" );
fclose(pFile);
out.open("out.exe", ios::binary);
out.write( buffer, length);
out.close();
delete[] buffer;
return 0;
}
out.exe isnt working properly, and after looking at it in winhex.exe
I see that the data has been modefied, while I'm not doing anything with it
Can anyone help me?
*the file is a simple hello world program, it messageboxes "hello world"
EDIT:
Sorry for my unresponsiveness, It was sleeping.
Anyways, I've opened both (the result and the original) programs inside an hex editor.
It seems that with everything I try this line:
Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F
00000200 4C 00 00 00 00 30 00 00 00 02 00 00 00 0D 0A 00 L 0
Changes into this:
Offset 0 1 2 3 4 5 6 7 8 9 A B C D E F
00000200 4C 00 00 00 00 30 00 00 00 02 00 00 00 0A 00 00 L 0
As you can or cannot see somehow during the reading or writing process a byte is being removed (or added, that sometimes happens as well)
Passing only ios_base::binary to fstream's ctor is not specified (in and/or out must be supplied too).
To avoid that, you could use ofstream (note the exra 'o') for out instead of fstream. As a bonus, this would avoid the need to first fopen with the "w" flag since ofstream's ctor creates the file by default.
is.read(buffer, length) is not guaranteed to read length bytes.
I forget if the same is true for out.write or not.
Lets make that a bit neater:
// Pass strings by const reference (just good habit)
// But may also save a copy. And it indicates that the function should
// not be messing with the name!
int Open(std::string const& Name, std::string const& out)
{
// Declare variables as close to use as possable.
// It is very C-Like to declare all the variables at the
// head of a function.
// Use the constructor to open the file.
std::ifstream is(Name.c_str(), ios::binary);
if (!is) // Failed to open
{ return -1;
}
// get length of file:
is.seekg (0, ios::end);
std::size_t length = is.tellg(); // Use the correct type. int is not correct
is.seekg (0, ios::beg);
// allocate memory:
// Using new/delete is risky. It makes the code not exception safe.
// Also because you have to manually tidy up the buffer you can not
// escape early. By using RAII the cleanup becomes automative and there
// is no need to track resources that need to be tidied.
//
// Look up the concept of RAII it makes C++ lfe so much easier.
// std::vector implements the new/delete internally using RAII
std::vector<char> buffer(length);
std::size_t read = 0;
do
{
// read does not gurantee that it will read everything asked for.
// so you need to do int a loop if you want to read the whole thing
// into a buffer.
is.read(&buffer[read], length - read);
std::size_t amount = is.gcount();
if (amount == 0)
{ return -2; // Something went wrong and it failed to read.
}
read += amount;
} while(length != read);
fstream out(out.c_str(), ios::binary );
if (!out)
{ return -3; // you may want to test this before spending all the time reading
}
// Probably need to loop like we did for read.
out.write( &buffer[0], length);
return 0;
}
Generally, files end in a newline. That 0d0a ("\r\n") might not be a readable part of the source file. Windows usually uses "\r\n" for newline, while UNIX uses just "\n". For some reason, when it writes a new file, it's using just 0a for the final newline. It might be interesting to see what happens if you read in and copy the file you wrote the first time.
The short answer is, this is just the kind of problem that crops up when you use a Windows system. :D
To hack it, you could always unconditionally write an extra "\r" as the last thing you output.
I think that
ifstream src(source.c_str(), ios::binary);
ofstream dest(destination.c_str(), ios::binary | ios::trunc);
dest << src.rdbuf();
src.close();
dest.close();
would do the trick.