I am trying to convert a managed byte array to std::string in my C++/CLI wrapper; however, I am seeing some corrupt memory in the heap later on. Just wanted to check if I am doing the conversion right. Below is my method is CLI:
string ByteArrayToStr(array<Byte>^ byteArray)
{
int size = byteArray.Length;
IntPtr pnt = Marshal::AllocHGlobal(size + 1);
char* chararray = (char*)pnt.ToPointer();
try
{
Marshal::Copy(byteArray, 0, pnt, size);
*(chararray + size) = 0;
return string(chararray);
}
finally
{
Marshal::FreeHGlobal(pnt);
}
}
Does anything seem wrong in above code?
You are doing an unnecessary explicit copy and playing with a manual memory allocation.
You could just pass the raw pinned pointer to std::string constructor:
string ByteArrayToStr(array<Byte>^ byteArray)
{
pin_ptr<unsigned char> temp = &byteArray[0];
return string(reinterpret_cast<char*>(temp), byteArray->Length);
}
Related
Environment: Windows 7 pro x64, Microsoft Visual Studio 2015 Enterprise, Version 14.0.25424.00 Update 3
int testFunction()
{
std::string _orig = "[188 80% (1/2)O:152]";
std::string _orig2 = "[999 99% (1/1)O:999]";
char *orig = NULL;
char *orig2 = NULL;
orig = new char[_orig.length() + 1];
strcpy(orig, _orig.c_str());
orig2 = new char[_orig2.length() + 1];
strcpy(orig2, _orig2.c_str());
*orig++;
*orig2++;
int a = atoi(orig);
int b = atoi(orig2);
delete[] orig;
delete[] orig2;
return 0;
}
Running the above code crashes with the "_CrtIsValidHeapPointer(block)" error.
If I don't iterate (*orig++ and *orig2++), then no issues.
So my question is, how can I iterate through the pointers and then when I'm done doing what I need to do with them, delete[] them correctly?
You did not delete the pointers you allocated!
delete must be called on the original memory address returned by new. Since you did orig++, you cant delete the address being pointed at!
Iterating can be done with an index, and using array subscription to dereference:
orig[i] = 'a';
Which is the same as doing this:
*(orig+i) = 'a';
Or you can get another pointer onto the same data, and modify this one.
char* pOrig = orig;
++pOrig;
Why did you write
*orig++; // why dereferencing?
Just ++ by itself would do the iteration.
Avoid to use raw pointers. Your code can be simpler:
std::string orig = "[188 80% (1/2)O:152]";
std::string orig2 = "[999 99% (1/1)O:999]";
int a = atoi(orig.c_str() + 1);
int b = atoi(orig2.c_str() + 1);
Your mistake is that you try to delete the shifted pointers instead of the original pointers. As the result heap manager gets wrong allocated block information usually put before the allocated pointer and you got heap corruption.
how can I iterate through the pointers and then when I'm done doing what I need to do with them, delete[] them correctly?
Create a copy of the pointer:
char* orig = new char[size];
char* i = orig;
*i++ = 'a';
delete orig;
A perhaps more common idiom is to dereference a temporary:
for(int i = 0; i < size - 1; i++)
orig[i] = 'a';
I would love to [use std::string], but I need to use atoi(), which won't work on std::string
You are mistaken. atoi works with std::string just fine. Simply use std::string::c_str() just like you did with strcpy. There is absolutely no reason to allocate a block of memory with new.
int testFunction()
{
std::string _orig = "[188 80% (1/2)O:152]";
int a = 0;
for (std::string::iterator it = _orig.begin(); it != _orig.end(); ++it)
{
if (isdigit((char)*it))
a = (atoi(it._Ptr));
}
return 0;
}
I got it. Thanks for everyone who helped me come to this conclusion. Staying with std::string was in fact the best approach.
I have a function which is used a million times in my code and there is a memory leak in it.
It takes a const wchar_t* as input and returns a const char*.
Now I understand that this pointer returned (const char*) would be required to get an explicit delete[] from the calling function (but I cannot afford it) as this would mean I need to change it on all locations.
The code is like:
inline const char * W2N(const wchar_t* wstr)
{
int cw=lstrlenW(wstr);
if (cw==0)
{
CHAR *psz=new CHAR[1]; *psz='\0'; return psz;
}
int cc=WideCharToMultiByte(CP_ACP,0,wstr,cw,NULL,0,NULL,NULL);
if (cc==0) return NULL;
CHAR *psz=new CHAR[cc+1];
cc=WideCharToMultiByte(CP_ACP,0,wstr,cw,psz,cc,NULL,NULL);
if (cc==0)
{
delete[] psz; return NULL;
}
psz[cc]='\0';
return psz;
}
Is there anything that I can do around this function to avoid memory leak.
You can, and you should, wrap your pointer inside a std::unique_ptr<char[]>. This will solve your exact problem, in an idiomatic C++ way.
This will change your function like this:
inline std::unique_ptr<char[]> W2N(const wchar_t *wstr) {
int cw = lstrlenW(wstr);
if(cw == 0) {
auto psz = std::make_unique<char[]>(1);
psz[0] = '\0';
return psz;
}
int cc = WideCharToMultiByte(CP_ACP, 0, wstr, cw, NULL, 0, NULL, NULL);
if(cc == 0)
return nullptr;
auto psz = std::make_unique<char[]>(cc + 1);
cc = WideCharToMultiByte(CP_ACP, 0, wstr, cw, psz, cc, NULL, NULL);
if(cc == 0) {
return nullptr;
}
psz[cc] = '\0';
return psz;
}
This, of course, assumes that you have access to a C++14 compliant compiler (for std::make_unique), or at least C++11 compliant (for std::unique_ptr).
Of course, as stated in the comments, you could return a std::string. If you do so, beware that returning nullptr or NULL from the function will probably segfault.
Without trying to solve this for multithreading, a global char buffer big enough to hold your largest string and the \0 returned would make for a fixed "leak".
I'm relatively new to C++ memory management, and I'm getting this weird error of heap corruption (plus an automatic breakpoint in Visual Studio before it). Here is the offending code:
z_world::z_world(char* name)
{
unsigned int i, skip;
char tmp;
//Load data from file
std::string* data = loadString(name);
//Base case if there is no world data
tiles = NULL;
w = 0;
h = 0;
if(data->length() > 0) {
//Set up the 'tiles' array
for(i = 0; i < data->length(); i++) {
if(data->at(i) == '\n')
h++;
if(h == 0)
w++;
}
tiles = new int[data->length()-h];
//Load Data
skip = 0;
for(i = 0; i < data->length(); i++) {
if(data->at(i) == '\n') {
skip++;
printf("\n");
continue;
}
tmp = data->at(i);
tiles[i+skip] = atoi(&tmp);
printf("%i ",tiles[i+skip]);
}
}
delete data;
}
Here's where I load in the string:
std::string* loadString(char* name)
{
ifstream in(name);
std::string* input = new string();
while(in) {
std::string line;
getline(in,line);
input->append(line);
input->append("\n");
}
in.close();
return input;
}
I get the breakpoint and error inside of "delete data;", which makes me think that "data" gets deleted somewhere before that, but I can't find where it would. For reference, this method is to create an object that contains world data for a game in the form of a virtual 2D integer array (for the ID's of the tiles).
Youre problem is probably here:
tiles[i+skip] = atoi(&tmp);
Problem 1:
It should be -skip
tiles[i - skip] =
Problem 2:
The atoi() command is being used incorrectly (tmp does not contain a string). But also I don't think atoi() is the appropriate method. I think what you are looking for is simple assignment. The conversion from char to int is automatic:
tiles[i - skip] = tmp;
Problem 3:
You are not using objects correctly. In this situation there is no need to generate dynamic objects and create a mess with dynamic memory management. It would be simpler to just to create automatic objects and pass those back normally:
std::string* loadString(char* name)
// ^ Don't do this.
std::string loadString(std::string const& name)
// ^^^^^^^ return a string by value.
// The compiler will handle memory management very well.
In general you should not be passing pointers around. In the few situations where you do need pointers they should be held within a smart pointer object or containers (for multiple objects) so that their lifespan is correctly controlled.
atoi(&tmp);
atoi expects a pointer to a null terminated string - not a pointer to a char
There's no need to dynamically allocate the string in the code you've shown. Change the loadString function to
std::string loadString(char* name)
{
ifstream in(name);
std::string input;
// ...
return input;
}
In the caller
std::string data = loadString( name );
Now there's no need to delete the string after you're done.
Instead of
int *tiles = NULL;
tiles = new int[data->length()-h];
use
std::vector<int> tiles;
tiles.resize(data.length() - h);
Also, if you do need to dynamically allocate objects you should be using smart pointers (std::unique_ptr and std::shared_ptr) instead of raw pointers.
There is a bug in
tiles[i+skip] = atoi(&tmp);
For example, for a string
Hello\n
World\n
and for the loop iteration at the point of i == 10, skip is already 1 (since we have encountered the first \n before) and you are writing to tiles[10 + 1], but tiles only has been allocated as an array with 10 elements.
May be the variable input is local to this function. So after returning from this the memory is freed. So, calling later delete on this string tries to free already freed memory.
char* timeNew = _com_util::ConvertBSTRToString(cpi->getTime());
if(timeFirst == true)
{
strcpy(timeOld,timeNew);
timeFirst = false;
}
how can I initiliase timeold if I dont know what the size of character array returned by cpi->getTime is?
Allocate memory for it based on length of timeNew:
delete[] timeOld;
timeOld = new char[strlen(timeNew) + 1];
or you could make timeOld a std::string and let it manage memory for you:
std::string timeOld;
timeOld = timeNew; // If timeNew is dynamically allocated you must still
// delete[] it when no longer required, as timeOld
// takes a copy of timeNew, not ownership of timeNew.
You can access the const char* using std::string::c_str() if really required.
Use strings where possible:
char *t= _com_util::ConvertBSTRToString(cpi->getTime());
std::string timeNew(t);
delete[] t;
if(timeFirst == true)
{
timeOld=timeNew;
timeFirst = false;
}
if you don't have to manage the memory returned by teh function simply:
std::string timeNew(_com_util::ConvertBSTRToString(cpi->getTime()));
if(timeFirst == true)
{
timeOld=timeNew;
timeFirst = false;
}
If you have to use ConvertBSTRToString then use boost::scoped_array<char> or boost::shared_array<char> to ensure you get clean-up.
boost::shared_array<char> time;
time.reset( _com_util::ConvertBSTRtoString( cpi->getTime() );
automatically reallocates. No calls to delete or delete[] necessary.
I need an array of this struct allocated in one solid chunk of memory. The length of "char *extension" and "char *type" are not known at compile time.
struct MIMETYPE
{
char *extension;
char *type;
};
If I used the "new" operator to initialize each element by itself, the memory may be scattered. This is how I tried to allocate a single contiguous block of memory for it:
//numTypes = total elements of array
//maxExtension and maxType are the needed lengths for the (char*) in the struct
//std::string ext, type;
unsigned int size = (maxExtension+1 + maxType+1) * numTypes;
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size);
But, when I try to load the data in like this, the data is all out of order and scattered when I try to access it later.
for(unsigned int i = 0; i < numTypes; i++)
{
//get data from file
getline(fin, line);
stringstream parser.str(line);
parser >> ext >> type;
//point the pointers at a spot in the memory that I allocated
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension);
//copy the data into the elements
strcpy(mimeTypes[i].extension, ext.c_str());
strcpy(mimeTypes[i].type, type.c_str());
}
can anyone help me out?
EDIT:
unsigned int size = (maxExtension+1 + maxType+1);
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size * numTypes);
for(unsigned int i = 0; i < numTypes; i++)
strcpy((char*)(mimeTypes + (i*size)), ext.c_str());
strcpy((char*)(mimeTypes + (i*size) + (maxExtension+1)), type.c_str());
You mix 2 allocation:
1) manage array of MIMETYPE and
2) manage array of characters
May be (I don't really understand your objectives):
struct MIMETYPE
{
char extension[const_ofmaxExtension];
char type[maxType];
};
would be better to allocate linear items in form:
new MIMETYPE[numTypes];
I'll put aside the point that this is premature optimization (and that you ought to just use std::string, std::vector, etc), since others have already stated that.
The fundamental problem I'm seeing is that you're using the same memory for both the MIMETYPE structs and the strings that they'll point to. No matter how you allocate it, a pointer itself and the data it points to cannot occupy the exact same place in memory.
Lets say you needed an array of 3 types and had MIMETYPE* mimeTypes pointing to the memory you allocated for them.
That means you're treating that memory as if it contains:
8 bytes: mime type 0
8 bytes: mime type 1
8 bytes: mime type 2
Now, consider what you're doing in this next line of code:
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
extension is being set to point to the same location in memory as the MIMETYPE struct itself. That is not going to work. When subsequent code writes to the location that extension points to, it overwrites the MIMETYPE structs.
Similarly, this code:
strcpy((char*)(mimeTypes + (i*size)), ext.c_str());
is writing the string data in the same memory that you otherwise want to MIMETYPE structs to occupy.
If you really want store all the necessary memory in one contiguous space, then doing so is a bit more complicated. You would need to allocate a block of memory to contain the MIMETYPE array at the start of it, and then the string data afterwards.
As an example, lets say you need 3 types. Lets also say the max length for an extension string (maxExtension) is 3 and the max length for a type string (maxType) is 10. In this case, your block of memory needs to be laid out as:
8 bytes: mime type 0
8 bytes: mime type 1
8 bytes: mime type 2
4 bytes: extension string 0
11 bytes: type string 0
4 bytes: extension string 1
11 bytes: type string 1
4 bytes: extension string 2
11 bytes: type string 2
So to allocate, setup, and fill it all correctly you would want to do something like:
unsigned int mimeTypeStringsSize = (maxExtension+1 + maxType+1);
unsigned int totalSize = (sizeof(MIMETYPE) + mimeTypeStringsSize) * numTypes;
char* data = (char*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, totalSize);
MIMETYPE* mimeTypes = (MIMETYPE*)data;
char* stringData = data + (sizeof(MIMETYPE) * numTypes);
for(unsigned int i = 0; i < numTypes; i++)
{
//get data from file
getline(fin, line);
stringstream parser.str(line);
parser >> ext >> type;
// set pointers to proper locations
mimeTypes[i].extension = stringData + (mimeTypeStringsSize * i);
mimeTypes[i].type = stringData + (mimeTypeStringsSize * i) + maxExtension+1;
//copy the data into the elements
strcpy(mimeTypes[i].extension, ext.c_str());
strcpy(mimeTypes[i].type, type.c_str());
}
(Note: I've based my byte layout explanations on typical behavior of 32-bit code. 64-bit code would have more space used for the pointers, but the principle is the same. Furthermore, the actual code I've written here should work regardless of 32/64-bit differences.)
What you need to do is get a garbage collector and manage the heap. A simple collector using RAII for object destruction is not that difficult to write. That way, you can simply allocate off the collector and know that it's going to be contiguous. However, you should really, REALLY profile before determining that this is a serious problem for you. When that happens, you can typedef many std types like string and stringstream to use your custom allocator, meaning that you can go back to just std::string instead of the C-style string horrors you have there.
You really have to know the length of extension and type in order to allocate MIMETYPEs contiguously (if "contiguously" means that extension and type are actually allocated within the object). Since you say that the length of extension and type are not known at compile time, you cannot do this in an array or a vector (the overall length of a vector can be set and changed at runtime, but the size of the individual elements must be known at compile time, and you can't know that size without knowing the length of extension and type).
I would personally recommend using a vector of MIMETYPEs, and making the extension and type fields both strings. You're requirements sound suspiciously like premature optimization guided by a gut feeling that dereferencing pointers is slow, especially if the pointers cause cache misses. I wouldn't worry about that until you have actual data that reading these fields is an actual bottleneck.
However, I can think of a possible "solution": you can allocate the extension and type strings inside the MIMETYPE object when they are shorter than a particular threshold and allocate them dynamically otherwise:
#include <algorithm>
#include <cstring>
#include <new>
template<size_t Threshold> class Kinda_contig_string {
char contiguous_buffer[Threshold];
char* value;
public:
Kinda_contig_string() : value(NULL) { }
Kinda_contig_string(const char* s)
{
size_t length = std::strlen(s);
if (s < Threshold) {
value = contiguous_buffer;
}
else {
value = new char[length];
}
std::strcpy(value, s);
}
void set(const char* s)
{
size_t length = std::strlen(s);
if (length < Threshold && value == contiguous_buffer) {
// simple case, both old and new string fit in contiguous_buffer
// and value points to contiguous_buffer
std::strcpy(contiguous_buffer, s);
return;
}
if (length >= Threshold && value == contiguous_buffer) {
// old string fit in contiguous_buffer, new string does not
value = new char[length];
std::strcpy(value, s);
return;
}
if (length < Threshold && value != contiguous_buffer) {
// old string did not fit in contiguous_buffer, but new string does
std::strcpy(contiguous_buffer, s);
delete[] value;
value = contiguous_buffer;
return;
}
// old and new strings both too long to fit in extension_buffer
// provide strong exception guarantee
char* temp_buffer = new char[length];
std::strcpy(temp_buffer, s);
std::swap(temp_buffer, value);
delete[] temp_buffer;
return;
}
const char* get() const
{
return value;
}
}
class MIMETYPE {
Kinda_contig_string<16> extension;
Kinda_contig_string<64> type;
public:
const char* get_extension() const
{
return extension.get();
}
const char* get_type() const
{
return type.get();
}
void set_extension(const char* e)
{
extension.set(e);
}
// t must be NULL terminated
void set_type(const char* t)
{
type.set(t);
}
MIMETYPE() : extension(), type() { }
MIMETYPE(const char* e, const char* t) : extension(e), type(t) { }
};
I really can't endorse this without feeling guilty.
Add one byte in between strings... extension and type are not \0-terminated the way do it.
here you allocate allowing for an extra \0 - OK
unsigned int size = (maxExtension+1 + maxType+1) * numTypes;
mimeTypes = (MIMETYPE*)HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, size);
here you don't leave any room for extension's ending \0 (if string len == maxExtension)
//point the pointers at a spot in the memory that I allocated
mimeTypes[i].extension = (char*)(&mimeTypes[i]);
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension);
instead i think it should be
mimeTypes[i].type = (char*)((&mimeTypes[i]) + maxExtension + 1);