Related
If I have a struct instanceData:
struct InstanceData
{
unsigned usedInstances;
unsigned allocatedInstances;
void* buffer;
Entity* entity;
std::vector<float> *vertices;
};
And I allocate enough memory for an Entity and std::vector:
newData.buffer = size * (sizeof(Entity) + sizeof(std::vector<float>)); // Pseudo code
newData.entity = (Entity *)(newData.buffer);
newData.vertices = (std::vector<float> *)(newData.entity + size);
And then attempt to copy a vector of any size to it:
SetVertices(unsigned i, std::vector<float> vertices)
{
instanceData.vertices[i] = vertices;
}
I get an Access Violation Reading location error.
I've chopped up my code to make it concise, but it's based on Bitsquid's ECS. so just assume it works if I'm not dealing with vectors (it does). With this in mind, I'm assuming it's having issues because it doesn't know what size the vector is going to scale to. However, I thought the vectors might increase along another dimension, like this?:
Am I wrong? Either way, how can I allocate memory for a vector in a buffer like this?
And yes, I know vectors manage their own memory. That's besides the point. I'm trying to do something different.
It looks like you want InstanceData.buffer to have the actual memory space which is allocated/deallocated/accessed by other things. The entity and vertices pointers then point into this space. But by trying to use std::vector, you are mixing up two completely incompatible approaches.
1) You can do this with the language and the standard library, which means no raw pointers, no "new", no "sizeof".
struct Point {float x; float y;} // usually this is int, not float
struct InstanceData {
Entity entity;
std::vector<Point> vertices;
}
This is the way I would recommend. If you need to output to a specific binary format for serialization, just handle that in the save method.
2) You can manage the memory internal to the class, using oldschool C, which means using N*sizeof(float) for the vertices. Since this will be extremely error prone for a new programmer (and still rough for vets), you must make all of this private to class InstanceData, and do not allow any code outside InstanceData to manage them. Use unit tests. Provide public getter functions. I've done stuff like this for data structures that go across the network, or when reading/writing files with a specified format (Tiff, pgp, z39.50). But just to store in memory using difficult data structures -- no way.
Some other questions you asked:
How do I allocate memory for std::vector?
You don't. The vector allocates its own memory, and manages it. You can tell it to resize() or reserve() space, or push_back, but it will handle it. Look at http://en.cppreference.com/w/cpp/container/vector
How do I allocate memory for a vector [sic] in a buffer like this?
You seem to be thinking of an array. You're way off with your pseudo code so far, so you really need to work your way up through a tutorial. You have to allocate with "new". I could post some starter code for this, if you really need, which I would edit into the answer here.
Also, you said something about vector increasing along another dimension. Vectors are one dimensional. You can make a vector of vectors, but let's not get into that.
edit addendum:
The basic idea with a megabuffer is that you allocate all the required space in the buffer, then you initialize the values, then you use it through the getters.
The data layout is "Header, Entity1, Entity2, ..., EntityN"
// I did not check this code in a compiler, sorry, need to get to work soon
MegaBuffer::MegaBuffer() {AllocateBuffer(0);}
MegaBuffer::~MegaBuffer() {ReleaseBuffer();}
MegaBuffer::AllocateBuffer(size_t size /*, whatever is needed for the header*/){
if (nullptr!=buffer)
ReleaseBuffer();
size_t total_bytes = sizeof(Header) + count * sizeof(Entity)
buffer = new unsigned char [total_bytes];
header = buffer;
// need to set up the header
header->count = 0;
header->allocated = size;
// set up internal pointer
entity = buffer + sizeof(Header);
}
MegaBuffer::ReleaseBuffer(){
delete [] buffer;
}
Entity* MegaBuffer::operator[](int n) {return entity[n];}
The header is always a fixed size, and appears exactly once, and tells you how many entities you have. In your case there's no header because you are using member variables "usedInstances" and "allocatednstances" instead. So you do sort of have a header but it is not part of the allocated buffer. But you don't want to allocate 0 bytes, so just set usedInstances=0; allocatedInstances=0; buffer=nullptr;
I did not code for changing the size of the buffer, because the bitsquid ECS example covers that, but he doesn't show the first time initialization. Make sure you initialize n and allocated, and assign meaningful values for each entity before you use them.
You are not doing the bitsquid ECS the same as the link you posted. In that, he has several different objects of fixed size in parallel arrays. There is an entity, its mass, its position, etc. So entity[4] is an entity which has mass equal to "mass[4]" and its acceleration is "acceleration[4]". This uses pointer arithmetic to access array elements. (built in array, NOT std::Array, NOT std::vector)
The data layout is "Entity1, Entity2, ..., EntityN, mass1, mass2, ..., massN, position1, position2, ..., positionN, velocity1 ... " you get the idea.
If you read the article, you'll notice he says basically the same thing everyone else said about the standard library. You can use an std container to store each of these arrays, OR you can allocate one megabuffer and use pointers and "built in array" math to get to the exact memory location within that buffer for each item. In the classic faux-pas, he even says "This avoids any hidden overheads that might exist in the Array class and we only have a single allocation to keep track of." But you don't know if this is faster or slower than std::Array, and you're introducing a lot of bugs and extra development time dealing with raw pointers.
I think I see what you are trying to do.
There are numerous issues. First. You are making a buffer of random data, telling C++ that a Vector sized piece of it is a Vector. But, at no time do you actually call the constructor to Vector which will initialize the pointers and constructs inside to viable values.
This has already been answered here: Call a constructor on a already allocated memory
The second issue is the line
instanceData.vertices[i] = vertices;
instanceData.vertices is a pointer to a Vector, so you actually need to write
(*(instanceData.vertices))[i]
The third issue is that the contents of *(instanceData.vertices) are floats, and not Vector, so you should not be able to do the assignment there.
I recently found this won't work in my global CUDA C++ code that I plan to compile and later to be called in Matlab:
int M = 10; float V[M];
or if I were to import M value from the matlab host code.
But this works:
float V[10];
I was told there exists a function called new that I can use to avoid this problem, but I read online and am still quite confused how to use this new function, and it seems only to apply to host code, is that right? If so, it won't apply to my case then, since my host code is in matlab. Is this a way to get around this, so that I don't have to change vector lengths one by one? Thank you!
I don't know anything about MATLAB or CUDA, but your problem is in C++. Arrays declared like that must have sizes fixed at compile-time.
Solution 1: Fix the size
Declare your variable M const. These are equivalent:
int const M = 10;
const int M = 10;
The compiler would then know that it can assume these variables will always have the same value no matter how you run the program.
Solution 2: C-style dynamic allocation
Dynamic allocation with new and delete. Arrays allocated on the abstract section of memory called the "free-store" (rather than on the "stack", like those arrays you have) can determine their sizes on the fly. You use it like this:
float * V = new V[M]; //V is a pointer to some freestore memory
//You use it and pass it like you would a normal array:
V[2] = 5.5;
int x = some_func(V);
//But after you're done, you should manually free the memory
delete [] V; //don't forget the [], since you used [] in the allocation
I don't recommend this, because of the possiblity of forgetting to delete the memory.
Solution 3: Automatic memory management with C++'s vector
In C++, the work of memory management can be hidden behind structures called classes.
#include<vector>
using std::vector;
vector<float> V(M); //V is a vector of floats, with initial size M
//You use it like a normal array
V[2] = 5.5;
//But to pass it as an array, you need to pass a pointer to the first element
int x = some_func(&V[0]); //read as &(V[0]): pass in the address of V[0]
Solution 3b: CUDA-compatible vector
Thrust is a C++ template library for CUDA based on the Standard Template Library (STL). Thrust allows you to implement high performance parallel applications with minimal programming effort through a high-level interface that is fully interoperable with CUDA C.
http://docs.nvidia.com/cuda/thrust/#vectors
Conclusion
If you're using fixed sizes, I recommend solution 1. If you're using sizes determined during runtime, I recommend vector.
(By the way, when you pass an ordinary array to a function, you are actually passing a pointer to the first element, NOT the array. The name of the array is automatically converted to a pointer type.)
I'm reading in values from a file which I will store in memory as I read them in. I've read on here that the correct way to handle memory location in C++ is to always use new/delete, but if I do:
DataType* foo = new DataType[sizeof(DataType) * numDataTypes];
Then that's going to call the default constructor for each instance created, and I don't want that. I was going to do this:
DataType* foo;
char* tempBuffer=new char[sizeof(DataType) * numDataTypes];
foo=(DataType*) tempBuffer;
But I figured that would be something poo-poo'd for some kind of type-unsafeness. So what should I do?
And in researching for this question now I've seen that some people are saying arrays are bad and vectors are good. I was trying to use arrays more because I thought I was being a bad boy by filling my programs with (what I thought were) slower vectors. What should I be using???
Use vectors!!! Since you know the number of elements, make sure that you reserve the memory first (by calling myVector.reserve(numObjects) before you then insert the elements.).
By doing this, you will not call the default constructors of your class.
So use
std::vector<DataType> myVector; // does not reserve anything
...
myVector.reserve(numObjects); // tells vector to reserve memory
You can use ::operator new to allocate an arbitrarily sized hunk of memory.
DataType* foo = static_cast<DataType*>(::operator new(sizeof(DataType) * numDataTypes));
The main advantage of using ::operator new over malloc here is that it throws on failure and will integrate with any new_handlers etc. You'll need to clean up the memory with ::operator delete
::operator delete(foo);
Regular new Something will of course invoke the constructor, that's the point of new after all.
It is one thing to avoid extra constructions (e.g. default constructor) or to defer them for performance reasons, it is another to skip any constructor altogether. I get the impression you have code like
DataType dt;
read(fd, &dt, sizeof(dt));
If you're doing that, you're already throwing type safety out the window anyway.
Why are you trying to accomplish by not invoking the constructor?
You can allocate memory with new char[], call the constructor you want for each element in the array, and then everything will be type-safe. Read What are uses of the C++ construct "placement new"?
That's how std::vector works underneath, since it allocates a little extra memory for efficiency, but doesn't construct any objects in the extra memory until they're actually needed.
You should be using a vector. It will allow you to construct its contents one-by-one (via push_back or the like), which sounds like what you're wanting to do.
I think you shouldn't care about efficiency using vector if you will not insert new elements anywhere but at the end of the vector (since elements of vector are stored in a contiguous memory block).
vector<DataType> dataTypeVec(numDataTypes);
And as you've been told, your first line there contains a bug (no need to multiply by sizeof).
Building on what others have said, if you ran this program while piping in a text file of integers that would fill the data field of the below class, like:
./allocate < ints.txt
Then you can do:
#include <vector>
#include <iostream>
using namespace std;
class MyDataType {
public:
int dataField;
};
int main() {
const int TO_RESERVE = 10;
vector<MyDataType> everything;
everything.reserve( TO_RESERVE );
MyDataType temp;
while( cin >> temp.dataField ) {
everything.push_back( temp );
}
for( unsigned i = 0; i < everything.size(); i++ ) {
cout << everything[i].dataField;
if( i < everything.size() - 1 ) {
cout << ", ";
}
}
}
Which, for me with a list of 4 integers, gives:
5, 6, 2, 6
I am doing a project converting some Pascal (Delphi) code to C++ and would like to write a function that is roughly equivalent to the Pascal "SetLength" method. This takes a reference to a dynamic array, as well as a length and allocates the memory and returns the reference.
In C++ I was thinking of something along the lines of
void* setlength(void* pp, int array_size, int pointer_size, int target_size, ....) {
void * p;
// Code to allocate memory here via malloc/new
// something like: p = reinterpret_cast<typeid(pp)>(p);
// p=(target_size) malloc(array_size);
return p;
}
My question is this: is there a way to pass the pointer type to a function like this and to successfully allocate the memory (perhaps via a typeid parameter?)? Can I use
<reinterpret_cast>
somehow? The ultimate aim would be something like the following in terms of usage:
float*** p;
p=setlength(100,sizeof(float***),sizeof(float**),.....);
class B;
B** cp;
cp=setlength(100,sizeof(B**),sizeof(B*),.....);
Any help would be most welcome. I am aware my suggested code is all wrong, but wanted to convey the general idea. Thanks.
Use std::vector instead of raw arrays.
Then you can simply call its resize() member method.
And make the function a template to handle arbitrary types:
If you want to use your function, it could look something like this:
template <typename T>
std::vector<T>& setlength(std::vector<T>& v, int new_size) {
v.resize(new_size);
return v;
}
But now it's so simple you might want to eliminate the function entirely and just call resize to begin with.
I'm not entirely sure what you're trying to do with the triple-pointers in your example, but it looks like you don't want to resize though, you want to initialize to a certain size, which can be done with the vector constructor:
std::vector<float>v(100);
If you wanted to do it literally, you would do it like this:
template <typename T>
T* SetLength(T* arr, size_t len) {
return static_cast<T*>(realloc(arr, sizeof(T) * len));
}
Note that the array must have been allocated with malloc or calloc. Also note that this does not actually resize the memory—it deallocates the memory and reallocates memory of the appropriate size. If there were any other pointers to the array being passed in, they will be invalid afterwards.
You're really better off using a more idiomatic C++ solution, like std::vector.
For a multidimensional array, probably the best option would be to use boost's multi_array library:
typedef boost::multi_array<float, 3> array_type;
array_type p(boost::extents[100][100][100]); // make an 100x100x100 array of floats
p[1][2][3] = 4.2;
This lets you completely abstract away the allocation and details of setting up the multidimensional array. Plus, because it uses linear storage, you get the efficiency benefits of linear storage with the ease of access of indirections.
Failing that, you have three other major options.
The most C++-y option without using external libraries would be to use a STL container:
std::vector<float **> p;
p.resize(100);
As with multi_array, p will then automatically be freed when it goes out of scope. You can get the vector bounds with p.size(). However the vector will only handle one dimension for you, so you'll end up doing nested vectors (ick!).
You can also use new directly:
float ***p = new float**[100];
To deallocate:
delete [] p;
This has all the disadvantages of std::vector, plus it won't free it for you, and you can't get the size later.
The above three methods will all throw an exception of type std::bad_alloc if they fail to allocate enough memory.
Finally, for completeness, there's the C route, with calloc():
float ***p = (float ***)calloc(100, sizeof(*p));
To free:
free((void*)p);
This comes from C and is a bit uglier with all the casts. For C++ classes it will not call the constructors for you, either. Also, there's no checking that the sizeof in the argument is consistent with the cast.
If calloc() fails to allocate memory it will return NULL; you'll need to check for this and handle it.
To do this the C++ way:
1) As jalf stated, prefer std::vector if you can
2) Don't do void* p. Prefer instead to make your function a template of type T.
The new operator itself is essentially what you are asking for, with the exception that to appropriately allocate for double/triple pointers you must do something along the following lines:
float** data = new float*[size_of_dimension_1];
for ( size_t i=0 ; i<size_of_dimension_1 ; ++i )
data[i] = new float[size_of_dimension_2];
...
// to delete:
for ( size_t i=0 ; i<size_of_dimension_1 ; ++i )
delete [] data[i];
delete [] data;
Edit: I would suggest using one of the many C++ math/matrix libraries out there. I would suggest uBlas.
I am observing strange behaviour of std::map::clear(). This method is supposed to call element's destructor when called, however memory is still accessible after call to clear().
For example:
struct A
{
~A() { x = 0; }
int x;
};
int main( void )
{
std::map< int, A * > my_map;
A *a = new A();
a->x = 5;
my_map.insert( std::make_pair< int, *A >( 0, a ) );
// addresses will be the same, will print 5
std::cout << a << " " << my_map[0] << " " << my_map[0]->x << std::endl;
my_map.clear();
// will be 0
std::cout << a->x << std::endl;
return 0;
}
The question is, why is variable a still accessible after its destructor was called by map::clear()? Do I need to write delete a; after calling my_map.clear() or is it safe to overwrite the contents of a?
Thanks in advance for your help,
sneg
If you store pointers on a map (or a list, or anything like that) YOU are the responsible for deleting the pointers, since the map doesn't know if they have been created with new, or not. The clear function only invokes destructors if you don't use pointers.
Oh, and one more thing: invoking a destructor (or even calling delete) doesn't mean the memory can't be accessed anymore. It only means that you will be accessing garbage if you do.
std::map does not manage the memory pointed to by the pointer values - it's up to you to do it yourself. If you don't want to use smart pointers, you can write a general purpose free & clear function like this:
template <typename M> void FreeClear( M & amap )
for ( typename M::iterator it = amap.begin(); it != amap.end(); ++it ) {
delete it->second;
}
amap.clear();
}
And use it:
std::map< int, A * > my_map;
// populate
FreeClear( my_map )
;
That's because map.clear() calls destructors of the data contained in the map, in your case, of the pointer to a. And this does nothing.
You might want to put some kind of smart pointer in the map for the memory occupied by a to be automatically reclaimed.
BTW, why do you put the template arguments in the call to make_pair? The template argument deduction should do pretty well here.
When you free a piece of heap memory, its contents don't get zeroed. They are merely available for allocation again. Of course you should consider the memory non accessible, because the effects of accessing unallocated memory are undefined.
Actually preventing access to a memory page happens on a lower level, and std libraries don't do that.
When you allocate memory with new, you need to delete it yourself, unless you use a smart pointer.
Any container stores your object Type and call corresponding constructors: internal code each node might look similar to:
__NodePtr
{
*next;
__Ty Val;
}
When you allocate it happens by constructing the val based on type and then linking. Something similar to:
_Ty _Val = _Ty();
_Myhead = _Buynode();
_Construct_n(_Count, _Val);
When you delete it calls corresponding destructors.
When you store references (pointers) it won't call any constructor nor it will destruct.
Having spent the last 2 months eating, sleeping, and breathing maps, I have a recommendation. Let the map allocate it's own data whenever possible. It's a lot cleaner, for exactly the kind of reasons you're highlighting here.
There are also some subtle advantages, like if you're copying data from a file or socket to the map's data, the data storage exists as soon as the node exists because when the map calls malloc() to allocate the node, it allocates memory for both the key and the data. (AKA map[key].first and map[key].second)
This allows you to use the assignment operator instead of memcpy(), and requires 1 less call to malloc() - the one you make.
IC_CDR CDR, *pThisCDRLeafData; // a large struct{}
while(1 == fread(CDR, sizeof(CDR), 1, fp)) {
if(feof(fp)) {
printf("\nfread() failure in %s at line %i", __FILE__, __LINE__);
}
cdrMap[CDR.iGUID] = CDR; // no need for a malloc() and memcpy() here
pThisCDRLeafData = &cdrMap[CDR.iGUID]; // pointer to tree node's data
A few caveats to be aware of are worth pointing out here.
do NOT call malloc() or new in the line of code that adds the tree node as your call to malloc() will return a pointer BEFORE the map's call to malloc() has allocated a place to hold the return from your malloc().
in Debug mode, expect to have similar problems when trying to free() your memory. Both of these seem like compiler problems to me, but at least in MSVC 2012, they exist and are a serious problem.
give some thought as to where to "anchor" your maps. IE: where they are declared. You don't want them going out of scope by mistake. main{} is always safe.
INT _tmain(INT argc, char* argv[]) {
IC_CDR CDR, *pThisCDRLeafData=NULL;
CDR_MAP cdrMap;
CUST_MAP custMap;
KCI_MAP kciMap;
I've had very good luck, and am very happy having a critical map allocate a structure as it's node data, and having that struct "anchor" a map. While anonymous structs have been abandoned by C++ (a horrible, horrible decision that MUST be reversed), maps that are the 1st struct member work just like anonymous structs. Very slick and clean with zero size-effects. Passing a pointer to the leaf-owned struct, or a copy of the struct by value in a function call, both work very nicely. Highly recommended.
you can trap the return values for .insert to determine if it found an existing node on that key, or created a new one. (see #12 for code) Using the subscript notation doesn't allow this. It might be better to settle on .insert and stick with it, especially because the [] notation doesn't work with multimaps. (it would make no sense to do so, as there isn't "a" key, but a series of keys with the same values in a multimap)
you can, and should, also trap returns for .erase and .empty() (YES, it's annoying that some of these things are functions, and need the () and some, like .erase, don't)
you can get both the key value and the data value for any map node using .first and .second, which all maps, by convention, use to return the key and data respectively
save yourself a HUGE amount of confusion and typing, and use typedefs for your maps, like so.
typedef map<ULLNG, IC_CDR> CDR_MAP;
typedef map<ULLNG, pIC_CDR> CALL_MAP;
typedef struct {
CALL_MAP callMap;
ULNG Knt;
DBL BurnRateSec;
DBL DeciCents;
ULLNG tThen;
DBL OldKCIKey;
} CUST_SUM, *pCUST_SUM;
typedef map<ULNG,CUST_SUM> CUST_MAP, CUST_MAP;
typedef map<DBL,pCUST_SUM> KCI_MAP;
pass references to maps using the typedef and & operator as in
ULNG DestroyCustomer_callMap(CUST_SUM Summary, CDR_MAP& cdrMap, KCI_MAP& kciMap)
use the "auto" variable type for iterators. The compiler will figure out from the type specified in the rest of the for() loop body what kind of map typedef to use. It's so clean it's almost magic!
for(auto itr = Summary.callMap.begin(); itr!= Summary.callMap.end(); ++itr) {
define some manifest constants to make the return from .erase and .empty() more meaningfull.
if(ERASE_SUCCESSFUL == cdrMap.erase (itr->second->iGUID)) {
given that "smart pointers" are really just keeping a reference count, remember you can always keep your own reference count, an probably in a cleaner, and more obvious way. Combining this with #5 and #10 above, you can write some nice clean code like this.
#define Pear(x,y) std::make_pair(x,y) // some macro magic
auto res = pSumStruct->callMap.insert(Pear(pCDR->iGUID,pCDR));
if ( ! res.second ) {
pCDR->RefKnt=2;
} else {
pCDR->RefKnt=1;
pSumStruct->Knt += 1;
}
using a pointer to hang onto a map node which allocates everything for itself, IE: no user pointers pointing to user malloc()ed objects, works well, is potentially more efficient, and and be used to mutate a node's data without side-effects in my experience.
on the same theme, such a pointer can be used very effectively to preserve the state of a node, as in pThisCDRLeafData above. Passing this to a function that mutates/changes that particular node's data is cleaner than passing a reference to the map and the key needed to get back to the node pThisCDRLeafData is pointing to.
iterators are not magic. They are expensive and slow, as you are navigating the map to get values. For a map holding a million values, you can read a node based on a key at about 20 million per second. With iterators it's probably ~ 1000 times as slow.
I think that about covers it for now. Will update if any of this changes or there's additional insights to share. I am especially enjoying using the STL with C code. IE: not a class in sight anywhere. They just don't make sense in the context I'm working in, and it's not an issue. Good luck.