I want to use the std::shared_ptr and std::list for my problem(memory bound) at disposal, notice that I need to interface with some legacy C code that requires the raw pointers and my main objective is to have kind of dynamic memory management(can reclaim memory whenever possible), my psudo-code is following,
// My "central" storage
typedef std::shared_ptr<double> data_type;
std::list<data_type> dynamic_storage;
// At each allocation point
dynamic_storage.push_back(data_type());
dynamic_storage.back.reset(new double[N],std::default_delete<double[]>());
// immediately after each allocation, store the shared_ptr to some map
std::map<data_type> map_for_job1; // may have several maps
int index; // unique index for later access, related to domain-specific problem
data_type ptr_in_map(dynamic_storage.back()); // reference counter +1
map_for_job1[index]=ptr_in_map; // store in map
// when I want to access again with a certain map and index
double *raw_data = map_for_job1.find(index).get();
// when I'm done with all the resources shared by buffer_map_1
map_for_job1.clear(); // reference counter -1
for (std::list<data_type>::iterator it=dynamic_storage.begin(); it != dynamic_storage.end(); ++it)
{
if (*it.use_count()==1)
dynamic_storage.erase(i) // release resource from central storage
}
So, my questions are,
Is this a valid memory-management pattern?
How to improve, perhaps traversing a list is too expensive?
Can't you simply do:
std::list< std::vector<data_type> > dynamic_storage;
And then go:
dynamic_storage.move_back( std::move( std::vector<data_type>(N) );
When you need a pointer, use iterator or as you did:
data_type* p = &dynamic_storage.back()[0];
Then the whole thing should clear up when falling out of scope so you don't need any particular cleanup code...
Related
I'm trying to populate a vector of doubles in C++ and pass the associated array to Fortran. But I'm having trouble freeing the rest of the memory associated with the vector. I'd like to avoid copying. Here's what I have:
std::vector<double> *vec = new std::vector<double>();
(*vec).push_back(1.0);
(*vec).push_back(2.0);
*arr = (*vec).data(); //arr goes to Fortran
How do I delete vec while keeping arr intact? Is there a way to nullify the pointer to arr in vec so that I can then delete vec?
Update
I see that I didn't give enough information here. A couple things:
I'm actually calling a C++ function in Fortran using iso_c_binding
I don't know how large the vec needs to be. The vector class looks good for this situation
I might try Guillaume's suggestion eventually, but for now, I'm passing vec to the Fortran and calling another C++ function to delete it once I'm done with the data
You need to rethink your program design.
Somehow, somewhere, you need to keep an array alive while Fortran is using it. So whatever context you're using to access Fortran should probably be responsible for ownership of this array.
class fortran_context {
/*Blah blah blah whatever API you're using to access Fortran*/
void * arr;
std::vector<double> vec; //Don't allocate this as a pointer; makes no sense!
public:
fortran_context() {
arr = //Do whatever is necessary to setup Fortran stuff. I'm assuming your
//api has some kind of "get_array_pointer" function that you'll use.
}
~fortran_context() {
//Do the cleanup for the fortran stuff
}
//If you want to spend time figuring out a robust copy constructor, you may.
//Personally, I suspect it's better to just delete it, and make this object non-copyable.
fortran_context(fortran_context const&) = delete;
std::vector<double> & get_vector() {
return vec;
}
std::vector<double> const& get_vector() const {
return vec;
}
void assign_vector_to_array() {
*arr = vec.data();
}
void do_stuff_with_fortran() {
assign_vector_to_array();
//???
}
};
int main() {
fortran_context context;
auto & vec = context.get_vector();
vec.push_back(1.0);
vec.push_back(2.0);
context.do_stuff_with_fortran();
return 0;
} //Cleanup happens automatically due to correct implementation of ~fortran_context()
I've abstracted a lot of this because I don't know what API you're using to access Fortran, and I don't know what kind of work you're doing with this array. But this is, by far, the safest way to ensure that
The vector's allocated memory exists so long as you are doing stuff in Fortran
The memory associated with the vector will be cleaned up properly when you're done.
How do I delete vec while keeping arr intact? Is there a way to nullify the pointer to arr in vec so that I can then delete vec?
The library does not provide any built-in capability to do that. You have to do the bookkeeping work yourself.
Allocate memory for the data and copy data from the vector.
Send the data to FORTRAN.
Decide when it is safe to deallocate the data and then delete them.
// Fill up data in vec
std::vector<double> vec;
vec.push_back(1.0);
vec.push_back(2.0);
// Allocate memory for arr and copy the data from vec
double* arr = new double[vec.size()];
std::copy(vec.begin(), vec.end(), arr);
// use arr
// delete arr
delete [] arr;
What you are asking for is not possible. std::vector, being a well behaved template class will release the internals that it owns and manages when it is destroyed.
You will have to keep vector alive while you are using its contents, which makes perfect sense. Otherwise, you will have to make a copy.
Also, I don't see why you are allocating the vector on the heap, it doesn't seem needed at all.
How do I delete vec while keeping arr intact? Is there a way to nullify the pointer to arr in vec so that I can then delete vec?
You don't.
I think you misuse or misunderstood what vector is for. It is not meant to expose memory management of the underlying array, but to represent a dynamically sized array as a regular type.
If you need to explicitly manage memory, I'd suggest you to use std::unique_ptr<T[]>. Since unique pointers offers a way to manage memory and to release it's resource without deleting it, I think it's a good candidate to meet your needs.
auto arr = std::make_unique<double[]>(2);
arr[0] = 1.;
arr[1] = 2.;
auto data = arr.release();
// You have to manage `data` memory manually,
// since the unique pointer released it's resource.
// arr is null here
// data is a pointer to an array, must be deleted manually later.
delete[] data;
I would like to create a vector of pointers to struct
vector<myStruct*> vec
For elements in the vector, not all of them contain data. Some of them may point to NULL.
So, should I create space by new in each of the element first
for(int i = 0; vec.size() ;i++){
if (thisSpaceIsValid(i))
vec.at(i) = new myStruct;
else
vect.at(i) = NULL;
}
The problem comes:
-If I use new for each element, it would be very slow. How can I speed it up a bit? Is there a way the create all the spaces that I need , that automatically access the pointer of such space to the vector(vec here)?
-If later I use delete to free the memory, would the problem of speed still bother me?
If I use "new" for each element, it would be very slow. How can I speed it up a bit? Is there a way the create all the spaces that I need , that automatically access the pointer of such space to the vector("vec" here)?
You can do that.
Let's say the size of your vector is M and you only need N of those elements to have pointers to objects and other elements are null pointers. You can use:
myStruct* objects = new myStruct[N];
and then, use:
for(int i = 0, j = 0; vec.size(); i++)
{
if (thisSpaceIsValid(i))
{
if ( j == N )
{
// Error. Do something.
}
else
{
vec[i] = objects+j;
++j;
}
}
else
{
vect[i] = NULL;
}
}
You have to now make sure that you are able to keep track of the value of objeccts so you can safely deallocate the memory by using
delete [] objects;
PS
There might be a better and more elegant solution to your problem. It will be worth your while to spend a bit more time thinking over that.
EDIT:
After reading the question again, it seems I misunderstood the question. So here is an edited answer.
If you only need to execute the code during some kind of initialization phase, you can create all the instances of myStruct in an array and then just point to those from the vector as already proposed by R Sahu. Note that the solution requires you to create and delete all instances at the same time.
However, if you execute this code several times and/or don't know exactly how many myStruct instances you will need, you could overwrite new and delete for the struct and handle memory allocation yourself.
See Callling object constructor/destructor with a custom allocator for an example of this. See the answer by Jerry Coffin.
BTW - you don't need vec.at(i) as you are iterating from 0 to size. vec[i] is okay and should perform a better.
OLD ANSWER:
You can do
vector<myStruct*> vec(10000, nullptr);
to generate a vector with for instance 10000 elements all initialized to nullptr
After that you can fill the relevant elements with pointer to the struct.
For delete just
for (auto e : vec) delete e;
cause it is safe to do deleteon a nullptr
If you need a vector of pointers, and would like to avoid calling new, then firstly create a container of structs themselves, then assign pointers to the elements into your vec. Be careful with choosing the container of structs. If you use vector of structs, make sure to reserve all elements in advance, otherwise its elements may move to a different memory location when vector grows. Deque on the other hand guarantees its elements don't move.
Multiple small new and delete calls should be avoided if possible in c++ when performance matters a lot.
The more I think about it, the less I like #RSahu's solution. In particular, I feel memory management in this scenario would be a nightmare. Instead I suggest using a vector of unique_ptr's owning memory allocated via custom alloctor. I believe, sequential allocator would do.
Hi i am doing my homeworks and i have a problem.
I need diferents maps and set in the application and i want reuse some variables.
I have this global variables
map<char,set<char> > IAF; //I Am Father
map<char,int> NBM; //Number Before Me
set<char> WCR; //Who can run
and every time in the main i need reset this variables.
I have done two things:
IAF = new map<char,set<char> >;
and
IAF = map<char,set<char> >;
But any has run.
Can someone help me?
use
IAF.clear()
NBM.clear()
WCR.clear()
(Edit: references to the spec and caveats)
map::clear()
set::clear()
Note that if you're storing pointers, clear() will remove the pointers, but it will not delete the memory pointed to by the pointers.
map<char,set<char> > IAF;
This is a definition of variable. This is not a pointer. If you want to do some kind of initialization you can use one of supported methods, e.g:
std::copy( differentContainer.begin(), differentContainer.end(), IAF.begin());
or
while( ...) {
IAF.insert( ...);
// or
IAF[ key] = value;
}
To delete the content of map you can do (this will not automatically delete memory pointed to by pointer in map - if you store pointers, use a smart pointers then):
IAF.clear();
In addition to the previous answers which are fairly clear. The new keyword, that you used at some point, is to allocate memory for a pointer.
map<char, set<char> > *IAF = new map<char, set<char> >;
//...
// free the memory
delete IAF;
Check the Dynamic memory allocation for more information and to understand when and how to use pointers.
Also, using
IAF = map<char,set<char> >;
Is incorrect. The map<char, set<char> > is a class name (combined with templates generic programming, see the answer to What is the meaning of "generic programming" in c++? ) and hence you cannot assign it to a variable in this way. What you want to do is to call the constructor that will return an instance of that class:
IAF = map >();
However, doing it this way is not efficient at all. It creates a temporary object, destroys IAF, copies the temporary, and then destroys the temporary (unless you are using C++11 and in this case you use move, but still...). So, it's better to use the clear() as stated by the other answers.
I'm new to C++ programming and would greatly appreciate replies that don't assume much prior knowledge.
Thanks to suggestions here, I've created an unordered map:
typedef std::tr1::unordered_map<std::string, Strain*> hmap;
The data in this map are pointers to instances of class Strain. As soon as these instances are created, I create pointers to them, and I then add these pointers to my hash table (hmap strainTable) and to another vector (vector< Strain *> liveStrains), e.g.,
string MRCA;
for ( int b = 0; b < SEQ_LENGTH; b++ ) {
int randBase = rgen.uniform(0,NUM_BASES);
MRCA.push_back( BASES[ randBase ] );
}
Strain * firstStrainPtr;
firstStrainPtr = new Strain( idCtr, MRCA, NUM_STEPS );
liveStrains.push_back( firstStrainPtr );
strainTable[ MRCA ]= firstStrainPtr;
Instances of class Strain are never deleted, nor are pointers to them removed from strainTable. Pointers do occasionally move between vector< Strain * > liveStrains and vector< Strain * > deadStrains, but once on strainTable, they stay on strainTable.
Is this kosher? As long as the underlying instances are never destroyed, will the pointers added to them remain intact?
Is it also correct that I should always be able to get member attributes from the pointers in strainTable by using, e.g., for the first entry,
hmap::const_iterator itr1 = strainTable.begin();
int id = (itr1->second)->getStrainID();
I'm finding that after a while, pointers in my strainTable point to garbage.
A pointer to any object allocated with new will remain valid until you call delete on the object. You can copy the pointer as much as you like, and it will be valid as long as the underlying object has not been deleted.
Secondly, yes, you are correct that you can access object attributes from the stored pointers via container iterators. But always check to make sure that the return value of hmap::find() is not equal to hmap::end().
So what you describe is fine. Now, as to why the pointers in your strainTable are ending up pointing to garbage, I couldn't say without more details. Are you sure you're not deleting any objects anywhere? Are you sure that when you copy pointers from one vector to the other, you are doing it correctly?
if you never delete them, then the pointers are still ok. On the other hand, you might want to keep things a bit tidier and use standard containers for the whole shebang.
typedef std::tr1::unordered_map<std::string, Strain> hmap;
typedef std::tr1::unordered_map<std::string, hmap::iterator> weakhmap;
hmap strainTable;
weakhmap liveStrains;
Strain firstStrain( idCtr, MRCA, NUM_STEPS );
strainTable[MRCA] = firstStrain;
liveStrains[MRCA] = strainTable.find(MRCA);
I am observing strange behaviour of std::map::clear(). This method is supposed to call element's destructor when called, however memory is still accessible after call to clear().
For example:
struct A
{
~A() { x = 0; }
int x;
};
int main( void )
{
std::map< int, A * > my_map;
A *a = new A();
a->x = 5;
my_map.insert( std::make_pair< int, *A >( 0, a ) );
// addresses will be the same, will print 5
std::cout << a << " " << my_map[0] << " " << my_map[0]->x << std::endl;
my_map.clear();
// will be 0
std::cout << a->x << std::endl;
return 0;
}
The question is, why is variable a still accessible after its destructor was called by map::clear()? Do I need to write delete a; after calling my_map.clear() or is it safe to overwrite the contents of a?
Thanks in advance for your help,
sneg
If you store pointers on a map (or a list, or anything like that) YOU are the responsible for deleting the pointers, since the map doesn't know if they have been created with new, or not. The clear function only invokes destructors if you don't use pointers.
Oh, and one more thing: invoking a destructor (or even calling delete) doesn't mean the memory can't be accessed anymore. It only means that you will be accessing garbage if you do.
std::map does not manage the memory pointed to by the pointer values - it's up to you to do it yourself. If you don't want to use smart pointers, you can write a general purpose free & clear function like this:
template <typename M> void FreeClear( M & amap )
for ( typename M::iterator it = amap.begin(); it != amap.end(); ++it ) {
delete it->second;
}
amap.clear();
}
And use it:
std::map< int, A * > my_map;
// populate
FreeClear( my_map )
;
That's because map.clear() calls destructors of the data contained in the map, in your case, of the pointer to a. And this does nothing.
You might want to put some kind of smart pointer in the map for the memory occupied by a to be automatically reclaimed.
BTW, why do you put the template arguments in the call to make_pair? The template argument deduction should do pretty well here.
When you free a piece of heap memory, its contents don't get zeroed. They are merely available for allocation again. Of course you should consider the memory non accessible, because the effects of accessing unallocated memory are undefined.
Actually preventing access to a memory page happens on a lower level, and std libraries don't do that.
When you allocate memory with new, you need to delete it yourself, unless you use a smart pointer.
Any container stores your object Type and call corresponding constructors: internal code each node might look similar to:
__NodePtr
{
*next;
__Ty Val;
}
When you allocate it happens by constructing the val based on type and then linking. Something similar to:
_Ty _Val = _Ty();
_Myhead = _Buynode();
_Construct_n(_Count, _Val);
When you delete it calls corresponding destructors.
When you store references (pointers) it won't call any constructor nor it will destruct.
Having spent the last 2 months eating, sleeping, and breathing maps, I have a recommendation. Let the map allocate it's own data whenever possible. It's a lot cleaner, for exactly the kind of reasons you're highlighting here.
There are also some subtle advantages, like if you're copying data from a file or socket to the map's data, the data storage exists as soon as the node exists because when the map calls malloc() to allocate the node, it allocates memory for both the key and the data. (AKA map[key].first and map[key].second)
This allows you to use the assignment operator instead of memcpy(), and requires 1 less call to malloc() - the one you make.
IC_CDR CDR, *pThisCDRLeafData; // a large struct{}
while(1 == fread(CDR, sizeof(CDR), 1, fp)) {
if(feof(fp)) {
printf("\nfread() failure in %s at line %i", __FILE__, __LINE__);
}
cdrMap[CDR.iGUID] = CDR; // no need for a malloc() and memcpy() here
pThisCDRLeafData = &cdrMap[CDR.iGUID]; // pointer to tree node's data
A few caveats to be aware of are worth pointing out here.
do NOT call malloc() or new in the line of code that adds the tree node as your call to malloc() will return a pointer BEFORE the map's call to malloc() has allocated a place to hold the return from your malloc().
in Debug mode, expect to have similar problems when trying to free() your memory. Both of these seem like compiler problems to me, but at least in MSVC 2012, they exist and are a serious problem.
give some thought as to where to "anchor" your maps. IE: where they are declared. You don't want them going out of scope by mistake. main{} is always safe.
INT _tmain(INT argc, char* argv[]) {
IC_CDR CDR, *pThisCDRLeafData=NULL;
CDR_MAP cdrMap;
CUST_MAP custMap;
KCI_MAP kciMap;
I've had very good luck, and am very happy having a critical map allocate a structure as it's node data, and having that struct "anchor" a map. While anonymous structs have been abandoned by C++ (a horrible, horrible decision that MUST be reversed), maps that are the 1st struct member work just like anonymous structs. Very slick and clean with zero size-effects. Passing a pointer to the leaf-owned struct, or a copy of the struct by value in a function call, both work very nicely. Highly recommended.
you can trap the return values for .insert to determine if it found an existing node on that key, or created a new one. (see #12 for code) Using the subscript notation doesn't allow this. It might be better to settle on .insert and stick with it, especially because the [] notation doesn't work with multimaps. (it would make no sense to do so, as there isn't "a" key, but a series of keys with the same values in a multimap)
you can, and should, also trap returns for .erase and .empty() (YES, it's annoying that some of these things are functions, and need the () and some, like .erase, don't)
you can get both the key value and the data value for any map node using .first and .second, which all maps, by convention, use to return the key and data respectively
save yourself a HUGE amount of confusion and typing, and use typedefs for your maps, like so.
typedef map<ULLNG, IC_CDR> CDR_MAP;
typedef map<ULLNG, pIC_CDR> CALL_MAP;
typedef struct {
CALL_MAP callMap;
ULNG Knt;
DBL BurnRateSec;
DBL DeciCents;
ULLNG tThen;
DBL OldKCIKey;
} CUST_SUM, *pCUST_SUM;
typedef map<ULNG,CUST_SUM> CUST_MAP, CUST_MAP;
typedef map<DBL,pCUST_SUM> KCI_MAP;
pass references to maps using the typedef and & operator as in
ULNG DestroyCustomer_callMap(CUST_SUM Summary, CDR_MAP& cdrMap, KCI_MAP& kciMap)
use the "auto" variable type for iterators. The compiler will figure out from the type specified in the rest of the for() loop body what kind of map typedef to use. It's so clean it's almost magic!
for(auto itr = Summary.callMap.begin(); itr!= Summary.callMap.end(); ++itr) {
define some manifest constants to make the return from .erase and .empty() more meaningfull.
if(ERASE_SUCCESSFUL == cdrMap.erase (itr->second->iGUID)) {
given that "smart pointers" are really just keeping a reference count, remember you can always keep your own reference count, an probably in a cleaner, and more obvious way. Combining this with #5 and #10 above, you can write some nice clean code like this.
#define Pear(x,y) std::make_pair(x,y) // some macro magic
auto res = pSumStruct->callMap.insert(Pear(pCDR->iGUID,pCDR));
if ( ! res.second ) {
pCDR->RefKnt=2;
} else {
pCDR->RefKnt=1;
pSumStruct->Knt += 1;
}
using a pointer to hang onto a map node which allocates everything for itself, IE: no user pointers pointing to user malloc()ed objects, works well, is potentially more efficient, and and be used to mutate a node's data without side-effects in my experience.
on the same theme, such a pointer can be used very effectively to preserve the state of a node, as in pThisCDRLeafData above. Passing this to a function that mutates/changes that particular node's data is cleaner than passing a reference to the map and the key needed to get back to the node pThisCDRLeafData is pointing to.
iterators are not magic. They are expensive and slow, as you are navigating the map to get values. For a map holding a million values, you can read a node based on a key at about 20 million per second. With iterators it's probably ~ 1000 times as slow.
I think that about covers it for now. Will update if any of this changes or there's additional insights to share. I am especially enjoying using the STL with C code. IE: not a class in sight anywhere. They just don't make sense in the context I'm working in, and it's not an issue. Good luck.