Initializing a large global constant object - c++

Question

How do you use a file to initialize a global const object that is too large to be created in the stack? This is my attempt so far:
// test.h
#pragma once
#include <boost/array.hpp>
typedef boost::array<int,100000> bigLut_t;
extern const bigLut_t constLut;
// test.cpp
#include <fstream>
#include <boost/filesystem.hpp>
#include "test.h"
bigLut_t& initializeConstLut()
{
if( boost::filesystem::exists("my_binary_file") == false ) {
std::ofstream outStream( "my_binary_file", ios::out | ios::binary );
bigLut_t* tempLut = new bigLut_t;
for(int i = 0; i < 100000; ++i) {
// Imagine this taking a long time,
// which is why we're using a file in the first place
tempLut->at(i) = i;
}
outStream.write( reinterpret_cast<char*>(tempLut), sizeof(bigLut_t) );
outStream.close();
delete tempLut;
}
// We can't write "bigLut_t lut;" because that would cause a stack overflow
bigLut_t* lut = new bigLut_t; // lut gets never deallocated
std::ifstream inStream( "my_binary_file", ios::in | ios::binary );
inStream.read( reinterpret_cast<char*>(lut), sizeof(bigLut_t) );
inStream.close();
return *lut;
}
const bigLut_t constLut = initializeConstLut();
AFAIK this works in a sense that constLut gets corretly initialized, but there's a memory leak since bigLut_t* lut gets never deallocated. I tried using a smart pointer for it, but that resulted in the values in constLut being quite random. I'm baffled by the lack of information I found by trying to google the solution.

Answer 1

How did you use shared_ptr ? Try the following :
// test.h
#pragma once
#include <boost/array.hpp>
typedef boost::array<int,100000> bigLut_t;
extern const bigLut_t constLut;
// test.cpp
#include <fstream>
#include <boost/filesystem.hpp>
#include "test.h"
boost::shared_ptr<bigLut_t> initializeConstLut()
{
if( boost::filesystem::exists("my_binary_file") == false ) {
std::ofstream outStream( "my_binary_file", ios::out | ios::binary );
bigLut_t* tempLut = new bigLut_t;
for(int i = 0; i < 100000; ++i) {
// Imagine this taking a long time,
// which is why we're using a file in the first place
tempLut->at(i) = i;
}
outStream.write( reinterpret_cast<char*>(tempLut), sizeof(bigLut_t) );
outStream.close();
delete tempLut;
}
// We can't write "bigLut_t lut;" because that would cause a stack overflow
boost::shared_ptr<bigLut_t> lut(new bigLut_t); // lut gets never deallocated
std::ifstream inStream( "my_binary_file", ios::in | ios::binary );
inStream.read( reinterpret_cast<char*>(lut), sizeof(bigLut_t) );
inStream.close();
return lut;
}
const bigLut_t constLut = *(initializeConstLut().get());

Answer 2

There are several good solutions to your problem. The solutions offered as answers so far are not among the good solutions (in particular, dynamic allocation and copying and relying on lifetime of temporary objects is just Very Bad™). I'll just give you one common solution.
A simple way to provide a huge constant is to use a Meyers' singleton, which means defining the constant as a static local variable in a function that returns a reference to it:
inline BigThing const& theBigThing()
{
static BigThing const theInstance; // Default constructor does the init job.
return theInstance;
}
This is not yet a complete solution, but let's first see how you can get rid of having to invoke a function, and instead dealing directly with what-looks-like-a-constant:
namespace detail {
inline BigThing const& theBigThing()
{
static BigThing const theInstance; // Default constructor does the init job.
return theInstance;
}
}
BigThing const& theBigThing = detail::theBigThing(); // No copying, just ref.
In your case where BigThing is an array that should be initialized from data in a file, you can't directly rely on the default constructor. You could if you defined a wrapper class, and that's one way. OK, let's do that (it's what I'd choose, I think):
namespace detail {
struct BigThingWrapper
{
BigThing thingy_;
BigThingWrapper()
{
// Initialize the thingy_ member here.
}
};
inline BigThing const& theBigThing()
{
static BigThingWrapper const theInstance;
return theInstance.thingy_;
}
}
BigThing const& theBigThing = detail::theBigThing(); // No copying, just ref.
Note 1: I've used inline so that the code conceivably could be placed in a header file. Just remove for implementation placed in an implementation file.
Note 2: This code is untouched by compiler's hands and so may contain ERORS, TYPPOS and whatnot. :-) But it's the idea that is your answer.
Note 3: There are, as mentioned, also other good ways to do this, so this is not "the" answer, and there is no "the" answer, but it's "an" answer.
Cheers & hth.,

Answer 3

Just doing it the old fashioned way is fine - create an auto pointer whose lifecycle is the whole application life-cycle, and an extern reference to the pointer:
// test.h
#pragma once
#include <boost/array.hpp>
typedef boost::array<int,100000> bigLut_t;
extern const bigLut_t& constLut; // make it a reference
// test.cpp
#include <fstream>
#include <boost/filesystem.hpp>
#include "test.h"
namespace {
std::auto_ptr<bigLut_t> initializeConstLut()
{
std::auto_ptr<bigLut_t> lut(new bigLut_t);
if( boost::filesystem::exists("my_binary_file") == false ) {
std::ofstream outStream( "my_binary_file", ios::out | ios::binary );
for(int i = 0; i < 100000; ++i) {
// Imagine this taking a long time,
// which is why we're using a file in the first place
lut->at(i) = i;
}
outStream.write( reinterpret_cast<char*>(lut), sizeof(bigLut_t) );
outStream.close();
// no point writing then reading the same data
} else {
std::ifstream inStream( "my_binary_file", ios::in | ios::binary );
inStream.read( reinterpret_cast<char*>(lut.get()), sizeof(bigLut_t) );
inStream.close();
}
return lut;
}
// local to this compilation unit, deletes object on exit
std::auto_ptr<bigLut_t> constLutPtr ( initializeConstLut() );
}
// the extern reference refers to the object held by the auto_ptr
const bigLut_t& constLut ( *constLutPtr.get() );
No extra copying, client code sees the extern variable as before, though the linker may have to have an extra indirection rather than the extern varibale being in a fixed address (&constLutPtr is on the heap rather than in static data area).
If having a fixed address for constLut is important, go back to having an extern value rather than an extern reference, read the data in using a reinterpret cast and a const_cast of &constLutPtr. pass reinterpret_cast<char*>(const_cast<bigLut_t*>(&constLut)) to the stream read.

Answer 4

Wouldn't it be easier to just make your global object a pointer, new a temporary object from initializeConstLut and set the global pointer to that object before exiting the function ?
If the object is so big, you should avoid any solution which involves copying and/or assignment. So you're left with several choices :
Make the global object a pointer (free copy)
Make the global object non-const and read directly in it
I can't believe I'm saying this, but use a Singleton like pattern and wrap the object. This will allow you to isolate initialization and provide a public method for access which will return a const reference to the array.
I'm not adding much details here because it really depends on your requirements (I'm thinking thread-safety for instance).

Answer 5

There is a line of thought that says known objects do not leak, if the only leak is at the end of application life. If all you do with the object is remove it from memory (and do not release other resources, nor update some external stuff like DB's and files) then you can just leave it there. After all, by explicitly deleting them from memory you are just blocking the application from finishing for no real benefit: memory will be released one way or the other.

Answer 6

Add an explicit destruction of the lut. You are responsible for destroying anything you create. Best is to do it explicitly:
void destroyLut(bigLut_t& lut)
{
delete &lut;
}
Then simply call this function before exiting main. You should never rely on static initialization and destruction in C++. Better is to always do explicit initialization and destruction.