a C++ program that loads over 1 billion records into memory? - c++

Question

I have a whole years worth of option prices in a database (along with other bits of information such as strike price, volatility etc) and would like to load it into memory for my program to process (hash maps, program was written in C++). The trouble is it takes over 10 hours to load into memory at startup (from a local database).
Can someone please suggest how i can overcome this issue ? I have workarounds where i only load the portions that i need but it would be great if people can share ideas to the problem. Would loading the data from shared memory after each startup help?
Here is the code so far, in short it queries the database and loads the data into containers
const std::string m_url = "mysql://localhost/test2?user=root&password=";
URL_T optionURL = URL_new(m_url.c_str());
ConnectionPool_T optionPool = ConnectionPool_new(optionURL);
ConnectionPool_setInitialConnections(optionPool,1);
ConnectionPool_setMaxConnections(optionPool,1);
ConnectionPool_start(optionPool);
Connection_T con = ConnectionPool_getConnection(optionPool);
ResultSet_T result = Connection_executeQuery(con,
"SELECT DISTINCT(underlying) FROM Options");
PreparedStatement_T prepareStatement = Connection_prepareStatement(con,"SELECT underlying,underlyingPrice,expiry,strike,issueDate,type,delta,volatility FROM Options o,RiskFreeRate r WHERE underlying = ? AND o.issueDate = r.date");
while(ResultSet_next(result))
{
const std::string symbol = ResultSet_getString(result,1);
PreparedStatement_setString(prepareStatement,1,symbol.c_str());
ResultSet_T resultDetail = PreparedStatement_executeQuery(prepareStatement);
while(ResultSet_next(resultDetail))
{
float strike = ResultSet_getDouble(resultDetail,4);
date expiry = from_string(ResultSet_getString(resultDetail,3));
std::string issueDate = ResultSet_getString(resultDetail,5);
float underlyingPrice = ResultSet_getDouble(resultDetail,2);
float riskFreeRate = 4; //tmp hack
float volatility = ResultSet_getDouble(resultDetail,8);
OptionDateMap optionMap = m_optionMap[symbol];
OptionVec optionVec = optionMap[issueDate];
optionVec.push_back(boost::shared_ptr<WallStreet::FixedIncome::Option::Option> (new WallStreet::FixedIncome::Option::Option(strike,expiry,underlyingPrice, riskFreeRate,volatility)));
optionMap[issueDate] = optionVec;
m_optionMap[symbol] = optionMap;
}
}