Increasing memory usage in sqlite3? - c++

Question

I've written a console app which receives events via boost::interprocess memory and dumps the info into an sqlite3 database. While running the app I've noticed that, in the Windows task manager, the memory usage was cyclically increasing every... 30s-1min. This led me to believe that the problem lies within the main loop in which I execute my SQL. I've added some monitoring and apparently the sqlite3_memory_usage returns increasing results every couple loop iterations.
Can somebody tell me what am I doing wrong? Am I missing something I should de-allocate ?
Here are 2 strings I use to generate SQL
const std::string sql_insert =
"INSERT INTO EventsLog "
"(Sec, uSec, DeviceId, PmuId, EventId, Error, Msg) "
"VALUES (%ld, %ld, %ld, %d, %ld, %d, %Q)";
const std::string sql_create =
"CREATE TABLE IF NOT EXISTS EventsLog("
"Id INTEGER PRIMARY KEY AUTOINCREMENT, "
"Sec INTEGER NOT NULL, "
"uSec INTEGER NOT NULL, "
"DeviceId INTEGER NOT NULL, "
"PmuId INTEGER NOT NULL, "
"EventId INTEGER NOT NULL, "
"Error INTEGER NOT NULL, "
"Msg TEXT"
")";
In here, I generate the SQL INSERT command
std::string construct_sql_query
(const ELMessageData & data)
{
std::string query = "";
ptime jan1st1970 = ptime(date(1970,1,1));
ptime now = boost::posix_time::microsec_clock::universal_time();
time_duration delta = now - jan1st1970;
TimeVal time((uint32)delta.total_seconds(),
(uint32)now.time_of_day().fractional_seconds());
char * const sql = sqlite3_mprintf(sql_insert.c_str(),
time.tv_sec,
time.tv_usec,
data.getDeviceId(),
data.getPmuId(),
data.getEventId(),
(data.getIsError() ? 1 : 0),
data.getExMsg().c_str());
if(sql == NULL)
post_event(EvIOError("Failed to create the SQL command",
"StLoggingEvents::_construct_sql_query"));
query = std::string(sql);
sqlite3_free(sql);
return query;
} // construct_sql_query
Here's the main loop in which I execute the INSERT commands
while(true)
{
m_exchange_obj->wait(); // wait for the semaphore to be raised
const std::string sql = construct_sql_query
(m_exchange_obj->receive());
char ** err = NULL;
const int rc = sqlite3_exec(m_db_handle,
sql.c_str(),
NULL,
NULL,
err);
sqlite3_free(err);
if(rc != SQLITE_OK)
{
LERR_ << "Error while inserting into the database";
LERR_ << "Last SQL Query : ";
LERR_ << sql;
}
else
{
LDBG_ << "Event logged...";
LDBG_ << "Sqlite3 memory usage : "
<< sqlite3_memory_used();
}
}

Answer 1

I second the suggestion of trying this under valgrind. You may also want to look at google's tcmalloc replacement... It can print pretty graphs showing you all your leaks... That said, I hope you get the answer for this... I plan on using SQLite in an upcoming project...

Answer 2

How are you determining your memory usage? You may not have a real leak.
If you are on a windows system and using visual studio, compile in debug mode and use the memory debugging macros to find your leaks.
If you are on a unix based system, try valgrind / memcheck.
I think OS X's xcode also detects leaks too.