I'm writing a program which updates MySQL tables every few seconds as new data is received. The function has an infinite loop to update constantly, with another loop inside it that iterates through table rows and picks data accordingly.
Unfortunately, no matter what I do, I encounter the most pathetic error:
Can't create more than max_prepared_stmt_count statements
I've tried simply deleting all mysql related values (except con and driver) after each iteration with statements seen in the example code here.
I've tried adding
stmt = NULL;
stmt->close();
Just in case before every value.
I've tried rewriting the program with auto_ptr, which allegedly clean up automatically, based on this answer.
I have even tried closing and reestablishing connection to the MySQL server after each iteration, but even that did not help me.
Please help.
More information on your loop may be required.
But it sounds like your stmt = con->createStatement(); is inside your loop. You only need one connection, before your loop starts.
Then you can execute a statment multiple times in your loop, res = stmt->executeQuery("SELECT 'Hello World!' AS _message");
When your application terminates then you close your stmt, delete stmt;
From mysql.com "The default value is 16,382." mysql.com-sysvar_max_prepared_stmt_count Which it sounds like you only require one.
Related
I am working on a project, and I need to push a list to my database. I want to iterate through the said list and execute a statement that adds said list to my table. Running the exact script within MySQL works perfectly fine. However, when I try to iterate through the list and run the script, it does it once before exiting my for loop. It won't even run any code past the executeQuery(). My code is as follows:
for (int i = 0; i < addList.size(); i++) {
stmt = con->createStatement();
stmt->executeQuery("INSERT INTO comboboxtable (combobox, user, text) VALUES (" + std::to_string(id) + ", \'" + username.toStdString() + "\', \'" + addList[i].toStdString() + "\')");
}
I have a driver and connection already established, and everywhere else in my code it seems to execute queries just fine. I also have this code in a try/catch set for catching SQL exceptions, but it isn't throwing anything. I am using the MySQL C++ connector to do this. The size of the loop is fine and everything should be working on the iteration side.
I seem to have figured it out after a bit of digging on the website and found that I needed to use execute() not executeQuery().
Ich have to shuffle a lot of data data from an application into an ORACLE 11g database. To execute SQL from within C++ I use to employ the following scheme, using bare SQL together with the Poco::Data framework and an ODBC connection to my database:
// get session from ODBC session pool
Session session = moc::SessionPoolMOC::get();
// prepare a statement as QString
QString q = ("INSERT INTO MOC_DATA_CONCENTRATOR"
"("
"EQUIPMENT_INSTANCE_ID, "
"STATION_NAME, "
"DATA_SRC, "
"ORG_ID)"
"values (%1, '%2', '%3', %4);"
);
/*
there would be also prepared statements, but I think that is not the
topic now...
*/
// set parameters within string from passed object 'info'
q = q.arg(info.equipment_instance_id); /* 1 */
q = q.arg(info.station_name.toUtf8().constData()); /* 2 */
q = q.arg(info.data_src.toUtf8().constData()); /* 3 */
q = q.arg(info.org_id); /* 4 */
// prepare statement
Statement query(session);
query << q.toUtf8().constData();
try
{
// execute query
query.execute();
}
catch (...)
{
...
}
I'm aware, that this is a very low level approach, but it worked really fine for all situations I encountered so far...
The problem is, that I have now a lot of data (about 400,000 records) to fill into one table. Data is available as different C++ objects, stored in a Qt-List. My naive approach is to call this code sequence for each object to insert. This works fine, but turns out to be quite slow (taking about 15 minutes or so). I asked around, and was told to better use a so called "bulk insert".
It seems, however, that this is more related to PLSQL - at least I have no Idea how to use that from the same context or in a similar way as shown in my example. I'm neither ORACLE expert nor administrator and usage of the database is only a side problem within my project. What would you recommend?
Many thanks,
Michael
I have an sqlite database that doesn't change.
Multiple processes that open a database connection each in SQLITE_OPEN_READONLY mode using sqlite3_open_v2. Each process is single threaded
The connections are made from an MSVC project using the official C/C++ Interface's single amalgamated C source file.
According to the SQLite FAQ multiple processes running SELECTs is fine
Each process after opening the database creates 4 prepared SELECT statements each with 2 bindable values.
Over the course of the execution the statements (one at a time) have the following called on them repeatedly as required
sqlite3_bind_int
sqlite3_bind_int
sqlite3_step (while SQLITE_ROW is returned)
sqlite3_column_int (while there was a row)
sqlite3_reset
The prepared statements are reused so finalize isn't called on each of them until near the end of the program. Finally the database is closed at the very end of execution.
The problem is any of these operations can fail with error code = 5: 'database is locked'
Error code 5 is SQLITE_BUSY and the website states that
"indicates a conflict with a separate database connection, probably in a separate process"
The rest of the internet seems to agree that multiple READONLY connections is fine. I've gone over and over the source and can't see that anything is wrong (I can't post it here sadly, I know, not helpful)
So I'm turning it to you guys, what could I possibly be missing?
EDIT 1:
Database is on a local drive, File system is NTFS, OS is Windows 7.
EDIT 2:
Wrapping all sqlite3 calls in infinite loops that check if SQLITE_BUSY was returned and then remake the call alleviates the problem. I don't consider this a fix but if that truly is the right thing to do then I'll do that.
So the working answer I have used is to wrap all the calls to sqlite in functions that loop that function while SQLITE_BUSY is returned. There doesn't seem to be a simple alternative.
const int bindInt(sqlite3_stmt* stmt, int parameterIndex, int value)
{
int ret;
do
ret = sqlite3_bind_int(stmt, parameterIndex, value);
while (ret == SQLITE_BUSY)
return ret;
}
I have some C++ code on Windows that uses plain old ADO (not ADO.NET) to retrieve data from a bunch of SQL Server databases. The code uses forward-only cursors to allow for fire hose cursors for maximum data throughput on queries that produce large Recordsets.
The code that processes the results looks like this, using the #import-generated wrapper for ADO 2.7:
ADODB::_RecordsetPtr records("ADODB.Recordset");
records->Open(cmd, _variant_t(static_cast<IDispatch *>(m_DBConnection)), ADODB::adOpenForwardOnly, ADODB::adLockReadOnly, ADODB::adCmdText);
if (!(records->BOF && records->EOF))
{
... Loop over the recordset and extract data from each record ...
}
Profiling shows that close to 40% of the above loop is spent in the call to BOF and this is having a massive impact on the overall database code's read performance. Because the code uses forward only cursors, it's not possible to check the RecordCount property as it is always -1 when using a forward only cursor.
Is there another way to either check for an empty Recordset that is not using the BOF/EOF check, or a way to speed up this check?
The other alternative that I can think of at the moment is to use one of the other cursor types and check how that will affect data throughput.
I have a C++ library that calls a stored procedure in MSSQL database using ADODB. Everything works fine on development database. But on test database, I am getting Com error 0x800a0e78: This option is not allowed if an object is closed.
The code looks like
ADODB::_CommandPtr cmd = NULL;
ADODB::_RecordsetPtr rs = NULL;
CHECKHR(cmd.CreateInstance( __uuidof( ADODB::Command) ));
cmd->ActiveConnection = m_connexion;
cmd->CommandText = "my_stored_procedure";
cmd->CommandType = ADODB::adCmdStoredProc;
CHECKHR(cmd->Parameters->Refresh());
//input param
cmd->Parameters->GetItem(paramIndex)->put_Value(paramValue);
// output param
cmd->Parameters->GetItem(paramIndex)->PutDirection(ADODB::adParamOutput);
rs = cmd->Execute( NULL, NULL, ADODB::adCmdStoredProc);
while(! rs->ADO_EOF ) { ...
rs->ADO_EOF is where it crashes the program if I use the test database.
note: The stored procedure is same in both the databases and returns same data.
There is one more flow where another SP is called. It works well with the test database. The problem appears only with this particular SP.
I tend to think that it is not a code issue because it works with development database. But I can consistently reproduce the problem with test database.
Please suggest of next actions I should take to resolve this issue
UPDATE
Due to some miracle this C++ exception has gone away after 1 day. But it is so very slow that the execution almost always times out. I do not know how to justify this behavior
UPDATE: 2013-07-18
After so much time, the error has appeared again. This time with development DB with the same SP
[Com error 0x800a0e78 : Operation is not allowed when the object is closed.
on the same line
while(! rs->ADO_EOF ) {
in rs I can see a memory address pointing to ADODB recordset object. But rs->ADO_EOF is generating the said error