How check fast, if database reachable? (Qt, QML, C++)- Linux - c++

I use qt with qml and c++. On my application i use a database.
It all works, if the database is reachable.
My problem is, that i would like to check, if database is reachable (like ping).
I tried
db.setDatabaseName(dsn);
if(db.isValid())
{
if(db.open())
{
//std::cout <<"Offene Datenbank";
connected=true;
}
else
{
connected=false;
}
}
else
{
connected=false;
}
and give the connected value as result. But that takes very long (maybe 30 seconds), if there is no connection. How i can check fast, if i have a database connection?
Is there maybe a way to break the command .open after 5 seconds not connected?

I think one easy solution is to just check the ping of the database sever. You can use platform specific ways for pinging.
This would work on Linux :
int exitCode = QProcess::execute("ping", QStringList() << "-c 2" << serverIp);
if (exitCode==0)
{
// is reachable
} else
{
// is not reachable
}

I have studied this question a bit. Here is what I found out.
The problem is in default db connection timeout - it is too long. Each db allows you to change it to an acceptable value, using their own API. In Qt there is one common db interface - QSqlDatabase. And it does not have such method. You can set connection settings by calling it's QSqlDatabase::setConnectOptions method, but it accepts only predefined list of options (which you can read in Qt's help).
For PostgreSQL there is an option connect_timeout, so you can write:
db.setConnectOptions("connect_timeout=5"); // Set to 5 seconds
For other databases there is no such parameter. Connection options of each db are parsed in it's 'driver' class, which derives QSqlDriver and is stored in a 'driver' library.
So, what you can do:
You can rewrite database's driver in order it to accept timeout option.
You can write separate code for each db, using it's native API.
UPDATE
Turns out, that ODBC has SQL_ATTR_CONNECTION_TIMEOUT option.
UPDATE 2
qsql_odbc.cpp:713
} else if (opt.toUpper() == QLatin1String("SQL_ATTR_CONNECTION_TIMEOUT")) {
v = val.toUInt();
r = SQLSetConnectAttr(hDbc, SQL_ATTR_CONNECTION_TIMEOUT, (SQLPOINTER) v, 0);
https://msdn.microsoft.com/en-us/library/ms713605(v=vs.85).aspx
SQL_ATTR_CONNECTION_TIMEOUT (ODBC 3.0)
An SQLUINTEGER value
corresponding to the number of seconds to wait for any request on the
connection to complete before returning to the application. The driver
should return SQLSTATE HYT00 (Timeout expired) anytime that it is
possible to time out in a situation not associated with query
execution or login.
If ValuePtr is equal to 0 (the default), there is no timeout.
Should work fine...

I suggest having some separate thread/class where you check connection and emit signal after some timeout if nothing happens (with a check - knowConnection - if we found out already if its connected).
This code is not tested and written from scratch on top of my head.. may contain some errors.
/// db connection validator in separate thread
void validator::doValidate() {
this->knowConnection = false;
db.setDatabaseName(dsn);
if(db.isValid())
{
QTimer::singleShot(1000, [this]() {
if (!this->knowConnection) {
emit connected(false);dm->connected=false;
}
});
if(db.open())
{
//std::cout <<"Offene Datenbank";
this->knowConnection = true;
dm->connected=true;
emit connected(true);
}
else
{
dm->connected=false;
this->knowConnection = true;
emit connected(false);
}
}
else
{
dm->connected=false;
this->knowConnection = true;
emit connected(false);
}
}
/// db manager in different thread
void dm::someDbFunction() {
if (connected) {
/// db logic
}
}
/// in gui or whatever
MainWindow::MainWindow() : whatever, val(new validator(..), .. {
connect(val, SIGNAL(connected(bool)), this, SLOT(statusSlot(bool));
....
}
void MainWindow::statusSlot(bool connected) {
ui->statusBar->setText((connected?"Connected":"Disconnected"));
}

Related

gRPC: What are the best practices for long-running streaming?

We've implemented a Java gRPC service that runs in the cloud, with an unidirectional (client to server) streaming RPC which looks like:
rpc PushUpdates(stream Update) returns (Ack);
A C++ client (a mobile device) calls this rpc as soon as it boots up, to continuously send an update every 30 or so seconds, perpetually as long as the device is up and running.
ChannelArguments chan_args;
// this will be secure channel eventually
auto channel_p = CreateCustomChannel(remote_addr, InsecureChannelCredentials(), chan_args);
auto stub_p = DialTcc::NewStub(channel_p);
// ...
Ack ack;
auto strm_ctxt_p = make_unique<ClientContext>();
auto strm_p = stub_p->PushUpdates(strm_ctxt_p.get(), &ack);
// ...
While(true) {
// wait until we are ready to send a new update
Update updt;
// populate updt;
if(!strm_p->Write(updt)) {
// stream is not kosher, create a new one and restart
break;
}
}
Now different kinds of network interruptions happen while this is happening:
the gRPC service running in the cloud may go down (for maintenance) or may simply become unreachable.
the device's own ip address keeps changing as it is a mobile device.
We've seen that on such events, neither the channel, nor the Write() API is able to detect network disconnection reliably. At times the client keep calling Write() (which doesn't return false) but the server doesn't receive any data (wireshark doesn't show any activity at the outgoing port of the client device).
What are the best practices to recover in such cases, so that the server starts receiving the updates within X seconds from the time when such an event occurs? It is understandable that there would loss of X seconds worth data whenever such an event happens, but we want to recover reliably within X seconds.
gRPC version: 1.30.2, Client: C++-14/Linux, Sever: Java/Linux
Here's how we've hacked this. I want to check if this can be made any better or anyone from gRPC can guide me about a better solution.
The protobuf for our service looks like this. It has an RPC for pinging the service, which is used frequently to test connectivity.
// Message used in IsAlive RPC
message Empty {}
// Acknowledgement sent by the service for updates received
message UpdateAck {}
// Messages streamed to the service by the client
message Update {
...
...
}
service GrpcService {
// for checking if we're able to connect
rpc Ping(Empty) returns (Empty);
// streaming RPC for pushing updates by client
rpc PushUpdate(stream Update) returns (UpdateAck);
}
Here is how the c++ client looks, which does the following:
Connect():
Create the stub for calling the RPCs, if the stub is nullptr.
Call Ping() in regular intervals until it is successful.
On success call PushUpdate(...) RPC to create a new stream.
On failure reset the stream to nullptr.
Stream(): Do the following a while(true) loop:
Get the update to be pushed.
Call Write(...) on the stream with the update to be pushed.
If Write(...) fails for any reason break and the control goes back to Connect().
Once in every 30 minutes (or some regular interval), reset everything (stub, channel, stream) to nullptr to start afresh. This is required because at times Write(...) does not fail even if there is no connection between the client and the service. Write(...) calls are successful but the outgoing port on the client does not show any activity on wireshark!
Here is the code:
constexpr GRPC_TIMEOUT_S = 10;
constexpr RESTART_INTERVAL_M = 15;
constexpr GRPC_KEEPALIVE_TIME_MS = 10000;
string root_ca, tls_key, tls_cert; // for SSL
string remote_addr = "https://remote.com:5445";
...
...
void ResetStreaming() {
if (stub_p) {
if (strm_p) { // graceful restart/stop, this pair of API are called together, in this order
if (!strm_p->WritesDone()) {
// Log a message
}
strm_p->Finish(); // Log if return value of this is NOT grpc::OK
}
strm_p = nullptr;
strm_ctxt_p = nullptr;
stub_p = nullptr;
channel_p = nullptr;
}
}
void CreateStub() {
if (!stub_p) {
ChannelArguments chan_args;
chan_args.SetInt(GRPC_ARG_KEEPALIVE_TIME_MS, GRPC_KEEPALIVE_TIME_MS);
channel_p = CreateCustomChannel(
remote_addr,
SslCredentials(SslCredentialsOptions{root_ca, tls_key, tls_cert}),
chan_args);
stub_p = GrpcService::NewStub(m_channel_p);
}
}
void Stream() {
const auto restart_time = steady_clock::now() + minutes(RESTART_INTERVAL_M);
while (!stop) {
// restart every RESTART_INTERVAL_M (15m) even if ALL IS WELL!!
if (steady_clock::now() > restart_time) {
break;
}
Update updt = GetUpdate(); // get the update to be sent
if (!stop) {
if (channel_p->GetState(true) == GRPC_CHANNEL_SHUTDOWN ||
!strm_p->Write(updt)) {
// could not write!!
return; // we will Connect() again
}
}
}
// stopped due to stop = true or interval to create new stream has expired
ResetStreaming(); // channel, stub, stream are recreated once in every 15m
}
bool PingRemote() {
ClientContext ctxt;
ctxt.set_deadline(system_clock::now() + seconds(GRPC_TIMEOUT_S));
Empty req, resp;
CreateStub();
if (stub_p->Ping(&ctxt, req, &resp).ok()) {
static UpdateAck ack;
strm_ctxt_p = make_unique<ClientContext>(); // need new context
strm_p = stub_p->PushUpdate(strm_ctxt_p.get(), &ack);
return true;
}
if (strm_p) {
strm_p = nullptr;
strm_ctxt_p = nullptr;
}
return false;
}
void Connect() {
while (!stop) {
if (PingRemote() || stop) {
break;
}
sleep_for(seconds(5)); // wait before retrying
}
}
// set to true from another thread when we want to stop
atomic<bool> stop = false;
void StreamUntilStopped() {
if (stop) {
return;
}
strm_thread_p = make_unique<thread>([&] {
while (!stop) {
Connect();
Stream();
}
});
}
// called by the thread that sets stop = true
void Finish() {
strm_thread_p->join();
}
With this we are seeing that the streaming recovers within 15 minutes (or RESTART_INTERVAL_M) whenever there is a disruption for any reason. This code runs in a fast path, so I am curious to know if this can be made any better.

Determine whether SQLite database is locked

I've read other answers on how to detect whether the SQLite database is locked, and it suggests to use sqlite3_busy_handler/sqlite3_busy_timeout. I'm using Qt which has support of SQLite databases but that doesn't matter here.
Because in the use case of my application another application might access the same database, I need to handle this case.
Here's the options I set to the database when opening it:
auto db = QSqlDatabase::database();
db.setDatabaseName(m_sDatabasePath);
db.setConnectOptions("QSQLITE_BUSY_TIMEOUT=0");
if (!db.open())
return false;
I have a function which should determine whether database is locked or not:
int callback(void *data, int)
{
std::cout << "callback" << std::endl;
return 0;
}
bool isDatabaseLocked(const QSqlDatabase &db)
{
if (auto driver = db.driver())
{
// get driver handler
QVariant v = driver->handle();
if (v.isValid() && qstrcmp(v.typeName(), "sqlite3*") == 0)
{
// v.data() returns a pointer to the handle
auto handle = *static_cast<sqlite3 **>(v.data());
if (handle)
{
std::cout << sqlite3_busy_handler(handle, callback, nullptr) << std::endl;
std::cout << sqlite3_busy_timeout(handle, 0) << std::endl;
}
}
}
return true;
}
When the database is lcoked, I execute this function and get printed two 0s (SQLITE_OK) while I'd expect to get 5 (SQLITE_BUSY). And the callback function isn't called either.
What am I doing wrong?
As clearly stated in the documentation, the sqlite3_busy_handler function sets a callback function as the busy handler for the connection. Such callback might be invoked whenever a lock is met. In your code, the call to sqlite3_busy_handler returns SQLITE_OK, which is perfectly fine: it means the attempt to set the callback succeeded, no reason to return SQLITE_BUSY at this time, since the call only affects the connection, not the database.
About the same applies to the sqlite3_busy_timeout function, which in turn sets a different kind of busy handler (which replace the callback you just set, by the way) which simply sleeps for the specified amount of milliseconds until the lock is released.
Now, generally speaking, how to detect if a resource is locked? The usual way is: try to lock it.
So you could use a function like this:
bool isDatabaseLocked(const QSqlDatabase & db)
{
QSqlQuery q(db);
if(q.exec("BEGIN EXCLUSIVE")) //tries to acquire the lock
{
q.exec("COMMIT"); //releases the lock immediately
return false; //db is not locked
}
return true; //failed to acquire the lock: returns true (db is locked)
}
To make it return immediately when the lock cannot be acquired, clear the busy handler, setting the busy timeout to zero for this connection (before opening it):
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
db.setConnectOptions("QSQLITE_BUSY_TIMEOUT=0");
To test the function, open a command line sqlite3 session on the database, and lock it by typing:
sqlite> BEGIN EXCLUSIVE;
When you want to release the lock, just type
sqlite> COMMIT;

Sleep inside QTConcurrent run method

I'm using QtConcurrent::run to execute some functions in background and not hang the GUI thread. In one function, I read logs from local SQlite database and send them to server by TCP socket.
Now I want to delay the execution after each log so the server has time to save it (TCP response is read in different thread). I'm stuck with Qt4.8 due to implementation limitations (many embeded devices - no chance to upgrade QT on them) and I can't use QThread::sleep(2) because it is protected in 4.8.
Is it possible to somehow pause the execution of thread inside QtConcurrent::run method or should I redesign it to implement my own class inheriting QThread?
void MainWindow::ReportFinishedHUs(bool asyncWork)
{
if(asyncWork == false)
{
QMutexLocker locker(&localDBmutex);
QList<QSqlRecord> HUsToReport = localDB->getHUsForBook();
qDebug() << "HUs to report" << HUsToReport.count();
if(!HUsToReport.isEmpty())
{
Cls_log::WriteDebugLog("HUs to report: " + QString::number(HUsToReport.count()));
foreach (QSqlRecord record, HUsToReport)
{
int _hu = record.indexOf("hu");
int _logTime = record.indexOf("logTime");
QString logTimeString = record.value(_logTime).toString();
QString hu = record.value(_hu).toString();
qDebug() << hu << logTimeString;
// creating message here ...
qDebug() << message;
emit sig_SendTCPMessage(message);
// this is where I need to wait for 2 seconds
QThread::sleep(2);
}
}
}
else
{
QtConcurrent::run(this, &MainWindow::ReportFinishedHUs, false);
}
}
EDIT:
Solved by usleep(2000000) which I somehow discarded for being platform specific... but hey, half of my aplication is platform specific and I only use it in embeded device with constant OS.
Keeping the question open if anyone can suggest more elegand solution using Qt methods. I like to get inspired.

qt waitforreadyread() inconsistent?

Under Qt 4.7.1, OS X 10.6.8
(have to use this -- later versions
of Qt and/or OS X introduce severe
incompatibilities for my users)
The following works. Sometimes. Then sometimes not.
When it doesn't work, it returns "Unknown Error"
hst is good in all cases, qDebug returns same correct
data for hst every time.
The idea is, use ->get to pull a CGI URL; the CGI
returns some data, which I can ignore in this case.
Then I'm done.
hst is a well formed URL,
http://yadda.com/cgi-bin/whatever.py
QString hst;
QNetworkReply *qnr;
QNetworkAccessManager *qqnap = NULL;
qqnap = new(std::nothrow) QNetworkAccessManager(tmw);
if (qqnap != NULL)
{
hst = loaduphst(); // get qstring to send
qnr = qqnap->get(QNetworkRequest(QUrl(hst))); // report in and fetch update info
if (qnr->waitForReadyRead(3000) == FALSE)
{
qDebug() << "waitForReadyRead() returned FALSE -- error or timeout:" << qnr->errorString();
}
}
else
{
qDebug() << "qqnap is NULL";
}
yadda.com is up; the target script is dead simple
and works fine from browser or cmd line every time.
This is running within the context of
MainWindow::closeEvent(QCloseEvent *ce)
before I emit ce->accept() GUI is still up,
etc.
Hints? Tips? Abuse? Thanks!
waitForReadyRead is not implemented in QNetworkReply. The default implementation does nothing:
bool QIODevice::waitForReadyRead(int msecs)
{
Q_UNUSED(msecs);
return false;
}
Use the readyRead signal to find out when there is data available to be read.
More-or-less synchronous use of async networking is very problematic in the context of the main GUI loop. Signals that don't appear (finished OR readyRead), URLs that sometimes send and sometimes don't... and of course, as the kind person above pointed out, unimplemented functions. Zebras!
What we can do, though, is fire up an event loop and a timer on our own, and this will in a more-or-less friendly way act synchronous.
Perhaps some poor soul will need to poke a website CGI as I do; here's the code. It works. At least under Qt 4.7.1 it does!
So anyway, here it is:
QNetworkReply *qnr;
QNetworkAccessManager *qqnap;
QNetworkRequest qnwr;
QEventLoop w;
QTimer arf;
if ((qqnap = new(std::nothrow) QNetworkAccessManager(this)))
{
qnwr.setUrl(myUrl()); // Build web goodness
qnwr.setRawHeader("User-Agent", myUserAgent());
arf.setSingleShot(true);
if (connect(&arf, SIGNAL(timeout()), // timer firing blows...
&w, SLOT(quit()) // ...out event loop
) == FALSE)
{ return(BAD_CONNECT_TOUT); }
if (connect(qqnap, SIGNAL(finished(QNetworkReply*)), // notify we finished...
this, SLOT(qqnapReplyQ(QNetworkReply*)) // ...cuz I need to know
) == FALSE)
{ return(BAD_CONNECT_FINISHED_NOTIFY); }
if (connect(qqnap, SIGNAL(finished(QNetworkReply*)), // finishing blows out...
&w, SLOT(quit()) // ...event loop
) == FALSE)
{ return(BAD_CONNECT_FINISHED_ELOOP); }
if ((qnr = qqnap->get(qnwr))) // Go if qnr is good
{
arf.start(6000); // timeout in ms // Watchdog timer on
w.exec(); // handle all that
if (arf.isActive()) { arf.stop(); } // kill timer if needed
}
else { return(BAD_WWWGET); } // FAIL
}
else
{
return(BAD_NWAM); // FAIL
}
return(ZEN_NETWORKING);

Simplest QT TCP client

I would like to connect to a listening server and transmit some data. I looked at the examples available but they seem to have extra functions that do not seem very helpful to me (i.e. connect, fortune, etc.). This is the code I have so far:
QTcpSocket t;
t.connectToHost("127.0.0.1", 9000);
Assuming the server is listening and robust, what do I need to implement to send a data variable with datatype QByteArray?
very simple with QTcpSocket. Begin as you did...
void MainWindow::connectTcp()
{
QByteArray data; // <-- fill with data
_pSocket = new QTcpSocket( this ); // <-- needs to be a member variable: QTcpSocket * _pSocket;
connect( _pSocket, SIGNAL(readyRead()), SLOT(readTcpData()) );
_pSocket->connectToHost("127.0.0.1", 9000);
if( _pSocket->waitForConnected() ) {
_pSocket->write( data );
}
}
void MainWindow::readTcpData()
{
QByteArray data = pSocket->readAll();
}
Be aware, though, that for reading from the TcpSocket you may receive the data in more than one transmission, ie. when the server send you the string "123456" you may receive "123" and "456". It is your responsibility to check whether the transmission is complete. Unfortunately, this almost always results in your class being stateful: the class has to remember what transmission it is expecting, whether it has started already and if it's complete. So far, I haven't figured out an elegant way around that.
In my case I was reading xml data, and sometimes I would not get all in one packet.
Here is an elegant solution. WaitForReadyRead could also have a time out in it and
then some extra error checking in case that timeout is reached. In my case I should never
receive an incomplete xml, but if it did happen this would lock the thread up indefinetly
without the timeout:
while(!xml.atEnd()) {
QXmlStreamReader::TokenType t = xml.readNext();
if(xml.error()) {
if(xml.error() == QXmlStreamReader::PrematureEndOfDocumentError) {
cout << "reading extra data" << endl;
sock->waitForReadyRead();
xml.addData(sock->readAll());
cout << "extra data successful" << endl;
continue;
} else {
break;
}
}
...