I am running a simple java program and trying to study the same using visual vm .
Not trying to debug application performance. app consist of 2 simple classes and creates 2 thread , which does nothing but sleep.
public class ThreadRunExample {
public static void main(String[] args){
Thread t1 = new Thread(new HeavyWorkRunnable(), "t1");
Thread t2 = new Thread(new HeavyWorkRunnable(), "t2");
System.out.println("Starting Runnable threads");
t1.start();
t2.start();
System.out.println("Runnable Threads has been started");
}
}
public class HeavyWorkRunnable implements Runnable{
#Override
public void run() {
System.out.println("Doing heavy processing - START "+Thread.currentThread().getName());
try {
Thread.sleep(1000);
//Get database connection, delete unused data from DB
doDBProcessing();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Doing heavy processing - END "+Thread.currentThread().getName());
}
private void doDBProcessing() throws InterruptedException {
System.out.println("Doing heavy processing - start "+Thread.currentThread().getName());
Thread.sleep(50000);
}
}
Visual VM Sampler tab shows the thread sleeping correctly
However profiler tab , after clicking for CPU profiling shows nothing .
During running of the app , profiler tab is blank . Any idea
Related
I'm using QtConcurrent::run to execute some functions in background and not hang the GUI thread. In one function, I read logs from local SQlite database and send them to server by TCP socket.
Now I want to delay the execution after each log so the server has time to save it (TCP response is read in different thread). I'm stuck with Qt4.8 due to implementation limitations (many embeded devices - no chance to upgrade QT on them) and I can't use QThread::sleep(2) because it is protected in 4.8.
Is it possible to somehow pause the execution of thread inside QtConcurrent::run method or should I redesign it to implement my own class inheriting QThread?
void MainWindow::ReportFinishedHUs(bool asyncWork)
{
if(asyncWork == false)
{
QMutexLocker locker(&localDBmutex);
QList<QSqlRecord> HUsToReport = localDB->getHUsForBook();
qDebug() << "HUs to report" << HUsToReport.count();
if(!HUsToReport.isEmpty())
{
Cls_log::WriteDebugLog("HUs to report: " + QString::number(HUsToReport.count()));
foreach (QSqlRecord record, HUsToReport)
{
int _hu = record.indexOf("hu");
int _logTime = record.indexOf("logTime");
QString logTimeString = record.value(_logTime).toString();
QString hu = record.value(_hu).toString();
qDebug() << hu << logTimeString;
// creating message here ...
qDebug() << message;
emit sig_SendTCPMessage(message);
// this is where I need to wait for 2 seconds
QThread::sleep(2);
}
}
}
else
{
QtConcurrent::run(this, &MainWindow::ReportFinishedHUs, false);
}
}
EDIT:
Solved by usleep(2000000) which I somehow discarded for being platform specific... but hey, half of my aplication is platform specific and I only use it in embeded device with constant OS.
Keeping the question open if anyone can suggest more elegand solution using Qt methods. I like to get inspired.
Consider a situation where you need to maintain 256 tcp connections with devices just for ocassionally sending commands. I want to do this in parallel(It needs to block until it gets the response), I'm trying to use QThreadPool for this purpose but I have some doubts if it is possible.
I tried to use QRunnable but I'm not sure how sockets will behave between threads (sockets should be used only in thread that they were created in?)
I'm also worried about efficiency of this solution, I would be glad if somebody could propose some alternatives, not necessarily using QT.
Below I'm posting some snippets of the code.
class Task : public QRunnable {
Task(){
//creating TaskSubclass instance and socket in it
}
private:
TaskSubclass *sub;
void run() override {
//some debug info and variable setting...
sub->doSomething( args );
return;
}
};
class TaskSubclass {
Socket *sock; // socket instance
//...
void doSomething( args )
{
//writing to socket here
}
}
class MainProgram : public QObject{
Q_OBJECT
private:
QThreadPool *pool;
Task *tasks;
public:
MainProgram(){
pool = new QThreadPool(this);
//create tasks here
}
void run(){
//decide which task to start
pool->start(tasks[i]);
}
};
My favorite solution for this problem is by multiplexing your sockets using select(). That way you don't need to create additional threads, and it is a "very POSIX" way to do it.
See for example see this tutorial:
http://www.binarytides.com/multiple-socket-connections-fdset-select-linux/
Or a related question in:
Using select(..) on client
As OMD_AT has allready pointed out the best solution is to use Select() and let the kernel do the job for you :-)
here you have an example of an Async approach and an Syncron multi thread approach.
In this example we create 10 connection to a google webservice and make a simple get request to the server, we measure how long all connections in each approach needed to receive the response from the google server.
Be aware that you should use a more faster webserver to make a real test, like the localhost because the network latency has a big impact on the result.
#include <QCoreApplication>
#include <QTcpSocket>
#include <QtConcurrent/QtConcurrentRun>
#include <QElapsedTimer>
#include <QAtomicInt>
class Task : public QRunnable
{
public:
Task() : QRunnable() {}
static QAtomicInt counter;
static QElapsedTimer timer;
virtual void run() override
{
QTcpSocket* socket = new QTcpSocket();
socket->connectToHost("www.google.com", 80);
socket->write("GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n");
socket->waitForReadyRead();
if(!--counter) {
qDebug("Multiple Threads elapsed: %lld nanoseconds", timer.nsecsElapsed());
}
}
};
QAtomicInt Task::counter;
QElapsedTimer Task::timer;
int main(int argc, char *argv[])
{
QCoreApplication app(argc, argv);
// init
int connections = 10;
Task::counter = connections;
QElapsedTimer timer;
/// Async via One Thread (Select)
// handle the data
auto dataHandler = [&timer,&connections](QByteArray data) {
Q_UNUSED(data);
if(!--connections) qDebug(" Single Threads elapsed: %lld nanoseconds", timer.nsecsElapsed());
};
// create 10 connection to google.com and send an http get request
timer.start();
for(int i = 0; i < connections; i++) {
QTcpSocket* socket = new QTcpSocket();
socket->connectToHost("www.google.com", 80);
socket->write("GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n");
QObject::connect(socket, &QTcpSocket::readyRead, [dataHandler,socket]() {
dataHandler(socket->readAll());
});
}
/// Async via Multiple Threads
Task::timer.start();
for(int i = 0; i < connections; i++) {
QThreadPool::globalInstance()->start(new Task());
}
return app.exec();
}
Prints:
Multiple Threads elapsed: 62324598 nanoseconds
Single Threads elapsed: 63613967 nanoseconds
Although, the answer is already accepted, I would like to share my)
What I understood from your question: Having 256 currently active connections, from time to time you send a request ("command" as you named it) to one of them and wait for the response. Meanwhile, you want to make this process multithreaded and, though you said "It needs to block until it gets the response", I assume you implied blocking a thread which handles request-response process, but not the main thread.
If I indeed understand the question right, here is how I suggest to do it using Qt:
#include <functional>
#include <QObject> // need to add "QT += core" in .pro
#include <QTcpSocket> // QT += network
#include <QtConcurrent> // QT += concurrent
#include <QFuture>
#include <QFutureWatcher>
class CommandSender : public QObject
{
public:
// Sends a command via connection and blocks
// until the response arrives or timeout occurs
// then passes the response to a handler
// when the handler is done - unblocks
void SendCommand(
QTcpSocket* connection,
const Command& command,
void(*responseHandler)(Response&&))
{
const int timeout = 1000; // milliseconds, set it to -1 if you want no timeouts
// Sending a command (blocking)
connection.write(command.ToByteArray()); // Look QByteArray for more details
if (connection.waitForBytesWritten(timeout) {
qDebug() << connection.errorString() << endl;
emit error(connection);
return;
}
// Waiting for a response (blocking)
QDataStream in{ connection, QIODevice::ReadOnly };
QString message;
do {
if (!connection.waitForReadyRead(timeout)) {
qDebug() << connection.errorString() << endl;
emit error(connection);
return;
}
in.startTransaction();
in >> message;
} while (!in.commitTransaction());
responseHandler(Response{ message }); // Translate message to a response and handle it
}
// Non-blocking version of SendCommand
void SendCommandAsync(
QTcpSocket* connection,
const Command& command,
void(*responseHandler) (Response&&))
{
QFutureWatcher<void>* watcher = new QFutureWatcher<void>{ this };
connect(watcher, &QFutureWatcher<void>::finished, [connection, watcher] ()
{
emit done(connection);
watcher->deleteLater();
});
// Does not block,
// emits "done" when finished
QFuture<void> future
= QtConcurrent::run(this, &CommandSender::SendCommand, connection, command, responseHandler);
watcher->setFuture(future);
}
signals:
void done(QTcpSocket* connection);
void error(QTcpSocket* connection);
}
Now you can send a command to a socket using a separate thread taken from a thread pool: under the hood QtConcurrent::run() uses the global instance of QThreadPool provided by Qt for you. That thread blocks until it gets a response back and than handles it with responseHandler . Meanwhile, your main thread managing all your commands and sockets stays unblocked. Just catch done() signal which tells that response was received and handled successfully.
One thing to note: asynchronous version sends request only when there is a free thread in the thread pool and waits for it otherwise. Of course, that is the behavior for any thread pool (that is exactly the point of such pattern) but just do not forget about that.
Also I was writing code without Qt in handy so may contain some errors.
Edit: As it turned out, this is not thread safe as sockets are not reentrant in Qt.
What you can do about it is to associate a mutex with a socket and lock it each time you execute its function. This can be done easily creating a wrapper around QTcpSocket class. Please, correct me if I wrong.
I instantiate a PostgreSQL connection through libpqxx. I query the database and get correct response. After that I tried the following error case: after instance of pqxx::connection has been created, I pause my program, manually kill the Postgre's connection process from a Linux's command shell and resume the program. It continues until it tries to create new transaction pqxx::work where it throws pqxx::broken_connection. I handle this exception and try to reconnect with a call to pqxx::connection::activate() but another pqxx::broken_connection gets thrown. How to reconnect to DB without instantiate another pqxx::connection?
P.S. reactivation is not inhibited. I use the standard connection type -
namespace pqxx
{
typedef basic_connection<connect_direct> connection;
}
Ok, nobody has answered. I noticed that after I manually kill the process behind the connection after several successive calls to pqxx::connection::activate, it gets reconnected, so that's my workaround.
class dbconnection : public pqxx::connection
{
public:
dbconnection(std::string options) : pqxx::connection(options) { };
void reconnect()
{
static int times = 0;
try
{
times++;
if(!this->is_open())
{
this->activate();
}
times = 0;
}
catch(const pqxx::broken_connection & e)
{
if(times > 10)
{
times = 0;
return;
}
this->reconnect();
}
};
};
I call dbconnection::reconnect each time after I catch a pqxx::broken_connection. Let me know do you have better solution?
I am writing a multi-threaded application in C++ using Boost threads (pthread). The application spawns 100 threads and each thread does the following task (I am writing a code snippet that will be running in each thread):
try {
driver = get_driver_instance();
con = driver->connect(SettingsClass.HostName, \
SettingsClass.UserName,SettingsClass.Password);
// SettingsClass is a global static class whose members
// (HostName, UserName, Password, etc) are initialized once
// before *any* thread is created.
con->setSchema("MyDatabase");
driver->threadInit();
string dbQuery = "select A, B, C from XYZTable where D=?";
prepStmt = con->prepareStatement(dbQuery);
prepStmt->setInt(1, 1);
rSet = prepStmt->executeQuery();
/* Do Something With rSet, the result set */
delete rSet;
delete prepStmt;
if (con != NULL && !con->isClosed()) {
con -> close();
driver->threadEnd();
delete con;
}
catch (SQLException &e)
{
/* Log Exception */
}
On running the process (the app, as earlier mentioned, i.e. with 100 such threads), I attach gdb midway and observe that more than 40% of the threads have hanged in the read() call. All the backtraces have mysql library functions (vio_read(), etc) and none are from my code as my code does not perform any I/O.
Could anyone point out why is this issue arising. Should I check my code / network or MySQL server configuration? Have I used the C++ connector library properly?
So I'm starting to do some research on alternatives for implementing a high volume client/server system, and I'm currently looking at Poco's Reactor framework since I'm using Poco for so much of my application frameworks now.
The incoming packet sizes are going to be pretty small, so I think it will work fine from the perspective of reading the data from the clients. But the operations that will be performed based on the client input will be relatively expensive and may need to be offloaded to another process or even another server. And the responses sent back to the client will sometimes be fairly large. So obviously I can't block the reactor thread while that is taking place.
So I'm thinking if I just read the data in the reactor event handler and then pass it to another thread(pool) that processes the data, it would work out better.
What I'm not too sure about is the process for sending the responses back to the client when the operations are complete.
I can't find too much information about the best ways to use the framework. But I've done some testing and it looks like the reactor will fire the WritableNotification event repeatedly while the socket is writable. So would the optimal process be to queue up the data that needs to be sent in the object that receives the WritableNotification events and send small chunks each time the event is received?
Update: So when I started testing this I was horrified to discover that server CPU usage went up to 100% on the CPU the server app was running on with a single connection. But after some digging I found what I was doing wrong. I discovered that I don't need to register for WritableNotification events when the service handler is created, I only need to register when I have data to send. Then once all of the data is sent, I should unregister the event handler. This way the reactor doesn't have to keep calling the event handlers over and over when there is nothing to send. Now my CPU usage stays close to 0 even with 100 connections. Whew!
i have wrote a class ServerConnector that copied from SocketConnector, but do not call the connect for socket, because the socket was connected already, if a reactor was started with a ServiceHandler for notifications in the run() function of TcpServerConnection, the class TcpServer would start a new thread. so, i got multithread of reactor-partten, but i do not konw it's best way or not.
class ServerConnector
template <class ServiceHandler>
class ServerConnector
{
public:
explicit ServerConnector(StreamSocket& ss):
_pReactor(0),
_socket(ss)
/// Creates a ServerConnector, using the given Socket.
{
}
ServerConnector(StreamSocket& ss, SocketReactor& reactor):
_pReactor(0),
_socket(ss)
/// Creates an acceptor, using the given ServerSocket.
/// The ServerConnector registers itself with the given SocketReactor.
{
registerConnector(reactor);
onConnect();
}
virtual ~ServerConnector()
/// Destroys the ServerConnector.
{
unregisterConnector();
}
//
// this part is same with SocketConnector
//
private:
ServerConnector();
ServerConnector(const ServerConnector&);
ServerConnector& operator = (const ServerConnector&);
StreamSocket& _socket;
SocketReactor* _pReactor;
};
the Echo-Service is a common ServiceHander
class EchoServiceHandler
{
public:
EchoServiceHandler(StreamSocket& socket, SocketReactor& reactor):
_socket(socket),
_reactor(reactor)
{
_reactor.addEventHandler(_socket, Observer<EchoServiceHandler, ReadableNotification>(*this, &EchoServiceHandler::onReadable));
_reactor.addEventHandler(_socket, Observer<EchoServiceHandler, ErrorNotification>(*this, &EchoServiceHandler::onError));
}
~EchoServiceHandler()
{
_reactor.removeEventHandler(_socket, Observer<EchoServiceHandler, ErrorNotification>(*this, &EchoServiceHandler::onError));
_reactor.removeEventHandler(_socket, Observer<EchoServiceHandler, ReadableNotification>(*this, &EchoServiceHandler::onReadable));
}
void onReadable(ReadableNotification* pNf)
{
pNf->release();
char buffer[4096];
try {
int n = _socket.receiveBytes(buffer, sizeof(buffer));
if (n > 0)
{
_socket.sendBytes(buffer, n);
} else
onError();
} catch( ... ) {
onError();
}
}
void onError(ErrorNotification* pNf)
{
pNf->release();
onError();
}
void onError()
{
_socket.shutdown();
_socket.close();
_reactor.stop();
delete this;
}
private:
StreamSocket _socket;
SocketReactor& _reactor;
};
The EchoReactorConnection works with class TcpServer to run reactor as a thread
class EchoReactorConnection: public TCPServerConnection
{
public:
EchoReactorConnection(const StreamSocket& s): TCPServerConnection(s)
{
}
void run()
{
StreamSocket& ss = socket();
SocketReactor reactor;
ServerConnector<EchoServiceHandler> sc(ss, reactor);
reactor.run();
std::cout << "exit EchoReactorConnection thread" << std::endl;
}
};
cppunit test case is same with TCPServerTest::testMultiConnections, but using EchoReactorConnection for multi-thread.
void TCPServerTest::testMultithreadReactor()
{
ServerSocket svs(0);
TCPServerParams* pParams = new TCPServerParams;
pParams->setMaxThreads(4);
pParams->setMaxQueued(4);
pParams->setThreadIdleTime(100);
TCPServer srv(new TCPServerConnectionFactoryImpl<EchoReactorConnection>(), svs, pParams);
srv.start();
assert (srv.currentConnections() == 0);
assert (srv.currentThreads() == 0);
assert (srv.queuedConnections() == 0);
assert (srv.totalConnections() == 0);
//
// same with TCPServerTest::testMultiConnections()
//
// ....
///
}