Qt GUI event recording and playback - c++

I'm attempting to implement a simple, lightweight system for recording Qt GUI events and playing them back from a script. I thought this would be fairly straightforward using the magic of Qt's event system, but I'm running into a problem I don't understand.
Here's quick summary of what I'm doing:
RECORDING:
I use QApplication.instance().eventFilter() to capture all GUI events I'm interested in* and save them to a Python script, in which each step looks something like this:
obj = get_named_object('MainWindow.my_menubar')
recorded_event = QMouseEvent(2, PyQt4.QtCore.QPoint(45, 8), 1, Qt.MouseButtons(0x1), Qt.KeyboardModifiers(0x0))
post_event(obj, recorded_event)
PLAYBACK:
I simply execute the script above, in a worker (non-GUI) thread. (I can't use the GUI thread because I want to keep sending scripted events to the application, even if the 'main' eventloop is blocked while a modal dialog eventloop is running.)
The important stuff happens in my post_event() function, which needs to do two things:
First, call QApplication.postEvent(obj, recorded_event)
Wait for all events to finish processing:**
Post a special event to the same eventloop that obj is running in.
When the special event is handled:
Call QApplication.processEvents()
Set a flag that tells the playback thread it's okay to continue
After the second part is complete, my expectation is that all effects of the first part (the recorded event) have completed, since the special event was queued after the recorded event.
The whole system mostly seems to work just fine for mouse events, key events, etc. But I'm having a problem with QAction handlers when I attempt to playback events for my main QMenuBar.
No matter what I try, it seems that I can't force my playback thread to block for the completion of all QAction.triggered handlers that result from clicking on my QMenu items. As far as I can tell, QApplication.processEvents() is returning before the QAction handler is complete.
Is there something special about QMenu widgets or QAction signals that breaks the normal rules for QApplication.postEvent() and/or QApplication.processEvents()? I need a way to block for the completion of my QMenu's QAction handlers.
[*] Not every event is recorded. I only record spontaneous() events, and I also filter out a few other types (e.g. Paint events and ordinary mouse movements).
[**] This is important because the next event in the script might refer to a widget that was created by the previous event.

I think your problem might best be served by using QFuture and QFutureWatcher (that is, if you're using the QtConcurrent namespace for threads, and not QThreads). Basically, the Qt Event handling system does NOT necessarily handle events in the order they're posted. If you need to block until a certain action is completed, and you're doing that action in a separate thread, you can use the QFuture object returned by QtConcurrent::run() with a QFutureWatcher to block until that particular thread finishes its processing.
Something else to consider is the way you handle events. When you use QApplication.postEvent(), the event you create gets added to the receiver's event queue to be handled later. Behind the scenes, Qt can reorder and compress these events to save processor time. I suspect this is more your problem.
In your function which handles playback, consider using QCoreApplication::processEvents(), which will not return until all events have finished processing. Documentation for QCoreApplication is here.

QMenu widgets and QAction signals are a special case. QMenu has an exec() function, normally used for popups. I suspect (but I don't know for sure) that QMenuBar would use this mechanism when it opens a regular pull-down menu. The docs are not clear about this, but Menus act a lot like dialog boxes in that they block all other user activity - how would Qt do this except by giving menus their own event loop? I can't fill in all the blanks from the information in your post, but I don't see how your playback thread would cope with a new event loop.

Related

How to monitor QT signals?

During debugging, I want to see what's awaits my program's event loop.
It's probably flooded, and I want to see by what signals, without (manually) adding specific log-message to every Q_EMIT.
Possible solutions might be watching some internal-qt-data structure that contains the events-queue (Is there such thing? how?)
Or -
Write a log message for every signal emitted (Is that possible?).
Any other ideas?
(QT 4.8 on Windows, using visual studio 2012)
Signals and events are two things that don't have anything to do with each other.
I want to see what's awaits my program's event loop. It's probably flooded.
First of all, let's get the nomenclature straight:
an event queue is where events are stored until delivery;
an event loop is what drains the event queue and delivers events to QObjects,
an event flood happens when, on average, during the delivery of each event there is more than one event added to the queue.
There are two reasons only why an event queue might get flooded:
It takes too long to process some events (e.g. when your code blocks): the drain rate of the queue is lower than the fill rate due to timing.
You're adding more than one event per each event delivered (on average): the fill rate of the queue is higher than the drain rate due to event multiplication - this is completely unrelated to any timing. An apt name for it would be an event storm.
To detect code the blocks for too long, you can use a tool I wrote for another answer.
To know how many events are waiting for any given thread, use the undocumented qGlobalPostedEventsCount(). You add that to the code of the tool linked-to above.
Not sure if this is sufficient for you but you can try installing event filters in between QObjects that implement eventFilter() like this:
class MyWidget : public QWidget
{
QGraphicsView myView;
MyWidget()
{
myView->installEventFilter(this);
// incoming events to myView are shown on the standard output
}
};
You can get more creative with this reading the docs.
The Qt documentation for Events and Filters states:
It is also possible to filter all events for the entire application, by installing an event filter on the QApplication or QCoreApplication object. Such global event filters are called before the object-specific filters. This is very powerful, but it also slows down event delivery of every single event in the entire application.
Therefore, you can create an event filter on the QApplication or QCoreApplication and monitor all events, checking their type.
Alternatively, QCoreApplication uses the virtual notify function to deliver events to objects. Overriding QCoreApplication would allow you to see both the event and QObject to which the event will initially* be delivered.
*Note that events are propagated to parent objects, if the receiving object ignores the event.
If you choose to use notify, be aware of the future direction for this function:
Future direction: This function will not be called for objects that live outside the main thread in Qt 6. Applications that need that functionality should find other solutions for their event inspection needs in the meantime. The change may be extended to the main thread, causing this function to be deprecated.

Are QMenus typically in a thread of their own? What's the right way to wait for them to finish handling events?

In the debugger, I can see that my QMenu objects do not reside in the same thread as the main gui thread. E.g., QApplication.instance().thread() == 0xdeadbeef, but mymenu.thread() == 0xdeadbabe
1) Is that expected? BTW, my QMenus are not native menus (I used setNativeMenuBar(False))
2) I want to block until the menu is finished processing all events. I tried QApplication.processEvents(), but that's a little tricky because it has to be called from the same thread as the the widget I want to wait for (in my case, the menu). I used a custom event filter class in combination with moveToThread() to make sure that processEvents() is called from the same thread that QMenu is running in, but it doesn't seem to matter.
No matter what I try, I can't seem to block for the completion of all QMenu event/action signal handling. My only option seems to be to sleep() long enough that I know the processing is finished. But that's not really acceptable for my use-case.

QT Event Problem

I am writing a qt program and have the following requirement.
When 30 sec passed without any click, lock the screen. If someone clicks again after these 30 secs, redirect him to a login screen.
I have read the qt doc about event and I believe that I need either method 1 or 2 to process mouse event.
1.Installing an event filter on qApp
An event filter on qApp monitors all events sent to all objects in the application.
2.Reimplementing QApplication::notify().
Qt's event loop and sendEvent() call this function to dispatch events. By reimplementing it, you get to see events before anybody else.
They also seems powerful to me, but I don't understand their difference.
Which one suits my requirement? Thank You.
You can basically achieve the same thing with either solution except for the fact that QApplication::notify (or its override) will be called before any event filter that may be on your application.
As the first approach does not require subclassing QApplication, it usually is the preferred one.The only reason to override QApplication::notify in your case would be if you needed to override it due to other reasons anyway, e.g. because you need to do anything related to your own custom events.
But looking at your requirements I would personally go for the following solution:
Install an event filter on qApp
Create a timer with a 30 seconds interval
Connect the timer to the lock screen method
Have your event filter reset the timer every time a mouse press is detected.
Dependent on your application you might also want to look for KeyPress events and maybe MouseMove events as well.

Qt events and signal/slots

In the Qt world, what is the difference of events and signal/slots?
Does one replace the other? Are events an abstraction of signal/slots?
In Qt, signals and events are both implementations of the Observer pattern. They are used in different situations because they have different strengths and weaknesses.
First of all let's define what we mean by 'Qt event' exactly: a virtual function in a Qt class, which you're expected to reimplement in a base class of yours if you want to handle the event. It's related to the Template Method pattern.
Note how I used the word "handle". Indeed, here's a basic difference between the intent of signals and events:
You "handle" events
You "get notified of" signal emissions
The difference is that when you "handle" the event, you take on the responsibility to "respond" with a behavior that is useful outside the class. For example, consider an app that has a button with a number on it. The app needs to let the user focus the button and change the number by pressing the "up" and "down" keyboard keys. Otherwise the button should function like a normal QPushButton (it can be clicked, etc). In Qt this is done by creating your own little reusable "component" (subclass of QPushButton), which reimplements QWidget::keyPressEvent. Pseudocode:
class NumericButton extends QPushButton
private void addToNumber(int value):
// ...
reimplement base.keyPressEvent(QKeyEvent event):
if(event.key == up)
this.addToNumber(1)
else if(event.key == down)
this.addToNumber(-1)
else
base.keyPressEvent(event)
See? This code presents a new abstraction: a widget that acts like a button, but with some extra functionality. We added this functionality very conveniently:
Since we reimplemented a virtual, our implementation automatically became encapsulated in our class. If Qt's designers had made keyPressEvent a signal, we would need to decide whether to inherit QPushButton or just externally connect to the signal. But that would be stupid, since in Qt you're always expected to inherit when writing a widget with a custom behavior (for good reason - reusability/modularity). So by making keyPressEvent an event, they convey their intent that keyPressEvent is just a basic building block of functionality. If it were a signal, it'd look like a user-facing thing, when it's not intended to be.
Since the base-class-implementation of the function is available, we easily implement the Chain-of-responsibility pattern by handling our special cases (up&down keys) and leaving the rest to the base class. You can see this would be nearly impossible if keyPressEvent were a signal.
The design of Qt is well thought out - they made us fall into the pit of success by making it easy to do the right thing and hard to do the wrong thing (by making keyPressEvent an event).
On the other hand, consider the simplest usage of QPushButton - just instantiating it and getting notified when it's clicked:
button = new QPushButton(this)
connect(button, SIGNAL(clicked()), SLOT(sayHello())
This is clearly meant to be done by the user of the class:
if we had to subclass QPushButton every time we want some button to notify us of a click, that would require a lot of subclasses for no good reason! A widget that always shows a "Hello world" messagebox when clicked is useful only in a single case - so it's totally not reusable. Again, we have no choice but to do the right thing - by connecting to it externally.
we may want to connect several slots to clicked() - or connect several signals to sayHello(). With signals there is no fuss. With subclassing you would have to sit down and ponder some class diagrams until you decide on an appropriate design.
Note that one of the places QPushButton emits clicked() is in its mousePressEvent() implementation. That doesn't mean clicked() and mousePressEvent() are interchangable - just that they're related.
So signals and events have different purposes (but are related in that both let you "subscribe" to a notification of something happening).
I don’t like the answers so far. – Let me concentrate on this part of the question:
Are events an abstraction of signal/slots?
Short answer: no. The long answer raises a “better” question: How are signals and events related?
An idle main loop (Qt’s for example) is usually “stuck” in a select() call of the operating system. That call makes the application “sleep”, while it passes a bunch of sockets or files or whatever to the kernel asking for: if something changes on these, let the select() call return. – And the kernel, as the master of the world, knows when that happens.
The result of that select() call could be: new data on the socket connect to X11, a packet to a UDP port we listen on came in, etc. – That stuff is neither a Qt signal, nor a Qt event, and the Qt main loop decides itself if it turns the fresh data into the one, the other or ignores it.
Qt could call a method (or several) like keyPressEvent(), effectively turning it into a Qt event. Or Qt emits a signal, which in effect looks up all functions registered for that signal, and calls them one after the other.
One difference of those two concepts is visible here: a slot has no vote on whether other slots registered to that signal will get called or not. – Events are more like a chain, and the event handler decides if it interrupts that chain or not. Signals look like a star or tree in this respect.
An event can trigger or be entirely turned into a signal (just emit one, and don’t call “super()”). A signal can be turned into an event (call an event handler).
What abstracts what depends on the case: the clicked()-signal abstracts mouse events (a button goes down and up again without too much moving around). Keyboard events are abstractions from lower levels (things like 果 or é are several key strokes on my system).
Maybe the focusInEvent() is an example of the opposite: it could use (and thus abstract) the clicked() signal, but I don’t know if it actually does.
The Qt documentation probably explains it best:
In Qt, events are objects, derived
from the abstract QEvent class, that
represent things that have happened
either within an application or as a
result of outside activity that the
application needs to know about.
Events can be received and handled by
any instance of a QObject subclass,
but they are especially relevant to
widgets. This document describes how
events are delivered and handled in a
typical application.
So events and signal/slots are two parallel mechanisms accomplishing the same things. In general, an event will be generated by an outside entity (for example, keyboard or mouse wheel) and will be delivered through the event loop in QApplication. In general, unless you set up the code, you will not be generating events. You might filter them through QObject::installEventFilter() or handle events in subclassed object by overriding the appropriate functions.
Signals and Slots are much easier to generate and receive and you can connect any two QObject subclasses. They are handled through the Metaclass (have a look at your moc_classname.cpp file for more), but most of the interclass communication that you will produce will probably use signals and slots. Signals can get delivered immediately or deferred via a queue (if you are using threads).
A signal can be generated.
Events are dispatched by the event loop. Each GUI program needs an event loop, whatever you write it Windows or Linux, using Qt, Win32 or any other GUI library. As well each thread has its own event loop. In Qt "GUI Event Loop" (which is the main loop of all Qt applications) is hidden, but you start it calling:
QApplication a(argc, argv);
return a.exec();
Messages OS and other applications send to your program are dispatched as events.
Signals and slots are Qt mechanisms. In the process of compilations using moc (meta-object compiler), they are changed to callback functions.
Event should have one receiver, which should dispatch it. No one else should get that event.
All slots connected to the emitted signal will be executed.
You shouldn't think of Signals as events, because as you can read in the Qt documentation:
When a signal is emitted, the slots connected to it are usually executed immediately, just like a normal function call. When this happens, the signals and
slots mechanism is totally independent
of any GUI event loop.
When you send an event, it must wait for some time until the event loop dispatches all events that came earlier. Because of this, execution of the code after sending event or signal is different. Code following sending an event will be run immediately. With the signals and slots mechanisms it depends on the connection type. Normally it will be executed after all slots. Using Qt::QueuedConnection, it will be executed immediately, just like events. Check all connection types in the Qt documentation.
There is an article that discusses event processing in some detail: http://www.packtpub.com/article/events-and-signals
It discussions the difference between events and signals here:
Events and signals are two parallel mechanisms used to accomplish the
same thing. As a general difference, signals are useful when using a
widget, whereas events are useful when implementing the widget. For
example, when we are using a widget like QPushButton, we are more
interested in its clicked() signal than in the low-level mouse press
or key press events that caused the signal to be emitted. But if we
are implementing the QPushButton class, we are more interested in the
implementation of code for mouse and key events. Also, we usually
handle events but get notified by signal emissions.
This seems to be a common way of talking about it, as the accepted answer uses some of the same phrases.
Note, please see helpful comments below on this answer from Kuba Ober, that make me wonder if it might be a bit simplistic.
TL;DR: Signals and slots are indirect method calls. Events are data structures. So they are quite different animals.
The only time when they come together is when slot calls are made across thread boundaries. The slot call arguments are packed up in a data structure and get sent as an event to the receiving thread's event queue. In the receiving thread, the QObject::event method unpacks the arguments, executes the call, and possibly returns the result if it was a blocking connection.
If we're willing to generalize to oblivion, one could think of events as as a way of invoking the target object's event method. This is an indirect method call, after a fashion - but I don't think it's a helpful way of thinking about it, even if it's a true statement.
'Event processing' by Leow Wee Kheng says:
Jasmine Blanchette says:
The main reason why you would use events rather than standard function calls, or signals and slots, is that events can be used both synchronously and asynchronously (depending on whether you call sendEvent() or postEvents()), whereas calling a function or invoking a slot is always synchronous. Another advantage of events is that they can be filtered.
Events (in a general sense of user/network interaction) are typically handled in Qt with signals/slots, but signals/slots can do plenty of other things.
QEvent and its subclasses are basically just little standardized data packages for the framework to communicate with your code. If you want to pay attention to the mouse in some way, you only have to look at the QMouseEvent API, and the library designers don't have to reinvent the wheel every time you need to figure out what the mouse did in some corner of the Qt API.
It is true that if you're waiting for events (again in the general case) of some sort, your slot will almost certainly accept a QEvent subclass as an argument.
With that said, signals and slots can certainly be used without QEvents, although you'll find that the original impetus for activating a signal will often be some kind of user interaction or other asynchronous activity. Sometimes, however, your code will just reach a point where firing off a certain signal will be the right thing to do. For example, firing off a signal connected to a progress bar during a long process doesn't involve a QEvent up to that point.
Another minor pragmatic consideration: emitting or receiving signals requires inheriting QObject whereas an object of any inheritance can post or send an event (since you invoke QCoreApplication.sendEvent() or postEvent()) This is usually not an issue but: to use signals PyQt strangely requires QObject to be the first super class, and you might not want to rearrange your inheritance order just to be able to send signals.)
In my opinion events are completely redundant and could be thrown out. There is no reason why signals could not be replaced by events or events by signals, except that Qt is already set up as it is. Queued signals are wrapped by events and events could conceivably be wrapped by signals, for example:
connect(this, &MyItem::mouseMove, [this](QMouseEvent*){});
Would replace the convenience mouseMoveEvent() function found in QWidget (but not in QQuickItem anymore) and would handle mouseMove signals that a scene manager would emit for the item. The fact that the signal is emitted on behalf of the item by some outside entity is unimportant and happens quite often in the world of Qt components, even though it is supposedly not allowed (Qt components often circumvent this rule). But Qt is a conglomerate of many different design decisions and pretty much cast in stone for fear of breaking old code (which happens often enough anyway).

QT Repaint/Redraw/Update/Do Something

I'm New to QT. I understand that you can force a display refresh, but I've pulled all my hair out trying to figure out how. Here is what I'm specifically trying to do.
I press a button (onClick signal event), which runs code that changes an image (QLabel) on the display, waits for input, and then proceeds by changing a new image (different QLabel). I've tried everything and the display doesn't refresh until the onclick signal event code is complete. Right now, I'm not waiting for user input, I'm using usleep(~500 ms) for testing purposes.
From what I read, QT is event driven meaning that I'm basically creating a bunch of events, that get put in a que, and executed when the (onClick signal event) returns to the (main loop)/(event handler). I don't want to wait until the function is complete, it's going to make programming extremely painful if I have to accomplish this routine entirely based on events.
How can I force the QLabel pixmap to refresh. I've tried everything. Below is all the code I have tried in my onClick signal event handler. (upButton is the name of the QLabel which is a pixmap)
update();
repaint();
ui->upButton->setUpdatesEnabled(TRUE);
update();
repaint();
QPaintEvent paintevent(ui->upButton->childrenRegion());
QPaintEvent * test = &paintevent;
paintEvent(test);
this->changeEvent(test);
ui->upButton->update();
ui->upButton->repaint();
ui->upButton->repaint(ui->upButton->childrenRegion());
repaint();
QApplication::sendPostedEvents();
this->parentWidget()->update();
usleep(100000);
As you can see, I'm just shooting in the dark at this point. I've tried to look at sample code and do all my homework, but I'm lost. Appreciate any help, advice, and or sample code.
I was using sleep to emulate a brief amount of time the computer was waiting for something to happen.
As I stated in my question, I didn't want to use events because it's a whole lot of unnecessary work to accomplish something extremely simply.
Also, the 'event' that needs to take place for the program to continue, is a USB event. Since I'm using an HID class device, there is no way to set an event to happen without a wait loop. USB HID classes don't permit setting interrupts, the OS claims the device.
I managed to get the above to work. I walked through the debugger and noticed the display would refresh before the sleep function. Running the program independently, I got random results with the display refreshing 1% of the time. I got rid of the sleep function, and added some other code in it's place to emulate a delay, and it was fine.
Just for everyone's knowledge, this is possible, it's not forbidden, and it's easy to do with the following:
qApp->processEvents();
qApp is a global external variable in the QApplication header.
Because this USB event is making my flow tricky, I stumbled upon the QWaitCondition Class. I was going to start a process waiting for the USB event. I would wait until the process releases the wait condition for my routine to continue.
But if anyone thinks this is a bad idea, please, speak out. I really do appreciate your feedback PiedPiper and Hostile Fork.
Thank you.
I noticed sometimes when you have multiple layered widgets, or widgets inside of widgets it helps to call their repaint() events.
For example
this->repaint();
this->parentWidget()->repaint();
this->parentWidget()->parentWidget()->repaint();
This is far easier then pushing out any processing to another Thread, or creating additional event handlers.
You shouldn't be waiting for input in your event handler. You need to rethink the logic of your program to use events the way they were intended. All the update() and repaint() calls in your code are unnecessary if you return to the event loop.
If i understood correctly, you have a slot and in this slot, you update the image shown in a QLabel. But you want this change to be displayed before the slot finishes.
If that is the case, issue an update() event, and call qApp->processEvents(). This method processes events that are waiting in the event queue and then returns, therefore this may be what you are after.
PS: an update() may not be necessary at all, i am not sure.