I am in need of drawing a custom mouse cursor using C++ for my application. I am using a device to track hand movements and translate them into a cursor position on the screen.
However, for this application the system cursor may or may not be displayed, so I need to somehow draw onto the screen. I also need to interact with any window. Because of this, I'm not sure of my course of action - I believe I can create a "transparent" window to draw on while passing clicks/drags through onto the next window but am not sure how to go about this.
EDIT: Clarification - This application might be run on a touchscreen device where the cursor is hidden by some means. I'm trying my best to create this application as "hands-off" of the host system as possible - this program is meant to be a drag and drop application that will just work on a system with none or very very minimal actions done to the settings of the host system itself.
Currently, I have the following which draws a cursor at the location of the "mouse" but I cannot figure out how to refresh the screen or draw over the old "cursor" before drawing the new one. This causes a repeated cursor across the screen:
// Some thread
DWORD WINAPI Cursor(LPVOID lpParam) {
auto dc = GetDC(NULL);
std::optional<POINT> point;
while (true) {
if (point.has_value()) {
auto p = point.value();
RoundRect(......); // Paint over old cursor
}
POINT p;
if (GetCursorPos(&p)) {
point.emplace(p);
RoundRect(......); // Paint new cursor
}
}
I've read that simply calling another RoundRect on the same location will clear it but that's obviously not the case or I'm doing it wrong.
I believe my program can have super privileges if it needs to. Working with WINAPI seems to be kind of a hassle compared to this kind of stuff on linux, regarding permissions of working with different windows and allowing clicks and such to go through....
Any suggestions for how to proceed?
Related
I am trying to deploy a cross platform Qt application written in c++. It works fine on my Ubuntu Linux but when I run it on Windows the application's main window's position gets set on the very top left point of the screen with the upper frame (that holds the minimize, maximize, close buttons) missing.
That is it until i resize the main window (in this case making the width smaller from the right). When this happens the upper frame and the control buttons appear as in the visualization I provided.
Note: I've removed all widgets on the app so they do not аppear as a distraction.
Note2: It appears the maximize button is disabled, which is not the case inside Ubuntu. I have not set any window flags.
How do i visualize the upper frame at the very start of the application without the need to resize the window. I understand its an OS specific behaviour. Setting the main window's geometry with a starting point with a higher y value does NOT help. It still pops at the very top left of the screen.
try to use QWidget::move to set the Window position after setGeometry.
If the widget is a window, the position is that of the widget on the
desktop, including its frame.
You ask a question about cross-platform UI code, and then you don't show the full code. Please show the full code.
The one line of code you do show does something in the wrong way: if you want to maximize a window, call the appropriate function, instead of setting its absolute size that you think shows the window maximized. Windows, their decorations and their placement are very very platform specific, and you should prefer their cross-platform abstractions over trying to do them yourself.
Concretely: the window positioning handles the decorations (title bar) differently on Windows and on Ubuntu. There is absolutely nothing you can do about it except not position your windows absolutely like this.
In the MainWindow constructor at the end this->setGeometry(0, 0, 1336, 600);
That's the problem. setGeometry deals with the geometry of the client area. This is well documented. You should use move to change the position of the widget frame. To set the size of the frame requires the knowledge of the frame's incremental width and height:
bool setWidgetFrameGeometry(QWidget *w, const QRect &r) {
auto frame = w->frameGeometry().size();
auto client = w->size();
auto delta = frame - client;
auto maxDelta = 128;
if (delta.width() > 0 && delta.width() < maxDelta
&& delta.height() > 0 && delta.height() < maxDelta) {
w->move(r.topLeft());
w->resize(r.size() - delta);
return true;
}
return false;
}
The call may need to be deferred to when the event loop had a chance to run:
auto setter = [this]{ return setWidgetFrameGeometry(this, {0,0,1336,600}); };
if (!setter())
QTimer::singleShot(0, this, setter);
I am trying to write a program that takes in the visual output from another program (we'll call it P2) and displays it on a screen in a 3d world.
The flow of the image capture is as follows:
P2 -> BitBlt -> OpenGL Texture -> 3d World
I want the user to be able to look around and click on elements in P2 using a crosshair which would be stationary in the middle of the viewport of my program. However, in my implementation it seems that I would need to have 2 cursors - one to control the camera in my 3d world and one to control P2 - which I don't think is trivial.
Additionally, since a window only updates what part of it is visible and I want to have P2 constantly updating in as high resolution as possible, it makes sense to have P2 on a second virtual desktop. This makes the issue of interaction more difficult.
Some solutions I can think of with their downsides:
Make a Compositing Window Manager
Sounds like a lot of work and I haven't been able to find any documentation on how this might be done.
Have the actual cursor over P2 but get the change in cursor position and use this to move the camera in the 3d world
This wouldn't be a linear transformation from planar to spherical coordinates. Additionally, I'm not sure if the cursor could be on a different Desktop to the current Desktop.
I am open to suggestions of alternative capture methods if they would help.
One example would be hooking the DirectX or OpenGL output of P2 and, if necessary, tricking it into rendering while not being active. This might allow P2 to be minimized but would not solve the issue of input.
Maybe it would be possible to hook the input functions of P2? Would this even be advised?
Here are some images to illustrate my program.
Update:
I have implemented SendNotifyMessage() and found that when I sent a WM_LBUTTONDOWN message to the application it became the foreground window. I set WS_EX_NOACTIVATE on P2 to stop this behaviour however, it still steals focus. I then need to Sleep(), presumably until P2 processes the messages, and then use SetForegroundWindow(). Note that SetFocus() and SetActiveWindow() do not result in my program regaining focus (so maybe focus is the wrong word). Are there any methods to remove this delay while keeping the SendNotifyMessage() asynchronous?
Additionally, in trying to use PostMessage() I found that the coordinate transformation was not correct. However, it worked perfectly in SendNotifyMessage(). What could be causing this behaviour?
Relevant Code
After getting the handle to the window I want to capture I run this:
prevWndStyle = GetWindowLong(hwnd, GWL_EXSTYLE);
SetWindowLong(hwnd, GWL_EXSTYLE, prevWndStyle | WS_EX_NOACTIVATE);
After receiving a WM_INPUT message in my WndProc and processing the input I run this code:
if (raw->data.mouse.usButtonFlags & RI_MOUSE_LEFT_BUTTON_DOWN) {
int hitX, hitY;
if (gls->Click(vec3(0.0f, 0.0f, 1.0f), 0, hitX, hitY)) {
LPARAM lParam = MAKELPARAM(hitX, hitY);
SendNotifyMessage(gls->ic.GetHWND(), WM_MOUSEMOVE, 0, lParam);
SendNotifyMessage(gls->ic.GetHWND(), WM_LBUTTONDOWN, MK_LBUTTON, lParam);
SendNotifyMessage(gls->ic.GetHWND(), WM_LBUTTONUP, 0, lParam);
// I want to get rid of these
Sleep(100);
SetForegroundWindow(gls->aw.GetHWND());
}
}
Update 2
I have found that WS_EX_NOACTIVATE stops the switch to the second desktop however when P2 is on the same desktop, P2 is brought to the foreground.
I think you can use the WIN32API to simulate clicks and also to capture window contents, I've seen it done before in different 3d "window" environments and also I've seen it done in things like Termina Server.
So maybe when "they click you, you pass the click forward" (while doing coordinate transformation)?
Some links for your perusal:
https://msdn.microsoft.com/en-us/library/windows/desktop/gg153548%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
http://www.cplusplus.com/forum/windows/24162/
I wrote my own cameraclass in C++ using DX11.
At the moment I use WM_MOUSEMOVE events to look around in the scene. To prevent the cursur to get out of the window, I call the function SetCursorPos to center the mouse whenever a WM_MOUSEMOVE event occurs.
But if I move my mouse very fast, the cursor gets out of the window.
A solution to that was using the function ClipCursor, but this leads to falter the rotation of the camera when the cursor hits the border of the rect. So ClipCursor solved the original problem, but ended in another.
Do you guys have any solution to that?
For a Windows desktop app, just use ClipCursor giving the rectangle of your window:
RECT rect;
GetClientRect(mWindow, &rect);
POINT ul;
ul.x = rect.left;
ul.y = rect.top;
POINT lr;
lr.x = rect.right;
lr.y = rect.bottom;
MapWindowPoints(mWindow, nullptr, &ul, 1);
MapWindowPoints(mWindow, nullptr, &lr, 1);
rect.left = ul.x;
rect.top = ul.y;
rect.right = lr.x;
rect.bottom = lr.y;
ClipCursor(&rect);
Be sure to have a way to escape this mode so that users can choose to interact with other windows if desired. Typically you call ClipCursor(nullptr); when you go to a 'pause' menu to get out of 'mouse-look' behavior like this.
You can use "raw" input in desktop apps as well, see Taking Advantage of High-Definition Mouse Movement. Keep in mind that raw input works pretty well for relative-movement 'mouse-look' behavior, but (a) it won't work over remote desktop, and (b) you won't get 'pointer-ballistics' which is a non-linear movement rate that makes a mouse more agile so you should generally stick with traditional WM_MOUSE messages when dealing with absolute-movement.
For Universal Windows Platform, you can't use "raw" input as there's no equivalent to WM_INPUT, but you do get high-precision data out of the MouseDevice.MouseMoved event via the MouseDelta property. You don't need to use ClipCursor for relative movement in UWP, just turning off the cursor by setting the CoreWindow.PointerCursor property to nullptr will prevent the system mouse position from being tracked. Again, you should restore the system cursor when you are in a 'pause' menu. See Developing mouse controls (DirectX and C++).
See DirectX Tool Kit Mouse helper class and more importantly the implementation file. For relative mode, it makes use of both ClipCursor and WM_INPUT for the Windows desktop Win32 implementation.
The regular windows message are not the best solution to drive precise controls. They inherit from the OS acceleration system, clipping and depends on other shenanigans as you can see.
The best API to receive mouse inputs is Raw Input. It has the advantage to expose the better dpi and polling rate the hardware can provide and is free of any under the hood manipulation. Once you read the mouse with this, you are free to use SetCapture and clipCursor to prevent unwanted click to an other window.
You can find the documentation here : https://msdn.microsoft.com/en-us/library/windows/desktop/ms645536(v=vs.85).aspx
I am coding little fun gadget. I want to be able to draw second (or more) mouse pointer icons at different location than the original mouse but to move it according to move of original mouse.
I know how to track movement of the mouse but I dunno how to draw/redraw mouse pointer; can anyone help?
You can use the following code:
CURSORINFO ci;
ci.cbSize = sizeof(CURSORINFO);
GetCursorInfo(&ci);
Next you can draw a cursor by calling:
DrawIcon(ContextDC, YourXPosition, YourYPosition, ci.hCursor);
If you need additional information about the cursor, like hotspot for example, check the ICONINFO structure:
ICONINFO ii;
GetIconInfo(ci.hCursor, &ii);
This could be done like:
grab the current mouse cursor from your application, using LoadCursor(). Just specify NULL, and the cursor you want. Or just load a bitmap for the cursor. Now, you have a bitmap.
Next step is to get the Device context of your Desktop: GetWindowDC(NULL). This will give you the opportunity to draw on the desktop anywhere.
There is a huge chance that you will need to apply CreateCompatibleBitmap() to the Image at #1 with the DC obtained at #2.
Now, use some BitBlt() to copy bits OUT from the DC obtained at #2 into a save image (YOU will need to create these) from the position you want to put your cursor.
Now, put the image obtained at #3 onto the DC of the Desktop obtained at #2 at the position you want.
When the user moved the mouse restore the image on the desktop with the saved data at #4. Release all the stuff you don't need (yes, this is mandatory).
And restart from #1.
These two more links might help:
Bitmaps, Device Contexts and BitBlt
Capturing an Image
Good luck!
I want to have a red line instead of mouse pointer in my (written in C++ with OpenGL) application. For example when I move the mouse over an OpenGL window, I would like the cursor to become a red line. How can I do that?
Also, how can I call the mouse related functions (OpenGL) for that line and specify the functionality?
As Nicol Bolas said in his comment, OpenGL knows nothing of the mouse. You'll need to interact with the windowing system one way or another (via direct Win32/X11/etc. API or via a portable windowing library a la Qt, wxWidgets, etc) to monitor mouse position.
If the cursor you're trying to draw is a bitmap, your best bet is likely to handle mouse enter/leave events sent to your window and respond to them by using an API function to change the cursor. This will handle automatically updating the cursor as the mouse moves around and will add minimal overhead to your application (windowing system cursors generally draw in an overlay plane that avoids sending redraw events to your window every time the mouse moves).
If you have a more procedural description of your cursor (that is, you intend to draw it with OpenGL drawing commands), you'll instead want to handle the mouse enter/leave events by using HideCursor()/ShowCursor() commands or the equivalent to turn off the windowing system's native cursor when the mouse is over your window. Then, you'll hook the mouse move callbacks and redraw your scene, adding whatever commands you need to draw the cursor at the position specified in the mouse move event.
The first approach is definitely preferred for performance & latency reasons--but there are some cursor types (think full screen crosshairs) that can't be accomodated that way.