I made a small program in C++ that can type text into a notepad document, it does so using FindWindow and FindWindowEx, but I was only able to do that because I knew the names of the windows I was trying to access (in this case it was "Notepad" for the doc and "Edit" for the text area).
If I didn't know the names how would I go about figuring that out? Lets hypothetically say I wanted to make text appear in Chrome's URL box, how would I figure out what the name of that box is so I can use FindWindowEx on it?
The least painful approach to automating 3rd party applications is to use UI Automation. It allows you to navigate UI hierarchies as well as operate on the individual objects.
All native windows controls support UI Automation out of the box. UI Automation also works with frameworks, that do not implement their UI using native controls (e.g. Qt). Particularly with Qt, tools like Spy++ will not give you the information you need.
Run the other app and then use a tool like Spy++, Winspector, etc to look at the other program's window hierarchy to find what you need.
Related
I just started developing an MFC application for the first time and I'm hoping to get more familiar with the whole "controls" concept. I am using the dialog editor in visual studio and so far I was not able to find functionality to add a simple table/grid. It seems quite basic to me, but I can't even find reliable information on how to do it on the internet.
I am looking for something similar to Qt's QTableWidget, something that I can later program with variable amount of rows and columns tailored to my application's use cases.
Do you have any ideas how to do it?
I use CGridCtrl which is very powerful and does a lot of the legwork for you.
Sounds like you're after a List View Control, which is wrapped by MFC's CListCtrl class. The dialog editor will enable you to add one and set its properties.
I'm making a GUI application in C++.
Is there a simple way to make the narrator of windows 10 to speak some text ?
Given of course that it is currently active.
A trick that sometimes works is to select some text in a text field and then focus it briefly. But
It doesn't work all the time
The focus is moved out and back. Even if it's for a short time, it's invasive and may disturb whatever the user is currently doing
The text field must be present at some place on screen, what is not always desired
If possible, I would like a solution without these three issues.
Other screen readers, in particular Jaws and NVDA, provide API to do this.
I'm even the author of a library, UniversalSpeech, which allow to make the currently running screen reader, if any, to speak text, abstracting the need to detect yourself which one is running.
Given that the narrator has greatly improved with the last 3 or 4 releases of windows 10, it would probably be interesting to support it, not only in my own program and for my particular usecase, but for everybody in my library.
However, I can't find any documentation or anything telling me if the narrator has an API similar to those of Jaws or NVDA.
In fact if there is currently no such API for Narrator, it would probably be interesting to suggest Microsoft to add one.
Note that this question is different from such as this one
where the answer suggests to use speech API directly. Using screen readers API and not speech API directly has great benefits:
Screen reader users are used to specific voice settings (voice, rate, pitch, language and regional accents, etc.). The default settings set in the control panel may not at all be similar. It implies
whether the user must configure speech settings in the control panel, what is global for all applications; not very good
and/or managing application-specific speech settings in a business application which isn't at all devoted to speech stuff; it would be rather strange to find speech settings in a financial app for example.
Using both screen reader and independent speech engine simultenously means that both can speak at the same time, what is of course extremely annoying. In fact in practice it happens quite often, I have already tested.
So, is there a simple way to make narrator to speak some text ?
My program is in C++, the library is in C, so in theory I have access to the whole winapi, through LoadLibrary/GetProcAddress if needed.
Please don't give any C# or VisualStudio-dependent solution.
Thank you.
After quite a while, I answer my own question !
IN fact, the simplest to make narrator speak something is probably to:
Define some label as being a live region
When something has to be spoken by narrator, change the text in the label and then send an update notification
Turning a label into a live region and sending a notification whenever the text changes is explained here:
https://learn.microsoft.com/en-us/accessibility-tools-docs/items/win32/text_livesetting
Live setting can be set to 0=off, 1=polite and 2=assertive. The meaning of polite and assertive are the same as in WAI ARIA.
Though as present (april 2021), narrator always interrupts speech as soon as the text of the label is replaced, even in polite mode.
What changes in assertive mode is that the text even take priority against interruptions due to normal keyboard navigation, i.e. you may not hear where you are when pressing tab, arrow keys, etc.
For that reason, I don't recommand it at all.
Note also that live setting only works on static text controls (win32 STATIC window class).
It is totally ignored when applied to text fields and text areas (win32 EDIT window class).
The label with live setting still works when placed off-screen or even hidden.
As far as I know, Microsoft Narrator doesn't expose the API for developers, and you can suggest a feature for it using Feedback App on Windows 10.
I want to achieve the same behavior as in a live region, i.e. have some text read by the SR as it appears at the bottom of a multiline rich text field, but in a native GUI app. For info, I'm using WXWidgets.
You can use the UI automation events to subscribe the property change of rich text field so that you can get notified when the text is updated. And since you are using third-party control, you also need to implement providers for any third party controls that do not include a provider. And below are links about UI automation provides and
UI Automation events for your reference:
UI Automation Events Overview
UI Automation Providers Overview
I was looking into trying to get my C++ application to do the following:
Open internet explorer
Open a webpage
Enter a name and password into specific forms in the webpage
Click the submit button on the webpage to open a new page
From my searching on the Internet it seems like using COM may make this possible, although I may be incorrect on that. I am doing my best to learn COM at the moment but some help would be great. I'm looking to do this without using MFC.
I have noticed this question which I kind of what I am looking for but I am having trouble understanding the suggested solutions. For example, I do not have a IWebBrowser2 option in my toolbox.
EDIT:
To make my question clearer, I had this task complete in a C# version by simply running a coded UI test but this will not work with C++. I am looking to open IE (not in the application itself), find the username and password forms, pass them string, then find the submit button on the page and click it.
This is very possible from c++. You will have to dive into the winapi to do some Keystroke stuff as well as window handling.
I'm not going to go into all of the code, but you have to do something like the following:
Start ie (if you give it a command line arg with the webpage, it will
open that page).
Make sure the ie window is focused (either just wait
if you want to keep it simple or use window's api to go through each
open HANDLE and find the window you want)
Use SendInput to send an Alt + D (to gain focus to the url bar, in firefox it will be a CTRL + L instead)
Use SendInput and javascript injection to modify the DOM as necessary
You can also submit the form (after everything is as you want it) using the above JS injection capability.
Yes, it is possible, but you have to embed a web browser control in your application, and it is not straightforward (I don't think you can automate DHTML in an external instance of Internet Explorer via COM).
I see that you don't want to use MFC, and this complicates even more the problem. Perhaps you can do it via ATL, I advise against even trying to do it without a framework.
If you could use MFC, then you could use a CDHtmlDialog form and access the underlying COM interfaces to automate the actions.
In the past, I developed an MFC application that used HTML as its user interface, but I used the CDHTMLView class that you can find here: http://www.codeproject.com/Articles/1783/Integrating-DHTML-into-MFC-Views
You can use this as an entry point for learning how to deal with DHTML events and how to play around with the IWebBrowser2 interface.
You should really take a look at WebDriver which is able to do exactly what you are describing. See (http://code.google.com/p/selenium/wiki/InternetExplorerDriverInternals) for more information about the InternetExplorerDriver internals. Even if you are not able to use the project directly, you can certainly browse the source to get a better idea of how what you want to do can be achieved.
What you want to do makes not much sense.
There are many APIs available to embed a browser view into your program. For example Qt offers this.
Then you can just do your HTTP POST request yourself and display the answer you get in your browser view.
That is a much cleaner solution.
Pro tip: Don't use Internet Explorer.
Can I use this search box control seen in Windows Explorer in my own Qt/C++ application? Or is it a custom control I'd have to implement manually?
I haven't seen this kind of control, however in Qt it's easy to implement it manually. You may follow this tutorial: http://labs.qt.nokia.com/2007/06/06/lineedit-with-a-clear-button/ and just replace a clear icon with a search one (and change slot from clear() to something which performs desired search, of course).
You can use the Windows Search Service to access the search functionality; I'm not sure you can embed the actual control that Explorer uses in your own app, but you could certainly create your own that works the same way.
What is the best way for, using WinApi, find a given control in a external program window?
For example, I'm trying to change Internet Explorer's url text box. I am having trouble getting programatically the handle to the text box. I know its type is "Edit" but I'd like to avoid having to search through all the child windows for the "Edit" control (that's how I'm currently doing).
Is there any kind of unique identifier for a given control on a window? I tried using "Control ID" but it doesn't seem to be working.
Thanks
When you're delving into the windows of another application that wasn't designed to give you any particular special access to its windows, then you don't really have any simple solution. Functions like FindWindowEx, GetWindow, EnumChildWindows, and the rest are what you have to work with.
However, it's often not a great idea to even do this. Internet Explorer may have certain types of windows in a certain hierarchy in the particular version that you're developing against right now. But those windows and hierarchy may be different in previous versions and could be considerably different in future versions. You have no guarantee about these things.
In some cases, you might do well to investigate if there are alternative and more official ways to control the other program. For instance, Internet Explorer exposes a COM object that can be used for many purposes. Because this is an official interface, you have better guarantees about what previous versions this will be supported on and that it won't break for future versions.
The best way to do it is find it step by step.. E.g. Find the IE window with FindWindow, then find the child of that with FindWindowEx, then find the child of that with FindWindowEx ... until you get down to the textbox.
There is 1 program I can think of that will generate VB code from dragging a icon from the application to any part of any other application.. VB is way old but it'll give you a very good idea how to do it!
It's called API Spy, found under 'Downloadable Applications (Windows Only)' on http://patorjk.com/blog/software/