I have been looking at the DXGI Desktop Duplication Sample on msdn, which looks very nice but it isn't clear that it will work for what I want to do.
What I would like to do is to use the Win32 Desktop api's to create a new desktop (CreateDesktop) then switch to that desktop (SwitchDesktop) and then in that new desktop I would like to create a window and render a live preview of the original desktop.
Despite the name the DXGI api's appear to be more related to "screens" not Desktops, in the Win32 sense. Before I invest a lot of time attempting to implement this only to find out that it doesn't work I was hoping someone more knowledgeable of these api's could chime in and steer me in the right direction.
Previous attempts to do this with other common screen capture methods have resulted in failure as the images are always black. I am hoping DXGI will allow me to render non-active desktops into a DirectX texture which I can then display in my app.
I think the answer to this is that you can't do it. I have since abandoned this approach and now believe that the RDP api's are what I actually need to accomplish this. I think that in DXGI when they say "Desktop" they actually mean "Screen".
I haven't been able to figure out how to do this with the RDP api's yet either but I believe it is possible.
Related
For Imageprocessing I want to get all pixel information from a given process.
Concrete its for testing an image hashing algorithm for identifying hearthstone cards, so i need to get a screenshot of the given process.
How can I solve it in windows?
My idea so far:
Get the process name.
Get the process ID
Get Window Handle
I have no idea how to go further from this point.
I hope it understandable what I want to achieve.
Unfortunately, there is no general method for getting the pixels of a particular window that I would be aware of. Depending on how the target application draws itself, this task can be very simple or very complicated. If we were talking about an application that uses good old GDI, then you could just get yourself an HDC to the window via GetWindowDC() and BitBlt/StretchBlt the content over into a bitmap of your own.
Unfortunately, the target application in your case appears to be a game. Games typically use 3D graphics APIs like Direct3D or OpenGL for drawing. Assuming that you cannot simply modify the target application to just send the desired data over to you out of its own free will, the only way to specifically record output from such applications that I'm aware of is to hook into the graphics API and capture the data from underneath the API. This can be done. However, implementing such a system is quite involved. There might be existing libraries to aid with writing such applications, but I don't know any that I could recommend here. If you don't have to capture the game content in real-time, you could just use a screen recording application to, e.g., record a video and then use that video as input for your algorithm. There are also graphics debugging tools like NSight Graphics or RenderDoc that you could use. Be aware that games, particularly online games, these days often have cheat protection systems that are likely to get very angry at you if you attempt to hook into the game…
Apart from all that, one alternative approach might be to use DXGI Output Duplication to just capture the entire desktop. While you won't be able to target one specific application (as far as I know), this would potentially have several advantages: First of all, it's only moderately complex to set up compared to a fully-fledged API-hook-based approach. Second, it should work regardless of what API the target application uses and even if the application is in fullscreen mode. Third, since you will have the data delivered straight from the operating system, you shouldn't have any issues with cheat protection. You can use MonitorFromWindow() to get the monitor your target window appears on and then enumerate all outputs of all DXGI adapters to find the one that corresponds to that HMONITOR…
So I am trying to figure out how get a video feed (or screenshot feed if I must) of the Desktop using OpenGL in Windows and display that in a 3D environment. I plan to integrate this with ARToolkit to make essentially a virtual screen. The only issue is that I have tried manually getting the pixels in OpenGl, but I have been unable to properly display them in a 3D environment?
I apologize in advance that I do not have minimum runnable code, but due to all the dependencies and whatnot trying to get an ARToolkit code running would be far from minimal. How would I capture the desktop on Windows and display it in ARToolkit?
BONUS: If you can grab each desktop from the 'virtual' desktops in Windows 10, that would be an excellent bonus!
Alternative: If you know another AR library that renders differently, or allows me to achieve the same effect, I would be grateful.
There are 2 different problems here:
a) Make an augmentation that plays video
b) Stream the desktop to somewhere else
For playing video on an augmentation you basically need to have a texture that gets updated on each frame. I recall that ARToolkit for Unity has an example that plays video.However.
Streaming the desktop to the other device is a problem of its own. There are tools that do screen recording, but you probably don't want that.
It sounds to me that what you want to do it to make a VLC viewer and put that into an augmentation. If I am correct, I suggest you to start by looking at existing open source VLC viewers.
I would like to have directcomposition render to a texture. Is this possible?
The reason for this is that I would like to be able to render a gpu accelerated windowless transparent flash player activex control to a texture. Something that is usually not possible, but which I hope to achieve with DirectComposition.
It's unlikely that this is possible, to quote MSDN (emphasis mine)
DirectComposition does not offer any rasterization services. An application must use some other software-based or hardware-accelerated rasterization library such as Direct2D or Direct3D to populate the bitmaps that are to be composed. After composing, DirectComposition passes composed bitmap content to Desktop Window Manager (DWM) for rendering to the screen.
As far as I know there are only official APIs to share your offscreen surfaces with DWM, but no API allowing you to get read-access to a DWM surface.
What DWM does allow you is redirecting HWND surfaces, so you can display the surfaces of other HWNDs on your window. This can be done either through DirectComposition (via CreateSurfaceFromHwnd) or the DWM API (via DwmRegisterThumbnail). For an example of the latter look here.
If you want to go the "hacking route" as indicated in your comment, there are undocumented APIs which look like they can give you access to the DWM surfaces, in particular DwmpDxGetWindowSharedSurface sounds promising. Someone else already did some reverse engineering and figured out the signature, but couldn't get it to work (texture works but renders black). This guy seems to have had more luck and was able to render window textures in 3d. I don't understand his language but you seem to have to use DwmpDxUpdateWindowSharedSurface (also undocumented).
You should be aware however that using undocumented functions is not a good idea, Microsoft can change them anytime (even in service pack releases) or remove them completely, since they are only used by Microsoft themselves they have no reason to maintain compatibility. Also there is a good chance that you are going to use them wrong (e.g. you might be missing necessary synchronization and cause random crashes, or worse).
However since the functionality is actually available there is hope that Microsoft may actually open it for puplic use in some future version of Windows.
let me first specify my development essentials. I am writing an Windows DLL. The programming language i do focus on is C/C++. Asm blocks are possible aswell when required for my task. Maybe even a driver, but i do not have any experience with them at all.
The DLL is being injected into a host process. That's always a Directx environment. Either Dx9, Dx10 or Dx11 and may run in fullscreen or windowed mode.
The method should support windows xp up to windows 7 and is being compiled in x86 only.
The goal is to come up with a function taking a screenshot of a given process-window. The screenshot is never being taken from the host process itself. Its always another process! The window may contain directx or gdi32 content. Maybe other contents are possible i do not think of at the moment (windows forms comes to my mind. i am not sure how that is being rendered internally). The windows may be minimized.
That screenshot needs to be accessable/convertable to an directx texture such as Texture2D, depending on the Directx environment i am working in. Saving the screenshot as an png/bmp is enough thoe, as i do know how to create such a texture from memory.
I've already tried the oldstyle BitBlt way, that didnt work on minimized applications thoe. The minimized applications are being drawn, when i send WM_PAINT messages to the targeting window. That aint a solution for me, as i also need to keep up with directx applications which doesnt react to such messages.
Maybe i need to hook each single DirectX window to accomblish my task, to access the backbuffer directly, i do hope for some better methods anyways.
For the reason that i do take a lot of screenshots from multiple windows, i would like to implement a fast method, which isnt such a cpu bogus. Copying from VideoRAM may be a bad way to go when having such performance needs.
I do hope for some ideas, maybe code samples as i am not familar with all the possibilities i could go for. I've looked at some windows thumbnail api, but that didnt support xp from what i could read.
Thanks in advance,
Frank
Simple example - on one side we see camera rendered via standard software rendered "Input" on other hand (labeled "Output") rendered via some directX stuff (at least it seems to me) :
So what function is provided by windows api or DirectX api for capturing such mixed scenes?
TightVNC Server can do it, you may want to look into what they are doing. From a simple glance through their source code it looks like they are creating a virtual screen that mirrors the primary screen.
Specifically though, look into the
CreateCompatibleDC and CreateDIBSection API's
As I known, there is not a direct way to capture DirectX render area, although we can see that on the screen. Because the real render action(aka render instruction) happens in hardware layer. So the API in standard SDK cannot know the finally render result, which lead to the black square.
The only way to do this maybe put your hope on the Render layer(such as DirectX engine) itself can support output interface as well as underlying render action. So I suggest to check some documentation to find if there indeed is.
DirectX can present to a limited subsection of the window that you give it, enabling you to create small regions of DX content in larger windows.