Capturing the screen behind the window - c++

I want to write a Windows C++ application where the contents of the window is whatever is behind the window (as if the window is transparent). That is, I want to retrieve the bounding box of my window; capture those coordinates below, and draw them on my window. Therefore it is crucial that I can exclude the window itself during the capture.
"Why not just make the window transparent?" you ask. Because the next step for me is to make modifications to that image. I want to apply some arbitrary filters on it. For example, let's just say that I want to blur that image, so that my window looks like a frosted glass.
I tried to use the magnification API sample at https://code.msdn.microsoft.com/windowsdesktop/Magnification-API-Sample-14269fd2 which actually provides me the screen contents excluding my window. However, re-rendering the image is done in a timer, which causes a very jittery image; and I couldn't figure out how to retrieve and apply arbitrary transformations to that image.
I don't know where to start and really could use some pointers at this point. Sorry if I'm approaching this from a stupid perspective.
Edit: I am adding a mock-up of what I mean:
Edit 2: Just like in the magnification API example, view would be constantly refreshed (as frequently as possible, say every 16 ms just for argument's sake). See Visolve Deflector for an example; although it does not apply any effects on the captured region.
Again, I will be modifying the image data afterwards; therefore I cannot use the Magnification API's kernel matrix support.

You did not specify if this is a one time activity or you need a continuous stream of whats behind your window (like Magnifier/etc). And if continuous, whats the frequency of updates you may need.
Anyway in either case I see two primary use cases:
The contents behind your app are constant: You may not believe, but
most of the time the contents behind your window will not change.
The contents behind your window are changing/animating: this is a
trickier case.
Thus if you can let go the non-constant/animated background usecase, the solution is pretty simple in both one shot and continuous stream cases:
Hide your application window
Take a screenshot, and cache it!
Show your app back (crop everything apart from your application main window's bounding box), and now user can apply the filter
Even if user changes the filter, reapply that to to cached image.
Track your window's WM_MOVE/WM_SIZE and repeat above process for new dimensions.
Additionally if you need to be precise, use SetWindowsHookEx for CBT/etc.
Corner cases from top of my head:
Notify icon/Balloon tool tips
Desktop background scheduling (windows third party app)
Application specific message boxes etc!
Hope this helps!

You can start by modifying MAGCOLOREFFECT . In MagnifierSample.cpp we have:
if (ret)
{
MAGCOLOREFFECT magEffectInvert =
{{ // MagEffectInvert
{ -1.0f, 0.0f, 0.0f, 0.0f, 0.0f },
{ 0.0f, -1.0f, 0.0f, 0.0f, 0.0f },
{ 0.0f, 0.0f, -1.0f, 0.0f, 0.0f },
{ 0.0f, 0.0f, 0.0f, 1.0f, 0.0f },
{ 1.0f, 1.0f, 1.0f, 0.0f, 1.0f }
}};
ret = MagSetColorEffect(hwndMag,&magEffectInvert);
}
Using a Color Matrix to Transform a Single Color.
For more advanced effects, you can blit contents to memory device context.

I've achieved something akin to this using the "GetForeGroundWindow" and "PrintWindow".
It's kind of involved, but here is a picture. The Image updates with it's source, but it's slow, so there is a significant lag (i.e .2 seconds=.5seconds)
Rather than a blur effect I opted for a SinWave effect. Also, using GetForeGroundWindow basically means it can only copy the contents of one window. If you want to hear more just respond and I'll put together some steps and an example repo.

Related

How to separate OpenGL drawing into classes

Say I wanted to draw just a simple OpenGL triangle. I know that I can draw a triangle in the main file, where all my OpenGL stuff is setup by doing something along the lines of:
glBegin( GL_TRIANGLES );
glVertex3f( 0.0f, 1.0f, 0.0f );
glVertex3f( -1.0f,-1.0f, 0.0f );
glVertex3f( 1.0f,-1.0f, 0.0f);
glEnd();
But instead of having all that clutter in my main file, I would like to draw a triangle instead by using a class named "Triangle" with a "Draw" function, so my code would look something like this:
Triangle TheTriangle;
TheTriangle.draw();
In short, how can I make a class with some OpenGL shapes that can be drawed by using a function?
Usual way is as follows:
TriangleArray tri;
tri.push_back(...);
tri.prepare();
while(1) {
clear();
tri.draw();
swapbuffers();
}
But usually the same class should handle array of objects, not just one object. So TriangleArray is good class name. prepare() is for setting up textures or vertex arrays. (Note: if your world is built from cubes, you'll create CubeArray instead.)
Like a few have said, OpenGL doesn't go well with object oriented programming, but that doesn't mean it can't be done. To be a little more theoretical, simply put, you could have a container of "Meshes" in which every frame you loop through and render each to the screen. The Render class could be thought of as a manager of the states, and the container of the various scene modules. In reality, most systems are much more complex than this and implement structures such as the scene graph.
To get started, try creating a mesh class and an object class (that maybe points to a mesh to be drawn) Add functionality to add and remove objects from a container. Every frame, loop through it and render each triangle(or whatever else you want) and there you have a very simple OO architecture. This would be a way to get you started.
Its very normal to find it odd to wrap very functional architecture with OOP but you do get used to it, and if done correctly, it can make your code much more maintainable and scalable. Having said that, the example I gave was quite simple, so here is an architecture that you may want to explore once you have that down.
The following link gives some useful info on what exactly a scene graph is: Chapter 6 covers the scene graph
Its a very powerful architecture that will allow you to partition and order your scenes in a very complex and efficient (if you take advantage of the benefits) manner. There are many other techniques but I find this one to be the overall most powerful for game dev. It can totally depend on what type of application you are seeking to create. Having said all this though, I would not advise making a FULLY object oriented renderer. Depending on your application, an OO scene graph could be enough. Anyways, Good Luck!
You can just put the OpenGL code in the Triangle::draw() function:
void Triangle::draw() {
glBegin( GL_TRIANGLES );
glVertex3f( 0.0f, 1.0f, 0.0f );
glVertex3f( -1.0f,-1.0f, 0.0f );
glVertex3f( 1.0f,-1.0f, 0.0f);
glEnd();
}
Of course, this assumes that you have correctly declared the draw() method in the Triangle class and that you have initialized the OpenGL enviornment.
OpenGL doesn't really map to well into OOP paradigms. It's perfectly possible to implement a object oriented rendering system, but the OpenGL API and many of its lower level concepts are very hard, to impossible to cast into classes.
See this answer to a similar question for details: https://stackoverflow.com/a/12091766/524368

How to draw transparent BMP with GDI+

I'm currently editing some old GDI code to use GDI+ and ran into a problem when it comes to draw a BMP file with transparent background. The old GDI code did not use any obvious extra code to draw the background transparent so I'm wondering how to achieve this using GDI+.
My current code looks like this
HINSTANCE hinstance = GetModuleHandle(NULL);
bmp = Gdiplus::Bitmap::FromResource(hinstance, MAKEINTRESOURCEW(IDB_BMP));
Gdiplus::Graphics graphics(pDC->m_hDC);
graphics.DrawImage(&bmp, posX, posY);
I also tried to create a new bitmap from the resource by using the clone method and by drawing the bitmap to a newly created one but neither did help. Both times I used PixelFormat32bppPARGB.
Then I tried to use alpha blending but this way the whole image gets transparent and not only the background:
Gdiplus::ColorMatrix clrMatrix = {
1.0f, 0.0f, 0.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f, 0.0f, 0.0f,
0.0f, 0.0f, 1.0f, 0.0f, 0.0f,
0.0f, 0.0f, 0.0f, 0.5f, 0.0f,
0.0f, 0.0f, 0.0f, 0.0f, 1.0f
};
Gdiplus::ImageAttributes imgAttr;
imgAttr.SetColorMatrix(&clrMatrix);
graphics.DrawImage(&bmp, destRect, 0, 0, width(), height(), Gdiplus::UnitPixel, &imgAttr);
The transparency information is already contained in the image but I don't have a clue how to apply it when drawing the image. How does one achieve this?
A late answer but:
ImageAttributes imAtt;
imAtt.SetColorKey(Color(255,255,255), Color(255,255,255), ColorAdjustTypeBitmap);
Will make white (255,255,255) transparent on any bitmap you use this image attribute with.
The simplest solution is to use some format other than BMP.
You need the bits to contain alpha data, and you need the Bitmap to be in a format that has alpha data. When you load a BMP with GDI+, it will always use a format without alpha, even if the BMP has an alpha channel. I believe the data is there in the image bits, but it's not being used.
The problem when you clone or draw to a PixelFormat32bppPARGB Bitmap is that GDI+ will convert the image data to the new format, which means discarding the alpha data.
Assuming it's loading the bits correctly, what you need to do is copy the bits over directly to another bitmap with the correct format. You can do this with Bitmap::LockBits and Bitmap::UnlockBits. (Make sure you lock each bitmap with its native pixel format so no conversion is done.)
I had the same problem. Transparent BMPs have not been shown correctly and unfortunately, PNGs cannot be loaded directly from resources (except by adding quite a bit of code which copies them into a stream and loads them from the stream). I wanted to avoid this code.
The bitmaps that I'm using also use only two colours (background and logo). Having an alpha channel means that I would need to save them with a much higher colour depth instead of only 2 bit colour depth.
Evan's answer was exactly was I was looking for :-)
Instead of white, I'm using the colour of the top left pixel as transparent colour:
Gdiplus::Color ColourOfTopLeftPixel;
Gdiplus::Status eStatus = m_pBitmap->GetPixel(0, 0, &ColourOfTopLeftPixel);
_ASSERTE(eStatus == Gdiplus::Ok);
// The following makes every pixel with the same colour as the top left pixel (ColourOfTopLeftPixel) transparent.
Gdiplus::ImageAttributes ImgAtt;
ImgAtt.SetColorKey(ColourOfTopLeftPixel, ColourOfTopLeftPixel, Gdiplus::ColorAdjustTypeBitmap);

glOrtho not working

I have a problem using glOrtho in a program that uses GLMDraw() function of GLM library to draw Google SketchUp 3D images. I wanted to see the image for only certain values of z in Projection mode and glOrtho() didn't seem to work so I made the following code to test it:
glOrtho(0.0f, 2.0f, 0.0f, 2.0f, 0.0f, 0.0f);
Since near and far planes are the same I thought I should see no image but I see the whole image.
What I am missing?
If you call glOrtho with znear=zfar, it generates a GL_INVALID_VALUE error, and probably just discards the call.
http://www.opengl.org/sdk/docs/man/xhtml/glOrtho.xml
Try giving it a range greater than zero.

C++ OpenGL load image in GL_QUAD, glVertex2f

Using WIN32_FIND_DATA and FindFirstFile I'm searching for files in a directory an with fileName.find(".jpg") != std::string::npos I filter the jpg images out.
I'm using OpenGL for creating Boxes with a red color:
glBegin( GL_QUADS );
glColor4f( 1.0f, 0.0f, 0.0f, 0.0f ); glVertex2f( 0.35f, 0.7f );
glColor4f( 1.0f, 0.0f, 0.0f, 0.0f ); glVertex2f( -0.35f, 0.7f );
glColor4f( 1.0f, 0.0f, 0.0f, 0.0f ); glVertex2f( -0.35f, -0.3f );
glColor4f( 1.0f, 0.0f, 0.0f, 0.0f ); glVertex2f( 0.35f, -0.3f );
This is the box in the center with a red color.
My Question is how can I load the Images each in a Cube instead of the Red color (glColor4f)?
I think this is not the best way to make this, but this code is not my own Code, I'm trying to make this better for a friend.
Thank you!
You need to learn about texturing. See NeHe's tutorial on the subject as an example.
However, that tutorial is a bit old (as is your code, since you use glVertex(), so it might not matter to you right now... :).
Anyway, starting from OpenGL 3.1 and OpenGL ES 2.0, you should do it with using GLSL, fragment shaders and samplers instead. See another tutorial for that. It's actually simpler than learning all the fixed function stuff.
It's not really a good practice to use WinAPI together with OpenGL applications unless you really have reasons to - and loading textures from the disk is not a good reason.
Think this way: OpenGL is a platform-independent API, why to dimnish this advantage by using non-portable subroutines when portable alternatives exist and are more convenient to use in most cases?
For the loading textures, I recommend the SOIL library. This is likely to be much better a solution than what the NeHe tutorials recommend.
For finding files on the disk, you might want to use boost::filesystem if you want to get rid of the WinAPI dependency. But that's not a priority now.
When you have the texture loaded by SOIL (a GLuint value being the texture ID), you can do the following:
enable 2D texturing (glEnable(GL_TEXTURE_2D)),
bind the texture as active 2D texture (glBindTexture(GL_TEXTURE_2D,tex);),
set the active color to pure white so that the texture image will be full-bright,
draw the vertices as usual, but for each vertex you'll need to specify a texture coordinate (glTexCoord2f) instead of a color. (0,0) is upper left coord of the texture image, (1,1) is the lower right.
Note that the texture image must have dimensions being powers of two (like 16x16 or 256x512). If you want to use any texture size, switch to a newer OpenGL version which supports GL_TEXTURE_RECTANGLE.
Not really a lot of explaining, as far as the basics are concerned. :)
BTW- +1 for what Marcus said in his answer. You're learning an outdated OpenGL version right now; while you can do a lot of fun things with it, you can do more with at least OpenGL 2 and shaders... and it's usually easier with shaders too.

Difficulties adjusting to OpenGL on the Mac

I know OpenGL itself is a frequently asked question, but I couldn't find a solution to this specific problem I'm having. I've been following NeHe's tutorials, and I've ran into some issues which I don't think should be happening:
When calling glRotatef, where the first parameter being the angle, it appears to be the speed of rotation instead.
Example:
glRotatef(0, 0.0f, 1.0f, 0.0f); // despite the constant numbers, the object rotates infinitely
I am using an NSTimer to loop through the drawing method, which I may think be part of the issue.
Instead of the object rotating 360 degrees around like it should, the object's angle will increment to 180 then decrement back to 0. This is the same with 2D and 3D objects.
I saw example code from Apple and other places that didn't have the same problem as I did, but I was never able to figure out what exactly I am doing wrong that gives me these issues.
The code you have there glRotatef(0,0.0f,1.0f,0.0f); does not change the rotation at all, it simply requests a rotation of 0 degrees around the Y axis. If you want an object to rotate smoothly as time progresses I would suggest the following:
Keep a counter that increments every time your timer triggers, then, before you draw whatever object you are displaying, reset the transformation matrix with glLoadIdentity() and then call glRotatef( counter , 0.0f, 1.0f , 0.0f )