I would like to make a program to capture video.
What is the best way to capture video?
I know C++ and I'm learning assembly. I found in my assembly book that I can get data from the video card. Would that be the best way?
I know FRAPS hooks into programs, but I would like my program to take video of the entire screen.
I would like something something fast, with low memory usage if possible. A requirement is that the program must be usable on other computers, despite dissimilar hardware.
The way Fraps works, it's impossible to capture the entire screen (unless you're running a full-screen DirectX application, of course). You're apparently trying to emulate the functionality of CamStudio, more so than Fraps.
CamStudio is open-source (here is the SorceForge page) so perhaps you could start by studying the source code? I would wager that it's not really for beginners, however.
Capturing an entire screen is simple, in short you get a desktop handle (GetWindowHandle(0)) and BitBlt() it to your bitmap.
Now you need to encode it to video, potentially full HD or more, in real time, using the best possible compression, ideally lossless because of text on the screen and vector graphics nature of traditional desktops. I don't know any good custom codec for such requirements so would recommend to use traditional h.264 and tune tradeoff between quality and performance. FFMPEG is probably the most popular library for this, just check license of h.264 encoding.
Related
I'm trying to write an application that records and saves the screen in C++ on the windows platform. I'm not sure where to start with this. I assume I need some sort of API, (FFMPEG, maybe OpenGL?). Could someone point me in the right direction?
You could start by looking at Windows remote desktop protocol, maybe some programming libraries are provided for that.
I know of a product that intercepts calls into the Windows GDI dll and uses that to store the screen drawing activities.
A far more simpler approach would be to do screenshots as often as possible and somehow minimize redundant data (parts of the screen that didn't change between frames).
If the desired output of your app is a video file (like mpeg) you are probably better off just grabbing frames and feeding them into a video encoder. I don't know how fast the encoders are these days. Ffmpeg would be a good place to start.
If the encoder turns out not fast enough, you can try storing the frames and encoding the video file afterwards. Consecutive frames should have many matching pixels, so you could use that to reduce the amount of data stored.
Well, I want to know.. maybe others too.
Is it possible to control each pixel separately on a screen by programming, especially C or C++?
Do you need special control over the drivers for the current screen? Are there operating systems which allow you to change pixels (for example draw a message/overlay on top of everything)?
Or does windows support this maybe in it's WinApi?
Edit:
I am asking this question because I want to make my computer warn me when I'm gaming and my processor gets too hot. I mainly use Windows but I have a dual boot ubuntu distro.
The lower you go, the more hassle you'll run into.
If you want raw pixel manipulation you might check out http://www.libsdl.org/ which helps you mitigate the hassle of creating surfaces/windows and that kind of stuff.
Linux has a few means to get you even lower if you want (ie without "windows" or "xwindows" or anything of the sort, just the raw screen), look in to the Linux Frame Buffer if you're interested in that.
Delving even lower (such as doing things with your own OS), the BIOS will let you go into certain video modes, this is what OS installers tend to use (at least they used to, some of the fancier ones don't anymore). This isn't the fastest way of doing things, but can get you into the realm of showing pixels in a few assembly instructions.
And of course if you wanted to do your own OS and take advantage of the video card (bypass the BIOS), you're then talking about writing video drivers and such, which is obviously a substantial amount of work :)
Re overlay messages ontop of the screen and that sort of thing, windows does support that sort of thing, so I'm sure you can do it with the WinAPI, although there are likely libraries that would make that easier. I do know you don't need to delve too deep to do that sort of thing though.
Let's look at each bit at a time:
Is it possible to control each pixel separately on a screen by
programming, especially C or C++?
Possibly. It really depends on the graphics architecture, and in many modern systems, the actual screen surface (that is "the bunch of pixels appearing on the screen") is not directly under software control - at least not from "usermode" (that is, from an application that you or I can write - you need to write driver code, and you need to co-operate sufficiently with the existing graphics driver).
It is generally accepted that drawing the data into an off-screen buffer and using a BitBlt [BitBlockTransfer] function to copy the content onto the screen is the prefferred way to do this sort of thing.
So, in reality, you probably can't manipulate each pixel ON the screen - but you may be able to appear like you do.
Do you need special control over the drivers for the current screen?
Assuming you could get direct access to the screen memory, your code certainly will have to have cooperation with the driver - otherwise, who's to say that what you want to appear on the screen doesn't get overwritten by something else [e.g. you want full screen access, and the clock-updater updates the time on screen every once a minute on top of what you draw, etc].
You may be able to set the driver into a mode where you have a "hole" that allows you to access the screen memory as a big "framebuffer". I don't think there's an easy way to do this in Windows. I don't remember one from back in 2003-2005 when I wrote graphics drivers for a living.
Are there operating systems which allow you to change pixels (for
example draw a message/overlay on top of everything)?
It is absolutely possible to create an overlay layer in the hardware of modern graphics cards. That's generally how video playback works - the video is played into a piece of framebuffer memory that is overlaid on top of the other graphics. You need help from the driver, and this is definitely available in the Windows API, via DirectX as far as I remember.
Or does windows support this maybe in it's WinApi?
Probably, but to answer precisely, we need to understand better what you are looking to do.
Edit: In your particular use-case, I would have thought that making sounds or ejecting the CD/DVD drive may be a more suitable opton. It can be hard to overlay something on top of the graphics drawn by a game, because games often try to use as much as possible of the available graphics resource, and you will probably have a hard time finding a way that works even for the most simple use-cases - never mind something that works for multiple different categories of games using different drawing/engine/graphics libraries. I'm also not entirely sure it's anything to worry overly about, since modern CPU's are pretty tolerant to overheating, so the CPU will just slow down, possibly grind to a halt, but it will not break - even if you take the heatsink off, it won't go wrong [no, I don't suggest you try this!]
Every platform supports efficient raw pixel block transfer "aka BitBlt()", so if you really want to go to frame buffer level you can allocate a bitmap and use pointers to set its contents directly then with one line of code efficiently flip this memory chunk into video ram buffer. Of course it is not as efficient as working with PCI framebuffers directly, but on the other hand this approach (BitBlt) was fast enough even in Win95 days to port Wolfenstein 3d on Pentium CPU WITHOUT the use of WinG.
HOWEVER, a care must be taken while creating this bitmap to match its format (i.e. RGB 16 bits, or 32 bits etc...) with actual mode that device is in, otherwise the graphics sub-system will do a lengthy recoding/dithering which will completely kill your speed.
So depending on your goals, If you want a 3d game your performance will suck with this approach. If you want just to render some shapes and dont need more than 10-15fps - this will work without diving into any device-driver levels.
Here is a few tips for overlaying in Windows:
hdc = GetDC(0);//returns hdc for the whole screen and is VERY fast
You can take HDC for screen and do a BItBlt(hdc, ..... SRCCOPY) to flip blocks of raster efficiently. There are also pre-defined Windows Handles for desktop but I dont recall the exact mechanics but if you are on multiple monitors you can get HDC for each desktop, look at "GetDesktopWindow", "GetDC" and the like...
I am trying to play a .flv file in the GLUT window using OpenGL and C++ in Linux, but I'm not sure where to start.
Is it possible to do this? If so, how?
Make sure you mean .flv not .swf.
It's quite easy. Decode the video with something like libavcodec and you can use raw frames as textures.
If you really want to do this, check out the source code of Gnash. They've a renderer that use OpenGL. However, rendering is just a small part of the job, you also have to decode audio/video, run actionscript, etc.. in order to run a flash file.
It so complicated that even Adobe didn't manage to make it right :)
If you want to play just some video, look at #Banthar's answer, otherwise:
OpenGL is a no-frills drawing API. It gives you the computer equivalent of "pens and brushes" to draw on some framebuffer. Period. No higher level functionality than that.
Flash it a really complex thing. It consists of a vector geometry object system, a script engine (ActionScript), provides sound and video de-/compression etc. All this must be supported by a SWF player. ATM there's only one fully featured SWF player and that's the one Adobe makes. There are free alternatives, but the are behind the official flash players by several major versions (Lightspark, Gnash).
So the most viable thing to do was loading the Flash player browser plugin in your program through the plugin interface, provide it, what a browser provided to a plugin (DOM, HTTP transport, etc.) and have the plugin render to a offscreen buffer which you then copy to OpenGL context. But that's not very efficient.
TL;DR: Complicated as sh**, and probably not worth the effort.
What should I use to perform screen capture on Windows for subsequent image processing?
I seek to do follow-up image processing in OpenCV.
Well the most straightforward thing to do is to use an off the shelf video capture tool to create an AVI file and then have image processing software operate on that, after the fact.
To get up and running:
CamStudio is free and open source and has a simple gui.
VirtualDub is also FOSS and is more powerful, but less intuitive to use. Its primarily a video editing and processing tool, but it actually has sophisiticated capture capabilities.
Both work on Windows and both can output uncompressed AVI files that OpenCV can read.
If you are completely new to OpenCV, then I recommend a O'Reilly's "Learning OpenCV". Its for the older OpenCV 1.1 but it will at least get you started.
If you crack open that book, and you find that its way above your head, then I would consider trying to do your image processing in a higher level language. MATLAB with the Image Processing Toolbox is well suited for rapid prototyping of image processing and its a much more forgiving development environment. Its an interpretative language, so you can see-as-you-code.
Based on the question as stated, this is as much info as I can provide. Perhaps consider providing more details about your specific application requirements?
I'm currently using MFC/GDI and Stingray to display bitmaps in my application and am looking for a better solution. Specifically;
Faster drawing speed - My current solution is slow, based on StretchDIBits
Better rendering quality - StretchDIBits rendering quality is awful when scaling a bitmap
Support for rotated bitmaps
Support for loading / saving in all popular formats
Support for large bitmaps - I'm regularly using aerial photographs that are ~64mb as 12,000x12,000 jpegs. GeoTIFF support would also be useful
Compatible with MFC document/view, including printing (e.g. must be able render to a CDC)
Access to source code is good but not necessary
Easy to use / port existing GDI code
While free is always nice, I don't mind spending a reasonable amount on a decent library, though no run time royalty costs. Googling suggests the following;
CImg
Graphics Magick
Lead Tools imaging SDK
Anyone got experience of these or can recommend an good alternative?
GDI+ is available on any Windows machine since early XP. It has codecs for all popular image formats, JPEG is included. Very nice filters for high-quality image rescaling. Unrestricted image rotation. Draws to a CDC through the Graphics class. Source code for the C++ wrappers are available in the SDK gdiplusXxx.h header files. Speed is likely to be equivalent however, rendering is software based to ensure compatibility.
You can #include <gdiplus.h> and use the C++ wrappers directly. The SDK docs are here. The CImage class is available in MFC, it doesn't expose all capabilities however.
I think it's unlikely you'll find something that performs faster than GDI on windows since it has kernel-level support which is something open source solutions will not have.
You might want to also look into OpenGL or Direct2D/Direct3D since these too have direct access to the frame buffer. With 3D APIs, texture size would probably be an issue since most standards limit to something like 4096x4096.
I have used CxImage in the past which is one to add to your evaluation list.