We're busy trying to render the frames we get from WebRTC, but we are having problems showing the video correctly.
Does anyone have experience in this or is there an guide online that shows how to render frames from WebRTC in win32?
We are building out application in Visual Studio 2013. We used OpenGL for an previous attempt and we use Direct3D now, a guide with either is fine for us.
What's the problem exactly? It's just a block of I420 format pixel data.
With any YUV -> RGB conversion method(even WebRTC library provide one), remain work to do is just drawing RGB bitmap to a window.
In my personal experience, conversion was really fast, and drawing to the buffer(memory DC, backbuffer) was sufficiently fast. Only issue was actual screen presentation. For this, Direct2D was most convenient.
Related
I've been trying to figure out how to playback a video file that is equirectangular (and adding movement controls.) I got the playback part using SDK samples. However, getting the video frames to texture to add to a skybox seems downright impossible. I've already looked at the custom EVR and DX11 renderer but can't seem to understand how all that works. Anyone have any ideas?
Thanks.
I think it possible to implement you idea, but you must know that all default renderers are used for simple renderer video. However, you can write own implementation IMFMediaSink class for your purpose. Or use simple frame grabber. You can get more by link - videoInput. It web site contains code for grabbing live video frames from web cam and rendering them via texturing of square object in OpenGL - very similar of your need.
I'm programming an application in c++ with a GUI in GTK3 that will show the images obtained from a genicam camera. I've got the official API of the camera that deals with it and extract returns an unsigned char* to the buffer where the image is contained.
The problem comes when I try to convert the image to a GTK format to render it in the GUI. I've tried with cairo and pixbuf, but I've problems in both of them, as the image is in MONO8 format, and pixbuf only deals with RGB. Cairo can deal with 8bit images, but only if they have an alpha channel, which is not the case.
Does someone know a way to approach this issue?
Thanks in advance
I am using OpenCV to show image on the projector. But it seems the cv::imshow is not fast enough or maybe the data transfer is slow from my CPU to GPU then to projector, so I wonder if there is a faster way to display than OpenCV?
I considered OpenGL, since OpenGL directly uses GPU, the command may be faster than from CPU which is used by OpenCV. Correct me if I am wrong.
OpenCV already supports OpenGL for image output by itself. No need to write this yourself!
See the documentation:
http://docs.opencv.org/modules/highgui/doc/user_interface.html#imshow
http://docs.opencv.org/modules/highgui/doc/user_interface.html#namedwindow
Create the window first with namedWindow, where you can pass the WINDOW_OPENGL flag.
Then you can even use OpenGL buffers or GPU matrices as input to imshow (the data never leaves the GPU). But it will also use OpenGL to show regular matrix data.
Please note:
To enable OpenGL support, configure OpenCV using CMake with
WITH_OPENGL=ON . Currently OpenGL is supported only with WIN32, GTK
and Qt backends on Windows and Linux (MacOS and Android are not
supported). For GTK backend gtkglext-1.0 library is required.
Note that this is OpenCV 2.4.8 and this functionality has changed quite recently. I know there was OpenGL support in earlier versions in conjunction with the Qt backend, but I don't remember when it was introduced.
About the performance: It is a quite popular optimization in the CV community to output images using OpenGL, especially when outputting video sequences.
OpenGL is optimised for rendering images, so it's likely faster. It really depends if the OpenCV implementation uses any GPU acceleration AND if the bottleneck is on rendering side of things.
Have you tried GPU accelerated OpenCV? - http://opencv.org/platforms/cuda.html
How big is the image you are displaying? How long does it take to display the image using cv::imshow now?
I know it's an old question, but I happened to have exactly the same problem. And from my observations I've concluded that the root of the problem is the projector's own latency, especially if one is using an older model.
How have I concluded it?
I displayed the same video sequence with cv::imshow() on the laptop monitor and on the projector. Then I waved my hand. It was obvious, that projector introduces significant latency.
To double-check, I've opended a webcam video, waved my hand in front of it and observed the difference on the monitor and on the projector. Webcam does no processing, no opencv operations, so in my understanding the only thing that would explain the latency would be the projector itself.
What is the best/easiest way to display a video (with sound!) in an application using XAudio2 and Direct3D9/10?
At the very least it needs to be able to stream potentially larger videos, and take care of the fact that the windows aspect ratio may differ from the videos (eg by adding letter boxes), although ideally Id like the ability to embed the video into a 3D scene.
I could of course work out a way to load each frame into a texture, discarding/reusing the textures once rendered, and playing the audio separately through XAudio2, however as well as writing a loader for at least one format, ive also got to deal with stuff like synchronising the video and audio components, so hopefully there is an eaier solution available or even a ready made free one with a suitable lisence (commercial distribution in binary form, dynamic linking is fine in the case of say LGPL).
In Windows SDK, there is a DirectShow example for rendering video to texture. It handles audio output too.
But there are limitations and I can't honestly call it easy.
Have you looked at Bink video? Its what lots of games use for video playback. Works great and you don't have to code all that video stuff yourself from scratch.
Does anyone know of a good resource that will show me how to load an image with C++ and spin it?
What I mean by spin is to do an actual animation of the image rotating and not physically rotating the image and saving it.
If I am not clear on what I am asking, please ask for clarification before downvoting.
Thanks
I would definitely default to OpenGL for this type of task, you can load the image into a Texture, then either redraw the image at different angles, or better yet you can just spin the 'camera' in the OpenGL engine. There are loads of OpenGL tutorials around, a quick google search will get you everything you need.
You could use SDL and the extension sdl_image and/or sdl_gfx
In Windows using GDI+ you could show rotated image in the following way:
Graphics graphics( GetSafeHwnd() ); // initialize from window handle
// You can construct Image objects from a variety of
// file types including BMP, ICON, GIF, JPEG, Exif, PNG, TIFF, WMF, and EMF.
Image image( L"someimage.bmp" );
graphics.RotateTransform( 30.0f ); // 30 - angle, in degrees.
graphics.DrawImage( &image, 0, 0 ); // draw rotated image
You could read here more detailed explanation.
Second solution is to use DirectX. You could create texture from file and later render it. It is not trivial solution, but it'll use hardware acceleration and will give you the best performance.
On Windows 7 there is available new API called Direct2D. I have not used it yet, but it looks promising.
Direct2D provides Win32 developers with the ability to perform 2-D graphics rendering tasks with superior performance and visual quality.
i agree with DeusAduro. OpenGL is a good way of doing this.
you can also do this with Qt
The "roll-your-own" solution is difficult.
I'd suggest looking into WPF - it might have some nice options in an image control.