Is it possible to get a rendered frame from a VTK visualization and pass it to OpenCV as an image without actually rendering a VTK window?
Looks like I should be able to follow this answer to get the rendered VTK frame from a window and then pass it to OpenCV code, but I don't want to render the VTK window. (I want to render a PLY mesh using VTK to control the camera pose, then output the rendered view to OpenCV so I can distort it for an Oculus Rift application).
Can I do this using the vtkRenderer class and not the vtkRenderWindow class?
Also, I'm hoping to do this all using the OpenCV VTK module if that is possible.
EDIT: I'm starting to think I should just be doing this with VTK functions alone since there is plenty of attention being paid to VTK and Oculus Rift paired together. I would still prefer to use OpenCV since that side of the code is complete and works nicely already.
You must make your render windows to render offline like this:
renderWindow->SetOffScreenRendering( 1 );
Then use a vtkWindowToImageFilter:
vtkSmartPointer<vtkWindowToImageFilter> windowToImageFilter =
vtkSmartPointer<vtkWindowToImageFilter>::New();
windowToImageFilter->SetInput(renderWindow);
windowToImageFilter->Update();
This is called Offscreen Rendering in VTK. Here is a complete example
You can render the image offscreen as mentioned by El Marce and then convert the image to OpenCV cv::Mat using
unsigned char* Ptr = static_cast<unsigned char*>(windowToImageFilter->GetOutput()->GetScalarPointer(0, 0, 0));
cv::Mat RGBImage(dims[1], dims[0], CV_8UC3, Ptr);
Related
I'm new to Imgui and Vulkan and I'm trying to display an image captured from a webcam using OpenCV.
Imgui documentation states that I should:
Load the raw decompressed RGBA image from RAM into a GPU texture. You'll want to use dedicated functions of your graphics API (e.g. OpenGL, DirectX11) to do this.
But their is no example of that using Vulkan API.
I used CreateWICTextureFromFile() from DirectXTK to load an animated GIF texture.
ID3D11Resource* Resource;
ID3D11ShaderResourceView* View;
hr = CreateWICTextureFromFile(d3dDevice, L"sample.gif",
&Resource, &View);
Then I displayed it on an ImageButton in dear IMGUI library:
ImGui::ImageButton((void*)View, ImVec2(width, height));
But it only displays a still image (the first frame of the GIF file).
I think I have to give it the texture of each frame separately. But I don't know how. Can you give me a hint?
The CreateWICTextureFromFile function in DirectX Tool Kit (a.k.a. the 'light-weight' version in the WICTextureLoader module) only loads a single 2D texture, not multi-frame images like animated GIF or TIFF.
The DirectXTex function LoadFromWICFile can load multiframe images if you give it the WIC_FLAGS_ALL_FRAMES flag. Because the library is focused on DirectX resources, it will resize them all to match the first image size.
That said, what WIC is going to return to you is a bunch of raw frames. You have query the metadata from WIC to actually get the animation information, and it's a little complicated to reconstruct. I have a simple implementation in the DirectXTex texassemble tool you can reference here. I focused on converting the animated GIF into a 'flip-book' style 2D texture array which is quite a bit larger.
The sample I referenced can be found on GitHub
I am using VTK to read an obj file, texture map the 3D model and transform it to another view (by applying rotateY/X/Z transforms to vtkActors) and writing it to file using vtkwindowtoImageFilter. Due to this pipeline, the rendered image is displayed on the screen before being written to file. Is there a way to do the same pipeline without the image being displayed on screen ?
If you are using VTK 5.10 or earlier, you can reander the geometry off screen.
I am not quite sure if this is what you are looking for. I am new to vtk and I found the above link looking for a way to convert a triangular surface to volume data, ie, to voxelize the surface. All profile I found on the internet is about using vtkwindowtoImageFilter to obtain a 2D section of the screen, have you worked out a way to access 3D data of the rendered window? Please tell me about that.
I'm programming an application in c++ with a GUI in GTK3 that will show the images obtained from a genicam camera. I've got the official API of the camera that deals with it and extract returns an unsigned char* to the buffer where the image is contained.
The problem comes when I try to convert the image to a GTK format to render it in the GUI. I've tried with cairo and pixbuf, but I've problems in both of them, as the image is in MONO8 format, and pixbuf only deals with RGB. Cairo can deal with 8bit images, but only if they have an alpha channel, which is not the case.
Does someone know a way to approach this issue?
Thanks in advance
How can I use both Opencv and OpenGl to draw some 3D shape onto a Mat frame ? The scenario is simple I am grabbing some frames ( cv::Mat) and I would like to visualise them with some 3D cool target arrows like this ---> target <---. Any ideas on how to do that? What would be a good reference to search for ?
Firstly use latest OpenCV version, or use the version which comes with Qt support. Make sure that you install OpenCV with Qt support, if you are building yourself then build with Qt support. Next go to this link http://docs.opencv.org/modules/highgui/doc/qt_new_functions.html#setopengldrawcallback
A single function will let you draw on the top of the image using a callback function.