I need to convert depth information aquired with a kinect sensor,
to real world 3D coordinates.
I know that the way to do this is by using a DepthGenerator
and call ConvertProjectiveToRealWorld
but this requires the sensor to be connected....
Does anyone knows a way to do it without the sensor connected?
How is your depth information stored?
The easiest way would probably be initializing OpenNI from a depth recording (.oni file). You can create .oni files using the NiViewer sample bundled with OpenNI (press on '?' to see the list of commands, one of them should let you record).
If your data isn't stored in an oni file, you should be able to create a dummy file with a single depth frame in it. That should be enough to cause the sensor parameters to be stored in the oni file as well - the parameters that are used in the projective to real world conversion.
Related
I have recently purchased an Orbbec Astra camera, which uses the same technology and produces the same style depth map as a Microsoft Kinect.
What would be the correct file format to save the depth map frames, How would I go about saving the videos recorded?
I have been able to load a stream but am not sure what format the frames should be saved in so that i can load them for testing at a later stage and still have all the same information.
I am using OpenNI2, OpenCV3.1.0 and C++.
I am currently working on a SLAM algorithm, and I succeeded in gathering the depth and RGB data on the form of a point cloud. However, I only display the frames that my Kinect 2.0 received to the screen and that is all.
I would like to gather those frames and as I move the Kinect, I construct a more elaborate Map (either 2D or 3D) so that it will help me in the localization or mapping.
My idea of the map construction would be just like when we create a Panorama image from many single snapshots.
Anyone has a clue, idea or an algorithm to do it?
You can use rtabmap to create 3D map and localizing your device. Its very simple to use and supports different devices.
I initialized the kinect sensor using NUI_INITIALIZE(NUI_INITIALIZE_FLAG_USES_SKELETON) to get the skeletal data.
I'm working on Augmented Reality Project where i can display a virtual ball/cube in the video feed that kinect generates by gathering the skeletal data in the background.
I will get the coordinates of hands and i'll render the cube with respect to the hand.
However i can't find a way to have a video feed and skeletal data together.
NUI_INITIALIZE(NUI_INITIALIZE_FLAG_USES_COLOR) gives you color data, you can only initialize the camera once. So it is either the video feed or the skeleton coordinates.
I tried to find the solution but i can't find any.
Note: I don't have any use of RGB except for preview so i can see the virtual object, since i'll be using the skeleton data to get the hand coordinates.
Found the Answer:
NuiInitialize(NUI_INITIALIZE_FLAG_USES_COLOR|NUI_INITIALIZE_FLAG_USES_SKELETON);
This will allow use of both the data.
I'm studying the use of multiple cameras for computer vision applications. E.g. there is a camera in every corner of the room and the task is human tracking. I would like to simulate this kind of environment. What I need is:
Ability to define dynamic 3D environment, e.g. room and a moving object.
Options to place cameras at different positions and get simulated data set for each camera.
Does anyone have any experience with that? I checked out blender (http://www.blender.org), but currently I'm looking for a faster/easier to use solution.
Could you give me guidance to similar software/libraries (preferably C++ or MATLAB).
you may find ILNumerics perfectly fits your needs:
http://ilnumerics.net
If I get it right! you are looking to simulate camera feed from multiple camera at different positions of an environment.
I dont know of any sites or a working ready made solution, but here is how I would proceed:
Procure 3d point clouds of a dynamic environment (see Kinect 3d slam benchmark datasets) or generate one of your own with Kinect(hoping you have Xbox kinect with you).
Once you got kinect point clouds in PCL point cloud format, you can simulate video feed from various cameras.
A pseudo code such as this will suffice:
#include <pcl_headers>
//this method just discards all 3d depth information and fills the pixels with rgb values
//this is like a snapshot in the pcd_viewer of pcl(point cloud library)
makeImage(cloud,image){};
pcd <- read the point clouds
camera_positions[] <- {new CameraPosition(affine transform)...}
for(camera_position in camera_positions)
pcl::transformPointCloud(pcd,
cloud_out,
camera_position.getAffineTransform()
);
//Now cloud_out contains point cloud in different viewpoint
image <- new Image();
make_image(cloud_out,image);
saveImage(image);
pcl provides a function to transform a point cloud given appropriate parameters pcl::trasformPointCloud()
If you wish not to use pcl then you may wish to check this post and then followed with remaining steps.
I am about to grab the video output of my raspberry pi to pass it to kinda adalight ambient lightning system.
The XBMC's player for PI, omxplayer, users OpenMAX API for decoding and other functions.
Looking into the code gives the following:
m_omx_tunnel_sched.Initialize(&m_omx_sched, m_omx_sched.GetOutputPort(), &m_omx_render, m_omx_render.GetInputPort());
as far as I understand, this sets a pipeline between the video scheduler and the renderer [S]-->[R].
Now my idea is to write a grabber component and plug-in it hardly into the pipeline [S]-->[G]->[R]. The grabber will extract the pixels from the framebuffer and pass it to a deamon which will drive the leds.
Now I am about to dig into OpenMAX API which seems to be pretty weird. Where should I start? Is it a feasible approach?
Best Regards
If you want the decoded data then just do not send to the renderer. Instead of rendering, take the data and do whatever you want to do. The decoded data should be taken from the output port of the video_decode OpenMAX IL component. I suppose you'll also need to set the correct output pixel format, so set the component output port to the correct format you need, so the conversion is done by the GPU (YUV or RGB565 are available).
At first i think you should attach a buffer to the output of camera component, do everything you want with that frame in the CPU, and send a frame through a buffer attached to the input port of the render, its not going to be a trivial task, since there is little documentation about OpenMax on the raspberry.
Best place to start:
https://jan.newmarch.name/RPi/
Best place to have on hands:
http://home.nouwen.name/RaspberryPi/documentation/ilcomponents/index.html
Next best place: source codes distributed across the internet.
Good luck.