Some pre-defined edges, such as EdgeProjectXYZ2UV, EdgeSE3ProjectXYZ, are widely used during BA. However, they can only set unchangeable camera parameters. I'm wondering if I can set optimizable camera parameters.
Thanks for any reply in advance!
I found Vertex VertexIntrinsics in g2o/types/sba/vertex_intrinsics.h. And its oplusImpl() function is implemented which means it can be optimized.
/**
* \brief Vertex encoding the intrinsics of the camera fx, fy, cx, xy, baseline;
*/
class G2O_TYPES_SBA_API VertexIntrinsics : public BaseVertex<4, Eigen::Matrix<number_t, 5, 1, Eigen::ColMajor> >
It seems to be the one I'm looking for. Yet I cannot find a type of Ternary-Edge that link this vertex, camera pose vertex, and 3D world point vertex. Or any other edge that takes this type of vertex.
It would be great if somebody could share an example or some explanation about how this vertex works. Or any other way to optimize camera intrinsic in g2o.
I managed to solve it by defining a Ternary edge that links the camera intrinsic Vertex (VertexIntrinsics), camera pose Vertex (VertexSE3 or VertexSE3Expmap), and Observation Target (Vertex SE3 in my case, or VertexPointXYZ ).
There will be only one VertexIntrinsics in the graph.
If you are looking for a ternary edge example, check edge_project_psi2uv.h.
Thank #RainerKuemmerle for such great work.
Related
I'm absolutely new to ROS/Gazebo world; this is probably a simple question, but I cannot find a good answer.
I have a simulated depth camera (Kinect) in a Gazebo scene. After some elaborations, I get a point of interest in the RGB image in pixel coordinates, and I want to retrieve its 3D coordinates in the world frame.
I can't understand how to do that.
I have tried compensating the distortions, given by the CameraInfo msg. I have tried using PointCloud with pcl library, retrieving the point as cloud.at(x,y).
In both cases, the coordinates are not correct (I have put a small sphere in the coords given out by the program, so to check if it's correct or no).
Every help would be very appreciated. Thank you very much.
EDIT:
Starting from the PointCloud, I try to find the coords of the points doing something like:
point = cloud.at(xInPixel, yInPixel);
point.x = point.x + cameraPos.x;
point.y = point.y + cameraPos.y;
point.z = point.z - cameraPos.z;
but the x,y,z coords I get as point.x seems not to be correct.
The camera has a pitch angle of pi/2, so to points on the ground.
I am clearly missing something.
I assume you've seen the gazebo examples for the kinect (brief, full). You can get, as topics, the raw image, raw depth, and calculated pointcloud (by setting them in the config):
<imageTopicName>/camera/color/image_raw</imageTopicName>
<cameraInfoTopicName>/camera/color/camera_info</cameraInfoTopicName>
<depthImageTopicName>/camera/depth/image_raw</depthImageTopicName>
<depthImageCameraInfoTopicName>/camera/dept/camera_info</depthImageCameraInfoTopicName>
<pointCloudTopicName>/camera/depth/points</pointCloudTopicName>
Unless you need to do your own things with the image_raw for rgb and depth frames (ex ML over rgb frame & find corresponding depth point via camera_infos), the pointcloud topic should be sufficient - it's the same as the pcl pointcloud in c++, if you include the right headers.
Edit (in response):
There's a magical thing in ros called tf/tf2. Your pointcloud, if you look at the msg.header.frame_id, says something like "camera", indicating it's in the camera frame. tf, like the messaging system in ros, happens in the background, but it looks/listens for transformations from one frame of reference to another frame. You can then transform/query the data in another frame. For example, if the camera is mounted at a rotation to your robot, you can specify a static transformation in your launch file. It seems like you're trying to do the transformation yourself, but you can make tf do it for you; this allows you to easily figure out where points are in the world/map frame, vs in the robot/base_link frame, or in the actuator/camera/etc frame.
I would also look at these ros wiki questions which demo a few different ways to do this, depending on what you want: ex1, ex2, ex3
I'm trying to implement AABBs/OOBBs with MathGeoLib since the ease to operate with BBs (and because I wanted to test some things with that library).
The problem is that the engine's objects transformations are based on glm since we started with glm (and they work properly) but when it comes to transform the OOBBs according to an object, it doesn't work very well.
What I basically do is to pass to a function the game object's translation, orientation and scale (I tried to pass a global matrix but it doesn't work, it seems to 'add' the transformation instead of setting it, and I can't access the oobb's matrix). That function does the next:
glm::vec3 pos = passedPosition - OOBBPreviousPos;
glm::mat4 Transformation = glm::translate(glm::mat4(1.0f), pos) *
glm::mat4_cast(passedRot) * glm::scale(glm::mat4(1.0f), passedScale);
glm::mat4 resMat = glm::transpose(Transformation);
math::float4x4 mat = math::float4x4::identity;
mat.Set(glm::value_ptr(resMat));
Which basically transposes the glm matrix (I have seen that that's they way of 'translating' them), passes it to a float* and then it constructs the MathGeoLib matrix with that. I have debugged it and the values seem to be right according to the object, so the next thing I do is actually transform the OOBB and then, enclose the AABB to have it inside, like this:
m_OBB.Transform(mat);
m_AABB.SetNegativeInfinity(); //Sets AABB to "null"
m_AABB.Enclose(m_OBB);
The final behaviour is pretty strange, believe me if I say that is the most close I've been from having it right, I've been some days testing different things and nothing works better (passing global/local matrices directly, trying different ways of passing/constructing transformation data, checking if the glm-MathGLib is correct...). It rotates but not around its own axis, and the scaling gets him crazy (although translation works). Its current behaviour can be seen here: https://gfycat.com/quarrelsomefineduck (blue cubes are AABBs, green ones are OOBBs).
Am I doing something wrong with the mathematics calculations or data transfer?
I still been looking on that but then some friend made me look into another direction, so I finally solved it (or better said: I "worked-around it") by storing an initial object's AABB and passing to the mentioned function the game object's global matrix. Then, inside the function, I used another MathGeoLib function to transform the OOBB.
That function finally looks like:
glm::mat4 resMat = glm::transpose(GlobalMatrixPassed);
math::float4x4 mat = math::float4x4::identity;
mat.Set(glm::value_ptr(resMat)); //"Translate" glm matrix passed into a MathGeoLib one
m_OOBB.SetFrom(m_InitialAABB); //Set OOBB from the initial aabb
m_OOBB.Transform(mat); //Transform it
m_AABB.SetFrom(m_OOBB); //Set the AABB in function of the transformed OOBB
I am trying to implement this SAO algorithm.
I am getting the following result :
I can't figure out why I have the nose on top of the walls, it seems to be a z-buffer issue.
Here are my input values :
const float projScale = 100.0;
const float radius = 0.9;
const float bias = 0.0005;
const float intensityDivR6 = pow(radius, 6);
I am using the original shader without modifications, except that I disable the usage of mipmaps of the depth buffer.
My depth buffer (on different scene, sorry) :
It should be an issue with the zbuffer linearization or it's not between -1 and 1.
Thank you Bruno, I finally figure out what were the issues.
The first was that I didn't transform my Z correctly, they use a specific pre-pass to make the Z linear and put it between -1 and 1. I was using an incompatible method to do it.
I also had to negate my near and far planes values directly in the projection matrix to compute correctly some uniforms.
Result :
I had a similar problem, having visual wrong occlusion, linked to the near/far, so I decided to give you what I've done to fix it.
The problem I had is discribed in a previous comment. I was getting self occlusion, when the camera was close to an object or when the radius was really too big.
If you take a closer look at the conversion from depth buffer value to camera-space value (the reconstructCSZ function from the g3d engine), you will see that replacing the depth by 0 will give you the near plane if you work with positive near/far. So, what it means is that every time you will get a tap outside the model, you will get a z component equals to near, which will give you wrong occlusion for fragments having a z close to 0.
You basically have to discard each taps that are located on the near plane, to avoid them being taken into account when comptuing the full contribution.
I have some problems with trying to animate the rotation of mesh objects.
If to make the rotation process once all is fine. Meshes are rotated normally and the final image from the WebGL buffer looks pretty fine.
http://s22.postimg.org/nmzvt9zzl/311.png
But if to use the rotation in loop (with each new frame record) then the meshes are starting to look very weird, look the next screenshot:
http://s22.postimg.org/j2dpecga9/312.png
I won't provide here the programming code, because the issue is depend on incorrect 3D graphics handling.
I think some OpenGL/Direct3D developers may give an advice how to fix it, because this question relates to the 3D-programming subject and to the some specific GL or D3D function/method. Also I think the way of work with rotation is the same both in OpenGL and Direct3D because of linear algebra and affine affine transformations.
If you are really interested what I'm using, so the answer is WebGL.
Let me describe how do I use the rotation of object.
The simple rotation is made using the quaternions. Any mesh object I define has its quaternion property.
If to rotate the object then the method rotate() is doing the next:
// Some kind of pseudo-code
function rotateMesh( vector, angle ) {
var tempQuatenion = math.convertRotationToQuaternion( vector, angle );
this.quaternion = math.multiplyQuaternions( this.quaternion, tempQuatenion );
}
I use the next piece of code in Renderer class to handle the mesh translation and rotation:
// Some kind of pseudo-code
// for each mesh, which is added to scene
modelViewMatrix = new IdentityMatrix()
translateMatrixByVector( modelViewMatrix, mesh.position )
modelViewMatrix.multiplyByMatrix( mesh.quaternion.toMatrix() )
So... I want to ask you if the logic above is correct then I provide some sources of math functions which are used for quaternions, rotations, etc...
If the logic above is incorrect so I think it makes no sense to provide something else. Because it's required to fix the main logic.
Quaternion multiplication is not commutative, i.e., if A and B are quaternions then A * B != B * A. If you want to rotate quaternion A by quaternion B, you need to do A = B * A, so this:
this.quaternion = math.multiplyQuaternions( this.quaternion, tempQuatenion );
should have its arguments reversed.
In addition, as mentioned by #ratchet-freak in comments, you should make sure your quaternions are always normalized, otherwise transformations other than rotation may happen.
I was studying Perlin's Noise through some examples # http://dindinx.net/OpenGL/index.php?menu=exemples&submenu=shaders and couldn't help to notice that his make3DNoiseTexture() in perlin.c uses noise3(ni) instead of PerlinNoise3D(...)
Now why is that? Isn't Perlin's Noise supposed to be a summation of different noise frequencies and amplitudes?
Qestion 2 is what does ni, inci, incj, inck stand for? Why use ni instead of x,y coordinates? Why is ni incremented with
ni[0]+=inci;
inci = 1.0 / (Noise3DTexSize / frequency);
I see Hugo Elias created his Perlin2D with x,y coordinates, and so does PerlinNoise3D(...).
Thanks in advance :)
I now understand why and am going to answer my own question in hopes that it helps other people.
Perlin's Noise is actually a synthesis of gradient noises. In its production process, we must compute the dot product of a vector pointing from one of the corners flooring the input point to the input point itself with the random-generated gradient vector.
Now if the input point were a whole number, such as the xyz coordinates of a texture you want to create, the dot product would always return 0, which would give you a flat noise. So instead, we use inci, incj, inck as an alternative index. Yep, just an index, nothing else.
Now returning to question 1, there are two methods to implement Perlin's Noise:
1.Calculate the noise values separately and store them in the RGBA slots in the texture
2.Synthesize the noises up before-hand and store them in one of the RGBA slots in the texture
noise3(ni) is the actual implementation of method 1, while PerlinNoise3D(...) suggests the latter.
In my personal opinion, method 1 is much better because you have much more flexibility over how you use each octave in your shaders.
My guess on the reason for using noise3(ni) in make3DNoiseTexture() instead if PerlinNoise3D(...) is that when you use that noise texture in your shader you want to be able to replicate and modify the functionality of PerlinNoise3D(...) directly in the shader.
My guess for the reasoning behind ni, inci, incj, inck is that using x,y,z of the volume directly don't give a good result so by scaling the the noise with the frequency instead it is possible to adjust the resolution of the noise independently from the volume size.