I'm currently trying to get the relative position of two Kinect v2s by getting the position of a tracking pattern both cameras can see. Unfortunately I can't seem to get the correct position of the patterns origin.
This is my current code to get the position of the pattern relative to the camera:
std::vector<cv::Point2f> centers;
cv::findCirclesGrid( registeredColor, m_patternSize, centers, cv::CALIB_CB_ASYMMETRIC_GRID );
cv::solvePnPRansac( m_corners, centers, m_camMat, m_distCoeffs, m_rvec, m_tvec, true );
// calculate the rotation matrix
cv::Matx33d rotMat;
cv::Rodrigues( m_rvec, rotMat );
// and put it in the 4x4 transformation matrix
transformMat = matx3ToMatx4(rotMat);
for( int i = 0; i < 3; ++i )
transformMat(i,3) = m_tvec.at<double>(i);
transformMat = transformMat.inv();
cv::Vec3f originPosition( transformMat(0,3), transformMat(1,3), transformMat(2,3) );
Unfortunately, when I compare originPosition to the point in the pointcloud that corresponds to the origin found in screenspace (saved in centers.at(0) above) I get a very different result.
The Screenshot below shows the pointcloud from the kinect with the point at the screenspace position of the pattern's origin in red in the red circle and the point at originPosition in light blue in the light blue circle. The screenshot was taken from directly in front of the pattern. The originPosition is also a bit more to the front.
As you can see, the red dot is perfectly in the first circle of the pattern while the blue dot corresponding to originPosition is not even close. Especially it is definitely not just a scaling issue of the vector from camera to origin. Also, the findCirclesGrid is done on the registered color image and the intrinsic parameters are taken from the camera itself to ensure that there is no difference in those between the image and the calculation of the point cloud.
You have transormation P->P' given by R|T, To get inverse transformation P'->P given by R'|T' just do:
R' = R.t();
T' = -R'* T;
And then
P = R' * P' + T'
Related
I have the following problem as shown in the figure. I have point cloud and a mesh generated by a tetrahedral algorithm. How would I carve the mesh using the that algorithm ? Are landmarks are the point cloud ?
Pseudo code of the algorithm:
for every 3D feature point
convert it 2D projected coordinates
for every 2D feature point
cast a ray toward the polygons of the mesh
get intersection point
if zintersection < z of 3D feature point
for ( every triangle vertices )
cull that triangle.
Here is a follow up implementation of the algorithm mentioned by the Guru Spektre :)
Update code for the algorithm:
int i;
for (i = 0; i < out.numberofpoints; i++)
{
Ogre::Vector3 ray_pos = pos; // camera position);
Ogre::Vector3 ray_dir = (Ogre::Vector3 (out.pointlist[(i*3)], out.pointlist[(3*i)+1], out.pointlist[(3*i)+2]) - pos).normalisedCopy(); // vertex - camea pos ;
Ogre::Ray ray;
ray.setOrigin(Ogre::Vector3( ray_pos.x, ray_pos.y, ray_pos.z));
ray.setDirection(Ogre::Vector3(ray_dir.x, ray_dir.y, ray_dir.z));
Ogre::Vector3 result;
unsigned int u1;
unsigned int u2;
unsigned int u3;
bool rayCastResult = RaycastFromPoint(ray.getOrigin(), ray.getDirection(), result, u1, u2, u3);
if ( rayCastResult )
{
Ogre::Vector3 targetVertex(out.pointlist[(i*3)], out.pointlist[(3*i)+1], out.pointlist[(3*i)+2]);
float distanceTargetFocus = targetVertex.squaredDistance(pos);
float distanceIntersectionFocus = result.squaredDistance(pos);
if(abs(distanceTargetFocus) >= abs(distanceIntersectionFocus))
{
if ( u1 != -1 && u2 != -1 && u3 != -1)
{
std::cout << "Remove index "<< "u1 ==> " <<u1 << "u2 ==>"<<u2<<"u3 ==> "<<u3<< std::endl;
updatedIndices.erase(updatedIndices.begin()+ u1);
updatedIndices.erase(updatedIndices.begin()+ u2);
updatedIndices.erase(updatedIndices.begin()+ u3);
}
}
}
}
if ( updatedIndices.size() <= out.numberoftrifaces)
{
std::cout << "current face list===> "<< out.numberoftrifaces << std::endl;
std::cout << "deleted face list===> "<< updatedIndices.size() << std::endl;
manual->begin("Pointcloud", Ogre::RenderOperation::OT_TRIANGLE_LIST);
for (int n = 0; n < out.numberofpoints; n++)
{
Ogre::Vector3 vertexTransformed = Ogre::Vector3( out.pointlist[3*n+0], out.pointlist[3*n+1], out.pointlist[3*n+2]) - mReferencePoint;
vertexTransformed *=1000.0 ;
vertexTransformed = mDeltaYaw * vertexTransformed;
manual->position(vertexTransformed);
}
for (int n = 0 ; n < updatedIndices.size(); n++)
{
int n0 = updatedIndices[n+0];
int n1 = updatedIndices[n+1];
int n2 = updatedIndices[n+2];
if ( n0 < 0 || n1 <0 || n2 <0 )
{
std::cout<<"negative indices"<<std::endl;
break;
}
manual->triangle(n0, n1, n2);
}
manual->end();
Follow up with the algorithm:
I have now two versions one is the triangulated one and the other is the carved version.
It's not not a surface mesh.
Here are the two files
http://www.mediafire.com/file/cczw49ja257mnzr/ahmed_non_triangulated.obj
http://www.mediafire.com/file/cczw49ja257mnzr/ahmed_triangulated.obj
I see it like this:
So you got image from camera with known matrix and FOV and focal length.
From that you know where exactly the focal point is and where the image is proected onto the camera chip (Z_near plane). So any vertex, its corresponding pixel and focal point lies on the same line.
So for each view cas ray from focal point to each visible vertex of the pointcloud. and test if any face of the mesh hits before hitting face containing target vertex. If yes remove it as it would block the visibility.
Landmark in this context is just feature point corresponding to vertex from pointcloud. It can be anything detectable (change of intensity, color, pattern whatever) usually SIFT/SURF is used for this. You should have them located already as that is the input for pointcloud generation. If not you can peek pixel corresponding to each vertex and test for background color.
Not sure how you want to do this without the input images. For that you need to decide which vertex is visible from which side/view. May be it is doable form nearby vertexes somehow (like using vertex density points or corespondence to planar face...) or the algo is changed somehow for finding unused vertexes inside mesh.
To cast a ray do this:
ray_pos=tm_eye*vec4(imgx/aspect,imgy,0.0,1.0);
ray_dir=ray_pos-tm_eye*vec4(0.0,0.0,-focal_length,1.0);
where tm_eye is camera direct transform matrix, imgx,imgy is the 2D pixel position in image normalized to <-1,+1> where (0,0) is the middle of image. The focal_length determines the FOV of camera and aspect ratio is ratio of image resolution image_ys/image_xs
Ray triangle intersection equation can be found here:
Reflection and refraction impossible without recursive ray tracing?
If I extract it:
vec3 v0,v1,v2; // input triangle vertexes
vec3 e1,e2,n,p,q,r;
float t,u,v,det,idet;
//compute ray triangle intersection
e1=v1-v0;
e2=v2-v0;
// Calculate planes normal vector
p=cross(ray[i0].dir,e2);
det=dot(e1,p);
// Ray is parallel to plane
if (abs(det)<1e-8) no intersection;
idet=1.0/det;
r=ray[i0].pos-v0;
u=dot(r,p)*idet;
if ((u<0.0)||(u>1.0)) no intersection;
q=cross(r,e1);
v=dot(ray[i0].dir,q)*idet;
if ((v<0.0)||(u+v>1.0)) no intersection;
t=dot(e2,q)*idet;
if ((t>_zero)&&((t<=tt)) // tt is distance to target vertex
{
// intersection
}
Follow ups:
To move between normalized image (imgx,imgy) and raw image (rawx,rawy) coordinates for image of size (imgxs,imgys) where (0,0) is top left corner and (imgxs-1,imgys-1) is bottom right corner you need:
imgx = (2.0*rawx / (imgxs-1)) - 1.0
imgy = 1.0 - (2.0*rawy / (imgys-1))
rawx = (imgx + 1.0)*(imgxs-1)/2.0
rawy = (1.0 - imgy)*(imgys-1)/2.0
[progress update 1]
I finally got to the point I can compile sample test input data for this to get even started (as you are unable to share valid data at all):
I created small app with hard-coded table mesh (gray) and pointcloud (aqua) and simple camera control. Where I can save any number of views (screenshot + camera direct matrix). When loaded back it aligns with the mesh itself (yellow ray goes through aqua dot in image and goes through the table mesh too). The blue lines are casted from camera focal point to its corners. This will emulate the input you got. The second part of the app will use only these images and matrices with the point cloud (no mesh surface anymore) tetragonize it (already finished) now just cast ray through each landmark in each view (aqua dot) and remove all tetragonals before target vertex in pointcloud is hit (this stuff is not even started yet may be in weekend)... And lastly store only surface triangles (easy just use all triangles which are used just once also already finished except the save part but to write wavefront obj from it is easy ...).
[Progress update 2]
I added landmark detection and matching with the point cloud
as you can see only valid rays are cast (those that are visible on image) so some points on point cloud does not cast rays (singular aqua dots)). So now just the ray/triangle intersection and tetrahedron removal from list is what is missing...
I am making an android application that will detect driver's fatigue from head nod pose.
What I did:
I have detected 68 facial landmarks from image using dlib library.
Used solvePnP to find out rotation matrix and from that matrix I have got roll, pitch and yaw angles.
Now, I have to detect head nod from pitch angle.
My Problem:
How to set a threshold value so that an angle below or above the threshold can be said as a head nod?
It results some negative angles. What does a negative angle mean in 3 dimensional axis?
My code:
void getEulerAngles(Mat &rotCamerMatrix,Vec3d &eulerAngles)
{
Mat cameraMatrix,rotMatrix,transVect,rotMatrixX,rotMatrixY,rotMatrixZ;
double* _r = rotCamerMatrix.ptr<double>();
double projMatrix[12] = {_r[0],_r[1],_r[2],0,
_r[3],_r[4],_r[5],0,
_r[6],_r[7],_r[8],0};
decomposeProjectionMatrix( Mat(3,4,CV_64FC1,projMatrix),
cameraMatrix,
rotMatrix,
transVect,
rotMatrixX,
rotMatrixY,
rotMatrixZ,
eulerAngles);
}
int renderToMat(std::vector<full_object_detection>& dets, Mat& dst)
{
Scalar color;
std::vector<cv::Point2d> image_points;
std::vector<cv::Point3d> model_points;
string disp;
int sz = 3,l;
color = Scalar(0,255,0);
double p1,p2,p3,leftear,rightear,ear=0,yawn=0.00,yaw=0.00,pitch=0.00,roll=0.00;
l=dets.size();
//I am calculating only for one face.. so assuming l=1
for(unsigned long idx = 0; idx < l; idx++)
{
image_points.push_back(
Point2d(dets[idx].part(30).x(),dets[idx].part(30).x() ) );
image_points.push_back(Point2d(dets[idx].part(8).x(),dets[idx].part(8).x() ) );
image_points.push_back(Point2d(dets[idx].part(36).x(),dets[idx].part(36).x() ) );
image_points.push_back(Point2d(dets[idx].part(45).x(),dets[idx].part(45).x() ) );
image_points.push_back(Point2d(dets[idx].part(48).x(),dets[idx].part(48).x() ) );
image_points.push_back(Point2d(dets[idx].part(54).x(),dets[idx].part(54).x() ) );
}
double focal_length = dst.cols;
Point2d center = cv::Point2d(dst.cols/2.00,dst.rows/2.00);
cv::Mat camera_matrix = (cv::Mat_<double>(3.00,3.00) << focal_length, 0, center.x, 0, focal_length, center.y, 0,
0, 1);
cv::Mat dist_coeffs = cv::Mat::zeros(4,1,cv::DataType<double>::type);
cv::Mat rotation_vector; //s Rotation in axis-angle form
cv::Mat translation_vector;
cv::Mat rotCamerMatrix1;
if(l!=0)
{
model_points.push_back(cv::Point3d(0.0f, 0.0f, 0.0f));
model_points.push_back(cv::Point3d(0.0f, -330.0f, -65.0f));
model_points.push_back(cv::Point3d(-225.0f, 170.0f, -135.0f));
model_points.push_back(cv::Point3d(225.0f, 170.0f, -135.0f));
model_points.push_back(cv::Point3d(-150.0f, -150.0f, -125.0f));
model_points.push_back(cv::Point3d(150.0f, -150.0f, -125.0f));
cv::solvePnP(model_points, image_points, camera_matrix, dist_coeffs,rotation_vector, translation_vector);
Rodrigues(rotation_vector,rotCamerMatrix1);
Vec3d eulerAngles;
getEulerAngles(rotCamerMatrix1,eulerAngles);
yaw = eulerAngles[1];
pitch = eulerAngles[0];
roll = eulerAngles[2];
/*My problem begins here. I don't know how to set a threshold value for pitch so that I can say a value below or
above the pitch is a head nod!*/
}
return 0;
}
First you have to understand how are the 3 angles used in a 3D context. Basically they rotate an object in the 3D space with respect to an origin (the origin can change depending on the context), but how are the 3 angles applied?.
This question can be express as: in which order do they rotate the object. If you apply yaw, then pitch and then roll it may give you a different orientation of the object that if you do it in the opposite order. Having said that, you must understand what are those values representing, to understand what to do with them.
Now, you ask what would be a good threshold, well it depends. On what? well, in the order they are applied. For instance if you apply pitch first with 45 degrees so it looks down and then you apply a roll 180 degrees, then it is looking up, so it is a little hard to define a threshold.
Since you have your model points, you can create a 3D rotation matrix and apply it to them with different pitch angles (the rest of the angles will be 0 so the order will not be important here) and visualize them and choose the one that you consider is nodding. This is a little bit subjective, so you should be the one doing it.
Now, to the second question. The answer again is, it depends. On what this time? you may ask. Simple, is your system left handed or right handed? in one this means that the rotation is apply clockwise and the other counter clockwise, the negative sign changes the direction. So, in a left handed system it will be clockwise and with the negative sign will be counterclockwise. The right handed system will be counterclockwise and the negative sign will make it clockwise.
As a suggestion, you can make a vector (0,0,-1) assuming your z axis looks towards the back. Then apply the rotation to it and proyect it to a 2D plane parallel to the z axis, here take the tip of your vector and see what is the angle here. This way you are sure what are you getting.
I am trying to render views of a 3D mesh in VTK, I am doing the following:
vtkSmartPointer<vtkRenderWindow> render_win = vtkSmartPointer<vtkRenderWindow>::New();
vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
render_win->AddRenderer(renderer);
render_win->SetSize(640, 480);
vtkSmartPointer<vtkCamera> cam = vtkSmartPointer<vtkCamera>::New();
cam->SetPosition(50, 50, 50);
cam->SetFocalPoint(0, 0, 0);
cam->SetViewUp(0, 1, 0);
cam->Modified();
vtkSmartPointer<vtkActor> actor_view = vtkSmartPointer<vtkActor>::New();
actor_view->SetMapper(mapper);
renderer->SetActiveCamera(cam);
renderer->AddActor(actor_view);
render_win->Render();
I am trying to simulate a rendering from a calibrated Kinect, for which I know the intrinsic parameters. How can I set the intrinsic parameters (focal length and principle point) to the vtkCamera.
I wish to do this so that the 2d pixel - 3d camera coordinate would be the same as if the image were taken from a kinect.
Hopefully this will help others trying to convert standard pinhole camera parameters to a vtkCamera: I created a gist showing how to do the full conversion. I verified that the world points project to the correct location in the rendered image. The key code from the gist is pasted below.
gist: https://gist.github.com/decrispell/fc4b69f6bedf07a3425b
// apply the transform to scene objects
camera->SetModelTransformMatrix( camera_RT );
// the camera can stay at the origin because we are transforming the scene objects
camera->SetPosition(0, 0, 0);
// look in the +Z direction of the camera coordinate system
camera->SetFocalPoint(0, 0, 1);
// the camera Y axis points down
camera->SetViewUp(0,-1,0);
// ensure the relevant range of depths are rendered
camera->SetClippingRange(depth_min, depth_max);
// convert the principal point to window center (normalized coordinate system) and set it
double wcx = -2*(principal_pt.x() - double(nx)/2) / nx;
double wcy = 2*(principal_pt.y() - double(ny)/2) / ny;
camera->SetWindowCenter(wcx, wcy);
// convert the focal length to view angle and set it
double view_angle = vnl_math::deg_per_rad * (2.0 * std::atan2( ny/2.0, focal_len ));
std::cout << "view_angle = " << view_angle << std::endl;
camera->SetViewAngle( view_angle );
I too am using VTK to simulate the view from a kinect sensor. I am using VTK 6.1.0. I know this question is old, but hopefully my answer may help someone else.
The question is how can we set a projection matrix to map world coordinates to clip coordinates. For more info on that see this OpenGL explanation.
I use a Perspective Projection Matrix to simulate the kinect sensor. To control the intrinsic parameters you can use the following member functions of vtkCamera.
double fov = 60.0, np = 0.5, fp = 10; // the values I use
cam->SetViewAngle( fov ); // vertical field of view angle
cam->SetClippingRange( np, fp ); // near and far clipping planes
In order to give you a sense of what that may look like. I have an old project that I did completely in C++ and OpenGL in which I set the Perspective Projection Matrix similar to how I described, grabbed the z-buffer, and then reprojected the points out onto a scene that I viewed from a different camera. (The visualized point cloud looks noisy because I also simulated noise).
If you need your own custom Projection Matrix that isn't the Perspective flavor. I believe it is:
cam->SetUserTransform( transform ); // transform is a pointer to type vtkHomogeneousTransform
However, I have not used the SetUserTransform method.
This thread was super useful to me for setting camera intrinsics in VTK, especially decrispell's answer. To be complete, however, one case is missing: if the focal length in the x and y directions are not equal. This can easily be added to the code by using the SetUserTransform method. Below is a sample code in python :
cam = self.renderer.GetActiveCamera()
m = np.eye(4)
m[0,0] = 1.0*fx/fy
t = vtk.vtkTransform()
t.SetMatrix(m.flatten())
cam.SetUserTransform(t)
where fx and fy are the x and y focal length in pixels, i.e. the two first diagnoal elements of the intrinsic camera matrix. np is and alias for the numpy import.
Here is a gist showing the full solution in python (without extrinsics for simplicity). It places a sphere at a given 3D position, renders the scene into an image after setting the camera intrinsics, and then displays a red circle at the projection of the sphere center on the image plane: https://gist.github.com/benoitrosa/ffdb96eae376503dba5ee56f28fa0943
I need to rotate the camera around a player from a third person view. (Nothing fancy).
Here it is how I try it:
// Forward, right, and position define the plane - they have x,y,z components.
void rotate ( float angle, Vector interestPoint )
{
Vector oldForward ( Forward );
forward = forward * cos(angle) + right * sin(angle);
forward.Normalize();
right = forward.CrossProduct ( up );
right.Normalize();
position = ( position + old_forward * position.Distance( interestPoint ) ) - (forward * position.Distance( interestPoint ) );
this->angle += angle;
}
The problem is that if, let's say just do turn left a lot, the distance between the object and the camera increases.
For a very simple orbit camera, like most 3rd person adventure games, you will need 4 things:
The position of the target
The distance from the target
The azimuthal angle
The polar angle
(If you want your camera to be always relative to the target in orientation, you need to provide the target's orientation as well, in this case I will simply use the world orientation)
See Spherical coordinate systems for a reference.
You should map your azimuthal angle on your horizontal control (and make it loop around when you reach 2 * PI) and your polar angle should be mapped on your vertical control (or inverted if the player selects that option and make it clamped between -PI and PI - watch out for calculations based on the world Up vector if you go parallel to it (-PI or PI)
The distance can be fixed or driven by a spline, for this case we will assume a fixed distance.
So, to compute your position you start with WorldForward, which is a unit vector pointing in the axis that you generally consider to be your forward, for example (1,0,0) (here, if we were building a relative camera, we would use our target's forward vector) and you invert it (* -1) to go "from the target" "to your camera".
(The following is untested pseudo code, but you should get the gist - also, keep note that it can be simplified, I just went for clarity)
Next step is to rotate this vector using our azimuth angle, which is the horizontal orientation component of your camera. Something like:
Vector toCamera = WorldForward * -1;
Matrix horizontalRotation = Matrix.CreateRotationZ(azimuth); // assuming Z is up
Vector horizontalRotationPosition = horizontalRotation.Transform(toCamera);
At this point, you have a camera that can rotate horizontally around your target, now to add the other axis, you simply transform again using the polar angle rotation:
Matrix verticalRotation = Matrix.CreateRotationY(polar); // assuming Y is right
Vector finalRotatedVector = verticalRotation.Transform(horizontalRotationPosition);
Now, what we have is a unit vector that points to the position where the camera should be, if you multiply it by the distance you want to keep from your target and add the position of your target, you should get your final position. Keep in mind that this unit vector, if negated, represents the forward vector of your camera.
Vector cameraPosition = targetPosition + finalRotatedVector * distanceFromTarget;
I am developing a small tool for 3D visualization of molecules.
For my project i choose to make a thing in the way of what Mr "Brad Larson" did with his Apple software "Molecules". A link where you can find a small presentation of the technique used : Brad Larsson software presentation
For doing my job i must compute sphere impostor and cylinder impostor.
For the moment I have succeed to do the "Sphere Impostor" with the help of another tutorial Lies and Impostors
for summarize the computing of the sphere impostor : first we send a "sphere position" and the "sphere radius" to the "vertex shader" which will create in the camera-space an square which always face the camera, after that we send our square to the fragment shader where we use a simple ray tracing to find which fragment of the square is included in the sphere, and finally we compute the normal and the position of the fragment to compute lighting. (another thing we also write the gl_fragdepth for giving a good depth to our impostor sphere !)
But now i am blocked in the computing of the cylinder impostor, i try to do a parallel between the sphere impostor and the cylinder impostor but i don't find anything, my problem is that for the sphere it was some easy because the sphere is always the same no matter how we see it, we will always see the same thing : "a circle" and another thing is that the sphere was perfectly defined by Math then we can find easily the position and the normal for computing lighting and create our impostor.
For the cylinder it's not the same thing, and i failed to find a hint to modeling a form which can be used as "cylinder impostor", because the cylinder shows many different forms depending on the angle we see it !
so my request is to ask you about a solution or an indication for my problem of "cylinder impostor".
In addition to pygabriels answer I want to share a standalone implementation using the mentioned shader code from Blaine Bell (PyMOL, Schrödinger, Inc.).
The approach, explained by pygabriel, also can be improved. The bounding box can be aligned in such a way, that it always faces to the viewer. Only two faces are visible at most. Hence, only 6 vertices (ie. two faces made up of 4 triangles) are needed.
See picture here, the box (its direction vector) always faces to the viewer:
Image: Aligned bounding box
For source code, download: cylinder impostor source code
The code does not cover round caps and orthographic projections. It uses geometry shader for vertex generation. You can use the shader code under the PyMOL license agreement.
I know this question is more than one-year old, but I'd still like to give my 2 cents.
I was able to produce cylinder impostors with another technique, I took inspiration from pymol's code. Here's the basic strategy:
1) You want to draw a bounding box (a cuboid) for the cylinder. To do that you need 6 faces, that translates in 18 triangles that translates in 36 triangle vertices. Assuming that you don't have access to geometry shaders, you pass to a vertex shader 36 times the starting point of the cylinder, 36 times the direction of the cylinder, and for each of those vertex you pass the corresponding point of the bounding box. For example a vertex associated with point (0, 0, 0) means that it will be transformed in the lower-left-back corner of the bounding box, (1,1,1) means the diagonally opposite point etc..
2) In the vertex shader, you can construct the points of the cylinder, by displacing each vertex (you passed 36 equal vertices) according to the corresponding points you passed in.
At the end of this step you should have a bounding box for the cylinder.
3) Here you have to reconstruct the points on the visible surface of the bounding box. From the point you obtain, you have to perform a ray-cylinder intersection.
4) From the intersection point you can reconstruct the depth and the normal. You also have to discard intersection points that are found outside of the bounding box (this can happen when you view the cylinder along its axis, the intersection point will go infinitely far).
By the way it's a very hard task, if somebody is interested here's the source code:
https://github.com/chemlab/chemlab/blob/master/chemlab/graphics/renderers/shaders/cylinderimp.frag
https://github.com/chemlab/chemlab/blob/master/chemlab/graphics/renderers/shaders/cylinderimp.vert
A cylinder impostor can actually be done just the same way as a sphere, like Nicol Bolas did it in his tutorial. You can make a square facing the camera and colour it that it will look like a cylinder, just the same way as Nicol did it for spheres. And it's not that hard.
The way it is done is ray-tracing of course. Notice that a cylinder facing upwards in camera space is kinda easy to implement. For example intersection with the side can be projected to the xz plain, it's a 2D problem of a line intersecting with a circle. Getting the top and bottom isn't harder either, the z coordinate of the intersection is given, so you actually know the intersection point of the ray and the circle's plain, all you have to do is to check if its inside the circle. And basically, that's it, you get two points, and return the closer one (the normals are pretty trivial too).
And when it comes to an arbitrary axis, it turns out to be almost the same problem. When you solve equations at the fixed axis cylinder, you are solving them for a parameter that describes how long do you have to go from a given point in a given direction to reach the cylinder. From the "definition" of it, you should notice that this parameter doesn't change if you rotate the world. So you can rotate the arbitrary axis to become the y axis, solve the problem in a space where equations are easier, get the parameter for the line equation in that space, but return the result in camera space.
You can download the shaderfiles from here. Just an image of it in action:
The code where the magic happens (It's only long 'cos it's full of comments, but the code itself is max 50 lines):
void CylinderImpostor(out vec3 cameraPos, out vec3 cameraNormal)
{
// First get the camera space direction of the ray.
vec3 cameraPlanePos = vec3(mapping * max(cylRadius, cylHeight), 0.0) + cameraCylCenter;
vec3 cameraRayDirection = normalize(cameraPlanePos);
// Now transform data into Cylinder space wherethe cyl's symetry axis is up.
vec3 cylCenter = cameraToCylinder * cameraCylCenter;
vec3 rayDirection = normalize(cameraToCylinder * cameraPlanePos);
// We will have to return the one from the intersection of the ray and circles,
// and the ray and the side, that is closer to the camera. For that, we need to
// store the results of the computations.
vec3 circlePos, sidePos;
vec3 circleNormal, sideNormal;
bool circleIntersection = false, sideIntersection = false;
// First check if the ray intersects with the top or bottom circle
// Note that if the ray is parallel with the circles then we
// definitely won't get any intersection (but we would divide with 0).
if(rayDirection.y != 0.0){
// What we know here is that the distance of the point's y coord
// and the cylCenter is cylHeight, and the distance from the
// y axis is less than cylRadius. So we have to find a point
// which is on the line, and match these conditions.
// The equation for the y axis distances:
// rayDirection.y * t - cylCenter.y = +- cylHeight
// So t = (+-cylHeight + cylCenter.y) / rayDirection.y
// About selecting the one we need:
// - Both has to be positive, or no intersection is visible.
// - If both are positive, we need the smaller one.
float topT = (+cylHeight + cylCenter.y) / rayDirection.y;
float bottomT = (-cylHeight + cylCenter.y) / rayDirection.y;
if(topT > 0.0 && bottomT > 0.0){
float t = min(topT,bottomT);
// Now check for the x and z axis:
// If the intersection is inside the circle (so the distance on the xz plain of the point,
// and the center of circle is less than the radius), then its a point of the cylinder.
// But we can't yet return because we might get a point from the the cylinder side
// intersection that is closer to the camera.
vec3 intersection = rayDirection * t;
if( length(intersection.xz - cylCenter.xz) <= cylRadius ) {
// The value we will (optianally) return is in camera space.
circlePos = cameraRayDirection * t;
// This one is ugly, but i didn't have better idea.
circleNormal = length(circlePos - cameraCylCenter) <
length((circlePos - cameraCylCenter) + cylAxis) ? cylAxis : -cylAxis;
circleIntersection = true;
}
}
}
// Find the intersection of the ray and the cylinder's side
// The distance of the point and the y axis is sqrt(x^2 + z^2), which has to be equal to cylradius
// (rayDirection.x*t - cylCenter.x)^2 + (rayDirection.z*t - cylCenter.z)^2 = cylRadius^2
// So its a quadratic for t (A*t^2 + B*t + C = 0) where:
// A = rayDirection.x^2 + rayDirection.z^2 - if this is 0, we won't get any intersection
// B = -2*rayDirection.x*cylCenter.x - 2*rayDirection.z*cylCenter.z
// C = cylCenter.x^2 + cylCenter.z^2 - cylRadius^2
// It will give two results, we need the smaller one
float A = rayDirection.x*rayDirection.x + rayDirection.z*rayDirection.z;
if(A != 0.0) {
float B = -2*(rayDirection.x*cylCenter.x + rayDirection.z*cylCenter.z);
float C = cylCenter.x*cylCenter.x + cylCenter.z*cylCenter.z - cylRadius*cylRadius;
float det = (B * B) - (4 * A * C);
if(det >= 0.0){
float sqrtDet = sqrt(det);
float posT = (-B + sqrtDet)/(2*A);
float negT = (-B - sqrtDet)/(2*A);
float IntersectionT = min(posT, negT);
vec3 Intersect = rayDirection * IntersectionT;
if(abs(Intersect.y - cylCenter.y) < cylHeight){
// Again it's in camera space
sidePos = cameraRayDirection * IntersectionT;
sideNormal = normalize(sidePos - cameraCylCenter);
sideIntersection = true;
}
}
}
// Now get the results together:
if(sideIntersection && circleIntersection){
bool circle = length(circlePos) < length(sidePos);
cameraPos = circle ? circlePos : sidePos;
cameraNormal = circle ? circleNormal : sideNormal;
} else if(sideIntersection){
cameraPos = sidePos;
cameraNormal = sideNormal;
} else if(circleIntersection){
cameraPos = circlePos;
cameraNormal = circleNormal;
} else
discard;
}
From what I can understand of the paper, I would interpret it as follows.
An impostor cylinder, viewed from any angle has the following characteristics.
From the top, it is a circle. So considering you'll never need to view a cylinder top down, you don't need to render anything.
From the side, it is a rectangle. The pixel shader only needs to compute illumination as normal.
From any other angle, it is a rectangle (the same one computed in step 2) that curves. Its curvature can be modeled inside the pixel shader as the curvature of the top ellipse. This curvature can be considered as simply an offset of each "column" in texture space, depending on viewing angle. The minor axis of this ellipse can be computed by multiplying the major axis (thickness of the cylinder) with a factor of the current viewing angle (angle / 90), assuming that 0 means you're viewing the cylinder side-on.
Viewing angles. I have only taken the 0-90 case into account in the math below, but the other cases are trivially different.
Given the viewing angle (phi) and the diameter of the cylinder (a) here's how the shader needs to warp the Y-Axis in texture space Y = b' sin(phi). And b' = a * (phi / 90). The cases phi = 0 and phi = 90 should never be rendered.
Of course, I haven't taken the length of this cylinder into account - which would depend on your particular projection and is not an image-space problem.