I have been transforming from screen space to world space like this:
//convert the window coordinates from gl_fragCoord to NDC coordinates:
ndc.x = 2.0*((gl_FragCoord.x)/width-0.5);
ndc.y = 2.0*((gl_FragCoord.y)/height-0.5);
ndc.z = (2.0*gl_FragCoord.z-far-near)/(far-near);
//convert the NDC coordinates to clip coordinates - gl_FragCoord.w = 1/w_clip:
clip = vec4(ndc,1.0)/gl_FragCoord.w;
//convert the clip coordinates to eye coordinates:
eye = invproject*clip;
//convert eye coordinates to world space (with inverse view matrix)
objectpos = invmodel*eye;
And this works fine, but the problem is that I am trying to perform some antialiasing, where I offset the screen coordinate by som random value, so that the new coordinate is still within the screen pixel. I imagined that this could be done like this:
ndc.x = 2.0*((gl_FragCoord.x+half_pixelsize*random.x)/width-0.5);
ndc.y = 2.0*((gl_FragCoord.y+half_pixelsize*random.y)/height-0.5);
ndc.z = (2.0*gl_FragCoord.z-far-near)/(far-near);
...
But then the problem is that I am still using the same z-coordinate, which should be different in eye space when changing the coordinates in screen space.
I've been told that this can be handled by reprojecting the z-coordinate: If one is given the original point, and the normal then one should reproject the distance from the original point to the new point, by using pythagoras. But I can't really understand what is meant by it, or if there maybe is another way to deal with this problem?
Related
I see objects beyond the far clipping plane in perspective projection and I don't think this is how it's suppose to work, so can someone give me an explanation why do I see objects beyond the far clipping plane such as a grid in this example.
The orthogonal projections works fine btw
I cleared all shapes from this demo and added two grids by changing the following code in Luna Frank Shapes Demo
void ShapesApp::BuildRenderItems()
{
auto gridRitem = std::make_unique<RenderItem>();
gridRitem->World = MathHelper::Identity4x4();
gridRitem->ObjCBIndex = 0;
gridRitem->Geo = mGeometries["shapeGeo"].get();
gridRitem->PrimitiveType = D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
gridRitem->IndexCount = gridRitem->Geo->DrawArgs["grid"].IndexCount;
gridRitem->StartIndexLocation = gridRitem->Geo->DrawArgs["grid"].StartIndexLocation;
gridRitem->BaseVertexLocation = gridRitem->Geo->DrawArgs["grid"].BaseVertexLocation;
mAllRitems.push_back(std::move(gridRitem));
gridRitem = std::make_unique<RenderItem>();
XMStoreFloat4x4(&gridRitem->World, XMMatrixTranslation(0,-1002,0)* XMMatrixRotationRollPitchYaw(1.5708, 0, 0));;
gridRitem->ObjCBIndex = 1;
gridRitem->Geo = mGeometries["shapeGeo"].get();
gridRitem->PrimitiveType = D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
gridRitem->IndexCount = gridRitem->Geo->DrawArgs["grid"].IndexCount;
gridRitem->StartIndexLocation = gridRitem->Geo->DrawArgs["grid"].StartIndexLocation;
gridRitem->BaseVertexLocation = gridRitem->Geo->DrawArgs["grid"].BaseVertexLocation;
mAllRitems.push_back(std::move(gridRitem));
gridRitem = std::make_unique<RenderItem>();
XMStoreFloat4x4(&gridRitem->World, XMMatrixTranslation(0, -1002, 0) * XMMatrixRotationRollPitchYaw(1.5708, 1.5708, 0));
gridRitem->ObjCBIndex = 2;
gridRitem->Geo = mGeometries["shapeGeo"].get();
gridRitem->PrimitiveType = D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
gridRitem->IndexCount = gridRitem->Geo->DrawArgs["grid"].IndexCount;
gridRitem->StartIndexLocation = gridRitem->Geo->DrawArgs["grid"].StartIndexLocation;
gridRitem->BaseVertexLocation = gridRitem->Geo->DrawArgs["grid"].BaseVertexLocation;
mAllRitems.push_back(std::move(gridRitem));
and change the grid size
GeometryGenerator::MeshData grid = geoGen.CreateGrid(200.0f, 200.0f, 60, 40);
and the projection matrix from on resize
XMMATRIX P = XMMatrixPerspectiveFovLH(0.25f * MathHelper::Pi, AspectRatio(), .1f, 1000.f);
XMStoreFloat4x4(&mProj, P);
Now I can still see the grid even though its beyond the far plane and even if the far plane is 900 the still appears on rotation at the edge of the screen. so I need to reduce the far plane or move the grid further, and as I keep changing the value of the far plane I can see a shape works as a brush, it hides everything beyond it but as camera rotate and the grid is no longer behind it the grid re-appear
and here's what I meant by the brush
I think you're thinking of the maximum view distance as being consistently 900 units away from the camera/eye position. If that was the case, it wouldn't be a clipping plane at all, it would be a curve - a sector of a sphere.
In reality the view frustum is a truncated pyramid made up of 6 planes. When the far plane is set to 900, then the view distance for the pixel in the centre of the view is 900, but the view distance at the corners is much higher (how much higher depends on the FOVs - you could work it out with a bit of trig).
So as you turn your camera left and right, an object approx 900 units away from the camera will come in and out of view as it intersects the far plane.
I've implemented a fps camera based on the up, right and view vectors from this.
Right now I want to be able to interact with the world by placing cubes in a minecraft style.
My lookAt vector is the sum of the view vector and the camera position, so my first attempt was to draw a cube at lookAt, but this is causing a strange behaviour.
I compute every vector like in the web I mentioned (such that lookAt = camera_position + view_direction) but the cube drawn is always arround me. I've tried several things like actually placing it (rounding the lookAt) and it appears near the wanted position but not at the place i'm looking at.
Given these vectors, how can I draw that's centered at the position that my camera is looking but a little bit further (exactly like minecraft)?
but the cube drawn is always arround me.
Yeah and that's obvious. You place cubes on the sphere surface of radius view_direction with center at camera_position.
Given these vectors, how can I draw that's centered at the position
that my camera is looking but a little bit further (exactly like
minecraft)?
You need to place cubes at the intersection of the view vector with the scene geometry. In the simplest case, it can be just "ground" plane, so you need intersect view vector with "ground" plane. Then you need to round the intersection xyz coordinates to the nearest grid node xyz = round(xyz / cubexyz)*cubexyz where cubexyz - cube size.
Approximate code:
Vector3D intersectPoint(Vector3D rayVector, Vector3D rayPoint, Vector3D planeNormal, Vector3D planePoint) {
Vector3D diff = rayPoint - planePoint;
double prod1 = diff.dot(planeNormal);
double prod2 = rayVector.dot(planeNormal);
double prod3 = prod1 / prod2;
return rayPoint - rayVector * prod3;
}
.......
Vector3D cubePos = intersectPoint(view_direction, camera_position, Vector3D(0, 1, 0), Vector3D(0, 0, 0));
cubePos = round(cubePos / cubeSize) * cubeSize;
AddCube(cubePos);
It's hard to tell without having images to look at, but lookAt is most likely your normalized forward vector? If i understood you correctly, you'd want to do something like objectpos = camerapos + forward * 10f (where 10f is the distance you want to place the object in front of you in 3d space units) to make sure that it's placed a few units in front of your fps controller.
actually, if view_direction is your normalized forward vector and your lookAt is camera_pos + view_direction, then you'd end up with something very close to your camera position, which would explain why the cube spawns inside you. either way, my suggestion should still work :)
perspectiveCamera = new PerspectiveCamera(90, 80, 48);
perspectiveCamera.position.set(0,0, 10f);
perspectiveCamera.lookAt(0,0,0);
perspectiveCamera.near = .01f;
perspectiveCamera.far = 300f;
My ScreenWidth x ScreenHeight are 800 x 480;
pCamera.unproject(mytouchPoint) shall suppose to give results between
x = 0 to 80
y = 0 to 48
but I m getting 0.000xyz for both x and y axis
Don't use such a small value for your camera's near member, it will cause floating point errors and/or z-fighting.
The width and height values you provided to PerspectiveCamera constructor, are used to calculate the aspect ratio. There is no single 2D resolution (the size of the screen-plane in world coordinates) in a 3D perspective.
You cannot simply unproject a 2D screen coordinate to a single 3D coordinate. For each 2D screen coordinate, there are an "infinite" amount of 3D coordinates possible. Therefor the unproject method of the camera will use the z-coordinate of the provided screen coordinate to decide which of those 3D coordinates to return. If z is zero, it will give the coordinate on the near-plane. If z is one, it will give the coordinate on the far-plane.
Assuming you used z=0 for myTouchPoint and given you have a very small near-plane (since you near value is very small), the unprojected value will be vary small and therefor (almost) equal to zero.
For more information, you might want to have a look at: http://blog.xoppa.com/interacting-with-3d-objects/
I found a way to easily do it. Its fast too.
Just create a plane at required depth z and find intersection of ray on it.
float zDepth=-10;//your decision or and object z position
public boolean touchDown(int screenX, int screenY, int pointer, int button) {
Ray ray = camera.getPickRay(screenX,screenY);
Plane plane=new Plane();
plane.set(0,0,1,0);// the xy plane with direction z facing screen
plane.d=zDepth;//***** the depth in 3d for the coordinates
Vector3 yourVector3Position=new Vector3();
Intersector.intersectRayPlane(ray, plane, yourVector3Position);
}
I am trying to render views of a 3D mesh in VTK, I am doing the following:
vtkSmartPointer<vtkRenderWindow> render_win = vtkSmartPointer<vtkRenderWindow>::New();
vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
render_win->AddRenderer(renderer);
render_win->SetSize(640, 480);
vtkSmartPointer<vtkCamera> cam = vtkSmartPointer<vtkCamera>::New();
cam->SetPosition(50, 50, 50);
cam->SetFocalPoint(0, 0, 0);
cam->SetViewUp(0, 1, 0);
cam->Modified();
vtkSmartPointer<vtkActor> actor_view = vtkSmartPointer<vtkActor>::New();
actor_view->SetMapper(mapper);
renderer->SetActiveCamera(cam);
renderer->AddActor(actor_view);
render_win->Render();
I am trying to simulate a rendering from a calibrated Kinect, for which I know the intrinsic parameters. How can I set the intrinsic parameters (focal length and principle point) to the vtkCamera.
I wish to do this so that the 2d pixel - 3d camera coordinate would be the same as if the image were taken from a kinect.
Hopefully this will help others trying to convert standard pinhole camera parameters to a vtkCamera: I created a gist showing how to do the full conversion. I verified that the world points project to the correct location in the rendered image. The key code from the gist is pasted below.
gist: https://gist.github.com/decrispell/fc4b69f6bedf07a3425b
// apply the transform to scene objects
camera->SetModelTransformMatrix( camera_RT );
// the camera can stay at the origin because we are transforming the scene objects
camera->SetPosition(0, 0, 0);
// look in the +Z direction of the camera coordinate system
camera->SetFocalPoint(0, 0, 1);
// the camera Y axis points down
camera->SetViewUp(0,-1,0);
// ensure the relevant range of depths are rendered
camera->SetClippingRange(depth_min, depth_max);
// convert the principal point to window center (normalized coordinate system) and set it
double wcx = -2*(principal_pt.x() - double(nx)/2) / nx;
double wcy = 2*(principal_pt.y() - double(ny)/2) / ny;
camera->SetWindowCenter(wcx, wcy);
// convert the focal length to view angle and set it
double view_angle = vnl_math::deg_per_rad * (2.0 * std::atan2( ny/2.0, focal_len ));
std::cout << "view_angle = " << view_angle << std::endl;
camera->SetViewAngle( view_angle );
I too am using VTK to simulate the view from a kinect sensor. I am using VTK 6.1.0. I know this question is old, but hopefully my answer may help someone else.
The question is how can we set a projection matrix to map world coordinates to clip coordinates. For more info on that see this OpenGL explanation.
I use a Perspective Projection Matrix to simulate the kinect sensor. To control the intrinsic parameters you can use the following member functions of vtkCamera.
double fov = 60.0, np = 0.5, fp = 10; // the values I use
cam->SetViewAngle( fov ); // vertical field of view angle
cam->SetClippingRange( np, fp ); // near and far clipping planes
In order to give you a sense of what that may look like. I have an old project that I did completely in C++ and OpenGL in which I set the Perspective Projection Matrix similar to how I described, grabbed the z-buffer, and then reprojected the points out onto a scene that I viewed from a different camera. (The visualized point cloud looks noisy because I also simulated noise).
If you need your own custom Projection Matrix that isn't the Perspective flavor. I believe it is:
cam->SetUserTransform( transform ); // transform is a pointer to type vtkHomogeneousTransform
However, I have not used the SetUserTransform method.
This thread was super useful to me for setting camera intrinsics in VTK, especially decrispell's answer. To be complete, however, one case is missing: if the focal length in the x and y directions are not equal. This can easily be added to the code by using the SetUserTransform method. Below is a sample code in python :
cam = self.renderer.GetActiveCamera()
m = np.eye(4)
m[0,0] = 1.0*fx/fy
t = vtk.vtkTransform()
t.SetMatrix(m.flatten())
cam.SetUserTransform(t)
where fx and fy are the x and y focal length in pixels, i.e. the two first diagnoal elements of the intrinsic camera matrix. np is and alias for the numpy import.
Here is a gist showing the full solution in python (without extrinsics for simplicity). It places a sphere at a given 3D position, renders the scene into an image after setting the camera intrinsics, and then displays a red circle at the projection of the sphere center on the image plane: https://gist.github.com/benoitrosa/ffdb96eae376503dba5ee56f28fa0943
In my scene I have terrain that I want to "grab" and then have the camera pan (with its height, view vector, field of view, etc. all remaining the same) as I move the cursor.
So the initial "grab" point will be the working point in world space, and I'd like that point to remain under the cursor as I drag.
My current solution is to take the previous and current screen points, unproject them, subtract one from the other, and translate my camera with that vector. This is close to what I want, but the cursor doesn't stay exactly over the initial scene position, which can be problematic if you start near the edge of the terrain.
// Calculate scene points
MthPoint3D current_scene_point =
camera->screenToScene(current_point.x, current_point.y);
MthPoint3D previous_scene_point =
camera->screenToScene(previous_point.x, previous_point.y);
// Make sure the cursor didn't go off the terrain
if (current_scene_point.x != MAX_FLOAT &&
previous_scene_point.x != MAX_FLOAT)
{
// Move the camera to match the distance
// covered by the cursor in the scene
camera->translate(
MthVector3D(
previous_scene_point.x - current_scene_point.x,
previous_scene_point.y - current_scene_point.y,
0.0));
}
Any ideas are appreciated.
With some more sleep :
Get the initial position of your intersected point, in world space and in model space ( relative to the model's origin)
i.e use screenToScene()
Create a ray that goes from the camera through the mouse position : {ray.start, ray.dir}
ray.start is camera.pos, ray.dir is (screenToScene() - camera.pos)
Solve NewPos = ray.start + x * ray.dir knowing that NewPos.y = initialpos_worldspace.y;
-> ray.start.y + x*ray.dir.y = initialpos_worldspace.y
-> x = ( initialpos_worldspace.y - ray.start.y)/rad.dir.y (beware of dividebyzeroexception)
-> reinject x in NewPos_worldspace = ray.start + x * ray.dir
substract initialpos_modelspace from that to "re-center" the model
The last bit seems suspect, though.