Orient object along surface normal - opengl

Question

When the user clicks on a surface I would like place an object at this position and orient it perpendicular to the surface normal.
After the user performs a click, I read the depth of three neighboring pixels from the buffer, unproject the pixels from screen coordinates to object space and then compute the surface normal from these points in object space:
glReadPixels(mouseX, mouseY, ..., &depthCenter);
pointCenter = gluUnProject(mouseX, mouseY, depthCenter, ...);
glReadPixels(mouseX, mouseY - 1, ..., &depthUp);
pointUp = gluUnProject(mouseX, mouseY - 1, depthUp, ...);
glReadPixels(mouseX - 1, mouseY, ..., &depthLeft);
pointLeft = gluUnProject(mouseX - 1, mouseY, depthLeft, ...);
centerUpVec = norm( pointCenter - pointUp );
centerLeftVec = norm( pointCenter - pointLeft );
normalVec = norm( centerUpVec.cross(centerLeftVec) );
I know that computing the normal just from three pixels is problematic (e.g. at edges or if the three points have vastly different depth), but for my initial test on a flat surface this must suffice.
Finally, in order to orient the object along the computed normal vector I create a rotation matrix from the normal and the up vector:
upVec = vec(0.0f, 1.0f, 0.0f);
xAxis = norm( upVec.cross(normalVec) );
yAxis = norm( normalVec.cross(xAxis) );
// set orientation of model matrix
modelMat(0,0) = xAxis(0);
modelMat(1,0) = yAxis(0);
modelMat(2,0) = normalVec(0);
modelMat(0,1) = xAxis(1);
modelMat(1,1) = yAxis(1);
modelMat(2,1) = normalVec(1);
modelMat(0,2) = xAxis(2);
modelMat(1,2) = yAxis(2);
modelMat(2,2) = normalVec(2);
// set position of model matrix by using the previously computed center-point
modelMat(0,3) = pointCenter(0);
modelMat(1,3) = pointCenter(1);
modelMat(2,3) = pointCenter(2);
For testing purposes I'm placing an objects on a flat surface after each click. This works well in most cases when my camera is facing downwards the up vector.
However, once I rotate my camera the placed objects are oriented arbitrarily and I can't figure out why!

Answer 1

Ok, I just found a small, stupid bug in my code that was unrelated to the actual problem. Therefore, the approach stated in the question above is working correctly.
In order to avoid some pitfalls, one could of course just use a math library, such as Eigen, in order to compute the rotation between the up vector and the surface normal:
upVec = Eigen::Vector3f(0.0f, 1.0f, 0.0f);
Eigen::Quaternion<float> rotationQuat;
rotationQuat.setFromTwoVectors(upVec, normalVec);