OpenGL ,World to Object coordinate mapping ? (inverse matrix) - opengl

Question

Greetings all,
As seen in the image,
I have an object named O (set of linestripes).Its object-coordinate system is (x',y',z').
I translate,rotate this object in my OpenGL scene using following code snippet:
glPushMatrix();
glTranslatef(Oz, Oy,Oz);
glRotatef(rotationX , 1.0, 0.0, 0.0);
glRotatef(rotationY, 0.0, 1.0, 0.0);
glRotatef(rotationZ, 0.0, 0.0, 1.0);
contour->render();
glPopMatrix()
;
I have a point called H ,which is translated to (hx,hy,hz) world coordinates using
glPushMatrix();
glTranslatef(hx,hy,hz);
glPopMatrix();
If I am correct, (Oz,Oy,Oz) and (hx,hy,hz) are world coordinates.
Now,what I want todo is calculate the position of H (hx,hy,hz) relative to O's object-coordinate system.(x',y',z');
As I understood,I can do this by calculating inverse transformations of object O and apply them to point H.
Any tips on this? Does OpenGL gives any functions for inverse-matrix calculation ? If I somehow found inverse-matrices what the order of multiplying them ?
Note : I want to implement "hammer" like tool where at point H ,I draw a sphere with radius R.User can use this sphere to chop the object O like a hammer.I have implemented this in 2D ,so I can use the same algorithm if I can calculate the hammer position
relative to (x',y',z')
Thanks in advance.

Answer 1

Inverting the matrix would be the general solution, but as far as I can see this isn't actually a "general" problem. Rather than undoing an arbitrary transformation, you are trying to do the reverse of a known sequence of transformations, each of which can be inverted very simply. If your object-to-world transformation is:
glTranslatef(Ox, Oy, Oz);
glRotatef(rotationX , 1.0, 0.0, 0.0);
glRotatef(rotationY, 0.0, 1.0, 0.0);
glRotatef(rotationZ, 0.0, 0.0, 1.0);
Then the world-to-object inverse is just:
glRotatef(-rotationZ, 0.0, 0.0, 1.0);
glRotatef(-rotationY, 0.0, 1.0, 0.0);
glRotatef(-rotationX , 1.0, 0.0, 0.0);
glTranslatef(-Ox, -Oy, -Oz);
Basically, just back out each applied transformation in the opposite order originally applied.

Answer 2

Yes, basically you're right that you can perform this operation by the translation matrix
M = O^-1 * H
any like you already guessed you need the inverse of O for this. OpenGL is not a math library though, it only deals with rendering stuff. So you'll have to implement the inversion yourself. Google for "Gauss Jordan" to find one possible algorithm. If you can be absolutely sure, that O consists only of rotation and translation, i.e. no shearing or scaling, then you can shortcut by transposing the upper left 3x3 submatrix and negating the uppermost 3 elements of the rightmost column (this exploits the nature of orthogonal matrices, like rotation matrices, that the transpose is also the inverse, the upper left 3x3 is the rotational part, the inverse of a translation is negating the elements of it's vector which is the rightmost upper 3 elements).