image tile registrations - computer-vision

Question

I'm trying to create a mapping for tile/pixel positions in one tiled image dataset to tile/pixel position in a different dataset (that roughly cover the same image area). Due to rotations of the imaged area, panning axis, translations, and scale changes, there are many unknowns to account for in this mapping. This situation is illustrated below, where the rectangles represent the imaged area for each tile, and the layout of the tiles is governed by the relationship between the panning axis (black lines) and the camera axis:
The problem reduces down to the following linear system which takes into account the scanning axis being rotated with respect to the camera axis differently in the two datasets, plus rotation,scale,and translations between the imaged areas in the two datasets. Unfortunately I'm not sure how to go about solving for the unknowns:
|tx ty|X|a b|+|px py|X|e f|+|i j|=|tx* ty*|X|k l|+|px* py*|
|c d| |g h| |m n|
The unknowns in this system are (a,b,c,d,e,f,g,h,i,j,k,l,m,n) and basically describe a mapping from tile and pixel positions in dataset1 (tx,ty,px,py) to tile and pixel positions in dataset2(tx*,ty*,px*,py*). Coming up with 14 (or more) such correspondences isn't too big a deal for me, as I have a log-polar image registration that works pretty well for affine mapping a tile from dataset1 to a tile in dataset2 if they contain sufficient image overlap. The problem is I'd rather get the above mapping determined rather than doing this registration between every single tile in dataset1 and 2 would take forever!!.
Any help greatly appreciated!
-Craig
Edit:
I just figured out I can rearrange the problem like so (I think?):
|tx ty px py 1|X|a b|=|tx* ty* px* py* 1|X|k l|
|c d| |m n|
|e f| |1 0|
|g h| |0 1|
|i j| |1 1|
This is looking closer to what I need, but I still need help trying to figure out how to convert the answer I'd get back into a form I could use (most importantly I think I'd need to explicitly know (k,l,m,n) to convert the mapped coordinate back into a tile and pixel position in the target.... no?
Edit2: Changed the notation to be correct for matrix multiplication
Edit3: Added illustration images since I now can!

Answer 1

OK, I've long since figured out the problem, but am posting here for documentation.
|tx ty px py 1|X|a b|=|tx* ty* px* py* 1|X|k l|
|c d| |m n|
|e f| |1 0|
|g h| |0 1|
|i j| |1 1|
can be rewritten conveniently as:
|tx ty px py| X A X B + T = |tx* ty* px* py*| X C
where the matrices A and C describe the transform from the slide coordinate system to a global |x y| coordinate system, and 'B + T' is an affine transform |x y| -> |x* y*|
rewritten again:
|tx ty px py 1 tx* ty*| X |A*| = |px* py*|
|B |
|T |
|C*|
shows that
|B| == |e f|
|T| |g h|
|i j|
A* X inv(B) = |a b|
|c d|
-C* = |k l|
|m n|
To determine the linear least-squares solution to the problem from correspondences:
points on slide 1
tp1 = [[tilex,tiley,pixelx,pixely],
...,]
to corresponding points on slide 2
tp2 = similar as above
def findABC(tp1,tp2):
dp1 = np.hstack([tp1,np.ones([len(tp1),1]),tp2[:,0:2]])
dp2 = tp2[:,2:4]
E,_,rank,_ = np.linalg.lstsq(dp1,dp2)
if rank == 7:
B = E[2:5]
A = np.dot(E[0:2,0:2],np.linalg.inv(E[2:4,0:2]))
C = -E[5:7]
return A,B,C
def SlideToGlobal(X,tp):
return np.dot(tp,X)
def GlobalToSlide(X,p):
tptp = np.dot(p,n.linalg.inv(X[0:2,0:2]))
tpoints = np.floor(tptp)
ppoints = tptp-np.dot(tpoints,X[0:2,0:2])
tp = np.hstack([tpoints,ppoints])
return tp