OpenCV image stacking - Alignment is off - c++

Question

I have some code that takes multiple images, aligns them, and stacks them together. For some reason, the alignment is off. A simplified version of the code is below
void stackImages(uint8_t **pixels, uint32_t width, uint32_t height, size_t len)
{
cv:: Mat firstImg;
cv::Mat stacked;
for (int i = 0; i < len; i++)
{
// Transformation matrix
cv::Mat1f M = cv::Mat1f(cv::Mat::eye(3, 3, CV_8UC1));
// Convert pixels (4 channel RGBA) to Mat
cv::Mat pixels = cv::Mat(height, width, CV_8UC4, pixels[i]);
cv::Mat gray;
cv::cvtColor(pixels, gray, cv::COLOR_RGBA2GRAY);
// skip the reference image
if(!i) {
firstImg = gray;
stacked = gray;
continue;
}
cv::Mat warped;
// create size struct
cv::Size size;
size.width = width;
size.height = height;
// create the transformation matrix
cv::findTransformECC(firstImg, gray, M, cv::MOTION_HOMOGRAPHY);
// warp the image according ot the transformation matrix
cv::warpPerspective(gray, warped, M, size);
// stack the image
stacked += warped;
}
// write the image
cv::imwrite("stacked.jpg", stacked);
}
I've tested this code with three images taken in rapid succession and the results are below. This is my first foray into image processing, so I'm mostly following online documentation.