glm rotation with scaling stretched aspect ratio - c++

Question

I am trying to rotate the texture image using glm,but the output looks stretched or not in the properly rotated along z axis. What might be the possible solution for this one.
float imgAspectRatio = imageWidth / (float) imageHeight;
float viewAspectRatio = viewWidth / (float) viewHeight;
if (imgAspectRatio > viewAspectRatio) {
yScale = viewAspectRatio / imgAspectRatio;
} else {
xScale = imgAspectRatio / viewAspectRatio;
}
glm::mat4 model(1.0f);
model = glm::translate(model, glm::vec3(0, 0, 0));
model = glm::rotate(model, glm::radians(angle),
glm::vec3(0, 0, 1));
model = glm::scale(model, glm::vec3(xScale, yScale, 1.0f));
Some suggested multiply with glm::ortho with but its giving square shape
glm::mat4 projection(1.0f);
projection = glm::ortho(
-imgAspectRatio,
imgAspectRatio,
-1.0f,
1.0f,
-1.0f,
1.0f
);

Answer 1

The code that you currently have would actually work if you had a square window.
By combining the image aspect ratio scaling and the view aspect ratio scaling into a single step, the order of operations you currently have is essentially as follows:
Scale image plane by image aspect ratio.
Scale all xy coordinates in your scene by the view aspect ratio.
Rotate image plane.
Translate image plane.
However, you really want to scale by the view aspect ratio at the very end. An easy exercise to help visualize why: imagine you have a perfectly square window in which you render a triangle with xy coordinates (-1, -1), (1, -1), (0, 1). Now suppose your window doubles in height, but you want the shape to stay the same: obviously, you just multiply each y-coordinate by 1/2. Now suppose you want to rotate the triangle by 90 degrees ccw, but again still keep the shape the same. Do you rotate first and then scale by the view aspect ratio? Or vice versa? By running through both options, it becomes clear that you have to scale by the view aspect ratio very last.
In other words, you need the following order:
Scale image plane by image aspect ratio.
Rotate image plane.
Translate image plane.
Scale all xy coordinates in your scene by the view aspect ratio.
Which is achieved by code that looks something like this:
float xScaleImg = 1.0f;
float yScaleImg = xScaleImg / imgAspectRatio;
float xScaleView = 1.0f;
float yScaleView = viewAspectRatio;
glm::mat4 model(1.0f);
model = glm::scale(model, glm::vec3(xScaleView, yScaleView, 1.0f));
model = glm::translate(model, glm::vec3(0, 0, 0));
model = glm::rotate(model, glm::radians(angle), glm::vec3(0, 0, 1));
model = glm::scale(model, glm::vec3(xScaleImg, yScaleImg, 1.0f));
The first four lines assume, as is the case in your example, that both the view and image widths are greater than the respective heights. You'll probably want to add some logic to change this around in the event that the reverse is true.
Update:
Depending on the effect you want to achieve for translation, you may want to move the glm::translate(...) command up one line. The order I gave in my original answer keeps the units of translation equal in pixels. E.g. if you pass in glm::vec3(1.0f, 1.0f, 0.0f), and the width of the window is, say, 1280 pixels, then the image will be translated 640 pixels to the left and 640 pixels upwards.
However, you may want to keep the OpenGL [-1, 1] range on both axes. That is, when you pass in glm::vec3(1.0f, 1.0f, 0.0f) for the translation, you may want the image to be translated right by half of the window's width and up by half of the window's height. In that case, you need to make the translation the last operation performed on the image, and this is done by making the glm::translate(...) the first line of code. This is what Nile Qor wanted. In this case, the last lines of the code become something like:
glm::mat4 model(1.0f);
model = glm::translate(model, glm::vec3(1.0f, 1.0f, 0));
model = glm::scale(model, glm::vec3(xScaleView, yScaleView, 1.0f));
model = glm::rotate(model, glm::radians(angle), glm::vec3(0, 0, 1));
model = glm::scale(model, glm::vec3(xScaleImg, yScaleImg, 1.0f));
For more context, you can see the discussion in the comments section of this answer.