I have a few questions on the usage of these two and how they effect textures.
What exactly is the ordering of the pixels? The pixels are provided via a buffer that's continuous. In what order does the texture read them so that they are distributed into the x and y planes?
Also these methods accept a type parameter for the data, does each pixel get 3 values? Because when we use the texture(uvMap, textureUV) method in the shader, it returns a vec3 of floats. So how exactly the data we provide to the texture via glteximage2d and gltexsubimage2d is read and organized in open gl?
Here is my assumption, correct me if I'm wrong:
The data buffer contains the pixel data. Each pixel is represented by the 3 values of the type sent in via glteximage2d and gltexsubimage2d methods. So the buffer we provide needs to have 3 * width * height number of values in it. OpenGL reads the buffer as follows(pseudo-code):
for(int y range 0 and height)
for(int x range 0 and width)
int index= (y * (width * 3)) + (x * 3);
pixels[y][x].x = buffer[index + 0];
pixels[y][x].y = buffer[index + 1];
pixels[y][x].z = buffer[index + 2];
The reference is very clear about that:
glTexImage2D
Description
The last three arguments (format, type, data) describe how the image is represented in memory.
If target is GL_TEXTURE_2D, ..., data is read from data as a sequence of signed or unsigned bytes, shorts, or longs, or single-precision floating-point values, depending on type. These values are grouped into sets of one, two, three, or four values, depending on format, to form elements.
The first element corresponds to the lower left corner of the texture image. Subsequent elements progress left-to-right through the remaining texels in the lowest row of the texture image, and then in successively higher rows of the texture image. The final element corresponds to the upper right corner of the texture image.
format determines the composition of each element in data. It can assume one of these symbolic values: ... (to be brief, i will not list the elements here, but it is a must read)
Related
I am looking for a solution to easily compute the pixel coordinate from two images.
Question: If you take the following code, how could I compute the pixel coordinate that changed from the "QVector difference" ? Is it possible to have an (x,y) coordinate and find on the currentImage which pixel it represents ?
char *previousImage;
char *currentImage;
QVector difference<LONG>;
for(int i = 0 ; i < CurrentImageSize; i++)
{
//Check if pixels are the same (we can also do it with RGB values, this is just for the example)
if(previousImagePixel != currentImagePixel)
{
difference.push_back(currentImage - previousImage);
}
currentImage++;
}
EDIT:
More information about this topic:
The image is in RGB format
The width, the height and the bpp of both images are known
I have a pointer to the bytes representing the image
The main objective here is to clearly know what is the new value of a pixel that changed between the two images and to know which pixel is it (its coordinates)
There is not enough information to answer, but I will try to give you some idea.
You have declared char *previousImage;, which implies to me that you have a pointer to the bytes representing an image. You need more than that to interpret the image.
You need to know the pixel format. You mention RGB, So -- for the time being, let's assume that the image uses 3 bytes for each pixel and the order is RGB
You need to know the width of the image.
Given the above 2, you can calculate the "Row Stride", which is the number of bytes that a row takes up. This is usually the "bytes per pixel" * "image width", but it is typically padded out to be divisible by 4. So 3 bpp and a width of 15, would be 45 bytes + 3 bytes of padding to make the row stride 48.
Given that, if you have an index into the image data, you first integer-divide it against the row stride to get the row (Y coordinate).
The X coordinate is the (index mod the row stride) integer-divided by the bytes per pixel.
From what I understand, you want compute the displacement or motion that occured between two images. E.g. for each pixel I(x, y, t=previous) in previousImage, you want to know where it did go in currentImage, and what is his new coordinate I(x, y, t=current).
If that is the case, then it's called motion estimation and measuring the optical flow. There are many algorithms for that, who rely on more or less complex hypotheses, depending on the objects you observe in the image sequence.
The simpliest hypothesis is that if you follow a moving pixel I(x, y, t) in the scene you observe, its luminance will remain constant over time. In other words, dI(x,y,t) / dt = 0.
Since I(x, y, t) is function of three parameters (space and time) with two unknowns, and there is only one equation, this is an ill defined problem that has no easy solution. Many of the algorithms add an additional hypothesis, so that the problem can be solved with a unique solution.
You can use existing libraries which will do that for you, one of them which is pretty popular is openCV.
I want to add up all channels of a Mat image to a Mat image with only one sum-channel. I've tried it this way:
// sum up the channels of the image:
// 1 .store initial nr of rows/columns
int initialRows = frameVid1.rows;
int initialCols = frameVid1.cols;
// 2. check if matrix is continous
if (!frameVid1.isContinuous())
{
frameVid1 = frameVid1.clone();
}
// 3. reshape matrix to 3 color vectors
frameVid1 = frameVid1.reshape(3, initialRows*initialCols);
// 4. convert matrix to store bigger values than 255
frameVid1.convertTo(frameVid1, CV_32F);
// 5. sum up the three color vectors
reduce(frameVid1, frameVid1, 1, CV_REDUCE_SUM);
// 6. reshape to initial size
frameVid1 = frameVid1.reshape(1, initialRows);
// 7. convert back to CV_8UC1
frameVid1.convertTo(frameVid1, CV_8U);
But somehow reduce does not touch the color channels as a Matrix Dimension. Is there another function that can sum them up?
Also why does using CV_16U in step 4.) not work? (I had to put a CV_32F in there)
Thanks in advance!
You can sum the RGB channels with a single line
cv::transform(frameVid1, frameVidSum, cv::Matx13f(1,1,1))
You may need one more line, as before applying the transform you shall convert the image to some appropriate type to avoid saturation (I assumed CV_32FC3). -Output array is of the same size and depth as source.
Some explanation:
cv::transform may operate on per-pixel channel values.
Having the third argument cv::Matx13f(a, b, c) for each pixel [u,v] it does the following:
frameVidSum[u,v] = frameVid1[u,v].B * a + frameVid1[u,v].G * b + frameVid1[u,v].R * c
By using third argument cv::Matx13f(1,0,1) you will sum only blue and red channels.
cv::transform is so clever, you can even use cv::Matx14f and then the fourth value will be added (offset) to each pixel in the frameVidSum.
Every 3rd element (in RGB) is one similar colour. Probably it will work if you grab every group of 3 elements (R, G and B) sum them up and store it in another 1-channel matrix. Before storing you should use saturate cast to avoid unexpected results. So, I think the better way is to use saturate cast instead of adapting your matrix.
Have a look at cv::split() and cv::add() functions.
You can use the split function to split the image into separate channels and then the add function to add the images. But be careful when using add because adding may lead to saturation of values. You may have to first convert types and then add. Have a look here: http://answers.opencv.org/question/13769/adding-matrices-without-saturation/
`Obtain the stride (the number of bytes between pixels on different rows)
screen_get_buffer_property_iv(mScreenPixelBuffer, SCREEN_PROPERTY_STRIDE, &mStride)`
I don't understand what the first line meant about having bytes between pixels on different rows. The function is what the stride is obtained through.
If we have a rectangular bunch of pixels (a screen, bitmap, or some such), there must be a way for a program to calculate the position of a pixel. Lets call this sort of bunch of pixels a "surface".
The surface can be split into individual pixels, and we could just put then in a very long row and number then from 0 to some large number (e.g. a 1280 x 1024 screen would have 1310720 pixels). But if you show this long row of pixels on a screen, it makes more sense to talk about lines of pixels that are 1280 pixels long, and have 1024 rows of them.
Now, let's say we want to draw a line from pixels 100,100 to 100,200. We can easily write that as:
int i;
for(i = 100; i < 200; i++)
{
setpixel(surface, 100, 100+i, colour);
}
Now, if we want to implement setpixel, what do we need to do? One thing would be to translate our x, y coordinates (100, 100+i) into a location of our "long row of pixels".
The general formula tends to be (x + y * width) * bytes_per_pixel. So if we have a 32bpp image (four bytes per pixel), that would make (100 + (100+i) * 1280) * 4
However, to make it easier to design the graphics chip there are often limits on things like "the width of a surface must be an even multiple of X", where X is usually 16, 32, 64 or some other power of 2. Sometimes, it has to be a power of two directly (for example textures in early opengl can only be 2^n x 2^n pixels in size - you don't have to USE the entire texture). And this is where stride comes in.
Say we want to have a bitmap of 100 x 100 pixels. But the graphics chip that we use to draw the bitmap to the screen has a rule that you MUST have a even multiple of 32 pixels wide surfaces. So we make something like this
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
XXXXXXXXXX...
The X's here represent the actual pixels (10 per X) in our bitmap, and the ... 28 pixels of "waste" that we have to have to make the graphics chip happy.
Now the formula of using width doesn't work, because from the software creating the bitmap, the width is 100 pixels. We need to change the math to make up for the "extra space at the end of each row of pixels":
(x + y * stride) * bytes_per_pixel
Now, the stride is 128, but the width is 100 pixels.
Stride here refers to array stride, the number of bytes between memory locations that correspond to the beginning of adjacent rows of an array, in this case of pixels.
In a fully packed array, the stride equals the size of an individual pixel multiplied with the number of pixels in the row. For performance reasons, arrays are frequently aligned so that each row takes a "round" number of bytes, typically an exponent of two. The byte size of the row, aka the stride, cannot be computed from other array parameters and must be known in order to correctly calculate the memory position of an arbitrary pixel.
I have a c/c++ program running on my Linux box that loads up a 24bit bitmap, reads the two headers and then stores the image data into a char* variable. I have verified this works by dumping that variables contents into a raw binary file and compared it to the original bitmap+offset. I used the code from HERE and unmodified and takes care or reordering into RGB and bottom up.
Now if I have a list of coordinates like X, Y, Width, Height how the heck do I translate these into the byte offsets of my image?!
In MY CODE you see that I am calculating the width of one scanline and the glyph location to find Y and then adding a scanline for each y+1. Similarly for X I am iterating over by three bytes at a time. And finally I store those three bytes sequentially into my temporary character array.
In truth I do not need the pixel data as the glyph is a 0xFF or 0x00 with no smoothing. I included it to make sure my bits where accounted for.
HERE is the image I am using.
EDIT: --------------------------------------------
As mentioned below my math was a bit quarky. fixed the line in the i,j,k loop to:
tmpChar[i][j][k] = img.data[(((Y+j) * imgWidth) + (X + i)) * imgBPP + k];
As for my programs output HERE as you can see it loads the bitmap fine and the header info is proper but when I try to display the contents of the tmpChar array its all 0xFF (I used a signed int so 0xFF = -1 and 0x00 = +0)
The layout in memory of the image is (ignoring that I might have reversed R, G and B):
[R of pixel 0] [G of pixel 0] [B of pixel 0] ....... [B of (0, imgWidth-1)] [R of pixel (1, 0)] .....
So to calculate the offset of any given pixel: offset = ((Y * imgWidth) + X) * imgBPP + colorByte.
Giving for your inner loop, as far as I can tell and assuming your X and Y for the character are correct:
tmpChar[i][j][k] = img.data[(((Y+j) * imgWidth) + (x + i)) * imgBPP + k];
I guess that the pixels are stored in a upside-down order in memory, as is usual with BMP file format:
Normally pixels are stored "upside-down" with respect to normal image
raster scan order, starting in the lower left corner, going from left
to right, and then row by row from the bottom to the top of the image
So your code may be reading the wrong block of pixels.
I know this may sound stupid but I'm goin crazy with this XD
I'm loading ad image (with ImageMagick) into a 1D vector, so that I have something like:
012345678...
RGBRGBRGB...
Where 0-. Are obviously the indexes of the vector, and R, G, and B are respectively the red byte, green byte, and blue byte.
So I have a WIDTHxHEIGHTx3 bytes vector.
Now, let's say I want to access the x,y,z byte, where z is the index of the color, which is the transformation formula to have a linear offset into the vector?
This expression produces an index to color component z at pixel (x,y):
((y * WIDTH) + x) * 3 + z
Assumptions are:
Data is placed in row-major order.
No padding/alignment bytes are used between rows.
Assuming your data is stored as a series of rows (not a crazy assumption), you can find byte x,y,z at y*WIDTH*3 + 3*x + z