I'm created and loaded this image:
int x, y, comps;
unsigned char* data = stbi_load(".//textures//heightMapTexture.png", &x, &y, &comps, 1);
Now, how do i get a RGB of a certain pixel of this image?
You are using the 8-bits-per-channel interface. Also, you are requesting only one channel (the last argument given to stbi_load). You won't obtain RGB data with only one channel requested.
If you work with rgb images, you will probably get 3 or 4 in comps and you want to have at least 3 in the last argument.
The data buffer returned by stbi_load will containt 8bits * x * y * channelRequested , or x * y * channelCount bytes.
you can access the (i, j) pixel info as such:
unsigned bytePerPixel = channelCount;
unsigned char* pixelOffset = data + (i + x * j) * bytePerPixel;
unsigned char r = pixelOffset[0];
unsigned char g = pixelOffset[1];
unsigned char b = pixelOffset[2];
unsigned char a = channelCount >= 4 ? pixelOffset[3] : 0xff;
That way you can have your RGB(A) per-pixel data.
Related
Below is te RGB output of a supposed YUV420SP buffer. No conversion, I' m just displaying the YUV420SP as if it were RGB, just to see some patterns.
The image is in a single unsigned char* buffer of size width*height*3. So if this is indeed YUV420SP, then I should have the Y as a black and white image, and then UV interleaved. I think I should see the Y as a black and white image, but why it repeats 3 times in my image? And should I see anything in the UV part?
Of course I tried to convert this buffer to RGB. I used https://github.com/andrechen/yuv2rgb/blob/master/yuv2rgb.h#L70 but I only get a completely black image.
The format looks like I420 format (also called YV12).
I420 is YUV 4:2:0 format with fully planar ordered format.
In YUV420, the Y color channel is the Luma (brightness) of each pixel.
U and V are the Chroma (color) channels.
The resolution of U and V is half of Y in both axes (downsampled by a factor of 0.5 in each axis).
I420 illustration:
Assume unsigned char* src is a pointer to the frame buffer, and the resolution is 640x480:
src -> YYYYYY
YYYYYY
YYYYYY
YYYYYY
src + 640*480 -> UUU
UUU
src + (320*240)*5 -> VVV
VVV
I used MATLAB code for restoring the RGB image from the image you have posted.
Here is the result:
MATLAB code (just for reference):
I = imread('Test.png');
R = I(:,:,1);G = I(:,:,2);B = I(:,:,3);
T = zeros(size(R,1), size(R,2)*3, 'uint8');
T(:, 1:3:end) = R;T(:, 2:3:end) = G;T(:, 3:3:end) = B;
T = T';T = T(:);
Y = T(1:640*480);
U = T(640*480+1:640*480+640*480/4);
V = T(640*480+640*480/4+1:640*480+(640*480/4)*2);
Y = (reshape(Y, [640, 480]))';
U = (reshape(U, [320, 240]))';
V = (reshape(V, [320, 240]))';
U = imresize(U, 2);
V = imresize(V, 2);
YUV = cat(3, Y, U, V);
RGB = ycbcr2rgb(YUV);
I've done a few YUV renderers before.
A YUV 420 buffer should contain width*height bytes for Y, followed by (width*height)/4) bytes for U. And another (width*height)/4) bytes for V. Hence, if your YUV byte buffer should contain (width*height*3)/2 bytes in size.
Just to see the grey scale pattern as you describe it, you'd need to convert the "Y" bytes into 24-bit RGB like the following:
Something like this:
unsigned char* YUV_BYTES = < some buffer of size (width*height*3)/2 with bytes copied in>
unsigned char* RGB_BYTES = < some buffer of size width*height*3 >
const unsigned char* dst = RGB_BYTES;
for (unsigned int r = 0; r < height; r++)
{
unsigned int row_offset = r*width;
for (unsigned int c = 0; c < width; c++)
{
*dst[0] = YUV[row_offset + c]; // R
*dst[1] = YUV[row_offset + c]; // G
*dst[2] = YUV[row_offset + c]; // B
dst += 3;
}
}
I think there's also an implicit assumption about the width and height of YUV images always being divisible by 4. Your renderer might draw this image upside down depending on your graphics library and platform.
Scenario
I want to create my own SetChannel function that will set a specific channel of an image. For example, I have an image input of type CV_16UC3 (BGR image of type ushort) and I want to change the green channel (=1 due to zero based index) to an ushort value of 32768. For this, I invoke SetChannel(input,1,32768).
template<typename T>
void SetChannel(Mat mat, uint channel, T value)
{
const uint channels = mat.channels();
if (channel + 1 > channels)
return;
T * data = (T*)mat.data;
// MBPR : number of Memory Block Per Row
// Mat.step : number of byte per row
// Mat.elemSize1() : number of byte per channel
const unsigned int MBPR = mat.step / mat.elemSize1();
// N : total number of memory blocks
const unsigned int N = mat.rows * MBPR;
for (uint i = channel; i < N; i += channels)
data[i] = value;
}
I prefer working in a single loop than nested looping so I define the number of iteration N as given above.
The code above works as expected but some other people said the part
T * data = (T*)mat.data;
is a code smell and regarded as a badly designed program.
Now I want to rewrite the new one with another approach as follows.
It is not working as expected because I don't know how to assign T value to data[i] of type uchar.
template<typename T>
void SetChannel(Mat mat, uint channel, T value)
{
const uint channels = mat.channels();
if (channel + 1 > channels)
return;
uchar * data = mat.data;
const unsigned int N = mat.rows * mat.step;// byte per image
const unsigned int bpc = mat.elemSize1();// byte per channel
const unsigned int bpp = mat.elemSize(); // byte per pixel
for (uint i = channel * bpc; i < N; i += bpp)
//data[i] = value;
}
Question
How to assign value of type T to data[i] of type uchar without any loss?
For those who don't know how Mat is, the following might be useful.
About OpenCV Mat class
OpenCV provides a bunch of types of images. For example,
CV_8UC1 represents gray scale image type in which each pixel has one channel of type uchar.
CV_8UC3 represents BGR (not RGB) image type in which each pixel has three channels, each of type uchar.
CV_16UC3 represents BGR (not RGB) image type in which each pixel has three channels, each of type ushort.
etc.
Mat is the class to encapsulate image. It has several attributes and functions. Let me list some of them that I will use in this question so you can understand my scenario better.
Mat.data: pointer of type uchar pointing to a block of image pixels.
Mat.rows: number of rows
Mat.channels(): number of channels per pixel
Mat.elemSize1() (ended with 1): number of byte per channel
Mat.elemSize(): number of byte per pixel.
Mat.elemSize() = Mat.channels() * Mat.elemSize1().
Mat.step: number of byte per row
Here Mat.step can be thought as the product of
- "effective" number of pixels per row (let me name it as EPPR),
- number of channels per pixel or Mat.channels(), and
- number of byte per channel or Mat.elemSize1().
Mathematically,
Mat.step = EPPR * Mat.elemSize()
Mat.step = EPPR * Mat.channels() * Mat.elemSize1()
Let me define EPPR * Mat.channels() as memory blocks per row (MBPR). If you know the correct term for MBPR, let me know.
As a result, MBPR = Mat.step / Mat.elemSize1().
I got it from someone offline. Hopefully it is useful for others as well.
template<typename T>
void SetChannel(Mat mat, uint channel, T value)
{
const uint channels = mat.channels();
if (channel + 1 > channels)
return;
uchar * data = mat.data;
const unsigned int N = mat.rows * mat.step;// byte per image
const unsigned int bpc = mat.elemSize1();// byte per channel
const unsigned int bpp = mat.elemSize(); // byte per pixel
const unsigned int bpu = CHAR_BIT * sizeof(uchar);// bits per uchar
for (uint i = channel * bpc; i < N; i += bpp)
for (uint j = 0; j < bpc; j++)
data[i + j] = value >> bpu * j;
}
I would like to extract a pixel's rgb value in every frame that is decoded using ffmpeg. I looked into ffplay source code
get_video_frame
video_refresh
queue_picture
I tried the above three methods to hook on to the frame but I do not understand how to get a pixel's rgb value. Could anyone kindly give some pointer into this
http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=15&t=805
Thats the source and this is the conversion source I used and it works as expected. Hope this helps someone
ColorRGB GetRGBPixel(const AVFrame& frame, int x, int y)
{
// Y component
const unsigned char y = frame.data[0][frame.linesize[0]*y + x];
// U, V components
x /= 2;
y /= 2;
const unsigned char u = frame.data[1][frame.linesize[1]*y + x];
const unsigned char v = frame.data[2][frame.linesize[2]*y + x];
// RGB conversion
const unsigned char r = y + 1.402*(v-128);
const unsigned char g = y - 0.344*(u-128) - 0.714*(v-128);
const unsigned char b = y + 1.772*(u-128);
return ColorRGB(r, g, b);
}
I am working with depth images retrieved from kinect which are 16 bits. I found some difficulties on making my own filters due to the index or the size of the images.
I am working with Textures because allows to work with any bit size of images.
So, I am trying to compute an easy gradient to understand what is wrong or why it doesn't work as I expected.
You can see that there is something wrong when I use y dir.
For x:
For y:
That's my code:
typedef concurrency::graphics::texture<unsigned int, 2> TextureData;
typedef concurrency::graphics::texture_view<unsigned int, 2> Texture
cv::Mat image = cv::imread("Depth247.tiff", CV_LOAD_IMAGE_ANYDEPTH);
//just a copy from another image
cv::Mat image2(image.clone() );
concurrency::extent<2> imageSize(640, 480);
int bits = 16;
const unsigned int nBytes = imageSize.size() * 2; // 614400
{
uchar* data = image.data;
// Result data
TextureData texDataD(imageSize, bits);
Texture texR(texDataD);
parallel_for_each(
imageSize,
[=](concurrency::index<2> idx) restrict(amp)
{
int x = idx[0];
int y = idx[1];
// 65535 is the maxium value that can take a pixel with 16 bits (2^16 - 1)
int valX = (x / (float)imageSize[0]) * 65535;
int valY = (y / (float)imageSize[1]) * 65535;
texR.set(idx, valX);
});
//concurrency::graphics::copy(texR, image2.data, imageSize.size() *(bits / 8u));
concurrency::graphics::copy_async(texR, image2.data, imageSize.size() *(bits) );
cv::imshow("result", image2);
cv::waitKey(50);
}
Any help will be very appreciated.
Your indexes are swapped in two places.
int x = idx[0];
int y = idx[1];
Remember that C++AMP uses row-major indices for arrays. Thus idx[0] refers to row, y axis. This is why the picture you have for "For x" looks like what I would expect for texR.set(idx, valY).
Similarly the extent of image is also using swapped values.
int valX = (x / (float)imageSize[0]) * 65535;
int valY = (y / (float)imageSize[1]) * 65535;
Here imageSize[0] refers to the number of columns (the y value) not the number of rows.
I'm not familiar with OpenCV but I'm assuming that it also uses a row major format for cv::Mat. It might invert the y axis with 0, 0 top-left not bottom-left. The Kinect data may do similar things but again, it's row major.
There may be other places in your code that have the same issue but I think if you double check how you are using index and extent you should be able to fix this.
In my application, once I load an image into an SDL_Surface object, I need to go through each RGB value in the image and replace it with another RGB value from a lookup function.
(rNew, gNew, bNew) = lookup(rCur, gCur, bCur);
It seems surface->pixels gets me the pixels. I would appreciate it if someone can explain to me how to obtain R, G, and B values from the pixel and replace it with the new RGB value.
Use built-in functions SDL_GetRGB and SDL_MapRGB
#include <stdint.h>
/*
...
*/
short int x = 200 ;
short int y = 350 ;
uint32_t pixel = *( ( uint32_t * )screen->pixels + y * screen->w + x ) ;
uint8_t r ;
uint8_t g ;
uint8_t b ;
SDL_GetRGB( pixel, screen->format , &r, &g, &b );
screen->format deals with the format so you don't have to.
You can also use SDL_Color instead of writing r,g,b variables separately.
Depending on the format of the surface, the pixels are arranged as an array in the buffer.
For typical 32 bit surfaces, it is R G B A R G B A
where each component is 8 bit, and every 4 are a pixel
First of all you need to lock the surface to safely access the data for modification. Now to manipulate the array you need to know the numbers of bit per pixels, and the alignment of the channels (A, R, G, B). As Photon said if is 32 bits per pixel the array can be RGBARGBA.... if it is 24 the array can be RGBRGB.... (can also be BGR, BGR, blue first)
//i assume the signature of lookup to be
int lookup(Uint8 r, Uint8 g, Uint8 b, Uint8 *rnew, Uint8* gnew, Uint8* bnew);
SDL_LockSurface( surface );
/* Surface is locked */
/* Direct pixel access on surface here */
Uint8 byteincrement = surface->format->BytesPerPixel;
int position;
for(position = 0; position < surface->w * surface->h* byteincrement; position += byteincrement )
{
Uint8* curpixeldata = (Uint8*)surface->data + position;
/* assuming RGB, you need to know the position of channels otherwise the code is overly complex. for instance, can be BGR */
Uint8* rdata = curpixeldata +1;
Uint8* gdata = curpixeldata +2;
Uint8* bdata = curpixeldata +3;
/* those pointers point to r, g, b, use it as you want */
lookup(*rdata, *gdata, *bdata, rdata,gdata,bdata);
}
.
SDL_LockSurface( surface );