I'm working on an edge detection algorithm for a NPR technique. I plan on just using difference of gaussians to find the edges.
I thought that I would take a copy of the current screen, then analyze and recolor the pixels so that I have a map to draw the edges with.
This is my screen copy logic so far:
int width = rd->width();
int height = rd->height();
GLbyte * data = (GLbyte *)malloc( width * height * 3 );
if( data ) {
glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, data);
}
float color = 0;
for (int i = 0; i < width; i++)
{
for (int j = 0; j < height; j++)
{
color = data[i*width+j];
}
}
Seeing as I'm just grabbing everything, I didn't think that the alpha component was necessary to copy. rd is my render device, and data is being output like this:
2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Wy2Vy2Vy2Vy2Vx2Vx2Vx2Vx2Vx2Vx2Vx2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy2Vy3Vy3Vy3Vy3Vy3Vy2Vy2Vy1Vy1Uy0Uy1Vy1Vy1Vy1Vy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy1Vy1Vy0Vy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vy0Vy0Vy0Vy0Vy0Vz0Vz0Vz0Vz0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Ux0Ux0Ux0Tx0Tx0Tx0Tx0Tx0Ux0Ux0Ux0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx/Tx/Tw/Tw/Tx/Tx0Tx0Tx0Tx/Tx/Tw.Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw-Tw.Tw.Tw.Tw.Tw/Tw/Tw/Tw/Tx/Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Tx0Ux0Ux0Ux0Ux0Ux0Ux0Ux0Ux0Ux0Ux0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vz0Vy0Vy0Vy0Vy0Vy0Vy0Vy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Uy0Vy1Vy1Vy2Vy2Vz2Vz3Wz3Wz3Vz3Vz3Vz3Vz...
And I have no idea how to handle that. I tried reading a value as shown below with the float color but that didn't really help me, as I don't really know what it means. Is each color I'm reading an intensity value of the pixel, or do I need to read three data points in a row to get all the channels?
What is a good way to get the data displayed on the screen, modify it, and redraw it?
You are telling glReadPixels that you want to read RGB values in 3 BYTES and you are putting it in a single float value. This cannot work.
Try the following instead:
unsigned char color[3];
for ...
color[0] = data[3*(i*width+j)];
color[1] = data[3*(i*width+j)+1];
color[2] = data[3*(i*width+j)+2];
I haven't tried it so there might be some mistakes. But you get the idea.
You could also tell glReadPixels that you only want GL_RED in GL_FLOAT and put it in a float buffer if you are processing black and white images and only want the intensity. Or GL_LUMINANCE; it's really up to you but you need to be coherent between the parameters you pass to glReadPixels and the way you parse that data.
Related
I am creating a program that allows you to view fractals like the Mandelbrot or Julia set. I would like to render them as quickly as possible. I would love a way to put an array of uint8_t pixel values onto the screen. The array is formatted like this...
{r0,g0,b0,r1,g1,b1,...}
(A one dimensional array or RGB color values)
I know I have the proper data because before I just set individual points and it worked...
for(int i = 0;i < height * width;++i) {
//setStroke and point are functions that I made that together just draw a colored point
r.setStroke(data[i*3],data[i*3+1],data[i*3+2]);
r.point(i % r.window.w,i / r.window.w);
}
This is a pretty slow operation especially if the screen is big (which I would like it to be)
Is there any faster way to just put all the data onto the screen.
I tried doing something like this
void* pixels;
int pitch;
SDL_Texture* img = SDL_CreateTexture(ren,
SDL_GetWindowPixelFormat(win),SDL_TEXTUREACCESS_STREAMING,window.w,window.h);
SDL_LockTexture(img, NULL, &pixels, &pitch);
memcpy(pixels, data, window.w * 3 * window.h);
SDL_UnlockTexture(img);
SDL_RenderCopy(ren,img,NULL,NULL);
SDL_DestroyTexture(img);
I have no idea what I'm doing so please have mercy
Edit (thank you for comments :))
So here is what I do now
SDL_Texture* img = SDL_CreateTexture(ren, SDL_PIXELFORMAT_RGB888,SDL_TEXTUREACCESS_STREAMING,window.w,window.h);
SDL_UpdateTexture(img,NULL,&data[0],window.w * 3);
SDL_RenderCopy(ren,img,NULL,NULL);
SDL_DestroyTexture(img);
But I get this Image... which is not what it should look like
I am thinking that my data is just formatted wrong, right now it is formatted as an array of uint8_t in RGB order. Is there another way I should be formatting it (note I do not need an alpha channel)
I am having some problems finding a solution on how to retrieve a specific color of a pixel on a SDL_Texture...
To be bit more specific: I am trying to calculate the average amount of color used in a given texture. Later on I want to devide for example the number of red pixels by the total amount of pixels. For this task I will need a method, which will get me each pixel color...
I tried to search for some functions, but unfortunately I wasnt able to figure it out..
I saw methods like SDL_RenderReadPixels and SDL_GetPixelFormatName, but none of those helped me out...
Do you have a solution for me?
To access an SDL_Texture's pixels, you must create a blank texture using SDL_CreateTexture() and pass in SDL_TEXTUREACCESS_STREAMING for the access parameter, then copy the pixels of a surface into it. Once that's done, you can use the SDL_LockTexture() function to retrieve a pointer to the pixel data which can then be accessed and modified. To save your changes, you'd call SDL_UnlockTexture(). Try something like this:
SDL_Texture *t;
int main()
{
// Init SDL
SDL_Surface * img = IMG_Load("path/to/file");
SDL_CreateTexture(renderer, SDL_PIXELFORMAT_RGBA8888, SDL_TEXTUREACCESS_STREAMING, img->w, img->h);
void * pixels;
SDL_LockTexture(t, &img->clip_rect, &pixels, img->pitch);
memcpy(pixels, img->pixels, img->w * img->h);
Uint32 * upixels = (Uint32 *) pixels;
// get or modify pixels
SDL_UnlockTexture(t);
return 0;
}
Uint32 get_pixel_at(Uint32 * pixels, int x, int y, int w)
{
return pixels[y * w + x];
}
You can get the colors from a pixel like this:
Uint32 pixel = get_pixel_at(pixels, x, y, img->w);
Uint8 * colors = (Uint8 *) pixel;
// colors[0] is red, 1 is green, 2 is blue, 3 is alpha (assuming you've set the blend mode on the texture to SDL_BLENDMODE_BLEND
If you want more information, then check out these SDL 2.0 tutorials: http://lazyfoo.net/tutorials/SDL/index.php. Tutorial 40 deals specifically with this problem.
Let me know if you have any questions or something is unclear.
Good luck!
Lets start with some code:
QByteArray OpenGLWidget::modifyImage(QByteArray imageArray, const int width, const int height){
if (vertFlip){
/* Each pixel constist of four unisgned chars: Red Green Blue Alpha.
* The field is normally 640*480, this means that the whole picture is in fact 640*4 uChars wide.
* The whole ByteArray is onedimensional, this means that 640*4 is the red of the first pixel of the second row
* This function is EXTREMELY SLOW
*/
QByteArray tempArray = imageArray;
for (int h = 0; h < height; ++h){
for (int w = 0; w < width/2; ++w){
for (int i = 0; i < 4; ++i){
imageArray.data()[h*width*4 + 4*w + i] = tempArray.data()[h*width*4 + (4*width - 4*w) + i ];
imageArray.data()[h*width*4 + (4*width - 4*w) + i] = tempArray.data()[h*width*4 + 4*w + i];
}
}
}
}
return imageArray;
}
This is the code I use right now to vertically flip an image which is 640*480 (The image is actually not guaranteed to be 640*480, but it mostly is). The color encoding is RGBA, which means that the total array size is 640*480*4. I get the images with 30 FPS, and I want to show them on the screen with the same FPS.
On an older CPU (Athlon x2) this code is just too much: the CPU is racing to keep up with the 30 FPS, so the question is: can I do this more efficient?
I am also working with OpenGL, does that have a gimmic I am not aware of that can flip images with relativly low CPU/GPU usage?
According to this question, you can flip an image in OpenGL by scaling it by (1,-1,1). This question explains how to do transformations and scaling.
You can improve at least by doing it blockwise, making use of the cache architecture. In your example one of the accesses (either the read OR the write) will be off-cache.
For a start it can help to "capture scanlines" if you're using two loops to loop through the pixels of an image, like so:
for (int y = 0; y < height; ++y)
{
// Capture scanline.
char* scanline = imageArray.data() + y*width*4;
for (int x = 0; x < width/2; ++x)
{
const int flipped_x = width - x-1;
for (int i = 0; i < 4; ++i)
swap(scanline[x*4 + i], scanline[flipped_x*4 + i]);
}
}
Another thing to note is that I used swap instead of a temporary image. That'll tend to be more efficient since you can just swap using registers instead of loading pixels from a copy of the entire image.
But also it generally helps if you use a 32-bit integer instead of working one byte at a time if you're going to be doing anything like this. If you're working with pixels with 8-bit types but know that each pixel is 32-bits, e.g., as in your case, you can generally get away with a case to uint32_t*, e.g.
for (int y = 0; y < height; ++y)
{
uint32_t* scanline = (uint32_t*)imageArray.data() + y*width;
std::reverse(scanline, scanline + width);
}
At this point you might parellelize the y loop. Flipping an image horizontally (it should be "horizontal" if I understood your original code correctly) in this way is a little bit tricky with the access patterns, but you should be able to get quite a decent boost using the above techniques.
I am also working with OpenGL, does that have a gimmic I am not aware
of that can flip images with relativly low CPU/GPU usage?
Naturally the fastest way to flip images is to not touch their pixels at all and just save the flipping for the final part of the pipeline when you render the result. For this you might render a texture in OGL with negative scaling instead of modifying the pixels of a texture.
Another thing that's really useful in video and image processing is to represent an image to process like this for all your image operations:
struct Image32
{
uint32_t* pixels;
int32_t width;
int32_t height;
int32_t x_stride;
int32_t y_stride;
};
The stride fields are what you use to get from one scanline (row) of an image to the next vertically and one column to the next horizontally. When you use this representation, you can use negative values for the stride and offset the pixels accordingly. You can also use the stride fields to, say, render only every other scanline of an image for fast interactive half-res scanline previews by using y_stride=height*2 and height/=2. You can quarter-res an image by setting x stride to 2 and y stride to 2*width and then halving the width and height. You can render a cropped image without making your blit functions accept a boatload of parameters by just modifying these fields and keeping the y stride to width to get from one row of the cropped section of the image to the next:
// Using the stride representation of Image32, this can now
// blit a cropped source, a horizontally flipped source,
// a vertically flipped source, a source flipped both ways,
// a half-res source, a quarter-res source, a quarter-res
// source that is horizontally flipped and cropped, etc,
// and all without modifying the source image in advance
// or having to accept all kinds of extra drawing parameters.
void blit(int dst_x, int dst_y, Image32 dst, Image32 src);
// We don't have to do things like this (and I think I lost
// some capabilities with this version below but it hurts my
// brain too much to think about what capabilities were lost):
void blit_gross(int dst_x, int dst_y, int dst_w, int dst_h, uint32_t* dst,
int src_x, int src_y, int src_w, int src_h,
const uint32_t* src, bool flip_x, bool flip_y);
By using negative values and passing it to an image operation (ex: a blit operation), the result will naturally be flipped without having to actually flip the image. It'll end up being "drawn flipped", so to speak, just as with the case of using OGL with a negative scaling transformation matrix.
Suppose we have a 32-bit PNG file of some ghostly/incorporeal character, which is drawn in a semi-transparent fashion. It is not equally transparent in every place, so we need the per-pixel alpha information when loading it to a surface.
For fading in/out, setting the alpha value of an entire surface is a good way; but not in this case, as the surface already has the per-pixel information and SDL doesn't combine the two.
What would be an efficient workaround (instead of asking the artist to provide some awesome fade in/out animation for the character)?
I think the easiest way for you to achieve the result you want is to start by loading the source surface containing your character sprites, then, for every instance of your ghost create a working copy of the surface. What you'll want to do is every time the alpha value of an instance change, SDL_BlitSurface (doc) your source into your working copy and then apply your transparency (which you should probably keep as a float between 0 and 1) and then apply your transparency on every pixel's alpha channel.
In the case of a 32 bit surface, assuming that you initially loaded source and allocated working SDL_Surfaces you can probably do something along the lines of:
SDL_BlitSurface(source, NULL, working, NULL);
if(SDL_MUSTLOCK(working))
{
if(SDL_LockSurface(working) < 0)
{
return -1;
}
}
Uint8 * pixels = (Uint8 *)working->pixels;
pitch_padding = (working->pitch - (4 * working->w));
pixels += 3; // Big Endian will have an offset of 0, otherwise it's 3 (R, G and B)
for(unsigned int row = 0; row < working->h; ++row)
{
for(unsigned int col = 0; col < working->w; ++col)
{
*pixels = (Uint8)(*pixels * character_transparency); // Could be optimized but probably not worth it
pixels += 4;
}
pixels += pitch_padding;
}
if(SDL_MUSTLOCK(working))
{
SDL_UnlockSurface(working);
}
This code was inspired from SDL_gfx (here), but if you're doing only that, I wouldn't bother linking against a library just for that.
I'm working with Kinect sensor and I'm trying to align depth and color frames so that I can save them as images which "fit" into each other. I've spent a lot of time going through msdn forums and modest documentation of Kinect SDK and I'm getting absolutely nowhere.
Based on this answer: Kinect: Converting from RGB Coordinates to Depth Coordinates
I have the following function, where depthData and colorData are obtained from NUI_LOCKED_RECT.pBits and mappedData is the output containing new color frame, mapped to depth coordinates:
bool mapColorFrameToDepthFrame(unsigned char *depthData, unsigned char* colorData, unsigned char* mappedData)
{
INuiCoordinateMapper* coordMapper;
// Get coordinate mapper
m_pSensor->NuiGetCoordinateMapper(&coordMapper);
NUI_DEPTH_IMAGE_POINT* depthPoints = new NUI_DEPTH_IMAGE_POINT[640 * 480];
HRESULT result = coordMapper->MapColorFrameToDepthFrame(NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480, NUI_IMAGE_RESOLUTION_640x480, 640 * 480, reinterpret_cast<NUI_DEPTH_IMAGE_PIXEL*>(depthData), 640 * 480, depthPoints);
if (FAILED(result))
{
return false;
}
int pos = 0;
int* colorRun = reinterpret_cast<int*>(colorData);
int* mappedRun = reinterpret_cast<int*>(mappedData);
// For each pixel of new color frame
for (int i = 0; i < 640 * 480; ++i)
{
// Find the corresponding pixel in original color frame from depthPoints
pos = (depthPoints[i].y * 640) + depthPoints[i].x;
// Set pixel value if it's within frame boundaries
if (pos < 640 * 480)
{
mappedRun[i] = colorRun[pos];
}
}
return true;
}
All I get when running this code is an unchanged color frame with removed (white) all pixels where depthFrame had no information.
With the OpenNI framework there an option call registration.
IMAGE_REGISTRATION_DEPTH_TO_IMAGE – The depth image is transformed to have the same apparent vantage point as the RGB image.
OpenNI 2.0 and Nite 2.0 works very well to capture Kinect information and there a lot of tutorials.
You can have a look to this :
Kinect with OpenNI
And OpenNi have a example in SimplerViewer that merge Depth and Color maybe you can just look on that and try it.
This might not be the quick answer you're hoping for, but this transformation is done successfully within the ofxKinectNui addon for openFrameworks (see here).
It looks like ofxKinectNui delegates to the GetColorPixelCoordinatesFromDepthPixel function defined here.
I think the problem is that you're calling MapColorFrameToDepthFrame, when you should actually call MapDepthFrameToColorFrame.
The smoking gun is this line of code:
mappedRun[i] = colorRun[pos];
Reading from pos and writing to i is backwards, since pos = depthPoints[i] represents the depth coordinates corresponding to the color coordinates at i. You actually want to iterate over writing all depth coordinates and read from the input color image at the corresponding color coordinates.
I think that in your code there are different not correct lines.
First of all, which kind of depth map are you passing to your function?
Depth data is storred using two bytes for each value, that means that the correct type of the pointer that you should use for your depth data
is unsigned short.
Second point is that from what i have understood, you want to map depth frame to color frame, so the correct function that you have
to call from kinect sdk is MapDepthFrameToColorFrame instead of MapColorFrameToDepthFrame.
Finally the function will return a map of point where for each depth data at position [i], you have the position x and position y where that point should
be mapped.
To do this you don't need for colorData pointer.
So your function should be modified as follow:
/** Method used to build a depth map aligned to color frame
#param [in] depthData : pointer to your data;
#param [out] mappedData : pointer to your aligned depth map;
#return true if is all ok : false whene something wrong
*/
bool DeviceManager::mapColorFrameToDepthFrame(unsigned short *depthData, unsigned short* mappedData){
INuiCoordinateMapper* coordMapper;
NUI_COLOR_IMAGE_POINT* colorPoints = new NUI_COLOR_IMAGE_POINT[640 * 480]; //color points
NUI_DEPTH_IMAGE_PIXEL* depthPoints = new NUI_DEPTH_IMAGE_PIXEL[640 * 480]; // depth pixel
/** BE sURE THAT YOU ARE WORKING WITH THE RIGHT HEIGHT AND WIDTH*/
unsigned long refWidth = 0;
unsigned long refHeight = 0;
NuiImageResolutionToSize( NUI_IMAGE_RESOLUTION_640x480, refWidth, refHeight );
int width = static_cast<int>( refWidth ); //get the image width in a right way
int height = static_cast<int>( refHeight ); //get the image height in a right way
m_pSensor>NuiGetCoordinateMapper(&coordMapper); // get the coord mapper
//Map your frame;
HRESULT result = coordMapper->MapDepthFrameToColorFrame( NUI_IMAGE_RESOLUTION_640x480, width * height, depthPoints, NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480, width * height, colorPoints );
if (FAILED(result))
return false;
// apply map in terms of x and y (image coordinates);
for (int i = 0; i < width * height; i++)
if (colorPoints[i].x >0 && colorPoints[i].x < width && colorPoints[i].y>0 && colorPoints[i].y < height)
*(mappedData + colorPoints[i].x + colorPoints[i].y*width) = *(depthData + i );
// free your memory!!!
delete colorPoints;
delete depthPoints;
return true;
}
Make sure that your mappedData has been initialized in correct way, for example as follow.
mappedData = (USHORT*)calloc(width*height, sizeof(ushort));
Remember that kinect sdk does not provide an accurate align function between color and depth data.
If you want an accurate alignment between two images you should use a calibration model.
In that case i suggest you to use the Kinect Calibration Toolbox, based on Heikkilä calibration model.
You can find it in the follow link:
http://www.ee.oulu.fi/~dherrera/kinect/.
First of all, you must calibrate your device.
That means, you should calibrate the RGB and the IR sensor and then find the transformation between RGB and IR.
Once you know this information, you can apply the function:
RGBPoint = RotationMatrix * DepthPoint + TranslationVector
Check OpenCV or ROS projects for further details on it.
Extrinsic Calibration
Intrinsic Calibration