Pixel Perfect Collision Detection in Cocos2dx - c++

I am trying to port the pixel perfect collision detection in Cocos2d-x the original version was made for Cocos2D and can be found here: http://www.cocos2d-iphone.org/forums/topic/pixel-perfect-collision-detection-using-color-blending/
Here is my code for the Cocos2d-x version
bool CollisionDetection::areTheSpritesColliding(cocos2d::CCSprite *spr1, cocos2d::CCSprite *spr2, bool pp, CCRenderTexture* _rt) {
bool isColliding = false;
CCRect intersection;
CCRect r1 = spr1->boundingBox();
CCRect r2 = spr2->boundingBox();
intersection = CCRectMake(fmax(r1.getMinX(),r2.getMinX()), fmax( r1.getMinY(), r2.getMinY()) ,0,0);
intersection.size.width = fmin(r1.getMaxX(), r2.getMaxX() - intersection.getMinX());
intersection.size.height = fmin(r1.getMaxY(), r2.getMaxY() - intersection.getMinY());
// Look for simple bounding box collision
if ( (intersection.size.width>0) && (intersection.size.height>0) ) {
// If we're not checking for pixel perfect collisions, return true
if (!pp) {
return true;
}
unsigned int x = intersection.origin.x;
unsigned int y = intersection.origin.y;
unsigned int w = intersection.size.width;
unsigned int h = intersection.size.height;
unsigned int numPixels = w * h;
//CCLog("Intersection X and Y %d, %d", x, y);
//CCLog("Number of pixels %d", numPixels);
// Draw into the RenderTexture
_rt->beginWithClear( 0, 0, 0, 0);
// Render both sprites: first one in RED and second one in GREEN
glColorMask(1, 0, 0, 1);
spr1->visit();
glColorMask(0, 1, 0, 1);
spr2->visit();
glColorMask(1, 1, 1, 1);
// Get color values of intersection area
ccColor4B *buffer = (ccColor4B *)malloc( sizeof(ccColor4B) * numPixels );
glReadPixels(x, y, w, h, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
_rt->end();
// Read buffer
unsigned int step = 1;
for(unsigned int i=0; i 0 && color.g > 0) {
isColliding = true;
break;
}
}
// Free buffer memory
free(buffer);
}
return isColliding;
}
My code is working perfectly if I send the "pp" parameter as false. That is if I do only a bounding box collision but I am not able to get it working correctly for the case when I need Pixel Perfect collision.
I think the opengl masking code is not working as I intended.
Here is the code for "_rt"
_rt = CCRenderTexture::create(visibleSize.width, visibleSize.height);
_rt->setPosition(ccp(origin.x + visibleSize.width * 0.5f, origin.y + visibleSize.height * 0.5f));
this->addChild(_rt, 1000000);
_rt->setVisible(true); //For testing
I think I am making a mistake with the implementation of this CCRenderTexture
Can anyone guide me with what I am doing wrong ?
Thank you for your time :)

Finally solved the problem.
Had to use custom opengl fragment shaders to shade one of the sprites completely RED and the other completely BLUE and then looping through glReadPixels values to find any pixel having both RED and BLUE pixels. (Blending has to be considered as well, we do not want to replace one pixel value by the other)
In-Depth information can be found on my blog post
http://blog.muditjaju.infiniteeurekas.in/?p=1

You are not stepping through the buffer properly.
// Read buffer
unsigned int step = 1;
for(unsigned int i=0; i<numPixels; i+=step)
{
ccColor4B color = buffer;
if (color.r > 0 && color.g > 0)
{
isCollision = YES;
break;
}
}
source: http://www.cocos2d-iphone.org/forums/topic/pixel-perfect-collision-detection-using-color-blending/#post-337907

Related

OpenGL imgui slider ranges

I have a 960x540 window I'm creating using OpenGL. Rendering one texture here and using imgui to implement a slider that let's me translate the x, y, and z positions at runtime.
int window_width = 960;
int window_height = 540;
window = glfwCreateWindow(window_width, window_height, "Object Oriented OpenGL", NULL, NULL);
First I set the slider default values.
glm::vec3 model_matrix_translation(200, 200, 0);
Later in my loop that renders the objects positions frame by frame as long as the window is open, I set the ImGui::SliderFloat3 v_min and v_max params to be the dimensions of my window. It appears all 3 sliders x,y, and z use these values, but is there a way to set a range for each individuals slider? As you can see my slider for the y axis can go beyond 540.
while (!glfwWindowShouldClose(window))
{
// renderer and other code before this point
{
ImGui::SliderFloat3("Model Matrix Translation", &model_matrix_translation.x, 0.0f, 960.0f);
}
ImGui::Render();
ImGui_ImplGlfwGL3_RenderDrawData(ImGui::GetDrawData());
glfwSwapBuffers(window);
glfwPollEvents();
}
The implementation of SliderFloat3 delegates to SliderScalarN, which is implemented as such:
bool ImGui::SliderScalarN(const char* label, ImGuiDataType data_type, void* v, int components, const void* v_min, const void* v_max, const char* format, float power)
{
ImGuiWindow* window = GetCurrentWindow();
if (window->SkipItems)
return false;
ImGuiContext& g = *GImGui;
bool value_changed = false;
BeginGroup();
PushID(label);
PushMultiItemsWidths(components, CalcItemWidth());
size_t type_size = GDataTypeInfo[data_type].Size;
for (int i = 0; i < components; i++)
{
PushID(i);
if (i > 0)
SameLine(0, g.Style.ItemInnerSpacing.x);
value_changed |= SliderScalar("", data_type, v, v_min, v_max, format, power);
PopID();
PopItemWidth();
v = (void*)((char*)v + type_size);
}
PopID();
const char* label_end = FindRenderedTextEnd(label);
if (label != label_end)
{
SameLine(0, g.Style.ItemInnerSpacing.x);
TextEx(label, label_end);
}
EndGroup();
return value_changed;
}
For your purposes, you will need to use a modified version of this function that moves v_min and v_max along with v, so in your modified function, change the last line of that for loop to:
v = (void*)((char*)v + type_size);
v_min = (void*)((char*)v_min + type_size);
v_max = (void*)((char*)v_max + type_size);
Or just specialize the whole function to take float* instead and then you can just do v++, v_min++, v_max++; instead.
Finally, you can call your modified function as follows:
float mins[] = {0.0f, 0.0f, 0.0f};
float maxs[] = {960.0f, 540.0f, 0.0f};
MyModifiedSliderFloat3("Model Matrix Translation", &model_matrix_translation.x, mins, maxs);

2D water shader in SFML

I want to implement the algorithm for a 2D water surface described here and here.
But instead of using two int arrays and calculating on the CPU I would like to use SFML's sf::RenderTexture's (FBO's basically) and a GLSL shader to run everything on the GPU. I want to use SFML, because it's so simple and I have worked with it before, so I know my way around it a little.
I've made some good progress so far. I was able to set up 3 sf::RenderTextures and ping-pong between them correctly (because other than int array you can't read and write to the same sf::RenderTexture at the same time). I was also able to adapt the algorithm for the height field creation form being in the range -32.767 to 32.767 to the range 0 to 1 (or to be more precise -0.5 to 0.5 for the calculation). Also adding new ripples works to some extend. So up to this point you can actually see a little of waves going on.
Here comes my problem now: The waves disappear really, really fast and I don't even apply any damping yet. According to the algorithm the ripples are not stopping if there is no damping applied. It's even the other way around. If I apply "amplification" the waves look close to what you would expect them to look like (but they still disappear without any damping applied to them). My first thought was that this is, because I use float's in range 0 - 1 instead of integers, but I only see this being a problem if multiplication is used, but I only use addition and subtraction.
Here is my SFML C++ code :
#include <SFML/Graphics.hpp>
#include <iostream>
int main()
{
sf::RenderWindow window(sf::VideoMode(1000, 1000), "SFML works!");
window.setFramerateLimit(12);
sf::RenderTexture buffers[3];
buffers[0].create(500, 500);
buffers[1].create(500, 500);
buffers[2].create(500, 500);
sf::RenderTexture* firstBuffer = buffers;
sf::RenderTexture* secondBuffer = &buffers[1];
sf::RenderTexture* finalBuffer = &buffers[2];
firstBuffer->clear(sf::Color(128, 128, 128));
secondBuffer->clear(sf::Color(128, 128, 128));
finalBuffer->clear(sf::Color(128, 128, 128));
sf::Shader waterHeightmapShader;
waterHeightmapShader.loadFromFile("waterHeightmapShader.glsl", sf::Shader::Fragment);
sf::Sprite spritefirst;
spritefirst.setPosition(0, 0);
spritefirst.setTexture(firstBuffer->getTexture());
sf::Sprite spritesecond;
spritesecond.setPosition(500, 0);
spritesecond.setTexture(secondBuffer->getTexture());
sf::Sprite spritefinal;
spritefinal.setPosition(0, 500);
spritefinal.setTexture(finalBuffer->getTexture());
while (window.isOpen())
{
sf::Event event;
while (window.pollEvent(event))
{
if(event.type == sf::Event::Closed)
window.close();
if(event.type == sf::Event::KeyReleased && event.key.code == sf::Keyboard::Escape)
window.close();
}
waterHeightmapShader.setParameter("mousePosition", sf::Vector2f(-1.f, -1.f));
// if mouse button is pressed add new ripples
if(sf::Mouse::isButtonPressed(sf::Mouse::Left))
{
sf::Vector2i mousePosition = sf::Mouse::getPosition(window);
if(mousePosition.x < 500 && mousePosition.y < 500)
{
sf::Vector2f mouse(mousePosition);
mouse.x /= 500.f;
mouse.y /= 500.f;
mouse.y = 1 - mouse.y;
std::cout << mouse.x << " " << mouse.y << std::endl;
waterHeightmapShader.setParameter("mousePosition", mouse);
}
}
waterHeightmapShader.setParameter("textureTwoFramesAgo", firstBuffer->getTexture());
waterHeightmapShader.setParameter("textureOneFrameAgo", secondBuffer->getTexture());
// create the heightmap
secondBuffer->display();
finalBuffer->clear(sf::Color(128, 128, 128));
finalBuffer->draw(sf::Sprite(secondBuffer->getTexture()), &waterHeightmapShader);
finalBuffer->display();
spritefirst.setTexture(firstBuffer->getTexture());
spritesecond.setTexture(secondBuffer->getTexture());
spritefinal.setTexture(finalBuffer->getTexture());
window.clear();
window.draw(spritefirst);
window.draw(spritesecond);
window.draw(spritefinal);
window.display();
// swap the buffers around, first becomes second, second becomes third and third becomes first
sf::RenderTexture* swapper = firstBuffer;
firstBuffer = secondBuffer;
secondBuffer = finalBuffer;
finalBuffer = swapper;
}
return 0;
}
And here is my GLSL shader code :
uniform sampler2D textureTwoFramesAgo;
uniform sampler2D textureOneFrameAgo;
uniform vec2 mousePosition;
const float textureSize = 500.0;
const float pixelSize = 1.0 / textureSize;
void main()
{
// pixels position
vec2 position = gl_TexCoord[0].st;
vec4 finalColor = ((texture2D(textureOneFrameAgo, vec2(position.x - pixelSize, position.y)) +
texture2D(textureOneFrameAgo, vec2(position.x + pixelSize, position.y)) +
texture2D(textureOneFrameAgo, vec2(position.x, position.y + pixelSize)) +
texture2D(textureOneFrameAgo, vec2(position.x, position.y - pixelSize)) - 2.0) / 2) -
(texture2D(textureTwoFramesAgo, position) - 0.5);
// damping
// finalColor.rgb *= 1.9; // <---- uncomment this for the "amplifiction" ie. to see the waves better
finalColor.rgb += 0.5;
// add new ripples
if(mousePosition.x > 0.0)
{
if(distance(position, mousePosition) < pixelSize * 5)
{
finalColor = vec4(0.9, 0.9, 0.9, 1.0);
}
}
gl_FragColor = finalColor;
}
Please remember that this is all just about the height field creation. There is no shading of the water yet.
Do you know why the waves disappear by them self without damping?
If I am reading the code correctly you sample the previous frame for the texture's colors/height and use four neighboring pixels/texels to determine the color/height of the current pixel.
As you are calculating (scaling) these neighbors you might run into missing the texel that contains the color/height you are looking for. It might not be the heighest texel, just one next to it a little bit lower causing the unexpected damping.
This is where you do not just use addition and subtraction:
const float pixelSize = 1.0 / textureSize;
By using this value you could just miss the texel you are looking for.
EDIT
Also: you are averaging the samples so the result will always be less than the maximum value of the samples. So instead of averaging you could select the maximum value. That might give weird results but also extra insight.
Here are some "Processing" codes which implements the same algorithm you've posted above, and its damping is correct, I hope you can get some points from it :
// codes begin
int Width = 800;
int Height = 600;
int FullSize = 0;
//int Spacing = 10;
int[] source, dest;
PImage bg;
void setup()
{
// if you want to run these codes by "Processing"
// please make a picture named "HelloWorld.png"
bg = loadImage("HelloWorld.png");
Width = bg.width;
Height = bg.height;
FullSize = Width * Height;
size(Width, Height);
source = new int[FullSize];
dest = new int[FullSize];
for (int i=0; i< FullSize; i++)
source[i] = dest[i] = 0;
}
void draw()
{
for (int i=Width; i< FullSize-Width; i++)
{
// check for bounds
int xi = i % Width;
if ((xi==0) || (xi==Width-1)) continue;
dest[i] = (
((source[i-1]+
source[i+1]+
source[i-Width]+
source[i+Width]) >>1) ) -dest[i];
int dampFactor = 1000;
dest[i] -= (dest[i] >> dampFactor); // Damping - Quick divde by 32 (5 bits)
}
//image(bg, 0, 0);
loadPixels();
for (int i=Width; i< FullSize-Width; i++)
{
// check for bounds
int xi = i % Width;
if ((xi==0) || (xi==Width-1)) continue;
int xoffset = dest[i-1] - dest[i+1];
int yoffset = dest[i-Width] - dest[i+Width];
int offset = i+xoffset+yoffset*Width;
if (offset>0 && offset<FullSize)
{
// TODO: make better map
pixels[i] = bg.pixels[offset];
}
}
//bg.updatePixels();
updatePixels();
//swap
int[] temp = source;
source = dest;
dest = temp;
}
void mouseDragged()
{
if (mouseX > 0 && mouseX < Width && mouseY > 0 && mouseY < Height)
source[mouseY*Width+mouseX] = (int)random(50, 100);
}
void mousePressed()
{
// TODO: make a area pulse value, like a radius circle
if (mouseX > 0 && mouseX < Width && mouseY > 0 && mouseY < Height)
source[mouseY*Width+mouseX] = (int)random(50, 100);
}
// codes end

SDL - drawing 'negative' circles (Fog of War)

I have this 800x600square I want to draw to the screen. I want to 'cut' circles in it (where alpha would be 0). Basically I'm drawing this whole rectangle over a map so in these 'circles' I drew, you can see the map, otherwise you see the grey square
So, I assume you're trying to add fog of war to one of you game?
I had a small demo I made for a local University a few weeks ago to show A* pathfinding, so I thought I could add fog of war to it for you. Here's the results:
Initial map
First, you start with a complete map, totally visible
Fog
Then, I added a surface to cover the entire screen (take note that my map is smaller than the screen, so for this case I just added fog of war on the screen, but if you have scrolling, make sure it covers each map pixel 1:1)
mFogOfWar = SDL_CreateRGBSurface(SDL_HWSURFACE, in_Width, in_Height, 32, 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000);
SDL_Rect screenRect = {0, 0, in_Width, in_Height};
SDL_FillRect(mFogOfWar, &screenRect, 0xFF202020);
Then, you need to draw it... I added this call after drawing the game objects and before drawing the UI
DrawSurface(mFogOfWar, 0, 0);
Where
void RenderingManager::DrawSurface(SDL_Surface* in_Surface, int in_X, int in_Y)
{
SDL_Rect Dest = { in_X, in_Y, 0, 0 };
SDL_BlitSurface(in_Surface, NULL, mScreen, &Dest);
}
Which should give you the following result:
"Punch Surface"
I then created a 32 bits .png that looks like this (checkerboard shows alpha)
When rendering my main character, I added this call:
gRenderingManager.RemoveFogOfWar(int(mX) + SPRITE_X_OFFSET, int(mY) + SPRITE_Y_OFFSET);
The offset is only there to center the punch with the sprite, basically, what I'm passing to RemoveFogOfWar is the center of my sprite.
Remove Fog Of War
Now the meat of the fog of war. I did two versions, one where Fog of War is removed permanently and one where the fog of war is reset. My fog of war reset relies on my punch surface to have a contour where the alpha is reset to 0 and the fact that my character moves of less pixels than the contour contains per frame, otherwise I would keep the Rect where my punch was applied and I would refill it before drawing again the new punch.
Since I couldn't find a "multiply" blend with SDL, I decided to write a simple function that iterates on the punch surface and updates the alpha on the fog of war surface. The most important part is to make sure you stay within the bounds of your surfaces, so it takes up most of the code... there might be some crop functions but I didn't bother checking:
void RenderingManager::RemoveFogOfWar(int in_X, int in_Y)
{
const int halfWidth = mFogOfWarPunch->w / 2;
const int halfHeight = mFogOfWarPunch->h / 2;
SDL_Rect sourceRect = { 0, 0, mFogOfWarPunch->w, mFogOfWarPunch->h };
SDL_Rect destRect = { in_X - halfWidth, in_Y - halfHeight, mFogOfWarPunch->w, mFogOfWarPunch->h };
// Make sure our rects stays within bounds
if(destRect.x < 0)
{
sourceRect.x -= destRect.x; // remove the pixels outside of the surface
sourceRect.w -= sourceRect.x; // shrink to the surface, not to offset fog
destRect.x = 0;
destRect.w -= sourceRect.x; // shrink the width to stay within bounds
}
if(destRect.y < 0)
{
sourceRect.y -= destRect.y; // remove the pixels outside
sourceRect.h -= sourceRect.y; // shrink to the surface, not to offset fog
destRect.y = 0;
destRect.h -= sourceRect.y; // shrink the height to stay within bounds
}
int xDistanceFromEdge = (destRect.x + destRect.w) - mFogOfWar->w;
if(xDistanceFromEdge > 0) // we're busting
{
sourceRect.w -= xDistanceFromEdge;
destRect.w -= xDistanceFromEdge;
}
int yDistanceFromEdge = (destRect.y + destRect.h) - mFogOfWar->h;
if(yDistanceFromEdge > 0) // we're busting
{
sourceRect.h -= yDistanceFromEdge;
destRect.h -= yDistanceFromEdge;
}
SDL_LockSurface(mFogOfWar);
Uint32* destPixels = (Uint32*)mFogOfWar->pixels;
Uint32* srcPixels = (Uint32*)mFogOfWarPunch->pixels;
static bool keepFogRemoved = false;
for(int x = 0; x < destRect.w; ++x)
{
for(int y = 0; y < destRect.h; ++y)
{
Uint32* destPixel = destPixels + (y + destRect.y) * mFogOfWar->w + destRect.x + x;
Uint32* srcPixel = srcPixels + (y + sourceRect.y) * mFogOfWarPunch->w + sourceRect.x + x;
unsigned char* destAlpha = (unsigned char*)destPixel + 3; // fetch alpha channel
unsigned char* srcAlpha = (unsigned char*)srcPixel + 3; // fetch alpha channel
if(keepFogRemoved == true && *srcAlpha > 0)
{
continue; // skip this pixel
}
*destAlpha = *srcAlpha;
}
}
SDL_UnlockSurface(mFogOfWar);
}
Which then gave me this with keepFogRemoved = false even after the character had moved around
And this with keepFogRemoved = true
Validation
The important part is really to make sure you don't write outside of your pixel buffer, so watch out with negative offsets or offsets that would bring you out of the width or height. To validate my code, I added a simple call to RemoveFogOfWar when the mouse is clicked and tried corners and edges to make sure I didn't have a "off by one" problem
case SDL_MOUSEBUTTONDOWN:
{
if(Event.button.button == SDL_BUTTON_LEFT)
{
gRenderingManager.RemoveFogOfWar(Event.button.x, Event.button.y);
}
break;
}
Notes
Obviously, you don't need a 32 bits texture for the "punch", but it was the clearest way I could think of to show you how to do it. It could be done using as little as 1 bit per pixel (on / off). You can also add some gradient, and change the
if(keepFogRemoved == true && *srcAlpha > 0)
{
continue; // skip this pixel
}
To something like
if(*srcAlpha > *destAlpha)
{
continue;
}
To keep a smooth blend like this:
3 State Fog of War
I thought I should add this... I added a way to create a 3 state fog of war: visible, seen and fogged.
To do this, I simply keep the SDL_Rect of where I last "punched" the fog of war, and if the alpha is lower than a certain value, I clamp it at that value.
So, by simply adding
for(int x = 0; x < mLastFogOfWarPunchPosition.w; ++x)
{
for(int y = 0; y < mLastFogOfWarPunchPosition.h; ++y)
{
Uint32* destPixel = destPixels + (y + mLastFogOfWarPunchPosition.y) * mFogOfWar->w + mLastFogOfWarPunchPosition.x + x;
unsigned char* destAlpha = (unsigned char*)destPixel + 3;
if(*destAlpha < 0x60)
{
*destAlpha = 0x60;
}
}
}
mLastFogOfWarPunchPosition = destRect;
right before the loop where the fog of war is "punched", I get a fog of war similar to what you could have in games like StarCraft:
Now, since the "seen" fog of war is semi transparent, you will need to tweak your rendering method to properly clip "enemies" that would be in the fog, so you don't see them but you still see the terrain.
Hope this helps!

OgreBullet incorrect HeightmapCollisionShape shape scale?

I am trying to load a HeightmapTerrainShape in OgreBullet by (mostly) using the demo code, but my terrain mesh is offset from the HeightmapTerrainShape. I have no clue why this is happening. This is my code:
void TerrainLoader::setTerrainPhysics(Ogre::Image *imgPtr)
{
unsigned page_size = terrainGroup->getTerrainSize();
Ogre::Vector3 terrainScale(4096 / (page_size-1), 600, 4096 / (page_size-1));
float *heights = new float[page_size*page_size];
for(unsigned y = 0; y < page_size; ++y)
{
for(unsigned x = 0; x < page_size; ++x)
{
Ogre::ColourValue color = imgPtr->getColourAt(x, y, 0);
heights[x + y * page_size] = color.r;
}
}
OgreBulletCollisions::HeightmapCollisionShape *terrainShape = new OgreBulletCollisions::HeightmapCollisionShape(
page_size,
page_size,
terrainScale,
heights,
true
);
OgreBulletDynamics::RigidBody *terrainBody = new OgreBulletDynamics::RigidBody(
"Terrain",
OgreInit::level->physicsManager->getWorld()
);
imgPtr = NULL;
Ogre::Vector3 terrainShiftPos(terrainScale.x/(page_size-1), 0, terrainScale.z/(page_size-1));
terrainShiftPos.y = terrainScale.y / 2 * terrainScale.y;
Ogre::SceneNode *pTerrainNode = OgreInit::sceneManager->getRootSceneNode()->createChildSceneNode();
terrainBody->setStaticShape(pTerrainNode, terrainShape, 0.0f, 0.8f, terrainShiftPos);
//terrainBody->setPosition(terrainBody->getWorldPosition()-Ogre::Vector3(0.005, 0, 0.005));
OgreInit::level->physicsManager->addBody(terrainBody);
OgreInit::level->physicsManager->addShape(terrainShape);
}
This is what it looks like with the debug drawer turned on:
My world is 4096*600*4096 in size, and each chunk is 64*600*64
heights[x + y * page_size] = color.r;
This Line gives you negative values. If you combine negative terrain height values with ogre bullet terrain, you get a wrong bounding box conversation.
You need to use the intervall 0-1 for height values.
Had the same problem with perlin noise filter that gives you values from -1 to 1.

Trouble fitting depth image to RGB image using Kinect 1.0 SDK

I'm trying to get the Kinect depth camera pixels to overlay onto the RGB camera. I am using the C++ Kinect 1.0 SDK with an Xbox Kinect, OpenCV and trying to use the new "NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution" method.
I have watched the image render itself in slow motion and looks as if pixels are being drawn multiple times in the one frame. It first draws itself from the top and left borders, then it gets to a point (you can see a 45 degree angle in there) where it starts drawing weird.
I have been trying to base my code off of the C# code written by Adam Smith at the MSDN forums but no dice. I have stripped out the overlay stuff and just want to draw the depth normalized depth pixels where it "should" be in the RGB image.
The image on the left is what I'm getting when trying to fit the depth image to RGB space, and the image on the right is the "raw" depth image as I like to see it. I was hoping this my method would create a similar image to the one on the right with slight distortions.
This is the code and object definitions that I have at the moment:
// From initialization
INuiSensor *m_pNuiInstance;
NUI_IMAGE_RESOLUTION m_nuiResolution = NUI_IMAGE_RESOLUTION_640x480;
HANDLE m_pDepthStreamHandle;
IplImage *m_pIplDepthFrame;
IplImage *m_pIplFittedDepthFrame;
m_pIplDepthFrame = cvCreateImage(cvSize(640, 480), 8, 1);
m_pIplFittedDepthFrame = cvCreateImage(cvSize(640, 480), 8, 1);
// Method
IplImage *Kinect::GetRGBFittedDepthFrame() {
static long *pMappedBits = NULL;
if (!pMappedBits) {
pMappedBits = new long[640*480*2];
}
NUI_IMAGE_FRAME pNuiFrame;
NUI_LOCKED_RECT lockedRect;
HRESULT hr = m_pNuiInstance->NuiImageStreamGetNextFrame(m_pDepthStreamHandle, 0, &pNuiFrame);
if (FAILED(hr)) {
// return the older frame
return m_pIplFittedDepthFrame;
}
bool hasPlayerData = HasSkeletalEngine(m_pNuiInstance);
INuiFrameTexture *pTexture = pNuiFrame.pFrameTexture;
pTexture->LockRect(0, &lockedRect, NULL, 0);
if (lockedRect.Pitch != 0) {
cvZero(m_pIplFittedDepthFrame);
hr = m_pNuiInstance->NuiImageGetColorPixelCoordinateFrameFromDepthPixelFrameAtResolution(
m_nuiResolution,
NUI_IMAGE_RESOLUTION_640x480,
640 * 480, /* size is previous */ (unsigned short*) lockedRect.pBits,
(640 * 480) * 2, /* size is previous */ pMappedBits);
if (FAILED(hr)) {
return m_pIplFittedDepthFrame;
}
for (int i = 0; i < lockedRect.size; i++) {
unsigned char* pBuf = (unsigned char*) lockedRect.pBits + i;
unsigned short* pBufS = (unsigned short*) pBuf;
unsigned short depth = hasPlayerData ? ((*pBufS) & 0xfff8) >> 3 : ((*pBufS) & 0xffff);
unsigned char intensity = depth > 0 ? 255 - (unsigned char) (256 * depth / 0x0fff) : 0;
long
x = pMappedBits[i], // tried with *(pMappedBits + (i * 2)),
y = pMappedBits[i + 1]; // tried with *(pMappedBits + (i * 2) + 1);
if (x >= 0 && x < m_pIplFittedDepthFrame->width && y >= 0 && y < m_pIplFittedDepthFrame->height) {
m_pIplFittedDepthFrame->imageData[x + y * m_pIplFittedDepthFrame->widthStep] = intensity;
}
}
}
pTexture->UnlockRect(0);
m_pNuiInstance->NuiImageStreamReleaseFrame(m_pDepthStreamHandle, &pNuiFrame);
return(m_pIplFittedDepthFrame);
}
Thanks
I have found that the problem was that the loop,
for (int i = 0; i < lockedRect.size; i++) {
// code
}
was iterating on a per-byte basis, not on a per-short (2 bytes) basis. Since lockedRect.size returns the number of bytes the fix was simply changing the increment to i += 2, even better would be changing it to sizeof(short), like so,
for (int i = 0; i < lockedRect.size; i += sizeof(short)) {
// code
}