I'm trying to isolate a single player through the Kinect depth camera. I'm opening an NUI_IMAGE_TYPE_DEPTH_AND_PLAYER_INDEX stream to process the player/depth info. The code that I'm using to draw a player is this:
if (LockedRect.Pitch != 0 ) {
USHORT* curr = (USHORT*) LockedRect.pBits;
const USHORT* dataEnd = curr + ((width/2)*(height/2));
index = 0;
while (curr < dataEnd && playerId != 0) {
USHORT depth = *curr;
USHORT realDepth = NuiDepthPixelToDepth(depth);
BYTE intensity = 255;
USHORT player = NuiDepthPixelToPlayerIndex(depth);
// Only colour in the player
if (player == playerId) {
for (int i = index; i < index + 4; i++)
dest[i] = intensity;
}
else {
for (int i = index; i < index + 4; i++)
dest[i] = 0;
}
index += 4;
curr += 1;
}
}
dest is an OpenGL texture.
The problem I'm having is that the variable player changes when a second person steps into the frame, and causes the person drawn in the texture to be the new person.
OK I figured out how to do it.
I needed to get the skeletal ID (0 through 5) which maps to depth pixel user (1 through 6). So when the sensor found a skeleton, it saved the ID, and set that to playerId. PlayerId is only cleared when its associated skeleton is lost by sensor.
Related
I'm making a neural network in c++ to traverse a maze knowing where the player is, which 2d direction it can move in (up,down,left,right) and where the goal is, i can at any time get the distance of the player to the goal as well
thus far this is my code
std::string NeuralNetwork::evolve(player p)
{
openMoves = p.getMoves();
playerPos = p.getListPos();
goalPos = p.getGoalPos();
values.clear();
for (auto item : openMoves)
values.push_back(item);
values.push_back(playerPos.x);
values.push_back(playerPos.y);
if (outputs.size() == 0)
outputs.resize(4);
if (inputs.size() == 0)
inputs.resize(values.size());
for (int i = 0; i < inputs.size(); i++)
{
inputs[i].srcValue = values[i];
if (inputs[i].weightList.size() == 0)
{
for (int j = 0; j < outputs.size(); j++)
{
inputs[i].weightList.push_back(dist(100) / 100.0f);
}
}
}
for (int i = 0; i < outputs.size(); i++)
{
outputs[i].out = 0;
if (outputs[i].theta == NULL)
outputs[i].theta = dist(-100, 100) / 100.0f;//rand funct to produce int between -100-100 then divide by 100 to get theta between -1 and 1
for (auto a : inputs)
{
outputs[i].out += (a.srcValue * a.weightList[i]);
}
outputs[i].finalOut = outputs[i].out / (1.0f + fabs(outputs[i].out));
//outputs[i].finalOut = 1 / (1 + std::pow(exp(1), -(outputs[i].out - outputs[i].theta)));
}
for (int i = 0; i < outputs.size(); i++)//backwards prop
{
float e = 1 - outputs[i].finalOut;
float delta = outputs[i].finalOut * (1 - outputs[i].finalOut)*e;
for (int j = 0; j < inputs.size(); j++)
{
inputs[j].weightList[i] += alpha * inputs[j].srcValue * delta;
}
outputs[i].theta += alpha * (-1)*delta;
}
the neural network is a function that's called every frame, the player starts in the top left and the goal is in the bottom right, the function returns a direction for the player to move in.
however, using backwards propagation each output final is never 1 which is what I use to determine its direction using
for (int i = 0; i < outputs.size(); i++)
{
if (outputs[0].finalOut == 1)//left
{
return "01";
}
else if (outputs[1].finalOut == 1)//up
{
return "10";
}
else if (outputs[2].finalOut == 1)//down
{
return "11";
}
else if (outputs[3].finalOut == 1)//right
{
return "00";
}
}
return "";}
however the function always returns no movement at all and i'm not sure why, even after a few minutes of waiting, there are no hidden layers to the network i'm starting simple with the inputs directly linking to the output
i'm unsure if my error calculation is correct to then adjust the weights. i'm unsure what i should be using to allow the ai to determine how to move
I'm struggling to load model via assimp with multiple meshes, here is code for loading model:
bool loadAssImp(
const char * path,
std::vector<unsigned short> & indices,
std::vector<glm::vec3> & vertices,
std::vector<glm::vec2> & uvs,
std::vector<glm::vec3> & normals
) {
Assimp::Importer importer;
const aiScene* scene = importer.ReadFile(path, aiProcess_Triangulate);
if (!scene) {
cout << importer.GetErrorString() << endl;
return false;
}
aiMesh* mesh;
for (unsigned int l = 0; l < scene->mNumMeshes; l++)
{
mesh = scene->mMeshes[l];
cout << l << endl;
// Fill vertices positions
vertices.reserve(mesh->mNumVertices);
for (unsigned int i = 0; i < mesh->mNumVertices; i++) {
aiVector3D pos = mesh->mVertices[i];
vertices.push_back(glm::vec3(pos.x, pos.y, pos.z));
}
// Fill vertices texture coordinates
uvs.reserve(mesh->mNumVertices);
for (unsigned int i = 0; i < mesh->mNumVertices; i++) {
aiVector3D UVW = mesh->mTextureCoords[0][i]; // Assume only 1 set of UV coords; AssImp supports 8 UV sets.
uvs.push_back(glm::vec2(UVW.x, UVW.y));
}
// Fill vertices normals
normals.reserve(mesh->mNumVertices);
for (unsigned int i = 0; i < mesh->mNumVertices; i++) {
aiVector3D n = mesh->mNormals[i];
normals.push_back(glm::vec3(n.x, n.y, n.z));
}
// Fill face indices
indices.reserve(3 * mesh->mNumFaces);
for (unsigned int i = 0; i < mesh->mNumFaces; i++) {
// Assume the model has only triangles.
indices.push_back(mesh->mFaces[i].mIndices[0]);
indices.push_back(mesh->mFaces[i].mIndices[1]);
indices.push_back(mesh->mFaces[i].mIndices[2]);
}
}
// The "scene" pointer will be deleted automatically by "importer"
return true;
}
model I'm trying to load has two meshes, head and torso
if I start loop like this: for (int l = scene->mNumMeshes - 1; l >= 0 ; l--) then it loads model almost correctly, some of the model vertices don't get drawn correctly but head and almost all of torso is drawn.
if I start loop like this: for (unsigned int l = 0; l < scene->mNumMeshes; l++) only torso is drawn (but is drawn fully without any missing parts)
strangely in both ways vertices and index count is the same.
You need to set the transformations descriped by the aiNode-instances as well. So when loading a scene loop over the nodes, set the local node transformation, the draw the assigned meshes to the current node.
Whithout applying these transformation information the rendering of the model will be broken.
My A* pathfinding function always gets to its intended destination, but it almost always goes a bit out of the way. Here's an example:
[I made a nice image to show my issue, but apparently it won't post until my reputation reaches 10; sorry, I'm new. :P]
Essentially, it pulls left or up as much as possible without actually adding more tiles to the path. It sounds like an issue with calculating the gScore or possibly the part where a tile's parent can be reassigned based on neighboring tiles' gScores, but I just can't figure out where it's going wrong. I've combed over my code for weeks and browsed dozens of online posts, but I'm still stuck. Fyi, the compiler/debugger I have to use doesn't support breakpoints or step-through debugging, so I'm stuck with simple text output. Can anyone spot what I'm doing wrong?
Here's the primary function (Note: this is all in Angelscript. It's based on C++, but there are small differences):
int CARDINAL_COST = 10;
int DIAGONAL_COST = 14;
array<vector2> findPath(vector2 startPosition, vector2 endPosition)
{
//Translate the start and end positions into grid coordinates
startPosition = _level.getTileGridPosition(startPosition);
endPosition = _level.getTileGridPosition(endPosition);
//The path to be returned
array<vector2> path(0);
//Create the closed
array<vector2> closedSet(0);
//Create the open set. These are nodes to be considered.
array<vector2> openSet(0);
//Add the startPosition to the open set.
openSet.insertLast(startPosition);
//Create the cameFrom (path) array. Each entry hods that tile's parent tile.
array<array<vector2>> cameFrom;
cameFrom = array<array<vector2>>(_level.width(), array<vector2>(_level.height()));
//Create the gScore array. gScore is the cost to get from the start to the current tile.
array<array<int>> gScore;
gScore = array<array<int>>(_level.width(), array<int>(_level.height()));
//Set the start position score to 0
gScore[startPosition.x][startPosition.y] = 0;
//Create the fScore array. fScore is the gScore + heuristic cost.
array<array<int>> fScore;
fScore = array<array<int>>(_level.width(), array<int>(_level.height()));
//Set the start position score to the estimated (heuristic) cost.
//gScore for start is 0, so that's not included in the equation.
fScore[startPosition.x][startPosition.y] = getHeuristicCost(startPosition, endPosition);
//Required variables
bool searchComplete = false;
vector2 currentTile = startPosition;
int x = 0;
int y = 0;
string tileType = "";
vector2 nextTile(0,0);
vector2 neighborTile(0,0);
int lowestScore = 0;
int tempScore = 0;
int index = 0;
while(!searchComplete)
{
//Find the tile in the openSet with the lowest fScore.
lowestScore = fScore[openSet[0].x][openSet[0].y];
neighborTile = openSet[0];//May not actually be a "neighbor" in this case, just looking for the lowest fScore.
for(int i = 0; i < openSet.length(); i++)
{
if(fScore[neighborTile.x][neighborTile.y] < lowestScore || i == 0)
{
lowestScore = fScore[neighborTile.x][neighborTile.y];
nextTile.x = neighborTile.x;
nextTile.y = neighborTile.y;
}
}
//Drop the "nextTile" from the openSet and add it to the closedSet
index = openSet.find(nextTile);
openSet.removeAt(openSet.find(nextTile));
closedSet.insertLast(nextTile);
//Set the currentTile
currentTile = nextTile;
//Get the fScore for each neighboring tile
for(x = currentTile.x - 1; x <= currentTile.x + 1; x++)
{
for(y = currentTile.y - 1; y <= currentTile.y + 1; y++)
{
//Safety: make sure x and y aren't out of bounds
if(x < 0)
x = 0;
else if(x > _level.width())
x = _level.width();
if(y < 0)
y = 0;
else if (y > _level.height())
y = _level.height();
//Set this x,y pair to be the neighborTile
neighborTile.x = x;
neighborTile.y = y;
//Get the tile type
if(_level.tileArray()[neighborTile.x][neighborTile.y] != null)
tileType = _level.tileArray()[neighborTile.x][neighborTile.y].GetString("type");
else
tileType = "";
//Make sure we aren't looking at the current tile, the tile is not closed, and the tile is a floor or door.
if(neighborTile != currentTile && closedSet.find(neighborTile) == -1 && (tileType == "floor" || tileType == "door"))
{
//If the neighboring tile is already in the open set, check to see if the currentTile's gScore would be less if that tile was its parent.
//If it is, set the it as the currentTile's parent and reset the fScore and gScore for it.
if(openSet.find(neighborTile) != -1)
{
if(gScore[neighborTile.x][neighborTile.y] < gScore[cameFrom[currentTile.x][currentTile.y].x][cameFrom[currentTile.x][currentTile.y].y])
{
cameFrom[currentTile.x][currentTile.y] = neighborTile;
//If the tile is a diagonal move
if(neighborTile.x - currentTile.x != 0 && neighborTile.y - currentTile.y != 0)
gScore[currentTile.x][currentTile.y] = gScore[neighborTile.x][neighborTile.y] + DIAGONAL_COST;
else//If the tile is a cardinal (N,S,E,W) move
gScore[currentTile.x][currentTile.y] = gScore[neighborTile.x][neighborTile.y] + CARDINAL_COST;
fScore[currentTile.x][currentTile.y] = gScore[currentTile.x][currentTile.y] + getHeuristicCost(currentTile, endPosition);
}
}
else//Add this tile to the open set
{
openSet.insertLast(neighborTile);
//Record this tile's parent
cameFrom[neighborTile.x][neighborTile.y] = currentTile;
//If the tile is a diagonal move
if(neighborTile.x - currentTile.x != 0 && neighborTile.y - currentTile.y != 0)
gScore[neighborTile.x][neighborTile.y] = gScore[currentTile.x][currentTile.y] + DIAGONAL_COST;
else//If the tile is a cardinal (N,S,E,W) move
gScore[neighborTile.x][neighborTile.y] = gScore[currentTile.x][currentTile.y] + CARDINAL_COST;
//Get the fScore for this tile
fScore[neighborTile.x][neighborTile.y] = gScore[neighborTile.x][neighborTile.y] + getHeuristicCost(neighborTile, endPosition);
}
}
}
}
//Check to see if we have arrived at the endTile
if(currentTile == endPosition)
{
searchComplete = true;
path = reconstructPath(cameFrom, startPosition, endPosition);
}
else
{
//Check to see if the openSet is empty
if(openSet.length() == 0)
searchComplete = true;
}
}//while(!searchComplete)
return path;
}
My heuristic uses the Manhattan method:
int getHeuristicCost(vector2 startPosition, vector2 endPosition)
{
//Using Manhattan method:
int x = abs(startPosition.x - endPosition.x)*10;
int y = abs(startPosition.y - endPosition.y)*10;
return x+y;
}
And finally, here's my path reconstructing function:
array<vector2> reconstructPath(array<array<vector2>> &in cameFrom, vector2 &in startPosition, vector2 &in endPosition)
{
//Start by adding in the end position
array<vector2> totalPath(1);
vector2 currentTile = endPosition;
totalPath[0] = endPosition;
int x = endPosition.x;
int y = endPosition.y;
int angle = 0;
while(vector2(x, y) != startPosition)
{
currentTile = cameFrom[x][y];
totalPath.insertAt(0,currentTile);
x = currentTile.x;
y = currentTile.y;
}
return totalPath;
}
for(int i = 0; i < openSet.length(); i++)
{
if(fScore[neighborTile.x][neighborTile.y] < lowestScore || i == 0)
{
lowestScore = fScore[neighborTile.x][neighborTile.y];
nextTile.x = neighborTile.x;
nextTile.y = neighborTile.y;
}
}
This loop just looks at neighborTile over and over. Did you mean to go over the elements of openSet?
I am new to kinect project
And I am implementing a depth threshold when distance is greater than 400mm
for (UINT y = 0; y < pImg->rows; ++y)
{
// Get row pointers for Mats
const USHORT* pDepthRow = depth->ptr<USHORT>(y);
for (UINT x = 0; x < pImg->cols; ++x)
{
USHORT raw_depth = pDepthRow[x];
SHORT realDepth = NuiDepthPixelToDepth(raw_depth);
// If depth value is valid, convert and copy it
if (raw_depth != 65535)
{
if(realDepth >400 ) //greater than 400mm
{
pImg->at<Vec4b>(y,x)[0] = 255;
pImg->at<Vec4b>(y,x)[1] = 255;
pImg->at<Vec4b>(y,x)[2] = 255;
pImg->at<Vec4b>(y,x)[3] = 255;
}
else
{
pImg->at<Vec4b>(y,x)[0] = 0;
pImg->at<Vec4b>(y,x)[1] = 0;
pImg->at<Vec4b>(y,x)[2] = 0;
pImg->at<Vec4b>(y,x)[3] = 0;
}
}
}
It seems get the correct result but reduces the frame rate massively.
When I want to get rid of the loop by using the cv::inRange, but this function only support 8U1C when the raw depth is 16U.
So what else can I use to segment the depth according to the real distance?
Try to improve performance by storing a reference to the pixel.
Change this:
if (realDepth > 400) //greater than 400mm
{
pImg->at<Vec4b>(y,x)[0] = 255;
pImg->at<Vec4b>(y,x)[1] = 255;
pImg->at<Vec4b>(y,x)[2] = 255;
pImg->at<Vec4b>(y,x)[3] = 255;
}
else
{
pImg->at<Vec4b>(y,x)[0] = 0;
pImg->at<Vec4b>(y,x)[1] = 0;
pImg->at<Vec4b>(y,x)[2] = 0;
pImg->at<Vec4b>(y,x)[3] = 0;
}
To this:
(I don´t know what T is because I dont know what pImg is.
T should be equal to the return value of the at method. I assume it is Vec4b.)
T& pixel = pImg->at<Vec4b>(y, x); // probably Vec4b& pixel = ..
if (realDepth > 400) //greater than 400mm
{
pixel[0] = 255;
pixel[1] = 255;
pixel[2] = 255;
pixel[3] = 255;
}
else
{
pixel[0] = 0;
pixel[1] = 0;
pixel[2] = 0;
pixel[3] = 0;
}
I'm doing some computing on raw video YUV file. The idea is to move a 8x8 window accross the entire frame, pixel by pixel. Here's my code:
while ( (frameNumber < maxFrame) &&
(fread_s(frame1, frameSize, 1, frameSize, file1)== frameSize) &&
(fread_s(frame2, frameSize, 1, frameSize, file2)) == frameSize)
{
unsigned char *p1 = frame1; //pointer to indicate start of a window
unsigned char *p2 = frame2;
unsigned char *p8_1 = NULL; //pointer used to navigate accross 8x8window
unsigned char *p8_2 = NULL;
for (int i = 0; i < countY; i += stepSize)
{
p1 += i*width; //move to next line
p2 += i*width;
for (int j = 0; j < countX; j += stepSize)
{
meanLuma1 = 0;
meanLuma2 = 0;
//8x8 window loop
for (int k = 0; k < windowSize; k++)
{
p8_1 = p1 + k*width; //move to next line of a window
p8_2 = p2 + k*width;
for (int l = 0; l < windowSize; l++)
{
meanLuma1 += *p8_1;
meanLuma2 += *p8_2; //Access violation here at i=60, others are 0
++p8_1;
++p8_2;
}
}
meanLuma1 = meanLuma1 / (windowSize*windowSize);
meanLuma2 = meanLuma2 / (windowSize*windowSize);
++p1;
++p2;
}
I keep getting access violation exception on p8_2 (i=60 ; j,k,l=0). I think it's weird, that p8_1 reads its value successfully but p8_2 does not, because both files have same size and dimensions. Variable states are following:
width=352;
height=288;
stepSize=4;
windowSize=8;
And I'm computing maximum number of steps like following:
int countX = ((width - windowSize) / stepSize)+1;
int countY = ((height - windowSize) / stepSize)+1;
Second strangeness is that if I set windowSize = 16 and stepSize = 8 , it compiles successfully.
Don't you see any obvious issues? I'm struggling with finding the bug for ages.
Seems here is an error in lines:
p1 += i*width; //move to next line
p2 += i*width;
You move on too far in arithmetical progression on each step. Try:
p1 += width; //move to next line
p2 += width;
or
p1 += stepSize*width; //move to next line
p2 += stepSize*width;
(depends on width and stepSize measure units).