I am new to OpenCV2 and working on a project in emotion recognition and would like to align a facial image in relation to a reference facial image. I would like to get the image translation working before moving to rotation. Current idea is to run a search within a limited range on both x and y coordinates and use the sum of squared differences as error metric to select the optimal x/y parameters to align the image. I'm using the OpenCV face_cascade function to detect the face images, all images are resized to a fixed (128x128). Question: Which parameters of the Mat image do I need to modify to shift the image in a positive/negative direction on both x and y axis? I believe setImageROI is no longer supported by Mat datatypes? I have the ROIs for both faces available however I am unsure how to use them.
void alignImage(vector<Rect> faceROIstore, vector<Mat> faceIMGstore)
{
Mat refimg = faceIMGstore[1]; //reference image
Mat dispimg = faceIMGstore[52]; // "displaced" version of reference image
//Rect refROI = faceROIstore[1]; //Bounding box for face in reference image
//Rect dispROI = faceROIstore[52]; //Bounding box for face in displaced image
Mat aligned;
matchTemplate(dispimg, refimg, aligned, CV_TM_SQDIFF_NORMED);
imshow("Aligned image", aligned);
}
The idea for this approach is based on Image Alignment Tutorial by Richard Szeliski Working on Windows with OpenCV 2.4. Any suggestions are much appreciated.
cv::Mat does support ROI. (But it does not support COI - channel-of-interest.)
To apply ROI you can use operator() or special constructor:
Mat refimgROI = faceIMGstore[1](faceROIstore[1]); //reference image ROI
Mat dispimgROI(faceIMGstore[52], faceROIstore[52]); // "displaced" version of reference image ROI
And to find the best position inside a displaced image you can utilize matchTemplate function.
Based on your comments I can suggest the following code which will find the best position of reference patch nearby the second (displaced) patch:
Mat ref = faceIMGstore[1](faceROIstore[1]);
Mat disp = faceIMGstore[52](faceROIstore[52]);
disp = disp.adjustROI(5,5,5,5); //allow 5 pixel max adjustment in any direction
if(disp.cols < ref.cols || disp.rows < ref.rows)
return 0;
Mat map;
cv::matchTemplate( disp, ref, map, CV_TM_SQDIFF_NORMED );
Point minLoc;
cv::minMaxLoc( map, 0, &minLoc );
Mat adjusted = disp(Rect(minLoc.x, minLoc.y, ref.cols, ref.rows));
Related
I'm trying to get the people detector provided by the OpenCV library running. So far I get decent performance on my iPhone 6 but the detection is super bad and almost never correct and I'm not really sure why this is since you can find example videos using the same default HOG descriptor with way better detection.
Here is the code:
- (void)processImage:(Mat&)image {
cv::Mat cvImg, result;
cvtColor(image, cvImg, COLOR_BGR2HSV);
cv::vector<cv::Rect> found, found_filtered;
hog.detectMultiScale(cvImg, found, 0, cv::Size(4,4), cv::Size(8,8), 1.5, 0);
size_t i;
for (i=0; i < found.size(); i++) {
cv::Rect r = found[i];
rectangle(image, r.tl(), r.br(), Scalar(0,255,0), 2);
}
}
The video input comes from the iPhone camera itself and "processImage:" is called for every frame. For the HOGDescriptor I use the default people detector:
_hog.setSVMDetector(cv::HOGDescriptor::getDefaultPeopleDetector());
I appreciate any help. :)
I'm new to openCV, so take this with a grain of salt:
The line cvtColor(image, cvImg, COLOR_BGR2HSV); converts the image from the BGR color space to the HSV color space. Essentially, it changes each pixel from being represented by how much blue, green, and red it has, to being represented by the components hue (color), saturation (how much color) and value (how bright). Clearly, the hogDescriptor acts on a BGR image, not an HSV image. You need to pass it a type CV_8UC3 image: An image with 3 channels per pixel (C3), ex. BGR, and an 8bit unsigned number for each channel (8U), This part is less important. What are you passing into the method processImage()? It should be one of those types. If not, you need to know the type and convert it to CV_8UC3 using the cvtColor() method
The problem is solved....I used cvGet2D,below is the sample code
CvScalar s;
s=cvGet2D(src_Image,pixel[i].x,pixel[i].y);
cvSet2D(dst_Image,pixel[i].x,pixel[i].y,s);
Where src_Iamge and dst_Image is the source and destination image correspondingly and pixel[i] is the selected pixel i wanted to draw in the dst image. I have include the real out image below.
have an source Ipl image, I want to copy some of the part of the image to a new destination image pixel by pixel. can any body tell me how can do it? I use c,c++ in opencv. For example if the below image is source image,
The real output image
EDIT:
I can see the comments suggesting cvGet2d. I think, if you just want to show "points", it is best to show them with a small neighbourhood so they can be seen where they are. For that you can draw white filled circles with origins at (x,y), on a mask, then you do the copyTo.
using namespace cv;
Mat m(input_iplimage);
Mat mask=Mat::zeros(m.size(), CV_8UC1);
p1 = Point(x,y);
r = 3;
circle(mask,p1,r, 1); // draws the circle around your point.
floodFill(mask, p1, 1); // fills the circle.
//p2, p3, ...
Mat output = Mat::zeros(m.size(),m.type()); // output starts with a black background.
m.copyTo(output, mask); // copies the selected parts of m to output
OLD post:
Create a mask and copy those pixels:
#include<opencv2/opencv.hpp>
using namespace cv;
Mat m(input_iplimage);
Mat mask=Mat::zeros(m.size(), CV_8UC1); // set mask 1 for every pixel you wanna copy.
Rect roi=Rect(x,y,width,height); // create a rectangle
mask(roi) = 1; // set it to 0.
roi = Rect(x2,y2,w2,h2);
mask(roi)=1; // set the second rectangular area for copying...
Mat output = 100*Mat::ones(m.size(),m.type()); // output with a gray background.
m.copyTo(output, mask); // copy selected areas of m to output
Alternatively you can copy Rect-by-Rect:
Mat m(input_iplimage);
Mat output = 100*Mat::ones(m.size(),m.type()); // output with a gray background.
Rect roi=Rect(x,y,width,height);
Mat m_temp, out_temp;
m_temp=m(roi);
out_temp = output(roi);
m_temp.copyTo(out_temp);
roi=Rect(x2,y2,w2,h2);
Mat m_temp, out_temp;
m_temp=m(roi);
out_temp = output(roi);
m_temp.copyTo(out_temp);
The answer to your question only requires to have look at the OpenCV documentation or just to search in your favourite search engine.
Here you've an answer for Ipl images and for newer Mat data.
For having an output as I see in your images, I'd do it setting ROI's, it's more efficient.
I'm trying to make a program to detect an object in any shape using a video camera/webcam based on Canny filter and contour finding function. Here is my program:
int main( int argc, char** argv )
{
CvCapture *cam;
CvMoments moments;
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* contours = NULL;
CvSeq* contours2 = NULL;
CvPoint2D32f center;
int i;
cam=cvCaptureFromCAM(0);
if(cam==NULL){
fprintf(stderr,"Cannot find any camera. \n");
return -1;
}
while(1){
IplImage *img=cvQueryFrame(cam);
if(img==NULL){return -1;}
IplImage *src_gray= cvCreateImage( cvSize(img->width,img->height), 8, 1);
cvCvtColor( img, src_gray, CV_BGR2GRAY );
cvSmooth( src_gray, src_gray, CV_GAUSSIAN, 5, 11);
cvCanny(src_gray, src_gray, 70, 200, 3);
cvFindContours( src_gray, storage, &contours, sizeof(CvContour), CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE, cvPoint(0,0));
if(contours==NULL){ contours=contours2;}
contours2=contours;
cvMoments(contours, &moments, 1);
double m_00 = cvGetSpatialMoment( &moments, 0, 0 );
double m_10 = cvGetSpatialMoment( &moments, 1, 0 );
double m_01 = cvGetSpatialMoment( &moments, 0, 1 );
float gravityX = (m_10 / m_00)-150;
float gravityY = (m_01 / m_00)-150;
if(gravityY>=0&&gravityX>=0){
printf("center point=(%.f, %.f) \n",gravityX,gravityY); }
for (; contours != 0; contours = contours->h_next){
CvScalar color = CV_RGB(250,0,0);
cvDrawContours(img,contours,color,color,-1,-1, 8, cvPoint(0,0));
}
cvShowImage( "Input", img );
cvShowImage( "Contours", src_gray );
cvClearMemStorage(storage);
if(cvWaitKey(33)>=0) break;
}
cvDestroyWindow("Contours");
cvDestroyWindow("Source");
cvReleaseCapture(&cam);
}
This program will detect all contours captured by the camera and the average coordinate of the contours will be printed. My question is how to filter out only one object/contour so I can get more precise (x,y) position of the object? If possible, can anyone show me how to mark the center of the object by using (x,y) coordinates?
Thanks in advance. Cheers
p/s:Sorry I couldn't upload a screenshot yet but if anything helps, here's the link.
Edit: To make my question more clear:
For example, if I only want to filter out only the square from my screenshot above, what should I do?
The object I want to filter out has the biggest contour area and most importantly has a shape(any shape), not a straight or a curve line
I'm still experimenting with the smooth and canny values so if anybody have the problem to detect the contours using my program please alter the values.
I think it can be solved fairly easy. I would suggest some morphological operations before contour detection. Also, I would suggest filtering "out" smaller elements, and getting the biggest element as the only one still in the image.
I suggest:
for filtering out lines (straight or curved): you have to decide what do you yourself consider a border between a "line" and a "shape". Let's say you consider all the objects of a thickness 5 pixel or more to be objects, while the ones that are less than 5 pixels across to be lines. An morphological opening that uses a 5x5 square or a 3-pixel sized diamond shape as a structuring element would take care of this.
for filtering out small objects in general: if objects are of arbitrary shapes, purely morphological opening won't do: you have to do an algebraic opening. A special type of algebraic openings is an area opening: an operation that removes all the connected components in the image that have (pixel) area smaller than a given threshold. If you have an upper bound on the size of uninteresting objects, or a lower bound on the size of interesting ones, that value should be used as a threshold. You can probably get a similar effect with a larger morphological opening, but it will not be so flexible.
for filtering out all the objects except the largest: it sounds like removing connected components from the smallest one to the largest one should work. Try labeling the connected components. On a binary (black & white image), this image transformation works by creating a greyscale image, labeling the background as 0 (black), and each component with a different, increasing grey value. In the end, pixels of each object are marked by a different value. You can now simply look at the gray level histogram, and find the grey value with the most pixels. Set all the other grey levels to 0 (black), and the only object left in the image is the biggest one.
The suggestions are written from the simplest to the most complex ones. Still, I think OpenCV can be of help with any of these. Morphological erosion, dilation, opening and closing are implemented in OpenCV. I think you might need to construct an algebraic opening operator on your own (or play with combining OpenCV basic morphology), but I'm sure OpenCV can help you with both labeling the connected components and examining the histogram of the resulting greyscale image.
In the end, when only pixels from one object are left, you do the Canny contour detection.
This is a blob processing problem that can not be solved (easily) by OpenCV itself. Have a look at cvBlobsLib. This library is extends OpenCV with functions/classes for connected component labeling.
http://opencv.willowgarage.com/wiki/cvBlobsLib
I am developing some image processing tools in iOS. Currently, I have a contour of features computed, which is of type InputArrayOfArrays.
Declared as:
std::vector<std::vector<cv::Point> > contours_final( temp_contours.size() );
Now, I would like to extract areas of the original RGB picture circled by contours and may further store sub-image as cv::Mat format. How can I do that?
Thanks in advance!
I'm guessing what you want to do is just extract the regions in the the detected contours. Here is a possible solution:
using namespace cv;
int main(void)
{
vector<Mat> subregions;
// contours_final is as given above in your code
for (int i = 0; i < contours_final.size(); i++)
{
// Get bounding box for contour
Rect roi = boundingRect(contours_final[i]); // This is a OpenCV function
// Create a mask for each contour to mask out that region from image.
Mat mask = Mat::zeros(image.size(), CV_8UC1);
drawContours(mask, contours_final, i, Scalar(255), CV_FILLED); // This is a OpenCV function
// At this point, mask has value of 255 for pixels within the contour and value of 0 for those not in contour.
// Extract region using mask for region
Mat contourRegion;
Mat imageROI;
image.copyTo(imageROI, mask); // 'image' is the image you used to compute the contours.
contourRegion = imageROI(roi);
// Mat maskROI = mask(roi); // Save this if you want a mask for pixels within the contour in contourRegion.
// Store contourRegion. contourRegion is a rectangular image the size of the bounding rect for the contour
// BUT only pixels within the contour is visible. All other pixels are set to (0,0,0).
subregions.push_back(contourRegion);
}
return 0;
}
You might also want to consider saving the individual masks to optionally use as a alpha channel in case you want to save the subregions in a format that supports transparency (e.g. png).
NOTE: I'm NOT extracting ALL the pixels in the bounding box for each contour, just those within the contour. Pixels that are not within the contour but in the bounding box are set to 0. The reason is that your Mat object is an array and that makes it rectangular.
Lastly, I don't see any reason for you to just save the pixels in the contour in a specially created data structure because you would then need to store the position for each pixel in order to recreate the image. If your concern is saving space, that would not save you much space if at all. Saving the tightest bounding box would suffice. If instead you wish to just analyze the pixels in the contour region, then save a copy of the mask for each contour so that you can use it to check which pixels are within the contour.
You are looking for the cv::approxPolyDP() function to connect the points.
I shared a similar use of the overall procedure in this post. Check the for loop after the findContours() call.
I think what you're looking for is cv::boundingRect().
Something like this:
using namespace cv;
Mat img = ...;
...
vector<Mat> roiVector;
for(vector<vector<Point> >::iterator it=contours.begin(); it<contours.end(); it++) {
if (boundingRect( (*it)).area()>minArea) {
roiVector.push_back(img(boundingRect(*it)));
}
}
cv::boundingRect() takes a vector of Points and returns a cv::Rect. Initializing a Mat myRoi = img(myRect) gives you a pointer to that part of the image (so modifying myRoi will ALSO modify img).
See more here.
As an aside: Apologies if I'm flooding SO with OpenCV questions :p
I'm currently trying to port over my old C code to use the new C++ interface and I've got to the point where I'm rebuilidng my Eigenfaces face recogniser class.
Mat img = imread("1.jpg");
Mat img2 = imread("2.jpg");
FaceDetector* detect = new HaarDetector("haarcascade_frontalface_alt2.xml");
// convert to grey scale
Mat g_img, g_img2;
cvtColor(img, g_img, CV_BGR2GRAY);
cvtColor(img2, g_img2, CV_BGR2GRAY);
// find the faces in the images
Rect r = detect->getFace(g_img);
Mat img_roi = g_img(r);
r = detect->getFace(g_img2);
Mat img2_roi = g_img2(r);
// create the data matrix for PCA
Mat data;
data.create(2,1, img2_roi.type());
data.row(0) = img_roi;
data.row(1) = img2_roi;
// perform PCA
Mat averageFace;
PCA pca(data, averageFace, CV_PCA_DATA_AS_ROW, 2);
//namedWindow("avg",1); imshow("avg", averageFace); - causes segfault
//namedWindow("avg",1); imshow("avg", Mat(pca.mean)); - doesn't work
I'm trying to create the PCA space, and then see if it's working by displaying the computed average image. Are there any other steps to this?
Perhaps I need to project the images onto the PCA subspace first of all?
Your error is probably here:
Mat data;
data.create(2,1, img2_roi.type());
data.row(0) = img_roi;
data.row(1) = img2_roi;
PCA expects a matrix with the data vectors as rows. However, you never scale the images to the same size so that they have the same number of pixels (so the dimension is the same), also data.create(2,1,...) - the 1 needs to be the dimension of your vector, i.e. the number of your pixels. Then copy the pixels from the crop to your matrix.