I am working with C++ and OpenCV on Visual Studio; for my application, I have a set of images and a .mask one that permits to select a ROI in each of them (it is 0 everywhere but in the ROI).
I load the images (duck1.jpg, duck2.jpg, etc.) with:
std::array<cv::Mat, n_imgs> img;
std::string folder = "C:/Users/imgs/";
std::string fname = "duck";
std::string format = ".jpg";
for (int i = 0; i < n_imgs; i++)
img[i] = cv::imread(folder + fname + std::to_string(i + 1) + format, 0);
Then, I apply a mask:
cv::Mat mask = cv::imread(folder + fname + ".mask" + format, 0);
for (int i = 0; i < img[0].rows; i++)
for (int j = 0; j < img[0].cols; j++)
for (int k = 0; k < n_imgs; k++)
if (mask.at<float>(i, j) == 0)
img[k].at<float>(i, j) = 0.f;
I keep getting 'Project.exe has triggered a breakpoint.' quite randomly in some recurring points of my successive code (which I won't post here because is quite long); however, these problems disappear when I comment the masking line.
Given the symptoms, I supposed it is an allocation problem, am I right? How can I fix it?
It will be more of a guess what is wrong but I will give you a hint.
Are you sure that img have float as underlying type? Because when you do cv::imread(file, IMREAD_GRAYSCALE ) this 0 stand for gray scale image which usually is CV_8UC1 (unsigned char of 8 bit) when you address it with float (which has size of 32 bit) you may end up writing memory after the end of image (24 bits after last pixel in image are written). This may sometimes trigger error and sometimes not, it depends on what is in memory after your image (is it allocated or not). I guess that if you run your program in Debug mode it will always fail.
So change:
for (int i = 0; i < img[0].rows; i++)
for (int j = 0; j < img[0].cols; j++)
for (int k = 0; k < n_imgs; k++)
if (mask.at<float>(i, j) == 0)
img[k].at<float>(i, j) = 0.f;
To:
for (int i = 0; i < img[0].rows; i++)
for (int j = 0; j < img[0].cols; j++)
for (int k = 0; k < n_imgs; k++)
if (mask.at<unsigned char>(i, j) == 0)
img[k].at<unsigned char>(i, j) = 0;
If you want your code to run faster and you have binary mask (with values of 0 and 1) you can just multiply these element-wise like this (note that this is only for one image):
cv::Mat afterApplyMask = img.mul(mask);
I was writing a C code in Visual Studio when I encountered this error. My project name and name of the solution in the source file was the same. Once I changed those and debugged, I stopped encountering the error.
Related
I'm attempting to use opencv2 to preprocess and display images for classification but the pixels do not seem to be formatting properly, I'm not sure if formatting is the proper term. The image was originally 1080p and I used ffmpeg to crop and scale to 480X800. I was getting weird results so testing the program using the following code to overlay a simple checkerboard pattern where all squares should be the same size and square -
std::string image_path = samples::findFile("/home/pi/test.jpg");
Mat img = imread(image_path, IMREAD_COLOR);
cv::cvtColor(img, img, COLOR_BGR2RGB);
for (int i = 0; i < 15 ; i++ ) {
for (int j = 0; j < 25 ; j++ ) {
int x;
if ((i + j) % 2 == 0) x = 1;
else x = 0;
for (int a = i * 32; a < (i + 1) * 32 ; a++) {
for (int b = j * 32; b < (j + 1) * 32 ; b++) {
img.at<int>(a, b) = x * img.at<int>(a, b);
}
}
}
}
I get the following
checkerboard_test
the original image looks exactly like it should without any stretching or other issues. This is being displayed on a small touch screen attached to a raspberry pi. Any help would be greatly appreciated.
I figured it out, code should be -
img.at<Vec3b>(a, b) = x * img.at<Vec3b>(a, b);
instead of
img.at<int>(a, b) = x * img.at<int>(a, b);
everyone I am trying to implement patter matching with FFT but I am not sure what the result should be (I think I am missing something even though a read a lot of stuff about the problem and tried a lot of different implementations this one is the best so far). Here is my FFT correlation function.
void fft2d(fftw_complex**& a, int rows, int cols, bool forward = true)
{
fftw_plan p;
for (int i = 0; i < rows; ++i)
{
p = fftw_plan_dft_1d(cols, a[i], a[i], forward ? FFTW_FORWARD : FFTW_BACKWARD, FFTW_ESTIMATE);
fftw_execute(p);
}
fftw_complex* t = (fftw_complex*)fftw_malloc(rows * sizeof(fftw_complex));
for (int j = 0; j < cols; ++j)
{
for (int i = 0; i < rows; ++i)
{
t[i][0] = a[i][j][0];
t[i][1] = a[i][j][1];
}
p = fftw_plan_dft_1d(rows, t, t, forward ? FFTW_FORWARD : FFTW_BACKWARD, FFTW_ESTIMATE);
fftw_execute(p);
for (int i = 0; i < rows; ++i)
{
a[i][j][0] = t[i][0];
a[i][j][1] = t[i][1];
}
}
fftw_free(t);
}
int findCorrelation(int argc, char* argv[])
{
BMP bigImage;
BMP keyImage;
BMP result;
RGBApixel blackPixel = { 0, 0, 0, 1 };
const bool swapQuadrants = (argc == 4);
if (argc < 3 || argc > 4) {
cout << "correlation img1.bmp img2.bmp" << endl;
return 1;
}
if (!keyImage.ReadFromFile(argv[1])) {
return 1;
}
if (!bigImage.ReadFromFile(argv[2])) {
return 1;
}
//Preparations
const int maxWidth = std::max(bigImage.TellWidth(), keyImage.TellWidth());
const int maxHeight = std::max(bigImage.TellHeight(), keyImage.TellHeight());
const int rowsCount = maxHeight;
const int colsCount = maxWidth;
BMP bigTemp = bigImage;
BMP keyTemp = keyImage;
keyImage.SetSize(maxWidth, maxHeight);
bigImage.SetSize(maxWidth, maxHeight);
for (int i = 0; i < rowsCount; ++i)
for (int j = 0; j < colsCount; ++j) {
RGBApixel p1;
if (i < bigTemp.TellHeight() && j < bigTemp.TellWidth()) {
p1 = bigTemp.GetPixel(j, i);
} else {
p1 = blackPixel;
}
bigImage.SetPixel(j, i, p1);
RGBApixel p2;
if (i < keyTemp.TellHeight() && j < keyTemp.TellWidth()) {
p2 = keyTemp.GetPixel(j, i);
} else {
p2 = blackPixel;
}
keyImage.SetPixel(j, i, p2);
}
//Here is where the transforms begin
fftw_complex **a = (fftw_complex**)fftw_malloc(rowsCount * sizeof(fftw_complex*));
fftw_complex **b = (fftw_complex**)fftw_malloc(rowsCount * sizeof(fftw_complex*));
fftw_complex **c = (fftw_complex**)fftw_malloc(rowsCount * sizeof(fftw_complex*));
for (int i = 0; i < rowsCount; ++i) {
a[i] = (fftw_complex*)fftw_malloc(colsCount * sizeof(fftw_complex));
b[i] = (fftw_complex*)fftw_malloc(colsCount * sizeof(fftw_complex));
c[i] = (fftw_complex*)fftw_malloc(colsCount * sizeof(fftw_complex));
for (int j = 0; j < colsCount; ++j) {
RGBApixel p1;
p1 = bigImage.GetPixel(j, i);
a[i][j][0] = (0.299*p1.Red + 0.587*p1.Green + 0.114*p1.Blue);
a[i][j][1] = 0.0;
RGBApixel p2;
p2 = keyImage.GetPixel(j, i);
b[i][j][0] = (0.299*p2.Red + 0.587*p2.Green + 0.114*p2.Blue);
b[i][j][1] = 0.0;
}
}
fft2d(a, rowsCount, colsCount);
fft2d(b, rowsCount, colsCount);
result.SetSize(maxWidth, maxHeight);
for (int i = 0; i < rowsCount; ++i)
for (int j = 0; j < colsCount; ++j) {
fftw_complex& y = a[i][j];
fftw_complex& x = b[i][j];
double u = x[0], v = x[1];
double m = y[0], n = y[1];
c[i][j][0] = u*m + n*v;
c[i][j][1] = v*m - u*n;
int fx = j;
if (fx>(colsCount / 2)) fx -= colsCount;
int fy = i;
if (fy>(rowsCount / 2)) fy -= rowsCount;
float r2 = (fx*fx + fy*fy);
const double cuttoffCoef = (maxWidth * maxHeight) / 37992.;
if (r2<128 * 128 * cuttoffCoef)
c[i][j][0] = c[i][j][1] = 0;
}
fft2d(c, rowsCount, colsCount, false);
const int halfCols = colsCount / 2;
const int halfRows = rowsCount / 2;
if (swapQuadrants) {
for (int i = 0; i < halfRows; ++i)
for (int j = 0; j < halfCols; ++j) {
std::swap(c[i][j][0], c[i + halfRows][j + halfCols][0]);
std::swap(c[i][j][1], c[i + halfRows][j + halfCols][1]);
}
for (int i = halfRows; i < rowsCount; ++i)
for (int j = 0; j < halfCols; ++j) {
std::swap(c[i][j][0], c[i - halfRows][j + halfCols][0]);
std::swap(c[i][j][1], c[i - halfRows][j + halfCols][1]);
}
}
for (int i = 0; i < rowsCount; ++i)
for (int j = 0; j < colsCount; ++j) {
const double& g = c[i][j][0];
RGBApixel pixel;
pixel.Alpha = 0;
int gInt = 255 - static_cast<int>(std::floor(g + 0.5));
pixel.Red = gInt;
pixel.Green = gInt;
pixel.Blue = gInt;
result.SetPixel(j, i, pixel);
}
BMP res;
res.SetSize(maxWidth, maxHeight);
result.WriteToFile("result.bmp");
return 0;
}
Sample output
This question would probably be more appropriately posted on another site like cross validated (metaoptimize.com used to also be a good one, but it appears to be gone)
That said:
There's two similar operations you can perform with FFT: convolution and correlation. Convolution is used for determining how two signals interact with each-other, whereas correlation can be used to express how similar two signals are to each-other. Make sure you're doing the right operation as they're both commonly implemented throught a DFT.
For this type of application of DFTs you usually wouldn't extract any useful information in the fourier spectrum unless you were looking for frequencies common to both data sources or whatever (eg, if you were comparing two bridges to see if their supports are spaced similarly).
Your 3rd image looks a lot like the power domain; normally I see the correlation output entirely grey except where overlap occurred. Your code definitely appears to be computing the inverse DFT, so unless I'm missing something the only other explanation I've come up with for the fuzzy look could be some of the "fudge factor" code in there like:
if (r2<128 * 128 * cuttoffCoef)
c[i][j][0] = c[i][j][1] = 0;
As for what you should expect: wherever there are common elements between the two images you'll see a peak. The larger the peak, the more similar the two images are near that region.
Some comments and/or recommended changes:
1) Convolution & correlation are not scale invariant operations. In other words, the size of your pattern image can make a significant difference in your output.
2) Normalize your images before correlation.
When you get the image data ready for the forward DFT pass:
a[i][j][0] = (0.299*p1.Red + 0.587*p1.Green + 0.114*p1.Blue);
a[i][j][1] = 0.0;
/* ... */
How you grayscale the image is your business (though I would've picked something like sqrt( r*r + b*b + g*g )). However, I don't see you doing anything to normalize the image.
The word "normalize" can take on a few different meanings in this context. Two common types:
normalize the range of values between 0.0 and 1.0
normalize the "whiteness" of the images
3) Run your pattern image through an edge enhancement filter. I've personally made use of canny, sobel, and I think I messed with a few others. As I recall, canny was "quick'n dirty", sobel was more expensive, but I got comparable results when it came time to do correlation. See chapter 24 of the "dsp guide" book that's freely available online. The whole book is worth your time, but if you're low on time then at a minimum chapter 24 will help a lot.
4) Re-scale the output image between [0, 255]; if you want to implement thresholds, do it after this step because the thresholding step is lossy.
My memory on this one is hazy, but as I recall (edited for clarity):
You can scale the final image pixels (before rescaling) between [-1.0, 1.0] by dividing off the largest power spectrum value from the entire power spectrum
The largest power spectrum value is, conveniently enough, the center-most value in the power spectrum (corresponding to the lowest frequency)
If you divide it off the power spectrum, you'll end up doing twice the work; since FFTs are linear, you can delay the division until after the inverse DFT pass to when you're re-scaling the pixels between [0..255].
If after rescaling most of your values end up so black you can't see them, you can use a solution to the ODE y' = y(1 - y) (one example is the sigmoid f(x) = 1 / (1 + exp(-c*x) ), for some scaling factor c that gives better gradations). This has more to do with improving your ability to interpret the results visually than anything you might use to programmatically find peaks.
edit I said [0, 255] above. I suggest you rescale to [128, 255] or some other lower bound that is gray rather than black.
Sorry for putting it out here but opencv site is down or something i cannot access the topic there. I am reading a RGB image in Mat and than trying to convert in into BYTE* using the following function but it is not giving me the exact image but distorted part of some of the image. Please can anyone help me out here where i am mistaking it. Thank you.
BYTE *change_to_BYTE(Mat matrix_value)
{
BYTE* v_char_new = new BYTE[matrix_value.rows * matrix_value.cols * 3]();
vector<byte> v_char;
for(int i = 0; i < matrix_value.rows; i++)
{
for(int j = 0; j < matrix_value.cols; j++)
{
v_char_new[((i*matrix_value.cols+j) * 3)+0] = (*(uchar*)((matrix_value.data+ i*matrix_value.step + j + 0)));
v_char_new[((i*matrix_value.cols+j) * 3)+1] = (*(uchar*)((matrix_value.data+ i*matrix_value.step + j +1)));
v_char_new[((i*matrix_value.cols+j) * 3)+2] = (*(uchar*)((matrix_value.data+ i*matrix_value.step + j +2)));
}
}
return v_char_new;
}
After checking it out in different ways this was the one that worked for me.
BYTE *Ctest_face_projectDlg::change_to_BYTE(Mat matrix_value)
{
BYTE* v_char_new = new BYTE[matrix_value.rows * matrix_value.cols * 3]();
for(int i = 0; i < matrix_value.rows; i++)
{
for(int j = 0; j < matrix_value.cols; j++)
{
Vec3b bgrPixel = matrix_value.at<Vec3b>(i, j);
v_char_new[((i*matrix_value.cols+j)*3)+0] = bgrPixel.val[0];
v_char_new[((i*matrix_value.cols+j)*3)+1] = bgrPixel.val[1];
v_char_new[((i*matrix_value.cols+j)*3)+2] = bgrPixel.val[2];
// do something with BGR values...
}
}
}
I have a Npp8u * image = new Npp8u[nWidth * nHeight]; array that stores grayscale image values. I defined Npp8u as typedef unsigned char Npp8u;. The problem occurs when I try to subtract the lowest image value from ever pixel.
I use this function to do the subtraction.
void SubtractValue(Npp8u * image, int width, int height, Npp8u value){
int iValue = value;
for (int i = 0; i < height; ++i){
for (int j = 0; j < width; ++j){
int indexVal = (image[i*width + j]);
int newVal = indexVal - iValue;
image[i*width + j] = (Npp8u) newVal;
}
}
}
Result of this function is quite strange. I use Lena Image to test, this is the original image.
If the value is 85 I get the following image
Note: The original image is .pgm
Reading procedure:
for (int i = 0; i < nHeight; ++i)
for (int j = 0; j < nWidth; ++j)
image[i*nWidth + j] = fgetc(fInput);
Writing procedure:
for (int i = 0; i < nHeight; ++i)
for (int j = 0; j < nWidth; ++j)
fputc(image[i*nWidth + j], fOutput);
Update: The problem occurs if the pixel has the value 10, which is new line feed.
Since you mentioned that character 10 was causing the issue, one would suspect that the image format you're reading/writing needs to processed in "binary" file mode, so that ASCII 10 is not treated as a line feed.
Therefore make sure that file is opened in binary mode (for fopen use the "rb" or "wb" flags, depending on whether you're reading or writing).
I'm trying to change a for loop's start variable to use an if statement instead. However, they don't seem to be equivalent..
I have a loop that looked like:
int K = 0;
std::uint32_t CheckSum = 0;
const std::uint8_t* BuffPos = static_cast<const std::uint8_t*>(Data);
int Start = Height < 12 ? 1 : 12;
for (std::size_t I = Start; I < Height; ++I)
{
for (std::size_t J = 0; J < Width; ++J, ++K)
{
BuffPos += 3; //skip RGB and move to alpha pixel.
CheckSum += *(BuffPos++); //Checksum = count of alpha pixels.
}
}
std::cout<<CheckSum;
However, I don't want my loop to start with I = Start.
I'd rather do that in an if statement. I tried the following:
int K = 0;
std::uint32_t CheckSum = 0;
const std::uint8_t* BuffPos = static_cast<const std::uint8_t*>(Data);
int Start = Height < 12 ? 1 : 12;
for (std::size_t I = 0; I < Height; ++I)
{
for (std::size_t J = 0; J < Width; ++J, ++K)
{
BuffPos += 3; //Skip RGB and move to alpha pixel.
if (I >= Start) //The if statement.
CheckSum += *(BuffPos++); //Checksum = count of alpha pixels.
}
}
std::cout<<CheckSum;
However, the second version with the if statement prints a totally different number than the first for the exact same bitmap.
Is there a reason why it does this? How can I fix it? I can't see why it would be any different :S
As you noticed, it is no longer the same.
Sure, you segregated the checksum bit, but now you are incrementing BufPos a different number of times. Your loop is now iterating more times, so BuffPos has a different value once you start creating the checksum than it did in the first version.
Look at it this way; the first version starts accumulating the checksum at BuffPos + 3. The second version begins accumulating the checksum from BuffPos + (Start * Width) + 3.
Honestly, the first version is better. I can't tell you which one is correct, but assuming the first version is (iterating from Start) then why introduce a branch at all? It just muddles up the code.
If you want the second version to be the same as the first you'll need to initialize BuffPos to Data + Start * Width or place both statements within the if. Of course, if you do that, you're just doing nothing at all until I == Start, and that should tell you something.
how's this version?
int K = 0;
std::uint32_t CheckSum = 0;
const std::uint8_t* BuffPos = static_cast<const std::uint8_t*>(Data);
int Start = Height < 12 ? 1 : 12;
for (std::size_t I = 0; I < Height; ++I)
{
if (I < Start) //The if statement.
continue;
for (std::size_t J = 0; J < Width; ++J, ++K)
{
BuffPos += 3; //Skip RGB and move to alpha pixel.
CheckSum += *(BuffPos++); //Checksum = count of alpha pixels.
}
}
std::cout<<CheckSum;
Both statements(Buffpos and cheksum) should come under if statement .