How to do Mul blending in opencv - c++

I want to blend two images like multiply blending in photoshop , i want to do the same in OpenCv using C++ for my app , I visit this many time and try to understand every time but i didn't get it , i search it alot but didn't get what i want other then this but this is little bit strange as conversion is alot from IplImages to ibl etc , Any help , guide, idea and example related opencv is needed . I go through Addweight but i think its quite different from Multiply Blending
Formula which i saw here
Target * Blend
and below is what i tried
Mat img1 = imread("E:\\img.jpg");
Mat img2 = Mat (img1.size(),img1.type());
vector<Mat> colors_1;
split(img2, colors_1);
colors_1[0] = 113;
colors_1[1] = 221;
colors_1[2] = 216;
merge(colors_1,img2);
Mat result(img1.size(), CV_32F);
for(int i = 0; i < img1.size().height; ++i){
for(int j = 0; j < img1.size().width; ++j){
for (int rgb=0 ; rgb<=img1.channels();rgb++){
float target = float(img1.at<uchar>(i, j)) / 255;
float blend = float(img2.at<uchar>(i, j)) / 255;
result.at<float>(i, j) = target*blend;
}
}
}
Result is in GrayScale and its not looking exact
Thank you

You are not accessing the image channels correctly. Moreover, you do not need to store the result in a float image, uchar is OK. Also, your loop on RGB channels should end when rgb<img1.channels().
Try this code:
cv::Mat img1 = cv::imread("E:\\img.jpg");
cv::Mat img2 = cv::Mat (img1.size(),img1.type());
std::vector<cv::Mat> colors_1;
cv::split(img2, colors_1);
colors_1[0] = 113;
colors_1[1] = 221;
colors_1[2] = 216;
cv::merge(colors_1,img2);
cv::Mat result(img1.size(), CV_8UC3);
for(int i = 0; i < img1.rows; ++i){
for(int j = 0; j < img1.cols; ++j){
for (int c=0 ; c<img1.channels();c++){
uchar target = img1.at<uchar>(i, 3*j+c);
uchar blend = img2.at<uchar>(i, 3*j+c);
result.at<uchar>(i, 3*j+c) = cv::saturate_cast<uchar>(target*blend/255.);
}
}
}

Related

How to convert a RGB Mat to HSV and set hue the value with OpenCv in c++?

I've worked on school project. And now I want to use a HSV Mat and then set the value of hue while maintain the sat and value at 100. Here is my code:
Mat imgRGB(Size(8, 8), CV_8U);
Mat imgHSV(Size(8, 8), CV_8U);
cvtColor(imgRGB, imgHSV, CV_RGB2HSV);
for (int i = 0; i < 8; ++i) {
for (int j = 0; j < 8; ++j) {
imgHSV.at<Vec3b>(i, j)[0] = 180 * i / 8;
imgHSV.at<Vec3b>(i, j)[1] = 100;
imgHSV.at<Vec3b>(i, j)[2] = 100;
}
}
Mat resizedHSV;
resize(imgHSV, resizedHSV, Size(640, 640));
imshow("HSV", resizedHSV);
waitKey(0);
Somehow VS2013 always told me that there is a cv::Exception. I found that the problem is on cvtColor() and for loop, but I often saw people use these function. I have no idea why this would fail. Could someone notice me. Thanks!!!

OpenCV-2.4.8.2: imshow differs from imwrite

I'm using OpenCV2.4.8.2 on Mac OS 10.9.5.
I have the following snippet of code:
static void compute_weights(const vector<Mat>& images, vector<Mat>& weights)
{
weights.clear();
for (int i = 0; i < images.size(); i++) {
Mat image = images[i];
Mat mask = Mat::zeros(image.size(), CV_32F);
int x_start = (i == 0) ? 0 : image.cols/2;
int y_start = 0;
int width = image.cols/2;
int height = image.rows;
Mat roi = mask(Rect(x_start,y_start,width,height)); // Set Roi
roi.setTo(1);
weights.push_back(mask);
}
}
static void blend(const vector<Mat>& inputImages, Mat& outputImage)
{
int maxPyrIndex = 6;
vector<Mat> weights;
compute_weights(inputImages, weights);
// Find the fused pyramid:
vector<Mat> fused_pyramid;
for (int i = 0; i < inputImages.size(); i++) {
Mat image = inputImages[i];
// Build Gaussian Pyramid for Weights
vector<Mat> weight_gaussian_pyramid;
buildPyramid(weights[i], weight_gaussian_pyramid, maxPyrIndex);
// Build Laplacian Pyramid for original image
Mat float_image;
inputImages[i].convertTo(float_image, CV_32FC3, 1.0/255.0);
vector<Mat> orig_guassian_pyramid;
vector<Mat> orig_laplacian_pyramid;
buildPyramid(float_image, orig_guassian_pyramid, maxPyrIndex);
for (int j = 0; j < orig_guassian_pyramid.size() - 1; j++) {
Mat sized_up;
pyrUp(orig_guassian_pyramid[j+1], sized_up, Size(orig_guassian_pyramid[j].cols, orig_guassian_pyramid[j].rows));
orig_laplacian_pyramid.push_back(orig_guassian_pyramid[j] - sized_up);
}
// Last Lapalcian layer is the same as the Gaussian layer
orig_laplacian_pyramid.push_back(orig_guassian_pyramid[orig_guassian_pyramid.size()-1]);
// Convolve laplacian original with guassian weights
vector<Mat> convolved;
for (int j = 0; j < maxPyrIndex + 1; j++) {
// Create 3 channels for weight gaussian pyramid as well
vector<Mat> gaussian_3d_vec;
for (int k = 0; k < 3; k++) {
gaussian_3d_vec.push_back(weight_gaussian_pyramid[j]);
}
Mat gaussian_3d;
merge(gaussian_3d_vec, gaussian_3d);
//Mat convolved_result = weight_gaussian_pyramid[j].clone();
Mat convolved_result = gaussian_3d.clone();
multiply(gaussian_3d, orig_laplacian_pyramid[j], convolved_result);
convolved.push_back(convolved_result);
}
if (i == 0) {
fused_pyramid = convolved;
} else {
for (int j = 0; j < maxPyrIndex + 1; j++) {
fused_pyramid[j] += convolved[j];
}
}
}
// Blending
for (int i = (int)fused_pyramid.size()-1; i > 0; i--) {
Mat sized_up;
pyrUp(fused_pyramid[i], sized_up, Size(fused_pyramid[i-1].cols, fused_pyramid[i-1].rows));
fused_pyramid[i-1] += sized_up;
}
Mat final_color_bgr;
fused_pyramid[0].convertTo(final_color_bgr, CV_32F, 255);
final_color_bgr.copyTo(outputImage);
imshow("final", outputImage);
waitKey(0);
imwrite(outputImagePath, outputImage);
}
This code is doing some basic pyramid blending for 2 images. The key issues are related to imshow and imwrite in the last line. They gave me drastically different results. I apologize for displaying such a long/messy code, but I am afraid this difference is coming from some other parts of the code that can subsequently affect the imshow and imwrite.
The first image shows the result from imwrite and the second image shows the result from imshow, based on the code given. I'm quite confused about why this is the case.
I also noticed that when I do these:
Mat float_image;
inputImages[i].convertTo(float_image, CV_32FC3, 1.0/255.0);
imshow("float image", float_image);
imshow("orig image", image);
They show exactly the same thing, that is they both show the same picture in the original rgb image (in image).
IMWRITE functionality
By default, imwrite, converts the input image into Only 8-bit (or 16-bit unsigned (CV_16U) in case of PNG, JPEG 2000, and TIFF) single-channel or 3-channel (with ‘BGR’ channel order) images can be saved using this function.
So whatever format you feed in for imwrite, it blindly converts into CV_8U with a range 0(black) - 255(white) in BGR format.
IMSHOW - problem
So when noticed your function, fused_pyramid[0].convertTo(final_color_bgr, CV_32F, 255); fused_pyramid is already under mat type 21 (floating point CV_32F). You tried to convert into floating point with a scale factor 255. This scaling factor 255 caused the problem # imshow. Instead to visualize, you can directly feed in fused_pyramid without conversion as already it is scaled to floating point between 0.0(black) - 1.0(white).
Hope it helps.

OpenCV: color extraction based on Gaussian mixture model

I am trying to use opencv EM algorithm to do color extraction.I am using the following code based on example in opencv documentation:
cv::Mat capturedFrame ( height, width, CV_8UC3 );
int i, j;
int nsamples = 1000;
cv::Mat samples ( nsamples, 2, CV_32FC1 );
cv::Mat labels;
cv::Mat img = cv::Mat::zeros ( height, height, CV_8UC3 );
img = capturedFrame;
cv::Mat sample ( 1, 2, CV_32FC1 );
CvEM em_model;
CvEMParams params;
samples = samples.reshape ( 2, 0 );
for ( i = 0; i < N; i++ )
{
//from the training samples
cv::Mat samples_part = samples.rowRange ( i*nsamples/N, (i+1)*nsamples/N);
cv::Scalar mean (((i%N)+1)*img.rows/(N1+1),((i/N1)+1)*img.rows/(N1+1));
cv::Scalar sigma (30,30);
cv::randn(samples_part,mean,sigma);
}
samples = samples.reshape ( 1, 0 );
//initialize model parameters
params.covs = NULL;
params.means = NULL;
params.weights = NULL;
params.probs = NULL;
params.nclusters = N;
params.cov_mat_type = CvEM::COV_MAT_SPHERICAL;
params.start_step = CvEM::START_AUTO_STEP;
params.term_crit.max_iter = 300;
params.term_crit.epsilon = 0.1;
params.term_crit.type = CV_TERMCRIT_ITER|CV_TERMCRIT_EPS;
//cluster the data
em_model.train ( samples, Mat(), params, &labels );
cv::Mat probs;
probs = em_model.getProbs();
cv::Mat weights;
weights = em_model.getWeights();
cv::Mat modelIndex = cv::Mat::zeros ( img.rows, img.cols, CV_8UC3 );
for ( i = 0; i < img.rows; i ++ )
{
for ( j = 0; j < img.cols; j ++ )
{
sample.at<float>(0) = (float)j;
sample.at<float>(1) = (float)i;
int response = cvRound ( em_model.predict ( sample ) );
modelIndex.data [ modelIndex.cols*i + j] = response;
}
}
My question here is:
Firstly, I want to extract each model, here totally five, then store those corresponding pixel values in five different matrix. In this case, I could have five different colors seperately. Here I only obtained their indexes, is there any way to achieve their corresponding colors here? To make it easy, I can start from finding the dominant color based on these five GMMs.
Secondly, here my sample datapoints are "100", and it takes about nearly 3 seconds for them. But I want to do all these things in no more than 30 milliseconds. I know OpenCV background extraction, which is using GMM, performs really fast, below 20ms, that means, there must be a way for me to do all these within 30 ms for all 600x800=480000 pixels. I found predict function is the most time consuming one.
First Question:
In order to do color extraction you first need to train the EM with your input pixels. After that you simply loop over all the input pixels again and use predict() to classify each of them. I've attached a small example that utilizes EM for foreground/background separation based on colors. It shows you how to extract the dominant color (mean) of each gaussian and how to access the original pixel color.
#include <opencv2/opencv.hpp>
int main(int argc, char** argv) {
cv::Mat source = cv::imread("test.jpg");
//ouput images
cv::Mat meanImg(source.rows, source.cols, CV_32FC3);
cv::Mat fgImg(source.rows, source.cols, CV_8UC3);
cv::Mat bgImg(source.rows, source.cols, CV_8UC3);
//convert the input image to float
cv::Mat floatSource;
source.convertTo(floatSource, CV_32F);
//now convert the float image to column vector
cv::Mat samples(source.rows * source.cols, 3, CV_32FC1);
int idx = 0;
for (int y = 0; y < source.rows; y++) {
cv::Vec3f* row = floatSource.ptr<cv::Vec3f > (y);
for (int x = 0; x < source.cols; x++) {
samples.at<cv::Vec3f > (idx++, 0) = row[x];
}
}
//we need just 2 clusters
cv::EMParams params(2);
cv::ExpectationMaximization em(samples, cv::Mat(), params);
//the two dominating colors
cv::Mat means = em.getMeans();
//the weights of the two dominant colors
cv::Mat weights = em.getWeights();
//we define the foreground as the dominant color with the largest weight
const int fgId = weights.at<float>(0) > weights.at<float>(1) ? 0 : 1;
//now classify each of the source pixels
idx = 0;
for (int y = 0; y < source.rows; y++) {
for (int x = 0; x < source.cols; x++) {
//classify
const int result = cvRound(em.predict(samples.row(idx++), NULL));
//get the according mean (dominant color)
const double* ps = means.ptr<double>(result, 0);
//set the according mean value to the mean image
float* pd = meanImg.ptr<float>(y, x);
//float images need to be in [0..1] range
pd[0] = ps[0] / 255.0;
pd[1] = ps[1] / 255.0;
pd[2] = ps[2] / 255.0;
//set either foreground or background
if (result == fgId) {
fgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0);
} else {
bgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0);
}
}
}
cv::imshow("Means", meanImg);
cv::imshow("Foreground", fgImg);
cv::imshow("Background", bgImg);
cv::waitKey(0);
return 0;
}
I've tested the code with the following image and it performs quite good.
Second Question:
I've noticed that the maximum number of clusters has a huge impact on the performance. So it's better to set this to a very conservative value instead of leaving it empty or setting it to the number of samples like in your example. Furthermore the documentation mentions an iterative procedure to repeatedly optimize the model with less-constrained parameters. Maybe this gives you some speed-up. To read more please have a look at the docs inside the sample code that is provided for train() here.

OpenCV 2.3: Convert Mat to RGBA pixel array

I am attempting to use OpenCV to grab frames from a webcam and display them in a window using SFML.
VideoCapture returns frames in OpenCV's Mat format. To display the frames, SFML requires a 1D array of pixels in its uint8 format, which (as far as I can tell) is interchangeable with uchar. This array is expected to represent 32 bits per pixel RGBA.
So, I have a uchar array, and I'm looping over the Mat data and copying each pixel:
VideoCapture cap(0);
Mat frame;
cap >> frame;
uchar* camData = new uchar[640*480*4];
uchar* pixelPtr = frame.data;
for(int i = 0; i < frame.rows; i++)
{
for(int j = 0; j < frame.cols; j++)
{
camData[i*frame.cols + j + 2] = pixelPtr[i*frame.cols + j + 0]; // B
camData[i*frame.cols + j + 1] = pixelPtr[i*frame.cols + j + 1]; // G
camData[i*frame.cols + j + 0] = pixelPtr[i*frame.cols + j + 2]; // R
camData[i*frame.cols + j + 3] = 255;
}
}
img.LoadFromPixels(640, 480, camData); //Load pixels into SFML Image object for display
Unfortunately, this doesn't quite work. Something in that loop is wrong, as the resulting image when I load and display camData is scrambled.
As far as I can discern, either my math in the loop is wrong so the pixels are being assigned wrong, or the Mat data is in some format other than BGR.
Any ideas?
OpenCV can do all job for you:
VideoCapture cap(0);
Mat frame;
cap >> frame;
uchar* camData = new uchar[frame.total()*4];
Mat continuousRGBA(frame.size(), CV_8UC4, camData);
cv::cvtColor(frame, continuousRGBA, CV_BGR2RGBA, 4);
img.LoadFromPixels(frame.cols, frame.rows, camData);
I like the accepted answer better but this snippet helps you understand what's going on.
for (int i=0; i<srcMat.rows; i++) {
for (int j=0; j<srcMat.cols; j++) {
int index = (i*srcMat.cols+j)*4;
// copy while converting to RGBA order
dstRBBA[index + 0] = srcMat[index + 2 ];
dstRBBA[index + 1] = srcMat[index + 1 ];
dstRBBA[index + 2] = srcMat[index + 0 ];
dstRBBA[index + 3] = srcMat[index + 3 ];
}
}
For me worked following code:
VideoCapture capture(0);
Mat mat_frame;
capture >> mat_frame; // get a new frame from camera
// Be sure that we are dealing with RGB colorspace...
Mat rgbFrame(width, height, CV_8UC3);
cvtColor(mat_frame, rgbFrame, CV_BGR2RGB);
// ...now let it convert it to RGBA
Mat newSrc = Mat(rgbFrame.rows, rgbFrame.cols, CV_8UC4);
int from_to[] = { 0,0, 1,1, 2,2, 3,3 };
mixChannels(&rgbFrame, 2, &newSrc, 1, from_to, 4);
The result (newSrc) is a premultiply image!

Fast Pixels Access opencv

i use this code to convert image to matrix ,so someone have any idea how can i convert this matrix to 1D one -->vector
i want to have image data as a 1D array ,in row major order that is all pixel values in the first row are listed first ,followed by pixel values in the second row and so on.
IplImage *img = cvLoadImage( "lena.jpg", CV_LOAD_IMAGE_COLOR);
CvMat *mat = cvCreateMat(img->height,img->width,CV_32FC3 );
cvConvert( img, mat );
for(int i=0;i<10;i++)
{
for(int j=0;j<10;j++){
CvScalar scal = cvGet2D( mat,j,i);
printf( "(%.f,%.f,%.f) ",scal.val[0], scal.val[1], scal.val[2] );}
printf("\n");}
cvNamedWindow("une_window");
cvShowImage("une_window", img);
cvWaitKey();
cvDestroyWindow("une_window");
Using the C++ API:
cv::Mat img = cv::imread("a.jpg");
std::vector<uchar> pixels;
pixels.reserve(img.rows * img.cols * 3);
if(img.isContinuous()) {
pixels = std::vector<uchar>(img.ptr(0), img.ptr(0) + img.rows * img.cols * 3 );
}
else {
for(int i = 0; i != img.rows; ++i) {
uchar* p = img.ptr(i);
for(int j = 0; j != img.cols * 3; ++j) {
pixels.push_back(p[j]);
}
}
}
I believe the fastest way for continuous Mats is to use the reshape command:
Mat colVec = img.reshape(1, img.rows*img.cols); // change to a Nx3 column vector
The reshape command just changes the header, so it does not require pixel access and therefore runs in O(1) time.
I think you should observe from video decoder output to know the video size information, other information collected from metadata in container parser might be not so accurate.
In C++ this is actually a one-liner:
cv::Mat_<float> img = cv::imread("a.jpg", 1);
std::vector<float> dest;
std::copy(img.begin(), img.end(), dest.begin());