I'm currently using the following function to apply my alpha channels (stored as separate GRAY cv::Mats), to images:
void percepUnit::applyAlpha() {
int x,y,w,h;
/*vector<cv::Mat> channels;
if (image.rows == mask.rows and image.cols == mask.cols) {
cv::split(image,channels); // break image into channels
channels.push_back(mask); // append alpha channel
cv::merge(channels,alphaImage); // combine channels
}*/
// Avoid merge
cv::Mat src[] = {this->image, this->mask};
int from_to[] = {0,0, 1,1, 2,2, 3,3};
this->alphaImage = Mat(image.rows, image.cols, CV_8UC4);
cv::mixChannels(src, 2, &(this->alphaImage), 1, from_to, 4); // &(*alphaImage)?
}
I've had to increase the resolution of the cv::Mats to 1280x720 (due to: How to replace an instance with another instance via pointer?) and now this function is running quite slowly, using up almost 50% of what is already a heavy meanshift segmentation application.
Any suggestions on how to apply these alpha channels faster? I'm running OpenCV with GPU, if you have any GPU based solutions.)
I ended up doing split / merge on the GPU:
void percepUnit::applyAlpha() {
cv::gpu::GpuMat tmpImage, tmpMask, tmpAlphaImage;
std::vector<cv::gpu::GpuMat> channels;
tmpImage.upload(this->image);
tmpMask.upload(this->mask);
cv::gpu::split(tmpImage,channels); // break image into channels
channels.push_back(tmpMask); // append alpha channel
cv::gpu::merge(channels,tmpAlphaImage); // combine channels
tmpAlphaImage.download(this->alphaImage);
tmpAlphaImage.release();
tmpImage.release();
tmpMask.release();
channels[0].release();
channels[1].release();
channels[2].release();
}
Related
I'm new to OpenCV and I've done a small POC for reading an image from some URL.
I'm reading the image from an URL using video capture. The code is as follows:
VideoCapture vc;
vc.open("http://files.kurento.org/img/mario-wings.png");
if(vc.isOpened() && vc.grab())
{
cv::Mat logo;
vc.retrieve(logo);
cv::namedWindow("t");
imwrite( "mario-wings-opened.png", logo);
cv::imshow("t", logo);
cv::waitKey(0);
vc.release();
}
This image is not opened correctly, possibly due to alpha channel.
What is the way to preserve alpha channel and get the image correctly?
Any help is appreciated.
-Thanks
Expected output
Actual output
if you are only loading an image, I recommend you to use imread instead, also, you will need to specified the second parameter of imread to load the alpha channel too, that is CV_LOAD_IMAGE_UNCHANGED or cv::IMREAD_UNCHANGED, depending on the version (in the worst case a -1 also works).
As far as I know, the VideoCaptureclass do not load images/video with a 4th channel. Since you are using a web url, loading the image won't work with imread, but you may use any method to download the data (curl for example) and then use imdecode with the data buffer to get the cv::Mat. OpenCV is a library for image processing, not for downloading images.
If you wanna draw it over another image you can do that:
/**
* #brief Draws a transparent image over a frame Mat.
*
* #param frame the frame where the transparent image will be drawn
* #param transp the Mat image with transparency, read from a PNG image, with the IMREAD_UNCHANGED flag
* #param xPos x position of the frame image where the image will start.
* #param yPos y position of the frame image where the image will start.
*/
void drawTransparency(Mat frame, Mat transp, int xPos, int yPos) {
Mat mask;
vector<Mat> layers;
split(transp, layers); // seperate channels
Mat rgb[3] = { layers[0],layers[1],layers[2] };
mask = layers[3]; // png's alpha channel used as mask
merge(rgb, 3, transp); // put together the RGB channels, now transp insn't transparent
transp.copyTo(frame.rowRange(yPos, yPos + transp.rows).colRange(xPos, xPos + transp.cols), mask);
}
I create a Bird-View-Image with the warpPerspective()-function like this:
warpPerspective(frame, result, H, result.size(), CV_WARP_INVERSE_MAP, BORDER_TRANSPARENT);
The result looks very good and also the border is transparent:
Bird-View-Image
Now I want to put this image on top of another image "out". I try doing this with the function warpAffine like this:
warpAffine(result, out, M, out.size(), CV_INTER_LINEAR, BORDER_TRANSPARENT);
I also converted "out" to a four channel image with alpha channel according to a question which was already asked on stackoverflow:
Convert Image
This is the code: cvtColor(out, out, CV_BGR2BGRA);
I expected to see the chessboard but not the gray background. But in fact, my result looks like this:
Result Image
What am I doing wrong? Do I forget something to do? Is there another way to solve my problem? Any help is appreciated :)
Thanks!
Best regards
DamBedEi
I hope there is a better way, but here it is something you could do:
Do warpaffine normally (without the transparency thing)
Find the contour that encloses the image warped
Use this contour for creating a mask (white values inside the image warped, blacks in the borders)
Use this mask for copy the image warped into the other image
Sample code:
// load images
cv::Mat image2 = cv::imread("lena.png");
cv::Mat image = cv::imread("IKnowOpencv.jpg");
cv::resize(image, image, image2.size());
// perform warp perspective
std::vector<cv::Point2f> prev;
prev.push_back(cv::Point2f(-30,-60));
prev.push_back(cv::Point2f(image.cols+50,-50));
prev.push_back(cv::Point2f(image.cols+100,image.rows+50));
prev.push_back(cv::Point2f(-50,image.rows+50 ));
std::vector<cv::Point2f> post;
post.push_back(cv::Point2f(0,0));
post.push_back(cv::Point2f(image.cols-1,0));
post.push_back(cv::Point2f(image.cols-1,image.rows-1));
post.push_back(cv::Point2f(0,image.rows-1));
cv::Mat homography = cv::findHomography(prev, post);
cv::Mat imageWarped;
cv::warpPerspective(image, imageWarped, homography, image.size());
// find external contour and create mask
std::vector<std::vector<cv::Point> > contours;
cv::Mat imageWarpedCloned = imageWarped.clone(); // clone the image because findContours will modify it
cv::cvtColor(imageWarpedCloned, imageWarpedCloned, CV_BGR2GRAY); //only if the image is BGR
cv::findContours (imageWarpedCloned, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
// create mask
cv::Mat mask = cv::Mat::zeros(image.size(), CV_8U);
cv::drawContours(mask, contours, 0, cv::Scalar(255), -1);
// copy warped image into image2 using the mask
cv::erode(mask, mask, cv::Mat()); // for avoid artefacts
imageWarped.copyTo(image2, mask); // copy the image using the mask
//show images
cv::imshow("imageWarpedCloned", imageWarpedCloned);
cv::imshow("warped", imageWarped);
cv::imshow("image2", image2);
cv::waitKey();
One of the easiest ways to approach this (not necessarily the most efficient) is to warp the image twice, but set the OpenCV constant boundary value to different values each time (i.e. zero the first time and 255 the second time). These constant values should be chosen towards the minimum and maximum values in the image.
Then it is easy to find a binary mask where the two warp values are close to equal.
More importantly, you can also create a transparency effect through simple algebra like the following:
new_image = np.float32((warp_const_255 - warp_const_0) *
preferred_bkg_img) / 255.0 + np.float32(warp_const_0)
The main reason I prefer this method is that openCV seems to interpolate smoothly down (or up) to the constant value at the image edges. A fully binary mask will pick up these dark or light fringe areas as artifacts. The above method acts more like true transparency and blends properly with the preferred background.
Here's a small test program that warps with transparent "border", then copies the warped image to a solid background.
int main()
{
cv::Mat input = cv::imread("../inputData/Lenna.png");
cv::Mat transparentInput, transparentWarped;
cv::cvtColor(input, transparentInput, CV_BGR2BGRA);
//transparentInput = input.clone();
// create sample transformation mat
cv::Mat M = cv::Mat::eye(2,3, CV_64FC1);
// as a sample, just scale down and translate a little:
M.at<double>(0,0) = 0.3;
M.at<double>(0,2) = 100;
M.at<double>(1,1) = 0.3;
M.at<double>(1,2) = 100;
// warp to same size with transparent border:
cv::warpAffine(transparentInput, transparentWarped, M, transparentInput.size(), CV_INTER_LINEAR, cv::BORDER_TRANSPARENT);
// NOW: merge image with background, here I use the original image as background:
cv::Mat background = input;
// create output buffer with same size as input
cv::Mat outputImage = input.clone();
for(int j=0; j<transparentWarped.rows; ++j)
for(int i=0; i<transparentWarped.cols; ++i)
{
cv::Scalar pixWarped = transparentWarped.at<cv::Vec4b>(j,i);
cv::Scalar pixBackground = background.at<cv::Vec3b>(j,i);
float transparency = pixWarped[3] / 255.0f; // pixel value: 0 (0.0f) = fully transparent, 255 (1.0f) = fully solid
outputImage.at<cv::Vec3b>(j,i)[0] = transparency * pixWarped[0] + (1.0f-transparency)*pixBackground[0];
outputImage.at<cv::Vec3b>(j,i)[1] = transparency * pixWarped[1] + (1.0f-transparency)*pixBackground[1];
outputImage.at<cv::Vec3b>(j,i)[2] = transparency * pixWarped[2] + (1.0f-transparency)*pixBackground[2];
}
cv::imshow("warped", outputImage);
cv::imshow("input", input);
cv::imwrite("../outputData/TransparentWarped.png", outputImage);
cv::waitKey(0);
return 0;
}
I use this as input:
and get this output:
which looks like ALPHA channel isn't set to ZERO by warpAffine but to something like 205...
But in general this is the way I would do it (unoptimized)
I'm new with OpenCV library, and I would like to use it to detect circles in a video stream captured from an iPad's back camera. I figured out how to do it and with OpenCV 2.4.2, it can be done in less than 10 lines of code. But it doesn't work for me, and I think I missed something because of some weird behaviours I obtain.
The code is very simple and begins in the Objective-C callback triggers each time a new frame is captured by the camera. Here is what I do in this callback:
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection
{
// Convert CMSampleBufferRef to CVImageBufferRef
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
// Lock pixel buffer
CVPixelBufferLockBaseAddress(imageBuffer, kCVPixelBufferLock_ReadOnly);
// Construct VideoFrame struct
uint8_t *baseAddress = (uint8_t*)CVPixelBufferGetBaseAddress(imageBuffer);
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
size_t stride = CVPixelBufferGetBytesPerRow(imageBuffer);
// Unlock pixel buffer
CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
std::vector<unsigned char> data(baseAddress, baseAddress + (stride * height));
// Call C++ function with these arguments => (data, (int)width, (int)height)
}
And here is the C++ function that process the image with OpenCV:
void proccessImage(std::vector<unsigned char>& imageData, int width, int height)
{
// Create cv::Mat from std::vector<unsigned char>
Mat src(width, height, CV_8UC4, const_cast<unsigned char*>(imageData.data()));
Mat final;
// Draw a circle at position (300, 200) with a radius of 30
cv::Point center(300, 200);
circle(src, center, 30.f, CV_RGB(0, 0, 255), 3, 8, 0);
// Convert the gray image to RGBA
cvtColor(src, final, CV_BGRA2RGBA);
// Reform the std::vector from cv::Mat data
std::vector<unsigned char> array;
array.assign((unsigned char*)final.datastart, (unsigned char*)final.dataend);
// Send final image data to GPU and draw it
}
The image retrieve from iPad's back camera is in BGRA (32 bits) format.
What I expected was an image from the iPad's back camera with a simple circle drawn at the position x = 300px, y = 200px and with a radius of 30px.
And this is what I got: http://i.stack.imgur.com/bWfwa.jpg
Do you know what is wrong with my code?
Thanks in advance.
Thanks for your help, I finally figured out what happen, and it's my entire fault...
When you create a new Mat you need to pass it the image's height as first argument, and not width. The circle is drawn properly if I switch the arguments.
I've just completed the Udacity Parallel programming stage 2 course, and I'm now implementing what I've learnt into a basic app with OpenCV which applies a gaussian blur to a constant stream of images coming through a webcam.
I'm loading frames into a Mat object, and whilst in my loop I want to call a method gaussian_cpu, the only problem is it requires a uchar4 to be passed to both the input and output parameters. How would I convert a Mat object to uchar4?
// Keep processing frames - Do CPU First
while(cpu_frames > 0)
{
cout << cpu_frames << "\n";
camera >> frameIn;
gaussian_cpu(frameIn, frameOut, numRows(), numCols(), h_filter__, 9);
imshow("Source", frameIn);
imshow("Dest", frameOut);
// 2ms delay to prevent system from being interrupted whilst drawing the new frame
waitKey(2);
cpu_frames--;
}
My method signature then looks like this:
void gaussian_cpu(
const uchar4* const rgbaImage, // input image from the camera
uchar4* const outputImage, // The image we are writing back for display
size_t numRows, size_t numCols, // Width and Height of the input image (rows/cols)
const float* const filter, // The value of sigma
const int filterWidth // The size of the stencil (3x3) 9
)
I need to use uchar4 so I can split the channels, do my convolution and then recombine the channels to return the output image. Is there any way to do this?
opencv generally uses bgr, 3 channel Mats, but a basic:
Mat bgra;
cvtColor( frameIn, bgra, CV_BGR2BGRA );
will generate an (unused) 4th channel. now you probably have to allocate mem for you outputImage:
Mat frameOut( bgra.size(), bgra.type() );
then you can feed those into your gaussian_cpu():
int filterWidth=5;
float *filter = ... // your job, not mine ;)
gaussian_cpu( (uchar4*)(bgra.data), (uchar4*)(frameOut.data), bgra.rows, bgra.cols, filter, filterWidth );
How can I overlay two images? Essentially I have a background with no alpha channel and than one or more images that have alpha channel that need to be overlaid on top of each other.
I have tried the following code but the overlay result is horrible:
// create our out image
Mat merged (info.width, info.height, CV_8UC4);
// get layers
Mat layer1Image = imread(layer1Path);
Mat layer2Image = imread(layer2Path);
addWeighted(layer1Image, 0.5, layer2Image, 0.5, 0.0, merged);
I also tried using merge but I read somewhere that it doesn't support alpha channel?
I don't know about a OpenCV function that does this. But you could just implement it yourself. It is similar to the addWeighted function. But instead of a fixed weight of 0.5 the weights are computed from the alpha channel of the overlay image.
Mat img = imread("bg.bmp");
Mat dst(img);
Mat ov = imread("ov.tiff", -1);
for(int y=0;y<img.rows;y++)
for(int x=0;x<img.cols;x++)
{
//int alpha = ov.at<Vec4b>(y,x)[3];
int alpha = 256 * (x+y)/(img.rows+img.cols);
dst.at<Vec3b>(y,x)[0] = (1-alpha/256.0) * img.at<Vec3b>(y,x)[0] + (alpha * ov.at<Vec3b>(y,x)[0] / 256);
dst.at<Vec3b>(y,x)[1] = (1-alpha/256.0) * img.at<Vec3b>(y,x)[1] + (alpha * ov.at<Vec3b>(y,x)[1] / 256);
dst.at<Vec3b>(y,x)[2] = (1-alpha/256.0) * img.at<Vec3b>(y,x)[2] + (alpha * ov.at<Vec3b>(y,x)[2] / 256);
}
imwrite("bg_ov.bmp",dst);
Note that I was not able to read in a file with the alpha channel because apparently OpenCV does not support this. That's why I computed an alpha value from the coordinates to get some kind of gradient.
Most probably channel number of merged is different from inputs. You can replace
Mat merged (info.width, info.height, CV_8UC4);
with this:
Mat merged;
This way you will let the addWeighted method create the destination matrix with the correct parameters.