How to overlay one image on top of another? C++ - c++

I have two RGB images (ppm format), and I want to be able to overlay any pixel that's not purely black of the top image onto the bottom image.
I can successfully load imaged, save images, copy images... but I'm not able to create an image out of the two images in the manner I've described above.
I'm not going to include all the code I have, but the important parts to achieve this are:
struct Pixel
{
unsigned int r;
unsigned int g;
unsigned int b;
}
I overloaded its == operator for easier comparison:
bool Pixel::operator==(const Pixel& other)
{
if(r != other.r)
{
return true;
}
else if(g != other.g)
{
return true;
}
else if(b != other.b)
{
return true;
}
else
{
return false;
}
}
In my Pic class I have this method:
Pic Pic::overlay(const Pic& top, Pixel mask)
{
for(int h = 0; h < height; h++)
{
for(int w = 0; w < width; w++)
{
if(!(top.pixels[h][w] == mask))
{
pixels[h][w] = top.pixels[h][w]; // pixels[][] is a Pixel array
}
}
}
return *this;
}
My main file has this:
Pic top;
Pic bot;
Pic overlay;
Pixel mask:
mask.r = 0;
mask.g = 0;
mask.b = 0;
top.loadimage("top.ppm"); //loadimage() loads the image in and all the data
bot.loadimage("bot.ppm"); //samme thing
overlay = bot.overlay(bot, mask);
overlay.saveimage("overlay.ppm");
The = operator is overloaded for the Pic class, obviously.
The kind of problems I have are these:
In the overlay method, if I leave this if statement as described above, top image will be displayed in the saved file. If I make it without !() part, it'll display the bottom image.
If I get rid of that if() statement completely, and simply try to alter the individual pixels, ex:
pixels[h][w].r = pixels[h][w].r - 50;
The saved image will be altered, all wacky looking, for obvious reasons.
However... .b and .g have no effect on the image.
I'm out of ideas... I've been playing with this for 2 days and I can't figure out what's wrong. Everything works as needed in my program, except this overlay method.
EDIT: So, I found one of the problems in my code and it went back to how I loaded the images with PPM P6 format. Instead of individually loading each pixel as 1 byte, I tried to load them all together, so it created that cushion stuff that happens with structures and binary reading in from compaction... Now I'm able to put the overlay of top image onto the bottom image, but not all colors are showing. Still, better than before.
Here's what I modified my overlay's nested for() loop to look like:
for(int h = 0; h < height; h++)
{
for(int w = 0; w < width; w++)
{
if(top.pixels[h][w].r != mask.r &&
top.pixels[h][w].g != mask.g &&
top.pixels[h][w].b != mask.b )
{
pixels[h][w].r = top.pixels[h][w].r;
pixels[h][w].g = top.pixels[h][w].g;
pixels[h][w].b = top.pixels[h][w].b;
}
}
}
Obviously it still requires work.

This line looks wrong:
overlay = bot.overlay(bot, mask);
Shouldn't it be:
overlay = bot.overlay(top, mask);
And if you want a shorter way to write your equality test then you might like this:
bool Pixel::operator==(const Pixel& other)
{
return (r==other.r && g==other.g && b==other.b);
}
Finally, since you've got an equality operator, then why not do add and assignment ('=') to keep your coder as neat as poss

Related

Am I checking for a white pixel correctly?

Cross posting as this may be more of a C++ question than a robotics one.
I am currently going through all the pixels in an image to determine what is a white pixel. I then have to decide where to drive the bot. I am also using sensor_msgs/Image.msg that I get from the /camera/rgb/image_raw channel.
However, I can't seem to locate any white image with the code but the RGBa values I set in my model in gazebo all have value 1 as shown in the image below the code .
I logged all my values(more than once) with ROS_INFO_STREAM but no values are 255, let alone 3 consecutive ones.
void process_image_callback(const sensor_msgs::Image img)
{
const int white_pixel = 255;
const int image_slice_width = img.step / 3;
int j = 0;
bool found = false;
for (int i = 0; not found and i < img.height; i++)
{
for (j; j < img.step-3; j += 3)
{
if (img.data[i*img.step + j] == white_pixel)
{
ROS_INFO_STREAM("img.data[i*img.step + (j + 0)]" + std::to_string(img.data[i*img.step + (j + 0)]));
ROS_INFO_STREAM("img.data[i*img.step + (j + 1)]" + std::to_string(img.data[i*img.step + (j + 1)]));
ROS_INFO_STREAM("img.data[i*img.step + (j + 2)]" + std::to_string(img.data[i*img.step + (j + 2)]));
}
// img.data only has one index
if (img.data[i*img.step + j ] == white_pixel and
img.data[i*img.step + (j + 1)] == white_pixel and
img.data[i*img.step + (j + 2)] == white_pixel)
{
found = true;
break;
}
}
ROS_INFO_STREAM("End of j loop");
}
if (found)
{
// go left, forward or right
}
else
{
// no white pixel seen so stop the bot
}
}
I'd suggest making your own custom structures such as these: The code below is not perfect syntax where it is only pseudo code to illustrate the overall concept.
Having your own custom classes and structures allows you to parse various file types of different image formats into a data structure format that is designed to work with your application.
Here, you would have a custom color structure that can be templated so that the color components can be either <integral> or <floating> types... Then having an external function that takes a Color object will check to see if it meets the criteria of being a white Color object. If the r,g,b color channels are indeed 255 or 1.0 and the alpha channel is not 0, then the pixel should be white!
I also provided specializations of the function template to work with both <integral> and <floating> type Color objects. Again the syntax isn't perfect as it is only pseudo-code but to implement these classes and structures should be trivial.
If the color encoding is not "RGBA" then you will have to figure out the actual encoding of the color channels and covert it to be in an "RGBA" format! Once this is done, we can then apply this to any pixel data.
template<typename T>
struct Color {
T r;
T g;
T b;
T a;
Color() : r{0}, g{0}, b{0}, a{1}
{}
Color(T red, T green, T blue, T alpha = 1) :
r{red}, g{green}, b{blue}, a{alpha}
{}
Color(T& red, T& green T& blue, T& alpha = 1) :
r{red}, g{green}, b{blue}, a{alpha}
{}
};
template<typename T, std::is_integral<T>>
bool isWhite(const Color<T>& color) {
return ( (color.r == 255) && (color.g == 255) &&
(color.b == 255) && (color.a != 0) );
}
template<typename T, std::is_floating<T>>
bool isWhite(const Color<T>& color) {
return ( (color.r == 1.0) && (color.g == 1.0) &&
(color.b == 1.0) && (color.a != 0.0) );
}
class Image {
private:
std::string filename_;
std::string encoding_;
uint32_t width_;
uint32_t height_;
uint32_t step_;
uint8_t bigEndian_;
std::vector<Color> pixelData_; // Where Color structures are populated
// from the file's `uint8[] data` matrix.
public:
Image(const std::string& filename) {
// Open The File parse it's contents from the header information
// and populate your internal data structure with the needed
// information from the file.
}
uint32_t width() const { return width_; }
uint32_t height() const { return height_; }
uint32_t stride() const { return step_; }
std::string getEncoding() const { return encoding_; }
std::string getFilename() const { return filename_; }
std::vector<Color> getPixelData() const { return pixelData_; }
};
Then somewhere else in your code where you are processing the information about the pixel data from the image.
void processImage(const Image& image) {
for (auto& pixel : image.getPixelData() ) {
if ( isWhite( pixel ) ) {
// Do something
} else {
// Do Something different
}
}
}
This should make it easier to work with since the Image object is of your own design. The hardest part would be to write the file loader - parser to obtain all of the information from their file format and to convert it to one of your own...
I've done this quite a bit since I work with 3D graphics using DirectX, OpenGL, and now Vulkan. In the beginning, I never relied on 3rd party libraries to load in image or texture files, I originally wrote my own loaders and parsers to accept TGAs, BMPs, PNGs, etc. and I had a single Texture or Image class that can be created from any of those file formats, file types.
This might help you out in the long run. What if you want to extend your application to use different "Cameras"? Now all you would have to do is just o write different file loaders for the next camera type. Parse it's data structures and convert it into your own custom data structure and format. Now you will end up having a plug and play system so too speak! You can easily extend the supported types your application can use.

Faster way to pick up the blob at a given point

I am trying to create an equivalent function for matlab's bwselect. So, I want to display the blob (which contains the points I will provide) and mask the rest.
Here's what I have tried.
cv::Mat bwselect(cv::Mat matImg, int x, int y)
{
cv::Mat img_labels, stats, centroids, mask;
if (matImg.data)
{
int numOfLables = connectedComponentsWithStats(matImg, img_labels, stats, centroids, 8, CV_32S);
if (numOfLables > 1)
{
for (int i = 1; i < numOfLables; i++)
{
mask = cv::Mat::zeros(img_labels.size(), CV_8UC1);
mask = mask | (img_labels == i);
if (mask.at<uchar>(y, x) > 0)
{
break;
}
}
}
}
return mask;
}
It does the job. But it's slow. Is there any faster and efficient way to do this?
If the input image is large and if it contains many objects, then the bottleneck could arise because you are allocating/deallocating a large mask buffer a lot of times.
Furthermore, if you call this function lots of times, it would be wise to call connectedComponentsWithStats only once and then use its results as additional input for your function.
I would suggest that you replace this entire loop
for (int i = 1; i < numOfLabels; i++){/*...*/}
with this
// img_labels data type is CV_32S
int label_at_pos = img_labels.at<int>(y, x);
if (label_at_pos > 0)
{
// create mask here and return it
}
EDIT: I made a correction to my code sample above. The connectedComponentsWithStats computes labels image that contains integer values by default.

Opencv: Get all objects from segmented colorful image

How to get all objects from image i am separating image objects through colors.
There are almost 20 colors in following image. I want to extract all colors and their position in a vector(Vec3b and Rect).
I'm using egbis algorithum for segmentation
Segmented image
Mat src, dst;
String imageName("/home/pathToImage.jpg" );
src = imread(imageName,1);
if(src.rows < 1)
return -1;
for(int i=0; i<src.rows; i=i+5)
{ for(int j=0; j<src.cols; j=j+5)
{
Vec3b color = src.at<Vec3b>(Point(i,j));
if(colors.empty())
{
colors.push_back(color);
}
else{
bool add = true;
for(int k=0; k<colors.size(); k++)
{
int rmin = colors[k].val[0]-5,
rmax = colors[k].val[0]+5,
gmin = colors[k].val[1]-5,
gmax = colors[k].val[1]+5,
bmin = colors[k].val[2]-5,
bmax = colors[k].val[2]+5;
if((
(color.val[0] >= rmin && color.val[0] <= rmax) &&
(color.val[1] >= gmin && color.val[1] <= gmax) &&
(color.val[2] >= bmin && color.val[2] <= bmax))
)
{
add = false;
break;
}
}
if(add)
colors.push_back(color);
}
}
}
int size = colors.size();
for(int i=0; i<colors.size();i++)
{
Mat inrangeImage;
//cv::inRange(src, Scalar(lowBlue, lowGreen, lowRed), Scalar(highBlue, highGreen, highRed), redColorOnly);
cv::inRange(src, cv::Scalar(colors[i].val[0]-1, colors[i].val[1]-1, colors[i].val[2]-1), cv::Scalar(colors[i].val[0]+1, colors[i].val[1]+1, colors[i].val[2]+1), inrangeImage);
imwrite("/home/kavtech/Segmentation/1/opencv-wrapper-egbis/images/inrangeImage.jpg",inrangeImage);
}
/// Display
namedWindow("Image", WINDOW_AUTOSIZE );
imshow("Image", src );
waitKey(0);
I want to get each color position so that
i can differentiate object positions. Please Help!
That's just a trivial data formatting problem. You want to turn a truecolour image with only 20 or so colours into a colour-indexed image.
So simply step through the image, look up the colour in your growing dictionary, and assign and integer 0-20 to each pixel.
Now you can turn the images into binary images simply by saying one colour is set and the rest are clear, and use standard algorithms for fitting rectangles.

Manipulating pixels of a cv::MAT just doesn't take effect

The following code is just supposed to load an image, fill it with a constant value and save it again.
Of course that doesn't have a purpose yet, but still it just doesn't work.
I can read the pixel values in the loop, but all changes are without effect and saves the file as it was loaded.
Think I followed the "efficient way" here accurately: http://docs.opencv.org/2.4/doc/tutorials/core/how_to_scan_images/how_to_scan_images.html
int main()
{
Mat im = imread("C:\\folder\\input.jpg");
int channels = im.channels();
int pixels = im.cols * channels;
if (!im.isContinuous())
{ return 0; } // Just to show that I've thought of that. It never exits here.
uchar* f = im.ptr<uchar>(0);
for (int i = 0; i < pixels; i++)
{
f[i] = (uchar)100;
}
imwrite("C:\\folder\\output.jpg", im);
return 0;
}
Normal cv functions like cvtColor() are taking effect as expected.
Are the changes through the array happening on a buffer somehow?
Huge thanks in advance!
The problem is that you are not looking at all pixels in the image. Your code only looks at im.cols*im.channels() which is a relatively small number as compared to the size of the image (im.cols*im.rows*im.channels()). When used in the for loop using the pointer, it only sets a value for couple of rows in an image ( if you look closely you will notice the saved image will have these set ).
Below is the corrected code:
int main()
{
Mat im = imread("C:\\folder\\input.jpg");
int channels = im.channels();
int pixels = im.cols * im.rows * channels;
if (!im.isContinuous())
{ return 0; } // Just to show that I've thought of that. It never exits here.
uchar* f = im.ptr<uchar>(0);
for (int i = 0; i < pixels; i++)
{
f[i] = (uchar)100;
}
imwrite("C:\\folder\\output.jpg", im);
return 0;
}

OpenCV function pointPolygonTest() does not act like what I thought

Please check my code, it does not work well. There is no errors occur during both build and debug session. I want to mark all the pixels with WHITE inside every contour. The contours are correct because I have drawn them separately. But the ultimate result is not right.
//Draw the Sketeches
Mat sketches(detected.size(), CV_8UC1, Scalar(0));
for (int j = ptop; j <= pbottom; ++j) {
for (int i = pleft; i <= pright; ++i) {
if (pointPolygonTest(contours[firstc], Point(j, i), false) >= 0) {
sketches.at<uchar> (i, j)= 255;
}
if (pointPolygonTest(contours[secondc], Point(j, i), false) >= 0) {
sketches.at<uchar> (i, j)= 255;
}
}
}
The variable "Mat detected" is another image used for hand detection. I have extracted two contours from it as contours[firstc] and contours[secondc]. And I also narrow down the hand part in the image to row(ptop:pbottom), and col(pleft,pright), and the two "for" loop goes correctly as well. So where exactly is the problem?.
Here is my result! Something goes wrong with it!