Am I checking for a white pixel correctly? - c++

Question

Cross posting as this may be more of a C++ question than a robotics one.
I am currently going through all the pixels in an image to determine what is a white pixel. I then have to decide where to drive the bot. I am also using sensor_msgs/Image.msg that I get from the /camera/rgb/image_raw channel.
However, I can't seem to locate any white image with the code but the RGBa values I set in my model in gazebo all have value 1 as shown in the image below the code .
I logged all my values(more than once) with ROS_INFO_STREAM but no values are 255, let alone 3 consecutive ones.
void process_image_callback(const sensor_msgs::Image img)
{
const int white_pixel = 255;
const int image_slice_width = img.step / 3;
int j = 0;
bool found = false;
for (int i = 0; not found and i < img.height; i++)
{
for (j; j < img.step-3; j += 3)
{
if (img.data[i*img.step + j] == white_pixel)
{
ROS_INFO_STREAM("img.data[i*img.step + (j + 0)]" + std::to_string(img.data[i*img.step + (j + 0)]));
ROS_INFO_STREAM("img.data[i*img.step + (j + 1)]" + std::to_string(img.data[i*img.step + (j + 1)]));
ROS_INFO_STREAM("img.data[i*img.step + (j + 2)]" + std::to_string(img.data[i*img.step + (j + 2)]));
}
// img.data only has one index
if (img.data[i*img.step + j ] == white_pixel and
img.data[i*img.step + (j + 1)] == white_pixel and
img.data[i*img.step + (j + 2)] == white_pixel)
{
found = true;
break;
}
}
ROS_INFO_STREAM("End of j loop");
}
if (found)
{
// go left, forward or right
}
else
{
// no white pixel seen so stop the bot
}
}

Answer 1

I'd suggest making your own custom structures such as these: The code below is not perfect syntax where it is only pseudo code to illustrate the overall concept.
Having your own custom classes and structures allows you to parse various file types of different image formats into a data structure format that is designed to work with your application.
Here, you would have a custom color structure that can be templated so that the color components can be either <integral> or <floating> types... Then having an external function that takes a Color object will check to see if it meets the criteria of being a white Color object. If the r,g,b color channels are indeed 255 or 1.0 and the alpha channel is not 0, then the pixel should be white!
I also provided specializations of the function template to work with both <integral> and <floating> type Color objects. Again the syntax isn't perfect as it is only pseudo-code but to implement these classes and structures should be trivial.
If the color encoding is not "RGBA" then you will have to figure out the actual encoding of the color channels and covert it to be in an "RGBA" format! Once this is done, we can then apply this to any pixel data.
template<typename T>
struct Color {
T r;
T g;
T b;
T a;
Color() : r{0}, g{0}, b{0}, a{1}
{}
Color(T red, T green, T blue, T alpha = 1) :
r{red}, g{green}, b{blue}, a{alpha}
{}
Color(T& red, T& green T& blue, T& alpha = 1) :
r{red}, g{green}, b{blue}, a{alpha}
{}
};
template<typename T, std::is_integral<T>>
bool isWhite(const Color<T>& color) {
return ( (color.r == 255) && (color.g == 255) &&
(color.b == 255) && (color.a != 0) );
}
template<typename T, std::is_floating<T>>
bool isWhite(const Color<T>& color) {
return ( (color.r == 1.0) && (color.g == 1.0) &&
(color.b == 1.0) && (color.a != 0.0) );
}
class Image {
private:
std::string filename_;
std::string encoding_;
uint32_t width_;
uint32_t height_;
uint32_t step_;
uint8_t bigEndian_;
std::vector<Color> pixelData_; // Where Color structures are populated
// from the file's `uint8[] data` matrix.
public:
Image(const std::string& filename) {
// Open The File parse it's contents from the header information
// and populate your internal data structure with the needed
// information from the file.
}
uint32_t width() const { return width_; }
uint32_t height() const { return height_; }
uint32_t stride() const { return step_; }
std::string getEncoding() const { return encoding_; }
std::string getFilename() const { return filename_; }
std::vector<Color> getPixelData() const { return pixelData_; }
};
Then somewhere else in your code where you are processing the information about the pixel data from the image.
void processImage(const Image& image) {
for (auto& pixel : image.getPixelData() ) {
if ( isWhite( pixel ) ) {
// Do something
} else {
// Do Something different
}
}
}
This should make it easier to work with since the Image object is of your own design. The hardest part would be to write the file loader - parser to obtain all of the information from their file format and to convert it to one of your own...
I've done this quite a bit since I work with 3D graphics using DirectX, OpenGL, and now Vulkan. In the beginning, I never relied on 3rd party libraries to load in image or texture files, I originally wrote my own loaders and parsers to accept TGAs, BMPs, PNGs, etc. and I had a single Texture or Image class that can be created from any of those file formats, file types.
This might help you out in the long run. What if you want to extend your application to use different "Cameras"? Now all you would have to do is just o write different file loaders for the next camera type. Parse it's data structures and convert it into your own custom data structure and format. Now you will end up having a plug and play system so too speak! You can easily extend the supported types your application can use.